Stochastic Processes: Martingales, Markov Chains, Poisson Jumps, and Ito Calculus

A Stochastic Process is a collection of random variables \{X_t \mid t \in T\} parameterized by time t, representing the probabilistic evolution of dynamic systems. From queuing networks in cloud architecture and molecular diffusion in statistical physics, to algorithmic trading and algorithmic risk modeling, stochastic mathematics provides the analytical framework for quantifying uncertainty over time.

This reference covers Discrete & Continuous-Time Markov Chains, Martingale Theory & Stopping Times, Poisson Point Processes, and Ito Stochastic Calculus.

1. Quick-Reference: Stochastic Process Taxonomy

+-----------------------------------------------------------------------------------------------------------------------+
|                                           STOCHASTIC PROCESS TAXONOMY                                                 |
+-----------------------------------------------------------------------------------------------------------------------+
| Process Type           | Time / State Space                     | Governing Equation / Invariant | Core Applications    |
+------------------------+----------------------------------------+--------------------------------+----------------------+
| Discrete Markov Chain  | Discrete $t \in \mathbb{N}$, State $S$ | $\pi = \pi \mathbf{P}$ (Stationary)| PageRank, Queue Model|
| Continuous Markov Chain| Continuous $t \ge 0$, Discrete $S$     | $rac{d\mathbf{P}}{dt} = \mathbf{P}\mathbf{Q}$| SRE Server Availability|
| Brownian Motion (Wiener)| Continuous $t \ge 0$, Continuous $S$  | $W_t - W_s \sim \mathcal{N}(0, t-s)$| Diffusion, Physics   |
| Poisson Point Process  | Continuous $t$, Discrete counts        | $P(N_t = k) = rac{(\lambda t)^k e^{-\lambda t}}{k!}$| Web Request Arrival  |
| Martingale             | Discrete or Continuous                 | $\mathbb{E}[X_{t+1} \mid \mathcal{F}_t] = X_t$| Fair Games, Asset Px |
+-----------------------------------------------------------------------------------------------------------------------+

2. Martingale Theory & Doob's Optional Stopping Theorem

A stochastic process \{X_t\}_{t \ge 0} adapted to filtration \mathcal{F}_t is a Martingale if:

  1. \mathbb{E}[|X_t|] < \infty for all t \ge 0.
  2. \mathbb{E}[X_{t+s} \mid \mathcal{F}_t] = X_t for all s \ge 0.

Doob's Optional Stopping Theorem

If au is a bounded stopping time, the expected value at stopping time equals the initial value:

\mathbb{E}[X_ au] = \mathbb{E}[X_0]

This theorem is the primary mathematical tool for proving the impossibility of guaranteed betting strategies in fair games and calculating ruin probabilities in actuarial risk.

3. Ito Calculus & The Stochastic Integral

Because sample paths of standard Brownian motion W_t are nowhere differentiable and have infinite total variation (with non-zero quadratic variation [W, W]_t = t), standard Riemann-Stieltjes integration fails.

The Ito Integral is defined as the mean-square limit:

\int_0^T f(t) dW_t = \lim_{n o \infty} \sum_{i=0}^{n-1} f(t_i) [W_{t_{i+1}} - W_{t_i}]

(Invariant: The integrand f(t_i) is evaluated strictly at the left endpoint, preserving the martingale property).

References

  1. Karatzas, I., & Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus (2nd ed.). Springer.
  2. Norris, J. R. (1998). Markov Chains. Cambridge University Press.
  3. Oksendal, B. (2013). Stochastic Differential Equations: An Introduction with Applications. Springer.