Markov Processes and Markov Families

Abstract

In this section we shall use intuitive arguments in order to find the distribution of M _T . Rigorous arguments will be provided later in this chapter, after we introduce the notion of a strong Markov family. Thus, the problem at hand may serve as a simple example motivating the study of the strong Markov property.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Distribution of the Maximum of Brownian Motion

Let ${W}_{t}$ be a one-dimensional Brownian motion relative to a filtration ${\mathcal{F}}_{t}$ on a probability space $({\it \Omega}, \mathcal{F},\mathrm{P})$. We denote the maximum of ${W}_{t}$ on the interval [0, T] by M _T ,

$${M}_{T}({\it \omega }) =\sup\limits_{0\leq t\leq T}{W}_{t}({\it \omega }).$$

In this section we shall use intuitive arguments in order to find the distribution of M _T . Rigorous arguments will be provided later in this chapter, after we introduce the notion of a strong Markov family. Thus, the problem at hand may serve as a simple example motivating the study of the strong Markov property.

For a non-negative constant c, define the stopping time ${\tau }_{c}$ as the first time the Brownian motion reaches the level c if this occurs before time T, and otherwise as T, that is

$${\tau }_{c}({\it \omega }) =\min (\inf \{t \geq 0 : {W}_{t}({\it \omega }) = c\},T).$$

Since the probability of the event ${W}_{T} = c$ is equal to zero,

$$\mathrm{P}({M}_{T} \geq c) =\mathrm{ P}({\tau }_{c}<T) =\mathrm{ P}({\tau }_{c}<T,{W}_{T}<c) +\mathrm{ P}({\tau }_{c}<T,{W}_{T}>c).$$

The key observation is that the probabilities of the events $\{{\tau }_{c}<T,{W}_{T}<c\}$ and $\{{\tau }_{c}<T,{W}_{T}>c\}$ are the same. Indeed, the Brownian motion is equally likely to be below c and above c at time T under the condition that it reaches level c before time T. This intuitive argument hinges on our ability to stop the process at time ${\tau }_{c}$ and then “start it anew” in such a way that the increment ${W}_{T} - {W}_{{\tau }_{c}}$ has symmetric distribution and is independent of ${\mathcal{F}}_{{\tau }_{c}}.$

Since ${\tau }_{c}<T$ almost surely on the event $\{{W}_{T}>c\}$,

$$\mathrm{P}({M}_{T} \geq c) = 2\mathrm{P}({\tau }_{c}<T,{W}_{T}>c) = 2\mathrm{P}({W}_{T}>c) = \frac{\sqrt{2}} {\sqrt{\pi T}}\int _{c}^{\infty }{e}^{-\frac{{x}^{2}} {2T} }dx.$$

Therefore,

$$\mathrm{P}({M}_{T} \leq c) = 1 -\mathrm{ P}({M}_{T} \geq c) = 1 - \frac{\sqrt{2}} {\sqrt{\pi T}}\int _{c}^{\infty }{e}^{-\frac{{x}^{2}} {2T} }dx,$$

which is the desired expression for the distribution of the maximum of Brownian motion.

2 Definition of the Markov Property

Let $(X,\mathcal{G})$ be a measurable space. In Chap. 5 we defined a Markov chain as a measure on the space of sequences with elements in X which is generated by a Markov transition function. In this chapter we use a different approach, defining a Markov process as a random process with certain properties, and a Markov family as a family of such random processes. We then reconcile the two points of view by showing that a Markov family defines a transition function. In turn, by using a transition function and an initial distribution we can define a measure on the space of realizations of the process.

For the sake of simplicity of notation, we shall primarily deal with the time-homogeneous case. Let us assume that the state space is ${\mathbb{R}}^{d}$ with the $\sigma$-algebra of Borel sets, that is $(X,\mathcal{G}) = ({\mathbb{R}}^{d},\mathcal{B}({\mathbb{R}}^{d}))$. Let $({\it \Omega}, \mathcal{F},\mathrm{P})$ be a probability space with a filtration ${\mathcal{F}}_{t}$.

Definition 19.1

Let $\mu$ be a probability measure on $\mathcal{B}({\mathbb{R}}^{d})$ . An adapted process ${X}_{t}$ with values in ${\mathbb{R}}^{d}$ is called a Markov process with initial distribution μ if:

• (1) $\mathrm{P}({X}_{0} \in {\it \Gamma} ) = \mu ({\it \Gamma} )$ for any ${\it \Gamma} \in \mathcal{B}({\mathbb{R}}^{d})$.

• (2) If $s,t \geq 0$ and ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$ is a Borel set, then

  $$\mathrm{P}({X}_{s+t} \in {\it \Gamma} \vert {\mathcal{F}}_{s}) =\mathrm{ P}({X}_{s+t} \in {\it \Gamma} \vert {X}_{s})\ \ \mathit{almost}\ \mathit{surely}.$$

  (19.1)

Definition 19.2

Let ${X}_{t}^{x}$, $x \in {\mathbb{R}}^{d}$ , be a family of processes with values in ${\mathbb{R}}^{d}$ which are adapted to a filtration ${\mathcal{F}}_{t}$ . This family of processes is called a time-homogeneous Markov family if:

• (1) The function $p(t,x,{\it \Gamma} ) =\mathrm{ P}({X}_{t}^{x} \in {\it \Gamma} )$ is Borel-measurable as a function of $x \in {\mathbb{R}}^{d}$ for any $t \geq 0$ and any Borel set ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$.

• (2) $\mathrm{P}({X}_{0}^{x} = x) = 1$ for any $x \in {\mathbb{R}}^{d}$.

• (3) If $s,t \geq 0$, $x \in {\mathbb{R}}^{d}$ , and ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$ is a Borel set, then

  $$\mathrm{P}({X}_{s+t}^{x} \in {\it \Gamma} \vert {\mathcal{F}}_{ s}) = p(t,{X}_{s}^{x},{\it \Gamma} )\ \ \mathit{almost}\ \mathit{surely}.$$

The function $p(t,x,{\it \Gamma} )$ is called the transition function for the Markov family ${X}_{t}^{x}$. It has the following properties:

• (1′) For fixed $t \geq 0$ and $x \in {\mathbb{R}}^{d}$, the function $p(t,x,{\it \Gamma} )$, as a function of ${\it \Gamma}$, is a probability measure, while for fixed t and ${\it \Gamma}$ it is a measurable function of x.

• (2′) $p(0,x,\{x\}) = 1$.

• (3′) If $s,t \geq 0$, $x \in {\mathbb{R}}^{d}$, and ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$ is a Borel set, then

  $$p(s + t,x,{\it \Gamma} ) =\int _{{\mathbb{R}}^{d}}p(s,x,dy)p(t,y,{\it \Gamma} ).$$

  The first two properties are obvious. For the third one it is sufficient to write

  $$p(s + t,x,{\it \Gamma} ) =\mathrm{ P}({X}_{s+t}^{x} \in {\it \Gamma} ) =\mathrm{ E}\mathrm{P}({X}_{ s+t}^{x} \in {\it \Gamma} \vert {\mathcal{F}}_{ s})$$
  $$=\mathrm{ E}p(t,{X}_{s}^{x},{\it \Gamma} ) =\int _{{\mathbb{R}}^{d}}p(s,x,dy)p(t,y,{\it \Gamma} ),$$

  where the last equality follows by Theorem 3.14.

Now assume that we are given a function $p(t,x,{\it \Gamma} )$ with properties (1′)–(3′) and a measure μ on $\mathcal{B}({\mathbb{R}}^{d})$. As we shall see below, this pair can be used to define a measure on the space of all functions $\widetilde{{\it \Omega} } =\{\widetilde{ {\it \omega }} : {\mathbb{R}}^{+} \rightarrow {\mathbb{R}}^{d}\}$ in such a way that $\widetilde{{\it \omega }}(t)$ is a Markov process. Recall that in Chap. 5 we defined a Markov chain as the measure corresponding to a Markov transition function and an initial distribution (see the discussion following Definition 5.17).

Let $\widetilde{{\it \Omega} }$ be the set of all functions $\widetilde{{\it \omega }} : {\mathbb{R}}^{+} \rightarrow {\mathbb{R}}^{d}$. Take a finite collection of points $0 \leq {t}_{1} \leq \ldots \leq {t}_{k}<\infty$, and Borel sets ${A}_{1},\ldots, {A}_{k} \in \mathcal{B}({\mathbb{R}}^{d})$. For an elementary cylinder $B =\{\widetilde{ {\it \omega }} :\widetilde{ {\it \omega }}({t}_{1}) \in {A}_{1},\ldots, \widetilde{{\it \omega }}({t}_{k}) \in {A}_{k}\}$, we define the finite-dimensional measure $\mathrm{{P}}_{{t}_{1},\ldots, {t}_{k}}^{\mu }(B)$ via

$$\mathrm{{P}}_{{t}_{1},\ldots, {t}_{k}}^{\mu }(B) =\int _{{\mathbb{R}}^{d}}\mu (dx)\int _{{A}_{1}}p({t}_{1},x,d{y}_{1})\int _{{A}_{2}}p({t}_{2} - {t}_{1},{y}_{1},d{y}_{2})\ldots $$

$$\int _{{A}_{k-1}}p({t}_{k-1} - {t}_{k-2},{y}_{k-2},d{y}_{k-1})\int _{{A}_{k}}p({t}_{k} - {t}_{k-1},{y}_{k-1},d{y}_{k}).$$

The family of finite-dimensional probability measures $\mathrm{{P}}_{{t}_{1},\ldots, {t}_{k}}^{\mu }$ is consistent and, by the Kolmogorov Theorem, defines a measure $\mathrm{{P}}^{\mu }$ on $\mathcal{B}$, the σ-algebra generated by all the elementary cylindrical sets. Let ${\mathcal{F}}_{t}$ be the $\sigma$-algebra generated by the elementary cylindrical sets $B =\{\widetilde{ {\it \omega }} :\widetilde{ {\it \omega }}({t}_{1}) \in {A}_{1},\ldots, \widetilde{{\it \omega }}({t}_{k}) \in {A}_{k}\}$ with $0 \leq {t}_{1} \leq \ldots \leq {t}_{k} \leq t$, and ${X}_{t}(\widetilde{{\it \omega }}) =\widetilde{ {\it \omega }}(t)$. We claim that ${X}_{t}$ is a Markov process on $(\widetilde{{\it \Omega} },\mathcal{B},\mathrm{{P}}^{\mu })$ relative to the filtration ${\mathcal{F}}_{t}$. Clearly, the first property in Definition 19.1 holds. To verify the second property, it is sufficient to show that

$$\mathrm{{P}}^{\mu }(B\bigcap \limits \{{X}_{s+t} \in {\it \Gamma} \}) =\int _{B}p(t,{X}_{s},{\it \Gamma} )d\mathrm{{P}}^{\mu }$$

(19.2)

for any $B \in {\mathcal{F}}_{s}$, since the integrand on the right-hand side is clearly $\sigma ({X}_{s})$-measurable. When $B =\{\widetilde{ {\it \omega }} :\widetilde{ {\it \omega }}({t}_{1}) \in {A}_{1},\ldots, \widetilde{{\it \omega }}({t}_{k}) \in {A}_{k}\}$ with $0 \leq {t}_{1} \leq \ldots \leq {t}_{k} \leq s$, both sides of (19.2) are equal to

$$\int _{{\mathbb{R}}^{d}}\mu (dx)\int _{{A}_{1}}p({t}_{1},x,d{y}_{1})\int _{{A}_{2}}p({t}_{2} - {t}_{1},{y}_{1},d{y}_{2})\ldots $$

$$\int _{{A}_{k}}p({t}_{k} - {t}_{k-1},{y}_{k-1},d{y}_{k})\int _{{\mathbb{R}}^{d}}p(s - {t}_{k},{y}_{k},dy)p(t,y,{\it \Gamma} ).$$

Since such elementary cylindrical sets form a π-system, it follows from Lemma 4.13 that (19.2) holds for all $B \in {\mathcal{F}}_{s}$.

Let $\widetilde{{\it \Omega} }$ be the space of all functions from ${\mathbb{R}}^{+}$ to ${\mathbb{R}}^{d}$ with the σ-algebra $\mathcal{B}$ generated by cylindrical sets. We can define a family of shift transformations ${\theta }_{s} :\widetilde{ {\it \Omega} } \rightarrow \widetilde{ {\it \Omega} }$, $s \geq 0$, which act on functions $\widetilde{{\it \omega }} \in \widetilde{ {\it \Omega} }$ via

$$({\theta }_{s}\widetilde{{\it \omega }})(t) =\widetilde{ {\it \omega }}(s + t).$$

If X _t is a random process with realizations denoted by ${X}_{\cdot }({\it \omega })$, we can apply θ _s to each realization to get a new process, whose realizations will be denoted by ${X}_{s+\cdot }({\it \omega })$.

If $f :\widetilde{ {\it \Omega} } \rightarrow \mathbb{R}$ is a bounded measurable function and ${X}_{t}^{x}$, $x \in {\mathbb{R}}^{d}$, is a Markov family, we can define the function ${\varphi }_{f}(x) : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$ as

$${\varphi }_{f}(x) =\mathrm{ E}f({X}_{\cdot }^{x}).$$

Now we can formulate an important consequence of the Markov property.

Lemma 19.3

Let ${X}_{t}^{x}$, $x \in {\mathbb{R}}^{d}$ , be a Markov family of processes relative to a filtration ${\mathcal{F}}_{t}$ . If $f :\widetilde{ {\it \Omega} } \rightarrow \mathbb{R}$ is a bounded measurable function, then

$$\mathrm{E}(f({X}_{s+\cdot }^{x})\vert {\mathcal{F}}_{ s}) = {\varphi }_{f}({X}_{s}^{x})\ \ \mathit{almost}\ \mathit{surely}.$$

(19.3)

Proof

Let us show that for any bounded measurable function $g : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$ and $s,t \geq 0$,

$$\mathrm{E}(g({X}_{s+t}^{x})\vert {\mathcal{F}}_{ s}) =\int _{{\mathbb{R}}^{d}}g(y)p(t,{X}_{s}^{x},dy)\ \ \mathrm{almost}\ \mathrm{surely}.$$

(19.4)

Indeed, if g is the indicator function of a Borel set ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$, this statement is part of the definition of a Markov family. By linearity, it also holds for finite linear combinations of indicator functions. Therefore, (19.4) holds for all bounded measurable functions, since they can be uniformly approximated by finite linear combinations of indicator functions.

To prove (19.3), we first assume that f is the indicator function of an elementary cylindrical set, that is $f = {\chi }_{A}$, where

$$A =\{\widetilde{ {\it \omega }} :\widetilde{ {\it \omega }}({t}_{1}) \in {A}_{1},\ldots, \widetilde{{\it \omega }}({t}_{k}) \in {A}_{k}\}$$

with $0 \leq {t}_{1} \leq \ldots \leq {t}_{k}$ and some Borel sets ${A}_{1},\ldots, {A}_{k} \subseteq {\mathbb{R}}^{d}$. In this case the left-hand side of (19.3) is equal to $\mathrm{P}({X}_{s+{t}_{1}}^{x} \in {A}_{1},\ldots, {X}_{s+{t}_{k}}^{x} \in {A}_{k}\vert {\mathcal{F}}_{s})$. We can transform this expression by inserting conditional expectations with respect to ${\mathcal{F}}_{s+{t}_{k-1}}$,…,${\mathcal{F}}_{s+{t}_{1}}$ and applying (19.4) repeatedly. We thus obtain

$$\begin{array}{rcl} & & \mathrm{P}({X}_{s+{t}_{1}}^{x} \in {A}_{ 1},\ldots, {X}_{s+{t}_{k}}^{x} \in {A}_{ k}\vert {\mathcal{F}}_{s}) \\ & & \quad =\mathrm{ E}({\chi }_{\{{X}_{s+{t}_{ 1}}^{x}\in {A}_{1}\}}\ldots {\chi }_{\{{X}_{s+{t}_{ k}}^{x}\in {A}_{k}\}}\vert {\mathcal{F}}_{s}) \\ & & \quad =\mathrm{ E}({\chi }_{\{{X}_{s+{t}_{ 1}}^{x}\in {A}_{1}\}}\ldots {\chi }_{\{{X}_{s+{t}_{ k-1}}^{x}\in {A}_{k-1}\}}\mathrm{E}({\chi }_{\{{X}_{s+{t}_{k}}^{x}\in {A}_{k}\}}\vert {\mathcal{F}}_{s+{t}_{k-1}})\vert {\mathcal{F}}_{s}) \\ & & \quad =\mathrm{ E}({\chi }_{\{{X}_{s+{t}_{ 1}}^{x}\in {A}_{1}\}}\ldots {\chi }_{\{{X}_{s+{t}_{ k-1}}^{x}\in {A}_{k-1}\}}p({t}_{k} - {t}_{k-1},{X}_{s+{t}_{k-1}}^{x},{A}_{ k})\vert {\mathcal{F}}_{s}) \\ & & \quad =\mathrm{ E}({\chi }_{\{{X}_{s+{t}_{ 1}}^{x}\in {A}_{1}\}}\ldots {\chi }_{\{{X}_{s+{t}_{ k-2}}^{x}\in {A}_{k-2}\}}\mathrm{E}({\chi }_{\{{X}_{s+{t}_{k-1}}^{x}\in {A}_{k-1}\}} \\ & & \qquad p({t}_{k} - {t}_{k-1},{X}_{s+{t}_{k-1}}^{x},{A}_{ k})\vert {\mathcal{F}}_{s+{t}_{k-2}})\vert {\mathcal{F}}_{s}) \\ & & \quad =\mathrm{ E}({\chi }_{\{{X}_{s+{t}_{ 1}}^{x}\in {A}_{1}\}}\ldots {\chi }_{\{{X}_{s+{t}_{ k-2}}^{x}\in {A}_{k-2}\}} \\ \end{array}$$

$$\begin{array}{c}\int _{{A}_{k-1}}p({t}_{k-1} - {t}_{k-2},{X}_{s+{t}_{k-2}}^{x},d{y}_{ k-1})p({t}_{k} - {t}_{k-1},{y}_{k-1},{A}_{k})\vert {\mathcal{F}}_{s}) = \ldots \\ =\int _{{A}_{1}}p({t}_{1} - s,{X}_{s}^{x},d{y}_{ 1})\int _{{A}_{2}}p({t}_{2} - {t}_{1},{y}_{1},d{y}_{2})\ldots \\ \int _{{A}_{k-1}}p({t}_{k-1} - {t}_{k-2},{y}_{k-2},d{y}_{k-1})p({t}_{k} - {t}_{k-1},{y}_{k-1},{A}_{k}).\end{array}$$

Note that ${\varphi }_{f}(x)$ is equal to $\mathrm{P}({X}_{{t}_{1}}^{x} \in {A}_{1},\ldots, {X}_{{t}_{k}}^{x} \in {A}_{k})$. If we insert conditional expectations with respect to ${\mathcal{F}}_{{t}_{k-1}},\ldots, {\mathcal{F}}_{{t}_{1}}$, ${\mathcal{F}}_{0}$ and apply (19.4) repeatedly,

$$\begin{array}{c}\mathrm{P}({X}_{{t}_{1}}^{x} \in {A}_{ 1},\ldots, {X}_{{t}_{k}}^{x} \in {A}_{ k}) =\int _{{A}_{1}}p({t}_{1} - s,x,d{y}_{1})\int _{{A}_{2}}p({t}_{2} - {t}_{1},{y}_{1},d{y}_{2})\ldots \\ \int _{{A}_{k-1}}p({t}_{k-1} - {t}_{k-2},{y}_{k-2},d{y}_{k-1})p({t}_{k} - {t}_{k-1},{y}_{k-1},{A}_{k}).\end{array}$$

If we replace x with ${X}_{s}^{x}$, we see that the right-hand side of (19.3) coincides with the left-hand side if f is an indicator function of an elementary cylinder.

Next, let us show that (19.3) holds if $f = {\chi }_{A}$ is an indicator function of any set $A \in \mathcal{B}$. Indeed, elementary cylinders form a π-system, while the collection of sets A for which (19.3) is true with $f = {\chi }_{A}$ is a Dynkin system. By Lemma 4.13, formula (19.3) holds for $f = {\chi }_{A}$, where A is any element from the σ-algebra generated by the elementary cylinders, that is $\mathcal{B}$.

Finally, any bounded measurable function f can be uniformly approximated by finite linear combinations of indicator functions.

Remark 19.4

If we assume that ${X}_{t}^{x}$ are continuous processes, Lemma 19.3 applies in the case when f is a bounded measurable function on $C([0,\infty ))$.

Remark 19.5

The arguments in the proof of the lemma imply that ${\varphi }_{f}$ is a measurable function for any bounded measurable f. It is enough to take s = 0.

It is sometimes useful to formulate the third condition of Definition 19.2 in a slightly different way. Let g be a bounded measurable function $g : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$. Then we can define a new function ${\psi }_{g} : {\mathbb{R}}^{+} \times {\mathbb{R}}^{d} \rightarrow \mathbb{R}$ by

$${\psi }_{g}(t,x) =\mathrm{ E}g({X}_{t}^{x}).$$

Note that ${\psi }_{g}(t,x) = {\varphi }_{f}(x)$, if we define $f :\widetilde{ {\it \Omega} } \rightarrow \mathbb{R}$ by $f(\widetilde{{\it \omega }}) = g(\widetilde{{\it \omega }}(t))$.

Lemma 19.6

If conditions (1) and (2) of Definition 19.2 are satisfied, then condition (3) is equivalent to the following:

• (3′) If $s,t \geq 0$, $x \in {\mathbb{R}}^{d}$ , and $g : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$ is a bounded continuous function, then

  $$\mathrm{E}(g({X}_{s+t}^{x})\vert {\mathcal{F}}_{ s}) = {\psi }_{g}(t,{X}_{s}^{x})\ \ \mathit{almost}\ \mathit{surely}.$$

Proof

Clearly, (3) implies (3′) as a particular case of Lemma 19.3. Conversely, let $s,t \geq 0$ and $x \in {\mathbb{R}}^{d}$ be fixed, and assume that ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$ is a closed set. In this case we can find a sequence of non-negative bounded continuous functions ${g}_{n}$ such that ${g}_{n}(x) \downarrow {\chi }_{{\it \Gamma} }(x)$ for all $x \in {\mathbb{R}}^{d}$. By taking the limit as $n \rightarrow \infty$ in the equality

$$\mathrm{E}({g}_{n}({X}_{s+t}^{x})\vert {\mathcal{F}}_{ s}) = {\psi }_{{g}_{n}}(t,{X}_{s}^{x})\ \ \mathrm{almost}\ \mathrm{surely},$$

we obtain

$$\mathrm{P}({X}_{s+t}^{x} \in {\it \Gamma} \vert {\mathcal{F}}_{ s}) = p(t,{X}_{s}^{x},{\it \Gamma} )\ \ \mathrm{almost}\ \mathrm{surely}$$

(19.5)

for closed sets Γ. The collection of all closed sets is a π-system, while the collection of all sets Γ for which (19.5) holds is a Dynkin system. Therefore (19.5) holds for all Borel sets Γ by Lemma 4.13.

3 Markov Property of Brownian Motion

Let ${W}_{t}$ be a d-dimensional Brownian motion relative to a filtration ${\mathcal{F}}_{t}$. Consider the family of processes ${W}_{t}^{x} = x + {W}_{t}$. Let us show that ${W}_{t}^{x}$ is a time-homogeneous Markov family relative to the filtration ${\mathcal{F}}_{t}$.

Since ${W}_{t}^{x}$ is a Gaussian vector for fixed t, there is an explicit formula for $\mathrm{P}({W}_{t}^{x} \in {\it \Gamma} )$. Namely,

$$p(t,x,{\it \Gamma} ) =\mathrm{ P}({W}_{t}^{x} \in {\it \Gamma} ) = {(2\pi t)}^{-\frac{d} {2} }\int _{{\it \Gamma} }\exp (-\vert \vert y - x\vert {\vert }^{2}/2t)dy$$

(19.6)

if $t>0$. As a function of x, $p(0,x,{\it \Gamma} )$ is simply the indicator function of the set Γ. Therefore, $p(t,x,{\it \Gamma} )$ is a Borel-measurable function of x for any $t \geq 0$ and any Borel set Γ.

Clearly, the second condition of Definition 19.2 is satisfied by the family of processes ${W}_{t}^{x}$.

In order to verify the third condition, let us assume that $t>0$, since otherwise the condition is satisfied. For a Borel set $S \subseteq {\mathbb{R}}^{2d}$ and $x \in {\mathbb{R}}^{d}$, let

$${S}_{x} =\{ y \in {\mathbb{R}}^{d} : (x,y) \in S\}.$$

Let us show that

$$\mathrm{P}(({W}_{s}^{x},{W}_{ s+t}^{x} - {W}_{ s}^{x}) \in S\vert {\mathcal{F}}_{ s}) = {(2\pi t)}^{-\frac{d} {2} }\int _{{S}_{{W}_{ s}^{x}}}\exp (-\vert \vert y\vert {\vert }^{2}/2t)dy.$$

(19.7)

First, assume that $S = A \times B$, where A and B are Borel subsets of ${\mathbb{R}}^{d}$. In this case,

$$\mathrm{P}({W}_{s}^{x} \in A,{W}_{ s+t}^{x} - {W}_{ s}^{x} \in B\vert {\mathcal{F}}_{ s}) = {\chi }_{\{{W}_{s}^{x}\in A\}}\mathrm{P}({W}_{s+t}^{x} - {W}_{ s}^{x} \in B\vert {\mathcal{F}}_{ s})$$

$$= {\chi }_{\{{W}_{s}^{x}\in A\}}\mathrm{P}({W}_{s+t}^{x} - {W}_{ s}^{x} \in B) = {\chi }_{\{{ W}_{s}^{x}\in A\}}{(2\pi t)}^{-\frac{d} {2} }\int _{B}\exp (-\vert \vert y\vert {\vert }^{2}/2t)dy,$$

since ${W}_{s+t}^{x} - {W}_{s}^{x}$ is independent of ${\mathcal{F}}_{s}$. Thus, (19.7) holds for sets of the form $S = A \times B$. The collection of sets that can be represented as such a direct product is a π-system. Since the collection of sets for which (19.7) holds is a Dynkin system, we can apply Lemma 4.13 to conclude that (19.7) holds for all Borel sets. Finally, let us apply (19.7) to the set $S =\{ (x,y) : x + y \in {\it \Gamma} \}$. Then,

$$\mathrm{P}({W}_{s+t}^{x} \in {\it \Gamma} \vert {\mathcal{F}}_{ s}) = {(2\pi t)}^{-\frac{d} {2} }\int _{{\it \Gamma} }\exp (-\vert \vert y - {W}_{s}^{x}\vert {\vert }^{2}/2t)dy = p(t,{W}_{ s}^{x},{\it \Gamma} ).$$

This proves that the third condition of Definition 19.2 is satisfied, and that ${W}_{t}^{x}$ is a Markov family.

4 The Augmented Filtration

Let ${W}_{t}$ be a d-dimensional Brownian motion on a probability space $({\it \Omega}, \mathcal{F},\mathrm{P})$. We shall exhibit a probability space and a filtration satisfying the usual conditions such that ${W}_{t}$ is a Brownian motion relative to this filtration.

Recall that ${\mathcal{F}}_{t}^{W} = \sigma ({W}_{s},s \leq t)$ is the filtration generated by the Brownian motion, and ${\mathcal{F}}^{W} = \sigma ({W}_{s},s \in {\mathbb{R}}^{+})$ is the σ-algebra generated by the Brownian motion. Let $\mathcal{N}$ be the collection of all $\mathrm{P}$-negligible sets relative to ${\mathcal{F}}^{W}$, that is $A \in \mathcal{N}$ if there is an event $B \in {\mathcal{F}}^{W}$ such that $A \subseteq B$ and $\mathrm{P}(B) = 0$. Define the new filtration $\widetilde{{\mathcal{F}}}_{t}^{W} = \sigma ({\mathcal{F}}_{t}^{W} \bigcup \limits \mathcal{N})$, called the augmentation of ${\mathcal{F}}_{t}^{W}$, and the new $\sigma$-algebra $\widetilde{{\mathcal{F}}}^{W} = \sigma ({\mathcal{F}}^{W} \bigcup \limits \mathcal{N})$.

Now consider the process ${W}_{t}$ on the probability space $({\it \Omega}, \widetilde{{\mathcal{F}}}^{W},\mathrm{P})$, and note that it is a Brownian motion relative to the filtration $\widetilde{{\mathcal{F}}}_{t}^{W}$.

Lemma 19.7

The augmented filtration $\widetilde{{\mathcal{F}}}_{t}^{W}$ satisfies the usual conditions.

Proof

It is clear that $\widetilde{{\mathcal{F}}}_{0}^{W}$ contains all the $\mathrm{P}$-negligible events from $\widetilde{{\mathcal{F}}}^{W}$. It remains to prove that $\widetilde{{\mathcal{F}}}_{t}^{W}$ is right-continuous.

Our first observation is that ${W}_{t} - {W}_{s}$ is independent of the $\sigma$-algebra ${\mathcal{F}}_{s+}^{W}$ if $0 \leq s \leq t$. Indeed, assuming that $s<t$, the variable ${W}_{t} - {W}_{s+\delta }$ is independent of ${\mathcal{F}}_{s+}^{W}$ for all positive $\delta$. Then, as $\delta \downarrow 0$, the variable ${W}_{t} - {W}_{s+\delta }$ tends to ${W}_{t} - {W}_{s}$ almost surely, which implies that ${W}_{t} - {W}_{s}$ is also independent of ${\mathcal{F}}_{s+}^{W}$.

Next, we claim that ${\mathcal{F}}_{s+}^{W} \subseteq \widetilde{{\mathcal{F}}}_{s}^{W}$. Indeed, let ${t}_{1},\ldots, {t}_{k} \geq s$ for some positive integer k, and let ${B}_{1},\ldots, {B}_{k}$ be Borel subsets of ${\mathbb{R}}^{d}$. By Lemma 19.3, the random variable $\mathrm{P}({W}_{{t}_{1}} \in {B}_{1},\ldots, {W}_{{t}_{k}} \in {B}_{k}\vert {\mathcal{F}}_{s}^{W})$ has a $\sigma ({W}_{s})$-measurable version. The same remains true if we replace ${\mathcal{F}}_{s}^{W}$ by ${\mathcal{F}}_{s+}^{W}$. Indeed, in the statement of the Markov property for the Brownian motion, we can replace ${\mathcal{F}}_{s}^{W}$ by ${\mathcal{F}}_{s+}^{W}$, since in the arguments of Sect. 19.3 we can use that ${W}_{t} - {W}_{s}$ is independent of ${\mathcal{F}}_{s+}^{W}$.

Let ${s}_{1},\ldots, {s}_{{k}_{1}} \leq s \leq {t}_{1},\ldots, {t}_{{k}_{2}}$ for some positive integers ${k}_{1}$ and ${k}_{2}$, and let ${A}_{1},\ldots, {A}_{{k}_{1}},{B}_{1},\ldots, {B}_{{k}_{2}}$ be Borel subsets of ${\mathbb{R}}^{d}$. Then,

$$\begin{array}{c}\mathrm{P}({W}_{{s}_{1}} \in {A}_{1},\ldots, {W}_{{s}_{{k}_{ 1}}} \in {A}_{{k}_{1}},{W}_{{t}_{1}} \in {B}_{1},\ldots, {W}_{{t}_{{k}_{ 2}}} \in {B}_{{k}_{2}}\vert {\mathcal{F}}_{s+}^{W})\\ = {\chi }_{\{{W}_{{s}_{ 1}}\in {A}_{1},\ldots, {W}_{{s}_{{k}_{1}}}\in {A}_{{k}_{1}}\}}\mathrm{P}({W}_{{t}_{1}} \in {B}_{1},\ldots, {W}_{{t}_{{k}_{ 2}}} \in {B}_{{k}_{2}}\vert {\mathcal{F}}_{s+}^{W}),\end{array}$$

which has a ${\mathcal{F}}_{s}^{W}$-measurable version. The collection of sets $A \in {\mathcal{F}}^{W}$, for which $\mathrm{P}(A\vert {\mathcal{F}}_{s+}^{W})$ has a ${\mathcal{F}}_{s}^{W}$-measurable version, forms a Dynkin system. Therefore, by Lemma 4.13, $\mathrm{P}(A\vert {\mathcal{F}}_{s+}^{W})$ has a ${\mathcal{F}}_{s}^{W}$-measurable version for each $A \in {\mathcal{F}}^{W}$. This easily implies our claim that ${\mathcal{F}}_{s+}^{W} \subseteq \widetilde{{\mathcal{F}}}_{s}^{W}$.

Finally, let us show that $\widetilde{{\mathcal{F}}}_{s+}^{W} \subseteq \widetilde{{\mathcal{F}}}_{s}^{W}$. Let $A \in \widetilde{{\mathcal{F}}}_{s+}^{W}$. Then $A \in \widetilde{{\mathcal{F}}}_{s+ \frac{1} {n} }^{W}$ for every positive integer n. We can find sets ${A}_{n} \in {\mathcal{F}}_{s+ \frac{1} {n} }^{W}$ such that $A{\it \Delta} {A}_{n} \in \mathcal{N}$. Define

$$B = \bigcap \limits_{m=1}^{\infty }\bigcup \limits _{n=m}^{\infty }{A}_{ n}.$$

Then $B \in {\mathcal{F}}_{s+}^{W}$, since $B \in {\mathcal{F}}_{s+ \frac{1} {m} }^{W}$ for any m. It remains to show that $A{\it \Delta} B \in \mathcal{N}$. Indeed,

$$B \setminus A \subseteq \bigcup \limits_{n=1}^{\infty }({A}_{ n} \setminus A) \in \mathcal{N},$$

while

$$\begin{array}{rcl} A \setminus B& =& A\bigcap \limits (\bigcup \limits _{m=1}^{\infty }\bigcap \limits_{n=m}^{\infty }({\it \Omega} \setminus {A}_{ n})) =\bigcup \limits _{m=1}^{\infty }(A\bigcap \limits (\bigcap \limits _{n=m}^{\infty }({\it \Omega} \setminus {A}_{ n}))) \\ & & \quad \subseteq \bigcup \limits _{m=1}^{\infty }(A\bigcap \limits ({\it \Omega} \setminus {A}_{m})) =\bigcup \limits _{m=1}^{\infty }(A \setminus {A}_{ m}) \in \mathcal{N}\end{array}.$$

Lemma 19.8 (Blumenthal Zero-One Law).

If $A \in \widetilde{{\mathcal{F}}}_{0}^{W}$ , then either $\mathrm{P}(A) = 0$ or $\mathrm{P}(A) = 1$.

Proof

For $A \in \widetilde{{\mathcal{F}}}_{0}^{W}$, there is a set ${A}_{0} \in {\mathcal{F}}_{0}^{W}$ such that $A{\it \Delta} {A}_{0} \in \mathcal{N}$. The set ${A}_{0}$ can be represented as $\{{\it \omega }\in {\it \Omega} : {W}_{0}({\it \omega }) \in B\}$, where B is a Borel subset of ${\mathbb{R}}^{d}$. Now it is clear that $\mathrm{P}({A}_{0})$ is equal to either 0 or 1, depending on whether the set B contains the origin. Since $\mathrm{P}(A) =\mathrm{ P}({A}_{0})$, we obtain the desired result.

5 Definition of the Strong Markov Property

It is sometimes necessary in the formulation of the Markov property to replace ${\mathcal{F}}_{s}$ by a $\sigma$-algebra ${\mathcal{F}}_{\sigma }$, where σ is a stopping time. This leads to the notions of a strong Markov process and a strong Markov family. First, we need the following definition.

Definition 19.9

A random process X _t is called progressively measurable with respect to a filtration ${\mathcal{F}}_{t}$ if ${X}_{s}({\it \omega })$ is ${\mathcal{F}}_{t} \times \mathcal{B}([0,t])$ -measurable as a function of $(\omega, s) \in {\it \Omega} \times [0,t]$ for each fixed $t \geq 0$.

For example, any progressively measurable process is adapted, and any continuous adapted process is progressively measurable (see Problem 1). If X _t is progressively measurable and τ is a stopping time, then ${X}_{t\wedge \tau }$ is also progressively measurable, and ${X}_{\tau }$ is ${\mathcal{F}}_{\tau }$-measurable (see Problem 2).

Definition 19.10

Let μ be a probability measure on $\mathcal{B}({\mathbb{R}}^{d})$ . A progressively measurable process X _t (with respect to filtration ${\mathcal{F}}_{t}$ ) with values in ${\mathbb{R}}^{d}$ is called a strong Markov process with initial distribution μ if:

• (1) $\mathrm{P}({X}_{0} \in {\it \Gamma} ) = \mu ({\it \Gamma} )$ for any ${\it \Gamma} \in \mathcal{B}({\mathbb{R}}^{d})$.

• (2) If $t \geq 0$, $\sigma$ is a stopping time of ${\mathcal{F}}_{t}$ , and ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$ is a Borel set, then

  $$\mathrm{P}({X}_{\sigma +t} \in {\it \Gamma} \vert {\mathcal{F}}_{\sigma }) =\mathrm{ P}({X}_{\sigma +t} \in {\it \Gamma} \vert {X}_{\sigma })\ \ \mathit{almost}\ \mathit{surely}.$$

  (19.8)

Definition 19.11

Let ${X}_{t}^{x}$, $x \in {\mathbb{R}}^{d}$ , be a family of progressively measurable processes with values in ${\mathbb{R}}^{d}$ . This family of processes is called a time-homogeneous strong Markov family if:

• (1) The function $p(t,x,{\it \Gamma} ) =\mathrm{ P}({X}_{t}^{x} \in {\it \Gamma} )$ is Borel-measurable as a function of $x \in {\mathbb{R}}^{d}$ for any $t \geq 0$ and any Borel set ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$.

• (2) $\mathrm{P}({X}_{0}^{x} = x) = 1$ for any $x \in {\mathbb{R}}^{d}$.

• (3) If $t \geq 0$ , σ is a stopping time of ${\mathcal{F}}_{t}$, $x \in {\mathbb{R}}^{d}$ , and ${\it \Gamma} \subseteq {\mathbb{R}}^{d}$ is a Borel set, then

  $$\mathrm{P}({X}_{\sigma +t}^{x} \in {\it \Gamma} \vert {\mathcal{F}}_{ \sigma }) = p(t,{X}_{\sigma }^{x},{\it \Gamma} )\ \ \mathit{almost}\ \mathit{surely}.$$

We have the following analog of Lemmas 19.3 and 19.6.

Lemma 19.12

Let ${X}_{t}^{x}$, $x \in {\mathbb{R}}^{d}$ , be a strong Markov family of processes relative to a filtration ${\mathcal{F}}_{t}$ . If $f :\widetilde{ {\it \Omega} } \rightarrow \mathbb{R}$ is a bounded measurable function and σ is a stopping time of ${\mathcal{F}}_{t}$ , then

$$\mathrm{E}(f({X}_{\sigma +\cdot }^{x})\vert {\mathcal{F}}_{ \sigma }) = {\varphi }_{f}({X}_{\sigma }^{x})\ \ \mathit{almost}\ \mathit{surely},$$

(19.9)

where ${\varphi }_{f}(x) =\mathrm{ E}f({X}_{\cdot }^{x})$.

Remark 19.13

If we assume that ${X}_{t}^{x}$ are continuous processes, Lemma 19.12 applies in the case when f is a bounded measurable function on $C([0,\infty ))$.

Lemma 19.14

If conditions (1) and (2) of Definition 19.11 are satisfied, then condition (3) is equivalent to the following:

• (3′) If $t \geq 0$ , σ is a stopping time of ${\mathcal{F}}_{t}$, $x \in {\mathbb{R}}^{d}$ , and $g : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$ is a bounded continuous function, then

  $$\mathrm{E}(g({X}_{\sigma +t}^{x})\vert {\mathcal{F}}_{ \sigma }) = {\psi }_{g}(t,{X}_{\sigma }^{x})\ \ \mathit{almost}\ \mathit{surely},$$

  where ${\psi }_{g}(t,x) =\mathrm{ E}g({X}_{t}^{x})$.

We omit the proofs of these lemmas since they are analogous to those in Sect. 19.3. Let us derive another useful consequence of the strong Markov property.

Lemma 19.15

Let ${X}_{t}^{x}$, $x \in {\mathbb{R}}^{d}$ , be a strong Markov family of processes relative to a filtration ${\mathcal{F}}_{t}$ . Assume that ${X}_{t}^{x}$ is right-continuous for every $x \in {\mathbb{R}}^{d}$ . Let σ and τ be stopping times of ${\mathcal{F}}_{t}$ such that $\sigma \leq \tau$ and τ is ${\mathcal{F}}_{\sigma }$ -measurable. Then for any bounded measurable function $g : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$,

$$\mathrm{E}(g({X}_{\tau }^{x})\vert {\mathcal{F}}_{ \sigma }) = {\psi }_{g}(\tau - \sigma, {X}_{\sigma }^{x})\ \ \mathit{almost}\ \mathit{surely},$$

where ${\psi }_{g}(t,x) =\mathrm{ E}g({X}_{t}^{x})$.

Remark 19.16

The function ${\psi }_{g}(t,x)$ is jointly measurable in (t, x) if ${X}_{t}^{x}$ is right-continuous. Indeed, if g is continuous, then ${\psi }_{g}(t,x)$ is right-continuous in t. This is sufficient to justify the joint measurability, since it is measurable in x for each fixed t. Using arguments similar to those in the proof of Lemma 19.6, one can show that ${\psi }_{g}(t,x)$ is jointly measurable when g is an indicator function of a measurable set. Approximating an arbitrary bounded measurable function by finite linear combinations of indicator functions justifies the statement in the case of an arbitrary bounded measurable g.

Proof of Lemma 19.15. First assume that g is a continuous function, and that $\tau - \sigma$ takes a finite or countable number of values. Then we can write ${\it \Omega} = {A}_{1} \cup {A}_{2} \cup \ldots$, where $\tau ({\it \omega }) - \sigma ({\it \omega }) = {t}_{k}$ for ${\it \omega }\in {A}_{k}$, and all t _k are distinct. Thus,

$$\mathrm{E}(g({X}_{\tau }^{x})\vert {\mathcal{F}}_{ \sigma }) =\mathrm{ E}(g({X}_{\sigma +{t}_{k}}^{x})\vert {\mathcal{F}}_{ \sigma })\ \ \mathrm{almost}\ \mathrm{surely}\ \mathrm{on}\ {A}_{k},$$

since $g({X}_{\tau }^{x}) = g({X}_{\sigma +{t}_{k}}^{x})$ on A _k , and ${A}_{k} \in {\mathcal{F}}_{\sigma }$. Therefore,

$$\mathrm{E}(g({X}_{\tau }^{x})\vert {\mathcal{F}}_{ \sigma }) =\mathrm{ E}(g({X}_{\sigma +{t}_{k}}^{x})\vert {\mathcal{F}}_{ \sigma }) = {\psi }_{g}({t}_{k},{X}_{\sigma }^{x}) = {\psi }_{ g}(\tau -\sigma, {X}_{\sigma }^{x})\ \ \mathrm{a.s.}\ \mathrm{on}\ {A}_{ k},$$

which implies

$$\mathrm{E}(g({X}_{\tau }^{x})\vert {\mathcal{F}}_{ \sigma }) = {\psi }_{g}(\tau - \sigma, {X}_{\sigma }^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

(19.10)

If the distribution of $\tau - \sigma$ is not necessarily discrete, it is possible to find a sequence of stopping times ${\tau }_{n}$ such that ${\tau }_{n} - \sigma$ takes at most a countable number of values for each n, ${\tau }_{n} \downarrow \tau$, and each ${\tau }_{n}$ is ${\mathcal{F}}_{\sigma }$-measurable. For example, we can take ${\tau }_{n}({\it \omega }) = \sigma ({\it \omega }) + k/{2}^{n}$ for all ω such that $(k - 1)/{2}^{n} \leq \tau ({\it \omega }) - \sigma ({\it \omega })<k/{2}^{n}$, where $k \geq 1$. Thus,

$$\mathrm{E}(g({X}_{{\tau }_{n}}^{x})\vert {\mathcal{F}}_{ \sigma }) = {\psi }_{g}({\tau }_{n} - \sigma, {X}_{\sigma }^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

Clearly, ${\psi }_{g}({\tau }_{n} - \sigma, x)$ is a Borel-measurable function of x. Since g is bounded and continuous, and X _t is right-continuous,

$$\lim\limits _{n\rightarrow \infty }{\psi }_{g}({\tau }_{n} - \sigma, x) = {\psi }_{g}(\tau - \sigma, x).$$

Therefore, $\lim _{n\rightarrow \infty }{\psi }_{g}({\tau }_{n} - \sigma, {X}_{\sigma }^{x}) = {\psi }_{g}(\tau - \sigma, {X}_{\sigma }^{x})$ almost surely. By the Dominated Convergence Theorem for conditional expectations,

$$\lim \limits_{n\rightarrow \infty }\mathrm{E}(g({X}_{{\tau }_{n}}^{x})\vert {\mathcal{F}}_{ \sigma }) =\mathrm{ E}(g({X}_{\tau }^{x})\vert {\mathcal{F}}_{ \sigma }),$$

which implies that (19.10) holds for all σ and τ satisfying the assumptions of the theorem.

As in the proof of Lemma 19.6, we can show that (19.10) holds if g is an indicator function of a measurable set. Since a bounded measurable function can be uniformly approximated by finite linear combinations of indicator functions, (19.10) holds for all bounded measurable g.

6 Strong Markov Property of Brownian Motion

As before, let W _t y be a $d$-dimensional Brownian motion relative to a filtration ${\mathcal{F}}_{t}$, and ${W}_{t}^{x} = x + {W}_{t}$. In this section we show that ${W}_{t}^{x}$ is a time-homogeneous strong Markov family relative to the filtration ${\mathcal{F}}_{t}$.

Since the first two conditions of Definition 19.11 were verified in Sect. 19.3, it remains to verify condition (3′) from Lemma 19.14. Let σ be a stopping time of ${\mathcal{F}}_{t}$, $x \in {\mathbb{R}}^{d}$, and $g : {\mathbb{R}}^{d} \rightarrow \mathbb{R}$ be a bounded continuous function. The case when $t = 0$ is trivial, therefore we can assume that $t>0$. In this case, ${\psi }_{g}(t,x) =\mathrm{ E}g({W}_{t}^{x})$ is a bounded continuous function of x.

First, assume that σ takes a finite or countable number of values. Then we can write ${\it \Omega} = {A}_{1} \cup {A}_{2} \cup \ldots$, where $\sigma ({\it \omega }) = {s}_{k}$ for ${\it \omega }\in {A}_{k}$, and all ${s}_{k}$ are distinct. Since a set $B \subseteq {A}_{k}$ belongs to ${\mathcal{F}}_{\sigma }$ if and only if it belongs to ${\mathcal{F}}_{{s}_{k}}$, and $g({W}_{\sigma +t}^{x}) = g({W}_{{s}_{k}+t}^{x})$ on A _k ,

$$\mathrm{E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}_{ \sigma }) =\mathrm{ E}(g({W}_{{s}_{k}+t}^{x})\vert {\mathcal{F}}_{{ s}_{k}})\ \ \mathrm{almost}\ \mathrm{surely}\ \mathrm{on}\ {A}_{k}.$$

Therefore,

$$\mathrm{E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}_{ \sigma }) =\mathrm{ E}(g({W}_{{s}_{k}+t}^{x})\vert {\mathcal{F}}_{{ s}_{k}}) = {\psi }_{g}(t,{W}_{{s}_{k}}^{x}) = {\psi }_{ g}(t,{W}_{\sigma }^{x})\ \ \mathrm{a.s.}\ \mathrm{on}\ {A}_{ k},$$

which implies that

$$\mathrm{E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}_{ \sigma }) = {\psi }_{g}(t,{W}_{\sigma }^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

(19.11)

If the distribution of $\sigma$ is not necessarily discrete, we can find a sequence of stopping times ${\sigma }_{n}$, each taking at most a countable number of values, such that ${\sigma }_{n}({\it \omega }) \downarrow \sigma ({\it \omega })$ for all ω. We wish to derive (19.11) starting from

$$\mathrm{E}(g({W}_{{\sigma }_{n}+t}^{x})\vert {\mathcal{F}}_{{ \sigma }_{n}}) = {\psi }_{g}(t,{W}_{{\sigma }_{n}}^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

(19.12)

Since the realizations of Brownian motion are continuous almost surely, and ${\psi }_{g}(t,x)$ is a continuous function of x,

$$\lim \limits_{n\rightarrow \infty }{\psi }_{g}(t,{W}_{{\sigma }_{n}}^{x}) = {\psi }_{ g}(t,{W}_{\sigma }^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

Let ${\mathcal{F}}^{+} =\bigcap _{n=1}^{\infty }{\mathcal{F}}_{{\sigma }_{n}}$. By the Doob Theorem (Theorem 16.11),

$$\lim \limits_{n\rightarrow \infty }\mathrm{E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}_{{ \sigma }_{n}}) =\mathrm{ E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}^{+}).$$

We also need to estimate the difference $\mathrm{E}(g({W}_{{\sigma }_{n}+t}^{x})\vert {\mathcal{F}}_{{\sigma }_{n}}) -\mathrm{ E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}_{{\sigma }_{n}})$. Since the sequence $g({W}_{{\sigma }_{n}+t}^{x}) - g({W}_{\sigma +t}^{x})$ tends to zero almost surely, and $g({W}_{{\sigma }_{n}+t}^{x})$ is uniformly bounded, it is easy to show that $\mathrm{E}(g({W}_{{\sigma }_{n}+t}^{x})\vert {\mathcal{F}}_{{\sigma }_{n}}) -\mathrm{ E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}_{{\sigma }_{n}})$ tends to zero in probability. (We leave this statement as an exercise for the reader.) Therefore, upon taking the limit as $n \rightarrow \infty$ in (19.12),

$$\mathrm{E}(g({W}_{\sigma +t}^{x})\vert {\mathcal{F}}^{+}) = {\psi }_{ g}(t,{W}_{\sigma }^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

Since ${\mathcal{F}}_{\sigma } \subseteq {\mathcal{F}}^{+}$, and ${W}_{\sigma }^{x}$ is ${\mathcal{F}}_{\sigma }$-measurable, we can take conditional expectations with respect to ${\mathcal{F}}_{\sigma }$ on both sides of this equality to obtain (19.11). This proves that ${W}_{t}^{x}$ is a strong Markov family.

Let us conclude this section with several examples illustrating the use of the strong Markov property.

Example.

Let us revisit the problem on the distribution of the maximum of Brownian motion. We use the same notation as in Sect. 19.1. Since ${W}_{t}^{x}$ is a strong Markov family, we can apply Lemma 19.15 with $\sigma = {\tau }_{c}$, $\tau = T$, and $g = {\chi }_{(c,\infty )}$. Since $\mathrm{P}({W}_{T}>c\vert {\mathcal{F}}_{{\tau }_{c}}) = 0$ on the event $\{{\tau }_{c} \geq T\}$,

$$\mathrm{P}({W}_{T}>c\vert {\mathcal{F}}_{{\tau }_{c}}) = {\chi }_{\{{\tau }_{c}<T\}}\mathrm{P}({W}_{t}^{c}>c){\vert }_{ t=T-{\tau }_{c}}.$$

Since $\mathrm{P}({W}_{t}^{c}>c) = 1/2$ for all t,

$$\mathrm{P}({W}_{T}>c\vert {\mathcal{F}}_{{\tau }_{c}}) = \frac{1} {2}{\chi }_{\{{\tau }_{c}<T\}},$$

and, after taking expectation on both sides,

$$\mathrm{P}({W}_{T}>c) = \frac{1} {2}\mathrm{P}({\tau }_{c}<T).$$

(19.13)

Since the event $\{{W}_{T}>c\}$ is contained in the event $\{{\tau }_{c}<T\}$, (19.13) implies $\mathrm{P}({\tau }_{c}<T,{W}_{T}<c) =\mathrm{ P}({\tau }_{c}<T,{W}_{T}>c)$, thus justifying the arguments of Sect. 19.1.

Example.

Let W _t be a Brownian motion relative to a filtration ${\mathcal{F}}_{t}$, and σ be a stopping time of ${\mathcal{F}}_{t}$. Define the process $\widetilde{{W}}_{t} = {W}_{\sigma +t} - {W}_{\sigma }$. Let us show that $\widetilde{{W}}_{t}$ is a Brownian motion independent of ${\mathcal{F}}_{\sigma }$.

Let Γ be a Borel subset of ${\mathbb{R}}^{d}$, $t \geq 0$, and let $f :\widetilde{ {\it \Omega} } \rightarrow \mathbb{R}$ be the indicator function of the set $\{\widetilde{{\it \omega }} :\widetilde{ {\it \omega }}(t) -\widetilde{ {\it \omega }}(0) \in {\it \Gamma} \}$. By Lemma 19.12,

$$\mathrm{P}(\widetilde{{W}}_{t} \in {\it \Gamma} \vert {\mathcal{F}}_{\sigma }) =\mathrm{ E}(f({W}_{\sigma +\cdot })\vert {\mathcal{F}}_{\sigma }) = {\varphi }_{f}({W}_{\sigma })\ \ \mathrm{almost}\ \mathrm{surely},$$

where ${\varphi }_{f}(x) =\mathrm{ E}f({W}_{\cdot }^{x}) =\mathrm{ P}({W}_{t}^{x} - {W}_{0}^{x} \in {\it \Gamma} ) =\mathrm{ P}({W}_{t} \in {\it \Gamma} )$, thus showing that ${\varphi }_{f}(x)$ does not depend on x. Therefore, $\mathrm{P}(\widetilde{{W}}_{t} \in {\it \Gamma} \vert {\mathcal{F}}_{\sigma }) =\mathrm{ P}(\widetilde{{W}}_{t} \in {\it \Gamma} )$. Since Γ was an arbitrary Borel set, $\widetilde{{W}}_{t}$ is independent of ${\mathcal{F}}_{\sigma }$.

Now let $k \geq 1$, ${t}_{1},\ldots, {t}_{k} \in {\mathbb{R}}^{+}$, B be a Borel subset of ${\mathbb{R}}^{dk}$, and $f :\widetilde{ {\it \Omega} } \rightarrow \mathbb{R}$ the indicator function of the set $\{\widetilde{{\it \omega }} : (\widetilde{{\it \omega }}({t}_{1}) -\widetilde{ {\it \omega }}(0),\ldots, \widetilde{{\it \omega }}({t}_{k}) -\widetilde{ {\it \omega }}(0)) \in B\}$. By Lemma 19.12,

$$\mathrm{P}((\widetilde{{W}}_{{t}_{1}},\ldots, \widetilde{{W}}_{{t}_{k}}) \in B\vert {\mathcal{F}}_{\sigma }) =\mathrm{ E}(f({W}_{\sigma +\cdot })\vert {\mathcal{F}}_{\sigma }) = {\varphi }_{f}({W}_{\sigma })\ \ \mathrm{almost}\ \mathrm{surely},$$

(19.14)

where

$$\begin{array}{c}{\varphi }_{f}(x) =\mathrm{ E}f({W}_{\cdot }^{x}) =\mathrm{ P}(({W}_{{ t}_{1}}^{x} - {W}_{ 0}^{x},\ldots, {W}_{{ t}_{k}}^{x} - {W}_{ 0}^{x}) \in B)\\ =\mathrm{ P}(({W}_{{t}_{1}},\ldots, {W}_{{t}_{k}}) \in B),\end{array}$$

which does not depend on x. Taking expectation on both sides of (19.14) gives

$$\mathrm{P}((\widetilde{{W}}_{{t}_{1}},\ldots, \widetilde{{W}}_{{t}_{k}}) \in B) =\mathrm{ P}(({W}_{{t}_{1}},\ldots, {W}_{{t}_{k}}) \in B),$$

which shows that $\widetilde{{W}}_{t}$ has the finite-dimensional distributions of a Brownian motion. Clearly, the realizations of $\widetilde{{W}}_{t}$ are continuous almost surely, that is $\widetilde{{W}}_{t}$ is a Brownian motion.

Example.

Let W _t be a d-dimensional Brownian motion and ${W}_{t}^{x} = x + {W}_{t}$. Let D be a bounded open domain in ${\mathbb{R}}^{d}$, and f a bounded measurable function defined on $\partial D$. For a point $x \in D$, we define ${\tau }^{x}$ to be the first time the process ${W}_{t}^{x}$ reaches the boundary of D, that is

$${\tau }^{x}({\it \omega }) =\inf \{ t \geq 0 : {W}_{ t}^{x}({\it \omega }) \in \partial D\}.$$

Since D is a bounded domain, the stopping time ${\tau }^{x}$ is finite almost surely. Let us follow the process ${W}_{t}^{x}$ till it reaches $\partial D$ and evaluate f at the point ${W}_{{\tau }^{x}({\it \omega })}^{x}({\it \omega })$. Let us define

$$u(x) =\mathrm{ E}f({W}_{{\tau }^{x}}^{x}) =\int _{\partial D}f(y)d{\mu }_{x}(y),$$

where ${\mu }_{x}(A) =\mathrm{ P}({W}_{{\tau }^{x}}^{x} \in A)$ is the measure on $\partial D$ induced by the random variable ${W}_{{\tau }^{x}}^{x}$ and $A \in \mathcal{B}(\partial D)$. Let us show that u(x) is a harmonic function, that is ${\it \Delta} u(x) = 0$ for $x \in D$.

Let ${B}^{x}$ be a ball in ${\mathbb{R}}^{d}$ centered at x and contained in D. Let ${\sigma }^{x}$ be the first time the process ${W}_{t}^{x}$ reaches the boundary of ${B}^{x}$, that is

$${\sigma }^{x}({\it \omega }) =\inf \{ t \geq 0 : {W}_{ t}^{x}({\it \omega }) \in \partial {B}^{x}\}.$$

For a continuous function $\widetilde{{\it \omega }} \in \widetilde{ {\it \Omega} }$, denote by $\tau (\widetilde{{\it \omega }})$ the first time $\widetilde{{\it \omega }}$ reaches $\partial D$, and put $\tau (\widetilde{{\it \omega }})$ equal to infinity if $\widetilde{{\it \omega }}$ never reaches $\partial D$, that is

$$\tau (\widetilde{{\it \omega }}) = \left \{\begin{array}{ll} \inf \{t \geq 0 :\widetilde{ {\it \omega }}(t) \in \partial D\}\ \ \ \text{ if}\,\widetilde{{\it \omega }}(t) \in \partial D\,\text{ for some}\,t \in {\mathbb{R}}^{+}, \\ \infty \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \text{ otherwise.} \end{array} \right. $$

Define the function $\widetilde{f}$ on the space $\widetilde{{\it \Omega} }$ via

$$\widetilde{f}(\widetilde{{\it \omega }}) = \left \{\begin{array}{ll} f(\widetilde{{\it \omega }}(\tau (\widetilde{{\it \omega }})))\ \ \ \mbox{ if\ $\widetilde{{\it \omega }}(t) \in \partial D$\ for some $\ t \in {\mathbb{R}}^{+}$,} \\ 0\ \ \ \ \ \ \ \ \ \ \qquad \mbox{ otherwise.} \end{array} \right. $$

Let us apply Lemma 19.12 to the family of processes ${W}_{t}^{x}$, the function $\widetilde{f}$, and the stopping time ${\sigma }^{x}$:

$$\mathrm{E}(\widetilde{f}({W}_{{\sigma }^{x}+\cdot }^{x})\vert {\mathcal{F}}_{{ \sigma }^{x}}) = {\varphi }_{\widetilde{f}}({W}_{{\sigma }^{x}}^{x})\ \ \mathrm{almost}\ \mathrm{surely},$$

where ${\varphi }_{\widetilde{f}}(x) =\mathrm{ E}\widetilde{f}({W}_{\cdot }^{x}) =\mathrm{ E}f({W}_{{\tau }^{x}}^{x}) = u(x)$. The function u(x) is measurable by Remark 19.5. Note that $\widetilde{f}(\widetilde{{\it \omega }}) =\widetilde{ f}(\widetilde{{\it \omega }}(s + \cdot ))$ if $s<\tau (\widetilde{{\it \omega }})$, and therefore the above equality can be rewritten as

$$\mathrm{E}(f({W}_{{\tau }^{x}}^{x})\vert {\mathcal{F}}_{{ \sigma }^{x}}) = u({W}_{{\sigma }^{x}}^{x})\ \ \mathrm{almost}\ \mathrm{surely}.$$

After taking expectation on both sides,

$$u(x) =\mathrm{ E}f({W}_{{\tau }^{x}}^{x}) =\mathrm{ E}u({W}_{{ \sigma }^{x}}^{x}) =\int _{\partial {B}^{x}}u(y)d{\nu }^{x}(y),$$

where ${\nu }^{x}$ is the measure on $\partial {B}^{x}$ induced by the random variable ${W}_{{\sigma }^{x}}^{x}$. Due to the spherical symmetry of Brownian motion, the measure ${\nu }^{x}$ is the uniform measure on the sphere $\partial {B}^{x}$. Thus u(x) is equal to the average value of u(y) over the sphere $\partial {B}^{x}$. For a bounded measurable function u, this property, when valid for all x and all the spheres centered at x and contained in the domain D (which is the case here), is equivalent to u being harmonic (see “Elliptic Partial Differential Equations of Second Order” by D. Gilbarg and N. Trudinger, for example). We shall further discuss the properties of the function u(x) in Sect. 21.2.

7 Problems

• 1. Prove that any right-continuous adapted process is progressively measurable. (Hint: see the proof of Lemma 12.3.)

• 2. Prove that if a process ${X}_{t}$ is progressively measurable with respect to a filtration ${\mathcal{F}}_{t}$, and $\tau$ is a stopping time of the same filtration, then ${X}_{t\wedge \tau }$ is also progressively measurable and ${X}_{\tau }$ is ${\mathcal{F}}_{\tau }$-measurable.

• 3. Let W _t be a one-dimensional Brownian motion. For a positive constant c, define the stopping time ${\tau }_{c}$ as the first time the Brownian motion reaches the level c, that is

  $${\tau }_{c}({\it \omega }) =\inf \{ t \geq 0 : {W}_{t}({\it \omega }) = c\}.$$

  Prove that ${\tau }_{c}<\infty$ almost surely, and find the distribution function of ${\tau }_{c}$. Prove that $\mathrm{E}{\tau }_{c} = \infty$.

• 4. Let ${W}_{t}$ be a one-dimensional Brownian motion. Prove that one can find positive constants c and $\lambda$ such that

  $$\mathrm{P}(\sup\limits_{1\leq s\leq {2}^{t}}\frac{\vert {W}_{s}\vert } {\sqrt{s}} \leq 1) \leq c{e}^{-\lambda t},\ \ t \geq 1.$$

• 5. Let ${W}_{t}$ be a one-dimensional Brownian motion and ${V }_{t} =\int _{0}^{t}{W}_{s}ds$. Prove that the pair $({W}_{t},{V }_{t})$ is a two-dimensional Markov process.

• 6. Let ${W}_{t}$ be a one-dimensional Brownian motion. Find $\mathrm{P}(\sup _{0\leq t\leq 1}\vert {W}_{t}\vert \leq 1)$.

• 7. Let ${W}_{t} = ({W}_{t}^{1},{W}_{t}^{2})$ be a standard two-dimensional Brownian motion. Let ${\tau }_{1}$ be the first time when ${W}_{t}^{1} = 1$, that is

  $${\tau }_{1}({\it \omega }) =\inf \{ t \geq 0 : {W}_{t}^{1}({\it \omega }) = 1\}.$$

  Find the distribution of ${W}_{{\tau }_{1}}^{2}$.

• 8. Let W _t be a one-dimensional Brownian motion. Prove that with probability one the set $S =\{ t : {W}_{t} = 0\}$ is unbounded.