Invariant, Super and Quasi-martingale Functions of a Markov Process

Abstract

We identify the linear space spanned by the real-valued excessive functions of a Markov process with the set of those functions which are quasimartingales when we compose them with the process. Applications to semi-Dirichlet forms are given. We provide a unifying result which clarifies the relations between harmonic, co-harmonic, invariant, co-invariant, martingale and co-martingale functions, showing that in the conservative case they are all the same. Finally, using the co-excessive functions, we present a two-step approach to the existence of invariant probability measures.

Dedicated to Michael Röckner on the occasion of his sixtieth birthday.

1 Introduction

Let E be a Lusin topological space endowed with the Borel $\sigma$-algebra $\mathscr {B}$ and $X = (\varOmega , \mathscr {F}, \mathscr {F}_t, X_t, \mathbb {P}^x, \zeta )$ be a right Markov process with state space E, transition function $(P_t)_{t \ge 0}$: $P_t u(x) = \mathbb {E}^x (u(X_t); t < \zeta )$, $t \ge 0$, $x\in E$.

One of the fundamental connections between potential theory and Markov processes is the relation between excessive functions and (right-continuous) supermartingales; see e.g. [1], Chap. VI, Sect. 10, or [2], Proposition 13.7.1 and Theorem 14.7.1. Similar results hold for (sub)martingales, and together stand as a keystone at the foundations of the so called probabilistic potential theory. For completeness, let us give the precise statement; a short proof is included in Appendix.

Proposition 1

The following assertions are equivalent for a non-negative real-valued $\mathscr {B}$-measurable function u and $\beta \ge 0$.

(i) $(e^{-\beta t}u(X_t))_{t\ge 0}$ is a right continuous $\mathscr {F}_t$-supermartingale w.r.t. $\mathbb {P}^x$ for all $x \in E$.

(ii) The function u is $\beta$-excessive.

Our first aim is to show that this connection can be extended to the space of differences of excessive functions on the one hand, and to quasimartingales on the other hand (cf. Theorem 1 from Sect. 2), with concrete applications to semi-Dirichlet forms (see Theorem 2 below).

Remark 1

Recall the following famous characterization from [3]: If u is a real-valued $\mathscr {B}$ -measurable function then u(X)  is an $\mathscr {F}_t$ -semimartingale w.r.t. all $\mathbb {P}^x$, $x\in E$  if and only if u is locally the difference of two finite 1-excessive functions.

The main result from Theorem 1 should be regarded as an extension of Proposition 1 and as a refinement of the just mentioned characterization for semimartingales from Remark 1. However, we stress out that our result is not a consequence of the two previously known results.

In Sect. 3 we focus on a special class of (0-)excessive functions called invariant, which were studied in the literature from several slightly different perspectives. Here, our aim is to provide a unifying result which clarifies the relations between harmonic, co-harmonic, invariant, and co-invariant functions, showing that in the Markovian (conservative) case they are all the same. The measurable structure of invariant functions is also involved. We give the results in terms of $L^p(E,m)$-resolvents of operators, where m is assumed sub-invariant, allowing us to drop the strong continuity assumption. In addition, we show that when the resolvent is associated to a right process, then the martingale functions and the co-martingale ones (i.e., martingale w.r.t. to a dual process) also coincide.

The last topic where the existence of (co)excessive functions plays a fundamental role is the problem of existence of invariant probability measures for a fixed Markovian transition function $(P_t)_{t\ge 0}$ on a general measurable space $(E,\mathscr {B})$. Recall that the classical approach is to consider the dual semigroup of $(P_t)_{t\ge 0}$ acting on the space of all probabilities P(E) on E, and to show that it or its integral means, also known as the Krylov-Bogoliubov measures, are relatively compact w.r.t. some convenient topology (metric) on P(E) (e.g. weak topology, (weighted) total variation norm, Wasserstein metric, etc.). In essence, there are two kind of conditions which stand behind the success of this approach: some (Feller) regularity of the semigroup $(P_t)_{t\ge 0}$ (e.g. it maps bounded and continuous (Lipschitz) functions into bounded and continuous (Lipschitz) functions), and the existence of some compact (or small) sets which are infinitely often visited by the process; see e.g. [4,5,6,7,8,9,10]. Our last aim is to present (in Sect. 4) a result from [11], which offers a new (two-step) approach to the existence of invariant measures (see Theorem 4 below). In very few words, our idea was to first fix a convenient auxilliary measure m (with respect to which each $P_t$ respects classes), and then to look at the dual semigroup of $(P_t)_{t\ge 0}$ acting not on measures as before, but on functions. In this way we can employ some weak $L^1(m)$-compactness results for the dual semigroup in order to produce a non-zero and non-negative co-excessive function.

At this point we would like to mention that most of the announced results, which are going to be presented in the next three sections, are exposed with details in [11,12,13].

The authors had the pleasure to be coauthors of Michael Röckner and part of the results presented in this survey paper were obtained jointly. So, let us conclude this introduction with a

$$ \text {``Happy Birthday, Michael!''} $$

2 Differences of Excessive Functions and Quasimartingales of Markov Processes

Recall that the purpose of this section is to study those real-valued measurable functions u having the property that u(X) is a $\mathbb {P}^x$-quasimartingale for all $x \in E$ (in short, “u(X) is a quasimartingale”, or “u is a quasimartingale function”). At this point we would like to draw the attention to the fact that in the first part of this section we study quasimartingales with respect to $\mathbb {P}^x$ for all $x \in E$, in particular all the inequalities involved are required to hold pointwise for all $x \in E$. Later on we shall consider semigroups or resolvents on $L^p$ or Dirichlet spaces with respect to some duality measure, and in these situations we will explicitly mention if the desired properties are required to hold almost everywhere or outside some exceptional sets.

For the reader’s convenience, let us briefly present some classic facts about quasimartingales in general.

Definition 1

Let $(\varOmega , \mathscr {F}, \mathscr {F}_t, \mathbb {P})$ be a filtered probability space satisfying the usual hypotheses. An $\mathscr {F}_t$-adapted, right-continuous integrable process $(Z_t)_{t \ge 0}$ is called $\mathbb {P}$-quasimartingale if

$$ {Var}^\mathbb {P}(Z):= \mathop {\sup }\limits _{\tau } \mathbb {E} \{ \mathop {\sum }\limits _{i = 1}^{n} |\mathbb {E}[Z_{t_i} - Z_{t_{i-1}}|\mathscr {F}_{t_{i-1}}]| + |Z_{t_n}|\} < \infty , $$

where the supremum is taken over all partitions $\tau : 0 = t_0 \le t_1 \le \ldots \le t_n < \infty$.

Quasimartingales played an important role in the development of the theory of semimartingales and stochastic integration, mainly due to M. Rao’s theorem according to which any quasimartingale has a unique decomposition as a sum of a local martingale and a predictable process with paths of locally integrable variation. Conversely, one can show that any semimartingale with bounded jumps is locally a quasimartingale. However, to the best of our knowledge, their analytic or potential theoretic aspects have never been investigated or, maybe, brought out to light, before.

We return now to the frame given by a Markov process. Further in this section we deal with a right Markov process $X = (\varOmega , \mathscr {F}, \mathscr {F}_t, X_t, \mathbb {P}^x, \zeta )$ with state space E and transition function $(P_t)_{t \ge 0}$. Although we shall not really be concerned with the lifetime formalism, if X has lifetime $\xi$ and cemetery point $\varDelta$, then we make the convention $u(\varDelta ) = 0$ for all functions $u: E \rightarrow [-\infty , + \infty ]$.

Recall that for $\beta \ge 0$, a $\mathscr {B}$-measurable function $f:E \rightarrow [0, \infty ]$ is called $\beta$-supermedian if $P_t^\beta f \le f$ pointwise on E, $t \ge 0$; $(P_t^\beta )_{t \ge 0}$ denotes the $\beta$-level of the semigroup of kernels $(P_t)_{t \ge 0}$, $P_t^\beta := e^{-\beta } P_t$. If f is $\beta$-supermedian and $\lim \limits _{t \rightarrow 0} P_t f = f$ point-wise on E, then it is called $\beta$-excessive. It is well known that a $\mathscr {B}$-measurable function f is $\beta$-excessive if and only if $\alpha U_{\alpha +\beta }f \le f$, $\alpha >0$, and $\lim \limits _{\alpha \rightarrow \infty } \alpha U_{\alpha }f = f$ point-wise on E, where $\mathscr {U} = (U_{\alpha })_{\alpha > 0}$ is the resolvent family of the process X, $U_\alpha := \int _0^\infty e^{-\alpha t} P_t dt$. The convex cone of all $\beta$-excessive functions is denoted by $E(\mathscr {U}_\beta )$; here $\mathscr {U}_\beta$ denotes the $\beta$-level of the resolvent $\mathscr {U}$, $\mathscr {U}_\beta := (U_{\beta +\alpha })_{\alpha > 0}$; the fine topology is the coarsest topology on E such that all $\beta$-excessive functions are continuous, for some $\beta > 0$. If $\beta = 0$ we drop the index $\beta$.

Taking into account the strong connection between excessive functions and supermartingales for Markov processes, the following characterization of M. Rao was our source of inspiration: a real-valued process on a filtered probability space $(\varOmega , \mathscr {F}, \mathscr {F}_t, \mathbb {P})$ satisfying the usual hypotheses is a quasimartingale if and only if it is the difference of two positive right-continuous $\mathscr {F}_t$-supermartingales; see e.g. [14], p. 116.

As a first observation, note that if u(X) is a quasimartingale, then the following two conditions for u are necessary:

(i) $\mathop {\sup }\limits _{t > 0} P_t|u|< \infty$ and (ii) u is finely continuous. Indeed, since for each $x\in E$ we have that $\mathop {\sup }\limits _{t} P_t|u|(x) = \mathop {\sup }\limits _{t}\mathbb {E}^x|u(X_t)| \le {Var}^{\mathbb {P}^x}(u(X)) < \infty$, the first assertion is clear. The second one follows by the result from [15] which is stated in the proof of Proposition 1 in the Appendix at the end of the paper.

For a real-valued function u, a partition $\tau$ of $\mathbb {R}^+$, $\tau : 0 = t_0 \le t_1 \le \ldots \le t_n < \infty$, and $\alpha >0$ we set

$$ V^\alpha (u) := \mathop {\sup }\limits _{\tau }V^\alpha _{\tau }(u), \quad V^\alpha _{\tau }(u) := \mathop {\sum }\limits _{i=1}^{n} P^\alpha _{t_{i-1}} |u - P^\alpha _{t_i - t_{i-1}}u| + P^\alpha _{t_n}|u|, $$

where the supremum is taken over all finite partitions of $\mathbb {R}_+$.

A sequence $(\tau _n)_{n \ge 1}$ of finite partitions of $\mathbb {R}_+$ is called admissible if it is increasing, $\mathop {\bigcup }\limits _{k \ge 1}\tau _k$ is dense in $\mathbb {R}_+$, and if $r \in \mathop {\bigcup }\limits _{k \ge 1}\tau _k$ then $r + \tau _n \subset \mathop {\bigcup }\limits _{k \ge 1}\tau _k$ for all $n \ge 1$.

We can state now our first result, it is a version of Theorem 2.6 from [12].

Theorem 1

Let u be a real-valued $\mathscr {B}$-measurable function and $\beta \ge 0$ such that $P_t|u| < \infty$ for all t. Then the following assertions are equivalent.

(i) $(e^{-\beta t}u(X_t))_{t\ge 0}$ is a $\mathbb {P}^x$-quasimartingale for all $x \in E$.

(ii) u is finely continuous and $\mathop {\sup }\limits _{n}V^\beta _{\tau _n}(u) < \infty$ for one (hence all) admissible sequence of partitions $(\tau _n)_n$.

(iii) u is a difference of two real-valued $\beta$-excessive functions.

Remark 2

The key idea behind the previous result is that by the Markov property is not hard to show that for all $x \in E$ we have ${Var}^{\mathbb {P}^x}((e^{-\alpha t}u(X_t)_{t \ge 0}) = V^\alpha (u)(x)$, meaning that assertion (i) holds if and only if $V^\alpha (u)<\infty$. But $V^\alpha (u)$ is a supremum of measurable functions taken over an uncountable set of partitions, hence it may no longer be measurable, which makes it hard to handle in practice. Concerning this measurability issue, Theorem 1, (ii) states that instead of dealing with $V^\alpha (u)$, we can work with $\mathop {\sup }\limits _{n}V^\alpha _{\tau _n}(u)$ for any admissible sequence of partitions $(\tau _n)_{n\ge 1}$. This subtile aspect was crucial in order to give criteria to check the quasimartingale nature of u(X); see also Proposition 1 in the next subsection.

2.1 Criteria for Quasimartingale Functions

In this subsection, still following [12], we provide general conditions for u under which $(e^{-\beta t}u(X_t))_{t\ge 0}$ is a quasimartingale, which means that, in particular, $(u(X_t))_{t\ge 0}$ is a semimartingale.

Let us consider that m is a $\sigma$-finite sub-invariant measure for $(P_t)_{t \ge 0}$ so that $(P_t)_{t \ge 0}$ extends uniquely to a strongly continuous semigroup of contractions on $L^p(m)$, $1 \le p < \infty$; $\mathscr {U}$ may as well be extended to a strongly continuous resolvent family of contractions on $L^p(m)$, $1 \le p < \infty$. The corresponding generators $(\mathsf{L}_p, D(\mathsf{L}_p) \subset L^p(m))$ are defined by

$$ D(\mathsf{L}_p) = \{ U_{\alpha } f : f \in L^p(m) \}, $$

$$ \mathsf{L}_p(U_{\alpha } f) := \alpha U_{\alpha } f - f \quad \mathrm{for \; all} \; f \in L^p(m), \ 1 \le p < \infty , $$

with the remark that this definition is independent of $\alpha > 0$.

The corresponding notations for the dual structure are $\widehat{P}_t$ and $(\widehat{\mathsf{L}}_p, D(\widehat{\mathsf{L}}_p))$, and note that the adjoint of $\mathsf{L}_p$ is $\widehat{\mathsf{L}}_{p^*}$; $\frac{1}{p} + \frac{1}{p^*}=1$. Throughout, we denote the standard $L^p$-norms by $\Vert \cdot \Vert _p$, $1 \le p \le \infty$.

We present below the $L^p$-version of Theorem 1; cf. Proposition 4.2 from [12].

Proposition 2

The following assertions are equivalent for a $\mathscr {B}$-measurable function $u \in \mathop {\bigcup }\limits _{1 \le p \le \infty } L^p(m)$ and $\beta \ge 0$.

(i) There exists an m-version $\widetilde{u}$ of u such that $(e^{-\beta t}\widetilde{u}(X_t))_{t\ge 0}$ is a $\mathbb {P}^x$-quasimartingale for $x \in E$ m-a.e.

(ii) For an admissible sequence of partitions $(\tau _n)_{n \ge 1}$ of $\mathbb {R}_+$, $\mathop {\sup }\limits _{n} V^\beta _{\tau _n}(u) < \infty$ m-a.e.

(iii) There exist $u_1, u_2 \in E(\mathscr {U}_\beta )$ finite m-a.e. such that $u = u_1 - u_2$ m-a.e.

Remark 3

Under the assumptions of Proposition 2, if u is finely continuous and one of the equivalent assertions is satisfied then all of the statements hold outside an m-polar set, not only m-a.e., since it is known that an m-negligible finely open set is automatically m-polar; if in addition m is a reference measure then the assertions hold everywhere on E exept a polar set.

Now, we focus our attention on a class of $\beta$-quasimartingale functions which arises as a natural extension of $D(\mathsf{L}_p)$. First of all, it is clear that any function $u \in D(\mathsf{L}_p)$, $1 \le p < \infty$, has a representation $u = U_{\beta } f = U_{\beta }(f^+) - U_{\beta }(f^-)$ with $U_{\beta }(f^{\pm }) \in E(\mathscr {U}_{\beta }) \cap L^p(m)$, hence u has a $\beta$-quasimartingale version for all $\beta > 0$; moreover, $\Vert P_t u - u \Vert _p = \left\| \int _0^t P_s\mathsf{L}_p u ds \right\| _p \le t \Vert \mathsf{L}_p u \Vert _p$. The converse is also true, namely if $1< p < \infty$, $u \in L^p(m)$, and $\Vert P_t u - u \Vert _p \le {const} \cdot t$, $t \ge 0$, then $u \in D(\mathsf{L}_p)$. But this is no longer the case if $p = 1$ (because of the lack of reflexivity of $L^1$), i.e. $\Vert P_t u - u \Vert _1 \le {const} \cdot t$ does not imply $u \in D(\mathsf{L}_1)$. However, it turns out that this last condition on $L^1(m)$ is yet enough to ensure that u is a $\beta$-quasimartingale function. In fact, the following general result holds; see [12], Proposition 4.4 and its proof.

Proposition 3

Let $1 \le p < \infty$ and suppose $\mathscr {A} \subset \{ u \in L^{p^{*}}_+(m) : \Vert u \Vert _{p^{*}} \le 1 \}$, $\widehat{P}_s\mathscr {A} \subset \mathscr {A}$ for all $s \ge 0$, and $E = \mathop {\bigcup }\limits _{f \in \mathscr {A}}\mathrm{supp}(f)$ m-a.e. If $u \in L^p(m)$ satisfies

$$ \sup \limits _{f \in \mathscr {A}}\int _E |P_tu - u| f d m \le {const} \cdot ~t~for~all~t \ge 0, $$

then there exists and m-version $\widetilde{u}$ of u such that $(e^{-\beta t}\widetilde{u}(X_t))_{t\ge 0}$ is a $\mathbb {P}^x$-quasimartingale for all $x \in E$ m-a.e. and every $\beta > 0$.

We end this subsection with the following criteria which is not given with respect to a duality measure, but in terms of the associated resolvent $\mathscr {U}$; cf. Proposition 4.1 from [12].

Proposition 4

Let u be a real-valued $\mathscr {B}$-measurable finely continuous function.

(i) Assume there exist a constant $\alpha \ge 0$ and a non-negative $\mathscr {B}$-measurable function c such that

$$ U_{\alpha }(|u| + c)< \infty , \quad \mathop {\lim \sup }\limits _{t \rightarrow \infty } P_t^{\alpha }|u| < \infty , \quad |P_t u - u| \le c t, t \ge 0, $$

and the functions $t \mapsto P_t(|u| + c)(x)$ are Riemann integrable. Then $(e^{-\alpha t}u(X_t))_{t \ge 0}$ is a $\mathbb {P}^x$-quasimartingale for all $x\in E$.

(ii) Assume there exist a constant $\alpha \ge 0$ and a non-negative $\mathscr {B}$-measurable function c such that

$$ |P_t u- u| \le c t, t \ge 0, \quad \mathop {\sup }\limits _{t \in \mathbb {R}_+} P_t^{\alpha }(|u| + c) = : b < \infty . $$

Then $(e^{-\beta t}u(X_t))_{t \ge 0}$ is a $\mathbb {P}^x$-quasimartingale for all $x\in E$ and $\beta > \alpha$.

(iii) Assume there exists $x_0 \in E$ such that for some $\alpha \ge 0$

$$ U_{\alpha }(|u|)(x_0) < \infty , \quad U_{\alpha }(|P_t u - u|)(x_0) \le {const} \cdot t, \; t \ge 0. $$

Then $(e^{-\beta t}u(X_t))_{t \ge 0}$ is a $\mathbb {P}^x$-quasimartingale for $\delta _{x_0}\circ U_\beta$-a.e. $x\in E$ and $\beta > \alpha$; if in addition $\mathscr {U}$ is strong Feller and topologically irreducible then the $\mathbb {P}^x$-quasimartingale property holds for q.e. $x\in E$.

2.2 Applications to Semi-Dirichlet Forms

Assume now that the semigroup $(P_t)_{t\ge 0}$ is associated to a semi-Dirichlet form $(\mathscr {E},\mathscr {F})$ on $L^2(E,m)$, where m is a $\sigma$-finite measure on the Lusin measurable space $(E,{\mathscr {B}})$; as standard references for the theory of (semi-)Dirichlet forms we refer the reader to [16,17,18,19], but also [20], Chap. 7. By Corollary 3.4 from [21] there exists a (larger) Lusin topological space $E_1$ such that $E\subset E_1$, E belongs to ${\mathscr {B}}_1$ (the $\sigma$-algebra of all Borel subsets of $E_1$), ${\mathscr {B}}={\mathscr {B}}_1|_E$, and $(\mathscr {E}, \mathscr {F})$ regarded as a semi-Dirichlet form on $L^2(E_1 , \overline{m})$ is quasi-regular, where $\overline{m}$ is the trivial extension of m to $(E_1, {\mathscr {B}}_1)$. Consequently, we may consider a right Markov process X with state space $E_1$ which is associated with the semi-Dirichlet form $(\mathscr {E},\mathscr {F})$.

If $u \in \mathscr {F}$ then $\widetilde{u}$ denotes a quasi continuous version of u as a function on $E_1$ which always exists and it is uniquely determined quasi everywhere. Following [22], for a closed set F we define $\mathscr {F}_{b, F} :=\{v\in \mathscr {F} : v \text{ is } \text{ bounded } \text{ and } v=0 \; m\text{-a.e. } \text{ on } E\setminus F\}$.

The next result is a version of Theorem 5.5 from [12], dropping the a priori assumption that the semi-Dirichlet form is quasi-regular.

Theorem 2

Let $u \in \mathscr {F}$ and assume there exist a nest $(F_n)_{n\ge 1}$ and constants $(c_n)_{n\ge 1}$ such that

$$ \mathscr {E}(u,v) \le c_n \Vert v\Vert _\infty \;\; \text{ for } \text{ all } \; v\in \mathscr {F}_{b, F_n}. $$

Then $\widetilde{u}(X)$ is a $\mathbb {P}^x$-semimartingale for $x\in E_1$ quasi everywhere.

Remark 4

The previous result has quite a history behind and we take the opportunity to recall some previous achievements on the subject. First of all, without going into details, note that if E is a bounded domain in $\mathbb {R}^{d}$ (or more generally in an abstract Wiener space) and the condition from Theorem 2 holds for u replaced by the canonical projections, then the conclusion is that the underlying Markov process is a semimartingale. In particular, the semimartingale nature of reflected diffusions on general bounded domains can be studied. This problem dates back to the work of [23], where the authors showed that the reflected Brownian motion on a Lipschitz domain in $\mathbb {R}^d$ is a semimartingale. Later on, this result has been extended to more general domains and diffusions; see [24,25,26,27,28]. A clarifying result has been obtained in [25], showing that the stationary reflecting Brownian motion on a bounded Euclidian domain is a quasimartingale on each compact time interval if and only if the domain is a strong Caccioppoli set. At this point it is worth to emphasize that in the previous sections we studied quasimartingales on the hole positive real semi-axis, not on finite intervals. This slight difference is a crucial one which makes our approach possible and completely different. A complete study of these problems (including Theorem 2 but only in the symmetric case) have been done in a series of papers by M. Fukushima and co-authors (we mention just [22, 29, 30]), with deep applications to BV functions in both finite and infinite dimensions.

All these previous results have been obtained using the same common tools: symmetric Dirichlet forms and Fukushima decomposition. Further applications to the reflection problem in infinite dimensions have been studied in [31, 32], where non-symmetric situations were also considered. In the case of semi-Dirichlet forms, a Fukushima decomposition is not yet known to hold, unless some additional hypotheses are assumed (see e.g. [19]). Here is where our study developed in the previous sections played its role, allowing us to completely avoid Fukushima decomposition or the existence of the dual process. On brief, the idea of proving Theorem 2 is to show that locally, the conditions from Proposition 3 are satisfied, so that u(X) is (pre)locally a semimartingale, and hence a global semimartingale.

Assume that $(\mathscr {E}, \mathscr {F})$ is quasi-regular and that it is local, i.e., $\mathscr {E}(u,v)=0$ for all $u,v \in \mathscr {F}$ with disjoint compact supports. It is well known that the local property is equivalent with the fact that the associated process is a diffusion; see e.g. [18], Chap. V, Theorem 1.5. As in [29], the local property of $\mathscr {E}$ allows us to extend Theorem 2 to the case when u is only locally in the domain of the form, or to even more general situations, as stated in the next result; for details see Sect. 5.1 from [12].

Corollary 1

Assume that $(\mathscr {E}, \mathscr {F})$ is local. Let u be a real-valued $\mathscr {B}$-measurable finely continuous function and let $(v_k)_k \subset \mathscr {F}$ such that $v_k \mathop {\longrightarrow }\limits _{k \rightarrow \infty } u$ point-wise outside an m-polar set and boundedly on each element of a nest $(F_n)_{n \ge 1}$. Further, suppose that there exist constants $c_n$ such that

$$ |\mathscr {E}(v_k, v)| \le c_n \Vert v\Vert _{\infty } \;\; for \; all \; v \in \mathscr {F}_{b, F_n}. $$

Then u(X) is a $\mathbb {P}^x$-semimartingale for $x \in E$ quasi everywhere.

3 Excessive and Invariant Functions on $L^p$-spaces

Throughout this section $\mathscr {U}=(U_\alpha )_{\alpha > 0}$ is a sub-Markovian resolvent of kernels on E and m is a $\sigma$-finite sub-invariant measure, i.e. $m( \alpha U_\alpha f ) \le m(f)$ for all $\alpha >0$ and non-negative $\mathscr {B}$-measurable functions f; then there exists a second sub-Markovian resolvent of kernels on E denoted by $\mathscr {\widehat{U}}=(\widehat{U}_\alpha )_{\alpha >0}$ which is in weak duality with $\mathscr {U}$ w.r.t. M in the sense that $\int _E fU_\alpha g dm = \int _E g\widehat{U}_\alpha f dm$ for all positive $\mathscr {B}$-measurable functions f, g and $\alpha >0$. Moreover, both resolvents can be extended to contractions on any $L^p(E,m)$-space for all $1\le p\le \infty$, and if they are strongly continuous then we keep the same notations for their generators as in Sect. 2.1. In this part, our attention focuses on a special class of differences of excessive functions (which are in fact harmonic when the resolvent is Markovian). Extending [33], they are defined as follows.

Definition 2

A real-valued $\mathscr {B}$-measurable function $v \in \bigcup _{1 \le p \le \infty } L^p(E, m)$ is called $\mathscr {U}$-invariant provided that $U_{\alpha }(vf) \,{=}\, v U_{\alpha } f$ m-a.e. for all bounded and $\mathscr {B}$-measurable functions f and $\alpha > 0$.

A set $A \in \mathscr {B}$ is called $\mathscr {U}$-invariant if $1_A$ is $\mathscr {U}$-invariant; the collection of all $\mathscr {U}$-invariant sets is a $\sigma$-algebra.

Remark 5

If $v \ge 0$ is $\mathscr {U}$-invariant, then by [13], Proposition 2.4 there exists $u \in E(\mathscr {U})$ such that $u = v$ m-a.e. If $\alpha U_{\alpha } 1 = 1$ m-a.e. then for every invariant function v we have that $\alpha U_{\alpha } v = v$ m-a.e, which is equivalent (if $\mathscr {U}$ is strongly continuous) with v being $\mathsf{L}_p$-harmonic, i.e. $v \in D(\mathsf{L}_p)$ and $\mathsf{L}_pv=0$.

The following result is a straightforward consequence of the duality between $\mathscr {U}$ and $\widehat{\mathscr {U}}$; for its proof see Proposition 2.24 and Proposition 2.25 from [13].

Proposition 5

The following assertions hold.

(i) A function u is $\mathscr {U}$-invariant if and only if it is $\widehat{\mathscr {U}}$-invariant.

(ii) The set of all $\mathscr {U}$-invariant functions from $L^p(E, m)$ is a vector lattice with respect to the point-wise order relation.

Let

$$ \mathscr {I}_p : = \{ u \in L^p(E, m) : \alpha U_\alpha u = u \; m\text{-a.e. }, \; \alpha > 0 \}. $$

The main result here is the next one, and it unifies and extends different more or less known characterizations of invariant functions; cf. Theorem 2.27 and Proposition 2.29 from [13].

Theorem 3

Let $u \in L^p(E, m)$, $1 \le p < \infty$, and consider the following conditions.

(i) $\alpha U_{\alpha }u = u$ m-a.e. for one (and therefore for all) $\alpha > 0$.

(ii) $\alpha \widehat{U}_{\alpha } u = u$ m-a.e., $\alpha > 0$.

(iii) The function u is $\mathscr {U}$-invariant.

(iv) $U_\alpha u = u U_\alpha 1$ and $\widehat{U}_{\alpha } u = u \widehat{U}_{\alpha } 1$ m-a.e. for one (and therefore for all) $\alpha > 0$.

(v) The function u is measurable w.r.t. the $\sigma$-algebra of all $\mathscr {U}$-invariant sets.

Then $\mathscr {I}_p$ is a vector lattice w.r.t. the pointwise order relation and (i) $\Leftrightarrow$ (ii) $\Rightarrow$ (iii) $\Leftrightarrow$ (iv) $\Leftrightarrow$ v).

If $\alpha U_\alpha 1 = 1$ or $\alpha \widehat{U}_{\alpha } 1 = 1$ m-a.e. then assertions (i)–(v) are equivalent.

If $p= \infty$ and $\mathscr {U}$ is m-recurrent (i.e. there exists $0\le f \in L^1(E,m)$ s.t. $Uf=\infty$ m-a.e.) then the assertions (i)–(v) are equivalent.

Remark 6

Similar characterizations for invariance as in Theorem 3, but in the recurrent case and for functions which are bounded or integrable with bounded negative parts were already investigated in [34]. Of special interest is the situation when the only invariant functions are the constant ones (irreducibility) because it entails ergodic properties for the semigroup resp. resolvent; see e.g. [13, 33, 35].

3.1 Martingale Functions with Respect to the Dual Markov Process

Our aim in this subsection is to identify the $\mathscr {U}$-invariant functions with martingale functions and co-martingale ones (i.e., martingales w.r.t some dual process); cf. Corollary 3 below. The convenient frame is that from [36] and we present it here briefly.

Assume that ${\mathscr {U}}=(U_{\alpha })_{\alpha >0}$ is the resolvent of a right process X with state space E and let ${\mathscr {T}}_0$ be the Lusin topology of E having ${\mathscr {B}}$ as Borel $\sigma$-algebra, and let m be a fixed ${\mathscr {U}}$-excessive measure. Then by Corollary 2.4 from [36], and using also the result from [21], the following assertions hold: There exist a larger Lusin measurable space $(\overline{E}, \overline{{\mathscr {B}}})$, with $E\subset \overline{E}$, $E\in \overline{{\mathscr {B}}}$, ${\mathscr {B}}=\overline{{\mathscr {B}}}|_{E}$, and two processes $\overline{X}$ and $\widehat{{X}}$ with common state space $\overline{E}$, such that $\overline{X}$ is a right process on $\overline{E}$ endowed with a convenient Lusin topology having $\overline{{\mathscr {B}}}$ as Borel $\sigma$-algebra (resp. $\widehat{{X}}$ is a right process w.r.t. to a second Lusin topology on $\overline{E}$, also generating $\overline{{\mathscr {B}}}$), the restriction of $\overline{X}$ to E is precisely X, and the resolvents of $\overline{X}$ and $\widehat{{X}}$ are in duality with respect to $\overline{m}$, where $\overline{m}$ is the trivial extension of m to $(E_1,{\mathscr {B}}_1): \; \overline{m}(A):=m(A\cap E), \; A \in \mathscr {B}_1$. In addition, the $\alpha$-excessive functions, $\alpha >0$, with respect to $\widehat{X}$ on $\overline{E}$ are precisely the unique extensions by continuity in the fine topology generated by $\widehat{X}$ of the $\widehat{{\mathscr {U}}}_{\alpha }$-excessive functions. In particular, the set E is dense in $\overline{E}$ in the fine topology of $\widehat{X}$.

Note that the strongly continuous resolvent of sub-Markovian contractions induced on $L^p(m)$, $1\le p<\infty$, by the process $\overline{X}$ (resp. $\widehat{{X}}$) coincides with ${\mathscr {U}}$ (resp. $\widehat{\mathscr {U}}$).

Corollary 2

Let u be function from $L^p(E, m)$, $1 \le p < \infty$. Then the following assertions are equivalent.

(i) The process $(u(X_t))_{t\ge 0}$ is a martingale w.r.t. $\mathbb {P}^x$ for m-a.e. $x\in E$.

(ii) The process $(u(\widehat{X}_t))_{t\ge 0}$ is a martingale w.r.t. $\widehat{\mathbb {P}}^x$ for m-a.e. $x\in E$.

(iii) The function u is $\mathsf{L}_p$-harmonic, i.e. $u \in D(\mathsf{L}_p)$ and $\mathsf{L}_p u=0$.

(iv) The function u is ${\widehat{\mathsf{L}}}_p$-harmonic, i.e. $u \in D({\widehat{\mathsf{L}}}_p)$ and ${\widehat{\mathsf{L}}}_p u=0$.

Proof

The equivalence $(iii) \Longleftrightarrow (iv)$ follows by Theorem 3, $i) \Longleftrightarrow ii)$, while the equivalence $(i) \Longleftrightarrow (iii)$ is a consequence of Proposition 2.    $\square$

We make the transition to the next (also the last) section of this paper with an application of Theorem 3 to the existence of invariant probability measures for Markov processes. More precisely, assume that $\mathscr {U}$ is the resolvent of a right Markov process with transition function $(P_t)_{t \ge 0}$. As before, m is a $\sigma$-finite sub-invariant measure for $\mathscr {U}$ (and hence for $(P_t)_{t\ge 0}$), while $\mathsf{L}_1$ and $\widehat{\mathsf{L}}_1$ stand for the generator, resp. the co-generator on $L^1(E,m)$.

Corollary 3

The following assertions are equivalent.

(i) There exists an invariant probability measure for $(P_t)_{t\ge 0}$ which is absolutely continuous w.r.t. m.

(ii) There exists a non-zero element $\rho \in D(\mathsf{L}_1)$ such that $\mathsf{L}_1 \rho = 0$.

Proof

It is well known that a probability measure $\rho \cdot m$ is invariant w.r.t. $(P_t)_{t \ge 0}$ is equivalent with the fact that $\rho \in D(\widehat{\mathsf{L}}_1)$ and $\widehat{\mathsf{L}}_1 \rho =0$ (see also Lemma 1, (ii) from below). Now, the result follows by Theorem 3.

Remark 7

Regarding the previous result, we point out that if $m(E) < \infty$ and $(P_t)_{t \ge 0}$ is conservative (i.e. $P_t1=1$ m-a.e. for all $t>0$) then it is clear that m itself is invariant, so that Corollary 3 has got a point only when $m(E)=\infty$. Also, we emphasize that the sub-invariance property of m is an essential assumption. We present a general result on the existence of invariant probability measures in the next section, where we drop the sub-invariance hypothesis.

4 $L^1$-harmonic Functions and Invariant Probability Measures

Throughout this subsection $(P_t)_{t \ge 0}$ is a measurable Markovian transition function on a measurable space $(E, \mathscr {B})$ and m is an auxiliary measure for $(P_t)_{t\ge 0}$, i.e. a finite positive measure such that $m(f) = 0 \Rightarrow m(P_tf) = 0$ for all $t > 0$ and all positive $\mathscr {B}$-measurable functions f. As we previously announced, our final interest concerns the existence of an invariant probability measure for $(P_t)_{t \ge 0}$ which is absolutely continuous with respect to m.

Remark 8

We emphasize once again that in contrast with the previous section, m is not assumed sub-invariant, since otherwise it would be automatically invariant. Also, any invariant measure is clearly auxiliary, but the converse is far from being true. As a matter of fact, the condition on m of being auxiliary is a minimal one: for every finite positive measure $\mu$ and $\alpha >0$ one has that $\mu \circ U_\alpha$ is auxiliary; see e.g. [11, 37].

For the first assertion of the next result we refer to [11], Lemma 2.1, while the second one is a simple consequence of the fact that $P_t1=1$.

Lemma 1

(i) The adjoint semigroup $(P_t^{*})_{t \ge 0}$ on $(L^{\infty }(m))^{*}$ maps $L^1(m)$ into itself, and restricted to $L^1(m)$ it becomes a semigroup of positivity preserving operators.

(ii) A probability measure $\rho \cdot m$ is invariant with respect to $(P_t)_{t \ge 0}$ if and only if $\rho$ is m-co-excessive, i.e. $P_t^*\rho \le \rho$ m-a.e. for all $t \ge 0$.

Inspired by well known ergodic properties for semigroups and resolvents (see for example [13]), our idea in order to produce co-excessive functions is to apply (not for $(P_t)_{t \ge 0}$ but for its adjoint semigroup) a compactness result in $L^1(m)$ due to [38], saying that an $L^1(m)$-bounded sequence of elements possesses a subsequence whose Cesaro means are almost surely convergent to a limit from $L^1(m)$.

Definition 3

The auxilliary measure m is called almost invariant for $(P_t)_{t \ge 0}$ if there exist $\delta \in [0, 1)$ and a set function $\phi : \mathscr {B} \rightarrow \mathbb {R}_+$ which is absolutely continuous with respect to m (i.e. $\mathop {\lim }\limits _{m(A) \rightarrow 0} \phi (A) = 0$) such that

$$m (P_t1_A) \le \delta m(E) + \phi (A) \quad \text{ for } \text{ all } \; t > 0. $$

Clearly, any positive finite invariant measure is almost invariant. Here is our last main result, a variant of Theorem 2.4 from [11].

Theorem 4

The following assertions are equivalent.

(i) There exists a nonzero positive finite invariant measure for $(P_t)_{t \ge 0}$ which is absolutely continuous with respect to m.

(ii) m is almost invariant.

Appendix

Proof of Proposition 1

(i) $\Rightarrow$ (ii). If $(e^{-\beta t}u(X_t))_{t\ge 0}$ is a right-continuous supermartingale then by taking expectations we get that $e^{-\beta t} \mathbb {E}^x u(X_t) \le \mathbb {E}^xu(X_0)$, hence u is $\beta$-supermedian. Now, by [20], Corollary 1.3.4, showing that $u \in E(\mathscr {U}_\beta )$ reduces to prove that u is finely continuous, which in turns follows by the well known characterization according to which u is finely continuous if and only if u(X) has right continuous trajectories $\mathbb {P}^x$-a.s. for all $x \in E$; see Theorem 4.8 in [15], Chap. II.

(ii) $\Rightarrow$ (i). Since u is $\beta$-supermedian, by the Markov property we have for all $0\le s \le t$

$$ \mathbb {E}^x[e^{-\beta (t+s)}u(X_{t+s}) | \mathscr {F}_s]=e^{-\beta (t+s)}\mathbb {E}^{X_s}u(X_t)=e^{-\beta (t+s)} P_tu(X_s) \le e^{-\beta s}u(X_s), $$

hence $(e^{-\beta t}u(X_t))_{t\ge 0}$ is an $\mathscr {F}_t$-supermartingale. The right-continuity of the trajectories follows by the fine continuity of u via the previously mentioned characterization.

   $\square$