S-strongly Time-Consistency in Differential Games

Abstract

In the paper the definition of S-strongly time-consistency in differential games is introduced. The approach of the construction of S-strong time-consistent subcore of the classical core on the base of characteristic function obtained by normalization of classical characteristic function is formulated. Its relation to another characteristic function obtained by an integral extension of the original characteristic function is studied.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

12.1 Introduction

Dynamic games theory has many applications in different areas (see [1, 2, 11, 13]). Particularly important are cooperative differential games that are widely used for modeling the situations of joint decision taking by many agents. When considering such problems, the realizability of cooperative solution in time turns out to be one of the central issues.

As it was mentioned earlier, [9, 13], an attempt to transfer the optimality principles (cooperative solution) from the static cooperative game theory to n-persons differential games leads to dynamically unstable (time inconsistent) optimality principles that renders meaningless their use in differential games. Hence, the notion of time consistent cooperative solution and an approach to determining such cooperative solution was proposed in [9].

A strong time-consistent optimality principle has even more attractive property. Namely, strong time consistency of the core considered as a cooperative solution implies that a single deviation from the chosen imputation taken from the core in favor of another imputation from the core does not lead to non-realizability of the cooperative agreement (the core) defined for the whole duration of the game, [7]. This implies that the overall payment for players will also be contained in the core.

In this paper, a cooperative differential game with the set of players N is studied in general setting on the finite time horizon. The work is of fundamental character, but may potentially have a big practical impact because it proposes a constructive approach to the definition of a new cooperative solution which satisfies the condition of strong time-consistency.

In the paper, we study different approaches to constructing a strongly time-consistent cooperative solution, which are based on the use of additional procedures for the imputation distribution on the time interval [t ₀, T] (IDP) for classical cooperative solution, i.e., the core, and on the transformations of the classical characteristic function V (S, ⋅), S ⊆ N. Furthermore, we present results illustrating the relationship between the introduced concepts.

In [7, 8], it was shown that it is possible to define a new type of characteristic function $\bar V(S, \cdot )$ on the base of integral transformation of the classical characteristic function V (S, ⋅) such that the resulting optimality principles are strongly time-consistent.

In [10], another approach to the construction of the characteristic function $\hat V(S, \cdot )$ on the base of normalizing transformation of V (S, ⋅) had been suggested and it was shown that the core constructed on the base of the new $\hat V(S, \cdot )$ belongs to the classical core.

In this contribution we track the connection between the optimality principles constructed on the basis of classical characteristic function and the constructions resulted from the new types of characteristic function. We study the property of strong-time consistency for all constructed optimality principles and suggest a modification of the notion of strong time-consistency as described below.

The notion of S-strong time-consistency can be considered as a weakening of the strong time-consistency and means the following: after a single deviation from the chosen imputation from the optimality principle $\hat M(x_0, t_0)$ in favor of another imputation from the same optimality principle $\hat M(x^*(t), t)$ the resulting imputation will belong to a larger set $M(x_0, t_0)\supset \hat M(x_0, t_0)$ even if the resulting solution does not belong to the initial set $\hat M(x_0, t_0)$. Note that S-strong time-consistency of the cooperative solution is considered with respect to another (bigger) set, hence the prefix S-.

The construction of a S-strongly dynamically stable subcore on the base of all described approaches is presented.

12.2 List of Key Notations

x :

trajectory of the system

u :

control vector u = {u ₁, …, u _n}

K _i (x, t, u):

payoff of the player i in a subgame starting at t from x

N :

set of players (the grand-coalition)

S :

subset of players (a coalition), S ⊆ N

V (S, x, t):

basic characteristic function (c.f.)

$\bar V(S,x,t)$ :

an integral extension of the c.f. V 

$\hat V(S,x,t)$ :

a normalized c.f. V 

L(x, t):

set of imputations associated with V 

$\bar L(x,t)$ :

set of imputations associated with $\bar V$

C(x, t):

core associated with V 

$\bar C(x,t)$ :

core associated with $\bar V$

$\hat C(x,t)$ :

core associated with $\hat V$

12.3 Basic Game

Consider the differential game Γ(x ₀, t ₀) starting from the initial position x ₀ and evolving on time interval [t ₀, T]. The equations of the system’s dynamics have the form

$$\displaystyle \begin{aligned} \begin{array}{l} \dot{x}=f(x,u_1,\ldots, u_n), \ x(t_0)=x_0,\\ \\ u_i\in U_i \subset \ \mbox{Comp} R^m, \ x\in R^l, \ i=1,\ldots,n. \end{array}\end{aligned} $$

(12.1)

The players’ payoffs are

$$\displaystyle \begin{aligned} K_i(x, t_0;u_1,\ldots , u_n)= \int_{t_0}^{ T} h_i(x(t))dt, \ i=1,\ldots, n, \ h_i (\cdot) \geq 0,\end{aligned} $$

where x(t) is the solution of system (12.1) with controls u ₁, …, u _n. The non-negativeness of the utility function h _i(⋅) is an important assumption of the model.

It is furthermore assumed that the system (12.1) satisfies all the conditions guaranteeing the existence and uniqueness of solution x(t) on the time interval [t ₀, T] for all admissible measurable open loop controls u ₁(t), …, u _n(t), t ∈ [t ₀, T]. Let there exist a set of controls

$$\displaystyle \begin{aligned} u^*(t)=\{u^*_1(t), \ldots, u^*_n(t)\}, \ t\in [t_0, T]\end{aligned} $$

such that

$$\displaystyle \begin{aligned} \max_{u_1,\ldots,u_n} \sum_{i=1}^{n} K_i(x_0,t_0;u_1(t),\ldots, u_n(t)) =\sum_{i=1}^{n} \int_{t_0}^{ T} h_i(x^*(t))dt =V(N;x_0,t_0).\end{aligned} $$

(12.2)

The solution x ^∗(t) of the system (12.1) corresponding to u ^∗(t), is called the cooperative trajectory.

In cooperative game theory, [6], it is assumed that the players initially agree upon the use of the controls $u^*(t)=\{u^*_1(t), \ldots , u^*_n(t)\}$ and hence, in the cooperative formulation the differential game Γ(x ₀, t ₀) always develops along the cooperative trajectory x ^∗(t).

Let N = {1, …, i, …, n} be the set of all players. Let S ⊆ N and denote by V (S;x ₀, t ₀) the characteristic function of the game Γ(x ₀, t ₀), [6]. Note that V (N;x ₀, t ₀) is calculated by the formula (12.2). Let V (S;x ^∗(t), t), S ⊆ N, t ∈ [t ₀, T] be a (superadditive) characteristic function of the subgame Γ(x ₀, t ₀) constructed by any relevant method [5].

So, we state the following properties for characteristic function:

$$\displaystyle \begin{gathered} {} V(\emptyset ;x_0,t_0)=0; \notag\\ V(N;x_0,t_0)=\sum_{i=1}^{n}\int_{t_0}^{ T} h_i(x^*(\tau))d\tau; \notag\\ V(S_1 \cup S_2;x_0,t_0) \geq V(S_1;x_0,t_0)+V(S_2;x_0,t_0). \end{gathered} $$

(12.3)

For the sake of definiteness we can assume that the characteristic function V (S;x ₀, t ₀) is constructed as the value of a zero-sum differential game based on the game Γ(x ₀, t ₀) and played between the coalition S (the first maximizing player) and the coalition N ∖ S (the second minimizing player), and in each situation the payoff of coalition S is assumed to be the sum of players’ payoffs from this coalition.

Consider the family of subgames Γ(x ^∗(t), t) of game Γ(x ₀, t ₀) along the cooperative trajectory x ^∗(t), i.e. a family of cooperative differential games from the initial state x ^∗(t) defined on the interval [t, T], t ∈ [t ₀, T] and the payoff functions

$$\displaystyle \begin{aligned} K_i(x^*(t),t;u_1, \ldots, u_n)= \int_{t}^{ T} h_i(x(\tau))d\tau, \ i=1,\ldots,n, \end{aligned}$$

where x(τ) is a solution of (12.1) from initial position x ^∗(t) with controls u ₁, …, u _n.

Let V (S;x ^∗(t), t), S ⊆ N, t ∈ [t ₀, T] be the (superadditive) characteristic function of subgame Γ(x ^∗(t), t), s.t. the properties (12.3) hold. For V (N;x ^∗(t), t), the Bellman optimality condition along x ^∗(t) holds, i.e.

$$\displaystyle \begin{aligned}V(N;x_0,t_0)=\int_{t_0}^{ t} \sum_{i=1}^{n}h_i(x^*(\tau))d\tau+ V(N; x^*(t),t).\end{aligned} $$

12.4 Construction of a Core with a New Characteristic Function

Define the new characteristic function $\bar {V}(S; x_0,t_0)$, S ⊆ N, similar to [7, 8], by the formula

$$\displaystyle \begin{aligned} \bar{V}(S;x_0,t_0)=\int_{t_0}^{ T} V(S; x^*(\tau),\tau) \dfrac {\sum\nolimits_{i=1}^{n}h_i(x^*(\tau))}{V(N;x^*(\tau),\tau)}d\tau. \end{aligned} $$

(12.4)

Similarly, we define for t ∈ [t ₀, T]

$$\displaystyle \begin{aligned} \bar{V}(S;x^*(t),t)=\int_{t}^{ T} V(S; x^*(\tau),\tau) \dfrac {\sum\nolimits_{i=1}^{n}h_i(x^*(\tau)) }{V(N;x^*(\tau),\tau)}d\tau. \end{aligned} $$

(12.5)

One can readily see that the function $\bar {V}(S; x_0,t_0)$ has the all properties (12.3) of the characteristic function of the game Γ(x ₀, t ₀). Indeed,

$$\displaystyle \begin{gathered} \bar{V}(\emptyset;x_0,t_0)=0,\\ \bar{V}(N;x_0,t_0)=V(N;x_0,t_0)=\sum_{i=1}^{n}\int_{t_0}^{ T} h_i(x^*(\tau))d\tau,\\ \bar{V}(S_1 \cup S_2;x_0,t_0)\geq \bar{V}(S_1;x_0,t_0)+\bar{V}(S_2;x_0,t_0). \end{gathered} $$

for S ₁, S ₂ ⊂ N, S ₁ ∩ S ₂ = ∅ (here we use the superadditivity of function V (S;x ₀, t ₀)). The similar statement is true also for function $\bar {V} (S; x^*(t), t)$ which is defined as the characteristic function of Γ(x ^∗(t), t).

Let L(x ₀, t ₀) be the set of imputations in Γ(x ₀, t ₀) determined by characteristic function of V (S;x ₀, t ₀), S ⊆ N, i.e.

$$\displaystyle \begin{aligned} L(x_0,t_0)= \left\{ \xi=\{\xi_i\}: \ \sum_{i=1}^{n} \xi_i= V(N;x_0,t_0), \ \xi_i \geq V(\{i\};x_0, t_0) \right\}. \end{aligned} $$

(12.6)

Similarly, we define the set of imputations L(x ^∗(t), t), t ∈ [t ₀, T] in the subgame Γ(x ^∗(t), t):

$$\displaystyle \begin{aligned} \begin{array}{c} L(x^*(t),t)= \left\{ \xi^t=\{\xi_i^t\} : \sum_{i=1}^{n} \xi_i^t= V(N;x^*(t),t), \right. \\ \\ \left. \xi_i^t \geq V(\{i\};x^*(t), t), i\in N \right\}. \end{array} \end{aligned} $$

(12.7)

We denote the set of imputations defined by characteristic functions $\bar {V}(S;x_0,t_0)$ and $\bar {V}(S;x^*(t),t)$ by $\bar {L}(x_0,t_0)$ and $\bar {L}(x^*(t),t)$, respectively. These imputations are defined in the same way as (12.6), (12.7).

Let ξ(t) = {ξ _i(t)}∈ L(x ^∗(t), t) be the integrable selector [9], t ∈ [t ₀, T], define

$$\displaystyle \begin{aligned} \bar{\xi_i} = \int_{t_0}^{ T} \xi_i(\tau) \frac{\sum\nolimits_{i=1}^{n}h_i(x^*(\tau)) }{V(N;x^*(\tau),\tau)}d\tau, \end{aligned} $$

(12.8)

$$\displaystyle \begin{aligned} \bar{\xi_i^t} = \int_{t}^{ T} \xi_i(\tau) \frac{\sum\nolimits_{i=1}^{n}h_i(x^*(\tau))}{V(N;x^*(\tau),\tau)}d\tau, \end{aligned} $$

(12.9)

where t ∈ [t, T] and i = 1, …, n.

One can see that

$$\displaystyle \begin{aligned} \sum_{i=1}^{n}\bar{\xi}_i=V(N;x_0,t_0),\end{aligned} $$

$$\displaystyle \begin{aligned} \sum_{i=1}^{n}\bar{\xi_i^t}=V(N;x^*(t),t).\end{aligned} $$

Moreover, we have

$$\displaystyle \begin{aligned} \bar{\xi}_i \geq \int_{t_0}^{ T} V(\{i\};x^*(\tau),\tau) \frac{ \displaystyle\sum_{i=1}^{n}h_i(x^*(\tau)) }{V(N;x^*(\tau),\tau)}d\tau = \bar{V}(\{i\};x_0,t_0)\end{aligned} $$

and similarly

$$\displaystyle \begin{aligned} \bar{\xi_i^t} \geq \bar{V}(\{i\};x^*(t),t), \ i=1,\ldots, n, \ t\in [t_0, T],\end{aligned} $$

i.e. the vectors $\bar {\xi }= \{\bar {\xi _i}\}$ and $\bar {\xi ^t}= \{\bar {\xi _i^t}\}$ are imputations in the games Γ(x ₀, t ₀) and Γ(x ^∗(t), t), t ∈ [t ₀, T], respectively, if the functions $\bar {V}(S;x_0,t_0)$ and $\bar {V}(S;x^*(t),t)$ are used as characteristic functions.

We have that $\bar {\xi }\in \bar {L}(x_0,t_0)$ and $\bar {\xi ^t} \in \bar {L}(x^*(t),t)$.

Denote by C(x ₀, t ₀) ⊂ L(x ₀, t ₀), C(x ^∗(t), t) ⊂ L(x ^∗(t), t), t ∈ [t ₀, T], the core of the game Γ(x ₀, t ₀) and of the subgame Γ(x ^∗(t), t), respectively (it is assumed that the sets C(x ^∗(t), t), t ∈ [t ₀, T], are not empty along the cooperative trajectory x ^∗(t)). For an application of the core in differential games see also [3].

So, we have

$$\displaystyle \begin{aligned} {C}(x_0, t_0)=\{\xi=\{\xi_i\}, s.t. \sum_{i\in S}{\xi}_i \geq {V}(S;x_0,t_0), \,\, \displaystyle\sum_{i\in N}{\xi}_i=V(N;x_0,t_0), \,\, \forall S \subset N \}.\end{aligned} $$

Let further $\tilde {C}(x_0,t_0)$ and $\tilde {C}(x^*(t),t)$, t ∈ [t ₀, T] be the core of the game Γ(x ₀, t ₀) and of Γ(x ^∗(t), t), constructed using the characteristic function $\bar {V}(S;x,t_0)$, defined by the formulas (12.4) and (12.5). Thus, $\tilde {C}(x_0,t_0)$ is the set of imputations $\{\tilde \xi _i \}$ such that

$$\displaystyle \begin{gathered} {}\displaystyle\sum_{i\in S}\tilde{\xi}_i \geq\bar{V}(S;x_0,t_0), \,\,\,\forall S \subset N; \qquad \displaystyle\sum_{i\in N}\tilde{\xi}_i=\bar{V}(N;x_0,t_0)=V(N;x_0,t_0)\vspace{-2pt} \end{gathered} $$

(12.10)

and $\tilde {C}(x^*(t),t)$ is the set of imputations $\{\tilde {\xi }_i^t\}$, s.t.

$$\displaystyle \begin{aligned} \displaystyle\sum_{i\in S}\tilde{\xi}_i^t \geq\bar{V}(S;x^*(t),t),\,\,\,\forall S \subset N; \qquad \displaystyle\sum_{i\in N}\tilde{\xi}_i^t=\bar{V}(N;x^*(t),t)=V(N;x^*(t),t). \end{aligned}$$

Let in the formulas (12.8) and (12.9) ξ(t) be an integrable selector, ξ(t) ∈ C(x ^∗(t), t ₀), t ∈ [t ₀, T]. Define the set

$$\displaystyle \begin{aligned} \bar C(x_0, t_0)=\left\{ \bar \xi: \bar{\xi} = \int_{t_0}^{ T} \xi(\tau) \dfrac{\sum\nolimits_{i=1}^{n}h_i(x^*(\tau)) }{V(N;x^*(\tau),\tau)}d\tau\},\, \forall \xi(\tau) \in C(x^*(\tau),\tau)\right\}. \end{aligned}$$

Similarly, we define

$$\displaystyle \begin{aligned} \bar C(x^*(t), t)=\left\{ \bar \xi^t: \bar{\xi^t} = \int_{t}^{ T} \xi(\tau) \dfrac{\sum\nolimits_{i=1}^{n}h_i(x^*(\tau)) }{V(N;x^*(\tau),\tau)}d\tau\},\, \forall \xi(\tau) \in C(x^*(\tau),\tau)\right\}. \end{aligned}$$

We have the following lemma.

Lemma 12.1

$$\displaystyle \begin{aligned} \bar C(x_0, t_0) \subseteq \tilde C(x_0, t_0), \qquad \bar C(x^*(t), t) \subseteq \tilde C(x^*(t), t), \qquad \forall t \in [t_0, T]. \end{aligned}$$

Proof

To prove this lemma, we use the necessary and sufficient conditions for imputations from the core (12.10).

We have $\forall \bar \xi \in \bar C(x_0)$:

$$\displaystyle \begin{aligned} \sum\limits_{i \in S}\bar \xi_i = \sum\limits_{i \in S}\int_{t_0}^{T} {\xi_i(\tau)} \frac{\sum_{i=1}^nh_i(x^*(\tau))}{V(x^*(\tau), \tau, N)} d{\tau}. \end{aligned}$$

For imputations from the (basic) core C(x ^∗(t), t) we have

$$\displaystyle \begin{aligned} \sum_{i\in S}\xi_i(t) \ge V(S,x^*(t), t),\quad \forall S \subset N. \end{aligned}$$

Hence,

$$\displaystyle \begin{aligned} \sum\limits_{i \in S}\bar \xi_i \geq \bar V(S,x_0, t_0),\quad \forall S \subset N, \end{aligned}$$

and $\bar C(x_0)\subseteq \tilde C(x_0)$.

The inclusion $\bar C(x^*(t), t) \subseteq \tilde C(x^*(t), t), \,\,\forall t \in [t_0, T]$ can be proved in a similar way. □

Moreover, we also have the converse result.

Lemma 12.2

$$\displaystyle \begin{aligned} \tilde C(x_0, t_0) \subseteq \bar C(x_0, t_0); \qquad \tilde C(x^*(t), t) \subseteq \bar C(x^*(t), t), \qquad \forall t \in [t_0, T]. \end{aligned}$$

Proof

We show that for each imputation $\tilde {\xi }_i\in \tilde {C}(x_0,t_0)$, $\tilde {\xi }_i^t \in \tilde {C}(x^*(t), t)$ there exists an integrable selector ξ(t) ∈ C(x ^∗(t), t), t ∈ [t ₀, T] such that

$$\displaystyle \begin{aligned} \tilde{\xi}_i= \int_{t_0}^{ T} \dfrac{\xi_i(\tau) \sum_{i\in N}h_i(x^*(\tau))}{V(N;x^*(\tau),\tau)}d\tau, \\ {} \tilde{\xi}_i^t= \int_{t}^{ T} \dfrac{ \xi_i(\tau) \sum_{i\in N}h_i(x^*(\tau))}{V(N;x^*(\tau),\tau)}d\tau, \\ i=1,\ldots, n. \end{aligned} $$

Since $\tilde {\xi }^t$ is an imputation, we have

$$\displaystyle \begin{aligned}\tilde{\xi}_i^t\geq \bar{V}(\{i\},x^*(t),t) = \int_{t}^T \displaystyle \frac{V(\{i\};x^*(\tau),\tau)}{V(N;x^*(\tau),\tau)} \displaystyle \sum_{i\in N}h_i(x^*(\tau))d\tau.\end{aligned}$$

Moreover, by summing up we get

$$\displaystyle \begin{aligned}\bar{V}(N;x^*(t), t)=\sum_{i=1}^{n}\tilde{\xi}_i^t.\end{aligned}$$

The non-negativeness of the utility functions h _i(⋅) implies that there exist α _i ≥ 0, i = 1, …, n such that

$$\displaystyle \begin{aligned}\tilde \xi_i^t= \int_{t}^T \dfrac{\alpha_i(\tau)+ V(\{i\};x^*(\tau),\tau) )}{V(N;x^*(\tau),\tau)} \sum_{i\in N}h_i(x^*(\tau))d\tau,\end{aligned}$$

and

$$\displaystyle \begin{aligned}\dfrac{\sum_{i=1}^{n} (\alpha_i(\tau)+ V(\{i\};x^*(\tau),\tau) )} {V (N; x^*(\tau),\tau)}=1.\end{aligned}$$

Obviously, that ξ(τ) = {ξ _i(τ) = α _i(τ) + V ({i};x ^∗(τ), τ))} is an imputation in the game with the characteristic function V (S;x ^∗(τ), τ)). But we can also prove that ξ(τ) = {ξ _i(τ) = α _i(τ) + V ({i};x ^∗(τ), τ))} belongs to the core C(x ^∗(τ), τ). For $\tilde {\xi }^t \in \tilde C(x^*(t), t)$ we have

$$\displaystyle \begin{aligned}\displaystyle \sum_{i \in S} \tilde{\xi}_i^t= \int_{t}^T \displaystyle \frac{\sum_{i \in S} (\alpha_i(\tau)+ V(\{i\};x^*(\tau),\tau) ))}{V(N;x^*(\tau),\tau)} \displaystyle \sum_{i\in N}h_i(x^*(\tau))d\tau \\\displaystyle \geq \bar{V}(S,x^*(t),t) = \int_{t}^T \displaystyle \frac{V(S;x^*(\tau),\tau)}{V(N;x^*(\tau),\tau)} \displaystyle \sum_{i\in N}h_i(x^*(\tau))d\tau, \end{aligned} $$

and hence we get

$$\displaystyle \begin{aligned} \sum_{i \in S} (\alpha_i(\tau)+ V(\{i\};x^*(\tau),\tau) )) \geq V(S;x^*(\tau), \tau).\end{aligned} $$

The lemma is proved. □

The preceding results imply that

$$\displaystyle \begin{aligned} \tilde C(x^*(t), t) \equiv \bar C(x^*(t), t),\,\, \forall t \in [t_0, T].\end{aligned} $$

It means, that the core $\tilde C(x_0, t_0)$ constructed by using characteristic function $\bar V$ coincides with the set of imputations $\bar C(x_0, t_0)$ constructed by formula (12.8) for any imputation ξ(t) from the initial core C(x ^∗(t), t). Later on we will use the unified notation $\bar C(x_0, t_0)$ for both sets.

12.5 Strong Time-Consistency

The property of strong dynamic stability (strong time consistency) coincides with the property of dynamic stability (time consistency) for scalar-valued principles of optimality such as the Shapley value [8] or the “proportional solution”. However, for set-valued principles of optimality it has significant and non-trivial sense, which is that any optimal behavior in the subgame with the initial conditions along the cooperative trajectory computed at some intermediate time t ∈ [t ₀, T], together with optimal behavior on the time interval [t, T] is optimal in the problem with the initial condition t ₀. This property is almost never fulfilled for such set-valued principles of optimality as the core or the NM-solution.

Let us formulate the definition of strong time-consistency for an arbitrary optimality principle M(x ₀, t ₀) based on previous results, [9]. A slightly different definition was given in [4].

Introduce the subset M(x ₀, t ₀) of the imputation set L(x ₀, t ₀) as the optimality principle in the cooperative game Γ(x ₀, t ₀). M(x ₀, t ₀) can be a core, a NM-solution, a Shapley value or another one. Similarly, we define this set for all subgames Γ(x ^∗(t), t) along the cooperative trajectory x ^∗(t).

Definition 12.1

The solution (optimality principle) M(x ₀, t ₀) is said to be strongly time-consistent in the game Γ(x ₀, t ₀) if

• 1. M(x ^∗(t), t) ≠ ∅, t ∈ [t ₀, T].

• 2. for any ξ ∈ M(x ₀, t ₀) there exists a vector-function β(τ) ≥ 0 such that

  $$\displaystyle \begin{aligned}M(x_0, t_0) \supset \int_{t_0}^{ t} \beta(\tau)d\tau \oplus {M}(x^*(t),t),\end{aligned} $$

  ∀t ∈ [t ₀, T], $\int _{t_0}^T \beta (t) dt= \xi \in M(x_0, t_0)$.

Here symbol ⊕ is defined as follows. Let a ∈ R ⁿ, B ⊂ R ⁿ, then

$$\displaystyle \begin{aligned} a \oplus B=\{ a+b: b\in B\}.\end{aligned} $$

Let us consider the core $\bar C(x_0, t_0)$ as the set M(x ₀, t ₀). Thus we have the following lemma.

Lemma 12.3

$\bar C(x_0, t_0)$ is a strongly time-consistent optimality principle.

Proof

From the definition of the set $\bar C(x_0, t_0)$ we have that any imputation $\bar \xi \in \bar C(x_0, t_0)$ has the form (12.8). Then for any $\bar \xi \in \bar C(x_0, t_0)$ there exists

$$\displaystyle \begin{aligned} \bar \beta_i(t)=\xi_i(t) \frac{\displaystyle \sum_{i\in N}h_i(x^*(t)}{V(N;x^*(t),t)} \geq 0, \,\, i=1,\ldots, , t \in [t_0, T]\end{aligned} $$

such that $\bar \xi = \int _{t_0}^T \bar \beta (t) dt \in \bar C(x_0, t_0)$.

Let us take another imputation $\hat \xi ^t$ from the core $\bar C(x^*(t), t)$. Then according to the definition of the set $\bar C(x^*(t), t)$ we have that there exists a selector $\hat \xi (t)$ from the initial basic core C(x ^∗(t), t) , i.e. $\hat \xi (t) \in C(x^*(t), t)$ such that

$$\displaystyle \begin{aligned} \hat \beta_i(t)=\hat \xi_i(t) \dfrac{\sum_{i\in N}h_i(x^*(t)}{V(N;x^*(t),T-t)} \geq 0, \,\, i=1,\ldots,N,\,\, t \in [t_0, T],\end{aligned} $$

such that $\hat \xi ^t = \int _{t}^T \hat \beta (t) dt \in \bar C(x^*(t), t)$.

Let us consider the vector-function

$$\displaystyle \begin{aligned} \check{\xi}(\tau)=\begin{cases} \xi(\tau) & \tau \in [t_0, t], \\ \hat \xi(\tau), & \tau \in (t,T], \end{cases}\end{aligned} $$

(12.11)

It is obvious that $\check {\xi }(\tau ) \in C(x^*(\tau ), \tau )$, ∀τ ∈ [t ₀, T]. Then we have a new vector

$$\displaystyle \begin{aligned} \check{\xi}=\int_{t_0}^t \bar \beta(\tau) d\tau + \hat \xi^t = \int_{t_0}^T\check{\xi}(\tau) \dfrac{\sum_{i\in N}h_i(x^*(\tau)}{V(N;x^*(\tau),\tau)} d\tau,\end{aligned} $$

where $\check {\xi }(\tau ) \in C(x^*(\tau ), \tau )$, ∀τ ∈ [t ₀, T].

From the definition of the set $\bar C(x_0, t_0)$ we have that new vector $\check {\xi } \in \bar C(x_0, t_0)$. The vector $\hat \xi ^t$ had been taken from the core $\bar C(x^*(t), t)$ arbitrarily.

So, we have shown that

$$\displaystyle \begin{aligned} \bar{C}(x_0, t_0) \supset \int_{t_0}^{ T} \xi(t) \dfrac{\sum_{i\in N}h_i(x^*(\tau)) }{V(N;x^*(\tau),\tau)}d\tau \oplus \bar{C}(x^*(t),t),\end{aligned} $$

t ∈ [t ₀, T].

The lemma is proved. □

The value

$$\displaystyle \begin{aligned} \xi_i(t) \dfrac{\sum_{i\in N}h_i(x^*(t)}{V(N;x^*(t),t)} \geq 0 \end{aligned}$$

is interpreted as the rate at which the ith player’s component of the imputation, i.e., $\bar {\xi }_i$, is distributed over the time interval [t ₀, T].

12.6 S-strongly Time-Consistency

As above we consider the subset M(x ₀, t ₀) of the imputation set L(x ₀, t ₀) as the optimality principle in the cooperative game Γ(x ₀, t ₀) which can be a core, a NM-solution, a Shapley value or another one. Similarly, we define this set for all subgames Γ(x ^∗(t), t) along the cooperative trajectory x ^∗(t).

Suppose we have two different optimality principles (cooperative solutions) M(x ₀, t ₀) and $\hat M(x_0, t_0)$ such that

$$\displaystyle \begin{gathered} \hat M(x_0, t_0) \subseteq M(x_0, t_0),\\ \hat M(x^*(t), t) \subseteq M(x^*(t), t), \end{gathered} $$

∀t ∈ [t ₀, T]. Again, we assume that these sets are non-empty during the whole game.

Definition 12.2

The cooperative solution $\hat M(x_0, t_0)$ is S-strongly time-consistent (dynamically stable) with respect to the set M(x ₀, t ₀) if for any imputation $\xi \in \hat M(x_0, t_0)$ there exists β(τ) ≥ 0 such that

$$\displaystyle \begin{aligned} M(x_0, t_0) \supset \int_{t_0}^{ t} \beta(\tau)d\tau \oplus \hat{M}(x^*(t),t), \end{aligned}$$

∀t ∈ [t ₀, T], $\int _{t_0}^T \beta (t) dt= \xi \in \hat M(x_0, t_0)$.

Here we introduce the definition of strong time-consistency of the optimality principle with respect to another (bigger) set, hence the prefix S-.

This definition means the following: even if the resulting solution will not belong to the initial set $\hat M(x_0, t_0)$ it will stay within the set M(x ₀, t ₀) which includes $\hat M(x_0, t_0)$.

From Definition 12.2 we have the following proposition.

Lemma 12.4

Let the optimality principle M(x ₀, t ₀) such that M(x ^∗(t), t) ≠ ∅, ∀t ∈ [t ₀, T] be strongly time-consistent. Then any subset $\hat M(x_0, t_0)$ , $\hat M(x_0, t_0) \subseteq M(x_0, t_0)$ such that $\hat M(x^*(t), t) \neq \emptyset ,\, \hat M(x^*(t), t) \subseteq M(x^*(t), t), \forall t \in [t_0,T]$ , is S-strongly time-consistent with respect to M(x ₀, t ₀).

12.7 The Construction of a S-strongly Dynamically Stable Subcore

In the following we identify a subset $\hat {C}(x_0,t_0)$ of the imputations in the set $\bar {C}(x_0,t_0)$, which would belong to the core C(x ₀, t ₀), defined on the basis of the classical characteristic function V (S;x ₀, t ₀).

Consider the value

$$\displaystyle \begin{aligned} \max_{t\leq \tau \leq T} \frac{V(S;x^*(\tau), \tau)}{V(N;x^*(\tau), \tau)}= \lambda(S,t_0),\end{aligned} $$

(12.12)

then the following inequality holds

$$\displaystyle \begin{aligned} \bar{V}(S;x_0, t_0) \leq \lambda (S,t_0) \int_{t_0}^{ T} \displaystyle \sum_{i\in N}h_i(x^*(\tau))d\tau = \lambda(S,t_0)V(N;x_0,t_0).\end{aligned} $$

(12.13)

We introduce a new characteristic function

$$\displaystyle \begin{aligned}\hat{V}(S;x_0,t_0)= \lambda(S,t_0)V(N;x_0,t_0).\end{aligned} $$

(12.14)

Similarly, for t ∈ [t ₀, T] define the respective characteristic function $\hat {V}(S;x^*(t),t)$ as

$$\displaystyle \begin{aligned} \hat{V}(S;x^*(t),t)=\lambda(S,t)V(N; x^*(t),t), \end{aligned} $$

(12.15)

where

$$\displaystyle \begin{aligned} \lambda (S,t)= \max_{t\leq \tau \leq T}\frac{V(S;x^*(\tau), \tau)}{V(N;x^*(\tau), \tau)}. \end{aligned} $$

(12.16)

From (12.12), (12.13), (12.15) and (12.16) we get

$$\displaystyle \begin{aligned} \hat{V}(S;x_0,t_0)\geq \bar{V}(S;x_0,t_0), \end{aligned}$$

$$\displaystyle \begin{aligned} \hat{V}(S;x^*(t),t)\geq \bar{V}(S;x^*(t),t). \end{aligned}$$

Notice that

$$\displaystyle \begin{aligned} \begin{array}{cc} \bar{V}(N;x_0,t_0)= \hat{V}(N;x_0,t_0),\quad \quad &\\ \\ \bar{V}(N;x^*(t),t)= \hat{V}(N;x^*(t),t).& \end{array} \end{aligned}$$

In addition, for all S ₁, S ₂, S ₁ ⊂ S ₂

$$\displaystyle \begin{aligned} \hat{V}(S_1;x^*(t),t)\leq \hat{V}(S_2; x^*(t), t), \ t\in[t_0, T]. \end{aligned}$$

Unfortunately, the property of superadditivity for the function $\hat {V}(S; x^*(t), t)$, t ∈ [t ₀, T] does not hold in general. One can write

$$\displaystyle \begin{aligned} \hat{V} (S;x^*(t),t)=\lambda (S,t) V(N;x^*(t),t)=\\ = \displaystyle \max_{t\leq \tau \leq T} \displaystyle\frac{V(S;x^*(\tau),\tau)}{V(N;x^*(\tau),\tau)}V(N;x^*(t),t) \geq \\ \geq V(N;x^*(t),t) \displaystyle\frac{V(S;x^*(t),t)}{V(N;x^*(t),t)}\geq V(S;x^*(t),t),\,\, S\subset N.\end{aligned} $$

(12.17)

The preceding inequality leads to the following lemma.

Lemma 12.5

The following inequality holds true:

$$\displaystyle \begin{aligned} V(S;x^*(t),t)\leq \hat{V}(S;x^*(t),t),\,\, \forall t \in [t_0,T].\end{aligned} $$

Denote by $\hat {C}(x_0,t_0)$ the set of imputations ξ = (ξ ₁, …ξ _n) such that

$$\displaystyle \begin{aligned} \begin{array}{cc} \displaystyle \sum_{i\in S}\xi_i \geq \hat{V}(S;x_0,t_0), \,\, \forall S \subset N,&\\ \\ \displaystyle \sum_{i\in N}\xi_i = \hat{V}(N;x_0,t_0).& \end{array}\end{aligned} $$

(12.18)

Assume that the set $\hat {C} (x^*(t), t)$ is not empty when t ∈ [t ₀, T]. It is easy to see that it is analogous to the core C(x ₀, t ₀), if the function $\hat {V}(S;x^*(t), t)$ is chosen as the characteristic function.

Thereby we have the statement.

Theorem 12.1 ([10])

The following inclusion takes place:

$$\displaystyle \begin{aligned} \hat{C} (x^*(\tau), \tau) \subset C (x^*(\tau), \tau) \cap \bar{C} (x^*(\tau), \tau),\,\, \forall \tau \in [t_0, T].\end{aligned} $$

(12.19)

We can also formulate the following Theorem (see Fig. 12.1 for an illustration).

Fig. 12.1

The basic core and the associated cores

Theorem 12.2

The subcore $\hat {C} (x_0, t_0) \subset C (x_0, t_0)$ is S-strongly time-consistent with respect to the set $\bar {C} (x_0, t_0)$.

Proof

From Theorem 12.1 we have that $\hat {C} (x_0, t_0) \subset C (x_0, t_0)\cap \bar {C} (x_0, t_0)$, and hence $\hat {C} (x_0, t_0) \subset \bar C (x_0, t_0)$. Lemma 12.3 implies that $\bar C (x_0, t_0)$ is strong-time consistent optimality principle.

Finally, the requested result follows from Lemma 12.4. □

The preceding theorem shows that using the new characteristic function (12.14) we constructed a subset of the classical core C(x ₀, t ₀) (subcore) in the game Γ(x ₀, t ₀) which is S-time-consistent with respect to $\bar C(x_0, t_0)$.

This gives an interesting practical interpretation of the subcore $\hat {C}(x_0, t_0)$. Selecting the imputation ξ from the subcore as a solution, we guarantee that if the players—when evolving along the cooperative trajectory in subgames—change their mind by switching to another imputation within the current subcore $\hat {C} (x^*(\tau ), \tau )$, the resulting imputation will not leave the set $\bar C(x_0, t_0)$ which is also a core in Γ(x ₀, t ₀), but with the characteristic function of the form $\bar V(S,\cdot )$ (12.3) obtained by an integral transformation of classical characteristic function V (S, x ^∗(τ), τ) in the games Γ(x ^∗(τ), τ).

From Theorem 12.1 it follows that the imputations of type $\hat {C}(x^*(t),t)$ belong to the classical core of the game Γ(x ^∗(t), t) for all t ∈ [t ₀, T]. In this sense, Theorem 12.1 establishes a new principle of optimality (cooperative solution).

12.8 Conclusion

In the paper we introduced the definition of S-strong time-consistency in differential games. The approach to the construction of an S-strong time-consistent subcore of the classical core is based on the use of normalized initial characteristic function. We also considered its relation to another characteristic function obtained by an integral extension of the original characteristic function.

We shown that the computed subset of the classical core can be considered as a new optimality principle (cooperative solution) in differential games.

In the future we plan to study the relationship of proposed approach with another constructive approach [12] which allows to identify another subset of the core which is strongly time-consistent.