Abstract
Optimal control theory deals with finding protocols to steer a system between assigned initial and final states, such that a trajectory-dependent cost function is minimized. The application of optimal control to stochastic systems is an open and challenging research frontier, with a spectrum of applications ranging from stochastic thermodynamics to biophysics and data science. Among these, the design of nanoscale electronic components motivates the study of underdamped dynamics, leading to practical and conceptual difficulties. In this work, we develop analytic techniques to determine protocols steering finite time transitions at a minimum thermodynamic cost for stochastic underdamped dynamics. As cost functions, we consider two paradigmatic thermodynamic indicators. The first is the Kullback–Leibler divergence between the probability measure of the controlled process and that of a reference process. The corresponding optimization problem is the underdamped version of the Schrödinger diffusion problem that has been widely studied in the overdamped regime. The second is the mean entropy production during the transition, corresponding to the second law of modern stochastic thermodynamics. For transitions between Gaussian states, we show that optimal protocols satisfy a Lyapunov equation, a central tool in stability analysis of dynamical systems. For transitions between states described by general Maxwell-Boltzmann distributions, we introduce an infinite-dimensional version of the Poincaré-Lindstedt multiscale perturbation theory around the overdamped limit. This technique fundamentally improves the standard multiscale expansion. Indeed, it enables the explicit computation of momentum cumulants, whose variation in time is a distinctive trait of underdamped dynamics and is directly accessible to experimental observation. Our results allow us to numerically study cost asymmetries in expansion and compression processes and make predictions for inertial corrections to optimal protocols in the Landauer erasure problem at the nanoscale.
Avoid common mistakes on your manuscript.
1 Introduction
In his remarkable paper [1] (English translation in [2]), Schrödinger addresses the problem of statistical reversibility of a physical system in contact with an environment. In doing so, he puts forward the idea of using entropic indicators to quantify deviations from thermodynamic equilibrium and, therefore, dissipation. Schrödinger identifies what is now commonly known as the Kullback–Leibler divergence or relative entropy [3] as a quantifier between the joint probability distribution of the system’s end states and those of a free diffusion.
In the last decades of the 20th century, Schrödinger’s trailblazing idea was reformulated into the language of stochastic optimal control [4,5,6], refining Schrödinger’s original “static bridge problem” [1] into a “dynamic Schrödinger bridge", where the relative entropy is computed between probability measures over the systems’ pathspace [7, 8]. Schrödinger bridges have active research interest because they allow computational optimal transport methods to be applied to dynamical models. This enables efficient computation in fields such as neuroscience [9, 10]; data science and machine learning [11]; and generative modeling, sampling, and dataset imputation [12, 13].
Technological advances in the last two decades have paved the way for the observation and manufacturing of nanomachines. At nanoscale, random fluctuations of thermal and topological origin may swamp out any mechanical behavior [14]. A fundamental question is, therefore, how natural or artificial nanosystems can efficiently harness randomness in order to generate controlled motion or perform thermodynamic work on larger scales. Schrödinger bridges find an optimal control protocol to rectify a system obeying stochastic dynamics, thus making it possible to devise systematic methods characterizing the efficiency of nanomachines [15].
In addition, the discovery of fluctuation relations (see Chapter 4 of [16] for a thorough conceptual and historical account) introduces a substantial development with respect to [1]. For Markov stochastic processes, fluctuation relations stem from considering the relative entropy of probability measures connected by a time reversal [17,18,19]: the corresponding generalized Schrödinger bridges minimize the thermodynamic cost associated with the transition. This means that we can consider thermodynamic cost functionals other than the Kullback–Leibler divergence [20,21,22,23,24,25,26,27]. Because the Kulllback-Leibler divergence is non-negative, the minimum mean entropy production in finite time transitions between assigned probability distributions is strictly larger than zero. Remarkably, the overdamped dynamics minimizer [28, 29] turns out to be the solution of a system of Monge–Ampère–Kantorovich optimal mass transport equations [30]. Two results of [28, 29] stand out. First, minimizers can be determined by efficient numerical algorithms even in the multi-dimensional case [31]. Second, the minimum entropy production is proportional to the squared Wasserstein distance between the probability distributions of the end states divided by the duration of the control horizon. This relation between mean entropy production and squared Wasserstein distance continues to hold as scaling limits for Markov jump processes [32] and underdamped dynamics [33, 34], see also [35, 36].
A detailed description of optimal protocols in the underdamped regime is urgent for several reasons. Optically levitated nanoparticles have become a common tool to study transitions in stochastic thermodynamics. Stable confinement and manipulation of nanoparticles within optical traps requires an account of the momentum dynamics. For instance, particle-environment energy exchanges during isoentropic (isochoric) transition within Brownian Carnot (Stirling) engines occurs through the momentum degrees of freedom [37, 38]. Understanding how to simultaneously control particles’ position and momentum is required to devise robust shortcuts to equilibration protocols [39,40,41,42,43].
A further motivation comes from the design of nanoscale electronic components [44,45,46]. Increasing the efficiency of such operations toward the bound prescribed by Landauer is a non-trivial task, with potentially relevant consequences for the design of information and computation technology [47]. The presence of inertia has been shown to lower the energetic cost needed to perform logic operations on bits [44, 48, 49]. This has sparked interest in the control of underdamped stochastic systems, with particular emphasis on the non-linear case, needed for the description of information bits [12, 50,51,52]. Ad-hoc experimental solutions have been found to realize controlled protocols for stochastic dynamics with inertia, confirming that inertial effects allow for fast and precise bit operations [49, 53,54,55].
With these motivations in mind, we introduce a systematic analytical derivation of optimal protocols for the underdamped dynamics. We consider two paradigmatic cases of running costs:
-
The underdamped version of the Schrödinger dynamic bridge problem, referred to as KL. The cost functional in this case is the Kullback–Leibler divergence between the probability measure of the controlled process and that of an assigned reference process. In [1], Schrödinger motivates this cost functional as a quantifier of the likelihood of non-equilibrium fluctuations, i.e., a large deviation functional in the current literature. Since then, Schrödinger dynamic bridge problems have emerged as relevant efficiency measure for diffusion-mediated transport processes with applications ranging from cybernetics [10] to molecular-scale engines [15]. More recently, it has been realized that when the reference process is a diffusion subject to inertia in the absence of a confining potential, Schrödinger dynamic bridge problems provide a viscous regularization of optimal mass transport [56,57,58]. This regularization has applications in machine learning [11]. Finally, when the reference process describes motion in a confining potential equal to that in the Maxwell-Boltzmann distribution of the final state, we obtain a model of an optimally controlled shortcut to adiabaticity.
-
The minimization of the mean entropy production, referred to as EP. This is the cost functional characterizing the second law of thermodynamics and Landauer’s principle [47]. We study this problem in the most general formulation compatible with detailed balance, and which can be self-consistently derived from Hamiltonian mechanics of a system coupled to an infinite bath described by harmonic oscillators. In this generalized formulation, the underdamped entropy production explicitly depends upon the control via a non-dimensional parameter g. Physically, g describes the intensity of momentum coupling between the system and bath. Interactions of these type have been recently observed in Josephson junctions [59, 60].
Besides the cost functional, the specification of an optimal control problem requires a definition of the functional space over which to carry out the optimization. This functional space is called the class of admissible controls. Our focus is on the class of admissible controls described by functions that are sufficiently regular to be differentiable in space and continuous time. Physically, this corresponds to the requirement that the control be slow with respect to the fastest time scale in the problem set by the Wiener process modeling the interaction to the bath. Under these hypotheses, we derive the stationary equations for the cost functionals by taking variations over the class of admissible controls specified by confining mechanical smooth potentials. In such a case [34], extremals of the cost solve a set of integro-differential equations, with features reminiscent of the Vlasov-Poisson-Fokker-Planck problem [61].
We obtain the following main results:
-
I
We show that the cumulants of the probability measure describing transitions between Gaussian states are amenable to the solution of a Lyapunov system of equations [62] in any number of dimensions. This immediately yields a body of rigorous results concerning existence, uniqueness and, when applicable, positivity of solutions (Sect. 4).
-
II
For transitions between states described by Maxwell-Boltzmann distributions in phase space, we introduce an infinite dimensional extension of Poincaré–Lindstedt multiscale perturbation theory [63] around the overdamped limit. This method allows us to treat all cumulants of the system probability measure on the same footing in the renormalization group fashion [64]. We hence obtain explicit predictions for the behavior in time of all phase space cumulants within second order accuracy. The method builds on ideas introduced in [65, 66] for dissipative and [67] for conservative dynamics. Although we restrict our analysis to a two-dimensional phase space, the analysis of the Gaussian case shows that extension to higher dimensional phase spaces is possible, albeit cumbersome (Sect. 6).
-
III
In the case of mean entropy production by an underdamped dynamics with purely mechanical coupling, our results support tightness of the lower bound provided by the overdamped dynamics [68, 69]. For more general couplings, both the mean entropy production and the cost of the dynamic Schrödinger bridge receive strictly positive corrections in the presence of inertia (Sect. 6.1.4).
-
IV
The cost of expansion is higher than that of compression when the initial states are thermodynamically equidistant (Sect. 8.1.1). This result is a manifestation of intrinsic asymmetries in thermal kinematics, recently pointed out in [70, 71].
The structure of the paper is as follows. In Sect. 2, we introduce the model of underdamped dynamics of a nanosystem weakly coupled to an environment by both mechanical and momentum dissipation interaction. When the intensity of the momentum coupling g vanishes, the model recovers the most widely applied underdamped dynamics. Next, we consider two thermodynamic cost functionals, KL and EP, and motivate their broad interest for applications to physics and other applied sciences. Our goal is to minimize these functionals towards the mechanical potential \(U_{t}\) governing the underdamped dynamics conditioned on the system’s initial and final probability distributions. For this reason, we present a brief overview of the mathematical results leading to known bounds for the cost functionals KL and EP in the second half of the section.
In Sect. 3, we introduce the Pontryagin-Bismut functional and derive its stationary equations. The Pontryagin-Bismut functional provides a description of optimal control dual to Bellman’s principle.
Section 4 focuses on the Gaussian case in a phase space of arbitrary dimension, and we derive our first main result here.
In Sect. 5, we set the stage for multiscale perturbation theory presented in Sect. 6. As usual, the idea is to use slow scales to cancel secular terms. Our main goal is to obtain a detailed analytical description of experimentally measurable indicators. We therefore summarize the logic of the derivation and the results before proofs. Readers only interested in our results may thus skip the second part of Sect. 6.
In Sect. 7, we briefly return to the Gaussian case and provide the analytic expression of the solution of the cell problem of the multiscale expansion [72]. The solution of the cell problem allows us to determine all cumulants within second order accuracy in the overdamped expansion.
Section 8 applies the results with some numerical computations. We have emphasized the Gaussian case for two reasons. Firstly, methods for accurate numeric integration of the exact optimal control equations are immediately available, meaning we can compare the perturbative approach with exact numeric predictions in the case of Gaussian boundary conditions. Secondly, transitions between Gaussian states are well adapted to model Brownian engines [70, 71, 73,74,75]. We therefore also study the cost of optimal protocols driving isothermal expansions and compressions of a system to an equilibrium state, which are modelled by a dynamic Schrödinger bridge. Additionally, we solve the cell problem in the case of Landauer’s erasure problem numerically and thus find inertial corrections to the erasure protocol, as well as predictions for the system’s probability measure cumulants.
The final section is devoted to conclusions and outlook. We defer further supplementary material to the Appendices.
2 Underdamped Control Model
We consider the dynamics of a nanosystem with mass m, whose position \(\varvec{\mathcalligra{q}}_{t}\) and momentum \(\varvec{\mathcalligra{p}}_{t}\) obey the Langevin–Kramers stochastic differential equations in \(\mathbb {R}^{2 d}\)
In Eqs. (1), \(\varvec{\mathcalligra{w}}^{(1)}_{t}\) and \(\varvec{\mathcalligra{w}}^{(2)}_{t}\) denote two d-dimensional independent Wiener processes. The Stokes time \(\tau \) is a constant parameter specifying the characteristic time scale of dissipation.
In (1a), a non-dimensional constant g couples the mechanical force \(\varvec{\partial }U\) and the fluctuating environment modeled by the Wiener process \(\varvec{\mathcalligra{w}}^{(1)}_{t}\) to the nanosystem position dynamics. For any \(g\ge 0\), Eq. (1) guarantees convergence towards a Maxwell-Boltzmann equilibrium whenever the potential U is time independent, confining and sufficiently regular. Setting g to zero recovers the standard Langevin–Klein–Kramers model [76].
We emphasize that the dynamics described by (1) are consistent with the general analysis [77] of the conditions guaranteeing the self-consistency of the harmonic environment hypothesis. In fact, (1) can be obtained from a microscopic Hamiltonian dynamics, in which the system interacts with a “bipartite harmonic” environment [78]. By bipartite harmonic environment, we mean an environment modeled by two kinds of oscillators: one type interacts with the system via the commonly assumed position-coupling [79], and the other via a linear momentum coupling [59, 60]. Linear momentum coupling models momentum dissipation observed e.g. in a single Josephson junction interacting with the blackbody electromagnetic field.
As for the force in (1), we only assume that it is the negative gradient of a confining and sufficiently regular mechanical potential, i.e. a potential depending only on the system position. We suppose that potentials of this type give rise to an open set of controls. Within this set, the controls ensure that at every instant of time t in a given time horizon \([{t}_{\iota },{t}_{\mathfrak {f}}]\) the probability density of the system
is well defined and satisfies the Fokker–Planck equation.
At an initial time \(t={t}_{\iota }\), we posit that the state of the nanosystem is statistically described by an assigned Maxwell-Boltzmann distribution at inverse temperature \(\beta \):
Furthermore, we require that at the end of the control horizon \(t={t}_{\mathfrak {f}}\) the probability density of the system satisfies the boundary condition
These assumptions on the probability distributions of the system end states are not necessary for the considerations that follow. They have the merit, however, to be both physically admissible and to lead to simplifications in the multiscale analysis of Sect. 6.
Stylized representation of a Schrödinger bridge modeling Landauer’s erasure of one bit of memory at minimum dissipation
The set of confining potentials \(U_{t}\) that give rise to phase space diffusions with probability marginals (2), (3) define the class of admissible controls of (1).
Our aim is to determine the optimal mechanical potentials \(U_{t}\) among the admissible ones that minimizes the thermodynamic cost functionals defined below conditioned on the initial and final probability distributions (2) and (3).
2.1 Thermodynamic Cost Functionals
We focus our attention on two physically relevant cases, hereafter referred to as KL and EP.
KL: Underdamped dynamic Schrödinger bridge [1]. The thermodynamic cost functional to minimize is the Kullback–Leibler divergence of the measure \(\mathcal {P}=\mathcal {P}_{\iota }^{\mathfrak {f}}\) generated by (1) subject to (2) and (3), from the measure \(\mathcal {Q}=\mathcal {Q}_{\iota }\) generated by (1) when the mechanical force is \( \varvec{\partial } U_{\star }\) and only the initial density (2) is assigned. The cost functional reads (see Appendix A)
The notation \({\text {E}}_{\mathcal {P}} \) emphasizes that the expectation value over the diffusion path is with respect to the measure \(\mathcal {P}\), and \(\text {d}\mathcal {P}/\text {d}\mathcal {Q}\) denotes the Radon-Nikodym derivative between \(\mathcal {P}\) and \(\mathcal {Q}\).
In the mathematics literature, the minimization of (4) at
is referred to as entropic interpolation [7] or entropic transportation cost [80]. This terminology is due to the discovery [56] that the minimization of the overdamped counterpart of (4) yields a viscous regularization of the Monge–Ampère–Kantorovich optimal transport problem (see [30]). Finally, [15] supports the use of the cost of a Schrödinger bridge as a natural efficiency measure for nano-engines in highly fluctuating environments; see [10, 81] for a wider class of applications. In Sect. 8.1.1 we show how the optimization of (4) provides a plausible model of shortcut to equilibration.
EP: Mean entropy production. In stochastic thermodynamics, the average entropy production is identified with the Kullback–Leibler divergence of the forward measure \(\mathcal {P}\) from a measure \(\mathcal {P}_{\mathcal {R}} \) obtained by a combined time-reversal and path-reversal operation (see e.g. [18, 19, 29] and Appendix A for further details):
Some observations are in order. To start with, the identification of (5) as mean entropy production of during a thermodynamic transition is legitimate in consequence of its relation with the heat release by the system evolving according to (1). This is a consequence of the general theory expounded in [19]. In Appendix B for reader convenience, we reproduce the calculation that justifies the identification.
For any g, the entropy production vanishes for a system in a Maxwell-Boltzmann equilibrium. For a bridge process, equilibrium means that the boundary conditions (2), (3) are specified by the same Maxwell-Boltzmann distribution. The corresponding optimal control problem becomes trivial. In any non-trivial case, the Gibbs-Shannon entropy difference appearing in the first row of (5) does not play a role in the optimization as it is fully specified by the boundary conditions.
Finally, the entropy production is non-coercive, i.e it is not a convex functional of the control at g equal zero. As a practical consequence, none of the infinitely many time-dependent protocols that connect the selected end states can be said to minimize the entropy production. Precise treatments of the optimal control problem in such a case are possible either by regularizing the problem [34], or in special cases [75], by considering non-purely mechanical controls [33]. Studying, as we propose here, the mean entropy production at finite g has the advantage of making the cost functional coercive with respect to the mechanical force.
At this point, it is worth commenting on our working hypotheses. The cost functionals in both case KL and EP are readily convex in the mechanical potential. We surmise the existence of an open set of admissible potentials that allows us to look for a minimum in the form of a regular extremal of a variational problem [82]. To justify this assumption we recall that Hörmander’s theorem (see e.g [83]) ensures that any potential \(U_{t}\) (1) that is sufficiently regular, bounded from below, and growing sufficiently fast at infinity results in a smooth density.
2.2 Bounds of the Thermodynamic Cost Functionals
In practice, the cost functionals (4) and (5) are the limit of Riemann sums on ratios of transition probability densities evaluated over increasingly small time increments. This construction is recalled in Appendix A. The construction immediately implies that (4) is bounded from below by the Kullback–Leibler divergence of the joint probability distribution of the system state at the end-times of the control horizon.
The measure theoretic analysis in Sect. 3 of [6] permits drawing more precise qualitative conclusions without making direct reference to the details of the dynamics. To summarize them, let us denote by \(\mathbb {S}\) the state space of dimension \(d_{S}\), where the stochastic process \(\big \{\varvec{\mathcalligra{x}}_{t},t\in [{t}_{\iota },{t}_{\mathfrak {f}}]\big \}\) with probability measure \(\mathcal {P}\) takes values. We also denote by \(\mathcal {P}_{\varvec{y}}^{\varvec{x}}\) (\(\mathcal {Q}_{\varvec{y}}^{\varvec{x}}\)) the probability measures subject to the bridge conditions
Under technical hypotheses guaranteeing that the optimization problem is well-posed, the main takeaways of [6] are the following. First, the Kullback–Leibler divergence is always amenable to the decomposition [4]
The first addend is the quantity originally considered by Schrödinger in [1], namely the static Kullback-Leibler divergence
of the joint probability density \(\wp \) of \(\varvec{\mathcalligra{x}}_{{t}_{\iota }}\) and \(\varvec{\mathcalligra{x}}_{{t}_{\mathfrak {f}}}\) from the two point probability
which is uniquely defined by the transition probability density \({T}_{{t}_{\mathfrak {f}},{t}_{\iota }}^{(\mathcal {Q})}(\cdot \mid \cdot )\) of the reference process and the probability distribution of \(\varvec{\mathcalligra{x}}_{{t}_{\iota }}\).
Both addends in (6) are positive. Furthermore, the static divergence (7) vanishes if and only if
Many possible \(\mathcal {P}\) are compatible with the same \(\wp \). Once \(\wp \) is fixed, \({\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})\) attains an infimum, in fact a minimum, for the \(\mathcal {P}\) that makes the second term of (6) vanish. A necessary condition [4, 6] enforcing this requirement is that for any \({t}_{\iota }\,\le \,s\,\le \,t\,\le \,{t}_{\mathfrak {f}}\)
where the function h is defined by
Once (8) holds true, the control of the abstract optimization problem is \( h_{{t}_{\mathfrak {f}},{t}_{\iota }}\). Correspondingly, (6) reduces to (7), which in turn we can couch into the form
The general form (8) of the necessary condition for the reduction to a static problem does not require the optimal process to enjoy the Markov property; the transition probability (8) may carry memory of the value taken by \(\varvec{\mathcalligra{x}}_{{t}_{\iota }}\). The results of [6] ensure that (8), under further regularity assumptions, reduces for any \(s\,\le \,t \in \left[ {t}_{\iota },{t}_{\mathfrak {f}}\right] \) to a Markov transition probability density
From the physics point of view, the assumptions leading to Markov transition probability densities immediately include an overdamped dynamics [5] or an underdamped dynamics driven by force field depending on both the position and momentum of the system [68, 84], and thus distinct from (1).
Our discussion so far refers to case KL. The connection to case EP stems from the Talagrand-Otto-Villani inequalities [85, 86]. These inequalities show that the static Kullback–Leibler divergence between probability densities is bounded from below by the squared Wasserstein distance between the densities multiplied by a proportionality factor. For the overdamped dynamics considered in [5], Mikami [56] (see also [57, 80, 87]) later proved that the bound becomes tight in a suitable scaling limit and the proportionality factor reduces to the inverse of the duration of the control horizon. More explicitly, the entropic transport cost (\(U_{\star }=0\)) multiplied by the viscosity becomes equal to the cost of a Monge–Ampère–Kantorovich optimal mass transport problem [30] in the limit of vanishing viscosity.
The connection to problem EP consists in the proof [28, 29] and [58] that the minimization of the mean entropy production by bridge processes obeying the overdamped dynamics can be exactly mapped into a Monge–Ampère–Kantorovich optimal mass transport. The reason is that the optimal control problem admits an equivalent reformulation, in which the current velocity of the admissible processes [88] play the role of control instead of the drift.
In the underdamped case, the presence of inertial effects complicates the picture. The mean entropy production cannot be written as the square of the current velocity. This prevents a direct application of the Benamou–Brenier inequality [89] (see also Appendix C). The Benamou–Brenier inequality allows one to couch the minimum mean overdamped entropy production into the squared Wasserstein distance between the densities at the end of the control horizon. It is, however, possible to show [33, 34] that the underdamped mean entropy production admits its overdamped counterpart as a lower bound. In particular, for (5), the following inequality
holds true. Bounds of the type (9) for the mean entropy production appeared in [33, 34] and later in [90]. The proof of (9) presented in [69] is motivated by [91]. For reader convenience, we reproduce the proof in Appendix C.
The above considerations suggest that for both the over- and underdamped dynamics (1), the inequality
should also hold true with \(\mathcal {C}_{\text {TOV}}\) a positive constant in agreement with the Talagrand-Otto-Villani theory [85, 86]. We refer to [68] for a mathematical proof of the bound for the underdamped dynamics, and for the overdamped dynamics [92] (see also [58, 80, 93]), including an explicit prediction of the constant \(\mathcal {C}_{\text {TOV}}\).
3 Optimal Control Formulation
Optimal control problems can be turned into variational problems by coupling the dynamics to the cost functional through Lagrange multipliers. In hydrodynamics, such an approach is referred to as the adjoint equation method, and has a long history going back to [94, 95]. By themselves alone, solutions of the variational equations only provide a necessary condition for the existence of regular extremals of an optimal control problem: extremals continuously satisfying (partial) differential equations. Optimality follows from convexity of the cost, as in our case, or from the study of the second variation. A mathematical formulation of the adjoint method in the context of stochastic optimal control is due to Bismut see e.g. [96]. Bismut theory may be regarded as extending the Pontryagin principle of deterministic optimal control [97] see also [82, 98]. Based on these considerations, we construct from KL and EP variational functionals by imposing that the mean forward derivative [88] of a Lagrange multiplier is an exact differential when the density \({f}_{t}(\varvec{x})\) with respect to the Lebesgue measure satisfies a Fokker–Planck equation associated with probability preserving boundary conditions. As in [33, 34] we refer to these functionals as Pontryagin-Bismut. In such a setup, we look for variational extremals of
Here we collectively denote phase space coordinates as
and define the running cost functional as
In writing (11) we conceptualize the fields \({f}\), V, and U as unknown variational fields. The existence of the functional requires integrability with respect to \({f}_{t}\) which we assume to be a probability density taking the values \({f}_{\iota }\) and \({f}_{\mathfrak {f}} \) at the start and end of the control horizon respectively, fixed by (2), (3). The field V becomes the value function of Bellman’s formulation of optimal control theory [97]. In (11), it plays the role of a Lagrange multiplier enforcing the dynamics.
Accordingly, we denote by \(\mathfrak {L}_{\varvec{x}}\) the differential generator of the dynamics determined by (1):
Thus, if \({f}_{t}\) is the instantaneous density of (1), then
is the mean forward derivative of V along the paths of (1) and by definition
This observation justifies the introduction of the value function as a Lagrange multiplier.
Our definition of the value function in EP omits the contribution from the variation of the Gibbs-Shannon entropy to the mean entropy production from the Pontryagin-Bismut functional. This is because the Gibbs-Shannon entropy in (5) is fully specified by the assigned boundary conditions and therefore does not enter the determination of the optimal control.
3.1 Variational Equations
We determine the optimal control equations by a stationary variation of (11). As expected, the variation with respect to the value function yields the Fokker–Planck equation for the probability density
The variation with respect to the probability density yields the dynamic programming equation [97]
In the overdamped case [5, 28, 58], and in the case when the control is a function of both position and momentum [68, 84, 99], the variation with respect to the potential yields a local, exactly integrable condition for the optimal control. In stark contrast, we find that the optimal control potential in the underdamped case must solve an integral equation coupled to the Fokker–Planck and dynamic programming equations [34]:
with \(\tilde{{f}}_{t}(\varvec{q})\) as the position marginal of \({f}_{t}(\varvec{q},\varvec{p}) \) (see Eq. (126) ) and
Finding regular extremals amounts to finding the simultaneous solutions of Eqs. (13), (14) and (15). The integro-differential stationary condition (15) is hard to approach due to its non-local nature (in momentum space). These issues are to some extent reminiscent of the Vlasov–Poisson–Fokker–Planck (see e.g. [61]) and the McKean–Vlasov (see e.g. [100]) equations. The condition somewhat simplifies when the configuration space is one dimensional. We can write
3.2 Dual Expression of the Optimal Cost
When the dynamic programming equation (14) holds, the Pontryagin-Bismut functional (11) reduces to
The optimum value of the cost hence coincides with the minimum, or at least infimum of (17), taken over all value functions satisfying the dynamic programming equation. This observation is the basis for the aforementioned duality relation used in [92], and later in [58, 68, 80, 93]. In what follows, we use (17) to compute the expression of minimum costs predicted by multiscale perturbation theory.
4 Gaussian Case
In view of the complexity of the optimal control condition (15), it is instructive to analyze the case of Gaussian boundary conditions. A similar analysis was performed in [34] for a case closely related to EP, but only in one dimensional configuration space.
Gaussian boundary conditions lead to major simplifications. The structure of the Fokker–Planck and dynamic programming equations preserve the space of Gaussian probability densities and second order polynomials in phase space for any at most quadratic control
and reference
potentials. In the above expressions \(\varvec{u}_{t} \), \( \varvec{u}_{\star }\) are vectors in \(\mathbb {R}^{d}\) and \( \mathscr {U}_{t}\), \( \mathscr {U}_{\star }\) are \(d\,\times \,d\) real symmetric matrices. Thus, the probability density is fully specified by the set of first and second order cumulants
Here and below, we use \(\otimes \) to denote the outer product of vectors in \(\mathbb {R}^{d}\). Correspondingly, a value function of the form
satisfies the dynamic programming equation (14). In (18), \(\mathscr {V}_{t}^{(q,q)}\) and \( \mathscr {V}_{t}^{(p,p)}\) are \(d\,\times \,d\) symmetric matrices, and
The Fokker–Planck equation (13) reduces to a closed system of differential equations of first order for the second order cumulants of the Gaussian statistics
and to a system of differential equations of first order for the first order cumulants sustained by the solution of second order ones:
The full system of cumulant equations (19)-(20) is complemented by boundary conditions at both ends of the control horizon:
and
The boundary conditions can be satisfied because the potential couples the cumulant equations to a first-order differential system of equal size for the coefficients of the value function in (18).
4.1 Analysis of Case KL
For case KL, we get
and
Finally, we find
The structure of (21)–(22) is analogous to that of the cumulant equations. The coefficients of second degree monomials in (18) satisfy a closed system whose solution sustains the equation for the coefficients of first order monomial.
We now turn to the solution of (15). A straightforward exercize in Gaussian integration yields the explicit expression of the “osmotic force” [88] or “score function” [11] of the position marginal
as well as an explicit expression for the conditional expectation
Upon inserting (24), (25) into (15) and matching the coefficients of monomials of same degree in \(\varvec{q}\), we arrive at the equations
with
and the dependent conditions
Clearly, the conditions (27) are always satisfied if (26) is solvable. In fact we recognize that equation (26a) is in fact a Lyapunov equation. Uniqueness, symmetry and positivity of the solution are very well understood [62]. In particular, for every \(t\in [{t}_{\iota },{t}_{\mathfrak {f}}]\) we can write the solution of (26a) as
The solution is well defined because by definition \( \mathscr {Q}_{t}\) is a positive matrix. Finally, taking the trace of both sides of (28) readily recovers (26b) thus completing the proof that the Gaussian case is solvable.
4.2 Analysis of Case EP
The equations that change are (21a)
Eq. (22a) which is replaced by
and, finally, Eq. (23) which for the mean entropy production reads
A qualitative difference with case KL occurs for vanishing g when the mean entropy production does not explicitly depend upon the control potential. This is most evident when inspecting (15). We get
where now
and
Whereas for \(g\,>\,0\) the optimal potential is uniquely determined by the solution of the Lyapunov equation (29a), the limit \(g\downarrow 0\) is singular. The Lyapunov equation becomes a constraint imposed on the coefficients of the value function. The upshot is that for vanishing g it is not possible to satisfy boundary conditions imposed on all phase space cumulants. In other words, the problem is not generically solvable for a generic assignment of Gaussian probability densities (2), (3). The problem admits, however, a solution if boundary data are just the position marginals. A detailed slow manifold analysis performed in the one-dimensional case in the supplementary material of [34] shows that the equation for g equal zero coincides with the slow manifold equations (see e.g. [101]) of the limit \(g \downarrow 0\) optimal control equations. This gives a precise mathematical meaning to the idea of \(\delta \)-Dirac optimal control upheld in [21]. It also shows that even if the optimal control does not exist for g equal zero, the strictly positive lower bound on the mean entropy production is always in agreement with (9).
5 General Case in One Dimension
A distinctive trait of the underdamped extremal equations (13), (14) and (15) is the integral term in Eq. (15), which introduces a non-local condition in the momentum variable. This is in stark contrast with the overdamped counterpart of (15). Indeed the latter is exactly integrable and thus reduces the extremal conditions to a pair of local hydrodynamics equations [5, 28]. In this section we construct a systematic multiscale expansion of (13) - (15) around the overdamped limit. By proceeding in this way we manage to reabsorb the non-locality in phase space into effective parameters of local equations—the cell problem—in configuration space. We perform our analysis in two-dimensional phase space. Extension to higher dimensional phase space is possible at the price of dealing with far more cumbersome algebra.
The approach we follow is inspired by [65, 66]. The first step is to project the momentum dependence in Eqs. (13)–(15) onto the basis of Hermite polynomials orthonormal with respect to the Maxwell thermal equilibrium distribution. We obtain an kinetic-theory-type hierarchy of coupled equations that do not depend on the momentum. Despite the additional complication of dealing with an infinite number of equations, this description turns out to be the ideal starting point for a multiscale expansion approach (in time).
5.1 Non-dimensional Variables
In order to neaten our notation, it is expedient to preliminary introduce non-dimensional variables:
where \(\ell \) is the typical length-scale set by the mechanical potentials in the boundary conditions.
Next, we introduce the non-dimensional counterparts of the phase space density, value function and mechanical control potential:
In non-dimensional variables, the generator of the phase space process (12) becomes
where now the order parameter of the overdamped expansion
explicitly appears. Equipped with these definitions, we rewrite the Fokker–Planck
the dynamic programming
and the stationary condition equations
where
denotes the position marginal of the probability density function.
5.2 Expansion in Hermite Polynomials
Calling \(H_n\) the n-th Hermite polynomial (see Appendix D for details), we expand the probability density and the value function as
and
The expansion coefficients \({\text {f}}_{{\text {t}}}^{(n)}\) and \({\text {V}}_{{\text {t}}}^{(n)}\) are scalar functions of the position and time. At equilibrium, the expansion for the probability density consists of the term \(n=0\) only. The remaining contributions are non-zero only in out-of-equilibrium conditions. In particular, all \({\text {f}}_{{\text {t}}}^{(n)}\) and \({\text {V}}_{{\text {t}}}^{(n)}\) for \(n\,>\,0\) vanish at the beginning and at the end of the control horizon because of the boundary conditions (2) and (3). The expansion in Hermite polynomials turns the extremal equations (33) into an infinite hierarchy of equations whose n-th elements are:
More detail on the derivation of the above equations is given in Appendix D. The hierarchy is complemented by equilibrium boundary conditions on the probability density, that, in the non-dimensional variables, read:
6 Multiscale Perturbation Theory
The hierachy (36) is equivalent to the original extremal equations (33), and holds for any value of \(\varepsilon \). We are interested in cases where \(\varepsilon \ll 1\) in order to solve (36) with a perturbative strategy. The limit of vanishing \(\varepsilon \) is, however, singular and cannot be handled by regular perturbation theory. We therefore resort to multiscale perturbation theory. In doing so, we need to take into account an essential difference with respect to the multiscale treatment of the overdamped limit of the underdamped dynamics [65, 66]. The difference is that the mechanical potential is not assigned but must be determined by solving the stationary conditions (36c). In addition, we are dealing with a time-boundary value problem rather than with an initial data problem. To overcome these difficulties, we formulate the multiscale expansion drawing from the Poincaré–Lindstedt technique [63] and renormalization group ideas that in recent years have been successfully applied to the resummation of perturbative series arising from Hamiltonian and dissipative dynamical systems [67]. Our strategy is based on the following considerations.
-
We suppose that the time variation of all functions in the hierarchy (36) occurs through effective time variables
$$\begin{aligned}&{\text {t}}_{j}=\varepsilon ^{j} \,{\text {t}}, & j \,\ge \, 0\,. \end{aligned}$$(38)occasioned by the overdamped order parameter \(\varepsilon \). As a consequence, the partial derivative with respect to \({\text {t}}\) breaks down into a differential operator
$$\begin{aligned} \partial _{{\text {t}}}=\partial _{{\text {t}}_{0}}+\varepsilon \,\partial _{{\text {t}}_{1}} +\varepsilon ^{2}\,\partial _{{\text {t}}_{2}}+\dots \end{aligned}$$(39)thus introducing a new dynamical variable at each order of the overdamped expansion.
-
We assume that the mechanical potential has a finite limit when \(\varepsilon \) tends to zero. This assumption [33, 34] is central in order to recover the overdamped dynamics [5, 28]. The a priori justification of the assumption is that momentum marginals of the boundary conditions already describe a Maxwell thermal equilibrium. The need for a controlled dynamics only arises in consequence of the boundary conditions imposed on the position process. In the generator (31), the mechanical potential is coupled to the dynamics by the overdamped expansion order parameter \(\varepsilon \). This fact leads to the inference that the control potential should admit a regular expansion in \(\varepsilon \) as a function of the position variable, varying in time on scales set by \(\varepsilon \).
-
The Poincaré–Lindstedt method is usually formulated for initial value problems. In such a context, the dynamics of the slow times \({\text {t}}_{j}\) with \(j\,>\,0\) is fixed by canceling secular terms (equivalently: resonances), i.e. polynomial terms in the time variable which as times increases would lead to a breakdown of perturbation theory. Such a secular term subtraction scheme is equivalent to a renormalization group type partial resummation of the perturbative expansion [67]. We need to adapt the subtraction scheme to a boundary problem. At \(\varepsilon \) equal zero the Fokker–Planck hierarchy (36a) is decoupled from the dynamic programming one (36b). As a consequence, the boundary conditions (37) cannot be satisfied at zero order of the regular perturbative expansion. We therefore use the boundary conditions to determine the slow time dependence of the \({\text {f}}^{(n)}\).
-
The value function expansion coefficients \({\text {f}}^{(n)} \) are not subject to anything other than satisfying the dynamics (36a). The logical basis for the resonance subtraction scheme is the duality relation (17). We reason that a cost can only be generated on the same time scales over which the mechanical control potential varies. We thus require that the dependence of \({\text {f}}^{(0)} \) must be constant with respect to the fastest time \( {\text {t}}_{0}\). We also observe that, although physically motivated, a non-uniqueness is intrinsic in any secular term cancellation or finite renormalization scheme exactly because these techniques involve a partial and not a complete resummation of the perturbative expansion [64]. Consistent alternative schemes may differ by higher order terms in the regular perturbative expansion.
-
The introduction of the slow time variables (38) is justified under a sufficiently wide scale separation. In principle, the perturbative expansion only holds for \(\varepsilon \ll 1\) and \({\text {t}}_{\mathfrak {f}}\gg 1\) (i.e. \({t}_{\mathfrak {f}}\gg \tau \)). Yet, we hope that extrapolating the results for finite control horizons will give sufficiently accurate results if the resummation scheme correctly captures the “turnpike behavior” of the exact solution of the optimal control. Turnpike behavior means the tendency of optimal controls to approximate the solution of the adiabatic limit, corresponding to a vanishing cost, as much as possible. We refer to [68] for further discussion and references on this point.
In summary, our aim is to look for a solution of (36) in the form of multiscale power series
where each addend of the above series depends, a priori, on all time scales
6.1 Results
We report the main results of the overdamped multiscale expansion, while deferring their derivation to Sect. 6.2. Without loss of generality, we set
to neaten the notation. Within second order in \(\varepsilon \) in the multiscale expansion, the solution of the Fokker–Planck equation takes the form
We emphasize that \({\text {t}}_{0}={\text {t}}\) and \({\text {t}}_{2}=\varepsilon ^2 \,{\text {t}}\) and that Eq. (42) is independent of \({\text {t}}_{1}=\varepsilon \,{\text {t}}\). We also neglect slower time scales \({\text {t}}_{j}\), \({j}\,>\,2\), as they only provide higher order corrections. Hence, for all the results presented in this Subsection, we drop the explicit dependence on \({\text {t}}_{1}\) and \({{\text {t}}}_{3}\).
6.1.1 Cell Problem Equations
As customary [72], we refer to the secular term subtraction conditions emerging at order \(O(\varepsilon ^{2})\) in regular perturbation theory as the cell problem. Secular term subtraction fixes the functional dependence upon the slow time \({\text {t}}_{2}\). As a consequence, we find it expedient to denote the unknown quantities of the cell problem as
and as an auxiliary field \(\sigma _{{\text {t}}_{2}}({\text {q}})\) related to \( v_{{\text {t}}_{\mathfrak {f}},{\text {t}}_{2}}^{(0:0)}({\text {q}})\) and \({\text {f}}_{{\text {t}}_{0},{\text {t}}_{2}}^{(0:0)}({\text {q}})\) by equation (101) in Sect. 6.2 below. The formulation of the cell problem in terms of the pair \(\rho _{{\text {t}}_{2}}\), \(\sigma _{{\text {t}}_{2}}\) exactly recovers the optimal control equations governing the overdamped limit in KL and EP:
We fully specify the cell problem by complementing (44) with the exact boundary conditions imposed by the position marginals of (2), (3) and written in non-dimensional variables as in (37)
In (44) we imply
for EP. Depending upon the problem under consideration, the constant \(\alpha \) takes the values
while the constants A and B are given by (see Eq. (98) below)
with
A and B always admit a finite limit as g tends to zero: by (48) the limit of vanishing g entails \(\omega \) tending to infinity in case EP. Furthermore, they depend upon the size of the control horizon \({\text {t}}_{\mathfrak {f}}\) so that
When \(U_{\star }=0\), the cell problem reduces to a coupled system of a Fokker–Planck and Burgers’ equation
By (49) and (46) in the limit of infinite control horizon (\({\text {t}}_{\mathfrak {f}}\nearrow \infty \)), we recover the result of [57] that in the overdamped limit optimal entropic transport is a viscous regularization of the minimization of the mean entropy production. As the optimal control of the latter problem [28] is equivalent to optimal mass transport, we also recover Mikami’s result [56]. In fact, for \({\text {t}}_{\mathfrak {f}}\) finite but sufficiently large to justify the scale separation required by the multiscale approach, the cell problem (50) allows us to extract information about corrections to the overdamped limit.
6.1.2 Cumulants and Marginal Distribution
Solving the cell problem allows us to evaluate the leading order corrections to the overdamped limit of all phase space cumulants within order \(O(\varepsilon ^{2})\). Namely, all cumulants turn out to be linear combinations of functionals of the pair \(\rho _{{\text {t}}_{2}}\) and \(\sigma _{{\text {t}}_{2}}\), weighed by pure functions of the fast time \({\text {t}}_{0}\).
We denote the non-dimensional counterparts of the position and momentum processes with a tilde
Unlike the cell problem, the cumulants are functions of both the fast \({\text {t}}_{0}\) and slow \({\text {t}}_{2}\) time variables. To neaten the expressions, we denote the moments of the position process with respect to the probability density specified by the cell problem as
Correspondingly, we also write
In particular,
is a constant, i.e.
for both the optimal entropic transport (KL with zero reference potential) and minimum mean entropy production EP problems. We justify this claim in Appendix F.
Momentum mean. Recalling (42), the expectation value of the momentum process conditioned on the position process is
In section 6.2, we show how to compute \({\text {f}}_{{\text {t}}_{0},{\text {t}}_{2}}^{(1:1)}\) from the solution of the cell problem. We obtain
Here \( a_{{\text {t}}_{0}}\) and \( b_{{\text {t}}_{0}}\) are pure functions of the fast time \({\text {t}}_{0}\):
It is straightforward to verify that
hence enforcing the boundary conditions imposed on (53). The derivation of an explicit expression of this quantity requires the solution of the \(O(\varepsilon ^{4})\) cell problem in the same way as (44a) specifies \({\text {f}}^{(1:1)}\).
Equipped with the above definitions, by integrating (52) in \({\text {q}}\) we arrive at
In order to interpret this result, we recall a standard result of multiscale analysis (see e.g. § 2.5.1 of [72]) ensuring that
when the separation of scales is sufficiently large: \({\text {t}}_{\mathfrak {f}}\gg O(1)\) with \( \varepsilon ^{2}\,{\text {t}}_{\mathfrak {f}}=O(1)\). The relation (98) between the integral over the functions (54) and the constants A and B (47) implies that as the duration of the control horizon grows, the momentum expectation tends to
Once recast into dimensional quantities, the identity reads
Correction to the position marginal distribution. Upon integrating out the momentum variable in (42), we get
In Sect. 6.2 we show that
Inspection of (58) reveals that the marginal (57) exactly satisfies the non-perturbative boundary conditions (45) and preserves normalization within accuracy. We avail ourselves of (57) to evaluate the remaining linear and second order cumulants.
Position mean. We readily obtain
As expected, at the boundaries of the control horizon the cumulant are fully specified by the boundary conditions and so are independent of \(\varepsilon \).
Position-momentum cross correlation. After straightforward algebra we find
where
and
Position variance. We obtain its expression by evaluating the difference between
and the squared mean value
After some algebra, we find that the expression of the variance reduces to
with \(\varsigma _{{\text {t}}_{2}} \) and \(\dot{\varsigma }_{{\text {t}}_{2}} \) defined by (61).
Momentum variance. The expectation value of the squared momentum conditioned on the position process is
with
After some tedious algebra we arrive at
We notice that the variance satisfies the boundary conditions in consequence of the identity
which follows from (92).
6.1.3 Optimal Control Potential
Similarly, the cell problem yields the leading order expression for the gradient of the optimal control potential
with
and
the current potential of the cell problem. In other words, the gradient of (66) is the current velocity [88] which allows us to represent (44a) as a mass conservation equation for any strictly positive \(\alpha \).
By definition, the current velocity vanishes when the system is in a Maxwell-Boltzmann equilibrium state. Hence, finite time transitions at minimum cost are not between Maxwell-Boltzmann equilibrium states, as we see from the explicit expression of the drift at the end times
and
From the physics point of view, this means transitions minimizing thermodynamic cost functionals have non-vanishing current velocity at the start and end of the protocol. Mathematically, this is unsurprising because the boundary conditions associated to the optimal control problem do not impose any conditions on the terminal values of the control potentials.
For all practical purposes, the shape of potential corresponding to the boundary equilibrium states can be matched at zero cost, through an instantaneous change of the control.
6.1.4 Minimum Cost
We evaluate the expression for the minimum cost using the duality relation (17).
-
KL
: The projection onto Hermite polynomials couches (17) into the form
$$\begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})=\int _{\mathbb {R}}\text {d}{\text {q}}\,\left( {\text {f}}_{0}^{(0)} v_{0}^{(0)}-{\text {f}}_{{\text {t}}_{\mathfrak {f}}}^{(0)} v_{{\text {t}}_{\mathfrak {f}}}^{(0)}\right) \,. \end{aligned}$$At leading order, multiscale perturbation theory yields the approximation
$$\begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})=\int _{\mathbb {R}}\text {d}{\text {q}}\,\left( {\text {f}}_{0,0}^{(0:0)} v_{{\text {t}}_{\mathfrak {f}},0}^{(0:0)}-{\text {f}}_{0,\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}^{(0:0)} v_{{\text {t}}_{\mathfrak {f}},\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}^{(0:0)}\right) +O(\varepsilon ^{2})\,. \end{aligned}$$This is because the non-perturbative boundary conditions only allow contributions that are proportional to \( {\text {f}}_{0,0}^{(0:n)}\). In addition, we subtract secular terms in the value function expansion by requiring
$$\begin{aligned} v_{{\text {t}}_{\mathfrak {f}},{\text {t}}_{2}}^{(0:2)}=v_{0,{\text {t}}_{2}}^{(0:2)}\,. \end{aligned}$$Thus, in our multiscale framework, the value of \(v_{{\text {t}}_{\mathfrak {f}},\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}^{(0:2)}\) can be only determined by higher order cell problems. To gain insight into the predicted features of the minimum, we couch the optimum value of the divergence into the form
$$\begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})=-\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d} {\text {t}}_{2}\int _{\mathbb {R}}\text {d}{\text {q}}\, \partial _{{\text {t}}_{2}} \left( {\text {f}}_{0,{\text {t}}_{2}}^{(0:0)} \,v_{{\text {t}}_{\mathfrak {f}},{\text {t}}_{2}}^{(0:0)}\right) +O(\varepsilon ^{2})\,. \end{aligned}$$The above representation allows us to express the divergence in terms of the cell problem density (43) and the identity
$$\begin{aligned}&v_{{\text {t}}_{\mathfrak {f}},{\text {t}}_{2}}^{(0:0)}=\frac{\sigma _{{\text {t}}_{2}}+(\alpha -A) \ln \rho _{{\text {t}}_{2}} }{2\,A}-\frac{{\text {U}}_{\star }}{2} -\frac{B\,{\text {q}}\,\dot{\mu }_{{\text {t}}_{2}}^{({1})}}{2\,A\,(A-B)} -\frac{B}{4\,A\,(A-B)}\int _{0}^{{\text {t}}_{2}}\text {d}{\text {s}}\, (\dot{\mu }_{{\text {s}}}^{({1})})^{2} \end{aligned}$$stemming from (89) and (101) in Sect. 6.2. Indeed, straightforward algebra yields
$$\begin{aligned} \begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})&=\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {s}}\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {s}}}\,\dfrac{\big (\partial _{{\text {q}}} (\sigma _{\text {t}_2}-\alpha {\text {U}}_{\star })\big )^2 }{4\,A}\\&\qquad +\frac{A-\alpha }{2\,A}\int _{\mathbb {R}}\text {d}{\text {q}}\,\left( \rho _{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\,\ln \frac{\rho _{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}}{\rho _{\star }}-\rho _{0}\,\ln \frac{\rho _{0}}{\rho _{\star }}\right) \\&\qquad +\frac{B}{4\,A\,(A-B)}\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {s}}\, (\dot{\mu }_{{\text {s}}}^{({1})})^{2}+O(\varepsilon ^{2}) \end{aligned} \end{aligned}$$(67)where
$$\begin{aligned}&\ln \rho _{\star }=-{\text {U}}_{\star }-\ln \int _{\mathbb {R}}\text {d}{\text {q}}\,e^{-{\text {U}}_{\star }} \end{aligned}$$and
$$\begin{aligned} A-B=(1+g)\left( 1-\frac{2 \tanh \frac{{\text {t}}_{\mathfrak {f}}}{2}}{{\text {t}}_{\mathfrak {f}}}\right) \end{aligned}$$which is positive definite when \({\text {t}}_{\mathfrak {f}}\,>\,2\). In (67), all terms but the first vanish in the limit of infinite scale separation \({\text {t}}_{\mathfrak {f}}\) tending to infinity. Further elementary considerations shed more light on the sign of the corrections. Recalling (66) and the properties of the current velocity, we obtain the identity
$$\begin{aligned} \begin{aligned} \int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}} \big (\partial _{{\text {q}}} (\sigma _{t_2}-\alpha {\text {U}}_{\star })\big )^2&=\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}\,(\partial _{{\text {q}}}\mathcalligra{c}_{{\text {t}}_2})^{2}+\alpha ^{2}\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}\left( \partial _{{\text {q}}} \ln \frac{\rho _{{\text {t}}_{2}}}{\rho _{\star }}\right) ^{2}\\&\quad +2\,\alpha \,\partial _{{\text {t}}}\int _{\mathbb {R}}\text {d}q\,\rho _{{\text {t}}_{2}}\ln \frac{\rho _{{\text {t}}_{2}}}{\rho _{\star }}\,. \end{aligned} \end{aligned}$$(68)We then re-write (67) as
$$\begin{aligned} \begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})&= \frac{1}{4\, \alpha }\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}} \big (\partial _{{\text {q}}} (\sigma _{{\text {t}}_{2}}-\alpha {\text {U}}_{\star })\big )^2\\&\quad -\frac{B}{4\,A\,(A-B)}\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\left( \int _{\mathbb {R}} \text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}(\partial _{{\text {q}}}\mathcalligra{c}_{{\text {t}}_{2}})^{2}-(\dot{\mu }_{{\text {t}}_{2}}^{({1})})^{2}\right) \\&\quad +\frac{\alpha -(A-B)}{4\,\alpha \,(A-B)} \int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}} (\partial _{{\text {q}}}\mathcalligra{c}_{{\text {t}}_{2}})^{2}\\&\quad +\alpha \,\frac{\alpha -A}{4\,A}\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\int _{\mathbb {R}} \text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}\left( \partial _{{\text {q}}} \ln \frac{\rho _{{\text {t}}_{2}}}{\rho _{\star }}\right) ^{2}+O(\varepsilon ^{2})\,. \end{aligned} \end{aligned}$$The identity
$$\begin{aligned} \dot{\mu }^{({1})}=\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}(\partial _{{\text {q}}}\mathcalligra{c}_{{\text {t}}_{2}}) \end{aligned}$$and the Cauchy–Schwarz inequality then ensure that all corrections are positive for \({\text {t}}_{\mathfrak {f}}\,>\,2\). Thus for any \({\text {t}}_{\mathfrak {f}}\) sufficiently large to ensure a separation of time scales, we arrive at the inequality
$$\begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})\,\ge \,\frac{1}{4\,\alpha } \int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}\big (\partial _{{\text {q}}} (\sigma _{{\text {t}}_{2}}-\alpha {\text {U}}_{\star })\big )^2 \end{aligned}$$whence we read the multiscale perturbation theory prediction of the Talagrand-Otto-Villani constant \(\mathcal {C}_{\text {TOV}}\) in (10). To do so, we focus on entropic transport and set \({\text {U}}_{\star } \) to zero. Next, we recall that any solution of the cell problem (50a)–(50b) enjoys the lower bound [102]
$$\begin{aligned} \int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\int _{\mathbb {R}}\text {d}{\text {q}}\, \rho _{{\text {t}}_{2}}\big (\partial _{{\text {q}}} \sigma _{{\text {t}}_{2}}\big )^2\,\ge \,\frac{{\text {E}}_{\widetilde{\mathcal {P}}} \left| \tilde{\mathcalligra{q}}_{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}-\tilde{\mathcalligra{q}}_{0}\right| ^{2}}{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}} \end{aligned}$$(69)where \( \widetilde{\mathcal {P}}\) is the measure generated by the overdamped Schrödinger bridge in \([0,\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}]\) associated to the stochastic differential equation
$$\begin{aligned} \text {d}\tilde{\mathcalligra{q}}_{{\text {t}}_{2}}=-(\partial \sigma _{{\text {t}}_{2}})(\tilde{\mathcalligra{q}}_{{\text {t}}_{2}})\, \text {d}{\text {t}}_{2}+\sqrt{2\,\alpha }\,\text {d}\mathcalligra{w}_{{\text {t}}_{2}}\,. \end{aligned}$$(70)The inequality (69) is a consequence of the law of iterated expectation (see e.g. [76] pag. 310). Indeed, it ensures that
$$\begin{aligned} \int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}\big (\partial _{{\text {q}}} \sigma _{{\text {t}}_{2}}\big )^2&\,\equiv \,\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}{\text {E}}_{\widetilde{\mathcal {P}}}\big ((\partial \sigma _{{\text {t}}_{2}})(\tilde{\mathcalligra{q}}_{{\text {t}}_{2}})\big )^{2}\\&=\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}{\text {E}}_{\widetilde{\mathcal {P}}}\left( {\text {E}}_{\widetilde{\mathcal {P}}}\left( \big ((\partial \sigma _{{\text {t}}_{2}})(\tilde{\mathcalligra{q}}_{{\text {t}}_{2}})\big )^{2}\,\big |\,\tilde{\mathcalligra{q}}_{0}\right) \right) \\&\,\ge \,\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}{\text {E}}_{\widetilde{\mathcal {P}}}\left( \Big ({\text {E}}_{\widetilde{\mathcal {P}}}\left( (\partial \sigma _{{\text {t}}_{2}})(\tilde{\mathcalligra{q}}_{{\text {t}}_{2}})\,\big |\,\tilde{\mathcalligra{q}}_{0}\right) \Big )^{2}\right) \,. \end{aligned}$$We now invert the order of integration and apply the Benamou–Brenier argument [89] to the stochastic paths generated by (70) and find
$$\begin{aligned}&\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}}\big (\partial _{{\text {q}}} \sigma _{{\text {t}}_{2}}\big )^2\,\ge \,\frac{ {\text {E}}_{\widetilde{\mathcal {P}}}\left( \Big ({\text {E}}_{\widetilde{\mathcal {P}}}\left( \tilde{\mathcalligra{q}}_{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}-\tilde{\mathcalligra{q}}_{0} -\sqrt{2\,\alpha }\,\tilde{\mathcalligra{w}}_{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}} \big |\tilde{\mathcalligra{q}}_{0}\right) \Big )^{2}\right) }{\varepsilon ^{2}\,{\text {t}}_{\mathfrak {f}}}\,. \end{aligned}$$The inequality now follows because \( \tilde{\mathcalligra{w}}\) is the Wiener process with respect to the measure \(\widetilde{\mathcal {P}} \) and as such has zero conditional expectation with respect to \(\tilde{\mathcalligra{q}}_{0} \). In Appendix E we present a path integral derivation of the same result. The upshot is that for entropic transport we get a Talagrand-Otto-Villani type inequality
$$\begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})\,\ge \,\frac{1}{4\,\alpha }\frac{{\text {E}}_{\widetilde{\mathcal {P}}}\left| \tilde{\mathcalligra{q}}_{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}-\tilde{\mathcalligra{q}}_{0}\right| ^{2}}{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\,. \end{aligned}$$In dimensional units, the same result reads
$$\begin{aligned} {\text {K}}(\mathcal {P}\mathrel {\Vert }\mathcal {Q})\,\ge \,\frac{1}{4\,\alpha }\frac{\beta \,m{\text {E}}_{\widetilde{\mathcal {P}}}\left| \mathcalligra{q}_{{t}_{\mathfrak {f}}}-\mathcalligra{q}_{0}\right| ^{2}}{\tau \,{t}_{\mathfrak {f}}}\,. \end{aligned}$$ -
EP
: Upon contrasting (5) with (11), the exact expression of the minimum mean entropy production reads
$$\begin{aligned} \mathcal {E}=\int _{\mathbb {R}}\text {d}{\text {q}}\, \left( {\text {f}}_{0}^{(0)} \big (v_{0}^{(0)}+\ln {\text {f}}_{0}^{(0)}\big )-{\text {f}}_{{t}_{\mathfrak {f}}}^{(0)} \big (v_{{t}_{\mathfrak {f}}}^{(0)}+\ln {\text {f}}_{{t}_{\mathfrak {f}}}^{(0)}\big )\right) \,. \end{aligned}$$The multiscale approximation then is
$$\begin{aligned} \mathcal {E}=-\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\hspace{-0.4cm}\text {d}{\text {t}}_{2}\int _{\mathbb {R}}\text {d}{\text {q}}\, \partial _{{\text {t}}_{2}} \left( {\text {f}}_{0,{\text {t}}_{2}}^{(0:0)} (v_{{\text {t}}_{\mathfrak {f}},{\text {t}}_{2}}^{(0:0)}+\ln {\text {f}}_{0,{\text {t}}_{2}}^{(0:0)})\right) +O(\varepsilon ^{2}) \end{aligned}$$where, by (89) and (101), the identity
$$\begin{aligned}&v_{{\text {t}}_{\mathfrak {f}},{\text {t}}_{2}}^{(0:0)}+\ln {\text {f}}_{0,{\text {t}}_{2}}^{(0:0)}=\frac{2\,\sigma _{{\text {t}}_{2}}}{A}+\frac{2\,B\,{\text {q}}\,\dot{\mu }^{({1})}}{A(A-B)} +\frac{B\,(\dot{\mu }^{({1})})^{2}\,{\text {t}}_{2}}{A\,(A-B)} \end{aligned}$$with
$$\begin{aligned} \dot{\mu }^{({1})}=\frac{\mu _{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}^{({1})}-\mu _{0}^{({1})}}{\varepsilon ^{2}\,{\text {t}}_{\mathfrak {f}}} \end{aligned}$$holds true. After some algebra, we arrive at
$$\begin{aligned} \begin{aligned} \mathcal {E}&=\dfrac{1}{1+g}\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}} (\partial _{{\text {q}}}\sigma _{{\text {t}}_{2}})^{2} +\dfrac{1+g-A}{(1+g)\,A}\int _{0}^{\varepsilon ^{2}{t}_{\mathfrak {f}}}\hspace{-0.4cm}\text {d}{\text {t}}_{2}\,\left( \int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}} (\partial _{{\text {q}}}\sigma _{{\text {t}}_{2}})^{2}-(\dot{\mu }^{({1})})^{2}\right) \\&\quad + \dfrac{1+g-(A-B)}{4\,(1+g)\,(A-B)}\,(\dot{\mu }^{({1})})^{2}\,\varepsilon ^{2}\,{\text {t}}_{\mathfrak {f}}+O(\varepsilon ^{2}) \end{aligned} \end{aligned}$$(71)with
$$\begin{aligned} A-B=(1+g)\left( 1-\frac{2 \tanh \frac{\omega \,{\text {t}}_{\mathfrak {f}}}{2}}{\omega \,{\text {t}}_{\mathfrak {f}}}\right) \,. \end{aligned}$$All addends in (71) are positive. Furthermore, the last two vanish both in the limit of infinite scale separation and upon recalling the definition (48) of \(\omega \) when the coupling constant g is vanishing
$$\begin{aligned} \lim _{g\searrow 0}\mathcal {E}=\int _{0}^{\varepsilon ^{2}{\text {t}}_{\mathfrak {f}}}\text {d}{\text {t}}_{2}\,\int _{\mathbb {R}}\text {d}{\text {q}}\,\rho _{{\text {t}}_{2}} \,(\partial _{{\text {q}}}\sigma _{{\text {t}}_{2}})^{2}\,. \end{aligned}$$We also emphasize for case EP, the field \(\sigma \) satisfies the compressible Euler equation. As a consequence, we can directly apply the Benamou-Brenier inequality [89] to (71) and straightforwardly recover the bound (9).
6.1.5 Accuracy of the Multiscale Approximation
Infinite hierarchies of equations such as (36) appear in the study of Liouville’s and Boltzmann equations [65, 103]. Many numerical methods resort to a phenomenological truncation of the hierarchy. The multiscale method provides a controlled truncation at the level of second order equations. In fact, all cumulants up to second order can be reconstructed from an effective first order system embodied by the cell problem.
In Fig. 2, we summarize how the secular term cancellation (or, equivalently, solvability) conditions allow us to re-order contributions of the regular perturbative expansion within the hierachy. The upshot is that the predictions for cumulants and total cost obtained from the solution of the cell problem have different accuracies in \(\varepsilon \).
6.2 Order-by-Order Solution
In this Section, we solve the hierarchy of equations (36) in a multiscale perturbative series in powers of \(\varepsilon \). To this goal, we insert Eqs. (40a)–(40c) into Eq. (36), and identify equations of distinct order in the power expansion, taking into account the time differentiation, which acts on the multiscale dependence of the probability density and value function according to (39). The derivation of the results is briefly outlined in words below.
At order zero in \(\varepsilon \), the equations for the density and value function give rise to two decoupled infinite systems of first order differential equations in the fast time \({\text {t}}_{0}\). These systems are trivially integrable with respect to the fast time \({\text {t}}_{0}\), implicitly keeping all information about the boundary condition in the unresolved dependence of the integration constants upon the slow times.
Remarkably, at order \(\varepsilon ^{1}\) the boundary (37) and stationary conditions reduce the non-trivial contribution of the two infinite hierarchies of equations to a system of two first order differential equations in the fast time for \( {\text {f}}^{(1:1)}\) and \(v^{(1:1)}\). Dependence upon higher order coefficients of the expansion in Hermite polynomials enters these equations in the form of functions of the slow time \({\text {t}}_{2}\) that must be determined at order \(\varepsilon ^{2}\) in the regular perturbative expansion. As no secular term appears at this order we can assume within accuracy independence of the solution of the extremal equations from \({\text {t}}_{1}\).
At order \(\varepsilon ^{2}\), we can determine all unknown quantities inherited from lower orders in the regular perturbative expansion by imposing the cancellation of secular terms. This fixes the dynamical dependence upon the slow time \({\text {t}}_{2}\) in the form of a cell problem. We enforce the correct boundary conditions in terms of \( {\text {f}}^{(0:2)}\), \( {\text {f}}^{(2:2)}\) and \(v^{(0:2)}\), \(v^{(2:2)}\). Finally, if we set all the \( {\text {f}}^{(n:0)}\), \({\text {f}}^{(n:1)}\) that are not sustained by the drift and all the \( v^{(n:0)}\), \( v^{(n:1)}\) that are not needed to control the non-vanishing contributions to the density to zero, it is self-consistent to set
Figure 2 is a stylized summary of the procedure. Additional details are provided in Appendix F.
In principle, it is possible to extend the analysis to orders higher than \(\varepsilon ^{2}\), as done in [65]. The appearance of spatial derivatives of higher order than the second may, however, call for the introduction of appropriate variables to perform partial resummations [103]. We return to this point in Sect. 9
Scheme of the multiscale approach presented in the text. The logical order of the calculation is represented by the black solid arrows, going through the sequence of solutions of the differential systems at the different orders. The quantities computed with this strategy are reported in the coloured boxes, where different colours correspond to different steps of the order-by-order multiscale calculation. Dashed arrows show the functional dependencies of the computed quantities. \(^\star\)The calculation actually shows that there is no dependence on the \({\text {t}}_{1}\)-time scale
6.2.1 Boundary Conditions
The boundary conditions (2), (3) are by hypothesis independent of the Stokes time and therefore remain the same once expressed in non-dimensional units. Consequently, all \({\text {f}}_{{{\text {t}}}_{0}}^{(n:i)}\)’s with \(n\,\ge \, 1\) vanish at the boundaries, so that
where \(\delta _{i,j}\) is a Kronecker delta. Without loss of generality, we set \({t}_{\iota }=0\).
The non-perturbative boundary behavior is not assigned a priori but is determined by that of the probability density. However, in multiscale perturbation theory, we have the freedom to choose how partial resummations to cancel secular terms are performed [63]. We have reasoned that contributions to the cost can only come from the same time scale as those where the control varies, which gives the following resonance subtraction condition
6.2.2 Solution of the Problem at Order Zero
The calculation starts at order zero of the \(\varepsilon \)-expansion. Equation (36a) can be written at order zero in \(\varepsilon \) as:
which implies
Here \(c_{{{\text {t}}}_{1}}\) is fixed by imposing the initial condition at time \({\text {t}}_{0}=0\):
following from Eq. (72). This observation leads to
Solving the value function equation (36b) at order zero in \(\varepsilon \) gives
This time we have no boundary conditions to impose. However, it follows from Eq. (36c) that
hence
The \({\text {t}}_{0}\)-dependence of the probability density and the value function is completely determined at order zero. We have no way to enforce the boundary condition at \(t={\text {t}}_{\mathfrak {f}}\) for \({\text {f}}_{{{\text {t}}}_{0}}^{(n:0)}\) at this stage: we will need to impose it on a slower time scale, in this way exploiting the additional freedom provided by the multiscale approach.
6.2.3 Solution of the Problem at Order One
By expanding Eq. (36a) at order one in \(\varepsilon \), one gets
The boundary conditions for the probability density force all terms of order higher than zero in \(\varepsilon \) vanish at \(t=0\) and \(t={\text {t}}_{\mathfrak {f}}\). This is a consequence of our assumption that the protocol starts and ends in equilibrium states, which cannot depend on the relaxation time scale \(\varepsilon \). They must coincide with the stationary states of the overdamped limit \(\varepsilon \rightarrow 0\). The \(n=0\) case of Eq. (6.2.3), by recalling Eq. (75), implies therefore that \({\text {f}}_{{{\text {t}}}_{0}} ^{(0:0)}\) is independent of \({\text {t}}_{1}\), hence
where we have introduced the notation
Similarly, the equations with \(n \ge 2\) lead to
The case \(n=1\) is less trivial and brings about the relation
Similarly, the value function equation (36b) at order one, for the case \(n=1\), gives
Once complemented with a condition for the drift \(\partial _{{\text {q}}} U^{(0)}_{{{\text {t}}}_{0}}\), Eqs. (82) and (83) form a closed system of differential equations. The missing relation can be obtained from the stationarity condition (36c), which at order one in \(\varepsilon \) reads
Eq. (84) provides an expression for the drift, which can be inserted into Eq. (82) to obtain a relation for \(v^{(1:1)}_{{{\text {t}}}_{0}}\) (see Eq. (140) in Appendix F). The system is then solved by differentiating the resulting equation with respect to \({\text {t}}_{0}\), and eliminating \(\partial _{{\text {t}}_{0}}v^{(1:1)}_{{{\text {t}}}_{0}}\) through (83) and \(v^{(1:1)}_{{{\text {t}}}_{0}}\) through (140). A second-order ODE for \({\text {f}}_{{{\text {t}}}_{0}}^{(1:1)}\) is found:
with \(\omega \) as defined in (48). The dependence of \(F_{{{\text {t}}}_{0}}({\text {q}})\) on \({\text {t}}_{0}\) is known; for its explicit expression, see Eq. (141). The equation can be solved by recalling that the Green function for the second order differential equation (85) is
with
with \(\theta (\cdot )\) being the Heaviside step-function. By introducing the notation
one obtains for \({\text {f}}_{{{\text {t}}}_{0}}^{(1:1)}\) the relation
where
and a notation analogous to (80) is adopted also for the value function. The last step of the solution of the order \(\varepsilon ^{1}\) consists in writing the optimal control potential as a function of (88). From Eq. (82) we find
Since there are no equations for \(\partial _{{\text {t}}_{1}}{\text {f}}_{{{\text {t}}}_{0}}^{(1:1)}\) nor \(\partial _{{\text {t}}_{1}}v^{(1:1)}_{{{\text {t}}}_{0}}\), i.e. no secular terms are found on the time scale \({\text {t}}_{1}\), we can assume the solution to be independent of \({\text {t}}_{1}\). Once \({\text {f}}_{{{\text {t}}}_{0}}^{(1:1)}\) is known, an explicit expression for \(v^{(1:1)}_{{{\text {t}}}_{0}}\) can also be found from (83) (see Eq. (140) in Appendix F). From the above equation it is possible to derive the expression (65) for the optimal drift, by using the expressions of \({\text {f}}_{{\text {t}}_{0};{{\text {t}}}_{2}}^{(0:0)}\) and \({\text {f}}_{{\text {t}}_{0};{{\text {t}}}_{2}}^{(1:1)}\) that will be found in the next subsection.
6.2.4 Solution of the Problem at Order Two
The order-two expansion of (36a) provides the following relations
The last equation (91d) ensures that all terms \({\text {f}}_{{{\text {t}}}_{0}}^{(n:2)}\) with \(n>2\) vanish once equilibrium boundary conditions are taken into account. Equation (91b) provides a relation for \({\text {f}}_{{{\text {t}}}_{0}}^{(1:2)}\) that requires knowledge of \(\partial _{{\text {q}}} {\text {U}}^{(1)}_{{{\text {t}}}_{0}}\). If we expand the stationary condition (36c) to second order in \(\varepsilon \) and assume that all \(v_{{{\text {t}}}_{0}}^{(n:0)} \), \(v_{{{\text {t}}}_{0}}^{(n:1)} \) that are not needed to control the non-vanishing \({\text {f}}_{{{\text {t}}}_{0}}^{(n:2)}\)’s can be set to zero, we get
We insert this result into (91b) and the corresponding equation for \(v_{{{\text {t}}}_{0}}^{(1:2)} \), and after straightforward, albeit tedious, algebra we arrive at
Taking into account the boundary conditions, we get
and we conclude that for any \({\text {t}}_{0}\),
The same applies to \(v_{{\text {t}}_{0},{{\text {t}}}_{1}}^{(1:2)}\).
Let us focus first on Eq. (91c). Integrating over \({\text {t}}_{0}\), one has
By substituting the expression of the drift obtained from Eq. (90) and integrating the term proportional to \(\partial _s{\text {f}}_{{\text {s}};{{\text {t}}}_{2}}^{(1:1)}\) by parts, we find
which is Eq. (63). This relation implies, recalling the boundary conditions, that
By substituting Eq. (88), an equation for the term \(\partial _{{\text {q}}}\left( v^{(2:0)}_{{\text {t}}_{\mathfrak {f}};{{\text {t}}}_{2}} \,{\text {f}}_{0;{{\text {t}}}_{2}}^{(0:0)}\right) \) can be derived (see Eq. (142) in Appendix F). Once plugged back into Eq. (88) itself, it yields
Here we introduce the functions whose explicit expression we gave in (54)
By (87) the two functions \(a_{{\text {t}}_{0}}\) and \(b_{{\text {t}}_{0}}\) are non homogeneous solution of the unstable oscillator equation weighed by constant coefficient also depending upon integrals over the Green function
In (93) we also introduce
where we use the function \(\zeta _{{{\text {t}}}_{2}}\) defined in Eq. (89). Equation (93) will be crucial in the following, as it allows to write a closed system of differential equations for \({\text {f}}_{0;{{\text {t}}}_{2}}^{(0:0)}\) and \(v^{(0:0)}_{{\text {t}}_{\mathfrak {f}};{{\text {t}}}_{2}}\), which can be reshaped as in Eqs. (44).
Taking into account the boundary conditions and Eq. (90), Eq. (91a) can be integrated over \({\text {t}}_{0}\) to give
If we now substitute Eq. (93) we get
where
The above relations lead to Eq. (47).
We now need to find an equation for \(\zeta _{{{\text {t}}}_{2}}\) in order to close the differential system and find the \({\text {t}}_{2}\)-dependence of \({\text {f}}_{0;{{\text {t}}}_{2}}^{(0:0)}\). To this aim, we consider the case \(n=0\) for the expansion of Eq. (36b) at order two in \(\varepsilon \). It reads
We integrate the above equation over \({\text {t}}_{0}\). By substituting (90) and making repeated use of Eqs. (85), (93) and (92) (see Appendix F for details), one finds
which is the closure equation for \(\zeta _{{{\text {t}}}_{2}}\). The constant \(\alpha \) is defined by Eq. (46).
The differential system for \({\text {f}}_{0;{{\text {t}}}_{2}}^{(0:0)}\) and \(\zeta _{{{\text {t}}}_{2}}\) can be rewritten in a much more convenient form by introducing the auxiliary field
Indeed, taking into account Eq. (149), it is easy to verify that Eqs. (97) and (100) are amenable to the form (44). Let us stress that, in terms of the field \(\sigma \), Eq. (97) becomes
Upon inserting this identity, (101), and (51), in (93), we recover the expression (53).
Finally, by plugging Eqs. (96) and (90) in Eq. (91a) one gets:
Integrating over \({\text {t}}_{0}\) leads to Eq. (58).
7 Analytic Results for the Gaussian Case
As discussed in Sect. 6.1 and shown analytically in Sect. 6.2, in order to find the explicit solution of the optimal problem, one first needs to address the differential system (44). In most cases, the solution can only be found numerically: this is discussed in the next Section. However, if the assigned initial and final conditions are Gaussian probability density functions (meaning that the particle is subject to harmonic confinement), the solution can be found analytically.
To do this, we plug a Gaussian ansatz for the density and a parabolic one for \(\sigma _{{\text {t}}_{2}}\), namely
where \(\mu ^{(1)}\) and \(\mu ^{(2)}\) are consistent with Eq. (51), into Eqs. (44).
Next, we solve for the coefficients, taking into account the boundary conditions. The derivation is straightforward and not carried out here. For both cases KL and EP, and \({\text {U}}_{\star }=0\), the explicit expressions for the relevant coefficients appearing in Eqs. (103) are
where
and for \(\dot{\varsigma }_{{\text {t}}_{2}}=\partial _{{\text {t}}_{2}}\varsigma _{{\text {t}}_{2}}\)
Knowing these coefficients allows us to compute the cumulants discussed in Sect. 6.1.2 for the general case.
It is worth noticing that these results result in a remarkably simple expression for mean entropy production at \(g=0\)
When
(104) remains valid, whereas it is possible to close the hierachy with a second order equation for the variance of the position process
The general solution of this equation takes the form
with the constants \(c_{i}\), \(i=1,2,3\), related by the algebraic equation
Unfortunately resolving the \(c_{i}\)’s in terms of generic boundary conditions leads to somewhat cumbersome expressions. In Sect. 8.1.1 we consider a special case of particular relevance.
8 Numerically Assisted Applications
In this section, we apply numerical methods to the multiscale expansion to analyze the underdamped dynamics, both in the case of Gaussian boundary conditions and in more complex boundary conditions, in particular, those modelling Landauer’s one bit of memory erasure.
In the Gaussian case, we have a system of differential equations specifying the non-perturbative solution, and we can therefore use numerical integration to solve the associated boundary value problems, from which we can obtain the first and second order phase space cumulants. We then use the perturbative approach to compute the same values, which show good agreement: see Figs. 3 for case KL and 4 for case EP. Additionally, we take a look at expansion and compression with Gaussian boundary conditions in Sect. 8.1.1.
Furthermore, the perturbative approach can be used to make predictions for the cumulants when no analytic solution is available. We demonstrate this using boundary conditions modelling Landauer’s one bit of memory erasure, as illustrated in Fig. 1. This requires numerically solving the cell problem (50), from which we obtain the optimal control protocol and the marginal distribution of the position in the overdamped dynamics. We can then compute leading order corrections to approximate the quantities in the underdamped dynamics.
8.1 Gaussian Case
In cases KL and EP, when the boundary conditions are assigned as Gaussian random variables, we have two boundary value problems for the first and second order cumulants. For case KL, we compute approximate solutions to the systems (19) and (20), and, for case EP, we make the amendments as described in Sect. 4.2.
The perturbative approach follows Sect. 7, and instead we have only one boundary value problem. The dependant quantities: momentum mean, momentum variance and the position-momentum cross correlation, as well as the higher order corrections to the position mean and variance can then be computed.
The respective boundary value problems are integrated numerically using the DifferentialEquations.jl [104] library in the Julia programming language. The results of the perturbative and non-perturbative integrations for case KL are in Fig. 3 and for case EP are in Fig. 4. In both case KL and case EP, we see that the perturbative expansion gives a very good approximation of the true solution.
Position mean (a) and variance (b); position-momentum cross correlation (c); and momentum mean (d) and variance (e) for the underdamped problem minimising the Kullback–Leibler divergence (Case KL) from a free diffusion with assigned Gaussian initial and final conditions. We plot the expressions of cumulants up to second order predicted by our perturbative approach, solid dark blue line, and contrast them with the numeric solution of the corresponding non-perturbative equation of Sect. 4.1 shown with a dashed light blue line. To plot the perturbative expressions we use the exact solution of the cell problem given in Sect. 7 and then we determine the full perturbative prediction for each of the cumulants using the expressions given in Sect. 6.1.2. We impose Gaussian boundary conditions through the first and second order cumulants of the position and momentum (tildes denoting non-dimensional coordinates): at initial time \({\text {t}}= 0\), we set the position and momentum variance \(\tilde{\mathscr {Q}}_{0}=\tilde{\mathscr {P}}_{0}=1\), the position-momentum cross correlation \(\tilde{\mathscr {C}}_{0} = 0\); and the position and momentum means \({\text {E}}_{\mathcal {P}}\tilde{\varvec{\mathcalligra{q}}}_{{\text {t}}_{\iota }} ={\text {E}}_{\mathcal {P}}\tilde{\varvec{\mathcalligra{p}}}_{{\text {t}}_{\iota }} =0\). At final time \({\text {t}}= {\text {t}}_{\mathfrak {f}}\): the momentum variance \(\tilde{\mathscr {P}}_{{\text {t}}_{\mathfrak {f}}}=1\); the position-momentum cross correlation \(\tilde{\mathscr {C}}_{{\text {t}}_{\mathfrak {f}}} = 0\); the position variance \(\tilde{\mathscr {Q}}_{{\text {t}}_{\mathfrak {f}}}=1.7\); the momentum mean \( {\text {E}}_{\mathcal {P}}\tilde{\varvec{\mathcalligra{p}}}_{{\text {t}}_{\mathfrak {f}}} =0\); and the position mean \({\text {E}}_{\mathcal {P}}\tilde{\varvec{\mathcalligra{q}}}_{{\text {t}}_{\mathfrak {f}}} = \sqrt{2}\). We use \({\text {t}}_{\mathfrak {f}}=5\), \(\varepsilon =0.2\) and \(g=0\). Numerical integration is performed by a fourth order co-location method in the DifferentialEquations.jl library [104]
Position mean (a) and variance (b); position-momentum cross correlation (c); and momentum mean (d) and variance (e) for the underdamped problem minimising the entropy production EP from a free diffusion with assigned Gaussian initial and final conditions. We use \({\text {t}}_{\mathfrak {f}}= 5\), \(\varepsilon = 0.2\), and compare the values computed by the perturbative approach (solid lines) with the numeric solution of the non-perturbative system at \(g=0.5\) (blue, dashed) and \(g=0.1\) (green, dotted), and \(\omega = \sqrt{(1+g)/g}\). We impose Gaussian boundary conditions through the first and second order cumulants of the position and momentum. We use the same values for the boundary conditions and the same numerical integration method as in Fig. 3. We compute the perturbative predictions in the same way as explained in the caption of Fig. 3. The equations of the non-perturbative system are those of Sect. 4.2
8.1.1 Asymmetry in Optimal Approaches to Equilibrium
Thermal kinematics in the underdamped dynamics. We compare a compression (blue) and expansion (orange) process starting from thermodynamically equidistant states from the final state. We fix the final position variance as \(\mathscr {Q}_{{t}_{\mathfrak {f}}} = 4\), the corresponding reference potential \(\mathscr {U}_{\star } = 1/4\), and \(\beta = 1\) in all panels. Panels (a)-(c) show the picture where the initial variances are chosen to be close together. We use \(\mathscr {Q}^{(c)}_{0} = 4.20\) for compression and \(\mathscr {Q}^{(e)}_{0} \approx 3.8065\) for expansion. For (a)-(b), we use \({\text {t}}_{\mathfrak {f}}=5\) and \(\varepsilon =0.1\). Panel (a) shows the variance of the position, with the solid lines computed non-perturbatively and the overlayed dashed lines computed using the linearized approach outlined in Sect. 8.1.1. Panel (b) shows the static Kullback–Leibler divergence (107), with the inset axes showing the difference between the compression and expansion process. Panel (c) shows the difference in the dynamic Kullback–Leibler divergence as a function of the time horizon \({\text {t}}_{\mathfrak {f}}\), with \(\varepsilon =0.5\). Panels (d)–(e) illustrate the case when the initial variances are chosen to be further apart. We use \(\mathscr {Q}^{(c)}_{0} \approx 10.3467\) for compression and \(\mathscr {Q}^{(e)}_{0} = 1\) for expansion. We use \({t}_{\mathfrak {f}}=5\) and \(\varepsilon =0.1\). Panel (d) shows the variances and (e) shows the static Kullback–Leibler divergence as functions of the time interval. Panel (f) shows the dynamic Kullback–Leibler divergence as a function of the time horizon \({\text {t}}_{\mathfrak {f}}\), with \(\varepsilon =0.5\). First order cumulants do not play a role in the analysis and are equal to 0 in all panels. All numerical integration is performed by DifferentialEquations.jl [104], as in Fig (3)
Very recently, [70, 71] highlighted the existence of a cooling versus heating asymmetry in the relaxation to a thermal equilibrium from hotter and colder states that are “thermodynamically equidistant”. Although not strictly a distance, the Kullback–Leibler divergence from the thermal state may be used to identify the dual processes [70]. We show that a similar asymmetry also occurs in optimally controlled isothermal compressions versus expansions of a small system.
To this goal we make the following observations. Choosing a reference potential \(U_{\star }\) in (4) equal to the potential in the final condition (3) forces the current velocity specified by the optimal protocol to be as small as possible at the end of the control horizon. In this sense, the optimal control problem models a relaxation to a thermal equilibrium in finite time. Well-established laboratory techniques [37, 105, 106] use the fact that the optical potential generated by a laser to trap a colloidal nanoparticle is effectively Gaussian. We combine these two observations to compare the compression versus the expansion of a nanosystem in an isothermal environment when the initial data are thermodynamically equidistant from the final equilibrium state. Mathematically, this means that the position marginals of the boundary conditions (2), (3) are centered Gaussians that differ only in the variance. In such a case, the only non-trivial optimal control equations are (19) and (21). Our aim is to compare a compression and an expansion process starting from “dual” initial states. Duality is with respect the Kullback–Leibler divergence from the end state whose value is initially the same for the two opposite processes. In the notation of Sect. 4 the Kullback–Leibler divergence for \(d=1\) reads
We fix the terminal condition
and compare the evolution of probability densities specified by initial conditions at \({t}_{\iota }=0\)
such that the initial position marginals have equal Kullback–Leibler divergence from the final state
The dynamic Schrödinger bridge with boundary conditions \((\mathscr {Q}_{0}^{(e)},\mathscr {Q}_{\star })\) / \((\mathscr {Q}_{0}^{(c)},\mathscr {Q}_{\star })\) provides a model of optimal expansion/compression of the system towards the equilibrium state characterized by \( \mathscr {Q}_{\star }\).
The multiscale prediction for the position variance is
where for the sake of simplicity we set \(g=0\). To relate non-dimensional quantities to their dimensional counterparts, we explicitly write the Stokes time \(\tau \) and the typical length-scale \(\ell \) of the transition. We suppose that the variance of the non-dimensional cell problem at the beginning of the control horizon is
How much the non-dimensional constant \(\textsf{v}\) differs from unity controls the thermodynamic distance from the final state. In such a case we find that the coefficients \(c_{i}\)’s in (106) are
with
The above expressions are exact and provide a useful benchmark for exact numerical integration of the cumulant hierarchy (see Fig 5).
For transitions describing small deformations of the position marginal of the system, it is however expedient to resort to simpler approximated expressions. We obtain these by linearizing (105) around the final condition of the transition. In other words, we look for a solution of the form
with dots corresponding to higher order terms in the non-linearity. We obtain
This expression allows us to analytically compare the behavior of the divergence from a common end state of system undergoing an expansion and a compression. We see that if we choose
for \(\eta \sim O(10^{-1})\) then within \(O(10^{-4})\) accuracy the initial data for the dual compression process is
A straightforward calculation then shows that within leading order accuracy
The result holds analytically for small deformations of the potential \(\eta \ll 1\) and close to the overdamped limit \(\varepsilon \,\ll \,1\).
Another thermodynamic indicator encoding similar information is the cost of the dynamic Schrödinger bridge (4). This quantity is a global indicator of the transition that can be studied versus the duration of the horizon. Consistently with the analytic perturbative result the evaluation of (4) shows that the divergence from equilibrium is larger for compression processes. The difference between compression and expansion tends to zero as the duration of the horizon tends to infinity, thus indicating symmetry restoration for adiabatic processes.
Our findings are summarized in Fig. 5. Our analysis is in line with the findings of [70]. If we interpret the divergence from equilibrium at any fixed time as an indirect quantifier of the speed with which the system ultimately thermalizes, our analytic and numerical results confirm that expansion is faster than compression for Gaussian models.
8.2 Landauer’s Erasure Problem
Predictions for the position mean (a) and variance (b); position-momentum cross correlation (c); momentum variance (d) and mean (e) in the underdamped dynamics minimizing the Kullback–Leibler divergence (Case KL) from a free diffusion computed using the perturbative expansion. The boundary conditions are assigned on the marginal density of the position: the initial state at \({\text {t}}= 0\) is a single peaked distribution centered at \(x_{\mathfrak {o}} = 1\) (see Eq. (108)) and the final is a double peaked distribution with peaks at \(-1\) and 1 (see Eq. (109)). We set \({\text {t}}_{\mathfrak {f}}= 5\), \(\varepsilon =0.2\), \(g=0\), \(\omega = 1\) and use \(\alpha = \sqrt{(1+g)\,A}\approx 0.64\), where A is as defined in (98a). The functions \(\rho _{{\text {t}}_{2}}({\text {q}})\) and \(\sigma _{{\text {t}}_{2}}({\text {q}})\) are approximated numerically for values of \({\text {t}}_{2}\in [0,\varepsilon ^2\, {\text {t}}_{\mathfrak {f}}]\) as the solution of Eq. (50) by a forward-backward iteration. We perform a total of 15 forward and backward passes of the iteration. At each step, the factors \(\phi \) and \(\hat{\phi }\) are computed by means of Eq. (112) and normalized for numerical stability. We use a total of 8000 sample points for \({\text {q}}\) in the interval \([-6,6]\) and evolve \(50\ 000\) independent Monte Carlo trajectories started from each \({\text {q}}\) by an Euler-Maruyama discretization of the SDE (113) with step-size \(h=0.005\). The forward-backward iteration is initialized by \(\hat{\phi }_{{\text {t}}_{\mathfrak {f}}}\) with a vector of ones. We recover the functions \(\rho \) and \(\sigma \) from \(\phi \) and \(\hat{\phi }\) using Eqns. (114); these are then smoothed using a convolution with a box filter with window size \(\delta = 0.06\). The predictions for the underdamped moments are then computed using expressions found in Sect. (6.1.2)
We model the Landauer’s one bit of memory erasure [47] as a Schrodinger bridge problem between an initial state single-peaked distribution and final state as a double-peaked distribution, as illustrated in Fig. 1. We can make predictions for the first and second order cumulants of the position and momentum distributions from the perturbative expansion, by computing the numerical solution to the cell problem (50) and hence the appropriate corrections. We focus only on case KL.
We assign the initial and final state of the position marginal distribution
where \(P_{\iota }\) and \(P_{\mathfrak {f}}\) denote the assigned initial and final distributions, and here take the explicit forms
with \(Z_{\iota },\ Z_{\mathfrak {f}}\) normalizing constants. The initial condition is a single peaked distribution centered at \(x_{\mathfrak {o}}\), and final condition is a double peaked distribution, with peaks at \(x_{\mathfrak {o}}\) and \(-x_{\mathfrak {o}}\).
We look at the case of \(U_{\star }=0\). The cell problem (44) can be approximated numerically using a forward-backward iteration. This specifically means computing the numerical solution of two coupled non-linear partial differential equations to obtain the functions \(\rho \) and \(\sigma \) of the slow time \({\text {t}}_{2} = \varepsilon ^2 {\text {t}}\).
We adopt the methodology of [84], beginning with the Hopf-Cole transform
yielding a pair of Fokker–Planck equations
with coupled boundary conditions
In this form, the cell problem can be solved using the forward-backward iteration, an adaptation of Algorithm 1 of [84]. We make a slight simplification, in that we perform the numerical integration of equations (110) by a Monte Carlo method, computing
using the forward and backward evolution respectively of the underlying auxiliary (Ito) stochastic process
where \(\{\mathcalligra{w}_{{\text {t}}_{2}}\}_{{\text {t}}_{2}\ge 0}\) denotes a standard Wiener process. The values of \(\mathcalligra{q}_{{\text {t}}_{2}}\) are approximated with discretized trajectories of (113) by the Euler-Maruyama scheme.
The forward-backward iteration goes as follows: We begin by sampling a set of values for \({\text {q}}\) from an interval on which both the initial and final assigned distributions \(P_{\iota }\) and \(P_{\mathfrak {f}}\) are compactly supported. We initialize the forward-backward iteration by taking a set of (positive) values for \(\hat{\phi }_{\varepsilon ^2{\text {t}}_{\mathfrak {f}}}\), which are then used to compute the boundary condition (111a) for equation (110a). We integrate equation (110a) using the expression (112b) to obtain \(\phi _0\) and recompute \(\hat{\phi }_0\) using (111b). By integrating (110b) using (112a) up to \({\text {t}}_{\mathfrak {f}}\), we once again obtain \(\hat{\phi }_{\varepsilon ^2{\text {t}}_{\mathfrak {f}}}\). This procedure is then repeated until convergence; we verify that the boundary condition relations (111) are satisfied, and the mean-squared difference between two iterations of \(\phi _{{\text {t}}_{\mathfrak {f}}}\) and \(\hat{\phi }_{{\text {t}}_{\mathfrak {f}}}\) is less than a specified tolerance. We can then recover the values of \(\rho _{{\text {t}}_{2}}\) and \(\sigma _{{\text {t}}_{2}}\) by the relations
The optimal control protocol in the overdamped case is \(\sigma \). From here, we use the relevant equations in Sects. 6.1.2 and 6.1.3 to make predictions for the first and second order cumulants of the position and momentum in the underdamped dynamics, which are shown in Fig. 6. The predicted marginal distribution of the position and the gradient of the optimal control protocol is shown in Fig. 7. Figure 8 contrasts the heights of the peaks of the marginal distribution of the position in the underdamped and overdamped dynamics over the time interval.
Predictions for the marginal density of the position (a)-(e) and (k) -(o) and the gradient of the optimal control protocol (f)-(j) and (p) -(t) in the overdamped (orange) and underdamped (blue) dynamics minimizing the Kullback–Leibler divergence (Case KL) from a free diffusion, \(U_{\star }=0\). We show the distribution \(\rho \) and the gradient of the optimal control protocol \(-\partial _{\text {q}} \sigma \) (114) for the overdamped dynamics, and compute the corrections needed to obtain the corresponding quantities \({\text {f}}_{{\text {t}}}\) and \(-\partial _{{\text {q}}} U_{{\text {t}}}\) for the underdamped dynamics. The shaded region in panels (a) and (o) show the assigned boundary conditions (108) and (109) respectively: the initial state at \({\text {t}}= 0\) is a single peaked distribution centered at \(x_{\mathfrak {o}} = 1\) and the final is a double peaked distribution with peaks at \(-1\) and 1. We set \({\text {t}}_{\mathfrak {f}}= 5\), \(\varepsilon =0.2\), \(g=0\), \(\omega = 1\) and \(\alpha = \sqrt{(1+g)\,A}\approx 0.64\), where A is as defined in (98a). The functions \(\rho _{{\text {t}}_{2}}({\text {q}})\) and \(\sigma _{{\text {t}}_{2}}({\text {q}})\) are computed as in Fig. (6), and predictions for the underdamped are computed using the expressions in 6.1. All distributions are normalized
Predictions for the heights of the peaks of the marginal distribution of the position in the overdamped (orange) and underdamped (blue) dynamics. Both peaks in the overdamped remain at roughly equal height, while the underdamped diverge. The distribution in the overdamped and the corrections to obtain the underdamped are computed as in Fig. 7, using \(\text {t}_{\mathfrak {f}} = 5\), \(\varepsilon = 0.2\), and \(g=0\)
9 Conclusions and Outlook
In this paper, we address the problem of finding optimal control protocols analytically for finite time stochastic thermodynamic transitions described by underdamped dynamics. To such end, we introduce a multiscale expansion whose order parameter vanishes in the overdamped limit. Within second order accuracy, we are able to find corrections for the linear and quadratic moments of the process. When the boundary conditions are Gaussian, our results are in excellent agreement with the solutions found by non-perturbative numerical methods.
We expect our theoretical predictions to provide a necessary benchmark for design and interpretation of experiments on nanomachine thermodynamics. In particular, this is the case for statistical indicators of the momentum process, whose dynamical properties are a distinctive trait of the underdamped regime. Our predictions for the momentum variance and the position-momentum cross correlation are in qualitative agreement with the very recent experimental observations in related laboratory setups [40].
We envisage several directions to extend the present work. In our view, the most urgent and possibly relevant for applications is devising efficient numerical algorithms to determine regular extremals for general (non-Gaussian) boundary conditions. The non-local nature of the equations determining the regular extremals hamper the direct application of proximal algorithms [84, 100] and Monte Carlo methods. We address the problem of generalizing these methods to the underdamped case in a companion contribution [107]. Here, we also compute inertial corrections to the numerical solution of the overdamped problem [31] for minimal entropy production in Landauer’s problem [28].
A second main result of the present work is the proof that the optimal control for transitions between Gaussian states solve a Lyapunov equation in any number of dimensions. This is a strong indication of the existence of regular extremals in phase spaces of any number of dimensions: in view of [66], the extension of the multiscale method is very cumbersome, but otherwise conceptually straightforward. A more subtle issue is instead the computation of corrections of orders higher than two, which are prone to instabilities already at third order. Ideas motivated by normal form theory [103] offer a promising way to overcome this difficulty. Yet, the application to optimal control on a finite time horizon is still an open challenge.
From the physics perspective, the multiscale expansion appears best suited to deal with nanoscale dynamics when inertial effects are present, but are small in comparison to thermal fluctuations. A possible alternative approach is the underdamped expansion (see e.g. Chapter 6 of [76]). This technique could be used to extract complementary information to that obtained here.
In terms of applications, our results are relevant for all physical contexts where random fluctuations and inertial effects cannot be disregarded. This is the case, for example, in bit manipulation in electronic devices. Information bits are encoded using bi-stable states governed by double-well potentials. Inertia is required to improve the efficiency of most logic operations [44].
Our results find natural applications also in biophysics. The control of biological systems such as bacteria suspensions and swarms is nowadays accessible to experimentation through several techniques [108,109,110,111]. This has generated increasing interest in the theoretical challenge of applying control theory to active matter models, i.e. out-of-equilibrium dynamics showing complex phenomena inspired by biology [26, 112,113,114]. So far, however, only overdamped dynamics have been considered. While this does describe the behaviour of microscopic biological systems at high Reynolds numbers (e.g., bacteria in liquid suspensions) fairly, it is well known that inertial effects do play a fundamental role in some classes of such systems [115,116,117]. A meaningful description of the collective behaviour of flocks and swarms requires taking into account inertial effects that allow efficient propagation of information within the system [118,119,120]. Any approach to the control of these models should therefore be carried out in the underdamped regime: even if our results cannot be straightforwardly applied to collective dynamics, they may provide a promising starting point for the development of control theory in this context.
Data Availibility
Data sets generated during the current study are available from the corresponding author on reasonable request.
References
Schrödinger, E.: Über die Umkehrung der Naturgesetze. Sitzungsberichte der preussischen Akademie der Wissenschaften, physikalische mathematische Klasse 8(9), 144–153 (1931). https://doi.org/10.1002/ange.19310443014
Chetrite, R., Muratore-Ginanneschi, P., Schwieger, K.: E. Schrödinger’s 1931 paper “On the Reversal of the Laws of Nature’’ [“Über die Umkehrung der Naturgesetze’’, Sitzungsberichte der preussischen Akademie der Wissenschaften, physikalisch-mathematische Klasse, 8 N9 144-153]. Eur. Phys. J. 46(1), 1–29 (2021). https://doi.org/10.1140/epjh/s13129-021-00032-7
Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Telecommunications and Signal Processing, p. 776. Wiley-Blackwell, Hoboken (2006). https://doi.org/10.1002/0471200611
Föllmer, H.: École d’Étè de Probabilitès de Saint-Flour XV-XVII. In: Hennequin, P.L. (ed.) Random Fields and Diffusion Processes. Springer, New York (1988). https://doi.org/10.1007/BFb0086180
Dai Pra, P.: A stochastic control approach to reciprocal diffusion processes. Appl. Math. Optim. 23(1), 313–329 (1991). https://doi.org/10.1007/BF01442404
Föllmer, H., Gantert, N.: Entropy minimization and Schrödinger processes in infinite dimensions. Ann. Probab. 25(2), 901–926 (1997)
Léonard, C.: A survey of the Schrödinger problem and some of its connections with optimal transport. Discrete Contin. Dyn.l Syst. Ser. A 34(4), 1533–1574 (2014). https://doi.org/10.3934/dcds.2014.34.1533. arXiv:1308.0215 [math.PR]
Chen, Y., Georgiou, T.T., Pavon, M.: Stochastic control liaisons: Richard Sinkhorn meets gaspard monge on a Schrödinger bridge. SIAM Rev. 2, 249–313 (2021). https://doi.org/10.1137/20m1339982
Todorov, E.: Optimality principles in sensorimotor control. Nat. Neurosci. 7(9), 907–915 (2004). https://doi.org/10.1038/nn1309
Todorov, E.: Efficient computation of optimal actions. Proc. Nat. Acad. Sci. 106(28), 11478–11483 (2009). https://doi.org/10.1073/pnas.0710743106
Peyré, G., Cuturi, M.: Computational optimal transport: with applications to data science. Found. Trends Mach. Learn. 5–6, 355–607 (2019). https://doi.org/10.1561/2200000073. arXiv: 1803.00567
De Bortoli, V., Thornton, J., Heng, J., Doucet, A.: Diffusion schrödinger bridge with applications to score-based generative modeling. NeurIPS 2021 (spotlight) and arXiv: 2106.01357 (2021)
Vargas, F., Ovsianas, A., Fernandes, D., Girolami, M., Lawrence, N.D., Nüsken, N.: Bayesian Learning via Neural Schrödinger-Föllmer Flows. arXiv:2111.10510 (2021)
Kay, E.R., Leigh, D.A., Zerbetto, F.: Synthetic molecular motors and mechanical machines. Angew. Chem. Int. Ed. 46(1–2), 72–191 (2007). https://doi.org/10.1002/anie.200504313
Filliger, R., Hongler, M.-O.: Relative entropy and efficiency measure for diffusion-mediated transport processes. J. Phys. A 38, 1247–1255 (2005). https://doi.org/10.1088/0305-4470/38/6/005
Peliti, L., Pigolotti, S.: Stochastic Thermodynamics. Princeton University Press, Princeton (2020)
Maes, C.: The fluctuation theorem as a Gibbs property. J. Stat. Phys. 95, 367–392 (1999). https://doi.org/10.1023/A:1004541830999. arXiv: math-ph/9812015
Maes, C., Redig, F., Moffaert, A.V.: On the definition of entropy production, via examples. J. Stat. Phys. 41(3), 1528–1554 (2000). https://doi.org/10.1063/1.533195
Chetrite, R., Gawȩdzki, K.: Fluctuation relations for diffusion processes. Commun. Math. Phys. 282(2), 469–518 (2008). https://doi.org/10.1007/s00220-008-0502-9. arXiv:0707.2725 [math-ph]
Schmiedl, T., Seifert, U.: Optimal finite-time processes in stochastic thermodynamics. Phys. Rev. Lett. 98, 108301 (2007). https://doi.org/10.1103/PhysRevLett.98.108301
Gomez-Marin, A., Schmiedl, T., Seifert, U.: Optimal protocols for minimal work processes in underdamped stochastic thermodynamics. J. Chem. Phys. 129(2), 024114 (2008). https://doi.org/10.1063/1.2948948. arXiv:0803.0269 [cond-mat.stat-mech]
Esposito, M., Broeck, C.V.: Second law and Landauer principle far from equilibrium. Europhys. Lett. 95(4), 40004 (2011). https://doi.org/10.1209/0295-5075/95/40004. arXiv: 1104.5165
Aurell, E., Mejía-Monasterio, C., Muratore-Ginanneschi, P.: Optimal protocols and optimal transport in stochastic thermodynamics. Phys. Rev. Lett. 106(25), 250601 (2011). https://doi.org/10.1103/PhysRevLett.106.250601. arXiv: 1012.2037
Sivak, D.A., Crooks, G.E.: Thermodynamic metrics and optimal paths. Phys. Rev. Lett. 108(19), 190602 (2012). https://doi.org/10.1103/PhysRevLett.108.190602. arXiv:1201.4166 [cond-mat.stat-mech]
Rotskoff, G.M., Crooks, G.E., Vanden-Eijnden, E.: Geometric approach to optimal nonequilibrium control: minimizing dissipation in nanomagnetic spin systems. Phys. Rev. E 95(1), 012148 (2017). https://doi.org/10.1103/PhysRevE.95.012148. arXiv:1607.07425 [cond-mat.stat-mech]
Baldovin, M., Guéry-Odelin, D., Trizac, E.: Control of active Brownian particles: an exact solution. Phys. Rev. Lett. 131, 118302 (2023). https://doi.org/10.1103/PhysRevLett.131.118302
Chennakesavalu, S., Rotskoff, G.M.: Unified, geometric framework for nonequilibrium protocol optimization. Phys. Rev. Lett. 10, 130 (2023). https://doi.org/10.1103/physrevlett.130.107101
Aurell, E., Gawȩdzki, K., Mejía-Monasterio, C., Mohayaee, R., Muratore-Ginanneschi, P.: Refined second law of thermodynamics for fast random processes. J. Stat. Phys. 147(3), 487–505 (2012). https://doi.org/10.1007/s10955-012-0478-x. arXiv:1201.3207 [cond-mat.stat-mech]
Gawȩdzki, K.: Fluctuation Relations in Stochastic Thermodynamics. Lecture notes, arXiv.org:1308.1518 (2013)
Villani, C.: Optimal Transport: Old and New. Grundlehren der mathematischen Wissenschaften, vol. 338, p. 973. Springer, Berlin (2009)
Brenier, Y., Frisch, U., Hénon, M., Loeper, G., Matarrese, S., Mohayaee, R., Sobolevskiǐ, A.: Reconstruction of the early Universe as a convex optimization problem. Monthly Not. R. Astronom. Soc. 346(2), 501–524 (2003). https://doi.org/10.1046/j.1365-2966.2003.07106.x. arXiv:astro-ph/0304214 [astro-ph]
Muratore-Ginanneschi, P., Mejía-Monasterio, C., Peliti, L.: Heat release by controlled continuous-time Markov jump processes. J. Stat. Phys. 150(1), 181–203 (2013). https://doi.org/10.1007/s10955-012-0676-6. arXiv:1203.4062 [cond-mat.stat-mech]
Muratore-Ginanneschi, P.: (2014) On extremals of the entropy production by “Langevin–Kramers’’ dynamics. J. Stat. Mech. 5, 05013 (2014). https://doi.org/10.1088/1742-5468/2014/05/p05013. arXiv:1401.3394 [cond-mat.stat-mech]
Muratore-Ginanneschi, P., Schwieger, K.: How nanomechanical systems can minimize dissipation. Phys. Rev. E 90(6), 060102 (2014). https://doi.org/10.1103/PhysRevE.90.060102. arXiv:1408.5298 [cond-mat.stat-mech]
Shiraishi, N., Funo, K., Saito, K.: Speed limit for classical stochastic processes. Phys. Rev. Lett. 121(7), 070601 (2018). https://doi.org/10.1103/PhysRevLett.121.070601. arXiv: 1802.06554
Remlein, B., Seifert, U.: Optimality of nonconservative driving for finite-time processes with discrete states. Phys. Rev. E 103, 5 (2021). https://doi.org/10.1103/physreve.103.l050105
Martínez, I.A., Roldán, E., Dinis, L., Petrov, D., Parrondo, J.M.R., Rica, R.A.: Brownian carnot engine. Nat. Phys. 12, 67–70 (2016). https://doi.org/10.1038/nphys3518. arXiv:1412.1282 [cond-mat.stat-mech]
Dinis, L., Martínez, I.A., Roldán, E., Parrondo, J.M.R., Rica, R.A.: Thermodynamics at the microscale: from effective heating to the Brownian Carnot engine. J. Stat. Mech. , 2016 (2016). https://doi.org/10.1088/1742-5468/2016/05/054003
Guéry-Odelin, D., Ruschhaupt, A., Kiely, A., Torrontegui, E., Martínez-Garaot, S., Muga, J.G.: Shortcuts to adiabaticity: concepts, methods, and applications. Rev. Modern Phys. 91, 045001 (2019). https://doi.org/10.1103/RevModPhys.91.045001
Raynal, D., Guillebon, T., Guéry-Odelin, D., Trizac, E., Lauret, J.-S., Rondin, L.: Shortcuts to equilibrium with a levitated particle in the underdamped regime. Phys. Rev. Lett. 131, 087101 (2023). https://doi.org/10.1103/PhysRevLett.131.087101. arXiv: 2303.09542
Plata, C.A., Prados, A., Trizac, E., Guéry-Odelin, D.: Taming the time evolution in overdamped systems: shortcuts elaborated from fast-forward and time-reversed protocols. Phys. Rev. Lett. 127, 190605 (2021). https://doi.org/10.1103/PhysRevLett.127.190605
Baldovin, M., Guéry-Odelin, D., Trizac, E.: Shortcuts to adiabaticity for Lévy processes in harmonic traps. Phys. Rev. E 106, 054122 (2022). https://doi.org/10.1103/PhysRevE.106.054122
Guéry-Odelin, D., Jarzynski, C., Plata, C.A., Prados, A., Trizac, E.: Driving rapidly while remaining in control: classical shortcuts from Hamiltonian to stochastic dynamics. Rep. Progress Phys. 86(3), 035902 (2023)
López-Suárez, M., Neri, I., Gammaitoni, L.: Sub-k bt micro-electromechanical irreversible logic gate. Nat. Commun. 7(1), 12068 (2016). https://doi.org/10.1038/ncomms12068
Deshpande, A., Gopalkrishnan, M., Ouldridge, T.E., Jones, N.S.: Designing the optimal bit: balancing energetic cost, speed and reliability. Proc. R. Soc. A 473(2204), 20170117 (2017). https://doi.org/10.1098/rspa.2017.0117
Ciampini, M.A., Wenzl, T., Konopik, M., Thalhammer, G., Aspelmeyer, M., Lutz, E., Kiesel, N.: Experimental nonequilibrium memory erasure beyond Landauer’s bound (2021)
Lent, C.S., Anderson, N.G., Sagawa, T., Porod, W., Ciliberto, S., Lutz, E., Orlov, A.O., Hänninen, I.K., Campos-Aguillón, C.O., Celis-Cordova, R., McConnell, M.S., Szakmany, G.P., Thorpe, C.C., Appleton, B.T., Boechler, G.P., Snider, G.L.: Energy Limits in Computation. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-93458-7
Ray, K.J., Boyd, A.B., Wimsatt, G.W., Crutchfield, J.P.: Non-markovian momentum computing: Thermodynamically efficient and computation universal. Phys. Rev. Res. 3, 023164 (2021). https://doi.org/10.1103/PhysRevResearch.3.023164
Dago, S., Pereda, J., Barros, N., Ciliberto, S., Bellon, L.: Information and thermodynamics: fast and precise approach to Landauer’s bound in an underdamped micromechanical oscillator. Phys. Rev. Lett. 126, 170601 (2021). https://doi.org/10.1103/PhysRevLett.126.170601
Proesmans, K., Ehrich, J., Bechhoefer, J.: Finite-time Landauer principle. Phys. Rev. Lett. 125(10), 100602 (2020). https://doi.org/10.1103/physrevlett.125.100602. arXiv: 2006.03242
Proesmans, K., Ehrich, J., Bechhoefer, J.: Optimal finite-time bit erasure under full control. Phys. Rev. E 3, 102 (2020). https://doi.org/10.1103/physreve.102.032105
Zhen, Y.Z., Egloff, D., Modi, K., Dahlsten, O.: Universal bound on energy cost of bit reset in finite time. Phys. Rev. Lett. 19, 127 (2021). https://doi.org/10.1103/physrevlett.127.190602
Gonzalez-Ballestero, C., Aspelmeyer, M., Novotny, L., Quidant, R., Romero-Isart, O.: Levitodynamics: Levitation and control of microscopic objects in vacuum. Science 6564, 374 (2021). https://doi.org/10.1126/science.abg3027
Dago, S., Bellon, L.: Dynamics of information erasure and extension of Landauer’s bound to fast processes. Phys. Rev. Lett. 128, 070604 (2022). https://doi.org/10.1103/PhysRevLett.128.070604
Dago, S., Pereda, J., Ciliberto, S., Bellon, L.: Virtual double-well potential for an underdamped oscillator created by a feedback loop. J. Stat. Mech. 5, 202 (2023)
Mikami, T.: Monge’s problem with a quadratic cost by the zero-noise limit of h -path processes. Probab. Theory Relat. Fields 129(2), 245–260 (2004). https://doi.org/10.1007/s00440-004-0340-4
Muratore-Ginanneschi, P.: On the use of stochastic differential geometry for non-equilibrium thermodynamics modeling and control. J. Phys. A 46(27), 275002 (2013). https://doi.org/10.1088/1751-8113/46/27/275002
Chen, Y., Georgiou, T.T., Pavon, M.: On the relation between optimal transport and Schrödinger bridges: a stochastic control viewpoint. J. Optim. Theory Appl. 169(2), 671–691 (2016). https://doi.org/10.1007/s10957-015-0803-z
Cuccoli, A., Fubini, A., Tognetti, V., Vaia, R.: Quantum thermodynamics of systems with anomalous dissipative coupling. Phys. Rev. E 6, 64 (2001). https://doi.org/10.1103/physreve.64.066124
Ankerhold, J., Pollak, E.: Dissipation can enhance quantum effects. Phys. Rev. E 4, 75 (2007). https://doi.org/10.1103/physreve.75.041103
Bonilla, L.L., Carrillo, J.A., Soler, J.S.: Asymptotic behavior of an initial-boundary value problem for the Vlasov–Poisson–Fokker–Planck System. SIAM J. Appl. Math. 57(5), 1343–1372 (1997). https://doi.org/10.1137/S0036139995291544
Bhatia, R., Elsner, L.: Positive linear maps and the lyapunov equation. In: Linear Operators and Matrices, pp. 107–120. Birkhäuser, Basel (2002). https://doi.org/10.1007/978-3-0348-8181-4_9
Verhulst, F.: Methods and Applications of Singular Perturbations: Boundary Layers and Multiple Timescale Dynamics. Texts in Applied Mathematics, Springer, New York (2005). https://doi.org/10.1007/0-387-28313-7
Amit, D.J., Martin-Mayor, V.: Field Theory, the Renormalization Group, and Critical Phenomena, 3rd edn. In: International series in pure and applied physics, p. 568. World Scientific Publishing, Singapore (2005). https://doi.org/10.1142/5715
Wycoff, D., Bálazs, N.L.: Multiple time scales analysis for the Kramers–Chandrasekhar equation. Physica A 146(1–2), 175–200 (1987). https://doi.org/10.1016/0378-4371(87)90227-5
Wycoff, D., Bálazs, N.L.: Multiple time scales analysis for the Kramers–Chandrasekhar equation with a weak magnetic field. Physica A 146(1–2), 201–218 (1987). https://doi.org/10.1016/0378-4371(87)90228-7
Gentile, G.: Quasi-periodic motions in dynamical systems. Review of a Renormalisation group approach. J. Math. Phys. 51, 015207 (2010). https://doi.org/10.1063/1.3271653. arXiv:0910.0755 [math.DS]
Chiarini, A., Conforti, G., Greco, G., Ren, Z.: Entropic turnpike estimates for the kinetic Schrödinger problem. eprint arXiv:2108.09161 [math.PR] (2021)
Muratore-Ginanneschi, P., Peliti, L.: Classical uncertainty relations and entropy production in non-equilibrium statistical mechanics. J. Stat. Mech. 8, 083202083202 (2023). https://doi.org/10.1088/1742-5468/ace3b3
Lapolla, A., Godec, A.: Faster uphill relaxation in thermodynamically equidistant temperature quenches. Phys. Rev. Lett. 11, 125 (2020). https://doi.org/10.1103/physrevlett.125.110602
Ibáñez, M., Dieball, C., Lasanta, A., Godec, A., Rica, R.A.: Heating and cooling are fundamentally asymmetric and evolve along distinct pathways. Nat. Phys. 20(1), 135–141 (2024). https://doi.org/10.1038/s41567-023-02269-z. arXiv:2302.09061
Pavliotis, G.A., Stuart, A.M.: Multiscale Methods: Averaging and Homogenization. Texts in Applied Mathematics, p. 307. Springer, New York (2008)
Schmiedl, T., Seifert, U.: Efficiency of molecular motors at maximum power. EPL (Europhys. Lett.) 83(3), 30005 (2008). https://doi.org/10.1209/0295-5075/83/30005. arXiv:0801.3743 [cond-mat.stat-mech]
Muratore-Ginanneschi, P., Schwieger, K.: Efficient protocols for Stirling heat engines at the micro-scale. EPL (Europhys. Lett.) 112, 20002 (2015). https://doi.org/10.1209/0295-5075/112/20002. arXiv:1503.05788 [cond-mat.stat-mech]
Dechant, A., Kiesel, N., Lutz, E.: Underdamped stochastic heat engine at maximum efficiency. EPL (Europhys. Lett.) 119, 50003 (2017). https://doi.org/10.1209/0295-5075/119/50003
Pavliotis, G.A.: Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations. Springer, New York (2014). https://doi.org/10.1007/978-1-4939-1323-7
Caldeira, A.O., Leggett, A.J.: Quantum tunnelling in a dissipative system. Ann. Phys. 149(2), 374–456 (1983). https://doi.org/10.1016/0003-4916(83)90202-6
Maile, D., Andergassen, S., Rastelli, G.: Effects of a dissipative coupling to the momentum of a particle in a double well potential. Phys. Rev. Res. 1, 2 (2020). https://doi.org/10.1103/physrevresearch.2.013226
Zwanzig, R.: Nonlinear generalized Langevin equations. J. Stat. Phys. 9(3), 215–220 (1973). https://doi.org/10.1007/bf01008729
Conforti, G., Ripani, L.: Around the entropic Talagrand inequality. Bernoulli 26, 1431–1452 (2020). https://doi.org/10.48550/ARXIV.1809.02062
Theodorou, E.A., Todorov, E.: Relative entropy and free energy dualities: Connections to Path Integral and Kullback–Leibler control. In: Annual Conference on Decision and Control (CDC), 2012 IEEE 51st, pp. 1466–1473 (2012). https://doi.org/10.1109/CDC.2012.6426381
Boscain, U., Sigalotti, M., Sugny, D.: Introduction to the pontryagin maximum principle for quantum optimal control. PRX Quantum 3, 2 (2021). https://doi.org/10.1103/prxquantum.2.030203
Rey-Bellet, L.: Ergodic Properties of Markov Processes. In: Quantum Open Systems II. The Markovian Approach. Lecture Notes in Mathematics, pp. 1–39. Springer, Berlin (2006)
Caluya, K.F., Halder, A.: Wasserstein proximal algorithms for the Schrödinger bridge problem: density control with nonlinear drift. IEEE Trans. Automatic Control 3, 67 (2022). https://doi.org/10.1109/TAC.2021.3060704
Talagrand, M.: Transportation cost for Gaussian and other product measures. Geometric Funct. Anal. 6(3), 587–600 (1996). https://doi.org/10.1007/bf02249265
Otto, F., Villani, C.: Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. J. Funct. Anal. 173(2), 361–400 (2000). https://doi.org/10.1006/jfan.1999.3557
Léonard, C.: From the Schrödinger problem to the Monge–Kantorovich problem. J. Funct. Anal. 262(4), 1879–1920 (2012). https://doi.org/10.1016/j.jfa.2011.11.026. arXiv:1011.2564 [math.OC]
Nelson, E.: Dynamical Theories of Brownian Motion, 2nd edn., p. 148. Princeton University Press, Princeton (2001). https://doi.org/10.2307/j.ctv15r57jg
Benamou, J.-D., Brenier, Y.: A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numerische Mathematik 84(3), 375–393 (2000). https://doi.org/10.1007/s002110050002
Dechant, A., Sasa, S.I.: Entropic bounds on currents in Langevin systems. Phys. Rev. 6, 97 (2018). https://doi.org/10.1103/physreve.97.062101
Gawedzki, K.: Improved 2nd Law of Stochastic Thermodynamics for underdamped Langevin process. Note written for Salambô Dago in July 2020, and communicated by the author to Erik Aurell (2021)
Mikami, T., Thieullen, M.: Duality theorem for stochastic optimal control problem. Stoch. Process. Appl. 116, 1815–1835 (2006)
Gentil, I., Léonard, C., Ripani, L.: About the analogy between optimal transport and minimal entropy. Ann. Facult. Sci. Toulouse 26(3), 569–600 (2017). https://doi.org/10.5802/afst.1546
Serrin, J.: Mathematical principles of classical fluid mechanics. In: Fluid Dynamics I / Strömungsmechanik I. Encyclopedia of Physics / Handbuch der Physik, vol. 3 / 8 / 1, pp. 125–263. Springer, Berlin, Heidelberg (1959). https://doi.org/10.1007/978-3-642-45914-6_2
Seliger, R.L., Whitham, G.B.: Variational principles in continuum mechanics. Proc. R. Soc. A 305(1480), 1–25 (1968). https://doi.org/10.1098/rspa.1968.0103
Bismut, J.M.: An introduction to duality in random mechanics. In: Kohlmann, M., Vogel, W. (eds.) Stochastic Control Theory and Stochastic Differential Systems. Lecture Notes in Control and Information Sciences, pp. 42–60. (1979). https://doi.org/10.1007/BFb0009375
Bechhoefer, J.: Control Theory for Physicists. Cambridge University Press, Cambridge (2021). https://doi.org/10.1017/9780511734809
Liberzon, D.: Calculus of Variations and Optimal Control Theory. A Concise Introduction. Princeton University Press, Princeton (2012)
Chen, Y., Georgiou, T.T., Pavon, M.: Fast cooling for a system of stochastic oscillators. J. Math. Phys. 11, 56 (2015). https://doi.org/10.1063/1.4935435
Caluya, K.F., Halder, A.: Gradient flow algorithms for density propagation in stochastic systems. IEEE Trans. Autom. Control 65(10), 3991–4004 (2020). https://doi.org/10.1109/tac.2019.2951348
Hayes, M., Kaper, T.J., Kopell, N., Ono, K.: On the application of geometric singular perturbation theory to some classical two point boundary value problems. Int. J. Bifurcat. Chaos 08(02), 189–209 (1998). https://doi.org/10.1142/s0218127498000140
Lehec, J.: Representation formula for the entropy and functional inequalities. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques 3, 49 (2013). https://doi.org/10.1214/11-aihp464
Bobylev, A.V.: Instabilities in the Chapman–Enskog expansion and hyperbolic Burnett equations. J. Stat. Phys. 124(2–4), 371–399 (2006). https://doi.org/10.1007/s10955-005-8087-6
Rackauckas, C., Nie, O.: Differentialequations.jl-: a performant and feature-rich ecosystem for solving differential equations in julia. J. Open Res. Softw. 1, 15 (2017)
Bérut, A., Arakelyan, A., Petrosyan, A., Ciliberto, S., Dillenschneider, R., Lutz, E.: Experimental verification of Landauer’s principle linking information and thermodynamics. Nature 483, 187–189 (2012). https://doi.org/10.1038/nature10872
Martínez, I.A., Petrosyan, A., Guéry-Odelin, D., Trizac, E., Ciliberto, S.: Engineered swift equilibration of a Brownian particle. Nat. Phys. 12, 843–846 (2016). https://doi.org/10.1038/nphys3758
Sanders, J., Baldovin, M., Muratore-Ginanneschi, P.: Minimal-work protocols for inertial particles in non-harmonic traps. Eprint arXiv:2407.15678 (2024) https://doi.org/10.48550/ARXIV.2407.15678
Sipos, O., Nagy, K., Di Leonardo, R., Galajda, P.: Hydrodynamic trapping of swimming bacteria by convex walls. Phys. Rev. Lett. 114, 258104 (2015). https://doi.org/10.1103/PhysRevLett.114.258104
Peng, C., Turiv, T., Guo, Y., Wei, Q.-H., Lavrentovich, O.D.: Command of active matter by topological defects and patterns. Science 354(6314), 882–885 (2016). https://doi.org/10.1126/science.aah6936
Cavagna, A., Giardina, I., Gucciardino, M.A., Iacomelli, G., Lombardi, M., Melillo, S., Monacchia, G., Parisi, L., Peirce, M.J., Spaccapelo, R.: Characterization of lab-based swarms of anopheles Gambiae mosquitoes using 3d-video tracking. Sci. Rep. 13(1), 8745 (2023)
Pellicciotta, N., Paoluzzi, M., Buonomo, D., Frangipane, G., Angelani, L., Di Leonardo, R.: Colloidal transport by light induced gradients of active pressure. Nat. Commun. 1, 14 (2023). https://doi.org/10.1038/s41467-023-39974-5
Shankar, S., Raju, V., Mahadevan, L.: Optimal transport and control of active drops. Proc. Nat. Acad. Sci. 35, 119 (2022). https://doi.org/10.1073/pnas.2121985119
Davis, L.K., Proesmans, K., Fodor, E.: Active matter under control: insights from response theory. Phys. Rev. X 14, 011012 (2024). https://doi.org/10.1103/PhysRevX.14.011012
Frim, A.G., DeWeese, M.R.: Shortcut engineering of active matter: run-and-tumble particles (2023)
Manacorda, A., Puglisi, A.: Lattice model to derive the fluctuating hydrodynamics of active particles with inertia. Phys. Rev. Lett. 119, 208003 (2017). https://doi.org/10.1103/PhysRevLett.119.208003
Scholz, C., Jahanshahi, S., Ldov, A., Löwen, H.: Inertial delay of self-propelled particles. Nat. Commun. 1, 9 (2018). https://doi.org/10.1038/s41467-018-07596-x
Löwen, H.: Inertial effects of self-propelled particles: from active Brownian to active Langevin motion. J. Chem. Phys. 152(4), 040901 (2019)
Attanasi, A., Cavagna, A., Del Castello, L., Giardina, I., Grigera, T.S., Jelić, A., Melillo, S., Parisi, L., Pohl, O., Shen, E., Viale, M.: Information transfer and behavioural inertia in starling flocks. Nat. Phys. 10(9), 691–696 (2014). https://doi.org/10.1038/nphys3035
Cavagna, A., Del Castello, L., Giardina, I., Grigera, T., Jelic, A., Melillo, S., Mora, T., Parisi, L., Silvestri, E., Viale, M., Walczak, A.M.: Flocking and turning: a new model for self-organized collective motion. J. Stat. Phys. 158(3), 601–627 (2014). https://doi.org/10.1007/s10955-014-1119-3
Cavagna, A., Conti, D., Creato, C., Del Castello, L., Giardina, I., Grigera, T.S., Melillo, S., Parisi, L., Viale, M.: Dynamic scaling in natural swarms. Nat. Phys. 13(9), 914–918 (2017). https://doi.org/10.1038/nphys4153
Acknowledgements
The authors are pleased to acknowledge discussions with Luca Peliti and Paolo Erdman. JS was supported by the Centre of Excellence in Randomness and Structures of the Academy of Finland and by a University of Helsinki funded doctoral researcher position, Doctoral Programme in Mathematics and Statistics. MB was supported by ERC Advanced Grant RG.BIO (Contract No. 785932).
Funding
Open access funding provided by Consiglio Nazionale Delle Ricerche (CNR) within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Communicated by Udo Seifert.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendices
Derivation of the Cost Functionals
The physics-style derivation of (4) and (5) proceeds by constructing finite dimensional approximation on families of time lattices \({t}_{\iota }\,\le \,t_{0}\,\le \,\dots \,\le \,t_{N+1}={t}_{\mathfrak {f}}\) with mesh size
The one-step approximation of the transition probability density of (1) in the pre-point prescription is
where \(\varvec{x}_{i}=\varvec{q}_{i}\oplus \varvec{p}_{i}\) and
while \(Z_{i}\) is a normalization constant irrelevant for the present considerations. Within accuracy (115) satisfies the Chapman–Kolmogorov equation [76]. Hence, we obtain the transition probability over any finite time interval by means of the limit
where we hold fixed in the limit \(\tilde{t}=s_{0}\,\le \,s_{N+1}=t\) and \(\varvec{z}_{0}=\varvec{\tilde{x}}\), \( \varvec{z}_{N+1}=\varvec{x}\). For any admissible potential, (117) satisfies by hypothesis the bridge boundary conditions
with \( {\text {f}}_{{t}_{\iota }}\), \({\text {f}}_{{t}_{\mathfrak {f}}}\) respectively assigned by (2) and (3).
1.1 Case KL
Proceeding in a similar fashion, the finite dimensional approximation of (4) is by definition
where \({\text {T}}_{\star }^{(h)}\) is defined with respect to the reference potential \(U_{\star }\). Using the properties of the logarithm and the normalization of the transition probability, the definition reduces to the sum
Next, we observe that
The outermost integrals in (119) over \(\varvec{q}_{i+1},\varvec{p}_{i+1}\) are Gaussian and equal to
We thus arrive at
Inserting this result into (119) and passing to the continuum limit recovers (4).
1.2 Case EP
The starting point is (118) where we replace \({\text {T}}_{\star }^{(h)}\) with the transition probability generated by the backward stochastic differential equations
where the label \(\flat \) recalls that the evolution proceeds backwards. The one-step approximation of the transition probability density on the lattice using the adapted post-point prescription yields
with
We recover (5) by contrasting ratios of (115) and (120) over the same time intervals and by identifying the sum of two finite dimensional approximations of stochastic integrals over the same integrand but evaluated in the pre-point and post-point prescription as twice the same integral in the Stratonovich prescription [76]. We refer to [18] for the details of the calculation or, e.g., to [69] a derivation directly in the continuum limit using stochastic calculus and Girsanov formula.
Consistency of the Definition of Mean Entropy Production with Stochastic Thermodynamics
The calculation follows the same steps as [33]. Let
the kinetic plus potential Hamiltonian specified by the control potential in (5). We define the work done on the system during a time interval [0, t] as
Here, \(\varvec{\mathcalligra{x}}_{t}\) ia a realization of (1). The partial derivative only affects the explicit time dependence of the control potential. Correspondingly, we identify the heat released by the system during the same realization with the Stratonovich stochastic integral
so that for any path satisfying (1) the identity
ensures the validity of the first law of thermodynamics. The dynamics (70) describes an open system in contact with an environment at constant temperature \(\beta ^{-1}\). Hence the average heat released by the system in [0, t] also specifies the change of entropy in the environment
The total entropy change is the sum of this quantity and the change of the Gibbs-Shannon entropy of the system [28]
In order to justify referring to (5) as mean entropy production, we need to show how it is related to (122). Indeed, from the properties of the Stratonovich integral
On the right hand side we introduce the current velocity
which is most conveniently written in terms of the \(2\,d \,\times \,2\, d\) real matrices
Probability conservation and antisymmetry of \(\textsf{J}\) imply
and thus allow us to arrive at
From this expression we readily verify that the thermodynamic mean entropy production (123) is positive definite. Next, we unfold the quadratic form in the integrand
Under our working hypothesis that confining mechanical potentials produce probability density decreasing at infinity sufficiently fast, an integration by parts yields
We are therefore in the position to apply the chain of identities
to couch (123) into the form
The last step is to make use of the kinetic plus potential form of the Hamiltonian. Straightforward algebra and an integration by parts yield
Hence, the Kullback–Leibler divergence (5) coincides with the thermodynamic entropy production. The above calculation can also be conceptualized as a special case of the general theory expounded in [19].
Proof of the Mean Entropy Production Lower Bound
It is well known (see e.g. [19, 33]) that (5) can be couched into the explicitly positive form
We add and subtract
in the squared norm of the first integrand in (124) and
to the second one. In both expressions we introduce the position marginal density
Upon expanding the norm squared into inner products, and taking advantage of the cancellation of the mixed term, we get
For any \(g>0\) and a, b arbitrary real numbers, the inequality
holds true. The upshot is
The vector field appearing on the right hand-side of the inequality
is exactly the current velocity transporting the position marginal distribution:
We are therefore in the position to apply the Benamou–Brenier inequality [89]
whence we finally recover (9).
Details of the Expansion in Hermite Polynomials
The Hermite polynomials are defined as
so that
and so on. They fulfill the following orthonormality condition:
Let us notice that from the definition of Hermite polynomials it follows:
and
The above identities, together with decomposition (34), can be used to write
Similarly, one has
and
Substituting the above relations into the Fokker–Planck equation (33a) and projecting onto \(H_n\) we get
which can be recast into Eq. (36a).
A similar approach can be followed for the value function. Recalling Eq. (35) one gets
hence Eq. (36b) follows. Equation (36c) comes from an analogous expansion of Eq. (33c).
Path Integral Proof of the Inequality (69)
We start from the discretized stochastic differential equation
where the label i runs over bins of the time discretization. We take uniform mesh h. b is a sufficiently regular drift, and the \( \eta _{i}\)’s are independent identically distributed centered Gaussian random variables with unit variance. We set out to compute
with
and \( Z_{h}\) a mesh dependent normalization constant. We emphasize the use of the pre-point prescription in our construction of finite dimensional approximations of the path integral. We perform the change of variables
in consequence whereof the chain of identities
holds true. As the change of variables in the pre-point prescription has unit Jacobian, we arrive at
by the Cauchy inequality, with
This inequality holds for any drift and therefore also for the one implementing the optimal bridge.
Further Details on the Order-by-Order Multiscale Expansion
In this Appendix, we present additional details about the order-by-order solution of the multiscale problem presented in Sect. 6.2. While it is not essential to follow the logic of our method, these intermediate steps may be a useful reference for the reader interested in the detailed derivation of the results.
In Sect. 6.2.3, we describe how to obtain a second order differential equation in \({\text {t}}_{0}\) for \({\text {f}}_{{{\text {t}}}_{0}}^{(1:1)}\), namely Eq. (85). The first step is to find a relation for \(v^{(1:1)}_{{{\text {t}}}_{0}}\) by plugging Eq. (84) in Eq. (82). We get
By differentiating in \({\text {t}}_{0}\) and eliminating \(v^{(1:1)}_{{\text {t}}_{0},{{\text {t}}}_{1}}\) and its time derivative through (140) itself and (83), one gets Eq. (85). The explicit expression of its right hand side reads
The dependence of the value function on \({\text {t}}_{1}\) can be actually dropped, since no resonant equation holds for \(v^{(0:0)}_{{{\text {t}}}_{0}}\) and \(v^{(2:0)}_{{{\text {t}}}_{0}}\) on that time scale. This result allows us to find (88) through the use of the Green function (86):
Once an explicit expression for \({\text {f}}_{{\text {t}}_{0};{{\text {t}}}_{2}}^{(1:1)}\) is known (i.e., Eq. (88)), it can be substituted into Eq. (92) to get the relation
where \(C_1\) and \(C_2\) are constants that can be evaluated considering the limits for \({\text {q}}\rightarrow \pm \infty \). One then has
In Sect. 6.2.4 we derive the differential equation (44b), which allows closing the system, providing the \({\text {t}}_{2}\)-dependence of \({\text {f}}_{0;{{\text {t}}}_{2}}^{(0:0)}\). To this aim, one first needs to substitute the expression for \(v^{(1:1)}_{{\text {t}}_{0};{{\text {t}}}_{2}}\) given by Eq. (99), obtaining
where we took into account Eq. (90) and we introduced
and
Because of the boundary conditions (73) one has
Besides, using (36b) at order 0,
hence
We also notice, recalling Eq. (85), that
where \(F_{{{\text {t}}}_{0}}\) obeys Eq. (141). Taking into account Eqs. (90), (92), (144), (145) and (146), one obtains from Eq. (143)
Now we can substitute the explicit expressions for \({\text {f}}_{{\text {t}}_{0};{{\text {t}}}_{2}}^{(1:1)}\) and \(\partial _{{\text {q}}}\left( {\text {f}}_{0;{{\text {t}}}_{2}}^{(0:0)}v_{{\text {t}}_{\mathfrak {f}};{{\text {t}}}_{2}}^{(0:0)}\right) \) provided by Eqs. (88) and (142). The integrals are evaluated by making repeated use of Eq. (94). We arrive at
By recalling Eq. (97), we obtain Eq. (100). It is now possible to compute the time derivative of \(\kappa _{{{\text {t}}}_{2}}\), by making use of (97) and (100):
The right hand side vanishes for case EP, and also for case KL when either \(U_{\star }\) is a linear function of \({\text {q}}\) (including the physically relevant case \(U_{\star }=0\)), or \(U_{\star }\) is symmetric with symmetric boundary conditions. By taking into account this result and the definition (101), Eq. (100) is straightforwardly recast into Eq. (44b).
We finally observe that, recalling (51) and (102), one has
Therefore,
when the right hand side of (149) vanishes.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Sanders, J., Baldovin, M. & Muratore-Ginanneschi, P. Optimal Control of Underdamped Systems: An Analytic Approach. J Stat Phys 191, 117 (2024). https://doi.org/10.1007/s10955-024-03320-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10955-024-03320-w