Maximum Principle and Sensitivity Relations in the Infinite Horizon Optimal Control H´el`ene Frankowska ´ PIERRE et MARIE CURIE CNRS and UNIVERSITE

In collaboration with Piermarco Cannarsa INdAM Workshop - New Trends in Control Theory and PDEs On the occasion of the 60th birthday of Piermarco Cannarsa July 3-7, 2017 H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Long Lasting Collaboration and Friendship (1989) Two characterisations of optimal trajectories for the Mayer problem, IFAC symposium Nonlinear Control Systems Design (1990) Quelques characterisations des trajectoires optimales dans la th´ eorie de contrˆ ole, Note CRAS (1990) Some characterizations of optimal trajectories in control theory, Proceedings of 29th CDC Conference (1991) Some characterizations of optimal trajectories in control theory, SICON (1992) Value function and optimality conditions for semilinear control problems, AMO (1996) On the value function of semilinear optimal control problems of parabolic type, II, AMO (Plant and Soil, Agroforestry Systems, Forest Ecology and Management, Field Crops Research) Cannarsa P., Frankowska H. & Sinestrari C. (1998) Properties of minimal time function in nonlinear control theory, JMSEC Cannarsa P., Frankowska H. & Sinestrari C. (2000) Optimality conditions and synthesis for the minimum time problem, SVA (2006) Interior sphere property of attainable sets and time optimal control problems, ESAIM COCV Cannarsa P., Frankowska H. & Marchini E. (2007) Lipschitz continuity of optimal trajectories in deterministic optimal control, View of ODE’s

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Cannarsa P., Frankowska H. & Marchini E. (2009) Existence and Lipschitz regularity of solutions to Bolza problems in optimal control, TAMS Cannarsa P., Frankowska H. & Marchini E. (2009) On Bolza optimal control problems with constraints, DCDS, Series B Cannarsa P., Da Prato G. & Frankowska H. (2010) Invariant measures associated to degenerate elliptic operators, Indiana University Mathematics Journal Cannarsa P., Frankowska H. & Marchini E. (2013) On optimal control problems with applications to systems with memory, JEE (2013) Local regularity of the value function in optimal control, Systems and Control Letters (2014) From pointwise to local regularity for solutions of Hamilton-Jacobi equations, Calculus of Variations and PDEs Cannarsa P., Frankowska H. & Scarinci T. (2014) Sensitivity relations for the Mayer problem with differential inclusions, Proceedings of 53rd CDC Conference Cannarsa P., Frankowska H. & Scarinci T. (2015) Sensitivity relations for the Mayer problem with differential inclusions, ESAIM COCV Cannarsa P., Frankowska H. & Scarinci T. (2016) Second-order sensitivity relations and regularity of the value function for Mayer’s problem in optimal control, SICON Cannarsa P., Da Prato G. & Frankowska H. (2016) Invariance for quasi-dissipative systems in Banach spaces, JMAA

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

(2017) Value function, relaxation, and transversality conditions in infinite horizon optimal control, JMAA (2017) Infinite horizon optimal control: transversality conditions and sensitivity relations, Proceedings of ACC Conference

- Three common European projects (HCM, TMR, ITN Marie Curie) - Two PhD students in co-tutelle (Teresa Scarinci and Vicenzo Basco) - Also Marco Mazzola as a post-doctoral - Many co-organised events, shared friends and co-authors, travels, diners, walks, discussions, confidences.... H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

(2017) Value function, relaxation, and transversality conditions in infinite horizon optimal control, JMAA (2017) Infinite horizon optimal control: transversality conditions and sensitivity relations, Proceedings of ACC Conference

- Three common European projects (HCM, TMR, ITN Marie Curie) - Two PhD students in co-tutelle (Teresa Scarinci and Vicenzo Basco) - Also Marco Mazzola as a post-doctoral - Many co-organised events, shared friends and co-authors, travels, diners, walks, discussions, confidences.... H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

(2017) Value function, relaxation, and transversality conditions in infinite horizon optimal control, JMAA (2017) Infinite horizon optimal control: transversality conditions and sensitivity relations, Proceedings of ACC Conference

- Three common European projects (HCM, TMR, ITN Marie Curie) - Two PhD students in co-tutelle (Teresa Scarinci and Vicenzo Basco) - Also Marco Mazzola as a post-doctoral - Many co-organised events, shared friends and co-authors, travels, diners, walks, discussions, confidences.... H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

(2017) Value function, relaxation, and transversality conditions in infinite horizon optimal control, JMAA (2017) Infinite horizon optimal control: transversality conditions and sensitivity relations, Proceedings of ACC Conference

- Three common European projects (HCM, TMR, ITN Marie Curie) - Two PhD students in co-tutelle (Teresa Scarinci and Vicenzo Basco) - Also Marco Mazzola as a post-doctoral - Many co-organised events, shared friends and co-authors, travels, diners, walks, discussions, confidences.... H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Infinite Horizon Optimal Control Problem Z ∞

V (t0 , x0 ) = inf

L(t, x (t), u(t)) dt t0

over all trajectory-control pairs (x , u), subject to the state equation (

x 0 (t) = f (t, x (t), u(t)), x (t0 ) = x0

u(t) ∈ U(t)

for a.e. t ≥ 0

x0 ∈ Rn , U : R+ ; Rm is a measurable set-valued map with closed 6= ∅ images, L : R+ × Rn × Rm → R, f : R+ × Rn × Rm → Rn . Controls u(t) ∈ U(t) are Lebesgue measurable selections. L is bounded from below by a function integrable on [0, ∞[ . Thus V takes values in (−∞, +∞]. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

A classical infinite horizon optimal control problem Z ∞

W (x0 ) = minimize

e −ρt `(x (t), u(t)) dt

0

over all trajectory-control pairs (x , u), subject to the state equation (

x 0 (t) = f (x (t), u(t)), x (0) = x0

u(t) ∈ U

for a.e. t ≥ 0

controls u(·) are Lebesgue measurable, ρ > 0. The literature addressing this problem deals with traditional questions of existence of optimal solutions, regularity of W , necessary and sufficient optimality conditions. A. Seierstad and K. Sydsaeter. Optimal control theory with economic applications, 1986. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Hamilton-Jacobi Equation ` ≥ 0. Under some technical assumptions W is the unique bounded lower semicontinuous solution with values in R+ of the Hamilton-Jacobi equation ρW (x ) + sup (h∇W (x ), f (x , u)i − `(x , u)) = 0 u∈U

in the following sense ρW (x )+ sup (hp, f (x , u)i − `(x , u)) = 0 ∀ p ∈ ∂ − W (x ), x ∈ Rn u∈U

∂ − W (x )

denotes the subdifferential of W at x . HF and Plaskacz 1999, in the presence of state constraints. If W is Bounded and Uniformly Continuous, then it is also the unique viscosity solution in the set of BUC functions Soner 1986, in the presence of state constraints. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Necessary Optimality Condition: Maximum Principle If (¯ x , u¯) is optimal, then ∃ p0 ∈ {0, 1} and a locally absolutely continuous p : [0, ∞[→ Rn with (p0 , p) 6= 0, solving the adjoint system −p 0 (t) = p(t)fx (t, x¯ (t), u¯(t))−p0 Lx (t, x¯ (t), u¯(t))

for a.e. t ≥ 0

and satisfying the maximality condition hp(t), f (t, x¯ (t), u¯(t))i − p0 L(t, x¯ (t), u¯(t)) = maxu∈U(t) (hp(t), f (t, x¯ (t), u)i − p0 L(t, x¯ (t), u))

for a.e. t ≥ 0

If p0 = 0 this maximum principle (MP) is called abnormal. Transversality condition like limt→∞ p(t) = 0 is, in general, absent, cf. Halkin 1974. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Main Differences with the Finite Horizon Case The maximum principle may be abnormal Transversality conditions are not derived from the cost : some authors, under appropriate assumptions, obtain a transversality condition at infinity in the form lim p(t) = 0 or

t→∞

lim hp(t), x¯ (t)i = 0

t→∞

However they are a consequence of the growth assumptions on f , L Main difficulty behind : Restriction of an optimal solution to a finite time interval is no longer optimal

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Some Approaches to Get Necessary Conditions To work with ”finitely” optimal controls Halkin 1974; Carlson, Haurie 1987. This leads to finite horizon problems with an end-point constraint Penalization and dealing with limits of finite horizon problems Aseev and Kryazhimskiy, 2004 Locally weakly overtaking optimal controls Aseev and Veliov, 2015 An alternative for linear control systems by Aubin and Clarke, 1979 : duality theory on weighted Sobolev spaces Lp (0, ∞; Rn ) with the measure e −ρt dt and, more recently, for more general measures Pickenhain 2010; Tauchnitz 2015 Transversality condition at the initial state −p(0) ∈ ∂W (x0 ) (generalized gradient of W at x0 ), Aubin and Clarke, 1979 Followed by works of Michel 1982; Ye 1993; Sagara 2010 H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Dynamic Programming Principle L is bounded from below by an integrable function, that is L(t, x , u) ≥ α(t) for a.e. t ≥ 0 and all x , u, where α : R+ → R is integrable on [0, +∞[. Thus V takes values in (−∞, +∞]. For every (t0 , x0 ) with V (t0 , x0 ) < ∞ the dynamic programming principle holds true: if (¯ x , u¯) is optimal, then for every T > t0 , Z T

V (t0 , x0 ) = V (T , x¯ (T )) +

L(t, x¯ (t), u¯(t)) dt t0

and for any other trajectory control pair (x , u) V (t0 , x0 ) ≤ V (T , x (T )) +

Z T

L(t, x (t), u(t)) dt t0

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Reduction to the Bolza Problem with Finite Horizon Introducing gT (y ) := V (T , y ) we get, using the dynamic programming principle, the Bolza type problem !

Z T

B

V (t0 , x0 ) := inf gT (x (T )) +

L(t, x (t), u(t)) dt t0

over all trajectory-control pairs (x , u), subject to the state equation (

x 0 (t) = f (t, x (t), u(t)), x (t0 ) = x0

u(t) ∈ U(t)

for a.e. t ∈ [t0 , T ]

gT may be discontinuous and so the (MP) is not immediate. Under assumptions (H) below, V B (s0 , y0 ) = V (s0 , y0 ) for all s0 ∈ [0, T ], y0 ∈ Rn . Furthermore, if (¯ x , u¯) is optimal for the infinite horizon problem at (t0 , x0 ) then the restriction of (¯ x , u¯) to [t0 , T ] is optimal for the above Bolza problem. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Assumptions (H) i) ∃ locally integrable c, θ : R+ → R+ such that for a.e. t ≥ 0 |f (t, x , u)| ≤ c(t)|x | + θ(t),

∀ x ∈ Rn , u ∈ U(t);

ii) ∀ R > 0, ∃ a locally integrable cR : R+ → R+ such that for a.e. t ≥ 0, ∀ x , y ∈ B(0, R), ∀ u ∈ U(t) |f (t, x , u) − f (t, y , u)| + |L(t, x , u) − L(t, y , u)| ≤ cR (t)|x − y |; iii) ∀ x ∈ Rn , f (·, x , ·), L(·, x , ·) are Lebesgue-Borel measurable ; iv) ∃ a locally integrable β : R+ → R+ and a locally bounded nondecreasing φ : R+ → R+ such that for a.e. t ≥ 0, L(t, x , u) ≤ β(t)φ(|x |),

∀ x ∈ Rn , u ∈ U(t);

v) U(·) is Lebesgue measurable and has closed nonempty images; vi) For a.e. t ≥ 0, ∀ x ∈ Rn the set F (t, x ) is closed and convex F (t, x ) :=

f (t, x , u), L(t, x , u) + r : u ∈ U(t) and r ≥ 0 H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Existence For any t0 ∈ R+ , x0 ∈ Rn such that V (t0 , x0 ) < +∞, a trajectory-control pair (¯ x , u¯) is called optimal for the infinite horizon problem at (t0 , x0 ) if for every trajectory-control pair (x , u) satisfying x (t0 ) = x0 we have Z ∞

L(t, x¯ (t), u¯(t)) dt ≤

t0

Z ∞

L(t, x (t), u(t)) dt t0

Proposition Assume (H). Then V is lower semicontinuous and for every (t0 , x0 ) ∈ dom(V ), there exists a trajectory-control pair (¯ x , u¯) R satisfying V (t0 , x0 ) = t∞ L(t, x ¯ (t), u ¯ (t)) dt. 0 H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Relaxation Consider the relaxed infinite horizon problem V

rel

(t0 , x0 ) = inf

Z ∞X n t0

λi (t)L(t, x (t), ui (t)) dt

i=0

over all trajectory-control pairs of Pn 0 u (t)) x (t) = i=0 λi (t)f (t, x (t), P i

u (t) ∈ U(t), λi (t) ≥ 0,

i x (t ) = x , 0 0

n i=0 λi (t)

=1

where ui (·), λi (·) are Lebesgue measurable. Clearly V rel ≤ V . The above corresponds to the convexification of the set F (t, x ) :=

f (t, x , u), L(t, x , u) : u ∈ U(t)

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Setting and Assumptions Existence and Relaxation

Continuity of V rel allows to omit (H) vi) Theorem Assume (H) i)-v) and that, for a.e. t ≥ 0 and all x ∈ Rn , the set

f (t, x , u), L(t, x , u) : u ∈ U(t)

is compact. If for every t ≥ 0, V rel (t, ·) : Rn → R is continuous, then V rel = V on R+ × Rn . In particular, if a trajectory-control pair (¯ x , u¯) is optimal, then it is also optimal for the relaxed problem. The above assumption is verified if U(t) is compact a.e. and f (t, x , ·), L(t, x , ·) are continuous. We introduce the Hamiltonian H(t, x , p) := sup (hp, f (t, x , u)i − L(t, x , u)) u∈U(t) H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Nonsmooth Analysis Lower Semicontinuous Value Function Continuous Value Function Locally Lipschitz Value Function

Generalized differentials and Limiting Normals hyp(ϕ) - hypograph of ϕ : Rn → R ∪ {±∞}. For x ∈ dom(ϕ) ∂ − ϕ(x ) := {p | lim inf y →x

For K ⊂

ϕ(y ) − ϕ(x ) − hp, y − x i ≥ 0} |y − x |

Rn

and x ∈ K dK (x + hv ) TK (x ) := {v | lim inf = 0}, NKL (x ) = Limsupy →K x [TK (y )]− h→0+ h where TK (y )− is the negative polar of TK (y ). Limiting superdifferential resp. horizontal superdifferential: L ∂ L,+ ϕ(x ) := {p | (−p, 1) ∈ Nhyp(ϕ) (x , ϕ(x ))} L ∂ ∞,+ ϕ(x ) := {p | (−p, 0) ∈ Nhyp(ϕ) (x , ϕ(x ))}

If ϕ is loc. Lipschitz, ∂ϕ(x ) - generalized gradient of ϕ at x . For loc. Lipschitz ψ : Rn → Rn , ∂ψ(x ) denotes the generalized Jacobian of ψ at x . H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Nonsmooth Analysis Lower Semicontinuous Value Function Continuous Value Function Locally Lipschitz Value Function

Maximum Principle for LSC Value Function Theorem (normal MP with a sensitivity relation) Let (¯ x , u¯) be optimal at (t0 , x0 ) and ∂x− V (t0 , x0 ) 6= ∅. If f (t, ·, u) and L(t, ·, u) are differentiable, then ∀ p0 ∈ ∂x− V (t0 , x0 ) the solution p(·) of the adjoint system −p 0 (t) = p(t)fx (t, x¯ (t), u¯(t)) − Lx (t, x¯ (t), u¯(t)), p(t0 ) = −p0 satisfies for a.e. t ≥ t0 the maximality condition hp(t), f (t, x¯ (t), u¯(t))i − L(t, x¯ (t), u¯(t)) = H(t, x¯ (t), p(t)) and the sensitivity relation −p(t) ∈ ∂x− V (t, x¯ (t)) H. Frankowska

∀ t ≥ t0 .

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Nonsmooth Analysis Lower Semicontinuous Value Function Continuous Value Function Locally Lipschitz Value Function

Maximum Principle for any (t0 , x0 ) ∈ dom(V ) Theorem (MP without transversality condition) Assume (H) and that (¯ x , u¯) is optimal at (t0 , x0 ). Then i) either normal (MP) holds: ∃ p(·) solving the adjoint inclusion − p 0 (t) ∈ p(t)∂x f (t, x¯ (t), u¯(t)) − ∂x L(t, x¯ (t), u¯(t)) a.e. t ≥ t0 and satisfying the maximality condition a.e. in [t0 , +∞[ ii) or abnormal (MP) holds: ∃ p(·) 6= 0 solving − p 0 (t) ∈ p(t)∂x f (t, x¯ (t), u¯(t)) a.e. t ≥ t0 , and satisfying a.e. in [t0 , +∞[ the abnormal maximality condition hp(t), f (t, x¯ (t), u¯(t))i = max hp(t), f (t, x¯ (t), u)i. u∈U(t)

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Nonsmooth Analysis Lower Semicontinuous Value Function Continuous Value Function Locally Lipschitz Value Function

Normality of the Maximum Principle The infinite horizon problem is called calm with respect to the state variable at (t0 , x0 ) ∈ dom(V ) if lim inf y →x0

V (t0 , y ) − V (t0 , x0 ) > −∞. |y − x0 |

Theorem Assume (H) i) − v ) and that the infinite horizon problem is calm with respect to the state variable at (t0 , x0 ) ∈ dom(V ). If a trajectory-control pair (¯ x , u¯) is optimal at (t0 , x0 ), then a normal (MP) holds true.

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Nonsmooth Analysis Lower Semicontinuous Value Function Continuous Value Function Locally Lipschitz Value Function

Maximum Principle with a Transversality Condition Theorem Let (H) i) − v ) hold and (¯ x , u¯) be optimal at (t0 , x0 ). If an upper semicontinuous function Φ : Rn → R satisfies Φ(·) ≤ V (t0 , ·) on B(x0 , r ) for some r > 0 and Φ(x0 ) = V (t0 , x0 ), then i) either the normal (MP) holds true with the transversality condition −p(t0 ) ∈ ∂ L,+ Φ(x0 ); ii) or the abnormal (MP) holds true with the transversality condition − p(t0 ) ∈ ∂ ∞,+ Φ(x0 ). H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Nonsmooth Analysis Lower Semicontinuous Value Function Continuous Value Function Locally Lipschitz Value Function

Transversality Condition and Sensitivity Relation Theorem Assume (H) i) − v ) and that V (T , ·) is locally Lipschitz for all large T > 0 . Then for every t ≥ 0, V (t, ·) is locally Lipschitz with the local Lipschitz constant depending only on the magnitude of t. Moreover, if (¯ x , u¯) is optimal at some (t0 , x0 ), then the normal (MP) holds true together with the sensitivity relations − p(t0 ) ∈ ∂x V (t0 , x0 ),

−p(t) ∈ ∂x V (t, x¯ (t)) for a.e. t > t0 .

If c, θ, β are bounded and F (t, x ) are closed, then (MP) holds true with the adjoint system in the Hamiltonian form with p satisfying in addition the sensitivity relation (H(t, x¯ (t), p(t)), −p(t)) ∈ ∂V (t, x¯ (t)) H. Frankowska

a.e.

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Second Order Jets Let ϕ : Rn → [−∞, ∞] and x ∈ dom(ϕ). A pair (q, Q) ∈ Rn × S(n) is a subjet of ϕ at x if ϕ(x ) + hq, y − x i +

1 hQ(y − x ), y − x i ≤ ϕ(y ) + o(|y − x |2 ) 2

for some δ > 0 and for all y ∈ x + δB. Then q ∈ ∂ − ϕ(x ). The set of all subjets of ϕ at x is denoted by J 2,− ϕ(x ). 2,1 We assume next that H ∈ Cloc , that f , L are differentiable with respect to x and consider an optimal trajectory-control pain (¯ x , u¯) starting at (t0 , x0 ).

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Second Order Jets Let ϕ : Rn → [−∞, ∞] and x ∈ dom(ϕ). A pair (q, Q) ∈ Rn × S(n) is a subjet of ϕ at x if ϕ(x ) + hq, y − x i +

1 hQ(y − x ), y − x i ≤ ϕ(y ) + o(|y − x |2 ) 2

for some δ > 0 and for all y ∈ x + δB. Then q ∈ ∂ − ϕ(x ). The set of all subjets of ϕ at x is denoted by J 2,− ϕ(x ). 2,1 We assume next that H ∈ Cloc , that f , L are differentiable with respect to x and consider an optimal trajectory-control pain (¯ x , u¯) starting at (t0 , x0 ).

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Riccati Equation For a fixed p0 ∈ ∂x− V (t0 , x0 ) let p¯ (·) solves the adjoint system with p¯ (t0 ) = −p0 . We already know that −¯ p (t) ∈ ∂x− V (t, x¯ (t)) for all t ≥ t0 . If for some T > t0 , V (t, ·) ∈ C 2 for all t ∈ [t0 , T ], then the Hessian −Vxx (t, x¯ (t)) solves the matrix Riccati equation: ˙ R(t) + Hpx [t]R(t) + R(t)Hxp [t] + R(t)Hpp [t]R(t) + Hxx [t] = 0 where Hpx [t] abbreviates Hpx (t, x (t), p(t)), and similarly for Hxp [t], Hpp [t], Hxx [t].

H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Forward Propagation of Subjets Theorem (corollary of Cannarsa, HF, Scarinci, SICON, 2016) Assume (p0 , R0 ) ∈ Jx2,− V (t0 , x0 ) for some R0 ∈ S(n). If the solution R of the matrix Ricatti equation ˙ R(t) + Hpx [t]R(t) + R(t)Hxp [t] + R(t)Hpp [t]R(t) + Hxx [t] = 0 with R(t0 ) = −R0 is defined on [t0 , T ], T > t0 , then the following second order sensitivity relation holds true: (−¯ p (t), −R(t)) ∈ Jx2,− V (t, x¯ (t)), ∀ t ∈ [t0 , T ]. Similar result is valid for backward propagation of second order superjets. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Local Lipschitz Continuity Maximum Principle

State Constraints We request, in addition, that for a closed set K ⊂ Rn trajectories of the control system have to satisfy the state constraint x (t) ∈ K , ∀ t ≥ t0 Assume that U is time independent and the following inward pointing condition (IPC) holds true: ∃ δ > 0 such that ∀ x ∈ ∂K and ∀ u ∈ U with max

hn, f (t, x , u)i ≥ 0

max

hn, f (t, x , w ) − f (t, x , u)i < −δ

n∈NKL (x )∩S n−1

∃ w ∈ U satisfying

n∈NKL (x )∩S n−1

(Conditions developed together with M. Mazzola) The usual H.M. Soner-type condition can not be applied in the non-autonomous case even for finite horizon problems : counterexamples to NFT theorems where given by A. Bressan. H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Local Lipschitz Continuity Maximum Principle

Lipschitz Continuity of Value Function Theorem Assume (IPC), that (H) holds true with time independent c, θ, U, that f is continuous, L(t, x , u) = e −ρt `(x , u), with ` : Rn × Rm → R bounded, continuous and locally Lipschitz in x uniformly in u. Then for all ρ > 0 sufficiently large, V is locally Lipschitz and V (t, ·) is locally Lipschitz uniformly in t. The proof ot this result is so that it provides an estimate of ρ and of the local Lipschitz constants of V and V (t, ·) from c, θ, cR and the inward pointing condition. This work is in progress with V. Basco and P. Cannarsa H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Local Lipschitz Continuity Maximum Principle

Maximum Principle under State Constraints Under the same assumptions, let (¯ x , u¯) be optimal at (t0 , x0 ) ∈ R+ × K . Then there exists a locally absolutely continuous p : [t0 , ∞[→ Rn with −p(t0 ) ∈ ∂xL,+ V (t0 , x0 ), a positive Borel measure µ on [t0 , ∞[ and a Borel measurable

ν(t) ∈ conv NKL (¯ x (t)) ∩ B

µ − a.e. t ≥ t0

such that for Z

η(t) :=

ν(s)dµ(s) ∀ t > t0 & η(t0 ) = 0,

[t0 ,t]

q(t) = p(t) + η(t) and for a.e. t ≥ t0 , we have (−p(t), ˙ x¯˙ (t)) ∈ ∂x ,p H(t, x¯ (t), q(t)) −q(t) ∈ ∂x V (t, x¯ (t)), (H(t, x¯ (t), q(t)), −q(t)) ∈ ∂ V (t, x¯ (t)) H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

H. Frankowska

Local Lipschitz Continuity Maximum Principle

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

H. Frankowska

Local Lipschitz Continuity Maximum Principle

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Local Lipschitz Continuity Maximum Principle

Local Lipschitz Continuity of V (t, ·) Assume (H) with c(t) ≡ c, θ(t) ≡ θ, cR (t) ≡ δ for all R > 0 and |L(t, y , u)−L(t, x , u)| ≤ k(t, |x |∨|y |)|x −y |, ∀ x , y ∈ Rn , u ∈ U(t) k : R+ × R+ → R+ is Lebesgue-Borel measurable, k(t, ·) is %, and Z ∞

e δt k(t, (R + θt)e ct )dt < +∞

∀ R ≥ 0.

0

If dom (V ) 6= ∅, then V is locally Lipschitz on R+ × Rn and for all t ≥ 0 and R > 0 |V (t, x2 ) − V (t, x1 )| ≤ e −δt Kt (R) |x2 − x1 |

∀ x1 , x2 ∈ B(0, R)

where for Mt (τ, R) = [R + θ(τ − t)]e c(τ −t) Z ∞

Kt (R) :=

e δτ k τ, Mt (τ, R) dτ

t

Remark. Less restrictive assumptions imply just continuity of V . H. Frankowska

Infinite Horizon Optimal Control Problem

Infinite Horizon Control Problem Maximum Principle and Sensitivity Second Order Sensitivity Relations State Constrained Case

Local Lipschitz Continuity Maximum Principle

Behavior of the Co-state at ∞ Corollary Under the same assumptions let (t0 , x0 ) ∈ R+ × Rn and (¯ x , u¯) be any trajectory-control pair satisfying x¯ (t0 ) = x0 . Then for all t ≥ t0 and x1 , x2 ∈ B(¯ x (t), 1) we have |V (t, x2 ) − V (t, x1 )| ≤ e −δt Kt0 (1 + |x0 |) |x2 − x1 | Consequently, if (¯ x , u¯) is optimal and (MP) is augmented by the sensitivity relation −p(t) ∈ ∂x V (t, x¯ (t)) a.e. t > t0 then p(t) → 0 exponentially when t → ∞. H. Frankowska

Infinite Horizon Optimal Control Problem