On the maximum principle Horia Cornean,
d. 25/09/2008.
These notes are not completely rigorous, and we often use stronger conditions than necessary in order to simplify the presentation.
1
The one dimensional case
Let f, f0 ∈ C 2 (R3 ) be two real functions, locally bounded in the C 2 norm. The control variables are generically denoted by u : [0, ∞) 7→ [−1, 1], where u is a piecewise continuous function. For a given u and T > 0, let us consider the initial value problem: x0 (t) = f (x(t), u(t), t),
0 < t < T,
x(0) = 0.
(1.1)
There exists a unique solution x, continuous on [0, T ], and also continuously differentiable except at the discontinuities of u.
1.1
Fixed T
We want to maximize the following integral: Z T I(u) := f0 (x(t), u(t), t)dt,
(1.2)
0
by choosing a piecewise continuous “optimal” function u∗ : [0, T ] 7→ [−1, 1]. We will provide sufficient conditions for the existence of such u∗ ’s. Clearly, u∗ generates an optimal state function x∗ which solves: x∗ 0 (t) = f (x∗ (t), u∗ (t), t),
0 < t < T,
∗
x (0) = 0. 1.1.1
(1.3)
A regularization procedure
We allow here the image of u to be the whole real axis, and also modify the functional to be optimized. If M > 0 is a large integer, consider the function gM : [−1, 1] → [0, 1/(2M )], 2 1 (1 − v 2M ). It is strictly concave, symmetric, has a maximum given by gM (v) := 2M 0 00 1/(2M ) at v = 0, gM (∓1) = 0, gM (∓1 ± 0) = ±M , and gM (∓1 ± 0) = −M (2M 2 − 0 1) =: −CM < 0. Moreover, gM converges pointwise to 0 on (−1, 1). Define the function GM : R → R such that: GM (v)
(1.4) 2
2
= [−CM (v + 1) /2 + M (v + 1)]θ(−v − 1) − [CM (v − 1) /2 + M (v − 1)]θ(v − 1) + gM (v)θ(1 − v 2 ). GM belongs to C 2 (R), is strictly concave and G0M is a bijection. Inside √ the interval √ [−1, 1] GM is bounded √ by 1/M , while outside the interval [−1 − 1/ M , 1 + 1/ M ] is as negative as − M . 1
We consider the maximizing problem for: Z T IM (u) := [f0 (x(t), u(t), t) + GM (u(t))]dt,
(1.5)
0
where now u : [0, T ] 7→ R, and u is piecewise continuous. 1.1.2
Approximation with a discrete model
Let N > 1 be an integer, and define: ∆t := T /N,
tk = k∆t,
0 ≤ k ≤ N.
(1.6)
Denote by uM a piecewise continuous control function, and by xM its corresponding state. Consider the discrete dynamical system: y(tk+1 ) := y(tk ) + (∆t)f (y(tk ), uM (tk ), tk ),
0 ≤ k ≤ N − 1,
(1.7)
y(t0 ) = 0.
(1.8)
Lemma 1.1. Denote by ek := |xM (tk ) − y(tk )|. Then lim
max ek = 0.
N →∞ 0≤k≤N
Proof. We can write xM (tk+1 ) − y(tk+1 ) = xM (tk ) − y(tk ) Z tk+1 + {f (xM (t), uM (t), t) − f (y(tk ), uM (tk ), tk )}dt,
0 ≤ k ≤ N − 1.
(1.9)
tk
Using the mean value theorem for f we can find three positive constants C1 , C2 , C3 such that for every tk ≤ t ≤ tk+1 we have: |f (xM (t), uM (t), t) − f (y(tk ), uM (tk ), tk )| ≤ C1 ek + C2
(1.10)
|uM (t) − uM (tk )| + C3 ∆t.
sup tk ≤t≤tk+1
Using this in (1.9) we can write: ek+1 ≤ αek + βk , α := 1 + C1 ∆t,
0 ≤ k ≤ N − 1, βk := C2 ∆t
(1.11) sup
2
|uM (t) − uM (tk )| + C3 (∆t) .
tk ≤t≤tk+1
By induction we derive the inequality: ej ≤ α j e0 +
j−1 X
αk βj−k−1 ,
1 ≤ j ≤ N.
(1.12)
k=0
Note that e0 = 0. Then since αj = ej ln(1+C1 ∆t) ≤ ejC1 ∆t ≤ eC1 T ,
j ≤ N,
there exists C4 > 0 such that ej ≤ C4
N −1 X
βk
(1.13)
k=0
≤ C4 C3 T 2 /N + C4 C2 T
N −1 1 X sup |uM (t) − uM (tk )|, N tk ≤t≤tk+1
1 ≤ j ≤ N.
k=0
Now the lemma is concluded since uM is piecewise uniformly continuous.
2
1.1.3
A finite dimensional optimization problem
Using the previous lemma, we have that for any piecewise continuous control function uM : Z T IM (uM ) = {f0 (xM (t), uM (t), t) + GM (uM (t))}dt 0 N −1 X
= lim
N →∞
{f0 (xM (tk ), uM (tk ), tk ) + GM (uM (tk ))}∆t
k=0 N −1 X
= lim
N →∞
{f0 (y(tk ), uM (tk ), tk ) + GM (uM (tk ))}∆t.
(1.14)
k=0
Now for a fixed N , define FM : RN × RN → R given by FM (a1 , . . . , aN , b0 , . . . bN −1 ) =
N −1 X
{f0 (ak , bk , tk ) + GM (bk )}∆t.
(1.15)
k=0
(Note that a0 = t0 = 0, while FM does not depend on aN ). We will maximize F subject to the restrictions φk (a, b) := ak+1 − ak − f (ak , bk , tk )∆t,
0 ≤ k ≤ N − 1,
a0 = 0.
(1.16)
We form the Lagrangian: FM (a, b) −
N −1 X
pk φk (a, b)
(1.17)
k=0
where the pk ’s are Legendre multipliers. Differentiating with respect to aN we obtain pN −1 = 0. Differentiating with respect to ak , 1 ≤ k ≤ N − 1 we obtain: (∂1 f0 )(ak , bk , tk )∆t − pk−1 + pk + pk (∂1 f )(ak , bk , tk )∆t = 0. Introduce the Hamilton function: H(a, b, p, t) := f0 (a, b, t) + pf (a, b, t).
(1.18)
Now we see that the above extremum conditions for p’s can be written as: pk = pk−1 − (∂1 H)(ak , bk , pk , tk )∆t,
1 ≤ k ≤ N − 1,
(1.19)
pN −1 = 0. From now on we make the following assumption: Assumption 1.2. H is smooth and jointly concave in the first two variables a and b. We see from (1.17) that an optimal bk must solve the equation ∂2 H(ak , b, pk , tk )+ G0M (b) = 0, which has a unique solution due to the fact that GM is strictly concave. Moreover, this solution can be expressed as a smooth function of its variables ˜bK (ak , pk , tk ). In conclusion, under the restrictions in (1.16), if the function FM has a global maximum then it must be attained in some pair of vectors a and b which obey the following relations: 0 = ∂2 H(ak , ˜bk , pk , tk ) + G0 (˜bk ), 0 ≤ k ≤ N − 1, (1.20) M
pk = pk−1 − (∂1 H)(ak , ˜bk , pk , tk )∆t, 1 ≤ k ≤ N − 1, ak+1 = ak + f (ak , ˜bk , tk )∆t, 0 ≤ k ≤ N − 1, a0 = 0.
pN −1 = 0,
It is not obvious how to solve such a system of equations. If we assume that p0 is known, then we can uniquely solve the system by iteration. Then p0 might be determined by imposing the condition pN −1 (p0 ) = 0. 3
1.1.4
A Mangasarian type sufficiency condition
We will prove here that under Assumption 1.2, any solution to (1.20) provides a global maximum point for FM . Let us assume that a∗ , b∗ , p is a solution of (1.20). Then we have: FM (a, b) − FM (a∗ , b∗ ) =
(1.21)
N −1 X
{H(ak , bk , pk , tk ) − H(a∗k , b∗k , pk , tk ) + GM (bk ) − GM (b∗k )}∆t
k=0
−
N −1 X
pk {f (ak , bk , tk ) − f (a∗k , b∗k , tk )}∆t.
k=0
Using the concavity of H and GM , and (1.20), we can write: H(ak , bk , pk , tk ) − H(a∗k , b∗k , pk , tk ) + GM (bk ) − GM (b∗k ) ≤ (ak − a∗k )∂1 H(a∗k , b∗k , pk , tk ) + (bk − b∗k )[∂2 H(a∗k , b∗k , pk , tk ) + G0M (b∗k )] = −(ak − a∗k )(pk − pk−1 ),
1 ≤ k ≤ N − 1.
(1.22)
Note that the above inequality is also formally true for k = 0, since a0 = a∗0 = 0. Now using (1.16), (1.22), and pN −1 = 0 in (1.21) we obtain: FM (a, b) − FM (a∗ , b∗ ) ≤−
(1.23)
N −1 X
{(ak − a∗k )(pk − pk−1 )}∆t
k=1
−
N −2 X
pk {[ak+1 − a∗k+1 ] − [ak − a∗k ]}∆t = 0.
k=1
Thus we have shown that FM attains its global maximum at any solution of (1.20), provided that such solutions exist. 1.1.5
Back to the continuum
Now let us assume that the following system of equations: 0 = ∂2 H(x, u, p, t) + G0M (u), 0
p = −(∂1 H)(x, u, p, t), x0 = f (x, u, t),
(1.24)
p(T ) = 0,
x(0) = 0,
has a solution x∗M , u∗M and p∗M . (The idea is to choose p(0) = p0 as a parameter, solve the system, and then find a compatible p0 from the condition p(T ; p0 ) = 0.) If the continuous solution exists and if ∂p0 p(T, p0 ) 6= 0, then if N is sufficiently large, the system in (1.20) will admit a solution which will ”converge” to the continuous one in the same manner as described in Lemma 1.1. We do not prove this statement. Now if we go back to (1.14), we see that we can write the inequality: IM (uM ) ≤ lim
N →∞
N −1 X
{f (a∗k , b∗k , tk ) + GM (b∗k )}∆t = IM (u∗M ).
k=0
4
(1.25)
1.1.6
Lifting the regularization
Going back to (1.4), we see that the function G0M is almost constant equal to zero on any interval of the form [−1 + , 1 − ] if M is large enough. On the other side, G0M varies very sharply near ±1. We have the following result: Lemma 1.3. Fix x, p, t and assume that H(x, ·, p, t) restricted to [−1, 1] uniquely attains its maximum at v ∈ [−1, 1]. Then the unique solution ˜bM of the equation ∂2 H(x, ˜bM , p, t) + G0M (˜bM ) = 0 fulfils: lim ˜bM = v.
(1.26)
M →∞
Proof. If v ∈ (−1, 1), it means that ∂2 H(x, v, p, t) = 0, ∂22 H(x, v, p, t) < 0 and then we can use the implicit function theorem if M is large enough. If v = 1, it means that ∂2 H(x, 1, p, t) ≥ 0. Because −G0M grows from almost zero to M on a very narrow interval near 1, then ˜bM must be near 1. The same argument goes for v = −1. While ˜bM is a smooth function of a, p, t, the maximum point v can vary sharply between the two extreme values −1 and +1. This is exactly what happens in the bang-bang controls. Assumption 1.4. Define the function v(x, p, t) as the largest maximum point of H(x, ·, p, t) on [−1, 1]. Assume that the system p0 = −(∂1 H)(x, v(x, p, t), p, t), 0
x = f (x, v(x, p, t), t),
p(T ) = 0,
x(0) = 0,
(1.27)
has a unique continuous, piecewise C 1 solution x∗ (t) and p∗ (t). Also assume that the function u∗ (t) := v(x∗ (t), p∗ (t), t) is piecewise continuous. Then one can prove that the solution (x∗M , p∗M , u∗M ) of (1.24) converges uniformly to (x∗ , p∗ , u∗ ) on compacts avoiding discontinuities. Moreover, u∗M (t) ∈ (−1, 1) for all 0 ≤ t ≤ T if M is large enough. Going back to (1.2), choose an arbitrary control function u. Using (1.5) we obtain: I(u) ≤ lim sup IM (u) ≤ lim sup IM (u∗M ) = lim sup I(u∗M ) = I(u∗ ). M →∞
1.2
M →∞
M →∞
Variable T , fixed end-point
Here we assume that f and f0 do not depend on time. The state function starts again from x0 = 0, and we want to get to a fixed x1 > 0 such that a certain functional is maximized. For a given control function u : [0, ∞) → [−1, 1], denote by T (u) > 0 the first time when x(t) = x1 . Then we want to maximize: Z I(u) =
T (u)
f0 (x(t), u(t))dt.
(1.28)
0
The minimal arrival time problem is covered by the case f0 (x, u) = −1. We introduce again the regularized functional IM , and we discretize the problem. One important difference is that we must allow the different times tk to be independent variables, although we still impose that they are a strictly increasing sequence and t0 = 0. 5
For a given u which generates a state x which will touch x1 at T (u), the discretization procedure from (1.6) and (1.7) will generate an approximate discrete solution which might not fulfil the endpoint condition: y(tN ) = x1 . The remedy is the following: we discretize exactly as before up to N − 1, and then we choose tN such that x1 = y(tN −1 ) + (tN − tN −1 )f0 (y(tN −1 ), uM (tN −1 )). Thus (1.14) can be written again, and we are left to a finite discretization problem, which we will describe next. For a fixed N , define FM : RN × RN → RN → R given by FM (a1 , . . . , aN , b0 , . . . bN −1 , t1 , . . . , tN ) =
N −1 X
{f0 (ak , bk ) + GM (bk )}(tk+1 − tk ).
k=0
(1.29) (Again a0 = t0 = 0, FM does not depend on aN ). We will maximize F subject to the restrictions φk (a, b, t) := ak+1 − ak − f (ak , bk )(tk+1 − tk ),
0 ≤ k ≤ N − 1.
(1.30)
Note that we do not impose the tk ’s to be increasing. We form the Lagrangian: FM (a, b, t) −
N −1 X
pk φk (a, b, t) − λ(aN − x1 ).
(1.31)
k=0
The necessary equations obeyed by a local extremum of this function are: 0 = ∂2 H(ak , ˜bk , pk ) + G0M (˜bk ), 0 ≤ k ≤ N − 1, pk = pk−1 − (∂1 H)(ak , ˜bk , pk )(tk+1 − tk ), 1 ≤ k ≤ N − 1,
(1.32)
pN −1 = λ, ak+1 = ak + f (ak , ˜bk )(tk+1 − tk ), 0 ≤ k ≤ N − 1, H(ak , ˜bk , pk ) + GM (˜bk ) = H(aN −1 , ˜bN −1 , pN −1 ) + GM (˜bN −1 ) = 0, aN = x1 ,
(1.33)
a0 = 0.
One major difference compared to the fixed T case is the appearance of N conservation equations (1.33). Here we have N extra unknowns, the different times. It is far from obvious how the general solution might look like, especially that we have not imposed monotonicity on t’s. But if we are only searching for strictly increasing times, and if for instance we assume that f > 0 (which indicates that the a’s can get closer to x1 after each step), then we can get rid of the times and simplify the system: 0 = ∂2 H(ak , ˜bk , pk ) + G0M (˜bk ), 0 ≤ k ≤ N − 1, ak+1 − ak , 1 ≤ k ≤ N − 1, pk = pk−1 − (∂1 H)(ak , ˜bk , pk ) f (ak , ˜bk ) pN −1 = λ, H(ak , ˜bk , pk ) + GM (˜bk ) = H(aN −1 , ˜bN −1 , pN −1 ) + GM (˜bN −1 ) = 0, a N = x1 ,
(1.34)
(1.35)
a0 = 0.
This system can be solved in a recursive way. From (1.34) and (1.35) we obtain aN −1 and ˜bN −1 as functions of λ. Then we have pN −2 from the iteration, and we start again by finding aN −2 and ˜bN −2 . Finally, we determine λ from the condition −ak a0 = 0. The time differences are then obtained as afk+1 and we are done. (a ,˜ b ) k
6
k
1.2.1
The continuum case
Now let us assume that the following system of equations: 0 = ∂2 H(x, u, p) + G0M (u), 0
p = −(∂1 H)(x, u, p), 0
x = f (x, u, t),
(1.36)
p(T ) = λ,
x(0) = 0,
x(T ) = x1 ,
0 = H(x, u, p) + GM (u), has a solution with x0 (T ) 6= 0 where T is the first time when x(T ) = x1 . Then the discrete problem will also have a solution which will ”converge” to the continuous one when N → ∞. We do not give details.
7
