CC BY
14
2016 ВЕСТНИК САНКТ-ПЕТЕРБУРГСКОГО УНИВЕРСИТЕТА Сер. 10 Вып. 3
ПРОЦЕССЫ УПРАВЛЕНИЯ
УДК 517.977.58 A. V. Fominyh
THE HYPODIFFERENTIAL DESCENT METHOD IN THE PROBLEM OF CONSTRUCTING AN OPTIMAL CONTROL*
St. Petersburg State University, 7—9, Universitetskaya nab., St. Petersburg, 199034, Russian Federation
This paper considers the problem of optimal control of an object, whose motion is described by a system of ordinary differential equations. The original problem is reduced to the problem of unconstrained minimization of a nonsmooth functional. For this, the necessary minimum conditions in terms of subdifferential and hypodifferential are determined. A class of problems, for which these conditions are also sufficient, is distinguished. On the basis of these conditions, the subdifferential descent method and the hypodifferential descent method are applied to the considered problem. The application of the methods is illustrated by numerical examples. Refs 16. Tables 4.
Keywords: nonsmooth functional, variational problem, program control, hypodifferential descent method.
А. В. Фоминых
МЕТОД ГИПОДИФФЕРЕНЦИАЛЬНОГО СПУСКА В ЗАДАЧЕ ПОСТРОЕНИЯ ОПТИМАЛЬНОГО УПРАВЛЕНИЯ
Санкт-Петербургский государственный университет, Российская Федерация, 199034, Санкт-Петербург, Университетская наб., 7—9
В статье рассматривается задача оптимального управления объектом, движение которого описывается системой обыкновенных дифференциальных уравнений. Исходная задача сводится к задаче безусловной минимизации некоторого негладкого функционала. Для него найдены необходимые условия минимума в терминах субдифференциала и гиподиф-ференциала. Выделен класс задач, для которых эти условия оказываются и достаточными. На основании данных условий к изучаемой задаче применяются метод субдифференциального спуска и метод гиподифференциального спуска. Приложение методов иллюстрируется на численных примерах. Библиогр. 16 назв. Табл. 4.
Ключевые слова: негладкий функционал; вариационная задача, оптимальное управление, метод гиподифференциального спуска.
Introduction. The technique of exact penalty functions was firstly used in the optimal control problems in [1, 2]. The general idea of such an approach is reduction of
Fominyh Alexander Vladimirovich — postgraduate student; [email protected] Фоминых Александр Владимирович — аспирант; [email protected]
* Работа выполнена при финансовой поддержке Российского фонда фундаментальных исследований (грант № 16-31-00056 мол-а) и Санкт-Петербургского государственного университета (НИР, проект № 9.38.205.2014).
© Санкт-Петербургский государственный университет, 2016
the original problem with restrictions to the unconstrained minimization of a nonsmooth functional. For this problem one should use nonsmooth optimization methods. The subdifferential descent method and the hypodifferential descent method belong to this class of methods.
The methods used in the paper may be refered to the direct methods of optimal control problems, since the optimization problem in functional space is being solved without necessity for integration of the system, which describes the controlled object. Among the vast arsenal of optimal control problem solving methods an approach based on the variations of the minimized functional is similar with the considered in the article method (see [3-6]).
In this paper the integral restriction on control is considered. Optimal control problems with such constraints were studied in some works, for example [7-9].
Approach used in the article is especially appropriate when it is important to take into account precisely the limitation on the final position of the object and the restriction in the form of differential equalities. It is therefore of interest in the spread of the use of exact penalties over optimal control problems with state constraints, the exact adherence of which is principal in many practical problems.
Statement of the problem. Let us consider a system of ordinary differential equations in normal form
x(t) = f (x,u,t), t e [0,T]. (1)
It is required to find such a control u* e Pm[0, T], satisfying an integral restriction
T
J (u(t),u(t))dt < 1, (2)
0
which brings system (1) from the given initial position
x(0) = x0 (3)
to the given final state
x(T) = xT (4)
and minimizes the integral functional
T
I(x,u) = J fo(x,x,u,t)dt. (5)
0
Suppose that there exists an optimal control u*. In system (1) T > 0 is a given moment of time, f (x, u,t) is a real n-dimensional vector-function, x(t) is an n-dimensional vector-function of the phase coordinates, which is supposed to be continuous with partially continuous in the interval [0,T] derivative, u(t) is an m-dimensional vector-function of control, which is supposed to be partially continuous in [0, T]. We consider f (x, u,t) to be continuously differentiable in x and u and continuous in all three of its arguments.
If t0 e [0, T) is a discontinuity point of the vector-function u(t), then we put
u(t0) = lim u(t). (6)
t[t0
At the point T we assume that
u(T ) = lim u(t). (7)
t \ 1
We consider that x(t0) is a right-handed derivative of the vector-function x at the point t0, x(T) is a left-handed derivative of the vector-function x at the point T.
In functional (5) f0(x,x,u,t) is a real scalar function, which is supposed to be continuously differentiable in x, x and u and continuous in all four of its arguments.
Reduction to the variational problem. Put z(t) = x(t), z G Pn [0, T]. Then from
(3) we get x(t) = x0 + / z(t)dr. With regard to the vector-function z(t) we make a Jo
suggestion, analogous to (6), (7). We have
f (x, u,t) = f I xo + z(t)dr, u, t
V 0 ) I f \
fo(x, z,u,t) = f xo + z(t)dT, z,u,t .
Let us introduce the functional
Fx(z, u) = I(z, u) + X
n ( T
z, u) + ^^ фъ(z) + max \0, (u(t), u(t)) dt — 1
i=l I n
where
(z(t) — f(x, u,t), z(t) — f(x, u,t))dt,
(8)
Ipi(z) = IV'iMl, r'Pi(z)=x0i+ J Zi(t)dt-xTi, i = l,n,
o
and x0i is an i-th component of the vector x0, xTi is an ¿-th component of the vector xT, i = 1, n, A > 0 is some constant. Denote
Ф(z, u) = y(z, u) + ^ фi(z) + max | 0, J (u(t), u(t))dt — 1 > .
(9)
It is not difficult to see that functional (9) is nonnegative for all z g Pn[0,T] and for all u G Pm[0, T] and vanishes at a point [z,u] e Pn[0, T] x Pm[0, T] if and only if the
vector-function u(t) satisfies constraint (2), and the vector-function x{t) =xo+ z(r)dT
_ Jo
satisfies system (1) at u(t) = u(t) and constraints (3), (4). Let us introduce the sets
Q={[z,u] e Pn[0,T] x Pm[0,T] | ФМ = 0},
a, = {[z,u] e Pn[0,T] x Pm[0,T] | $(z,u) <4, here 6 > 0 is some number. Then
Qg \ Q = {[z,u] e Pn[0, T] x Pm[0, T] I 0 < $(z, u) < 6}.
Using the same technique as in [1, 10], it can be shown that the following theorem takes place.
Theorem 1. Suppose there exists such a positive number Ao < ro that VA > Ao there exists a point [z(A),u(A)] e Pn[0,T] x Pm[0,T], for which F\(z(A),u(A)) = inf F\(z,u).
[z,u]
Let the functional I(z,u) be Lipschitz on the set Qg \ Q. Then functional (8) will be an exact penalty function.
Thus, under the assumptions of Theorem 1 there exists such a number 0 < A* < ro that VA > A* the initial problem of minimization of functional (5) on the set Q is equivalent to the problem of minimization of functional (8) on the whole space. Further we suppose that the number A in functional (8) is fixed and the condition A > A* holds.
Lemma 1. If system (1) is linear in the phase variables x and in control u, and the functional I(z,u) is convex, then the functional F\(z,u) is convex. Proof. Let us present functional (8) in the form
Fx(z,u) = I(z, u) + Ap(z, u) + AFi(z) + AF2(u),
where I(z,u), p(z,u), Fi(z), F2(u) are the corresponding summands from the right-hand side of (8). The functionals Fi(z) and F2(u) are convex as maximum of convex functionals. The functional I(z, u) is convex by the lemma assumption. Let us show the convexity of the functional z,u) in the case of the linearity of system (1). Let system (1) be of the form
X = A(t)x + B(t)u + c(t),
where A(t) is an n x n-matrix; B(t) is an n x m-matrix; c(t) is an n-dimensional vector-function. Suppose A(t), B(t), c(t) be real and continuous in [0,T]. Let zi,z2 e Pn[0,T], mi, «2 G Pm[0, T], a £ (0,1). Denote Tp(z, u, t) = z(t) — f(z, u, t). We have
^2(a(zi,ui) + (1 - a)(z2,u2)) = ||azi(t) + (1 - a)z2(t) -
t
- A(t) [xo + J (azi(r) + (1- a)z2(T))dr] - B(t)[aui(t) + (1 - a)u2(t)\ - c(t)112 =
o
T
|2 2
= | \a(p{zi,ui) + (1 - a)(fi(z2,u2)\\2 = a2 J (tp{z\, u\,t), (p{zi,ui,t))dt + (10)
0
T T
+ 2a(l — a) J {Tp{zi,ui,t),Tp{z2,u2,t))dt + (1 - a)2 J (Tp(z2,u2,t),Tp{z2,u2,t))dt,
0
T
(ay>(zi,wi) + (1 - a)ip(z2,u2))2 = a2 J (^(z1,u1,t),^(z1,u1,t))dt +
0
+ 2а(1 — а)
T
(p(z1,u1,t),'ip(z1,u1,t))dt J fa(z2,u2,t),^(z2,u2,t))dt +
0
T
+ (1 - a)2 J (Tp{z2,U2,t),Tp{z2,U2,t))dt. (11)
Using Holder's inequality, for all zi, z2, ui, u2 one gets
T
/ (Tp{zi,ui,t),Tp{z2,U2,t))dt <
<
(cp(z1,u1,t),cp(z1,u1,t))dt
(<p(z2,U2,t), (p(z2,u2,t))dt,
hence from (10) and (11) we obtain
p2(a(zi,ui) + (1 - a)(z2, u)) < (a^(zi,ui) + (1 - a)^(z2,u2))2. (12)
Since a(zi,ui) + (1 — a)(z2,u2)) ^ 0, a^(zi,ui) + (1 — a)^(z2,u2) ^ 0, then from inequality (12) V zi, z2, ui, u2 and a G (0,1) follows:
p(a(zuui) + (1 — a)(z2,u2)) < a^(zi,ui) + (1 — a)p(z2,u),
that proves the convexity of the functional ^(z, u) in the case of the original system linearity.
Now note that the functional F\(z,u) is convex (in the case of the initial system linearity) as a sum of convex functionals. Lemma 1 is proved.
Necessary minimum conditions. Let us introduce the sets Vi = \z G Pn [0,T]\x0 + J z(t)dt = xt \ ,
П2 = |u e Pm[0,T] | J (u(t),u(t))dt < 1
Ms = {[z,u] G Pn[0,T] x Pm[0,T] \ p(z,u)=0} and the following index sets:
l0 = {i = l^\lpi(z)=0},
I- = {i = | ^i(z) < 0}, I+ = {i = T^I0i(z)>O}.
Let us also introduce the control sets
U0 = e Pm[0,T] | T (u(t),u(t))dt - 1 = 0 j ,
f I T
U- = In e Pm [0,T] I j (u(t), u(t)) dt - 1 < 0
U+ = I u
e Pm[0,T]| j {u(t),u(t))dt - 1 > oj.
Using the same technique as in [1, 10], it is easy to see, that the following two theorems take place.
Theorem 2. If [z,u] G Q.3, then the functional F\(z,u) is subdifferentiable, and its subdifferential at the point [z, u] is expressed by the formula
/Qf df P df / '
t t ы j=1
dfo
df
+ A[ - (^J w(t) + 2i/u(t)]J toi G [-1, 1], г G /0,
Hj = 0, j e Io, Hj = 1, j e I+, Hj = -1, j e I-, (13) v e [0,1], u e U0, v = 1, u e U+, v = 0, u e U-,
z(t) — f (x, u, t) '
<t) =
<p(z,u)
Theorem 3. If [z,u] G then the functional F\(z,u) is subdifferentiable, and its subdifferential at the point [z, u] is expressed by the formula
9Fx(z, u) = {[f ^dr + Ц + A[«(i) - J (%)'v{r)dr + ]T uiiCi + ¿^e,-],
iei0
j=i
dfo du
+ \[-(^)'v(t) + 2,su(t)]] \ vePn[0,T], IblKl}- (14)
In (14) u>i G [—1,1], i G Io, Hj, j = 1,"-; v are defined by (13).
Corollary 1. If [z,u] G O3, z G ^i, u G then the functional F\(z,u) is subdifferentiable, and its subdifferential at the point [z,u] is expressed by the formula
T
T
dFx(z, u) = {[j fdr + ^ + A[«(i) - J (%)'v{T)dr + ]T
dx dz J \3x
tt
ieio
uJi G [-1, 1], i = 1, n,
du \8u
V e [0,1], u e Uo, V = 0, u e U-, v e Pn[0,T], ||v|| < 1}.
(15)
It is known [11] that necessary and in the case of the convexity also sufficient condition for the minimum of functional (8) at the point [z*,u*] in terms of subdifferential is the condition
0n+m e dF\(z * ,u*),
where 0n+m is a zero element of the space Pn[0,T] x Pm[0,T]. Hereof and in view of Lemma 1 we conclude that the following theorem takes place.
Theorem 4. For the control u* e to bring system (1) from initial position (3) to final state (4) and to minimize functional (5), it is necessary, and in the case of the linearity of system (1) and the convexity of functional (5) also sufficient that
0n+m e dF\(z*,u*),
(16)
where the expression for the subdifferential dF\(z,u) is given by (15).
The subdifferential descent method. Let us find the smallest by norm subgradient h = h(t, z, u) e dF\(z, u) at the point [z, u], i. e. solve the problem min
Fix a point [z, u] and consider two cases. A. Let p(z, u) > 0. In this case
heOFx(z,u)
mm
hedFx(z,u)
where
2 ._
mm
Ui, i^Io, V
T T
J (s1(t)+^YJ^iei)2dt + J (s2(t) + 2Xvu(t))2dt
0 ^o 0
S1(t)=i1(t)+\J2
He
j=1
(17)
Si(t)
dx
dz
\dx
(t )dr\,
and numbers Wj, i G Jo, Mj, J = 1,^ and the vector-function w(t) are defined by (13).
Problem (17) is a problem of quadratic programming with linear constraints and can be solved using one of the known methods [12]. Denote w*, i e Io, v* its solution. Then the vector-function
G(t,,
:= h* =
si(t) + X V u*ei, s2(t) + 2Xv*u(t)
ieio
(18)
is the smallest by norm subgradient of the functional Fx at a point [z, u] in this case (if ^(z,u) > 0). If ||G|| > 0, then the vector-function —G(t,z,u)/\\G\\ is the subdifferential descent direction of the functional Fx at the point [z, u].
B. Let ¥>(z, u) = 0. In this case
mm
hedFx(z,u)
2
mm
\\hi\\2 + \\h2\\2] = min
Ui, iElo, v, v
dx dz
j
//■) -f / 2 {-£) <T)dT + Е<*е<}dt + (19)
t ieio j=l
du
where h\ = hi(t,z,u), h2 = h2(t,z,u), and numbers u>i, i g Io, l^j, j = 1 ,n, v and the vector-function v(t) are defined by (14). Construct the functional
+ /x[ max{0, |H|2 - l} + max{0,772 - 1} + ^ max{0, uJj - 1}], (20)
ieio
here v = 2v — 1, and the vector lv g i?'io' consists of the components Wj, i g Io-
Under some natural assumptions it can be shown, that the functional HM is an exact penalty function, then one may use any method (for example, the subdifferential descent method) for the unconstrained minimization of functional (20) to find v*, w*, v*.
Remark 1. The subdifferential dF\(z,u) is a convex compact set, therefore necessary minimum condition of the functional H^(v,u>,T/) will be also sufficient. Denote v*, w*, v* the solution of problem (19). Then the vector-function
G(t,z,u) := h* =
'fV + f + A
dx dz
v*(t)
T
/ \dx
dfo du
'(т )dr ш*ei ej
ieio j=i
+ A
(Щ
\du)
v* (t) + 2v* u(t)
(21)
2
is the smallest by norm subgradient of the functional F\ at the point [z,u] in this case (if ^(z,u) = 0). If ||G|| > 0, then the vector-function —G(t,z,u)/\\G\\ is the subdifferential descent direction of the functional F\ at a point [z,u].
Thus, in items A and B the problem of finding subdifferential descent direction of the functional Fx at a point [z, u] has been solved. In case of p(z, u) > 0 (item А) this problem is solved relatively easily, as it is a problem of quadratic programming with linear constraints. In case of p(z, u) = 0 (item B) besides unknown values w, v one must also find the vector-function v(t). It is a more complicated problem, which can be solved with numerical methods, for example, with subdifferential descent method, as it was noted in item B.
Now we can describe the subdifferential descent method for finding stationary points of the functional Fx(z,u). Choose an arbitrary point [z\,u\] G Pn[0,T] x Pm[0,T] and assume that the point [zk, uk] G Pn[0, T] x Pm[0, T] is already found. If minimum condition (16) holds, then the point [zk,uk] is the stationary point of the functional Fx(z,u) and the process terminates. Otherwise put
[zk+i,uu+i] = [zu, uu] - akGk,
where the vector-function Gk = G(t,zk,uk) is the smallest by norm subgradient of the functional Fx at the point [zk,uk]. The value for the functional Gk is given either by
formula (18) if <^>(zk,uk) > 0, or by formula (21) if ,uk) = 0. The value ak is the solution of the following one-dimensional minimization problem
min Fx([zk,uk] - aGk) = Fx([zk,uk] - akGk).
Then Fx(zk+1,uk+1) < F\(zk,uk). If the sequence {[zk,uk]} is finite, then its last point is the stationary point of the functional Fx(z, u) by construction. If the sequence {[zk, uk]} is infinite, then the described process may not lead to the stationary point of the functional Fx(z,u), because the subdifferential mapping dFx(z,u) is not continuous in Hausdorff metric.
The hypo differential descent method. Using formulas of codifferential calculus [11], it can be shown that the following two theorems take place.
Theorem 5. If [z, u] <// then the functional Fx(z, u) is hypodifferentiable, and its hypodifferential at a point [z, u] is expressed by the formula
dF\{z, u) = [0,si(i),s2(i)] +
n
+ A^cojfV'jl» - tpi(z),ei,0m], [-ipil\z) - V>i(z),-ei;0m]} +
i=i
T
+ \co{[J (u(t),u(t))dt - 1 - max{0, ||u||2 - 1}, 0n, 2u(t)], [ - max{0, ||u||2 - 1}, 0n, 0m]},
0
where
n j=i
t t с
(t) =
z(t) — f (x, u, t)
Hj = 0, j e ^ Hj = ~1, j e 1+, Hj = —11, j e I-.
Theorem 6. If [z, u] G then the functional Fx(z, u) is hypodifferentiable, and its hypodifferential at a point [z, u] is expressed by the formula
dFx{z,u) = \ A[ j (z(t)-f(x,u,t))'v(t)dt-<p(z,u)], [^dT + ^1 +
+
+ - Vi(*),ei,Om], [-ipi(z)-ipi(z),-ei, Om] } + (22)
i=i
T
+ Xco{[J (u(t),u(t))dt - 1 - max{0, ||u||2 - 1}, 0n, 2u(t)],
0
[ - max{0, M2 - 1}, 0n, 0ni]} | v e Pn[0,T], ||v|| < l}.
It is known [11] that necessary and in the case of the convexity also sufficient condition for the minimum of functional (8) at the point [z*,u*] in terms of hypodifferential is the condition
0n+m+l e dF\ (z*,u*),
where 0n+m+1 is a zero element of the space Pn [0,T] x Pm [0,T] x R. Hereof and in view of Lemma 1 we conclude that the following theorem takes place.
Theorem 7. For the control u* e to bring system (1) from initial position (3) to final state (4) and to minimize functional (5), it is necessary, and in the case of the linearity of system (1) and the convexity of functional (5) also sufficient that
0n+m+1 e dFx (z*,u*), (23)
where the expression for the hypodifferential dFx(z,u) is given by (22).
Let us find the smallest by norm hypogradient g = g(t, z, u) e dFx(z, u) at the point [z, u], i. e. solve the problem min ||g||2.
gEdFx (z,u)
Fix a point [z,u] and consider two cases. A. Let p(z,u) > 0. In this case
min \\g\\2 = min_|| [0, si(t),s2(t)] +
gedFx(z,u) /3i£[0,l], i=l,n+l
n
+ A^i/M^iW - V>i(*),ei,0m] +(1 - A) - Vi(*),-ei,0m]} +
i=i
T
+ Авп+i
J (u(t),u(t))dt - 1 - max{0, ||u||2 - 1}, 0n, 2u(t) .0
+ (24)
+ X(1 - /3n+i)[ - max{0, M2 - 1}, 0ri, 0m] 11 .
Problem (24) is a problem of quadratic programming with linear constraints and can be solved using one of the known methods [12]. Denote its solution /?*, i = l,n+ 1. Let g = [g1,g2], where the vector-function g2 consists of the last n + m components of g. Then the vector-function
n
G(t, z, u) := g2 = [si(i), s2(t)] + A ^ {¡3* [<*, 0m] + (1 - ¡3*) [ - eu 0m] } +
i=i
+ X^n+i [0n, 2u(t)] + X(1 - fj*n+i) [0n, 0m] (25)
consists of the last n+m componets of the smallest by norm hypogradient of the functional Fx at the point [z,u] in this case (if z,u) > 0). If ||G|| > 0, then the vector-function
-G(t,z,v)/HGH is the hypogradient descent direction of the functional Fx at the point [z,u].
B. Let z, u) = 0. In this case
mm |
gEdFx(z,u)
min
/з»е[о,1], i= i,n+i, v
\[ (z(t) - f (x,u,t))'v(t)dt - ^(z,u)\,
dfo, ^ df0 dx dz
v{T)dT]—-X(^)v(t)
dx
+
+ \f3n+i[ (u(t),u(t)) dt - 1 - max{0, ||u||2 - 1}, 0n, 2u(t)] +
+ A(1 - pn+i)[ - max{0, ||u||2 - 1}, 0n, 0m\
\[ (z(t) - f (x,u,t))'v(t)dt - ф(z,u)\,
/з»е[о,1], i= i,n+i,
dx dz
ал'
dx
+
i, оm\ + [ - ФА?) - фг(г), -ei; 0m] } +
i=l
+ Apn+i[ (u(t),u(t)) dt - 1,0n, 2u(t)]+ A[ - max{0, ||u||2 - 1}, 0n, 0m\
This expression can be rewritten as follows:
mm |
gEdFx(z,u)
2
min
min
i=i,n+i, v
-A
i=i
{z)+ipi[z)) + — I I [U
2 + Ш2 + Ш2] =
n
A[ I (z(t)-f(x, u, t))'v(t)dt-<p(z, и)] +A Y^Piiz) (Pi + 1)-
i=i
(u(t), u(t) I dt 1J (/3n+i + 1 - A max{0, HuH2 - +
t n } 2
+ dt +
T ( T T
+ fl fr2JldT+f2Jjl+x\v(t)-
J W dx dz J \dx
0 Kt t
2
vt
2
v(t)
2
T
where g\ = gi(t, z, u), g2 = gi{t, z, u), g3 = g3(t, z, u), f3i = 2/?j — 1, i = 1, n + 1, and the vector-function v(t) is defined in (22).
Let the vector /3 G Rn+1 consist of the components /?i; i = l,n+l. Write the functional
n+1
|2
ff>,/3) = \\g\\2 +M[max{0, |M|2 - 1} + ]T max{0, ft - 1}]. (27)
i=i
Denote
n+1
112 — 1} + max{0j Pi ~ 1}] •
i=i
Introduce the sets
Tl= {[v,~p\ G Pn[0,T] x Rn+1 | = 0},
Tls = {M] G Pn[0,T] x i?n+1 | < 5}.
Then _ _
QS\Q= {[v,/3] ePn[0,T] xRn+1 | 0 <*(«,/?) <S}.
Also introduce the following sets
___9
Bi0 = {Pi G R | Pi - 1 = 0},
___9
= {A G R | ft - 1 < 0},
_ _2
Bi+ = {Pi e R \ Pi - l > 0},
where i = 1, n + 1.
Lemma 2. Suppose there exists such a positive number fj,o < oo that VyU, > /xo f/iere exists a point [v(p), P(p)] G P„[0, T] x i?n+1; /or which /?(/x)) = inf H^vj.3).
_ _ [v,P]
Let the functional g{v,p) be Lipschitz on the set ils \ T7ien functional (27) will be an exact penalty function.
Thus, under the assumptions of Lemma 2 there exists such a number 0 < j* < <x that Vj > j* problem (26) is equivalent to the problem of minimization of functional (27) on the whole space. Further we suppose that the number j in functional (27) is fixed and the condition j > j* holds.
Lemma 3. Functional (27) is hypodifferentiable, and its hypodifferential at a point [v,p] is expressed by the formula
dHfl(v,f3)=[0,gv,gpi,...,gpn+i] +
r /"Tii , ,1
+ j
[IMI2 - 1 - max{0, IMI2 - 1}, 2v(t), 0n+1], [- max{0, ||v||2 - 1}, 0n, 0„+i]} +
_2 _2 _ _2
+ co{ [ft - 1 - max{0, ft - 1}, 0n, 2ft, 0n], [ - max{0, ft - 1}, 0n, 0n+1] } + •••+ (28)
_2 _2 _ _2
+ co{ [/Vi - 1 - max{°, Pn+1 - 1}, On, On, 2/?n+1], [ - max{0, /?п+1 - 1}, On, On+1] }
Calculate the following vector-functions in formula (28):
ffv = giv + g2v + gsv,
where
giv = 2A2< / (z{t) +
i=i
i=i
+ + о / («(i),«(i))<ft-l (/?n+i + 1) "
i=i
- max{0, ||u ||2 - 1H (z(t) - f (x,u,t)),
92v = 2A Av{t) - A J {^)'v(r)dr - Ag | «(r)dr + A^ / / (£ ) «(O^dr +
t T dj_ [ [
dx J J \dxs
0 т
, fdfo,, ,dfo ^
'dfo a/o dx dz
Q J- n
dT ~ Xt7TrY,Piei
=
г = 1,n,
where
S1!^. = 2A2 < / (z(t) - f(x,u,t))'v(t)dt - (p(z,u) +
i=i
i=i
i=i
T
J (u(t), u(t))dt - 1
(fln+i + 1) — max{0, | |м| | — 1} / Фг(г
= 2A
dfo , d/0 .
"5" ®T + "5--Л
dx dz
df
+ А^/^е* > ejdi,
ffe = 2A
du
du
v(t) + f3n+1u(t) + u(t)\ J u{t)dt.
t
1
Remark 2. The hypodifferential dF\(z,u) is a convex compact set, therefore necessary minimum condition of the functional H^(v,/3) will be also sufficient.
Lemma 4. For the point [v*,/3 ] G Pn[0,T] x Rn+1 to minimize functional (27), it is necessary and sufficient that
—i
0n+n+2 G dH^v* ,3 ), (29)
where the expression for the hypodifferential dH^v,^) is given by (28).
Let us find the smallest by norm hypogradient g = g(t, v, /3) g dH^(v, /3) at the point [v,/3], i. e. solve the problem
min_ll<?H2= min_ [0,9v,ff3,---,ff3 1 +
gedH^v,/3) TiG[0,l], i=l,n+2 Pl '
+ M
71 [IIvII2 -1 -max{0, ||v||2 -1}, 2v(t), 0n+i] +(1-71) [-max{0, ||v||2 -1}, 0n, 0n+i] +
_9 _2 ___2
+ 72 [ft -1 -max{0, ft -1}, 0„, 2ft, 0„] +(l-72) [-max{0, ft -1}, 0„, 0n+1] +• • • + (30)
_2 _2 _
+ 7n+2 [Pn+i ~ 1 - max{0, (3n+1 - 1}, 0„, 0„, 2(3n+1] +
_2
+ (1 - 7„+2)[ - max{0, (3n+1 - l},0„,0„+i]
Problem (30) is a problem of quadratic programming with linear constraints and can be solved using one of the known methods [12]. Denote its solution 7*, i = l,n + 2. Let 9 = 92)1 where the vector-function ~g2 consists of the last n + n + 1 components of g. Then the vector-function
n +
2
+ M
G(t,vJ3) \= g*2 = [gv, g^^, .. ., i?âri+1] + 7Î [2«(t),0n+1] +(l-7Î)[0„,0n+1] +72*[0„,2ft,0„] +(l-72*)[0„,0n+1] +••• + + 7:+2 [On, 0n, 2/?n+1] + (1 - 7:+2) [0„, 0n+i]
consists of the last n + n + 1 components of the smallest by norm hypogradient of the functional HM at the point [v,[3\. If ||G|| > 0, then the vector-function —G(t,v,f3)/\\G\\ is the hypogradient descent direction of the functional HM at the point [v,[3\.
Let us describe the following hypodifferential descent method for finding minimum points of the functional H^(v, /3). Choose an arbitrary point [vi, ft] g Pn[0, T] x Rn+1 and assume that the point [vk,/3k] g Pn[0,T] x Rn+1 is already found. If minimum condition (29) holds, then the point [vk,/3k] is the minimum point of the functional H^(v,/3) and the process terminates. Otherwise put
[vk+iJ3k+1] = [vkJ3k] - akGk,
where the vector-function Gk = G(t,vk,/3k) consists of the last n + n+ 1 components of the smallest by norm hypogradient of the functional Hf at the point [vk, ft] and the value ak is the solution of the following one-dimensional minimization problem:
minH^([vk,/3k] - aGk) = H^([vk,f3k] - akGk). (31)
a>0
Then H^(vk+i, fik+i) ^ Hfj,(vk, (3k). If the sequence {[«fc,/3fc]} is infinite, then it can be shown that the hypodifferential descent method converges in the following sense:
If the sequence {[vk, f3k]} is finite, then its last point is the minimum point of the functional Hfj,(v,(3) by construction.
Denote v*, /* the solution of problem (26). Let g = [gi,g2], where the vector-function g2 consists of the last n + m components of g. Then the vector-function
G(t,z,u) ■= g2 dfo_x(dfy.
+ + Л dx dz
du
duJ
v2 (t)
+ 0m] +(1- Ю[ - eu +
(32)
i=i
+ \l3*n+i [0„, 2u(t)} + X(1 - en+i) [0n, 0m]
consists of the last n+m components of the smallest by norm hypogradient of the functional Fx at the point [z,u] in this case (if z,u) = 0). If HGH > 0, then the vector-function —G(t,z,u)/\\G\\ is the hypogradient descent direction of the functional Fx at the point [z,u].
Thus, in the points A and B the problem of finding the hypogradient descent direction of the functional Fx at the point [z, u] was solved. In the case p(z, u) > 0 (point A) this problem is sufficiently easy, as it is a problem of quadratic programming with linear constraints. In the case ip(z, u) = 0 (point B) besides the unknown values fy, i = 1, n + 1, one also has to find the vector-function v(t). This is a more difficult problem, which may be solved with numerical methods, for example, with the hypodifferential descent method as it has been described in the point B.
Remark 3. Note that due to functional H^ structure problem (31) of finding the descent step can be solved analytically. Moreover, problem (30) of finding the descent direction can be solved in finite number of iterations using quadratic programming methods.
Now we can describe the hypodifferential descent method for finding stationary points of the functional Fx(z,u). Choose an arbitrary point [z\,u\] e Pn[0,T] x Pm[0,T] and assume that the point [zk, uk] e Pn[0, T] x Pm[0, T] is already found. If minimum condition (23) holds, then the point [zk,uk] is the stationary point of the functional Fx(z,u) and the process terminates. Otherwise put
[zk+i,ut+i] = [zk, uk] - a.kGk, where the vector-function Gk = G(t, zk,uk) consists of the last n + m components of the smallest by norm hypogradient of the functional Fx at the point [zk ,uk]. The value for the functional Gk is given either by formula (25) if ^(zk,uk) > 0, or by formula (32) if ^>(zk, uk) = 0. The value ak is the solution of the following one-dimensional minimization problem
min Fx([zk,uk] - aGk) = Fx([zk,uk] - a.kGk)-
Then Fx(zk+1,uk+1) < Fx(zk,uk). If the sequence {[zk,uk]} is infinite, then it can be shown that the hypodifferential descent method converges in the sense
\\g(zk,uk)\\ ^ 0 if k
If the sequence {[zk, uk]} is finite, then its last point is the stationary point of the functional F\(z,u) by construction.
Numerical examples. Let us consider some examples of the application of the hypodifferential descent method.
Example 1. Consider the system
¿1 = ¿2, ¿2 = Ul, X3 = ¿4,
¿4 = U2 — 9.8
with boundary conditions
x(0) = [—1, 0, 0,0], x(1) = [0,0, 0,0].
It is required to minimize the functional
l
I = J ul(t) +u2(t) dt.
0
For this problem the analytical solution is known [13], which is as follows:
ul(t) = —12t + 6,
u*2 (t) = 9.8, z^(t) = —6t2 + 6t, z2,(t) = —12t + 6,
zs (t) = 0,
z44 (t) = 0, I (z*,u*) = 108.04.
Table 1 presents the hypodifferential descent method results. Here we put u = [0,1], z(t) = [1,0,0,0] as initial approximation, then x(t) = [—1 +t, 0,0,0]. Table 1 shows that on the 30-th iteration error does not exceed the value 3 x 10~3.
Table 1. Example 1
к I(zk,'U'k) II«* - «fell l|G(^,«fc)ll
1 1.06044 3.47062 3.21367 197.96324
2 0.94422 3.20293 3.22259 707.22868
10 0.34105 1.15682 1.38112 848.13142
20 0.20739 0.72749 0.69893 256.2921
30 108.0425 0.05774 0.02886 0.425
Example 2. Let us consider another example. Let the following system be given
X1 = X2 + ui, X 2 = U2
with boundary conditions
x(0) = [2,0.5], x(1) = [xl(1), 0] and the restriction on the control
i
U2(t) + uV
J ul(t)+ul(t) dt < 1.
It is required to minimize the functional
i
I = J z1(t) dt.
For this problem the analytical solution is also known [7], which is as
[(t) = -
13'
/9 I/9 1
1 /9.2 1
91 13 + 1 Г+2" V 13
z2*(t)
9 1/9 1 Î3t ~ 2VÎ3 ~ 2' 1
I(z*,u*) = -( 1-VT3).
Table 2 presents the hypodifferential descent method results. Here we put u = [0,0], 2(t) = [0,0] as initial approximation, then x(t) = [2,0.5]. Table 2 shows that on the 7-th iteration error does not exceed the value 5 x 10~3.
Table 2. Example 2
к I(Zk,Uk) II«* -«fell l|G(^,«fc)ll
1 1.0 1.00004 0.86826 188.77058
2 0.51873 0.91483 0.90879 76.71471
5 0.00243 0.79148 0.85081 112.2858
6 -0.61768 0.23167 0.23273 0.70711
7 -0.6464 0.08873 0.1132 0.21357
Example 3. Let the following system be given
x i = u, X 2 = x\
u
with boundary conditions
x(0) = [0.25, 0], x(1) = [0.25, x2(1)] and the restriction on the control
1
J u2(t) dt < 1.
0
It is required to minimize the functional
1
I = / z2(t) dt.
This example was considered in the paper [14] with the heavier restriction on the control |w(t)| < 1, t e [0,1], where one may also find the optimal value of the functional
I (z*,u*) =
1
96'
Table 3 presents the hypodifferential descent method results. Here we put u = 10t — 5, z(t) = [10t — 5, (0.25 + 5t2 — 5t)2] as initial approximation, then x(t) = [0.25 + 5t2 — 5t, 5t5 — 12.5t4 + 9.1(6)t3 — 1.25t2 + 0.0625t]. Table 3 shows that on the 8-th iteration error does not exceed the value 5 x 10~3, however, due to the considered weaker restriction on the control and the nonlinearity of the system we can not guarantee that the obtained value is a global minimum in this problem.
Table 3. Example 3
к I(Zk,Uk)
1 8.3333 486.44
2 0.43953 102.93801
5 0.10272 130.33683
7 0.00025 99.303
8 0.01579 0.1127
Example 4. Let us consider one more example. There is a system given
x i = cos(x3), x 2 = sin(x3), x з = u
with boundary conditions
x(0) = [0, 0, 0], x(1) = [3.85, 2.85, x3(1)] and the restriction on the control
5.1228
J u2(t) dt < 1.2807.
0
It is required to minimize the functional
5.1228
I = J z3(t) dt.
0
This example with other boundary conditions was considered in the papers [15, 16].
Table 4 presents the hypodifferential descent method results. Here we put u = 0.5, z(t) = [0.5,0.5,0.5] as initial approximation, then x(t) = [0.5t,0.5t,0.5t]. Analogous to the previous example due to the nonlinearity of the system we can not guarantee that the obtained value is a global minimum in this problem.
Table 4. Example 4
к I(zk,uk)
1 328.4571 373.594
2 232.7861 350.5031
10 27.879 81.23427
15 7.18531 48.2351
20 -0.06627 50.3464
25 0.42832 22.2662
30 -0.157194 0.21303
35 -0.19294 0.0573
Conclusion. The considered problem of constructing an optimal control in the form of Lagrange with integral restriction on control reduces to the variational problem of minimizing a nonsmooth functional on the whole space. For this functional the subdifferential and the hypodifferential are obtained, the necessary minimum conditions are found, which are also sufficient in a partial case. The methods of the subdifferential descent and the hypodifferential descent are applied to the problem. The results are illustrated with numerical examples.
References
1. Karelin V. V. Penalty functions in a control problem. Automation and Remote Control, 2004, no. 3, pp. 483-492.
2. Demyanov V. F., Giannessi F., Karelin V. V. Optimal control problems via exact penalty functions. Journal of Global Optimization, 1998, vol. 12, pp. 215-223.
3. Baranov A. Y., Khazarinov Y. F., Khomenuk V. V. Gradient optimization methods of nonlinear systems of automatic control. Proc. "Applied Tasks of Technical Cybernetics", 1966, pp. 307-316.
4. Kelly H. J. Gradient theory of optimal flight paths. ARS Journal, 1960, vol. 30, no. 10, pp. 947-954.
5. Kelly H. J. Method of gradients. Optimiz. techn. applic. aerospace syst. New York, London, Acad. Press, 1962, pp. 205-254.
6. Moiseev N. N. Jelementy teorii optimal'nyh sistem [Elements of the theory of optimal systems]. Moscow, Nauka Publ., 1975. 528 p. (In Russian)
7. Egorov A. I. Osnovy teorii upravlenija [Basics of the control theory]. Moscow, Fizmatlit Publ., 2004, 504 p. (In Russian)
8. Antipin A. S., Khoroshilova E. V. Linear programming and dynamics. Proc. of the Institute of Mathematics and Mechanic UrO RAS, 2013, vol. 19, no. 2, pp. 7-25.
9. Demyanov V. F., Rubinov A. M. Priblizhjonnye metody reshenija jekstremal'nyh zadach [Approximate methods of solving extremal problems]. Leningrad, Leningrad University Publ., 1968, 178 p. (In Russian)
10. Tamasyan G. Sh. Numerical methods in problems of calculus of variations for functionals depending on higher order derivatives. Journal of Mathematical Sciences, 2013, vol. 188, no. 3, pp. 299321.
11. Demyanov V. F., Rubinov A. M. Osnovy negladkogo analiza i kvazidifferencial'noe ischislenie [Basics of nonsmooth analysis and quasidifferentiable optimization]. Moscow, Nauka Publ., 1990, 432 p. (In Russian)
12. Daugavet V. A. Chislennye metody kvadratichnogo programmirovanija [Numerical methods of quadratic programming]. Saint Petersburg, Saint Petersburg State University Publ. House, 2001, 128 p. (In Russian)
13. Krasovskij N. N. Teorija upravlenija dvizheniem [Motion control theory]. Moscow, Nauka Publ., 1968, 476 p. (In Russian)
14. Kumar V. A control averaging technique for solving a class of singular optimal control problems. Intern. J. Control, 1976, vol. 23, no. 3. pp. 361—380.
15. Srochko V. A., Khamidulin R. G. The method of successive approximations in optimal control problems with boundary conditions. USSR Computational Mathematics and Mathematical Physics, 1986, vol. 188, issue 2, pp. 113-122.
16. Evtushenko Yu. G. Metody reshenija jekstremal'nyh za,d,a,ch i ih prilozhenie k sistemam optimizacii [Methods for solving extreme problems and their application to systems of optimization]. Moscow, Nauka Publ., 1982, 432 p. (In Russian)
For citation: Fominyh A. V. The hypodifferential descent method in the problem of constructing an optimal control. Vestnik of Saint Petersburg University. Series 10. Applied mathematics. Computer science. Control processes, 2016, issue 3, pp. 106-125. DOI: 10.21638/11701/spbu10.2016.310
Статья рекомендована к печати доц. А. П. Жабко. Статья поступила в редакцию 1 февраля 2016 г. Статья принята к печати 26 мая 2016 г.
