CC BY
13URAL MATHEMATICAL JOURNAL, Vol. 3, No. 1, 2017
AN ALGORITHM FOR COMPUTING BOUNDARY POINTS OF REACHABLE SETS OF CONTROL SYSTEMS UNDER
INTEGRAL CONSTRAINTS1
Abstract: In this paper we consider a reachability problem for a nonlinear affine-control system with integral constraints, which assumed to be quadratic in control variables. Under controllability assumptions it was proved in [8] that any admissible control that steers the control system to the boundary of its reachable set is a local solution to an optimal control problem with an integral cost functional and terminal constraints. This leads to the Pontriagyn maximum principle for boundary trajectories. We propose here a numerical algorithm for computing the reachable set boundary based on the maximum principle and provide some numerical examples.
Key words: Optimal control, Reachable set, Integral constraints, Boundary points, Pontriagyn maximum principle.
We consider here the reachable sets of a nonlinear affine-control system with joint integral constraints on the state and the control. The numerical algorithms for constructing approximations of reachable sets of control systems were investigated in many works (see, for example [2, 4, 7, 912, 14, 15, 17]). The properties of reachable sets under integral constraints and algorithms for their construction were studied in [1, 5, 6, 16]. For systems with pointwise constraints on the control it is known (see, for example, [13]) that the control, which steers the trajectory to the boundary of the reachable set, satisfies the Pontryagin maximum principle. In the paper [8] we have considered the reachability problem for a nonlinear affine-control system with constraints on the control variables given by the quadratic integral inequality. Assuming the controllability property of the linearized system, we proved that any admissible control that steers the control system to the boundary of its reachable set is a local solution to an optimal control problem with an integral cost functional and a terminal constraint. This leads to the maximum principle for boundary trajectories. The last result admits a generalization to the case of joint integral constraints on the state and the control given by the inequality
The reachable set in this case may be considered as the solution to the inverse optimal control problem: to find the terminal states reachable from the given initial state by the trajectories satisfying the constraints on the value of the cost functional. The aim of the present paper is to propose a numerical algorithm for computing boundary points of the reachable set. This algorithm is based on the solution of equations following from the maximum principle for boundary trajectories.
xThe research is supported by Russian Science Foundation, project No. 16-11-10146.
Mikhail I. Gusev
Krasovskii Institute of Mathematics and Mechanics,
Ural Branch of the Russian Academy of Sciences, 16 S.Kovalevskaya str., 620990, Ekaterinburg, Russia [email protected]
Introduction
1. Notation and definitions
Further by AT we denote the transpose of a real matrix A, In is an identity n x n-matrix, 0n is a zero n x n-matrix, 0 stands for a zero vector of appropriate dimension. For x, y € Rn let (.x,y) = xTy denotes the inner product, xT = (xi,... ,xn), ||x|| = (x,x)a be the Euclidean norm, and Br(x): Br(x) = {x € Rn : ||x — x|| < r} be a ball of radius r > 0 centered at x. For a set
d/
5cK™ let dS be the boundary of S; ~q~(x) is the Jacobi matrix of a vector-valued function /(x).
For a real k x m matrix A a matrix norm is denoted as || A ||. The symbol Rnxr denotes a space of n x r real matrices, the symbols Li, L2 and C stand for the spaces of summable, square summable and continuous vector-functions respectively. The norms in these spaces are denoted as || ■ ||Li,
We consider the control system
x(t) = A(t,x(t)) + /2(t,x(t))u(t), x(t0) = x0, (1.1)
on the fixed interval [t0, t1], where t0 < t < t1, x € Rn, u € Rr, /1 : Rn+1 ^ Rn, /2 : Rn+1 ^ Rnxr are continuous mappings.
The functions /1 and /2 are assumed to be continuously differentiable in x and satisfying the following conditions:
|| /1(t,x) || < l1(t)(1+ || x ||), || /2(t,x) ||< I2(t), (1.2)
where 11(-) € L1, 12(-) € L2. Under these assumptions for any u(-) € L2 there exists a unique absolutely continuous solution x(t) of system (1.1) which satisfies the initial condition x(t0) = x0 and is defined on the interval [t0,t1 ].2
Denote as J(u(-)) the following integral functional
f ti
J(u(-)) ^ (Q(t,x(t))+ uT(t)R(t,x(t))u(t))dt.
J to
Here x(t) is a solution of system (1.1) corresponding to the control u(t) and the initial vector x0. The function Q(t,x) and the positive definite symmetric matrix R(t,x) are assumed to be continuous on [t0,t1] x Rn and satisfying the inequalities Q(t,x) > 0, uTR(t, x)u > a||u||2 for some a > 0 and any (t,x,u) € [t0,t1] x Rn x Rr. Define the set
U = {u(-) € L2 : J(u(-)) < ^2},
where ^ > 0 is a given number, and let P be a m x n full rank real matrix, m < n. Denote by G(t1) the (output) reachable set of the system (1.1) at the time t1 for the fixed x0 and the integral constraints:
G(t1) = {y € Rm : 3u(0 € U, y = Px(t1 ,u(-)^,
where x(t,u(-)) is a trajectory of system (1.1), corresponding to u(-). The reachable set is a compact set in Rm, but it may be empty. Recall the following definitions: the linear control system
x(t) = A(t)x(t) + B(t)u(t), t € [t0,t1 ], x(t0)= x0,
a) is said to be controllable on [t0 ,t1] with respect to the output y = Px if for any y1 € Rm there exists a control u(-) € L2 that transfers the system from the zero initial state x(t0) = 0 to the final
2We use the same notation for the space L2 in the case of a scalar function ¿2(-) and a vector-function u(-).
state x(t1) such that Px(t1) = y1;
b) is said to be the linearization of the system X = F(t,x,u) along the trajectory x(t),u(t) if
dF dF
A(t) = —(t,x(t),u(t)), B(t) = —(t,x(t),u(t)).
2. The Maximum Principle for Boundary Trajectories 2.1. Extremal Properties of Boundary Points
Let us show that any admissible control that steers the control system to the boundary of its reachable set is a local solution to an optimal control problem with an integral cost functional and terminal constraints.
Theorem 1. Assume that:
1) y1 € dG(t1);
2) u(-) € U is a control that steers the system from the state x(t0) = x0 to the point x(t1), Px(t1) = y1, x(t) is the corresponding trajectory;
3) the linearization along (x(t),u(t)) of system (1.1) is controllable on [t0,t1 ] w.r.t. output y = Px;
Then there exists a > 0 such that J(v(-)) > i2 for any v(-) € B(u(-),a) C L2 satisfying the condition Px(t1) = y1. Since J(u(-)) < ¡2, this implies that J(u(-)) = ¡2 and the control u(-) provides a local minimum in the optimal control problem
J(u(-)) ^ min, u(') € L2, x(to) = x0, Px(t1) = y1 (2.3)
with terminal constraint Px(t1) = y1.
Proof. The proof follows the scheme of the proof of the Theorem 1 [8] and uses the Graves theorem [3]. □
Since the local minimum in L2 admits the needle variations of the control, the local L2-minimizer satisfies Pontryagin's maximum principle. Introduce the Pontryagin function (Hamiltonian) associated with (2.3)
H (p,t,x,u) = -p0f0(t,x,u) + ,pT(f1(t,x) + f2(t,x)u),
p0 > 0, f0(t,x,u) = Q(t,x)+uTR(t,x)u. Assume additionally that Q(t,x), R(t,x) are continuously differentiable in x. A locally optimal control for (2.3) satisfies the maximum principle: there exist p0 > 0, l € Rm, (p0, l) = 0, and a function p(t) such that
H (p(t),t,x(t),u(t)) = max H (p(t),t,x(t),v),
veRr
dH d f
p(t) = ~ — (p(t),x(t),u(t)) = -AT(t)p(t) +p0^(t,x(t),u(t)), p(t i) = PTl.
Since the terminal constraints are regular (rankP = m), we have p0 + ||p(t)|| = 0, t € [t0,t1]. As previously, we denote here by (A(t),B(t)) the matrices of the linearization along (x(t),u(t)) of system (1.1). Applying the maximum principle to the solution of problem (2.3) we come the following
Corrolary 1. Suppose that u(t) satisfies the assumptions of Theorem 1. Then there exist l € Rm, l = 0 and a function p(t) such that
pit) = -^(p(t),x(t),u(t)) = -AT(t)p(t) + ±C2A(t,x(t),u(t)), p(t 1) = PTl,
u(t) = R-1(t,x(t))/T(t,x(t))P(t), t € [t0,t1].
Proof. If a pair (A(t), B(t)) is controllable w.r.t. y = Px, then p0 > 0. Indeed, if it turned out that p0 = 0, then p(-) is a non zero solution of the equation
p(t) = —AT(t)p(t), p(t1) = P T1,
and from the maximum principle we would obtain
pT(t)B(t)u(t) = max pT(t)B(t)v
almost everywhere in t. The last is valid only if pT(t)B(t) = 0. Represent p(t) in the form p(t) = XT(t1,t)PTl, then ||1TPX(t1,t)B(t)||2 = 0, t € [t0,t1 ]. Integrating both sides of the last equality over [t0,t1], we get 1TV1 = 0. This contradicts to the controllability of (A(t),B(t)) w.r.t. y = Px, since I / 0. Thus we can take po = from the maximum principle it follows that Hu(p(t), t, x(t), u(t)) = 0, hence u(t) = u(t, x(t),p(t)), where u(t,x,
p) = R-1(t,x)/2T(t,x)p. □
2.2. Algorithm
Let us describe the following algorithm for calculating boundary points of reachable sets based on the results of previous subsection. Further we assume that P = [Im, 0] if m < n or P = In if m = n. In this case the transversality conditions p(t1) = PTl take the form pi(t1) = 0, i = m + 1,.., n. Letting
x0(t) = /0(t,x(t),u(t)), x0(t0) = 0,
we get J(u(-)) = x0(t1). Substituting u(t, x,p) into differential equations, we obtain the following system
x(t) = /1(t,x(t))+ /2 (t, x(t))u(t, x(t), p(t)), x(t0) = x0,
p(i) =-|f-H{p{t),x{t),u{t,x{t),p{t))), p(t0) = q, (2.1)
x 0(t) = /0(t,x(t),u(t,x(t),p(t))), x0(t0) = 0.
Denote by X the following (2n + 1)-column vector X = [x;p; x0] and write equations (2.1) as the system
X(t) = F(t, X(t)), X(t0) = [x0; q; 0], (2.2)
By F(t, X) we denote the right-hand side of (2.1). Since x0 is fixed, the solution of (2.2) depends only on the vector q € Rn, denote this solution as X(t, q) = [x(t, q); p(t, q); x0 (t, q)]. These functions have continuous derivatives Xq(t,q) with respect to q, which can be found by integrating the linearization of (2.2) along the trajectory X(t, q)
dF
Xq(t,q) = —(f,X(f,q))Xq(t,q), Xq(f0, q) = [0ra; /„; 0]. (2.3)
The integration of equations (2.1)and (2.2) over the interval [t0,t1 ] may be performed simultaneously. To this end, we unite both systems into one system of dimension (2n + 1)(n + 1)
X(t) = F(t, X(t)), X(t0) = [x0; q;0],
dF (2.4)
Xq(t, q) = —(t,X(t,q))Xg(t,q), Xg(t0,q) = [0ra;Jra;0].
Consider the following continuously differentiable functions
00 (q) = x0(t1,q) - I2, 0i(q) = pm+i(t1,q), i = 1,...,n — m,
their derivatives in q may be found by numerical integration of differential equations (2.4). The calculations of boundary points require the solution of the system of equations
0i (q) = 0, i = 0,...,n — m, (2.5)
and also the integration of system (2.1) with zeros of system (2.5) as the initial points for (2.1). In case m = n the system (2.5) consists of a single equation 00(q) = 0.
Let us describe a simple version of the algorithm for calculating zeros of 0i(q) in the case m = n = 2. Represent q € R2 in polar coordinates: q1(d) = r(d)cos(d + d0) + q0, q2(d) = r(d) sin(0 + d0) + q20. Here r(d) is a distance from a reference point q0 and d is an angle between q — q0 and the reference direction q = (cos d0, sin d0). Differentiating the identity 00(q(d)) = 0, we get a differential equation for r(d)
r(e) r(e) ^iq{e)) s[n(e + 9o) ~ ^iq{e)) cos(e + 9o) o < o < 2ir (2 6) {) { )iPoqMд))coS(д + дo)+iPoq2(qmMO + Oo), - - '
To start the solution we use a one-dimensional search procedure for finding the root of equation 0(q0 + rq) = 0 and after this take this root as the initial state for differential equation (2.6).
3. Examples
Here we illustrate the above procedure for two examples of 2-dimensional control systems.
E x a m p l e 1. Consider the Duffing equation
x 1 = x2, x2 = ^(x1) + u, t € [0,t1], x1(0) = 0, x2(0) = 0, (3.1)
^(x1) = — ax1 — ftxf, a, ft > 0, which describes the motion of nonlinear stiff spring on impact of an external force u. Consider the integral constraint on the state and the control
f11
/ (axf(t) + 6x2(t) + u2(t))dt < 2, 0
where a, b are nonegative parameters and take P = I2.
It is easy to verify that the controllability assumptions of Theorem 1 are satisfied here. Really, consider any trajectory (x(t),u(t)) of (3.3). The linearization of (3.3) along (x(t),u(t)) has the matrices
A(t) = UW, 0 ■ B(t) = (J
An adjoint system = — AT(t)s is as follows
S1(t) = —<p (x1(t))S2(t), S2(t) = —S1(t).
Thus, the identity sT(t)B(t) = s2(t) = 0 for t € [t0,t1] implies s1(t) = 0. This means the controllability of the pair (A(t),B(t)).
Figure 1. Reachable sets for different values of t\.
.........
• * » ' * • * * * ♦* • • « -
* » ' * ..... %
....... * • • * . . * «
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
Figure 2. Reachable sets for different values of a, b.
The system (2.4) takes the following form
X1 = X2,
X 2 = p(X1)+ X4,
X3 = aX1 — <p (X1)X4,
X4 = bX2 — X3,
X 5 = aX2 + bX22 + X42,
. 1 2 (3.2)
X5+i = X6+i,
X6+i = (X1)X5+i + X8+i,
X7+i = aX5+i — (X1)X4X5+i — (X1)X8+i, X8+i = bX6+i — X7+i,
X 9+i = 2aX1X5+i + 2bX2X6+i + 2X4X8+^
In equations (3.2) i = 1,6, so (3.2) is a system of 15-th order. Integrating this system over [0, t1] for initial state XT(0) = (0, 0,q1 ,q2, 0, 0, 0,1, 0, 0, 0, 0, 0,1, 0) we get
Mq)=X5(h,q)-/x2, ^(q)=X10(t1,q), ^(q) = X15(t1} q), qT = (Ql, q2).
Since x(0) = 0 and ^(x1) is an odd function having even derivative it is not difficult to prove that the set {q : 00(q) = 0} is symmetric with respect to the origin. In this case it is natural to take the reference point q0 = 0. As the reference direction we choose q = (1,0). The results of numerical simulations for the case a = = 10 are shown in Fig. 1-2.
The Fig. 1 shows the plot of the reachable sets boundaries for t1 = 0.5, 1, 1.5, and 2 respectively, and for a = 0, b = 0. The reachable sets boundaries for the values of a = 0, b = 0; a = 5, b = 10; a = 30, b = 15 and t1 = 2 are presented in Fig. 2.
E x a m p l e 2. Consider the following system [16]
x 1 = x2, x2 = f(x1)+ u, t € [0,2n], x1(0)=0, x2(0) = 0, (3.3)
where ^(x1) = — sin x1. The integral constraint on the state and the control are given by the inequality
/ (ax2(t) + bx2(t) + u2(t))dt < 2
2.5r 21.51 -0.50-0.5-1 --1.5-2-
1
-3 -2 -1 0 1 2 3 4 5
-0.2 0 0.2 0.4 0.6 0.8 1
q1
Figure 3. Reachable sets for different values of a, b
Figure 4. Zero-level lines of V'o(q) for different values of a, b
-3 -2 -1
12 3 4
0 1 2 3 4 5 6 7
8
Figure 5. Reachable sets for different values of
Figure 6. Graph of the function r(0).
as in Example 1. As above the controllability assumptions of Theorem 1 are satisfied for the considered system.
The results of numerical simulation are shown in the Fig. 3-6. The Fig. 3 shows the plot of the reachable sets boundaries for t1 = 2, and for a = 0, b = 0; a = 0.1, b = 0; a = 0.5, b = 0.1 respectively. This plot demonstrates that reachable sets are nonconvex for a = 0, b = 0 and became convex under increase of parameters a, b.
The next plot (Fig. 4) exhibits the zero-level lines of ^0(q) corresponding to the curves of Fig. 3.
The Fig. 5 demonstrates the dependence of reachable sets on the value = 0.5,1,1.5,2,2.2. It shows that reachable sets that are convex for small loose their convexity as increases (see [16]). In this example the method fails for > 2.2 because a numerical integration of (2.6) unable to meet integration tolerances. Note that the considered procedure may by applied if the zero-level line ^0(q) = 0 is a differentiable curve. Differentiability can be violated in the points where ^0qi (q) = ^0q2 (q) = 0 or the right-hand side of (2.6) is singular. The graph of the solution of (2.6) corresponding to the value = 2.2 is shown in Fig. 6.
15
4. Conclusion
This paper describes an algorithm for computing the boundaries of the reachable sets under joint integral constrains on state and control variables. The reachable set may be considered here as the solution to the inverse optimal control problem: to find the terminal states reached from the given initial state by the trajectories satisfying the constraints on the value of the cost functional. The Pontriagyn maximum principle for boundary trajectories is applied to construct a numerical algorithm for computing the boundary points. The results of numerical simulation for two examples of second order nonlinear control systems are presented.
REFERENCES
1. Anan'ev B.I. Motion correction of a statistically uncertain system under communication constraints // Automation and Remote Control, 2010. Vol. 71, no. 3. P. 367-378. DOI: 10.1134/S0081543810060039
2. Baier R., Gerdts M., Xausa I. Approximation of reachable sets using optimal control algorithms // Numerical Algebra, Control and Optimization, 2013. Vol. 3, no. 3. P. 519-548. DOI: 10.3934/naco.2013.3.519
3. Donchev A. The graves theorem revisited // Journal of Convex Analysis, 1996. Vol. 3, no. 1, P. 45-53.
4. Filippova T.F., Matviichuk O.G. Algorithms to estimate the reachability sets of the pulse controlled systems with ellipsoidal phase constraints // Automation and Remote Control, 2011. Vol. 72, no. 9. P. 1911-1924. DOI: 10.1134/S000511791109013X
5. Guseinov K.G., Ozer O., Akyar E., Ushakov V.N. The approximation of reachable sets of control systems with integral constraint on controls // Nonlinear Differential Equations and Applications, 2007. Vol. 14, no. 1-2. P. 57-73. DOI: 10.1007/s00030-006-4036-6
6. Guseinov Kh.G., Nazlipinar A.S. Attainable sets of the control system with limited resources // Trudy Inst. Mat. i Mekh. Uro RAN, 2010. Vol. 16, no. 5. P. 261-268.
7. Gusev M.I. Internal approximations of reachable sets of control systems with state constraints // Proc. Steklov Inst. Math., 2014. Vol. 287, Suppl. 1. P. 77-92. DOI: 10.1134/S0081543814090089
8. Gusev M.I., Zykov I.V. On extremal properties for boundary points of reachable sets under integral constraints on the control // Trudy Inst. Mat. Mekh. UrO RAN, 2017. Vol. 23, no. 1. P. 103-115. (in Russian) DOI: 10.21538/0134-4889-2017-23-1-103-115
9. Kostousova E.K. On the boundedness of outer polyhedral estimates for reachable sets of linear systems // Comput. Math. and Math. Phys., 2008. Vol. 48. P. 918-932. DOI: 10.1134/S0965542508060043
10. Kurzhanski A.B., Varaiya P. Dynamic optimization for reachability problems // J. Optim. Theory Appl., 2001. Vol. 108, no. 2. P. 227-251. DOI: 10.1023/A:1026497115405
11. Kurzhanski A.B., Varaiya P. On ellipsoidal techniques for reachability analysis. Part I. External approximations // Optim. Methods Software, 2002. Vol. 17, no. 2. P. 177-206. DOI: 10.1080/1055678021000012426
12. Kurzhanski A.B., Varaiya P. Dynamics and control of trajectory tubes. Theory and computation. Basel, 2014. DOI: 10.1007/978-3-319-10277-1
13. Lee E.B., Marcus L. Foundations of optimal control theory. NY-London-Sydney: John Willey and Sons, Inc., 1967.
14. Lempio F., Veliov V.M. Discrete approximations of differential inclusions // GAMM Mitt. Ges. Angew. Math. Mech., 1998. Vol. 21. P. 103-135.
15. Patsko V.S., Pyatko S.G., Fedotov A.A. Three-dimensional reachability set for a nonlinear control system // J. Comput. Syst. Sci. Int., 2003. Vol. 42, no. 3. P. 320-328.
16. Polyak B.T. Convexity of the reachable set of nonlinear systems under L2 bounded controls // Dynamics of Continuous, Discrete and Impulsive Systems, Series A: Mathematical Analysis, 2004. Vol. 11. P. 255-267.
17. Sinyakov V.V. Method for computing exterior and interior approximations to the reachability sets of bilinear differential systems // Differential Equations, 2015. Vol. 51, no. 8. P. 1097-1111. DOI: 10.1134/S0012266115080145
