автореферат диссертации по информатике, вычислительной технике и управлению, 05.13.18, диссертация на тему:Стохастическое управление со стабилизирующими лидерами

Уральский государственный университет

На правах рукописи

ЧОЙ Ен Сан

СТОХАСТИЧЕСКОЕ УПРАВЛЕНИЕ СО СТАБИЛИЗИРУЮЩИМИ ЛИДЕРАМИ

Специальность 05.13.18- математическое моделирование, численные методы и комплексы программ

Диссертации на соискание учёной степени кандидата физико-математических наук

Научный руководитель: Доктор физ-мат наук, профессор А.Н.Красовский

Екатеринбург 2001

Ural State University

On the right of the manuscript

CHOI Young Sang

STOCHASTIC CONTROL WITH THE LEADERS- STABILIZERS

Speciality 05.13.18- mathematical modeling, numerical methods and software complexes

PhD thesis

Scientific advisor:

Dr. Sci., professor A.N.Krasf" 1

Ekaterinburg - 2001

Contents

Introduction 4

Chapter 1. Mutual tracking of stochastic motions 17

1. Equation of the Motion 17

2. The z-model-leader 22

3. Control actions for the x-object2 24

4. Disturbance actions for the x-object 26

5. Constructions of the control actions for x-object and z-model. Extremal shift. 32

6. The mutual tracking in combined process 35

7. The proofs of the theorems and lemmas 39

8. Geometrical interpretation of Lemma 1 50

9. Asymptotically stable system 54

10.The model problem 1 61

Chapter 2. Application of mutual tracking algorithms to

some control problems and differential games 64

11 .Tracking along an arbitrary curve 65

12.Tracking u-control with some quality index for the zu -model -

,;u-leader 71

13.Optimal и -control for the x -object with minimizing ensured

results of the given quality index 79 14.Optimal v-control for the x-object with maximizing ensured result

of the given quality index 84

Introduction

The Mathematical theory of control, essentially developed during the last decades, is used for solving many problems of practical importance. The efficiency of its applications has increased in connection with the refinement of computer techniques and the corresponding mathematical software. Real-time control schemes that include computer-realized blocks are, for example, attracting ever more attention. The theory of control provides abstract models of controlled systems and the processes realized in them. This theory investigates these models, proposes methods for solving the corresponding problems and indicates ways to construct control algorithms and the methods of their computer realization.

One of the branches of modern theory of control is the investigation of the problems that are usually called the problems of stochastic control. These problems can be "roughly" classified according to the following features:

1. Employment of probability problems is determined in general by the stochastic nature of controlled object itself. It seems that many problems on Biology, Medicine, Economics, Sociology etc can be referred to this category of problems.

2. Feedback control process that is based on the current informational image y[t] of the system faces difficulties because informational image gives deformed information about the real current condition of controlled object. One often try's to overcome such difficulties explaining these deformations on the base of probability

concepts. Thus the employment of methods of the probability processes theory becomes natural.

3. The probability models are generally used because of artificial inclusion of probability mechanisms into the control scheme. Even in the case of quite satisfactory handling of original controlled object and the quality index of process in deterministic form we can hope on any improvement of the quality of process.

The assumed work can be referred to the class of the problems that are connected with 2 and 3 from this list of features.

Let's name shortly some of those general theories and results that determined the modern control theory, particularly in the branch connected with this work.

On account of the boundary for this text it is inevitably that the proposal historical transfer has many essential gaps. In advance we apologize for this.

Let us mention the book of Cine Cue-cane [9] in which it was formulated the conception of control and observation for the dynamical system according to the feedback control principle and with regard to the corresponding mathematical apparatus and elementary basis in the technical cybernetic that was developed to that time. Note that in this extending monograph already the essential place took up the optimal control problems in stochastic interpretation.

As most essential step for the development of the optimal control theory became the work of L.S.Pontryagin and his collaborators V.G.Boltyanskii, R.V.Gamkrelidze and E.F.Mishchenko [58]. The result of this work - the maximum principle of Pontryagin for the solution of the problems on the optimal programming control. This principle rendered and render now and continue its strong influence now strong

influence to the all consequent development of the mathematical control theory.

It should be noted руку the theory of dynamical programming that in its foundation has the works of R.Bellman [5]. The dynamical programming in significant extent determines the ideology of the recent statement of the problem of control by the feedback control principle and the methods of its solutions.

In general mathematical theory of control and observation for the dynamical systems, after the classic works ofN.Winner and S.Shennon most well known and influential became the works of R.Kalman [26]. Among them the big influence in science and applications renders the construction of Kalman filter.

Especially it must be called here that branch of the control theory, which is related to the control problems under the conditions of uncertainty and conflict. They are often formalized in the form of constituents in some differential game. Here among basic works one has to call the book of R.Isaacs [24]. In stochastic interpretation of the differential games among the initiating works we have to call the work by I.V.Girsanov [19].

The methods of the solution of the differential games are often connected with the differential equations in partial derivatives in type of Hamilton-Jacoby. But under the conditions, most interesting for the control theory, for these equations the smooth classic solutions can be obtained only in some rare cases. In order to overcome these difficulties it requires the utilization of the nonsmooth analysis [12, 14, 60] and the generalized solutions. The history of these solutions is connected with the works of S.N.Kruzhkov [38]. At recent time the generalized equations type Hamilton-Jacoby connected with the works of V.P.Maslov and his collaborators [45]. Now very intensive development

of the theory of the minimax solutions of A.I.Subbotin [62] and the theory of adequate viscosity solutions connected with the works of P.L.Lions and M.G.Crandall [13] is working out.

As it was said above the actuality of applications, namely the stochastic control, is stipulated in significant degree by the extensive spreading of the stochastic models for the study of the real processes which have place in techniques, natural science and social science. These models are connected with the consideration of the differential equations containing indeterminate and poorly predicted parameters which are treated as random functions. At the same time, one of the main conditions for the physical realizability of the process is its stability, hence the special attention is spared to such procedures of stochastic control that must be stable and satisfactorily resistance to the dynamical, informational and other disturbances, i.e. - provide the satisfactory result under the action of this disturbances. Note, that one of the difficulties consists in the fact, that the restrictions for the disturbances, as the rule, it is convenient to form on the base of its mean values - mathematical expectations from its norms or the norms squared, and the required result it is desirable to obtain with the probability as close as possible to unit.

In connection with the discussed above, it is necessary to say about the outstanding works of A.N.Kolmogorov by the theory of probability and its recent applications. We shall call here the works of A.N.Shiryaev and R.Sh.Liptser [42] and their collaborators.

In connections with the problems of stability it is necessary to call the classic results of A.M.Lyapunov and its numerous extensions as for the deterministic, so and for the stochastic systems.

Speaking more exactly, we say that the investigations in this work are based on the approaches, methods and constructions from the theory

of stochastic processes, theory of stability, theory of optimal control, tracking and observation of the processes and so on, which were proposed and are developed in the works of Albreht E.G., Balakrishnan A.V. Basar T, Batuhtin V.D., Bellman R., Bensoussan A., Boltyanskii V.G., Bryson A.E., Chencov A.G., Chernous'ko F.L., Chikrii A.A., Clarke F., Crandall M.G., Dem'janov V.P., Cine Cue-cane, Elliot R.J., Friedman A., Fleming W.H., Gabasov R, Gamkrelidze R.V., Girsanov I.V., Grigorenko N.L., Has'minskii R.Z., Ho Y-S., Isaacs R., Kabanov Yu.M., Kalman R., Kalton N.J., Karlin C., Kirillova F.M., Kleimenov A.F., Kolmanovskii V.B., Kolmogorov A.N., Krasovskii N.N., Krilov I.A., Kruzhkov S.N., Kryazhimskii A.V., Kurzhanskii A.B., Kushner H.J., Leitman G., Lions P.L., Logunov V.N., Maslov V.P, Melikyan A.A., Mil'shtein G.N., Mishchenko E.F., Myshkis A.D., Nikolskii M.S., von Neumann J., Olech C., Olsder J., Osipov Yu.S., Patsko V.S., Petrosyan L.A., Petrov N.N., Pojarickii G.K., Pontryagin L.S., Pshenichnii B.N., Shiryaev A.N., Shorikov A.F., Subbotin A.I., Subbotina N.N., Tret'yakov V.E., Ushakov V.N., Varaiya P., Warga J., Zelikin M.I. and many other authors.

The investigation of the problems of stochastic control are conjugated also with such difficulties, that analytical solution of considered problems can be obtained only in exceptional cases. Thus it is actual the development of the algorithms of numerical-computer modeling of such problems and the creation of the corresponding programming product.

The statement of the considered here problem and the methods of its solutions are based on the mathematical formalization for the problems on the optimization of the process under the criterion of ensured result and in particular for the problems on the control in conception of differential games that is developed in Ekaterinburg at

Ural State University ^and Institute of Mathematics and Mechanics of the Urals Branch of Russian Academy of Science.

Aim of the work.

Investigation of combined control processes which are composed of basic real x-object and its appropriate z-model-leaders or observer-models that are realized in computer. Statement and solution of the problem of mutual stable tracking of the motions of the stochastically controlled object and model under the sufficiently small informational and dynamical errors. Application of the elaborated algorithms of stable motion tracking to differential game problems. Developing of the software programs on the basis of the constructed algorithms. The main aim of the work is to provide concretely the general results of the optimal stochastic control theory, including ,£uch type of control in the framework of the differential game theory basing on the simplified approximating constructions. This should promote to the effective utilization of the corresponding methods and results for elaboration of the effective computer procedures of control and computer simulation of the process. The work also has the aim to promote teaching process and popularization of the corresponding theories for the specialists in applied areas.

Contents of the work

In the first chapter the statement of the problem about the stable mutual tracking of the motions of real x-object and its computer z-model is given. The equations, describing the evolution in time t of these objects that are nonlinear in the control parameters и and the "basic" uncertain disturbances are given. The restrictions for the control actions and disturbances an so on also are given. It is important that for the

considered controlled dynamical system the so-called saddle point condition in the small game that is called often as the Isaacs-Bellman condition [5, 24] is not valid. As it was mentioned above, this fact essentially defines application of the stochastic mechanism for constructing the control actions, and, sometimes - stochastic interpretation of basic uncertain disturbances.

This stochastic control mechanism is working out on the feedback

control principle on the basis of the available distorted informational

*

image x [t] = x[/] + Ajnf [/] for the real current state x[/] of the controlled object. Besides, x-object is subjected to uncertain basic disturbance v[f] and additional dynamical disturbance hcnn\t\ •

Informational random disturbance A^ [/] and the random disturbance

hrfinlt] are assumed small in measure. The control mechanism should

provide closeness of the motions of x-object and z -model in some combined probabilistic {x,z} process. The considered in this work control mechanism is based on the so called minimax and maximin extremal shifts. Sense of these extremal shifts consists in the following: minimax shift has the aim of approaching the motion xj/] to the motion z[/], and the maximin shift approaches the motion z\t\ to the motion x[/]. We investigate the problem in approximate discrete in time t scheme with small step At = -.

The first theorem states that this method of the extremal shifts provides at every current moment t^ = [0,Г] .the desired

closeness of the states x|7] and z\t] in the sense of satisfactory small

estimation of the — } if only the following estimations

are valid:

Е{ |Д

inf < 8т{, E{\hdjn(t)\} < Sdin, At <S, —z^qI ^ S0. (I)

Here ... is the Euclead norm of the vector, E{...} is the mathematical

expectation, positive numbers S-mf, S, Sq .are sufficiently small.

However in many control problems it is desirable that the closeness of the motions x[t] and z[t] would be small not only with

i |2

respect to the estimation of the value ~ zl//t| ) or

] - z\tk ]|} at current moments tbut also there would be closeness every or almost every realization x\t,CO] to the corresponding realization z[t,0)\ on the whole interval |/o,i9].

Theorem 2 gives us the corresponding estimation. It states that under the mentioned in Chapter 1 conditions, for any number /3 < 1 the considered control mechanism provides the desired closeness of the motions and z[t] with respect to the estimation

max x(7, of\ — z[l, oj, i.e. we have satisfactory small of the value

0 <t<T 1

max \x[t,co]- z[t,aA with the probability P> fi, if the estimations 0 <t<T

(I) are valid. In the end of Chapter 1 one can find the illustrative examples for not very complicated model examples.

Material of this work essentially utilizes the ideas, for example, of the books [34, 36]. It is necessary to note that theorem 1 and theorem 2 correspond to analogical general theorems from these books where the theorems are presented in the rather formalized form. However here theorem 1 and theorem 2 are formulated and proved in informative and

visual forms for describing and constructing motions of x -object and z-model-leader constructed on the basis of the discrete approximate scheme which is based on the difference equations. This allows one to simplify the corresponding proofs and constructions, and make them more visual. From another side - and this is here the most essential moment - this allows to realize the corresponding control algorithms in computer included in the loop, and gives us the possibility of elaboration the constructive control procedures in the form of the effective software toolboxes.

In the second chapter we apply the results of the first chapter (the scheme of forming the mutual stochastic tracking of x-object and z-model) to some typical differential games on minimax and maximin of some positional functional p that defines the quality index of the control process.

In particular, one could consider one specific class of the differential games that combined differential games with matrix games.

The main resume of this chapter is the following. If for some game for x-object the saddle point condition is not satisfied and to this game corresponds (according to some rule) the appropriate differential game for the appropriate z-model, then the solution of the original differential game for x-object can be obtained according to some scheme of the informational exchange between elements of the stable mutual stochastic tracking in combined {x, z} -motion and elements of the corresponding solution of the game for z-model. It is important to note that for the game for the computer z-model the saddle point condition in a small game is already valid.

It is necessary to note again that this procedure corresponds to the general procedures that described for example in the book [34, 36].

However the auxiliary differential game in pure strategies for models-leaders is constructed here not on the basis of и-stable and v-stable bridges as it is constructed in that book [34]. But in this work the auxiliary differential game in pure strategies for models-leaders are

constructed on the basis of so called the accompanying points Wy [^ ] and "И^[tfc] which represent some abstract auxiliary motions wu[t] and

This again defines the construction that is convenient for

realization of the control algorithms on the basis of the virtual computer models included into the loop of feedback control.

It is also necessary to note that the results of the game on the basis of the considered in the work procedure are related to an arbitrary positional functional.

The illustrative examples of not very complicated differential games are given. However, in these games we discuss sufficiently clearly all circumstances and constructions.

Summary. For the considered differential games is given approximating scheme of the forming of whole algorithm for the solution of the problem for one or another opponents. This scheme is constructive and consist of three constructive elements: 1. The block of mutual stochastic tracking of real x-object and its computer z-model.

2. The block of the deterministic solution of an auxiliary deterministic game for the virtual-computer z-model.

3. The block of the pasting together the 1-st and the 2-nd blocks on the base of exchange: of the current value of the phase vector z\t^ ] by the transition it from the 1-st block to the 2-nd block and by the

transition of the current value of the control action that

generated in the 1-st blok for the z-model from the 1-st block to the 2-nd block.

Therefore, it has determined the possibility of the constructive formation of the corresponding programming product.

Scientific novelty.

Investigated and solved the problem of the stable mutual tracking of stochastically controlled dynamical object (nonlinear with respect to disturbances) and some computer model-leader. Solution of the problem is justified in details in geometrical interpretation and analytical form under dynamical and informational disturbances. Informational random error (that is included into the informational image) and the dynamical disturbance should be rather small. It is shown that the closeness between the motions of the object and the model is ensured with the probability that is closed to one. The complete solving algorithms of control that have combined some blocks partial algorithms are given. The combination of such blocks programs is used for the investigation of some differential games. On this basis software toolbox for MATLAB could be developed. The toolbox allows one numerically simulate motions of the system and models under various values of the parameters and informational disturbances.

In order to avoid misunderstandings it is important to make the following remark. The results presented in this work are based, in essence, on the results and methods of the books [34, 36] where these results are given in the very general form. However in this work these results are described and proved for the approximating scheme of the motions x-object and z-model, and, thus, are oriented on developing the software programs for the constructing control procedures on the

computer included into the loop, and also for the convenient computer modeling of the process. Besides all presented in this work, results have sufficiently visual character and oriented on familiarity with the corresponding results and methods of the theory of stochastic optimal control for sufficiently wide classes of readers, including engineers. As shows the experience, familiarity with general results, presented on the basis of the rigorous and formal theory and methods of the highest probability theory is sufficiently complicated.

Theoretical and applied significance of the work.

The results of the work can be used for further developing of the control algorithms for dynamical systems under the conflict and uncertainties. On the basis of the elaborated algorithms and software one can numerically investigate problems of control for real systems and processes.

Approbation of the work.

The results of the work were presented and discussed at the International IF AC conference "Non-smooth and discontinuous problems of control and optimization" (Chelyabinsk, June, 1998); at International conference on automatic control (Pusan, Republic of Korea, October, 1998); at seminars of the Automation control center (Seoul National University); at seminars of the Dynamic system department of the Institute of Mathematics and Mechanics of the Urals Branch of Russian Academy of Science; at seminars of the Chair of theoretical mechanics of the Ural State University.

Publications.

Main results of the work are published in 2 papers [32, 33].

Structure of the work.

The work consists of introduction, 2 chapters and appendix. Total volume of the work is 92 pages. The work contains 14 figures and 68 bibliographic references.

Chapter 1

Mutual tracking of stochastic motions

1 Equation of the Motion

We will consider the control dynamic system that is described by the ordinary vector differential equation

x = A(t)x + f(t,u,v) + hdin(t\ 0 <t<T (1.1)

which is non-linear with respect to the actions u and v. Here x is n-dimensional phase vector, t is a time, = 0 is a fixed time moment of

the beginning of the control process, T is a time moment of the end of this process; и is r-dimensional vector of the control, v is s-dimensional vector of the disturbance; Aft) is matrix-valued function piecewise continuous in te [0,7], f(t,u,v) is vector-function piecewise continuous in t, for every fixed t function f(t,u,v) is continuous in и and v. The points of discontinuous /(•) are independent of u, v. The function f{t,•) is continuous on the right and has finite limits on the left in these points discontinuous.

Taking into account the computer simulation, it is assumed that admissible value of и and v are restricted by the inclusions

W€? (1.2)

veQ

(1.3)

In this work we consider the case when the sets P and Q have a

form

(1.4)

Q = J}

(1.5)

where M and N are given numbers.

Also we assume that r-dimensional vectors j=l, ...,M in (1.4) satisfy the conditions

max

/=1,

U

Ш

= M

(1.6)

where M is some given number.

Here I и | denotes the Euclidean norm of vector u, i.e.

и

= (uf + u\ +... + и/)

2 \l/2

(1.7)

In (1.5) the .y-dimensional vectors vm, / = 1, conditions

yN satisfy the

max

V

= N

(1-8)

where N is given number. So, the sets P (1.2) and Q (1.3) are the given and fixed collections of the vectors (1.4) and (1.5) in the Euclidean spaces /Tand Rs, respectively.

In (1.1) hdjn(t) is the random vector-function restricted by the conditions

\hdin(t)\<H , E{\hdin(t)\) < ddin , te[ 0,71, (1.9)

where Я is a sufficiently large constant, ^„is a sufficiently small constant. Here and below E{...} is the mathematical expectation.

It is supposed that matrix A(t) in (1.1) satisfies the inequality |<4(0|| < ^ j where Я — const and the symbol denotes the norm

of the matrix A(t), i.e. ||Ж0|| = sup|y4(?)x|.

|x|<l

In our work we shall consider the case when for function /(t,u,v) in the differential equation (1.1) the so-called saddle point condition in a small game [34, 36]

maxmin <l,f(t,u,v) > =minmax </,/(f,w,v) > (1.10)

vejQ ueP u&P vgQ

(for every IgR") is not valid. Here the symbol < /, /> denotes the scalar product of two ^-dimensional vectors I and f. Here and further vectors are understood as column vectors, hence

</,/> =/т/ =[/„...,/„]

л

fn

/,/,+...+/„/„ (l.ii)

where the upper index T denotes transposition. Note that the condition (3.1) is often called as Isaacs-Bellman condition [5, 24].

So in considered case for the function f(t,u,v) in (1.1) there exist the vector I ei?" and the point t such that inequality

эр Ф

maxmin < / ,f(t,u,v) > Ф minmax < / v) > (1.12)

veQ ueP ueP veQ

holds.

Then in many control problems (including differential game problems) it is useful to form the motion of the x-object (1.12) along with corresponding motions of the appropriate z-model-leader {pilot) or z-model-observer as a mutual stochastic process (appropriate z-model-leader will be described below in section 2 ).

Combined motion of {xftj, z[t]} is constructed in this work with using computer modeling in discrete in time t form

In this work we will consider and use the well-known scheme of the control - the feedback control in discrete in time scheme. So, we assume that for the given time interval [0,7] we will use any partition

A{tk} = {t0,tb-,tk <*к+1>->*к =т) (1ЛЗ)

Where К is some large number. This partition include all the discontinues points of the functions Aft) and f(t,u,v). On each of the semi-intervals [tk,tk+ i), k = 0, ...,K the functions Aft) and fft,u,v) are substitued by constants. A(t)= A(tk) and f(t,u,v) = f(tk,u,v), t E l>nd also we assume that hdin(t) = hdin(tk), te [tA,xA ), where tk <lk < Thus below we shall consider the .x-object that is described by the finite difference equation

*('jfc+l) = x<fk) + Шк )x(!k ) + f if k »«,v) + hdin (tk)) A t (1.14)

where At = tk+l-tk, tkeA{tk} (1.13).

The actions u-u[t] E P (1.4) , t E [tk,tkJ(.\) and v = v[t] E Q

(1.5) , t E [tk,tk+\) are defined by the probability tests which will be described below in the section 3.

2 The z-model-leader

Let us consider a motion of some abstract z-model together with the real x-object (1.14). The current state of this z-model at time te[/0,.9 ] is determined by its n-dimensional phase vector z\t\. The

motion of z-model on any time interval t/(<t<t/c+\ ,

(1.13), к = 0,..,K, is determined by finite difference equation

Фш] =ФкJ + MhMhl+ fpqih)) ы, С2-1)

where

~ M N

fpq(tk)=i: Zf(tk,U[lKvW)piqj (2.2)

/=1 7=1

In (2.2) i = 1,..,M and v^ ,j = I,.. ,N are the vectors of the given finite sets P (1.4) and Q (1.5), respectively. The function /(•) in

(2.2) coincides with the function/(•) in (1.1), (1.14). The numbers , i

= l,..,M and qj, j = 1,..,N satisfy the following conditions

M

Pi >0, / = 2>/ =1, (2.3)

/=1

N

q -> 0, ] = (2.4)

7=1

where the numbers M and N are the same as in (1.4) and (1.5), respectively. We assume that the motion of the z-model (2.2) is simulated by the computer included in the regulator. As has been said above the proposed z-model (2.1) will be the "leader " for the motion of the x-object (1.14).

The numbers (2.3) and (2.4) are connected by some rules with the probabilities that define the random choice of the actions и and v for the x-object (1.14). This relation will be discussed in the next section.

3 Control actions for the x-object

We will call as usual the "position of the controlled system (1.14) at time moment tk " - the pair {tk, x [tk ]}, tke A {tk} (1.13), к — 0,..,K.

Respectively, the "position of the z-model (2.2) at time moment tk " is the pair {tk, z[th]}, tk e A {tk } (1.13), k= 0,..,K. Note that we will also consider the case when the position of x-object (1.14) is estimated with some informational error Ajnfsuch that at each time moment tk e A { ^} (1.13),k = 0,. .,K, we know only the distorted position { x*[tk]}, where

x*[tk]=x[tk] + Ainftt*], (3.1)

where Ajnf [t k ] is a random vector.

Let us describe probability tests that define the random choice of actions u[t] - u[tk] e P (1.4), t e ftt^k+0-

As the ideal case, we accept that at moment tk one can make instant probability test on choosing vector u[tk] eP (1.4).

*

This test is defined by the suitable probabilities {p; }:

* v-i * * m

Pi - 0, 2. Pi ~ U Pi ~ probability of the event u[tk]=uv \ i.e. i=l

P(u[tk] = u[i]) = pi i=l, ...,M, (3-2)

here the symbol P(...) denotes the probability [20, 59].

This idealization (of the possibility of the instant probability test) can be relaxed if we consider that the informational error (3.1) in the position measurement is also caused by the delay of measurement of the current position of the x-object x[t^J. This small delay can be

considered as time which is necessary for realization of the probability test.

4. Disturbance actions for the x-object

It is assumed that the realization of the random function v[t] are

piecewise constant on the semi-interval [tk,tk+\). Mechanism of the

forming these functions v[t] is connected with one important condition.

Namely, the functions v[t] and u[tk] satisfy the condition of the

stochastic independe