# Necessary conditions of optimality for stochastic switching control systems.

1. INTRODUCTIONOptimal control problems of stochastic systems have a multitude practical applications in fields such as pricing an option, forecasting the growth of population and determining optimal portfolio of investments,etc. [1, 2, 3, 4, 5]. Modern theory of stochastic optimal control in the main has been developed along the two lines: maximum principle and dynamic programming [6, 7]. The analogue of maximum principle for stochastic systems has been first obtained by Kushner [8]. Earlier results on the developments of Pontryagin's maximum principle for stochastic control systems are met in [9, 10, 11, 12]. Investigation of stochastic maximum principle by using random convex analysis was obtained by Bismut [13]. In [14, 15, 16] are obtained the modern presentation of maximum principle for stochastic systems with backward stochastic differential equations . Many real systems have unpredictable structural changes in their behavior from causes of random failures, sudden disturbances, abrupt variation of the connecting points on a mechanisms. These processes have been described by the collection of stochastic differential equations [17, 18] are known as hybrid systems. A switching systems are special class of hybrid systems and have the advantage of modeling nature phenomena with the continuous changing law of system's structure. Therefore optimization problems of switching systems provide both theoretical and practical interest [19, 20, 21, 22, 23].

This paper is dedicated to the stochastic optimal control problems of switching systems with controlled drift and controlled diffusion coefficients. We obtain necessary condition of optimality in the form of a maximum principle for such systems, where the restrictions on transitions are described by functional constraints in the each of constituent interval.

In present paper, backward stochastic differential equations have been used to prove a maximum principle for stochastic optimal control problems of switching systems. Optimal control problems of stochastic switching systems with uncontrolled diffusion coefficients have been considered by the authors in [24, 25, 26, 27]. The problem with controlled diffusion coefficients without endpoint constraints is studied in [28]. Stochastic switching systems with controlled diffusion and with the special type of restrictions were investigated in [29, 30].

This paper contains five sections. Notations, definitions and the statement of main problem are given in Section 2. Section 3 is devoted to problem of optimality of stochastic switching systems with controlled coefficients. In section 4 stochastic optimal control problem of switching system with endpoint restrictions is treated. It is proved some important facts for our goal and is established necessary condition of optimality in form of maximum principle. Conclusion finalizes the present paper.

2. DESCRIPTION OF MAIN PROBLEM

Following notations are used throughout present paper. [R.sup.m] represents the m dimensional real vector space; [absolute value of x] denotes the Euclidean norm. We use N as notation for some positive constant; [bar 1, r] denotes the set of integer numbers 1,..., r. Assume that [sigma]-algebras [F.sup.t.sub.l] = [bar.[sigma]] ([w.sup.l.sub.t],[t.sub.l- 1] [less than or equal to] t [less than or equal to] [t.sub.l)] are generated by independent Wiener processes [w.sup.1.sub.t], [w.sup.2.sub.t],..., [w.sup.r.sub.t]. Let ([SIGMA], F, P) be a complete probability space with filtration {[F.sub.t],t [member of] [0,T]}, where [F.sub.t] = [bar.[sigma]] ([F.sup.l.sub.t],l = [bar.1,r]). [L.sup.2.sub.F]l(0,T-, [R.sup.m]) denotes the space of all predictable processes [x.sub.t](w) in [R.sup.m] such that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. [R.sup.kxm] represents the space of all linear transformations from [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], be open sets. Unless specified otherwise, we will use following notations: t = ([t.sub.0],...,[t.sub.r]),u = ([u.sup.1],...,[u.sup.r]) and x = ([x.sup.1],...,[x.sup.r])

Consider the following stochastic control system:

(2.1) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

(2.2) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

(2.3) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where {[t.sub.k]} denote the time that [x.sub.t] is heavily changed, which are a series of unknown moments satisfying [t.sub.1] < [t.sub.2] < [t.sub.3] < ... .

Elements of [U.sub.[partial derivative]] = [U.sup.1.sub.[partial derivative]] x [U.sup.2.sub.[partial derivative]] x ... x [U.sup.r.sub.[partial derivative]] are called admissible controls. Our main goal is to find optimal inputs ([x.sup.1,] [x.sup.2],... ,[x.sup.r],[u.sup.1],[u.sup.2],..., [u.sup.r]) and switching sequence [t.sub.1],[t.sub.2],...,[t.sub.r], which are minimize following cost functional:

(2.4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

on the decisions of the system (2.1)-(2.3), which are generated by all admissible controls at conditions:

(2.5) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

[G.sup.l] are a closed convex sets in [R.sup.1.]

To establish necessary condition of optimality for the stochastic control problem

(2.1)-(2.5) we need to the following assumptions.

(H1) Functions [g.sup.l], [f.sup.l],[p.sup.l], [[PHI].sup.l],[q.sup.l] are twice continuously differentiable with respect to x for each l = 1, 2,..., r.

(H2) For each l = 1, 2,..., r functions [g.sup.l], [f.sup.l],[p.sup.l], and all their derivatives are continuous in (x, u). [g.sup.l.sub.x], [9.sup.l.sub.xx], [f.sup.l.sub.x], [f.sup.l.sub.xx],[P.sup.l.sub.xx] are bounded and hold the linear growth conditions.

(H3) Functions [psi](x) : [R.sup.nr] [right arrow] R are twice continuously differentiable and hold the condition:

|[psi](x)| + |[[psi].sub.x](x)| <[less than or equal to] N (1 + [absolute value of x]), |[[psi].sub.xx](x)| [less than or equal to] N.

(H4) For each l = 1, 2,... , r - 1 functions [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] are twice continuously differentiable with respect to (t) and satisfy the condition:

|[[PHI].sup.l](x,t)| + |[[PHI].sup.l.sub.x](x,t)| [less than or equal to] N(1 + [absolute value of x]), |[[PHI].sup.l.sub.xx](x,t)| [less than or equal to] N.

(H5) Functions [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] are continuously differentiable with respect to (x) and meet the condition:

|[q.sup.l](x)| + |[[q.sup.l.sub.x] (x)| [less than or equal to] N(1 + [absolute value of x]), |[q.sup.l.sub.xx](x)| [less than or equal to] N.

Consider the sets:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

with the elements

[[pi].sup.i] = ([t.sub.0], [t.sub.1],...,[t.sub.i], [x.sup.1.sub.t], [x.sup.2.sub.t],...,[x.sup.i.sub.t], [u.sup.1], [u.sup.2],..., [u.sup.i]).

Definition 2.1. Collection of stochastic processes {[x.sup.l.sub.t] = [x.sup.l](t,[[pi].sup.l])}, t [member of] [[t.sub.l- 1],[t.sub.l]],l = 1, r is called a solution of variable structure system (2.1)-(2.2) corresponding to an element [[pi].sup.r] [member of] [A.sub.r,] if stochastic process [x.sup.l.sub.t] [member of] [O.sub.l] almost certainly continuous on the interval [[t.sub.l-1],[t.sub.l]], holds the condition (2.2) at point [t.sub.l] and satisfies the equation (2.1) almost everywhere.

Definition 2.2. [[pi].sup.r] [member of] [A.sub.r] is called the admissible element if pairs ([x.sup.l.sub.t],[u.sup.l.sub.t]),t [member of] [[t.sub.l-l],[t.sub.l]], l = [bar. 1,r] satisfy (2.1)-(2.3) and conditions (2.5).

Definition 2.3. Let [A.sup.o.sub.r] be the set of admissible elements. The element [[??].sup.r] [member of] [A.sup.o.sub.r], is called an optimal solution of problem (2.1)-(2.5) if there exist admissible controls [[??].sup.l.sub.t], t [member of] [[t.sub.l-1],[t.sub.l]] and corresponding solutions of system (2.1)-(2.2) such that pairs ([[??].sup.l.sub.t], [[??].sup.l.sub.t]), l = [bar.1,r] minimize the functional (2.4).

3. MAXIMUM PRINCIPLE OF STOCHASTIC SWITCHING SYSTEMS

Applying the similar technique as in [28] following necessary condition of optimality for stochastic control system (2.1)-(2.4) is obtained.

Theorem 3.1. Suppose that,

[[pi].sup.r] = ([t.sub.0],[t.sub.1],...,[t.sub.r],[x.sup.l.sub.t],...,[x.sup.r.sub.t], [u.sup.1],...,[u.sup.r])

is an optimal solution of problem (2.1)-(2.4) and conditions (H1)-(H4) hold . Then, a) there exist stochastic processes [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]which are the solutions of the following conjugate equations:

(3.1) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

(3.2) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

b) almost everywhere in [theta] [member of] [[t.sub.l-1], [t.sub.l]], and [for all] [[??].sup.l] [member of] [U.sup.l], l = [bar.1,r], almost certainly (a.c.) fulfills the maximum principle:

(3.3) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

c) following transversality conditions hold:

(3.4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Here we used following notations:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Proof. Let [[bar.u].sup.l.sub.t] = [[u.sup.l.sub.t] + [DELTA][ [bar.u].sup.l.sub.t] l = [bar.1, r] represent some admissible controls and [[bar.u].sup.l.sub.t] = [x.sup.l.sub.t] + [DELTA][ [bar.u].sup.l.sub.t] l = [bar.l,r] represent the corresponding trajectories of system (2.1)- (2.3). Let 0 = [t.sub.0] < [t.sub.1] < ... < [t.sub.r] = T be switching sequence. Then following identities are obtained for some sequence 0 = [[bar.t].sub.0] < [[bar.t].sub.1] < ... < [[bar.t].sub.r] = T:

(3.5)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

According to Ito's formula implies that following identities are true:

(3.6)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

and

(3.7)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Note that linear terms in (3.5) can be handled in the following way. Consider the following matrix-valued equations:

d[Z.sub.t] = [A.sub.t] [Z.sub.t]dt + [B.sub.t] [Z.sub.t]dwt,

[Z.sub.0] = I,

which have a unique solution [Z.sub.t] with E sup [[parallel][Z.sub.t][parallel].sub.2s] < [infinity], s [greater than or equal to] 1, if [A.sub.t] and [B.sub.t] are the predictable and bounded matrices (see [11]). It is easy to show that the matrix [Z.sub.t] has an inverse and [G.sub.t] = [Z.sup.-1.sub.t] is a solution of the equation:

d[G.sub.t] = - ([G.sub.t][A.sub.t] - [G.sub.t][B.sub.t][B.sub.t]) - [G.sub.t][B.sub.t][dw.sub.t],

[G.sub.0] = I.

In order to establish the existence and uniqueness of solution of adjoint stochastic differential equations, it is enough to follow the method described in the article [10] and to make use the independence of Wiener processes [w.sup.1.sub.t] ,..., [w.sup.r.sub.t] in the each interval [[t.sub.l-1],[t.sub.i]], l = l,...,r. The stochastic processes [[psi].sup.l.sub.t], [[PSI].sup.l.sub.t], = [bar 1, r], at the points [t.sub.1], [t.sub.2],...,[t.sub.r] are defined as:

(3.8) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

and

(3.9) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Using the expressions (3.5)-(3.9) for the increment of a functional (2.4) we obtain the form as indicated below:

(3.10)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where

(3.11) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

and

(3.12)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

According to (3.1), (3.2), (3.8) and (3.9) , through the simple transformations expression (3.10) may be written as:

(3.13)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Based on fact that [[pi].sup.r] = (t, x, u) is optimal solution, using the independence of increments respective to different arguments and assumption (H4) from expression (3.13), we obtain that (3.4) is true.

Consider the following spike variations:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where [[epsilon].sub.l] are enough small numbers. Then the expression (3.13) takes the form of:

(3.14) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

In order to obtain estimation for increment (3.14), we introduce following lemma.

Lemma 3.2 (Gronwall's inequality [6]). Let m(t) is a continuous function satisfying:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], here g(t) is continuous,h(t) is bounded functions and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. Then following holds:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Proof of following lemma can be found in [29]. Here, the brief proof will be given in due to make comprehensible content for this paper.

Lemma 3.3. Suppose that conditions (H1)-(H2) are satisfied. Then, the following is obtained:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Here [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] are the solutions of system (2.1)-(2.2), corresponding to the controls [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Proof. Let's denote the following: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. It is clear that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. Then for [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

or

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Therefore from the conditions (H1)-(H2) and using the Lemma 3.2 we have

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Hence: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. According to identity (3.5) for special spike variation of control [for all]t [member of] [[[theta].sub.l] + [[epsilon].sub.l], [t.sub.l]]:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

which can be rewritten as follow:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Hence: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Thus: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

From the expression (3.12), due to Lemma 3.3 for each l implies following estimation:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Then according to fact that [[bar.u].sub.t] = ([[bar.u].sup.1.sub.t],..., [[bar.u].sup.2.sub.t],..., [[bar.u].sup.r.sub.t] is optimal of control from (3.14) for each l it follows that:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Finally, due to the smallness and arbitrariness of [[epsilon].sub.l] (3.3) is achieved.

4. NECESSARY CONDITION OF OPTIMALITY FOR SWITCHING SYSTEMS WITH CONSTRAINTS

First, we recall notion Ekeland's variational principle to use in our main result.

Theorem 4.1 (Ekeland's variational principle [31]). Let (X, d) is complete metric space and f : X [right arrow] R[Universal] (+ [infinity]) be a semi-continuous function from below. [epsilon], [lambda] are positive numbers and for some point [x.sub.0] [member of] X is satisfied: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], there exist [bar.x] [member of] X such that:

1) f ([bar.x]) [less than or equal to] f (x0 ),

2) d([bar.x], [x.sub.0] ) [less than or equal to] [lambda],

3) [for all]x [member of] X, f ([bar.x]) [less than or equal to] f (x) + [epsilon][lambda]d. ([bar.x], x).

By applying the Theorem 3.1 and Theorem 4.1 the necessary condition of optimality for stochastic control problem of switching systems (2.1)-(2.5) is obtained.

Theorem 4.2. Assume that, assumptions (H1)-(H5) satisfy and [[pi].sup.r] = (t,x,u) is an optimal solution of problem (2.1)-(2.5). Then,

a) there exist non-zero vector ([[lambda].sub.0], [[lambda].sub.1],..., [[lambda].sub.r]) [member of] [R .sup.r+1] and stochastic functions ([[psi].sup.l.sub.t], [[beta].sup.l.sub.t]) [member of] [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] which are the solutions of the following conjugate equations:

(4.1) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

(4.2) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

b) a.e. [theta] [member of] [[t.sub.l-1],[t.sub.l]] and [for all] [[??].sup.l] [member of] [U.sup.l,] l = [bar. 1,r], a.c. holds the maximum principle:

(4.3) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

c) following transversality conditions holds:

(4.4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Proof. First, we discuss the existence of uniquely solutions of adjoint equations (4.1) and (4.2). In fact from [10, 15, 16], the first-order adjoint processes ([[psi].sup.l.sub.t], [[beta].sup.l.sub.t]) and second order adjoint processes ([[PSI].sup.l.sub.t], [K.sup.l.sub.t] described in a unique way by (4.1) and (4.2) respectively. Using Theorem 4.1 , the problem is convert into the sequence of unconstrained problems. Finally, we obtain maximum principle in the case when and endpoint constraints are imposed. Consider following approximating functional for any natural j :

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], [y.sub.1] [member of] [G.sup.1],... ,[y.sub.r] [member of] [G.sup.r]}, [J.sup.0] minimal value of the functional in the problem (2.1)-(2.5). Introduce space of controls [V.sup.l] [equivalent to] ([U.sup.l.sub.[partial derivative]], d) obtained by means of the metric:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

[V.sup.1,] [V.sup.2,...,] [V.sup.r] are complete metric spaces [31]. It is easy to prove the following fact:

Lemma 4.3. Assume that conditions (H1)-(H4) hold, for each l [u.sup.l'n.sub.t] be the sequence of admissible controls from [V.sup.l,] and [x.sup.l'n.sub.t] be the sequence of corresponding trajectories of the system (2.1)-(2.3). If the following condition is met: d([u.sup.l,n.sub.t],[u.sup.l.sub.t]) [right arrow] 0, then, [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] = 0, where [x.sup.l.sub.t] is a trajectory corresponding to an admissible controls [u.sup.l.sub.t], l = [bar.1,r].

Due to continuity of the functionals[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] , according to Ekeland's variational principle, there are controls such as: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

[V.sup.l] follows: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

This fact can be treated following way: ([[t.sub.1],..., [t.sub.r], [x.sup.1,j.sub.t],..., [x.sup.rj.sub.t], [u.sup.1,j.sub.t],..., [u.sup.r,j.sub.t]) is a solution of the following problem:

(4.5) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Here

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Then according to the Theorem 3.1, it is obtained as follows:

(4.6) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

and the random processes (4.5) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], which are solutions of the system:

(4.7) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where non-zero ([[lambda].sup.j.sub.0], [[lambda].sup.j.sub.1],..., [[lambda].sup.j.sub.r]) [member of] [R.sup.r+1] meet the following requirement:

([[lambda].sup.j.sub.0], [[lambda].sup.j.sub.1],..., [[lambda].sup.j.sub.r]) = ([-c + 1/j + EM([x.sup.j], [u.sup.j], t)],

(4.8) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

here

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

2) a.e. t [member of] [[t.sub.l-1,] [t.sub.l]] and [for all] [[??].sup.l] [member of] [V.sup.l], l = [bar.1,r], a.c. is satisfied:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

(4.9) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

3) following conditions of transversality satisfy:

(4.10) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Since the following has existed |([[lambda].sup.j.sub.0], [[lambda].sup.j.sub.1],..., [[lambda].sup.j.sub.r])| = 1, then according to (4.8) and conditions (H1)-(H5) it is implied that ([[lambda].sup.j.sub.0], [[lambda].sup.j.sub.1],..., [[lambda].sup.j.sub.j]) [right arrow] ([[lambda].sub.0], [[lambda].sub.1],..., [[lambda].sub.r]) if j [right arrow] [infinity].

We now state the following results which will be needed in the future.

Lemma 4.4. Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] be a solution of system [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] be a solution of system (4.6). If d([u.sup.l,j.sub.t], [u.sup.l.sub.t]) [right arrow] 0, then

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Proof. It is clear that [for all]t [member of] [[t.sub.l-1],[t.sub.l]]:

(4.11) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Squaring both sides of the equation, according to Ito formula [for all]s [[t.sub.l-1], [t.sub.l]] we obtain:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Now, using the assumptions (H1)-(H5) we have:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Hence, by the Lemma 3.2 we establish that:

(4.12) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where (4.10) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. Hence from (4.1),(4.6) and conditions (H3),(H5) it follows that (4.10) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and D [right arrow] 0. Consequently, from (4.12) we obtain that [[psi].sup.r,j.sub.s] [right arrow] [[psi].sup.r.sub.s] in [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and thus [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Then from expression (4.11) in view of assumptions (H1)-(H5) and according to Lemma 3.2 we get:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

here (4.12) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], which D [right arrow] 0 according to (4.1), (4.6) and conditions (H3)-(H4). Hence, from (4.12) implies that (4.12) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Lemma 4.5. Let [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] be a solution of system (4.2), and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], be a solution of system (4.7). Then

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Proof. Due to Ito's formula from expressions (4.2) and (4.7) for [for all]s [member of] [[t.sub.l-1],[t.sub.l]):

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Then with help simple transformations we obtain:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

According to Gronwall inequality a.e. in [[t.sub.l-1],[t.sub.l]) we have:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where the constant D defined as:

Follow the same steps as in Lemma 4.4 in view of (4.2), (4.7) and assumptions (H3), (H5) we establish that: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. Further, according to assumptions (H1)-(H4) and expressions (4.2), (4.6) we obtain: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

According to sufficient smallness of [epsilon] follows that D [right arrow] 0. [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Due to Lemma 4.4 and Lemma 4.5 it can be proceed to the limit in systems (4.6) , (4.7) and the fulfilment of (4.1),(4.2) are obtained. Follow a similar scheme by taking limit in (4.9) and (4.10) it is proved that (4.3),(4.4) are true. Theorem 4.2 is proved.

5. CONCLUSION

A lot of theoretical and numerical advances have recently been realized in the field of modelling and control related with randomness [3, 4, 5, 32, 33]. Necessary conditions satisfied by an optimal solution, play an important role for investigation of optimization and optimal control problems. The present paper is devoted to optimal control problem of stochastic switching systems with the endpoint state restrictions in the form of functional constraints. The necessary conditions developed in this study can be viewed as a stochastic analogues of the problems that are formulated in [19, 20, 23]. However, Theorem 4.2 is a natural evolution of the results given in [28, 26, 29].

Received April 2, 2015

CHARKAZ AGHAYEVA

Department of Industrial Engineering, Anadolu University

Turkey Istitute of Control Systems of ANAS, Baku, Azerbaijan

REFERENCES

[1] L. Arnold, Stochastic Differential Equations: Theory and Applications, Wiley, New York, 1974.

[2] V. B. Kolmanovsky, A. D. Myshkis, Applied Theory of Functional Differential Equations, Kluwer Academic Publishers, 1992.

[3] O. Kallenberg, Foundations of Modern Probability, Springer, New York, 2002.

[4] B. 0ksendal Stochastic Differential Equations with Applications, Universitext, Springer, 2013.

[5] A. N. Shiryaev, Essentials of Stochastic Finance: Facts, Models,Theory, World Scientific, 2003.

[6] W. H. Fleming, R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer, 1975.

[7] J. Yong, X. Y. Zhou, Stochastic controls: Hamiltonian Systems and HJB equations,New York, Springer, 1999.

[8] H. J. Kushner, Necessary conditions for continuous parameter stochastic optimization problems, SIAM,10:550-565, 1976.

[9] V. I. Arkin, M. T. Saksonov, The necessary conditions of optimality in control problems of stochastic differential equations, DAN SSSR, 244,4: 11-16, 1979.

[10] A. Bensoussan, Lectures on Stochastic Control in Nonlinear Filtering and Stochastic Control, Springer Lectures Notes in Mathematics, 972:1-62, Springer Verlag, 1982.

[11] J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control,14:419-444, 1976.

[12] U. G. Haussman, General necessary conditions for optimal control of stochastic systems, Stochastic Systems: Modeling, Identification and Optimization, II Mathematical Programming Studies, 6: 30-48, 1976.

[13] J. M. Bismut, An Introductory Approach to Duality in Optimal Stochastic Control, SIAM Review, 20: 62-78, 1978.

[14] V. Borkar, Controlled diffusion processes, Probability surveys, 2: 213-244, 2005.

[15] N. I. Makhmudov, General necessary optimality conditions for stochastic systems with controllable diffusion Statistics and control of random processes, 135-138, Nauka, Moscow, 1989.

[16] S. Peng, A general stochastic maximum principle for Optimal control problem, SIAM: Journal on Control and Optimization, 28:966-979, 1990.

[17] P. J. Antsaklis, A. Nerode, Hybrid Control Systems: An Introductory Discussion to the Special Issue IEEE Transaction on Automatic Control 43(4):457-460, 1998.

[18] E. Boukas, Stochastic switching systems: analysis and design, Birkhauser, Canada, 2006.

[19] S. C. Bengea, A. C. Raymond, Optimal Control of Switching systems, Automatica, 41:11-27, 2005.

[20] D. I. Capuzzo, L. C. Evans, Optimal Switching for ordinary differential equations, SIAM, Journal on Control and Optimization, 22:143-161, 1984.

[21] R. Gao, X. Liu, J. Yang, On optimal control problems of a class of impulsive switching systems with terminal states constraints, Nonlinear Analysis: Theory, Methods and Applications, 73:1940-1951, 2010.

[22] B. Picolli, Hybrid systems and optimal control Proceedings of the 37th IEEE Conference on Decision and Control, 13-18, 1989.

[23] T. I. Seidmann, Optimal control for switching systems Proceedings of the 21st annual conference on in formations science and systems,485-489, 1987.

[24] Ch. Agayeva, Q. Abushov, Necessary condition of optimality for stochastic control systems with variable structure, Proceedings of the 20th International Conference :Continuous, Optimization and Knowledge - based technologies, 77-81 Vilnius,Technika, 2008.

[25] Ch. Agayeva, Q. Abushov, The maximum principle for some stochastic control problem of switching systems, Selected papers. 24th Mini EURO Conference on Continuous Optimization and Information-Based Technologies in the Financial Sector,100-105, Vilnius, Technika, 2010.

[26] Ch. Aghayeva, Q. Abushov, The maximum principle for the nonlinear stochastic optimal control problem of switching systems, Journal of Global Optimization, 56(2):341-352, 2013.

[27] Q. Abushov, Ch. Aghayeva, Stochastic maximum principle for the nonlinear optimal control problem of switching systems, Journal of Computational and Appllied Mathematics,259:371376, 2014.

[28] Ch. Aghayeva, Q. Abushov, The maximum principle for some nonlinear stochastic optimal control problem of swithching system, Transactions ANAS, mathematics and mechanics series of physical-technical and mathematical science, 3:74- 81, 2011.

[29] Ch. Aghayeva, Q. Abushov, Stochastic maximum principle for switching systems, Proceedings of the 4th International Conference "Problem of Cybernetics and Informatics",3:198-201, 2012.

[30] Ch. Aghayeva, Q. Abushov, Stochastic Optimal Control Problem for Switching System with Controlled Diffusion Coefficients,Proceedings book of the World Congress on Engineering, 1:202-207, 2013.

[31] I. Ekeland, On the variational principle, Journal Mathematical Analysis and Applications, 47:324-353, 1974.

[32] E. Kiyak, O. Cetin, A. Kahvecioglu, Aircraft sensor fault detection based on unknown input observers, Aircraft Engineering and Aerospace Technology, 80(5):545-548, 2008.

[33] O. Turan, Exergetic Effects of Some Design Parameters on the Small Turbojet Engine for Unmanned Air Vehicle Applications, Energy, 46(1):51-61, 2012.

Printer friendly Cite/link Email Feedback | |

Author: | Aghayeva, Charkaz |
---|---|

Publication: | Dynamic Systems and Applications |

Article Type: | Report |

Date: | Sep 1, 2015 |

Words: | 4726 |

Next Article: | Existence and monotone iteration of concave positive symmetric solutions for a three-point second-order boundary value problems with integral... |

Topics: |