# On the effect of non-linearity on linear quadratic regulator stability and performance.

INTRODUCTIONIn a majority of control applications, the design of control systems is based on a linear approximation of the system dynamics and the subsequent design of a corresponding linear controller that can guarantee some degree of (robust) performance for the (nominal) linearized system. Although this strategy can yield acceptable control system performance in a number of cases, such an approach is problematic when the underlying system cannot be represented adequately by a linear system model. It remains difficult to measure the extent of the performance degradation that results from the non-linear behaviour that was ignored at the design stage. The development of measures of non-linearity and its effect has been considered by many authors. These measures are primarily designed to assist control engineers in the efficient and effective development of suitable control architectures for non-linear control systems.

Many authors (e.g. Desoer and Wang, 1980; Allgower, 1995a, b; Stack and Doyle III, 1997b; Haber, 1985; Ogunnaike et al., 1993; Guay et al., 1995) have considered the assessment of process non-linearity as a means of justifying the need for nonlinear control techniques. Most of the non-linearity measures that have been developed focus on the quantification of the non-linearity of the open-loop response of a non-linear system. The main assumption related to the use of such measures in controller design is that highly non-linear systems will generally require non-linear controllers. Although this may be true in a significant number of applications, it is well known that linear feedback controllers can effectively diminish the extent of open-loop non-linearity in a non-linear control system (Eker and Nikolaou, 2002). In an effort to provide a more precise description of non-linearity for use in control systems design decisions, a number of authors have proposed so-called control relevant non-linearity measures.

In Eker and Nikolaou (2002), the authors present a measure of closed-loop non-linearity suitable for the analysis of linear control for non-linear systems. The non-linear system, a stable non-linear input-output operator N, is assumed to be controlled by a controller Q resulting from the Youla parameterization of all stabilizing controllers of the linear approximation of the process, a stable linear operator L. Using an IMC framework, the closed-loop non-linear control system operator provides a direct measure of the discrepancy between the non-linear and the nominal linear closed-loop. The induced incremental norm, subject to a linear low-pass filter, of the closed-loop operator is used as a measure of closed-loop non-linearity. Assuming robust stability and invertibility of the operator (I + NQ - LQ), explicit bounds on the control-relevant non-linearity are obtained. The main advantage of the measure, which is parameterized by the choice of controller, Q, is its applicability in the design of linear controllers for non-linear systems; however, the analysis is limited to open-loop stable non-linear systems. The computations required to evaluate the induced norm of the closed-loop operator, although shown to be greatly reduced in Nikolaou (1993), can be complex for larger systems.

In Stack and Doyle III (1995, 1997a, b), an objective function is included with the problem definition, and a Lagrangian optimization is performed to determine an optimal control structure. The measure of non-linearity is obtained from the state to input controller operator. The degree of non-linearity of this operator provides an indication that a non-linear controller may be required for this application. A coherence test, discussed in Haber (1985), is chosen as a measure of non-linearity between the system input and output reflected in the optimal control structure. The main difficulty with this approach is associated with the computation of the optimal control structure. It seems obvious, at first hand, that a control structure that is deemed optimal with respect to a given objective function provides a direct indication of the need for non-linear controller; however, this approach is counter-productive, since one needs to design a non-linear controller structure. The main advantage of this approach is that it provides a very accurate description of the non-linear control structure.

In Schweickhardt (2003), a closed-loop control law non-linearity measure is defined. The measure, which is largely based on the OCS approach (Stack and Doyle III, 1995), attempts to compute the closed-loop optimal control law non-linearity using the measure proposed in Allgower (1995a) subject to set of initial conditions. As in Stack and Doyle III (1995), the assessment of non-linearity requires the computation of optimal closed-loop trajectories and addresses the question of the need for nonlinear control directly. From a practical perspective, it is unclear whether the measurement of control law non-linearity precludes the successful application of a linear controller. Furthermore, the computation of optimal closed-loop trajectories over a set of initial conditions remains a considerable task, as in other similar operator-based approaches.

In recent work by the authors Guay et al. (2005) and Guay and Forbes (2004), a performance sensitivity measure was developed for the quantification of the effect of non-linearity on LQG controller performance in continuous-time non-linear control systems. This method is based on a different approach to the problem of non-linearity assessment that is primarily motivated by the general approach to the design of a control system. In most applications, one seeks a local linear approximation of the process and designs a linear controller to achieve locally optimal performance objectives. The degree to which these objectives are realizable will depend on the underlying structure of the process. Naturally, if the model used in the analysis is poor then the resulting objectives will not be attained. When the process displays significant non-linearity, the intended LQR optimal performance will not be realized. The non-linearity thus becomes an obstacle to one's ability to achieve the intended controller performance. In this context, the quantification of non-linearity in the closed-loop system is reduced to the effect of non-linearity on the intended controller performance. That is, if the linear controller performance is subject to large deviations in its nominal performance then such a performance can only be ensured via the design of a non-linear controller.

In this paper, we adopt this point of view to develop a performance based non-linearity measure for continuous-time non-linear systems. The approach differs from the approach presented in Guay et al. (2005) and Guay and Forbes (2004) for continuous-time systems. The measure proposed in Guay et al. (2005) is based on the computation of the sensitivity of the nonlinear control system subject to a linear control strategy with input perturbations. Although informative, the resulting performance sensitivity measure provides a test that is primarily focused on the internal stability of the control systems and only provides a conservative assessment of non-linearity. In addition, the quantification of non-linearity considered in Guay et al. (2005) and Guay and Forbes (2004) is based on a truncated Taylor series of the performance measure. As a result, it provides only an approximate assessment of the impact of local non-linearity effects that can often be misleading.

In this paper, we consider the quantification of non-linearity subject to changes in the initial conditions of the closed-loop system. The technique is applied (but not limited) to the study of non-linear continuous-time control systems subject to LQR control. Two measures of non-linearity are proposed. As in Guay et al. (2005) and Guay and Forbes (2004), we propose a measure of the impact of non-linearity on the closed-loop performance. The main contribution of the approach proposed is the development of an exact measure that does not rely on any approximation of the cost functional in a neighbourhood of an equilibrium point of interest. The measure can be calculated effectively and provides a very accurate assessment of the impact of non-linearity on closed-loop performance. We also propose a measure of the impact of non-linearity on the stability of the closed-loop system. The stability measure and the performance measure provide a comprehensive set of tools for the study of non-linear control systems subject to LQR control. The main difference between this technique and all other techniques presented in the literature is that the analysis can be used directly in the control design step. If a linear controller can be shown to operate as intended, despite the non-linearity, then a practitioner can apply a linear controller to the non-linear systems with the guarantee that a certain degree of performance can be assured, at the design step.

The paper is as follows. In the next section we present the proposed sensitivity measures. We first introduce the class of systems and the class of the linear quadratic regulators of interest. A stability sensitivity measure is presented first followed by a performance sensitivity measure. The primary focus is on the computation of the measures and their interpretation. In the third section process examples are studied and compared. Brief conclusions are provided in the fourth section.

PERFORMANCE SENSITIVITY MEASURE

In this section, we propose two measures of sensitivity that quantify the effect of non-linearity on stability and performance of a non-linear control system subject to a linear quadratic controller. The stability sensitivity measure (SSM) quantifies the expected loss of stability when a non-linear system is regulated by a linear quadratic regulator (LQR) in a region of the state space. The performance sensitivity measure (PSM) attempts to characterize the extent of performance degradation expected for a non-linear system subject to LQR control.

Class of Non-Linear Systems

Consider the non-linear time-invariant system:

[??] = f(x, u(t)) (1)

y = h(x(t))

where f : D x u [right arrow] [R.sup.n] is smooth vector valued function on a set D [member of] [R.sup.n] containing the origin and a set u of admissible control inputs. The input u(t)[member of] [R.sup.p] is the available control input, y(t) [member of] [R.sup.m] is the observed process output, and x(t) [R.sup.n] represents the state variables of the system at time t.

The linearization of the system Equation (1) about the origin is given by the linear time-invariant system:

[??] = Ax + Bu(t) (2)

y(t) = Cx(t)

where

A = [partial derivative]f (0,0)/[partial derivative]x B = [partial derivative]f (0,0)/[partial derivative] x [partial derivative]h (0)/[partial derivative]x

It is assumed that the triple (A, B, C) constitutes an observable and controllable linear system. By letting C be the identity matrix, full state information is available for use in the control strategy.

The linear quadratic regulator

For the linear system Equation (2), the linear quadratic regulator is given by:

u(t) = - [R.sup.-1] [B.sub.T] Px(t) = Kx(t) (3)

where K = - [R.sup.-1] [B.sup.T]P is the controller gain matrix. The resulting control action minimizes, for every initial condition x(0) = [x.sub.0], the quadratic objective function:

J = [[integral].sup.[infinity].sub.0] ([x.sup.T] (t) [Q.sub.x] (t) + [u.sup.T] (t) Ru(t)) dt (4)

where Q [member of] [R.sup.nxn] and R [member of] [R.sup.pxp] are problem-specific, non-negative definite state- and input-penalty matrices. The matrix P is the positive-definite, symmetric solution matrix of the algebraic Riccati equation:

[A.sub.T]P + PA - [PBR.sup.-1] [B.sup.T]P + [C.sup.T] QC = 0 (5)

The cost to regulate to the origin when the system starts at any point at any time t can be approximated by the value function:

[J.sup.*] = [x.sup.T] (t)Px(t) (6)

Sensitivity Measures of a Non-Linear System

If one implements the LQR controller on the nominal non-linear plant Equation (1), the degree to which the intended linear controller performance, [J.sup.*], and the stability of the closed-loop system is realized depends on the extent of non-linearity of the process. If the non-linearity of the process is high, then the sensitivity of the nominal controller performance and stability to changes in the nominal conditions will be significant.

In the following, we propose two measures of sensitivity that quantify the effect of process non-linearity on the closed-loop system stability and performance.

Effect of Non-Linearity on Stability

If one implements the LQR controller on the nominal non-linear plant, the implicitly assumed stability may be precarious when the underlying system is significantly non-linear. In this section, we propose a measure of the effect of non-linearity on the stabilization of a non-linear system using an LQR controller. We consider the full state information situation noting that the partial state measurement case can be treated effectively with a straightforward derivation.

For the LQR controller, u = Kx, it is well known that the value function J = [x.sup.T]Px constitutes a Lyapunov function for the closed-loop linear system. The derivative of J along the trajectories of the closed-loop linear system is given by:

dJ/dt = [x.sup.T] ([PA.sub.cl] + [A.sup.T.sub.cl]P) x = [x.sup.T] Qx (7)

where [A.sub.cl] = A + BK corresponds to the dynamics of the closed-loop linear system. For a non-linear system:

[??] = f(x, u)

subject to an LQ controller, u = Kx, the rate of change of J is given by:

dJ/dt = [x.sup.T] PF(x) + F[(x).sup.T] Px (8)

where F(x) = f(x, Kx) corresponds to the closed-loop system dynamics.

For linear systems, the closed-loop system is such that Equation (7) holds. As a result, the rate of decrease of J along the trajectories will be the same for all states in the ellipsoid {x [member of] [R.sup.n]| [x.sup.T] Qx = [c.sub.q]} where [c.sub.q] is a positive constant. Thus, the rate of decrease is invariant of state variables along these ellipsoids. For non-linear systems, expression Equation (8) may deviate significantly from the nominal expression and this invariance is lost. More importantly, the function J = [x.sup.T] Px will cease to be a candidate Lyapunov function for the closed-loop system. Thus, the stability of the closed-loop system may not be confirmed simply from the nominal LQ controller design. In the presence of significant non-linearity, one may require an alternative candidate Lyapunov function to prove the stability of the closed-loop system.

In order to capture this lack of invariance, we attempt to quantify the sensitivity of dJ/dt with respect to x, for the closed-loop non-linear system in a neighbourhood of the origin. In this way, we attempt to measure how one can rely on the LQ controller design and use J = [x.sup.T] Px as a candidate Lyapunov function for the system. In cases where this assumption does not hold, the underlying design may be susceptible to instabilities that were not foreseen in the nominal design of the LQ controller. Although the closed-loop system may remain stable in a neighbourhood of the origin, its stability cannot be assessed from the LQ design step.

In the following development, we propose a measure of the sensitivity of the nominal stability of the closed-loop to process non-linearity.

For the closed-loop non-linear system, the gradient of Equation (8) with respect to x is given by:

[[nabla].sub.x] dJ/dt = PF (x) + [x.sup.T] P [partial derivative]F/[partial derivative]x + [partial derivative][F.sup.T]/[partial derivative]x Px + F [(x).sup.T] P

The value of this gradient vanishes identically when evaluated at the origin x = 0. The second order derivatives of Equation (8) at x(0) = 0 are given by:

[[nabla].sup.2.sub.x] dJ/dt = 2P [partial derivative]F/[partial derivative]x + 2 [partial derivative][ F.sup.T]/[partial derivative]x + P + 2[x.sup.T]P [[partial derivative].sup.2]F/[partial derivative]x[partial derivative][x.sup.T] (9)

Note that at the equilibrium x = 0, we have [partial derivative]F/[partial derivative]x = [A.sub.cl] and, therefore:

[[nabla].sup.2.sub.x] dJ/dt = -2Q (10)

Proceeding in a similar fashion, it is straightforward to show that the third order derivatives evaluated at x = 0 are given by:

[[nabla].sup.3.sub.x] dJ/dt = 6 [P] [[partial derivative].sup.2]F/[partial derivative]x [partial derivative][x.sup.T]] (11)

where the square bracket notation is introduced to summarize matrix multiplication involving a three-dimensional array. The bracket indicates that the summation in the multiplication must be performed on the numerator index. Left and right multiplication of each matrix face of the three-dimensional array involves summation over the denominator indices.

Based on the expressions given above, one can write the Taylor series of dJ/dt about x as follows:

dJ/dt = - [delta][x.sup.T] Q[delta]x + [alpha] ([delta]x) (12)

where

[alpha]([delta]x) = [delta][x.sup.T]PF([delta]x) + F[([delta]x).sup.T] P[delta]x + [delta][x.sup.T]Q[delta]x (13)

with [delta]x representing a value of the state variables in some neighbourhood of the origin. The non-linear term [alpha]([delta]x) represents the contribution of all higher order derivatives of dJ/dt to the Taylor series.

For linear systems, it is clear that this non-linear term vanishes identically for all x. Closed-loop stability is thus ensured by Equation (10). For non-linear systems, the contribution of [alpha]([delta]x) will provide a measure of the effect of non-linearity to the closed-loop stability.

For any initial condition [delta]x, the quadratic term -[delta][x.sup.T] Q[delta]x represents the nominal LQ system performance. The term [alpha]([delta]x) is a measure of the departure of the closed-loop non-linear system from its nominal LQ behaviour. The contribution of the non-linear term [alpha]([delta]x) must be evaluated for each [delta]x. In order to develop a meaningful summary measure, we propose to evaluate the contribution of [alpha]([delta]x) over a set of possible state variable values.

As in previous work, we consider a scaling region defined using a suitable positive definite matrix S. Assume that a suitable scaling matrix S has been defined. We define the change of variables, z = S[delta]x where S is a non-singular positive matrix that defines the scaling region [OMEGA] = {[delta]x [member of] [R.sup.n]|[delta][x.sup.T] [S.sup.T] S[delta]x [less thano r equal to] 1}. It follows that [parallel]z[parallel] = 1 [x.sub.0] [member of] [OMEGA] (where [parallel] x [parallel] represents the Euclidean norm). The Taylor series Equation (12) can be re-expressed in z-coordinates as follows:

dJ/dt [approximately equal to] - [Z.sup.T] [S.sup.-T] [QS.sup.-1] z + [alpha] (z) (14)

The second quantity evaluated on the right-hand side of inequality Equation (14) is defined as the stability sensitivity measure (SSM):

SSM = [alpha](z) (15)

The magnitude of the SSM is relative to the magnitude of the quadratic term on the right-hand side of Equation (14). The SSM provides an informative assessment of the potential application of the cost function J = [x.sup.T]Px as a candidate Lyapunov for the closed-loop system. A large value indicates that further analysis is required to provide a suitable Lyapunov function (if any) for the closed-loop system over the region of interest. It does not constitute a strict measure of stability, in the sense of Lyapunov for example.

A more representative measure can be obtained by evaluating a mean of the SSM over the area, A, of the unit ball in the z-coordinates defined as:

MSSM [[integral].sub.A] [alpha] (z)dA (16)

Note that the value of SSM can be evaluated as follows:

MSSM = [[integrall].sub.A] dJ/dt (z) dA + [[integral].sub.A] [z.sup.T] [S.sup.-T] [QS.sup.-1] zdA

where dJ/dt is given by Equation (8).

The measure MSSM is used as a measure of the potential instability of the closed-loop non-linear system subject to LQR control. MSSM is expressed relative to the integral:

[lambda](t) = [[integral].sub.A] [z.sup.T] [S.sup.T] [QS.sup.-1] zdA

A positive value exceeding [lambda] indicates that the closed-loop system does not inherit the properties of the nominal linear system for initial conditions in the scaling region [OMEGA]. One may want to reduce the resulting operating region until a negative is obtained. Note that a negative value of MSSM does not necessarily imply that the non-linear closed-loop system is stable; however, it does indicate an overall tendency to provide a closed-loop stable non-linear control system. That is, given the cost function J = [x.sup.T]Px, one can construct positively invariant sets using level sets on J contained in [OMEGA]. It is clear that in situations where the instability region over [OMEGA] is much smaller than the stability region, the MSSM can still yield a negative measure, leading to an ambiguous result. In such cases, one may need to increase the size of the scaling region and repeat the analysis. Increasing the region will amplify the non-linear effects in the closed-loop and hence provide a better assessment of non-linearity. Another approach would be to increase the control penalty matrix R in the LQ cost functional.

To remedy this problem, one can consider alternative measures. In order to develop a suitable measure, we may reiterate the objective of the SSM. In using the SSM, we seek a measure that quantifies the degree to which the derivative of the cost function J, corresponding to the design of a LQ controller about a specific point (re-centred to the origin x = 0), is negative. Clearly, if it can be established that dJ/dt (in Equation (8)) is negative everywhere in a specified region then the non-linear system is not susceptible to instabilities. As a result, the non-linear control system performs as intended with the LQ controller that is stabilizing over the specified region. On the other hand, it would be informative to have a measure of the proportion of the set of states where the sign of dJ/dt is positive and, hence, where the non-linear control system is not performing as intended. That is the set of points at which the cost functional J ceases to act as a Lyapunov function for the control system.

Given a region [OMEGA] of the state space with area A, one can compute the following measure, called the sensitivity measure:

SM = [[integral].sub.A] I (dJ/dt) dA (17)

where

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (18)

is the indicator function and [OMEGA] is the region of interest in the state space. The value of SM provides a measure of the fraction of states that lead to unstable conditions. A nonzero value indicates the potential for instability in the closed-loop system.

The evaluation of the SSM and SM is straightforward. For any value of the scaled variable z, one computes the corresponding value of [delta]x using the user-defined scaling matrix. For each [delta]x, one evaluates the right-hand side of Equation (8) and the LQ closed-loop system contribution [delta][x.sup.T] Q[delta]x. The value of the nonlinear contribution [alpha] to the closed-loop system is then obtained directly using Equation (13).

The mean values MSSM and [lambda](t) and the stability measure, SM, can be computed numerically using functions in Matlab's Numerical Integration Toolbox, such as those based on a Gaussian quadrature technique. Due to the potential singularities arising from the evaluation of SM, a Gauss-Chebyshev integration procedure is used. Alternatively, the value of SM can be approximated using the following expression:

[SM.sub.[beta]] = [[integral].sub.A] (1/2 + 1/[pi] [tan.sup.-1] ([beta] dJ/dt))dA (19)

where [beta] is a positive constant such that [lim.sub.[beta][arrow right][infinity]] [SM.sub.[beta]]. = SM.

We have found these techniques to be both efficient and accurate for integration over the area, A, or the volume, V, of the scaling region.

Yet another measure can be considered in the current context. In contrast to the measures provided above, we seek a measure of the degree of positive invariance of the scaling region of interest S for the non-linear closed-loop system subject to a LQ controller. Consider a set of initial conditions, [[OMEGA].sub.0] [subset] S. We assume that [[OMEGA].sub.0] is a user-defined compact subset of S. In most cases, the set [[OMEGA].sub.0] is chosen as some inner approximation of S. We want to measure the proportion of the set [[OMEGA].sub.0], which yield trajectories that remain in S. For this purpose, let us define the indicator function:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (20)

where x(0) [member of] [[OMEGA].sub.0] is an initial condition for the closed-loop system and x(t; x(0)) represents the trajectory of the closed-loop system starting from initial condition x(0). Using the indicator function, we define the following stability measure, [SM.sub.inv] ([[OMEGA].sub.0], S), as follows:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (21)

where [A.sub.0] is area covering the boundary of the compact set [[OMEGA].sub.0] and x(0) [member of] [A.sub.0].

To compute [SM.sub.inv]([[OMEGA].sub.0], S), we consider the set of trajectories chosen over a fixed horizon, T, and compute the integral over [A.sub.0] using a Gauss-Chebyshev quadrature method. This noted by adding the variable T as an argument of the measure, as [SM.sub.inv] ([[OMEGA].sub.0], S, T). In most cases, it will not be possible to compute the value as T [arrow right] [infinity]. In comparison to SM and SSM, the computation of [SM.sub.inv] ([[OMEGA].sub.0], S, T) constitutes a major challenge. Its application is limited to simple system. Further research is ongoing to evaluate the validity of the measure for more general problems.

Effect of Non-Linearity on Performance

If one designs a LQR controller based on the linearization of a non-linear system at a specific equilibrium point then theory guarantees that the nominal controller performance can be achieved locally at that point if the linearization is both controllable and observable. The extent to which the nominal performance can be achieved is directly related to the extent of non-linearity of the plant. If one initiates the system at an initial condition x(0) = [x.sub.0] in a neighbourhood of the equilibrium point, the optimal linear controller performance dictates the optimal path of the closed-loop system unless the effects of non-linearity are significant. This effect is directly related to the choice of initial conditions x0. In what follows, we propose to quantify the change in nominal controller performance due to non-linearity by evaluating the sensitivity of the nominal linear quadratic controller performance to changes in the initial conditions, x(0) = [x.sub.0].

To ascertain the effect of [x.sub.0] on the performance of the closed-loop system, Equation (6) is differentiated with respect to [x.sub.0] and evaluated along the closed-loop trajectory starting with initial conditions x(0) = 0. Differentiating [J.sup.*] with respect to [x.sub.0], we obtain:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Let [x.sub.l0] and [x.sub.m0] (l = 1, ..., n, m = 1, ..., n) be elements of the vector [x.sub.0]. The second and third order derivatives of J with respect to [x.sub.0] are given by:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], (22)

and

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

respectively.

All the derivatives of [J.sup.*] are evaluated at x(0) = 0 to obtain a local measure of sensitivity that applies to the closed-loop system operating at the set point, (x, u) = (0, 0).

The computation of the derivatives of J requires the calculation of the first and second order sensitivity coefficients of x(t) with respect to x0. The sensitivity coefficients are computed by the integration of the sensitivity equations. Differentiating (1) with respect to x0 and reversing the order of differentiation, we obtain:

d/dt [partial derivative]x/[partial derivative][x.sub.i0] = [partial derivative]F/[partial derivative][x.sub.m] [partial derivative][x.sup.m]/[partial derivative][x.sub.i0] (23)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (24)

where [x.sub.n] and [x.sub.m] are elements of the vector x(t). All summations along indices m and n are presented following the Einstein notation (i.e., < x, y >= [x.sub.m] [y.sup.m] where <,> represents the inner product between two vectors, x and y). The initial conditions for the first and second order sensitivity coefficients are:

[partial derivative]x/[partial derivative][x.sub.0] ([t.sub.0]) = [I.sub.nxn], [[partial derivative].sup.2]x/[partial derivative][x.sub.0] [partial derivative][x.sup.T.sub.0] ([t.sub.0] = [0.sub.nxnxn] (25)

where [I.sub.n x n] is the n dimensional identity matrix and [0.sub.nxnxn] is an n x n x n three-dimensional array of zeroes.

Let us consider the linearized system Equation (2) subject to the LQ control. If one computes the first and second order sensitivities for this linear system as given in Equations (23) and (24), one obtains:

[partial derivative]x/[partial derivative][x.sub.0](t) = exp ((A - BK) (t - [t.sub.0])), [[partial derivative].sup.2]x/[partial derivative][x.sub.0] [partial derivative][x.sup.T.sub.0](t) [equivalent to] [0.sub.nxnxn] (26)

Consequently, the third order derivatives, [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], and all higher order derivatives of the cost functional with respect to [x.sub.0] must be identically zero and hence, for linear systems, we see that the value function [J.sup.*] is the quadratic function of [x.sub.0] at each time t given by Equation (22).

Clearly, this will not be the case for non-linear systems (except in a small number of special cases). Hence, the relative magnitude of [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and all other higher order derivatives provides an indication of the non-linear effect in the closed-loop system.

The second and third order derivatives can be rewritten in matrix form as:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (27)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (28)

The Taylor series expansion about x0 = 0 is then given by:

[J.sup.*] (z, t) [approximately equal to] [z.sup.T] [LAMBDA](t)z + [alpha] (z, t) (29)

where [alpha](z) = [J.sup.*](z) - [z.sup.T] [LAMBDA]z and

[LAMBDA](t) = [S.sup.-T] [partial derivative]x (t)/[partial derivative][x.sub.0] P [partial derivative]x(t)/[partial derivative][x.sub.0] [S.sup.-1] (30)

for each scaled initial condition, z, and time t. The significance of the non-linearity effect depends on the relative contribution of [alpha] (z, t) to Equation (29). The quantity [alpha] (z) provides a measure of the degree of departure of the linear controller performance in the actual non-linear control system from the nominal linear controller performance. The magnitude of [alpha](z) is normalized by considering its value on the unit ball {z| [[parallel]z[parallel].sup.2] = 1}. The quantity [alpha](z), is called the performance sensitivity measure or PSM.

PSM(z) = [alpha](z) (31)

This normalized measure, as taken over a region of unit norm in the scaled variable z, provides a meaningful value relative to the magnitude of the quadratic term [z.sup.T][LAMBDA]z. In this study, we propose the following measure called the performance sensitivity measure (PSM). It is given by:

PSM(z,t) = [alpha] (z,t) (32)

The quantity PSM(z, t) depends explicitly on a specific value of the initial condition z and time t. A summary measure is the following mean measure, called the mean performance sensitivity measure (MPSM):

MPSM(t) = [[integral].sub.A] ([alpha] (z,t)) dA = [[integral].sub.A] (J(z,t) - [z.sup.T] [LAMBDA](t)z)dA (33)

where the integration is taken over the area, A, of the unit sphere ([parallel]z[parallel] = 1). To compute the measure MPSM(t), we compute the the multi-dimensional integrals:

[bar.J](t) [[integral].sub.A] J(z, t)dA (34)

and

[rho](t) = [[integral].sub.A] (z.sup.T] [LAMBDA](t)z)dA (35)

The resulting value of MPSM(t) = [bar.J](t) - [rho](t). Note that Equation (34) is a function of the initial condition z. The magnitude of MPSM(t) relative to [rho](t) provides an assessment of the effect of non-linearity on the intended nominal controller performance. This leads to a measure called the relative mean performance sensitivity measure (RMPSM) defined as:

RMPSM (t) = MPSM(t)/[rho](t) 36)

If the system behaves as a linear system with the intended linear system performance then the value of RMPSM(t) should be much smaller than 1. As the value of RMPSM(t) increases, the contribution of the non-linear terms to the closed-loop performance will become significant and the intended nominal performance cannot be achieved.

In general, the degree of sensitivity that can be tolerated will change from application to application. If a very high degree of performance must be obtained then a relatively small value of the PSM can be viewed as important (<0.1). If one needs a linear controller that operates satisfactorily (but not strictly optimal) then a larger value would be tolerable (<0.5).

A more conceptual argument leads to the following assessment of the relative magnitude of the RMPSM(t), which is based on the following six [sigma] argument.

Suppose that we develop a control system that provides six [sigma] performance. That means that the linear controller system (based on the linearized model) is such that:

[rho](t) = 36[[sigma].sup.2]

Assume that we deem the change in performance to be significant when the resulting process performance yields a decrease of one [sigma]. That is, we assume the resulting closed-loop system yields:

[bar.J] = 25[[sigma].sup.2]

This would mean that the contribution from the non-linear term would be equal to:

MPSM(t) = 36[[sigma].sup.2] - 25[[sigma].sup.2] - 9[[sigma].sup.2]

In this case, the resulting value of the RMPSM(t) would be 9/36=0.25. Thus, at the six [sigma] level, a value of 0.25 would result in a change of one [sigma] unit. Applying the same argument, a two [sigma] change would result in a RMPSM(t) of (36-16)/36=0.56 and for three [sigma], we get a RMPSM(t) of (36-9)/36=0.75.

This simple six [sigma] argument provides a clear representation of the implications of the measure. It also gives a clearer picture to the practitioner of the potential performance changes that would result from the implementation of a linear controller on a nonlinear process. Depending on the degree of performance that can be sacrificed, the RMPSM provides a measure of the resulting inflation of the cost function that is due to the process non-linearity. In general, a value of 0.25 to 0.3, leading to a change of approximately one [sigma] in a six [sigma] framework, provides a noticeable change in process performance.

The measure RMPSM(t) can be evaluated at each instant, t, for any initial conditions, x(0). For any given sequence, the overall average, [bar.RMPSM](t) = [integral].sup.t.sub.0] RMPSM ([tau]) d[tau], provides a summary value of the PSM for this process. The magnitude of RMPSM is evaluated relative to the overall average value given by:

[bar.[LAMBDA]](t) = [[integral].sup.t.sub.0] [rho] ([tau])d[tau]

We propose the following relative performance sensitivity measure RPSM:

RPSM = [bar.RMPSM](t)/[bar.[LAMBDA]](t) (37)

The interpretation of the RPSM is the same as for the measure RMPSM(t).

APPLICATION OF THE SENSITIVITY MEASURES

Example 1: Application to a Bioreactor Control Problem In this example, we consider the bioreactor model given in Hernjak and Doyle III (2003):

[[??].sub.1] = [mu](x)[x.sub.1] - u/V [x.sub.1]

[[??].sub.2] = - 1/[Y.sub.x/s] [mu](x) [x.sub.1] + u/V ([S.sub.f] - [x.sub.2]) (38)

[[??].sub.3] ([alpha][mu](x) + [beta] [x.sub.1] - u/V [x.sub.3]

where

[mu](x) = [mu].sub.m] (1 - [x.sub.3]/[P.sub.m]) [x.sub.2]/[K.sub.m] + [x.sub.2] + [x.sup.2.sub.2]/[K.sub.i]

and, where [x.sub.1] is the biomass concentration, [x.sub.2] is the substrate concentration, [x.sub.3] is the product concentration and u is the substrate feed flow rate. We consider the steady-state conditions outlined in Hernjak and Doyle III (2003). They are listed in Table 1.

We will apply the SM, SSM and PSM measures to assess the effect of non-linearity when the process is controlled with a LQR controller.

We first compute the linearization of the system Equation (38) about the steady-state conditions listed in Table 1. It is given by the following linear dynamical system:

[??] = Ax + Bu

where

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (39)

The LQR controller tuning matrices, Q and R, are given by:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

For the assessment of non-linearity, we consider the operating region about the steady-state conditions designated by the following scaling matrix:

[S.sub.0] = Diag[(1, 0.25, 0.01)]

For these settings, the value of MSSM is -368 compared to the value [lambda] of 322. The negative value of MSSM indicates that the average contribution of the non-linearity to the stability of the closed-loop is beneficial. In contrast, the value of SM is equal to 0.5. This value indicates that, for about half of the volume of the scaling region of interest, the functional J = [x.sub.T] Px is strictly increasing along the trajectories of the closed-loop system. Thus, the nominal stability conditions of the LQ control are not inherited in the non-linear closed-loop process. A different analysis is required to establish the positive invariance of the scaling region. For a fixed time horizon, T = 400 and the set of initial conditions, [[OMEGA].sub.0] = {[delta]x [member of] [R.sup.n] |[delta][x.sup.T] [S.sup.T] S[delta]x [less than or equal to] r} with r = 0.9, the value of [SM.sub.inv] ([[OMEGA].sub.0], S, 400) is 0.27. The result demonstrates that the scaling region cannot be considered to be a positively invariant set of the closed-loop dynamics. If one reduces r to 0.75 and 0.5 however, one obtains [SM.sub.inv] ([OMEGA].sub.0], S, 400) values of 0.024 and 0, respectively. The results confirm that there exist a set of initial conditions such that the scaling region is positively invariant (at least up to time t = 400 h). This would suggest that the system is subject to some mild non-linearities over the scaling region and the system can operate safely in that region.

Figure 1 shows the value of the MPSM and the value of [rho] as a function of time. The average value RMPSM is evaluated at 0.1843 when averaged from 0 to 400 h. The results suggest that this non-linear process is subject to weak non-linear effects. This corroborates the results given in Hernjak and Doyle III (2003). The process can, therefore, be expected to perform as intended for the nominal linear quadratic design. To demonstrate the significance of this result, we show in Figures 2 and 3 the simulation of the system from an initial condition that lies on the boundary of the scaling region. The state variables are shown on Figure 2 while the input variables are in Figure 3. On the same plots, we show the simulation of the linearized system subject to the LQR controller. The results demonstrate that the non-linear closed-loop process with LQR control behaves very closely to its linear approximation. Thus, the closed-loop operates as designed for the nominal linearized model.

[FIGURE 1 OMITTED]

The analysis, thus, provides a very effective technique to test whether a non-linear system can be effectively controlled using a linear quadratic controller. For this bioreactor system, the answer is affirmative for the scaling region considered.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

Example 2: Application to a Benchmark Chemical System Non-linearity in chemical systems can arise from several sources, including bilinear terms in heat transfer expressions for energy balances, and reaction kinetics governed by the Arrhenius equation. The selection of the controlled input may also result in different non-linearities, depending upon the process under consideration.

In this section, the control of a continuously stirred tank reactor (CSTR) model is examined. The sensitivity measures are computed for numerous operating points.

The van de Vusse reactor, often employed as a benchmark non-linear process in the literature, contains one feed species, identified by the letter A, which reacts irreversibly to form three unique compounds, B, C and D as follows:

A [right arrow] B [right arrow] C

2A [right arrow] D

The model equations, taken from (Gatzke and Doyle III, 1998), are:

[dx.sub.1]/dt = u [C.sub.A0] - [x.sub.1]) - [k.sub.1][x.sub.1] - [k.sub.3][x.sup.2.sub.1] [dx.sub.2]/dt = - [ux.sub.2] + [k.sub.1][x.sub.1] - [k.sub.2][x.sub.2] (40)

where [x.sub.1] and [x.sub.2] are the concentrations of species A and B in mol/L, respectively, and T is the temperature of the reactor in K. The model parameters u, and [C.sub.A0] represent the dilution rate, in [h.sup.-1], and inlet concentration of species A, in mol/L, respectively. The constant model parameters (Perez et al., 2002) are given in Table 2.

[FIGURE 4 OMITTED]

It is generally presumed that the desired product is species B, with C and D being unwanted by-products of the reaction. We consider the control of the outlet stream composition by manipulation of the dilution rate, u. Substituting in the parameter values given in Table 2 yields the steady-state locus shown in Figure 4 for dilution rates, u, in the range [2.19, 26.19]. The locus shows that there is a point of operation where the concentration of species B is maximized, near "d". Although this is the most desirable condition, we consider other points along the locus in computing the RMPSM. The objective function is chosen as:

J = [[integral].sup.[infinity].sub.0] (100 [(x(t) - [x.sub.s]).sup.T] (x(t) - [x.sub.s]) + [(u(t) - [u.sub.s]).sup.2])dt

The operating interest in this case consists of the state variable values that lie in the region [[??].sub.1] [member of] [x.sup.nom.sub.1] [+ or -] 0.4], and [[??].sub.2] [member of] [[x.sup.nom.sub.2] [+ or -] 0.04]. The corresponding scaling matrix S is given by:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Several points were chosen along the steady-state locus by fixing the input dilution rate. The list of steady-state locus points and the sensitivity measures are given in Table 3. The computed values for [bar.MPSM] [bar.MSSM] , and SM indicate that the non-linear effects are negligible for the performance requirement and operating region considered. Therefore, a LQ controller could be used effectively for the control of this process with guarantee of stability and performance in the region of interest. Figure 6 shows the dynamic behaviour of MPSM and [rho] at the steady-state point "a". Similarly, Figure 5 shows the dynamic behaviour of SSM and [lambda] at the steady-state point "a". Since the value of SSM is negative and, hence, consistently less than [lambda], we conclude that the effect of non-linearity on the closed-loop system stability is minimal for the operating region chosen. To illustrate this result, the closed-loop behaviour of the nonlinear control system with LQ control is compared to the closed-loop behaviour of the nominal linear control system with LQ control. The simulation results are in Figures 7 and 8. The deviation of the state variables from the steady-state values for point "a" are shown in Figure 7. The deviation of the control variable are shown in Figure 8. As predicted by the analysis, the non-linear system behaves like the nominal linear system and achieves the LQ performance requirement.

[FIGURE 5 OMITTED]

In this example, the analysis confirms that the LQ control system is guaranteed to operate as designed. Furthermore, the SM measure guarantees that the cost function J = [x.sub.T]Px can act as a Lyapunov function for the system over the scaling region.

[FIGURE 6 OMITTED]

[FIGURE 7 OMITTED]

[FIGURE 8 OMITTED]

In closing, we consider the value of the measure [SM.sub.inv] ([OMEGA].sub.0], S, T) at point "a". In this case, we consider a final time T of 0.1. The set of initial conditions [[OMEGA].sub.0] is chosen as [[OMEGA].sub.0] = {[delta]x [member of] [R.sup.n]|[delta][x.sup.T] [S.sup.T] S[delta]x [less than or equal to] r}. For the values r = 0.75 and 0.5, the corresponding values of [SM.sub.inv]([[OMEGA].sub.0], S, T) are 0.94 and 0.69. Thus, in this case the scaling region is not positively invariant. This is to be expected in this case since the choice of scaling region does not take into account the dynamics of the closed-loop process.

CONCLUSIONS

In this paper, we have introduced two sensitivity measures that provide an assessment of the effect of non-linearity on LQR control performance in non-linear control systems. In particular, we have proposed a summary measure, the RMPSM, which provides a local assessment of the sensitivity of nominal controller performance in a non-linear control system. We have also developed the SSM that measures the impact of non-linearity on closed-loop system stability. Using two simulation studies, the applicability of the measures was presented. The results demonstrate that the measures can be used to predict the viability of LQ control in non-linear systems.

Manuscript received December 12, 2005; revised manuscript received April 24, 2006; accepted for publication May 28, 2006.

REFERENCES

Allgower, F., "Definition and Computation of a Nonlinearity Measure," IFAC Nonlinear Control Systems Design, 257-262 (1995a).

Allgower, F., "Definition and Computation of a Nonlinearity Measure and Application to Approximate i/o-Linearization," Technical Report, Universitat Stuttgart (1995b).

Desoer, C. A. and Y.-T. Wang, "Foundations of Feedback Theory for Nonlinear Dynamical Systems," IEEE Transactions on Circuits and Systems 25(2), 104-123 (1980).

Eker, S. A. and M. Nikolaou, "Linear Control of Nonlinear Systems: Interplay Between Nonlinearity and Feedback," AIChE J. 48(9), 1957-1980 (2002).

Gatzke, E. P. and F. J. Doyle III, "Moving Horizon Parameter Estimation Using Qualitative Knowledge and Multiple Linear Models," Technical Report, University of Delaware (1998).

Guay, M., R. Dier, J. Hahn and P. J. McLellan, "Effect of Process Nonlinearity on Linear Quadratic Regulator Performance," J. Proc. Control 15, 113-124 (2005).

Guay, M. and J. F. Forbes, "On the Effect of Process Nonlinearity on Linear Quadratic Regulator Stability and Performance," in Proc. 43rd IEEE CDC, Paradise Island, Bahamas (2004), pp. 3299-3304.

Guay, M., P. J. McLellan and D. W. Bacon, "Measurement of Nonlinearity in Chemical Process Control Systems: The Steady-state Map," CJChE 73, 868-882 (1995).

Haber, R., "Nonlinearity Tests for Dynamic Processes," IFAC Identification and System Parameter Estimation, 409-414 (1985).

Hernjak, N. and F. J. Doyle III, "Correlation of Process Nonlinearity with Closed-Loop Disturbance Rejection," Ind. Eng. Chem. Res. 42, 4611-4619 (2003).

Nikolaou, M., "When is Nonlinear Dynamic Modelling Necessary?," in Proc. 1993 ACC, San Francisco, CA, U.S.A. (1993), pp. 1067-1071.

Ogunnaike, B. A., R. K. Pearson and F. J. Doyle III, "Chemical Process Characterization: Applications in the Rational Selection of Control Strategies," Proc. Europ. Cont. Conf., June 28-July 1 (1993), pp. 1067-1071.

Perez, H. B., B. Ogunnaike and S. Devasia, "Output Tracking between Operating Points for Nonlinear Processes: van de Vusse Example," IEEE Trans. Contr. Syst. Tech. 10, 611-617 (2002).

Stack, A. J. and F. J. Doyle III, "A Measure for Control Relevant Nonlinearity," Proc. of 1995 American Control Conference, June 21-23, 1995.

Stack, A. J. and F. J. Doyle III, "Application of a Control-Law Nonlinearity Measure to the Chemical Reactor Analysis," AIChE J. 43(2), 425-439 (1997a).

Stack, A. J. and F. J. Doyle III, "The Optimal Control Structure: An Approach to Measuring Control-Law Nonlinearity," Comp. Chem. Eng. 21(9), 1009-1019 (1997b).

Schweickhardt, T., F. Allgower and F. J. Doyle III, "Nonlinearity Quantification for the Optimal State Feedback Controller," in Proc. of ECC 2003, Cambridge, U.K. (2003), pp. 4611-4617.

Martin Guay (1) * and J. Fraser Forbes (2)

(1.) Department of Chemical Engineering, Queen's University, Kingston, ON, Canada K7L 3N6

(2.) Department of Chemical and Materials Engineering, University of Alberta, Edmonton, AB, Canada T6G 2G6

* Author to whom correspondence may be addressed. E-mail address: guaym@chee.queensu.ca

Table 1. Simulation parameters--bioreactor system u = 0.08 L [h.sup.-1], [x.sub.1] = 1.80 g [L.sup.-1], [x.sub.2] = 10.5 g [L.sup.-1], [x.sub.3] = 48.7 g [L.sup.-1] V = 9.92 L, [S.sub.f] = 15.0 g [L.sup.-1], [K.sub.m] = 1.2 g [L.sup.-1], [K.sub.i] = 22.0 g [L.sup.-1] [P.sub.m] = 0.40 g [L.sup.-1], [Y.sub.X/S] = 0.40 g [g.sup.-1], [alpha] = 2.20 g [g.sup.-1], [beta] = 0.2 [h.sup.-1], [[mu].sub.m] = 0.48 [h.sup.-1] Table 2. Model parameters for the isothermal CSTR with van de Vusse kinetics Parameter Value [k.sub.1] 50 [h.sup.-1] [k.sub.2] 100 [h.sup.-1] [k.sub.3] 100 mol [A.sup.-1][h.sup.-1] [C.sub.A0] 1 mol/L Table 3. RMPSM and SSM values for various operating points of the isothermal van de Vusse CSTR Point Dilution label rate [x.sub.1] [x.sub.2] a 19.6 0.215 0.090 b 32.6 0.292 0.110 c 78.0 0.451 0.127 d 180.9 0.618 0.309 e 293.6 0.708 0.090 Point label [bar.RMPSM] MSSM SM a 0.089 -0.014 0 b 0.062 -0.018 0 c 0.033 -0.019 0 d 0.009 -0.003 0 e 0.006 -0.003 0

Printer friendly Cite/link Email Feedback | |

Author: | Guay, Martin; Forbes, J. Fraser |
---|---|

Publication: | Canadian Journal of Chemical Engineering |

Date: | Feb 1, 2007 |

Words: | 8314 |

Previous Article: | Hydrogen yield from low temperature steam reforming of ethanol. |

Next Article: | Simultaneous measurement bias correction and dynamic data reconciliation. |