Group Title: inverse optimal linear regulator problem
Title: The Inverse optimal linear regulator problem
CITATION PDF VIEWER THUMBNAILS PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00097608/00001
 Material Information
Title: The Inverse optimal linear regulator problem
Physical Description: viii, 152 leaves. : ; 28 cm.
Language: English
Creator: Elder, Joseph Marcus, 1946-
Publication Date: 1972
Copyright Date: 1972
 Subjects
Subject: Automatic control   ( lcsh )
Control theory   ( lcsh )
Electrical Engineering thesis Ph. D
Dissertations, Academic -- Electrical Engineering -- UF
Genre: bibliography   ( marcgt )
non-fiction   ( marcgt )
 Notes
Thesis: Thesis--University of Florida.
Bibliography: Bibliography: leaves 148-152.
Additional Physical Form: Also available on World Wide Web
General Note: Typescript.
General Note: Vita.
 Record Information
Bibliographic ID: UF00097608
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
Resource Identifier: alephbibnum - 000585258
oclc - 14205973
notis - ADB3891

Downloads

This item has the following downloads:

PDF ( 5 MBs ) ( PDF )


Full Text















THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM


Dy

Joseph Marcus Elder, Jr.




















A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF
THE UNIVERSITY OF FLORIDA IN PARTIAL
FULFILLMENt OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY


UNIVERSITY OF FLORIDA
1972





























To my grandfather
Roy 1W. Estridge
who never doubted
and to my wife Dina
who never despaired

















ACKNOWLEDGMENTS


The author wishes to express his sincere appreciation to the

members of his supervisory committee: Dr. T. E. Bullock, chairman;

Dr. W. H. Boykin, and Dr. A. E. Durling for their counsel and patience.

Special thanks are due to Dr. Bullock who gave unselfishly of his time

and energy; and abject apologies are due to his wife, Jeannette, who

endured the many late hours and missed dinners.

Thanks also are due to Dr. C. V. Shaffer who had an uncanny

knack for dispatching the multitude of small problems encountered along

the way, to Dr. L. E. Jones who was always ready to help, and to

Mr. Y. T. Parker for a great deal of food for thought.

The author is grateful to the University of Florida Graduate

School, the Department of Electrical Engineering, and the National

Aeronautics and Space Administration for financial assistance.














TABLE OF CONTENTS


Page

ACKNOWLEDGMENTS .. . . .. . . . . . . . . iii

LIST OF FIGURES . . . . . . . . ... . . vi

ABSTRACT .. . . . . . .. .... . . . vii

CHAPTER

I INTRODUCTION . . . . . . . . 1

1.1 Background . . . . . . . . . . 1
1.2 Survey of Previous Work . . . . . . 4

II THE LINEAR, QUADRATIC COST OPTIMIZATION PROBLEM . . 7

2.1 Introduction . . . . . . . . . 7
2.2 The Linear System, Quadratic Cost Problem . . 8
2.3 The Optimal Control Problem . . . . . 9

The Euler-Lagrange Equations . . . . 9
The Riccati Equation . ... ... . 10
The Conjugate Point Condition . . . ... 12
Sufficient Conditions Existence . . . 18

2.4 The Optimal Linear Regulator Infinite
Final Time . . . . . . . . . 20

Existence and Stability .. . . . . 20

2.5 Summary . . . . . . . .. . . 25

III THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM . 27

3.1 Introduction . . . .. . . . . . . 27
3.2 When Is a Linear Control System Optimal? . . 28
3.3 Implications of Optimality ... . . . .. 29
3.4 Companion Matrix Canonical Form . . ... 33
3.5 Characterization of the Equivalence Class of Q's 35
3.6 Resume of Y-Invariant 'Matrices . . . ... 57

General Structure . . . . . . 58
Spectral Factorization . . . . 59
Diagonal . . . . . . . . . 59

3.7 Summary . . . . . . ..... . 61











TABLE OF CONTENTS (Continued)


CHAPTER

IV THE INVERSE PROBLEM AND LlNEAR REGULATOR DESIC'. . .

4.1 Introduction . . . . . . . . . .
4.2 Pole Placement by Performance Index Desiratil. ..
4.3 Design by Performance Index Iteratior. . . .
4.4 Design by Explicit Performance Irdex
Specification . . . . . . . . .
4.5 Sampled-Data Controller Design . . . . .

V CONCLUSIONS . . . . . . . . . . .

5.1 Summary of Results . . . . . . . .
5.2 Suggestions for Future Research . . . . .

APPENDICES

A ADDITIONAL PERFORMANCE INDICES ACCOMMODATED
BY THE THEORY . . . . . . . . . . .


A.1 Introduction . . . . . . .
A.2 Quadratic Performance Index with
Exponential Weighting . . .
A.3 Quadratic Performance Index with
Cross-Products . . . . . .
A.4 Quadratic Performance Indices with
Derivative Weighting . . . . .

B PROOF OF SUFFICIENCY OF THEOREM 3.1 . . .


C THE INVERSE PROBLEM NUMERICAL DETAILS . . . .


C.1 Introduction . . . .
C.2 Computation of Magnitude -
C.3 Test for Non-negativity of
Even Polynomials . . .
C.4 Spectral Factorization .
C.5 Sample Program . . .
C.6 Subprogram Listing .


118
118


Square Polyramials
Real,


REFERENCES . . . . . . . . . . . .


Page

63

63
65
80

92
98

103

103
104


. . 107

. . 107

. . 109

. 111


148













LIST OF FIGURES


Figure Page

2.1 Conjugate Trajectory. . . . . . . 12

3.1 Return Difference . . . . . . . .... . 30

3.2 Nyquist Plot of Optimal System . . . . ... 31

3.3 System Pole Locations . . . . . . . . 38

3.4 Sparse Equivalent Matrix . . . . . . . 51

3.5 Zero Locations of Ev(F) and Od(F) . . . .... . 57

3.6 Structure of Y-Invariant Matrices . . . . . 58

4.1 Control System Design Procedures . . . . . 64

4.2 Proposed Closed-loop Pole Configuration . . ... 66

4.3 Procedure for Specification of a Performance Index
from Proposed Pole Locations . . . . ... . 70

4.4 Eigenvalue Distributions for Q . . . . . . 76

4.5 Preliminary Design Pole Configuration . . . ... 87

4.6 Transient Response vs q. . . . . . . . . 88

4.7 Nyquist Plot of Final Design . . . . ... . 89

4.8 Root-Locus Plot of Final Design . . . . ... 89

4.9 System Responses to a Step Input Preliminary
and Final Designs . . . . . . . .. . 90

A.1 Modified Plant . . . . . . . . 112

A.2 Synthesis of Optimal Control for Performance
Index with Control Derivative Weighting . . ... 114









Abstract of Dissertation Presented to the
Graduate Council of the University of Florida in Partial Fulfillment
of the Requirements for the Degree of Doctor of Philosophy


THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM

By

Joseph Marcus Elder, Jr.

March, 1972


Chairman: Dr. T. E. Bullock
Major Department: Electrical Engineering


The principal objective of this research is to extend the results

of R. E. Kalman's original analysis of the inverse optimal control prob-

lem in order to create a design tool which allows the designer to employ

the powerful and sophisticated techniques of optimal control theory

when the correct choice of a performance index is not apparent. Specif-

ically, methods are developed for the design of optimal linear regulators

(both continuous and discrete) which satisfy classical performance

specifications in addition to the minimality of a quadratic functional.

This permits the great computing power of optimal control theory to be

brought to bear on problems which previously could be accommodated only

by the cut-and-try methods of classical design schemes. In passing,

insights into the basic nature of optimal regulators are developed in

terms of classical concepts, unifying some important notions in clas-

sical and modern control theory.

The contributions of this research can be summarized as follows:

1. A complete theoretical investigation of the inverse

optimal linear regulator problem for quadratic per-

formance indices with positive semidefinite state









weighting matrices and scalar input systems is reviewed

and preliminary results for the case where the weighting

matrix is allowed to be sign indefinite are given.

2. The equivalence of performance indices for scalar input

linear quadratic loss problems is resolved and a proce-

dure for generating the entire equivalence class of cost

functions equivalent to a given performance index is

developed.

3. Practical numerical methods for determination of system

optimality and computation of solutions to the scalar

input inverse problem are discussed.

4. Some of the effects of specific elements of the perform-

ance index on optimal system performance and pole loca-

tions are determined.

5. The problem of designing optimal systems to meet classical

performance specifications is encountered and some defin-

itive results obtained.

6. A solution to the problem of designing a sampled-data

controller to approximate the performance of continuous

controller is given.














CHAPTER I


INTRODUCTION




1.1 Background

In 1964 R. E. Kalman published a paper [K1]1 which has come to

have considerable impact on the theory of optimal control. It dealt

not with the conventional optimal control problem of computing a system

trajectory which extremizes a specified performance index, but rather

with the "inverse" problem of determining what performance indices, if

any, are extremized by a specified control. Kalman restricted his

attention to the case where 1he system is linear and the integral per-

formance index is quadratic in the states and control. No immediate

direct application was made of Kalman's results, but they were of great

value in the application and interpretation of his earlier analysis

[K2] of what is now called the "linear quadratic loss" optimization

problem.

Quadratic functionals have long been studied in the calculus

of variations [Gl] but not until Kalman was the minimization of a func-

tional quadratic in the states and input of a linear dynamical system

considered in the context of an optimal control problem. Other authors,

notably S. S. L. Chang fCl] and Newton, Gould, and Kaiser [NI],


Brackets contain reference information. A letter followed
by a numeral indicates, respectively, the first letter of the first
author's surname and the order of appearance within the given alpha-
betic group of the specific reference.








had since the middle 1950's been occupied with the "analytical design" of

linear control systems through the use of a performance index quadratic

in the error (i.e., the difference between actual and desired system

responses to a specified input). Employment of Parseval's Theorem and

spectral factorization [N1] resulted in a solution strongly related to

the optimal linear filter of Wiener [Wl] of almost fifteen years earlier.

The solutions for the ISE (integral of the square of the error) problems

were very cumbersome to compute and seldom applied to systems of

greater than third order.

In the early 1960's the fledgling field of optimal control

theory underwent a metamorphosis. Difficult aerospace problems had

arisen which could only be conveniently approached as optimal control

problems. Rapidly there began appearing almost as many different

optimal control formulations as there were proponents. Superficially,

optimal control theory appeared to be a panacea; however, formidable

problems in computing and often even greater difficulties in the

implementation of some optimal designs limited their utility. In

general, the computation of optimal controls requires the use of

iterative algorithms which converge, at best, slowly. Further, optimal

controls are usually open-loop in nature.

The linear quadratic loss problem suffers from neither of

these difficulties: the solution is numerically straightforward and

always results in a linear feedback control law which is easily

implemented; in addition both continuous and sampled-data controllers

can be accommodated. If the system is to operate in a noisy environment,

an extension of the quadratic loss formulation, the Kalman-Bucy filter,

can easily be included in the design.









Unfortunately, performance indices as a whole seldom relate

well to system design requirements which are given as classical time

and frequency.domain specifications. In this regard, the quadratic

loss problem suffers as well.

The principal objective of this research is to extend those

original results of Kalman in order to create a design tool which

allows the designer to employ the powerful and sophisticated techniques

of optimal control theory when the correct choice of a performance

index is not apparent. Specifically, methods are developed for the

design of optimal linear regulators (both continuous and discrete)

which satisfy classical performance specifications in addition to the

minimality of a quadratic functional. This would permit the great

computing power of optimal control theory to be brought to bear on

problems which previously could be accommodated only by the cut-and-try

methods of classical design schemes. In passing, insights into the

basic nature of optimal regulators are developed in terms of classical

concepts, unifying some important notions in classical and modern

control theory.

The original impetus for this investigation was the requirement

to develop a method for designing a digital controller to replace an

existing continuous compensator without significantly affecting system

performance. It was believed that determination of a performance index

minimized by the continuous system would allow for performance invariant

design by computing the optimal control law for the sampled-data ver-

sions of the continuous system and performance index. In Chapter IV

this scheme is discussed as a solution to a problem which will









undoubtedly occur with increasing frequency as digital controllers

become more commonplace. It was in the course of this investigation

that the potential of the inverse problem for regulator design of wider

latitude was discovered.


1.2 Survey of Previous Work

Perhaps the first attempts at relating optimal control theory

and classical design were early (c. 1950) investigations of so-called

"standard forms" (e.g., [G2]). The object was to tabulate forms of

closed-loop system transfer functions which were optimal with respect

to a specified performance measure (for instance, ISE) and input.

This approach was rather restrictive and difficult to apply to problems

of interest. Its impact was nonetheless considerable and a recent

paper by Rugh [R11] indicates that the linear quadratic loss problem

has much in common with these early results.

Shortly after the linear quadratic loss problem became widely

known, the task of selecting performance indices which result in

optimal systems with desired characteristics was attacked primarily on

an experimental basis. The hope was that massive experience and some

insight into the mechanics of computing the optimal control laws would

lead to guidelines for the choice of a quadratic performance index.

One such procedure, developed by Tyler and Tuteur [Tl) consisted of

computing root-locus plots as functions of weighting terms in the

performance index and choosing a suitable compromise.

Another study of optimal systems in classical terms was made

in 1965 by R. J. Leake [Ll]. Leake, in developing a computational

scheme for solutions to the linear quadratic loss problem, shows how









an estimate of optimal system bandwidth may be made without computing

the optimal control and comments on an intriguing geometric interpreta-

tion (in the complex plane) of Kalman's criteria for optimality.

An important step in the analysis that follows will be the

determination of when two different quadratic performance indices

applied to the same plant will result in identical optimal control laws.

Kalman's paper on the inverse problem, by its nature, encounters the

problem theoretically but does not consider it in detail. A partial

practical solution to the equivalence problem appeared simultaneously

with Kalman's paper. Wonham and Johnson observed in their paper on

the linear quadratic problem with a bounded control [W2] that an arbi-

trary weighting matrix in the performance index could be replaced by

a diagonal one which will result in the same optimal control. Their

result arises from the observation that when the system is expressed

in a canonical form (companion matrix form [R2]) the performance index

may be reduced by repeated integration by parts to a diagonal form;

they fail to realize, however, that the diagonalized version may no

longer possess a solution. Kalman and Englar later recognize [K3,

p. 304-306] that, in the same canonical form as used by Wonham and

Johnson, certain terms may be discarded and equivalence maintained.

Beyond these early results and an occasional rediscovery of

them (e.g., Kreindler and Hedrick [K4]) the study of equivalent quad-

ratic performance indices has remained dormant.








In summary:

1. A great deal of experimental work has been done in hopes of

relating optimal design to classical criteria and providing

intuition into a procedure for choosing performance indices.

Very little basic theoretical research has been invested

in this area.

2. The practical problems of determining the optimality of an

actual system and the computation of a performance index

have not been considered.

3. Some elementary equivalence relations have been developed,

but the difficulties arising from naively applying them

have been generally overlooked. Nor have the advantages of

one equivalent form over another been explored.

4. No real effort has been made to use the study of the inverse

problem as a foundation for linear regulator design.


This work begins with a review of the linear plant, quadratic cost

variational problem. The conditions for the solution to exist are dis-

cussed along with techniques for computing the optimal control.

The inverse problem for a linear plant and quadratic performance

index is next considered. The conditions for optimality are developed,

the meaning of optimal control is studied in terms of classical criteria

and the equivalence of performance indices is resolved.

The fourth chapter develops computational procedures for the

determination of solutions to the inverse problem and goes on to con-

sider practical techniques for the design of optimal regulators which

meet classical performance specifications.














CHAPTER II


THE LINEAR, QUADRATIC COST OPTIMIZATION PROBLEM




2.1 Introduction

As a first step toward the establishment of viable design

techniques based on the linear, quadratic cost optimal control problem,

the optimization problem itself must be reviewed in some detail. The

great power, latitude and computational elegance of this optimal control

formulation account in large part for its popularity as an object of

study. In the investigation that follows its limitations will also

have considerable impact.

This chapter will first define what is meant by the linear

plant, quadratic cost optimization problem. A method for the compu-

tation of solutions which also provides a necessary and sufficient

test for existence will then be considered. The chapter closes with

a specific review of the particular subproblem which will be of

principal interest in the remainder of this work, the time-invariant

optimal linear regulator.








2.2 The Linear System, Quadratic Cost Droblem

Consider an n-dimensional linear system with state feedback

described by the state equation

dx
Fx+ Gu (1)


where u = -K (t)x

and a companion performance index

t
J (xf Qx + uTRu)dt, Q = Q1 and R = RT (2)
t


where u is m-dimensional; F, G, Q, and R are real, constant matrices;

and K is a real (possibly time-varying) matrix; all are of appropriate

dimensions.

It will be shown later (Appendix A) that several other

performance indices can be accommodated in the framework established

1
for this particular one. The 2 preceding the integral in (2) is for

algebraic simplicity in the present discussion and has no effect on

the result of the optimization process; it will occasionally be

deleted in the sequel without comment.

Taken together there are a variety of problems inferred by

(1) and (2). The most obvious is the optimal control problem [K2].

i.e., compute the control law K(t), if any, which minimizes the

performance index (2). A second, somewhat more obscure problem, is

the so-called "inverse problem" of optimal control theory [K1,E1].

The "inverse problem" seeks to determine what, if any, performance

index is minimized by a specific feedback control law. In the fol-

lowing, both of these topics will play an important role in uncover-

ing the nature of optimal systems.









2.3 The Optimal Control Problem


The Euler-Lagrange Equations


The necessary conditions for a control u to minimize the

performance index (2) is that the system equations (1) and the

celebrated Euler-Lagrange equations [Gl] be simultaneously satisfied,

i.e.,

m = (t) = O (3)

6H
u 0 (4)


where H is the Hamiltonian

I T I T T
H =- x +x u Ru + X (Fx + Gu).
2 2


If R is non-singular, equation (4) (sometimes called the "stationarity

condition" [B3]) determines a candidate for the optimal control, i.e.,


u(t) = -R -G X(t) (5)

The simultaneous solution of equations (1) and (3) with the substi-

tution of (5) results in the two-point boundary value problem:


d Fx -GR-1G x(t ) = x (given),
L]L \G T ] (6)
-Q -F T X(t) = .
Q rx]









The Riccati Equation


The optimal control problem for a linear system subject to

a quadratic performance index has been studied in great detail over

the past several years (e.g., [K2], [K33) and several eloquent solu-

tions for the necessary conditions have been offered [Bl1. Most of

these solutions differ only in the approach taken to solve the two-

point boundary value problem (6); for the purpose of this chapter the

so-called "sweep method" [Gl] is most illustrative and will be pre-

sented by way of review.

Hypothesize a matrix P(t,t ) such that


X(t) = P(t,tf)x(t), (7)


then P(t,t ) would in effect provide a boundary value which is

"swept back" in time to the initial time and the initial value for

\ would simply be


X(to)= P(t ,tf )x


hence, the separation of the boundary values would be resolved.

By requiring that the Euler-Lagrange equations be satisfied, it may

be shown that

/dP T -1 T
-+ PF + F P -PGR G P + Qx = 0.
Wdt

Since x(t) is arbitrary, P(t,t ) must satisfy

dP T -1 T
dt -PF F P + PGR G P Q, P(t ,t ) = (8)

which is a matrix version of the familiar Riccati equation [Dl].


1This problem is often referred to as the "optimal linear
regulator" problem in the literature; however, this designation will
be reserved for the infinite final time case here.









At first it may seem rather surprising that the solution to

a 2n order two-point boundary value problem (6) can be obtained from

the solution of an nth order non-linear equation (8), but recalling that

the scalar Riccati equation was solved in elementary differential

equation theory with the aid of a 2nd order linear differential equation

relates it to a familiar problem.

Finally, substitution of (7) into (5) leads to a candidate for

the optimal control (a control which satisfies the necessary condi-

tions) in the form of a feedback control law, i.e.,


u = -K (t)x, where K(t) = P(t,t )GR~1 (9)


and P is the solution to the Riccati equation (8).

Before proceeding any further, another necessary condition is

immediately available which has been previously ignored in this analysis.

From observation of the performance index (2), it is obvious that R

must be positive semidefinite; otherwise, a control could be hypoth-

esized with sufficient high frequency content to have negligible effect

on the states of the system, hence allowing the integral (2) to become

unbounded (negatively). The requirement in the Euler-Lagrange equa-

tions that R be non-singular further constrains R to be positive definite.

This assumption is equivalent to the "strengthened Legendre-Clebsche"

necessary condition of the calculus of variations; therefore, the

requirement that R be positive definite is sometimes referred to by

that nomenclature.

In summary, a solution to the matrix Riccati equation (8)

specifies, in the form of equations (9), a control which satisfies

the necessary conditions (Euler-Lagrange equations) for optimality;








all that remains is to investigate sufficient conditions for the

optimality of (9). The existence of a solution to the Riccati equation

and an additional sufficient condition are related to the non-existence

of conjugate points which are defined below.




The Conjugate Point Condition [Gl]


Definition 2.1

A point t1 is called a conjugate point to
tf if there exists a solution to the Euler-Lagrange
equations (6) with the boundary conditions

x(t ) = x(t ) = 0, where t S t1 < tf,

which is not zero everywhere on the interval.

Graphically, a conjugate point can be illustrated as in Figure 2.1

(cf. Figure 1 in Reference [B3]).


x(t)
2




tt




Figure 2.1 Conjugate Trajectory



Lemma 2.1 [B3]

When a conjugate point exists, both the
trivial solution (Path 1 in Figure 2.1) and the
conjugate trajectory (Path 2 in Figure 2.1)
result in a zero value for the integral


f (xTQx + uTRu)dt.
t
1









Proof

Consider the identically zero integral on the conjugate path


X (x Fx + GR -G )dt = 0

t t

or SJ xTidt 1 X(Fx GR G X)dt = 0.
1 1


Applying integration by parts to the first term results in


Tx + (Fx GR-1GT)dt = 0.



The first term is clearly zero and with the substitution for X from

the Euler-Lagrange equations (6) and for u from (5)
t
S(xTQx + uT Ru)dt = 0,


which completes the proof.

When a conjugate point does exist at tl, M(t1) must be non-zero

to be distinct from the identically zero solution to (6). If the system

is controllable, it must respond to the input (5) resulting from

X(t ) f 01 hence by continuity the conjugate path (A and x) must be non-

zero for some finite interval within (t,t f). Then the conjugate path

cannot be a duplicate of the identically zero path with the addition

of a (non-zero) discontinuity; thus the presence of a conjugate point

represents more than merely misbehavior at a single point. The occur-

rence of a conjugate point coincident with t, is-'obviouslycdisconcerting
0

because it indicates that there are at least two candidates (that'

satisfy the necessary conditions) for an optimal control which lead









to entirely different trajectories at the same (zero) cost. When a

conjugate point occurs at t 7 to, the results are equally catastrophic

but this case does not lend itself as well to heuristic interpretation.


Theorem 2.1

For a linear, completely controllable system subject
to a quadratic performance index (2) with R positive
definite, a control u* satisfying the Euler-Lagrange
equations is globally optimal if and only if there exist
no conjugate points on the interval (to,t ).

A discussion of this theorem and its proof in a somewhat dif-

ferent context can be found in Breakwell and Ho [B3]. The requirement

that conjugate points be non-existent is generally referred to as the

"conjugate point condition" or occasionally in the classical calculus

of variations as the Jacobi condition [Gl].

Complete controllability requires that there exists an input,

u(t), which will drive the system


x = Fx + Gu(t) x(t ) = x
o o


from any initial state, x o, to the origin within an arbitrary time

interval; this can be shown to be equivalent to requiring that the

matrix [G,FG,...,F G] have full rank [K5]. Controllability is not

actually a severe criterion in a practical sense. A plant which is

not completely controllable can be transformed into two canonical

subsystems, one containing the completely controllable part and the

other subsystem containing the remainder of the plant dynamics [K5].

Hence, a given design problem can be thought of as two related designs

on the canonical subsystems and it is only necessary to be certain that

the non-controllable part is stable and/or not reflected in the

performance index.









Now the solution to the linear, quadratic cost optimization

problem is essentially complete; it is merely necessary to solve the

Riccati equation for a candidate control law and to insure that the

conjugate point condition is satisfied. Although there appears to be

no convenient way to test for the presence of conjugate points, the

following theorem demonstrates that a test for the conjugate point

condition is actually implicit in the solution of the Riccati equation.


Theorem 2.2

The solution to the Riccati equation (8) for the
optimization problem specified by (1) (completely con-
trollable) and (2) (with R positive definite) fails to
exist (becomes unbounded) at t t ot if tl is a conjugate point to t .f


A formal proof can be found in Lee [L2] which is very much in

the spirit of the following heuristic justification.

From the definition of a conjugate point, x(tl) = 0; however,

X(t ) 1 0, since this could only come about in the trivial solution.

Recall that the solution to the Riccati equation was defined by (7) as


X(t) = P(t,t )x(t).

Then as t approaches tl, x approaches zero but X remains non-zero;

hence, j P(t)|| must correspondingly increase and finally become

unbounded at t = tl

Thus far the conjugate point condition has been presented in

a rather esoteric format with little physical interpretation. Again

consideration of the general case is difficult but the occurrence of

a conjugate point at the initial time (t ) has an interesting

interpretation. The absence of a conjugate point at t (in the linear
O









case) insures that no two system trajectories which satisfy the Euler-

Lagrange equations (extremals) will ever intersect,2 that is, there will

never exist two distinct optimal trajectories ianating from a single

system state [B4].

The solution to the Riccati equation (8) also has a physical

interpretation which will be useful later.


Lemma 2.2

The value of the performance index for the
optimal control law is

o lT
J (t ,xo tf) xoP(totf)xo (10)

where P is the solution to the matrix Riccati
equation (8) which is bounded on (t ,tf), i.e.)
there exist: no conjugate points.


Proof


J = (xTQx + u Ru)dt



Substitution of the optimal control law,

-1 T
u = -R G Px,


results in

tf

J = xT(Q + PG -1GTP)x dt. (11)


2
Intersection here is taken so as to exclude the case of
tangential coincidence. In the general (non-linear) case extremals
taken sufficiently close (i.e., neighboring trajectories) must
not cross.










Define
-1 T
F = F GR GTP


the closed-loop state matrix and k (tot) as its corresponding state

transition matrix, i.e.,


4k = F k and x(t) = kx o


Equation (11) can now be rewritten as


J = ft xTok(Q + PGR -1GP)4kx dt (12)


Substitution of the definition of Fk into the Riccati equation

results in


P= F P PFk PGR-G P Q


and when employed in (12) to replace Q, (12) becomes


= T (--PFk P dt
0

or
tf t
J = J o xkP4 xodt (xTPT p x + x+ TkT, x )dt.
2 o k ok o ok ko
o o


Application of integration by parts to the first integral above results

in terms which cancel the second integral, leaving


1 TT 1 T 1 T
J = TP kxo = oP(t o' t) X (t f)P(tft )x(t).

0


Since P(t,t ) = O0, this is the desired result.

This proof is considerably different from the standard one

(e.g., [Al, p. 251) in that it does not require the use of








Hamilton-Jacobi theory. This lemma not only provides a necessary link

in the solution of the problem considered in the next section but

provides a tie with dynamic programming, in that formulation equation (10)

is referred to as an "optimal return function" [B5].




Sufficient Conditions Existence


The conjugate point condition is not entirely satisfactory from

the standpoint of a working sufficiency condition. It is not known at

the onset whether or not the optimization problem has a solution; only

after the complete control law is computed is one assured that it was

not all for naught. What is required then, it would seem, is a suffi-

cient condition somewhat more restrictive than the conjugate point

condition with the advantage that success of the optimization problem

is predetermined.


Theorem 2.3

The optimization problem of minimizing a quadratic
performance index


J = (x Qx + u Ru)dt, R = R positive definite (13)
o


subject to a completely controllable linear plant

dx
S= Fx + Gu (14)
dt

always has the global minimum

u(t) =- K T (t)x(t) where K(t) = P(t,t )GR-1


and = -F P PF + PGR-1GP Q, P(t ,t ) =0 (15)


for all finite (totf) if Q is positive semidefinite and
symmetric.









Proof

Suppose that Q is positive semidefinite and the Riccati equa-

tion (15) diverges at t = t1 < tf. A solution for (15) must exist in

a sufficiently small neighborhood of tf further P(t) exists for all

t e-(t +C,t ) e > 0; hence, by Lemma 2.2


J (x(t +e),t +e) = x (t +)P(t +t)x(t +) 2 0, (16)

for e < t t1


the inequality is due to the positive definiteness of the integrand

of J. As e approaches zero, some entry of P becomes unbounded. It can

be assumed without loss of generality that at least one diagonal element

of P becomes infinite; otherwise, some 2 x 2 principal ninor of P would

be negative, contradicting the positive semidefiniteness of P inferred

in (16). Let e. be a vector which is all zeros except for the ith ele-

ment, a one, which corresponds to a diagonal term p.. of P which becomes

unbounded as C approaches zero; then,


J(et +e) = Pii (t1+, t'


Since J is the optimal performance index, a performance index

resulting from any arbitrary control, say u = 0, must be greater.

Then if tl + E and ei are chosen as the initial time and state of the

optimization problem, i.e., to = t + e

t
J0(e. .tl+E)=pi (t r.t95 fe eT(r-t)Q5r-tO)e. dT ( 7)



where (t-t ) is the system state transition matrix and 0(t-t )e.
o o i
is the resulting free trajectory. Clearly, the integral in (17)








remains bounded over the finite interval (t +'t ), while the left-

hand side becomes unbounded as c 0. This contradicts the original

supposition of a conjugate point at tl and proves the theorem.

An interesting proof of this theorem using Lyapunov's second

method to determine the stability of the Riccati equation is given in

[K2].

Throughout the literature there appear sporadically references

to theorems such as 2.3 as necessary and sufficient conditions for a

solution of the optimization problem to exist (e.g., [D3,p. 557]);

this is patently false and numerous counterexamples exist [G3].

No necessary and sufficient condition for the non-existence of conjugate

points (other than integrating the Riccati equation) is presently known,

although many researchers are actively pursuing these conditions and

there is reason to believe they will be found in the near future [B6].




2.4 The Optimal Linear Regulator Infinite Final Time


Existence and Stability


The Euler-Lagrange equations and the conjugate point condition

apply as well to the case when the final time is no longer finite.

However, as before, the conjugate point condition is not satisfactory

as an existence test; the following theorem extends Theorem 2.3 to

the infinite time case.











Theorem 2.4

If a linear plant (14) is completely controllable
and a companion quadratic performance index,


J = S (xQx + u Ru)dt, R = R positive definite (18)
T
has Q = Q positive semidefinite and if P(t,t ) is a
solution to the matrix Riccati equation (15) with
P(t ,t ) = 0, then
lim P(t,t ) = P(t) (19)
tf-

exists for all t and is a solution of (15).


Proof [K2]

First, it must be shown that the limit (19) exists for all t.

Since the plant is completely controllable for every xo, there exists

a control u'(t) which transfers x to 0 by some t t l. Set u(t) = 0

for t > tl. Then


J(txo't t)=x TP(t 't )x J(t ,x t J(t ,x
1 o o 1 0 o t1) t 0 0 )
u u


is bounded for all t > t The optimal costs are also non-decreasing
1 o

as t Suppose that this were not the case, that is, suppose


Jo(t ,x 't ) > JC(t ,x ot ), for t2 > t
1 0 o 1 2 o o' 2 2 11


where u (t) is the optimal control corresponding to J1 and u (t) is

the optimal control resulting in J. Then by the positive definiteness

of the integrand of (18), use of control u2 will result in a lower cost

at time t than ul; thus contradicting the optimality of ul and

demonstrating the assertion that the optimal costs are monotonic.

The limit therefore exists for all t by the well-known result that all

bounded monotonic sequences possess a limit.








Now it is necessary to demonstrate that (19) is a solution to

the Riccati equation. Define P(t,t f;A) as a solution of (15) with

the boundary value P(t ,t ) = A. Then, using the continuity of solu-

tions of (15) with respect to its boundary values,


P(t) = lim P(t,t 2;0) = lim P(t,t IP(t t 2;0))

2 2

= P(t,t ;lim) = 't;O)) = P(t,t ;P(t1))
t 2

and P(t) is a solution of (15) for all t.

The price that is paid for guaranteeing the existence of a

solution over all time (in addition to requiring Q to be positive

semidefinite) is the complete controllability of the plant.


Corollary 2.1

The solution P(t) to the matrix Riccati equation (15)
for the optimal linear regulator problem of Theorem 2.4 is
constant and the unique positive definite solution of

PF + F PGR10P + Q = 0, (20)

the steady-state (algebraic) Riccati equation.


The stationarity of P follows from the arbitrariness of the

choice of the initial time to, since all initial times must result in

the same value of the optimal performance index. Equation (20) ensues

when the effect on P of the irrelevance of t is considered; the choice

of the positive definite solution is dictated by the positive definite-

ness of (18) and uniqueness is guaranteed by the conjugate point con-

dition.









It is now clear that the optimal linear regulator is the optimal

control theory analog of the classical design using state feedback.

As pointed out by Kalman [K2], the literature contains many

references where the stability of an optimal system is tacitly assumed.

For example, Letov [Zl, p. 378] equates (without proof) optimal systems

and those which are stable in the sense of Lyapunov; although probably

true for the class of optimal systems which are also stable, it is not

in general. This can be easily demonstrated with a simple example.

Let a scalar input plant,

x = Fx + gu,

have one or more eigenvalues with positive real parts and let the per-

formance index to be minimized be,

t

J = S u2dt,
0


that is, Q = 0 and R = 1. The problem so defined clearly has a solu-

tion (Theorem 2.3), which is

u(t) = 0.

Stability was not a result of optimization in this case primarily

because the states are not reflected in the cost (performance index);

had the plant been stable, however, the optimal system would have been

also. The final theorem in this section formalizes this observation

into a general result.








Theorem 2.5 [K11]

An optimal linear regulator problem satisfying the
conditions of Theorem 2.3 results in an asymptotically
stable control law if

i) the pair [F,H], where Q = ITI is completely
observable, i.e., [H,F H,... (F )n-1H] has
full rank;
and only if

ii) the linear subspace
X, = (x 0o 1) He x 11J= 0
Ft
of the state space is null (i) or e x, x E X
is an asymptotically stable response in the
sense of Lyapunov.


Proof

i) By the assumption of observability and the positive semi-

definiteness of Q, the integrand of the performance index must be posi-

tive along any non-zero trajectory of the plant. Then


Jo(t,x(t)) = xT (t)x(t) > 0, x(t) 9 0 (21)


along optimal trajectories. Differentiation of the performance index,


J(t,x(t)) = (xT Qx + uTRu)dT,



results in J(t,x(t)) = (x Qx + u Ru)< 0, x(t) 1 0, which is

negative along all non-trivial trajectories. Then (21) is a Lyapunov

function [L2] and the system is asymptotically stable by Lyapunov's

Second Stability Theorem [L3, p. 37].

ii) When certain of the states are not observable in the cost,

Lyapunov's method is effectively applied only to a subsystem (i.e., the









states observed in the cost) and the more general result follows when

the stability of the remainder of the system is considered.

The requirements for stability placed on Q by Theorem 2.5 are

not unduly restrictive in that they reflect good engineering judgment;

that is, if a state has significant effect on system performance it

should influence the design process. Some authors choose to include

stability tacitly in the definition of the performance index by con-

sidering only the case where Q is positive definite (e.g., [SI]).

Stability is assured since a positive definite Q must have a non-

singular factor and condition (i) of Theorem 2.5 is clearly satisfied.


2.5 Summary

Over the past ten years the optimal linear regulator problem

has become one of the most widely studied optimal control problems.

Perhaps the best way to summarize this unique problem is to review the

qualities which set it apart.

1. The solution to the problem is always approached in the

same fashion regardless of the specific system or weight-

ing matrices employed and in contrast to the majority of

optimal control problems, the solution is explicit.

2. In a restricted, but very large, set of weighting matrices

(i.e., Q positive semidefinite) the solution is guaranteed

to exist.

3. The solution is always in the form of a linear, constant

feedback control law, as opposed to the general optimal

control problem where a feedback formulation is not directly




26



obtainable. With another minor concession to generality

(Theorem 2.5) asymptotic stability will be assured from

the onset.

4. Numerical computation of the control laws is both straight-

forward and relatively inexpensive (in terms of computing

time)[B7]; in addition, a multitude of numerical schemes

are available [Bl].

Under suitable assumptions of continuity and corresponding

definitions of controllability and observability [D2] most of these

remarks are also applicable to linear time-varying systems and non-

constant weighting matrices [K2].














CHAPTER III


THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM




3.1 Introduction

Optimality in itself is not necessarily a desirable character-

istic for a system to possess. For instance, suppose it is desired

to minimize the sensitivity of an existing system's response to com-

ponent variations by use of an appropriate controller. A system

design which obviously has a minimal sensitivity is one that does

nothing whatever; then one candidate for an optimal controller would

be one that turns the system off. In this case, the optimal design would

probably not be satisfactory because it was not implicit in the procedure that

the resulting system should perform in some acceptable manner in addi-

tion to minimizing the performance index.

The lesson is clear, optimality may be a frivolous notion

unless its ramifications are thoroughly understood in the context of

total system behavior. In the present chapter the implications of

optimality are considered for a single input linear plant subject to

a quadratic performance index to set the stage for its use as a viable

design tool.

The system to be considered in this chapter can be described

by the state equation,

dx
=t Fx + gu, (1)









and occasionally it will require an associated feedback control law,


u = k x, (2)


where F is a constant n x n matrix and g,k are constant n-vectors.

In conjunction with this plant, a quadratic performance index,


J = (xQx + ru )dt, (3)


will be studied. The control weighting factor, r, will be taken with-

out loss of generality as unity throughout the remainder of this work.


3.2 When Is a Linear Control System Optimal?

The original impetus to investigate the "inverse optimal control

problem" is generally considered to be a 1964 paper by R. E. Kalman,

bearing the same title as this section [K1], although it has a much

older history in the Calculus of Variations. In that paper, Kalman

considers the theoretical criterion for a linear system to be optimal

with respect to a restricted class of quadratic performance indices.

His principal result is reviewed next.


Theorem 3.1 [Kl]

Consider a completely controllable linear plant
(1) with a stable completely observable control law k
(2). Then k is an optimal control law with respect
to a quadratic performance index (3), with Q positive
semidefinite, if and only if,

11 + k (jw)gj2 2 1 (4)

for all real wu, where (s) = (sl F)-1









Proof

i) Necessity: If k is an optimal control law, then by

Theorems 2.3 and 2.4

k = Pg (5)

where P is a solution to the steady-state Riccati equation,


PF + F P Pgg P + Q = 0. (6)

Substituting (5) into (6) permits it to be rewritten as


PF + FP kk + Q = 0. (7)


Adding and subtracting sP to (7) results in


-P(sI F) (-sI F )P kk + Q = 0.


Premultiplying and postmultiplying the above by g T (-s) and (s)g,

respectively (where $(s) is as defined in equation (4)) and substitut-

ing (5) where appropriate, leads to


gT T(-s)k + k T(s)g + gT T T(-s)kkT(s)g = g TT-s)Q(s)g

which may be factored as


[1 + g (-s)kll + k T(s)g] = 1 + gTT (-s)Qs(s)g. (8)


Noting the semidefiniteness of Q proves the necessary condition.

ii) Sufficiency: Proof of the sufficient part of the theorem

is not particularly enlightening and requires concepts that are yet

to appear, hence it will be deferred to the appendix.


3.3 Implications of Optimality

It is somewhat surprising that the condition for optimality (4)

should be a frequency domain criteria in that all of the previous









analysis of the problem (Chapter II) was carried out in the time domain.

The study of optimal systems in the frequency domain may be responsible

for the current interest shown by Kalman [KG1 and others in realization

tl.ciry from the point of view of invariants and thus promises to have

far greater i influence in years to come.

Examination of the optimality criterion (4) reveals that the

quantity whose modulus is taken on the left-hand side of the equation,

i.e.,

f (ju) = 1 + k T(ju)g


is the "return difference" of classical feedback theory [B81]. It can

be interpreted as the difference between a unity input (in the frequency

domain) to the system and what would consequently be returned as feed-

back. Graphically, this is illustrated in Figure 3.1 as the difference

between the input at node a and the feedback at node b.



Plant

a a x
0 ; (j )g
1I

b T
_T
0 -k]

Feedback
Gains
fk = a b


Figure 3.1 Return Difference




The requirement of (4) that the modulus of the return differ-

ence be greater than unity is the celebrated result of classical

feedback theory that the sensitivity of system response to variations








in plant parameters is reduced by the addition of feedback [B8].

Further analysis of inequality (4) reveals that it requires that the

Nyquist locus avoid a circle of unit radius about the point (-1,0).

Figure 3.2 pictures a Nyquist plot of a hypothetical optimal system

constructed from a plant with two eigenvalues with positive real parts.











-I
A









Figure 3.2 Nyquist Plot of Optimal System


Two additional observations may be immediately made from

Figure 3.2. First, the phase margin of an optimal system must be at

least 600, since the closest points (in phase) to the negative real

axis that the locus can cross the unity magnitude contour (points A

and A' on Figure 3.2) are displaced 600 from the axis. Second, the

system will clearly remain stable regardless of how the feedback gain

is increased. Then if gain margin is defined as that factor by which

feedback gain may be increased before system instability occurs [E2] ,

the gain margin of an optimal system may be said to be infinite.

However, if gain margin is defined as the reciprocal of the magnitude

of the loop gain at phase crossover, i.e.,









1 T
GM 1 i-- where Arg fk Q(jw )g] = 180,
k $(ju )g c


then an optimal system has a gain margin of at least or approximately

-6 db. Kuo indicates that the two definitions given are equivalent

[K7, p. 398], but consideration of the example of Figure 3.2 reveals

that this is not the case when the plant is non-minimum phase.

A third definition, and the one that should be understood in

any references to gain margin in the sequel, is the following [T2]:

Let g. be the factor by which the feedback gain may be
increased until instability occurs and g be the factor
by which the gain may be decreased and stability main-
tained, then

GI = smaller of fgi,l/g d.


This definition is more reasonable in that it reflects the actual

gain disturbance required for instability and is easily determined

from the Nyquist diagram. The gain margin for an optimal regulator,

in the sense of Theorem 3.1, then is at least 6 db.

It is instructive at this point to compare these figures of

gain and phase margin with those required in practice. Jones, Moore

and Tecosky in Truxal's classic handbook [T2, PP. 19-14] recommend

phase margins of 40' to 600 and gain margins of at least 10 db for

applications typical of chemical process control. Generalities of

this sort are of course subject to severe criticism but nonetheless

it can be concluded that minimum optimal regulator specifications

compare favorably with those desired of classical designs.









3.4 Companion Matrix Canonical Form

It is well known that for any non-derogatory matrix, F, there exists

a similarity transformation, T, such that

-1
F = TFT

is the companion matrix [Ml] of F. A companion matrix being a matrix

of the form:

0 1 0 ... O

0 0 1 ... O
F = (9)

-al1 a2 -a3'.-*an

where


n n k-1
cp(s) = s + E aks = det (sl F>.
k=l


That is, the last row of the companion matrix of F contains the nega-

tive of the normalized coefficients of the characteristic polynomial

of F (and P), with the remainder of the matrix null save 'a superdiag-

onal of ones. Some authors choose to refer to the transpose of the

matrix defined above as the companion matrix [G4] In the following

either definition-will suffice; however, (9) will be assumed for con-

sistency.

When F assumes the role of the state matrix in a completely

controllable linear system (1) the similarity transformation, T, which

transforms F to its companion matrix, places the system as a whole in

a canonical form. Specifically, if z = Tx represents the change of

basis described above and the linear completely controllable system is









x l'x + goL

T
y = x,

then FK ]]

z = Fz + gu




F = TFT the companion matrix of F


0
0

S= Tg= and H = (T1) T1.

0
1


Complete controllability allows the specified form of g to be chosen.

Since there is no loss of generality, controllable systems will be

assumed in this coordinate system and the "hat" notation will be omitted.

The problem of actually computing the required transformation

has lately occupied a good deal of the literature, e.g., [J1,R2,R3].

Computation of the transformation is conceptually not very difficult

and somp straightforward numerical solutions have been formulated

resp. R3].

The companion matrix canonical form is often referred to as the

phase-variable canonical form [S2]; this cognomen alludes to the prop-

erty that each state is the derivative of the preceding state.

A second property that will find application is the fact that g has

only one non-zero element, hence, the control directly affects only

the last state (highest derivative).









3.5 Characterization of the Equivalence
Class of Q's

Employment of the coordinate system reviewed in the last section

permits additional insight to be gleaned from Theorem 3.1. First it

will be necessary to exploit a characteristic of the companion matrix

canonical form.


Lemma 3.1

For a completely controllable linear system in the
companion matrix canonical form (1)
1


S(s) = p(s)i(s)g = s2

sn-1

where c(s) = (sI F)-1
and tp(s) = det (sI F).

Proof

To prove the theorem it is sufficient to show that


(sI-F)cp(s)(s)g = (sI-F)S(s) = ((s)g.

With F in companion matrix form


s -1 0 ... 0 1 0
0 s -1 ... 0 s 0
(sI-F)S(s) = =

a a a ... a +s sn-1 sI s a. si-1
1 2 3 ni


which is clearly Cp(s)g and the theorem is proven.

This lemma allows the simplification of equation (8) used in

the proof of Theorem 3.1 to

kTS(ju) 2 ST(-jL)QSju)
(jw) = + |(10)
+pju ____ 2= Jp(jwfl2









Multiplying by |p(ju) 2 results in


Jcp(jw) + kSC(jwu)2 = Jp(jw)|2 + S (-ju)QS(ju).

With the observation that the expression inside the absolute value marks

is the closed-loop characteristic polynomial (which will be denoted

p (s)) permits further simplification to

c(Ik jW)2 Icp(jw)|2 = sT(-jw)QS(jw). (11)

This relation (11) defines a polynomial by which the open- and

closed-loop characteristic equations of an optimal system must be

related to the state weighting matrix of a corresponding performance

index. This polynomial will appear frequently; consequently, it will

be convenient to refer to it with the following notation:

Y(u) = pPk (j)2 IP(jw)12 = ST(-jw)QS(jw), (12)

or occasionally as


Y(Q;w) = S (-ju)QS(ju)

to emphasize the functional relationship of Q.

Recalling that characteristic polynomials are invariant under

similarity transformation and the assumption of positive semidefiniteness

for Q leads to a useful corollary of Theorem 3.1.


Corollary 3.1

A completely controllable scalar linear system
(1) with a completely observable stable control law k
is optimal with respect to a quadratic performance index
(2) with Q positive semidefinite and completely observ-
able if and only if

S(j 2 cp(juj)2 0 for all real w,








where cp(s) = det (sI -F) and p k(s) = det (sI -F+gk ),
the open- and closed-loop characteristic polynomials,
respectively.1

The requirement that the feedback control be completely observ-

able again appears without apparent justification. Suppose that k were

not completely observable, then the pair of polynomials which comprise

the expression


T K(jw)
k (j(j)g -


will have a common factor [K5]. This can be thought of, in the clas-

sical control sense, as having a zero on top of a pole which prevents

the response of the pole from being observed at the output. Let the

common factor be y(jw) and denote the polynomials with y factored with

a prime, then equation (10) becomes
2
1 + K (ju)y(ju) 1+ ST (-jw)QS(jw)
cP (jw)y(jw) ICp'(juw)2ly(u)j)2

or


Jcp'(ju) + K'(jw) J2y(jmu)2 = lcp(jw)2y(jW)12+ST(-ju)QS(ju).


The implication is that S (-jw)QS(ju) must contain the common factor

IY(j))12 as well. Since Q is positive semidefinite, it can be factored

as Q = HHT; then by the preceding statement the vector of polynomials

formed from



Through a minor abuse of the language, the terms "completely
observable Q" and observabilityy from the cost" should be taken to
mean that the pair [F H] are completely observable when H is any factor
of Q such that Q = HH Similarly, "completely observable k" means
that the pair [F,k] are completely observable.









IH T(jU)g (13)


may have the common factor y(jw) and the Q may not be completely observ-

able. The reason for the uncertainty in the observability of Q is the

ambiguity in the location of the zeros of the factor of (13) correspond-

ing to y(jCu)12. Depending on the choice of H, (13) can have zeros in

the right or left half-plane which may or may not eclipse a pole of the

system. If k is completely observable, however, there will be no common

factor regardless of how H is chosen and any Q which satisfies (13) will

be completely observable. This is best illustrated with an example.


Example 3.1

Consider the second order plant,




1-1 -2 1]


with the feedback law, k = 1


The feedback is clearly not observable and in fact cancels one of the

plant poles at -1 as shown in Figure 3.3.



Imt


x plant poles: -1, -1

0 closed-loop poles: -1, -2



-2 -1 Re


Figure 3.3 System Pole Locations









The control law is nonetheless optimal and three performance indices

which are minimized by it have Q's,2




1 3 3 h 1



2 3 -3 h2 L-l


and 3

Q3 0



The first is not observable; (13) has a zero at -1 which

coincides with a plant pole. The second is identical to Q1 except

that the zero of (13) is reflected about the axis (at + 1) and is

consequently observable. The last is non-singular and avoids the

difficulty entirely. The important point to note is that the control

is optimal for all of these Q's. Complete observability from the cost

is not required for optimality; it is only necessary that any plant

poles which are unobservable appear in the closed-loop system as well.

That is, optimal feedback cannot move system poles which are unobserv-

able in the cost.

The requirement that k be completely observable is, in a sense,

an "inverse" sufficient condition to the observability of Q; if the

condition is met, then any Q for which the system is optimal must be

completely observable. If, however, k is not completely observable but

Y is non-negative, there may still exist one or more positive semidefi-

nite Q's which are completely observable for which the system is optimal.


2
These statements will be justified later.








Another corollary to Theorem 3. 1 results from consideration of the

preceding remarks.


Corollary 3.2

A completely controllable scalar linear system with
a stable feedback control law is optimal with respect to
a quadratic performance index with Q positive semidefinite if

p (jum)2 _- p(ju) 2 2 0 for all real u,

and
a) C(s) and c(pk(s) are relatively prime which
further insures that any Q for which the
control law is optimal is completely observable,

or
b) the plant is stable.


Physically, the (a) condition can be interpreted as meaning that

the aforementioned observability difficulties do not arise if the feed-

back moves all the plant poles. This is the case because the common

factor in the discussion preceding the corollary does not exist. The

second (b) condition removes the ambiguity by restricting the plant

poles to the left-half complex plane (including the imaginary axis)

and no difficulties of the sort discussed will occur. Some necessary

and sufficient conditions similar to those of Corollary 3.2 will-be

considered coincidentally with later results.

In order to provide greater insight into the optimality condi-

tion in a strictly mathematical sense, it will be convenient to consider

the requirements for Q implied by equation (11). In the same fashion

that the two preceding corollaries dealt with the left-hand side of the

equation and its relation to the system, it is desirable to consider

what the right-hand side, i.e.,

Y(Q;w) = ST (-jw)QS(j) ,

portends for the corresponding performance index.









Before proceeding, two well-known lemmas on factorization will

be stated; the first concerns a factorization of real, even polynomials

and the second a factorization of positive semidefinite matrices.


Lemma 3.2

If and only if F(w) is a real, even polynomial and

r(w) 0 for all real w,

then there exists a unique "spectral" factor y(s) such
that
r(w) = y(jw)y(-ju))

and y(s) has only zeros with non-positive real parts.


Many proofs of this lemma are recorded in the literature; for

example, see Brockett [B2, p. 173 ff].


Lemma 3.3

A real, symmetric matrix Q may be factored as
Q = HH1 ,
where H is a real matrix of rank (H) = rank (Q) if
and only if Q is positive semidefinite.


This is a fundamental theorem which is encountered in the study

of quadratic forms; see [A2, p. 139].

The groundwork has now been laid to present a theorem which

concisely places the system-theoretic results of Theorem 3.1 into a

mathematical framework.


Theorem 3.2

For every real, even polynomial r(w) of order
2(n-1), there exists a real, symmetric, positive semi-
definite, nth order matrix Q such that

Y(Q;x) = r(u),
(Y as defined in (12)) if and only if,

r((u) 0 for all real wu.








Proof

i) Sufficiency: If F(u) is real, even and non-negative,

Lemma 3.2 assures that a real factor y(s) exists such that

y(jw)y(-jwf)= r(w). (14)

Let h be a real vector composed of the coefficients of y(s) ordered

with the constant term first and the coefficient of the (n-l)st term

last. Then, clearly,

y(ju) = h S(jm) and y(-ju) = ST (-j)h,

where S is as defined in equation (12) of Lemma 3.2. The product (14)

can be identified as

Y(hhT ;w) = S (-ju)hhTS(jW) = y(-jw)y(jW) = F(w)

and hh provides a positive semidefinite Q which satisfies the suffi-

ciency part of the theorem.

ii) Necessity: It must be shown that any positive semidefi-

nite Q results in a Y(Q;w) that,is a real, even, non-negative poly-

nomial. Since Q is positive semidefinite, the Hermitian form,

F(W) = Y(Q;i) = ST (-jw)QS(j)),

is non-negative and is clearly real and even. The theorem is proved.

Theorem 3.2 relates the optimality condition of Corollary 3.1 to

a realizability condition for positive semidefinite Q's. Corollary 3.1

states that if the polynomial

y(Wu) = 1k(jiW)I2 |y(j)|12

is non-negative and the feedback control law is stable, the closed-loop

system is optimal. Theorem 3.2 states that if an arbitrary real even

polynomial is non-negative, then there exists a positive semidefinite

matrix Q that generates the polynomial by equation (12). If the system








is in companion matrix canonical form, the polynomial studied for

optimality and the polynomial of Theorem 3.2 are one and the same as

revealed by equation (11). Together they constitute all that is

required to determine the optimality of a given closed-loop system

configuration with respect to a positive semidefinite Q and to

construct a corresponding performance index.

The proof of Theorem 3.2 provides a "recipe" for computing

at least one performance index which is minimized by a given canonical

optimal system; that is, by spectral factorization. This realization

also prevents the performance index from obscuring unstable system

poles and thus avoids the observability problem. At this point, it

would be instructive to return to Example 3.1 and verify system

optimality and the choices of Q.


Example 3.2

Consider the second order plant and feedback law of Example 3.1,


F = ,[ k = ] .
1- -2 1L

The open- and closed-loop characteristic polynomials are


(p(s) = s2 + 2s + 1,


p (s) = s2 + 3s + 2,

and

Y(w) = |Yk (jW) 2 |(j) 12 = 3ju2 + 3,

is clearly non-negative. Then since the plant is stable, the opti-

mality of the feedback law follows directly from Corollary 3.2b.

Theorem 3.2 insures that a positive semidefinite Q can be found which








will form a performance index minimized by the control law,and the

proof hints at how one such Q can be constructed. Following the proof

the spectral factor of Y is computed, i.e.,

I(s) = /1(s+1) and

Y = V(-jw)4(jw).

Now a vector h is constructed composed of the coefficients of 4(s)

ordered with the constant term first,



h = /3f and Q = hhT =[3



which is the choice for Q1 in Example 3.1.

In order to avoid an increasing awkwardness in notation, the

following definition is required.


Definition 3.1

A state weighting matrix Q2 is said to be equivalent
to another weighting matrix Q1 if for a given linear system (1)
the quadratic performance indices (3) formed from Q1 and Q2
are minimized by the same control law. This equivalence
will be denoted by a tilde, i.e.,


and the set of all matrices which are equivalent for a
given system and optimal control law will be referred to as
the "equivalence class of Q's" for that optimal system.


Note that this equivalence relation is correctly defined in the

strictest sense [A3], that is, it is:

i) reflective Q Q

ii) symmetric if Q, ~ Q2, then Q2 ~ Q

iii) transitive if Q1 ~ 2 and Q1 Q then Q1 Q3









The definition fails to draw a distinction between symmetric and

non-symmetric matrices. Consistent with the remainder of this work,

symmetric weighting matrices will be tacitly assumed. There is no loss

of generality in the assumption of Q symmetric; if A is a non-symmetric

matrix, then it is easy to see that it can be replaced in a quadratic

1 T
form by the symmetric matrix -(A + A ) without altering the value of

the form fill].

The next theorem, which may be considered a central result of

this section, prescribes how all quadratic performance indices minimized

by a given system are related.


Theorem 3.3

For a scalar system, x = Fx + gu, in companion matrix
canonical form with a stable optimal control law k, a state
weighting matrix Q2 is equivalent to a weighting matrix Q1

which forms a quadratic performance index minimized by k
if and only if

a) Y(Q2;w) = Y(Q1;w),

b) the linear subspace of the state space,

T


Ft
is null or e x for x e X is an asymptotically stable
response in the sense of yapunov, and

c) the optimization problem for Q2 and the given system
has no conjugate points on t e [0,-).

Proof

i) Sufficiency: By Theorem 2.1 and (c) above, the minimiza-

tion of the performance index with Q2 results in a unique optimal control

law which by Theorem 2.5 and (b) is also stable; denote this control law

as k2. Suppose that k 2 k1, where k1 is the optimal control law








corresponding to Q 1. Then, since the coefficients of the closed-loop

characteristic polynomials are the sum of the plant characteristic

polynomial coefficients and the entries of k1 (or k ),


Ck2 (s) k 1 (s),
2 1

and by the stability of k, and k2


k 2(ju) )2 p kkl(jO) 2


(i.e., both characteristic equations have zeros in the left half-plane).

This last inequality with the definition of Y (12) contradicts part (a)

of the hypothesis and sufficiency is demonstrated.

ii) Necessity: It must now be shown that if Q2 ~ 1, then

(a), (b), and (c) follow. Parts (b) and (c) are implicit in that they

are necessary and sufficient conditions for stability and existence,

respectively, of the optimization problem with Q 2. The first part

results from the equality of k, and k2 and the definition of Y (12).

This theorem could easily have been rewritten to include the

case where the resulting control law was not stable; however, this

complication would have no usefulness and would obscure insight

into the mechanism of equivalent Q's. Many of the results to follow

can be extended to include unstable optimal control laws but any

apparent increase in relevance is purely artificial. Hence the praxis

of considering control laws to be only stable will be continued

throughout the sequel.

As is often the case, in this theorem practicality is the price

of generality. In the discussion of Theorem 2.3 it was observed that

the conjugate point condition was not very satisfactory as a sufficiency









condition for the optimization. The same remarks apply here: no con-


jug points will exist for Q2 positive semidefinite; then the conju-

gatt point condition need only be examined where Q2 is not positive

semidefinite. The condition that Y be identical for equivalent Q's,

part (a), can also be simplified.

The next lemma provides the last link in reformulating the

results of Theorem 3.3 into a workable equivalence relation for

performance indices.



Lemma 3.4

For an arbitrary real, symmetric, nth order matrix Q,
the real even polynomial resulting from the quadratic form

Y(Q;W) = S (-jm)QS(ju) (15)
where



s
S(s) =
.n-1

can be determined from the relation

n i-i
Y(Q;w) = S [q.. + 2 S (-1) q i+] -1 (16)
i=l j=1

(q. = 0 for i < 1 or j > n).


Proof

Rewrite the vector S as the sum of two orthogonal vectors,

1 0

0 s
2
S(s) = a(s) + b(s) a(s) = s b(s) = 0
0 s
4
s 0








and clearly

S(-s) = a(s) b(s)

since a is an even function of s and b is odd. Substitution of this

relation for S into (15) leads to


Y(Q;s) = (a(s) b(s)) Q(a(s) + b(s))
aTha bT T T
= a Qa b Qb + (a Qb b Qa).


By the symmetry of Q the bilinear terms in parentheses total zero.

Since a(s) and b(s) have odd or even elements respectively zero,

Y(Q;s) contains no terms with q. with i + j odd. The non-zero

off-diagonal terms resulting from a Qa have i and j both even, while

the non-zero off-diagonal terms from b Qb have i and j both odd.

Hence, Y may be rewritten as

a i+l 2(i-1) i+j-2 i+j-2
Y(Q;s) = (-1)i+q..s2 + 2 q..s -2 q i+j-2
i=1 i,j odd i,j even


which reduces easily to (16).

This lemma is important in its own right in that it provides

an algorithm for computation of 'Y for a given Q without the necessity

of evaluating the quadratic form. Its primary value however is that

it allows the equivalence condition of Theorem 3.3 to be redefined from

the invariance of a polynomial (T) to specific algebraic constraints on

the entries of the matrices in question.









Corollary 3.3

A weighting matrix Q is equivalent to a matrix Q1 for
the system of Theorem 3.3 if and only if the entries
of the matrices are related so that the quantities

p. = ii 2q i + 2q . (17)

i = 1,2,3...,n

(q. = 0 for i < 1 or j > n)

are equal for q.. taken to be elements Q or Q and
parts (b) and (c of Theorem 3.3 are satisfied.


Proof

It must be shown that the relation (17) given in the corollary

is equivalent to part (a) of Theorem 3.3, that is, if it is satisfied,

the Y polynomials for Q1 and Q2 are identical. This is accomplished

by demonstrating that the coefficients of the respective Y polynomials

are coincident. The proof follows directly from determination of the

coefficients of Y from (16) which are then related to the p. of (17).

With additional study of equations (16) and (17) a somewhat

startling phenomenon comes to light. The only entries of a given

weighting matrix Q which influence the polynomial Y(Q;w) are of the form


qkk and qk-P,k+V.


which excludes any element q.i where i+j is odd. This indicates

that approximately half of the elements of an arbitrary weighting matrix

are irrelevant (assuming that parts b and c of Theorem 3.3 are still

satisfied) with respect to the optimal control law. Kalman and Englar

(K3] noted from the structure of the companion matrix canonical form

that the q.ij elements where i+j is odd are "irrelevant" without identi-

fying the underlying structural relation (17) for equivalent Q's.








Their suggestion that the "irrelevant" terms in a given weighting matrix

be nulled at the onset of the optimization procedure, in order to simplify

computation, is potentially hazardous. There is the possibility that

nulling these elements will alter the observability qualities of the

original matrix to the extent that an unstable plant pole is obscured.

Fortunately, this occurrence appears to be extremely unlikely; the

author was unable to construct a matrix which behaved in this fashion

after a very exhaustive search. It seems that an unobservable matrix

will remain so after the i+j-odd terms have been struck and conversely

an observable matrix will still be observable after these terms have

been removed. A very convincing heuristic argument can be made in

support of this observation; first, however, it will be convenient to

state two lemmas.


Lemma 3.5 (Gerschgorin, 1931 [G5])

All the eigenvalues of a square matrix A = (a. .) lie in
the union of circular regions,

n
laii zj E l a I i = 1,2,3,...,n,
j=1
j/i

of the complex plane.


This is, of course, the touted Gerschgorin Circle Theorem

which has found wide application in the numerical eigenvalue problem.

A laconic proof of this famous theorem can be found in Cullen [C2,

p. 197]. Of principal immediate interest, however, will be a second

lemma which, although a corollary of Gerschgorin's Theorem, has found

prominence as a separate result.








Lemma 3.6

Diagonally dominant matrices are non-singular. A square
matrix A is said to be diagonally dominant if
n
i a. > S a..j for i = 1,2,3,...,n (18)
j=1l
jgi
where a.. is the (i,j)th element of A; that is, if
the diagonal entries are larger in modulus than the sum
of the magnitudes of the remaining constituents of their
respective rows.


The justification for this lemma follows easily from Lemma 3.5.

If a matrix is diagonally dominant, the region of permitted eigenvalue

locations excludes the origin and the matrix is consequently non-singular.

Consider Figure 3.4 which is a schematic drawing of a symmetric

matrix with the i + j odd terms removed and the X's representing the

remaining entries.




Q = 0 X 0 X 0 ...

X 0 X 0 X ...




Figure 3.4 Sparse Equivalent Matrix



It is clear from observation of the figure that removing the indicated

entries tends to increase the dominance of the diagonal and in a sense

makes it "more non-singular"; thus it would be reasonable to expect an

aggrandizement of observability rather than a deterioration.

A possible second concern is that discarding elements in the

manner described may destroy the positive semidefiniteness of the








weighting matrix. A second application of Gerschgorin's Theorem,

this time in the form of Lemma 3.5, reveals that the likelihood of

degrading the positive semidefiniteness of a Q matrix by removing

the irrelevant terms is extremely small.

If the matrix is positive semidefinite, the union of circles

which form the permissible regions for eigenvalues must include parts

of the right-half complex plane (including, perhaps, the origin); and

by a well-known result of matrix theory [H11], the diagonal elements of

a positive semidefinite matrix, hence the centers of these circles,

must be non-negative. Then as the off-diagonal terms are removed, as in

Figure 3.4, it is apparent from relation (18) that the radii of the

circles which may contain eigenvalues are reduced and the eigenvalues

will consequently be restricted to fall in a region that is, if anything,

more positive. The only case which is not resolved by this argument is

the one where the permissible region of the original matrix includes

part of the left half-plane and reduction of the radii does not retrieve

the region wholly into the right half-plane; thus admitting the possi-

bility of a negative eigenvalue in the reduced matrix. In any case,

the migration of any of the eigenvalues of a Q matrix to the left, as

a result of the simplifying operation, appears extremely unlikely.

In general the observability and absence of conjugate points

for the sparser matrix must be tested. However, for a specific but

very important case both conditions (and hence equivalence) can be

guaranteed a priori.







Theorem 3.4

If a weighting matrix Q for a quadratic performance
index (3) operating on a scalar system in companion matrix
canonical form is unity rank, i.e.,
T
hh = Q11
where h is a vector, and the resulting optimal control law
is stable, then the matrix Q resulting from nulling the
entries q of Q for i+j od is equivalent to Q1


Proof

Since Q1 is unity rank, its elements can easily be written as

functions of the elements of the factor vector; that is,


S

Q hh T = 1h2

hIh


hh hh hh
12 13 1 4
2
h2 h2h h2h .
2
h2h3 h3 h3h4
23 3 3


where h. is the ith entry of h. The matrix Q constructed from Q by
i 2 where ij is odd is
discarding the elements qij where i +j is odd is


2



bh1
2-


0 h h3 0
2
12 0 h h2h ..
2 2
0 hh O
o


It is easily verified that a rank two factor of Q2 is


h
21
0

iH2 h3
0


Q2 = H22








Part (a) of Theorem 3.3, i.e., Y(Q1 ;) = Y(Q ;w), is satisfied

by virtue of Corollary 3.3 and part (c) is fulfilled because Q2 is

shown to be positive semidefinite as a result of the existence of the

factor H2. Then all that is required is to show that no unstable

poles will be unobservable by H2 (part b) to prove Q Q1. In fact,

an even stronger condition will be shown; that is, the only plant poles

which are unobservable through Q2 are precisely those which are unobserv-

able through Q This will effectively satisfy part (b) since Q1 is

required by the hypothesis to result in a stable control law and hence

cannot fail to observe any unstable plant poles

It must now be demonstrated that the only common factors of

the polynomial entries of the vector,


H S(s), (19)


are also factors of the corresponding polynomial for Q1,


h S(s) .

If H2 is thought of as defining two outputs to the system, the above

requires that the outputs not simultaneously obscure any system pole

which is observable through Q1. The polynomials formed by (19) are

2 4
hI + h3s + h5s +


3 5
and h s + h s + h s + ..


These polynomials can be recognized as the result of "separating" the

polynomial h TS(s) into even and odd functions of s. This is a common

procedure in Neicj.ork Theory in which the polynomial operators

Ev (*) and Od (*) are often defined [T31 to simplify notation.








In this notation


F(s) = h S(s) = Ev(F) + Od(F) = Ev(F) + sEv (F),


where Ev (F) is Od(F) with an s factored out. Making use of the prop-
o

erty of even polynomials that their zeros lie symmetrically about the

imaginary axis and denoting the highest even power coefficient of F

as c and the highest odd power coefficient as c F(s) can be
e o

rewritten as

2 2
F(s) = c IT (s + a.) + s c 7T (s + b.).
ei=l 1 ojl=1

In the above vaS and Vb are zeros of'Ev(F) and Ev (F), and
1 3 o
the upper limits on the products must be chosen with regard to whether

the dimension of the system is even or odd. Now if Ev(F) and Ev (F)

possess a common factor, then a = b for some k and m and
k m
2 2 2
F(s) = (s2 + ak)c i=l (s +a) + c IT (s +b.)
e i= 1 0 j=1 3
igk jdm


which is clearly a factor of F(s) as well, and the theorem is proven.

This proof of the observability of the rank two Q will be use-

ful in the next theorem. A simpler analysis follows easily from

consideration of the composition of the two outputs defined by H2'

Summing these outputs results in a single output identical to the one

defined by h, thus any response which is observable from h is also

observable from H2.

This theorem guarantees that if a system and an optimal control

minimize a single output in the mean-square sense (h is unity rank),

then with little effort a pair of outputs can be defined which are also

minimized. The real value of this result is, however, that it allows








enormous simplification of the unity rank performance index at the onset

of the problem, with attendant savings in numerical quality and quantity.

This result will also find application in the next chapter in another

context.

A special case of Theorem 3.4 reveals an interesting structure

which is well worth recording.


Corollary 3.4

If matrix Q1 of Theorem 3.4 is such that its factor h
forms a polynomial hTS(s) which has all zeros with non-
positive real parts; then the factor H2 of matrix Q2 con-
structed as described in the proof will form two polynomials,


Fl(s)



having simple zeros (except possibly at the origin) which
are restricted to lie on the imaginary axis where they
occur in conjugate pairs and alternate with each other.


Proof

The proof only requires that the root location property

described be demonstrated. The polynomials of interest are easily

recognized as

F (s) = Ev(F)

F2(s) = Od(F)

and F(s) = h S(s).


It is a well-known theorem of Network Theory that the

Ev(F) and Od(F) functions of a Hurwitz polynomial, F, have the root

partitioning property recounted in the corollary [G6]. This phenom-

enon is referred to as the Alternation [G6] or Separation [T3] property.









A complex plane diagram of the respective zero locations of a typical

case is useful to visualize this result.





o

o zeros of Ev(F)
o
x zeros of Od(F)
Re
0 F a Hurwitz polynomial








Figure 3.5 Zero Locations of Ev(F) and Od(F)




This corollary implies more than appears in the hypothesis.

By Theorem 3.2 a unity rank Q, with h S(s) a Hurwitz polynomial, can

always be constructed if Y(u) is non-negative, then the sparse matrix

of the corollary can always be constructed as well.




3.6 Resum4 of Y-Invariant Matrices

Because of the necessity of investigating observability and

the testing for the absence of conjugate points much of the heuristic

appeal of the form of matrices invariant under the operator Y(Q;u)

is obscured. In order to distinguish between matrices equivalent in

the sense of Theorem 3.3 and matrices which result in identical Y's,

without resorting to cumbersome phraseology, the latter will be

referred to as "Y-invariant matrices." The conjugate point condition









is satisfied a priori for a very large class of weighting matrices

(i.e., positive semidefinite) and the observability restriction has

been shown to be a relatively innocuous constraint between T-invariant

matrices. Then a great deal is to be gained from a review of the

structure of invariant matrices.



General Structure


Corollary 3.3 reveals that two matrices are Y-invariant if and

only if their elements are such that the n-tuple


p. = q.. 2q + 2q .. i = 1,2,...,n
S 1ii i-l,i+l i-2,i+2 -

(q.. = O for i < 1 or j > n)


is equal for both matrices. For instance, in the third order case this

can be interpreted as meaning that Y will remain invariant.if a quantity

is added to q1,3 and q3,1 and twice that quantity is added to q22

In general, matrices which are Y-invariants of a given mt trix may be: con-

structed by manipulating the elements on "diagonals" running from

lower left to upper right. This can be represented pictorially as in

Figure 3.6.

1 x 2 x 3

x 2 x 3

2 x 3 x

x 3 ... x n-I

3 x n-l x

x n-1 x n


Figure 3.6 Structure of Y-Invariant Matrices









The integer entries of the illustration indicate which term of the

n-tuple is affected by the element of a Q matrix which would be in that

location; the x's are, of course, entries which influence no term.

From this it is easy to see that matrices Y-invariantAo a given matrix

can be identified by inspection.



Spectral Factorization


A special case which is of considerable interest is the Q

formed from the coefficients of the Hurwitz spectral factor of Y(w).

It is a form which can always be constructed when 'Y(u) is non-negative

and always meets the observability and conjugate point criteria. When

the i+j odd terms are zeroed the resulting matrix is always equivalent

and possesses the root partitioning property of Corollary 3.4. This

form is theoretically straightforward to compute but in practice is

among the most difficult.



Diagonal


Another special case which is appealing in its simplicity is

the diagonal Q. This matrix is formed by placing the coefficients of

Y(2 ) along the diagonal with the constant term first. AY-invariant

diagonal matrix also can always be formed; no observability difficulties

will be encountered if the terms are all non-zero and the conjugate

point condition will be satisfied if the terms are non-negative.

A simple example helps to demonstrate how this all fits together.








Example 3.3

Consider the real, even polynomial,


F(W) = W- + 2u2 +1,


and some members of the set of (Y-invariant) matrices,


fQIY(Q;wu) = F(w))3

i) Diagonal

The diagonal Y-invariant matrix is simply,




Q1 = 0 2 :. O ,

0 0 1


which is non-singular (thus observable) and also positive definite.

ii) Unity Rank

The roots of F(w) are sketched in the complex plane'diagram

below.






-1 +1


If the left half-plane zeros are chosen, the resulting unity rank

Q = I1 is

S1 2 1 -1

Q2 = 2 4 2 = 2 [1 2 11

1 2 1_








and zeroing the i+j odd terms results in



S o f' 1 o

Q = 0 4 0 = 0 2 0

J1 0 1 1 0


This matrix is clearly rank two and can easily be shown a -invariant

of Q1 by adding -1 to q13 and q31 and twice that (-2) to q22. The

zeros of H S(s) for Q3 are, respectively, j for the first output and 0

for the second, demonstrating the separation property of Corollary 3.4.

If the factor is composed of one zero from each half-plane, the

resulting Q is


1 0 -1

Q4 = 0 0 0 ,

-1 0 1


which is again clearly an invariant of Q 1

Note that in every Q matrix gener ;ed in this example the q11

and qnn entries remain fixed. From Corollary 3.3 it is obvious that

this must always be the case and that these terms are, respectively,

the first (constant term) and last coefficients of Y(w). The constancy

of these elements has a very interesting physical interpretation which

will be discussed in the next chapter.


3.7 Summary

The introduction to this chapter alluded to a concern which has

been often expressed, with varying degrees of vehemence, by those well-

versed in classical design techniques; that is, the minimality of some









performance measure is seldom directly relevant to a practical design

problem. For the quadratic performance index and a linear system,

however, it was shown that resulting optimal system configurations

possess characteristics which are very much in the spirit of good

system design. The minimum optimal system stability margins of

600 in phase and 6 db in gain compare favorably with those encountered

in practice.

The problem of actually constructing a performance index which

is minimized by an optimal system was found to lead, with due consider-

ation for certain cancellation and existence difficulties, to an equiv-

alence class of weighting matrices which enjoy some extremely enlight-

ening and useful mutual properties. When a single member of an equiva-

lence class has been determined.the remainder of the class can be

constructed with relative ease.













CHAPTER IV


THE INVERSE PROBLEM AND LINEAR REGULATOR DESIGN



4.1 Introduction

The previous chapters have dealt with optimal systems, their

properties and construction. It is essential that these concepts be

well in hand if the techniques of optimal control theory are to be

applied to design problems where a relevant performance measure is not

readily discernible. In general, this is the case when a linear regu-

lator is to be designed using optimal control theory. This chapter

will investigate how optimal control theory and classical techniques

may be used to complement one another in practical design problems.

For the purposes of this chapter it is helpful to think of

classical synthesis techniques for linear control systems in terms of

Figure 4.1. The problem originates with a linear system and a number

of performance specifications that the compensated system is to satisfy

(node A). The desired result is a realization (node C) which meets the

performance criteria and contains a compensator which is satisfactory

from the standpoint of practicality constraints, such as realizability

and noisy or incomplete measurements.

One way of approaching the problem is to prescribe pole loca-

tions which will meet the performance specifications (node B) and then

to construct a compensator which will place the plant poles in approx-

imately the desired locations while not contributing significantly









to system response (path II). This is a quite prevalent philosophy,

although it is often obscured by the specific design procedure used.

Path I can be thought of as classical synthesis and analysis procedures

used iteratively to arrive at these pole locations.

Specifications


III


Pole
Locations


Realization


Classical Design Procedure Alternatives

I. Prescribe pole locations 1. Compute performance index

a. from specifications
II. Design compensator a. from specifications
b. from pole locations

III. Direct design 2. Optimal control problem

3. Automated optimal compensator
design

Figure 4.1 Control System Design Procedures




A second approach is to determine at the onset the form of the

compensator required and manipulate its parameters to arrive directly

at a realization (path III). Well-known variations of this approach

are Evans' root locus and lead-lag design [E2]. These techniques have

the decided advantage of defining a clear-cut way to proceed but it may

be difficult to accommodate noisy measurements.

The procedures of this chapter provide alternative paths in the

design scheme as illustrated in Figure 4.1 by the Arabic numbered









branches. This provides for much greater flexibility in how the problem

is attacked and employs the power of optimal control theory to relieve

some of the procedural or computational burden in some or all phases

of the design.

In some cases it will be possible to define a performance index

directly from classical specifications (path la) which will be minimized

by a control law which satisfies the specifications. The specific con-

trol law can then easily be computed (path 2) or the corresponding

compensators can be designed (path 3), using Kalman-Bucy filter theory

[K8,K9] or one of the new techniques for automated compensator design

[P1,P2].

As indicated by path lb of the figure, it is possible to compute

a performance index corresponding to a closed-loop (optimal) pole con-

figuration and the remainder of the synthesis can be completed through

the use of compensator design techniques discussed in the last paragraph.

Figure 4.1 does not reveal some of the additional flexibility

allowed by these procedures. For instance, a performance index may

be specified which meets only a subset of the specifications and may

then be used as a basis for design iteration. The techniques also

provide for the design of sampled-data compensators (Section 4.4).


4.2 Pole Placement by Performance
Index Designation

If a set of closed-loop system poles are proposed as a prelim-

inary or final design configuration, the prescription of a performance

index which is minimized by this configuration (path Ib of Figure 4.1)

may be of considerable value in the completion or refinement of the

design. Figure 4.2 illustrates such a case.










j


x x plant poles

-- -- -n-- --x---- proposed closed-loop poles

x




Figure 4.2 Proposed Closed-loop Pole Configuration




This configuration may have resulted from a root-locus type

analysis, the exact scheme used is not germane to the present discus-

sion; the important point is that it is not bound to any technique or

form of compensation. The construction of a performance index which

is minimized by this configuration (if possible) provides at most

2 1
approximately n /4 nominal parameters which may be varied to perturb

the design or used to design the required compensator (path 3 of

Figure 4.1).

The problem of computing a quadratic performance index which

is minimized by specified closed-loop pole locations for a given

plant is basically the problem entertained in great detail in the

preceding chapter. This section seeks to deal with this problem on

a more pragmatic basis.

The plant to be considered is a completely controllable,

constant, linear system taken without loss of generality to be in

companion matrix canonical form and the performance index is the

infinite final time case of the last chapter.


The precise number of pertinent parameters will be given later.









The stable closed-loop pole configuration of Figure 4.2 will be

optimal with a positive semidefinite weighting matrix Q if and only if


p(w) = Jp

where cp(s) and p k(s) are the normalized (highest coefficient is unity)

open- and closed-loop characteristic polynomials. Although (1) is a

succinct optimality criterion, it is not obvious how best to proceed

to test a given polynomial (1) for non-negativity.

The necessity for testing the sign semidefiniteness of real

even polynomials arises in many other applications, for instance, tests

for positive reality in Network Theory (VI]. It can be shown by

factoring Y(w2) or through the use of Sturm's Theorem that (1) is

equivalent to requiring that Y(w 2) have no positive real roots of odd

multiplicity [Vl, p. 106].

There is still not a computationally satisfactory approach

evident. Calculation of the zeros of Y(w2) if the order is large,

is difficult numerically and best avoided as a test. Recent results

by Siljak [S3] and Karmarkar [K10] extend the interpretation of the

sign changes in the first column of the Routh table [G4] to test for

positive real zeros of odd multiplicity. This test is probably the

only alternative currently available to the computation of the zeros

of Y(w 2) as a necessary and sufficient test for non-negativity.

Appendix C outlines a numerical implementation (in Fortran IV)

of a modification of Siljak's method, which is both accurate and effi-

cient; it is believed to be the only such program in existence.








It should be reiterated that condition (1) applies only to

optimality with respect to a positive semidefinite Q; a system with

Y(u') 2 0 may well be optimal for a sign indefinite Q. In such a case

the properties conditioned on Q being positive semidefinite (Section

3.3) would not in general hold but the Q equivalence relations and

remarks concerning the observability requirements for Q (Chapter III)do.

If criterion (1) is satisfied, a positive semidefinite weighting

matrix can be generated, in fact, an entire equivalence class of posi-

tive semidefinite Q's. In general the initial member of the equivalence

class must be computed by spectral factorization. This operation, like

the test for non-negativity, is not a numerically trivial one. The

spectral factorization of Y(u), if approached naively, consists of

determining the 2(n-1) roots of T, separating them by the sign of their

real parts, constructing the factor composed of the zeros with negative

real parts and multiplying it by the appropriate constant. This pro-

cedure is entirely unsatisfactory. It is well known that the confidence

in approximation for root locations generally decreases drastically for

polynomials of large order; this coupled with the error induced by

constructing the spectral factor from its roots makes this procedure

numerically hazardous.

An obvious alternative is to reduce the order of Y(wu) by sub-
2
stituting c = w and computing the roots of an n 1 order polynomial

which are the squares of the actual roots of interest. The roots of

Y(w) with negative real parts can be obtained directly. A refinement

of this procedure is to compute quadratic factors of Y(o) only and

from these factors, through some rather intricate logic, garner the









related left half-plane quadratic factors of Y(w). This serves to

reduce the total number of numerical manipulations, saving computing

time and decreasing the sources of error propagation. The implementa-

tion of the spectral factorization algorithm discussed in the appendix

takes this approach, utilizing an efficient technique due to Bairstow

[Kll], for the approximation of quadratic factors. Bairstow's iter-

ation has been shown to have a rapid rate of convergence, although it

is somewhat more sensitive to starting values than competing methods

[Kll, p. 101ff].

One objection to this procedure is that the roots of "(w) are

not directly available as a secondary test for non-negativity. The

test for non-negativity described earlier seems wholly adequate and

any sacrifice of computational efficiency in the spectral factorization

in favor of a redundant test does not appear justifiable.

The flow diagram of Figure 4.3 summarizes concisely the steps

to be taken in the generation of a quadratic performance index which

is minimized by a given plant and stable proposed (optimal) closed-loop

pole configuration. The first step () is to compute the closed-loop

and plant normalized characteris'-ic polynomials (if not already known).

The magnitude-square of the open- and closed-loop characteristic poly-

nomials are next computed; this is another operation where the obvious

procedure, i.e., multiplication of the respective polynomials by their

complex conjugates, is not the best choice. Since the magnitude-square

polynomials are even, half of the coefficients are zero. The-technique

outlined in Appendix C makes use of this structure by computing only

the non-zero terms and in a way which obviates the use of complex



































Yes


k Optimal Coordinate
Yes for Transformation
Q (if required)


Figure 4.3 Procedure for Specification of a Performance
Index from Proposed Pole Locations

Numbers in circles are referred to in the text.








arithmetic. When the difference of the magnitude-square polynomials

is taken (, the resulting Y(w) is an even polynomial of order 2(n-1)

due to the normalization of the highest power coefficients of Cp and ck

to one.

A decision based on the non-negativity of Y is made in blockG.

If Y is non-negative the system is clearly optimal and the specifica-

tion of the Q's may proceed. The unity-rank Q is computed from the

left half-plane factor of Y ), using the techniques for spectral

factorization discussed earlier, which further insures the observabil-

ity conditions are satisfied, and the equivalence class of positive

semidefinite Q's is immediately available.

If Y is not non-negative the procedure is no longer straight-

forward and positive results are not guaranteed. An initial Q1 is

constructed so that

Y(Q1 ;) = y(u),

and all the unstable plant poles are observable ). A reasonable
2
choice for an initial Q is the diagonal case (coefficients of Y(w )

on diagonal); this is the easiest to generate and a desirable form

for the answer. This choice of Q is inserted in the Riccati equation

for the system in companion matrix canonical form,


P(t) = -FTP(t) -P(t)F+ P(t)ggT P(t) Q, P(tf) )=0. (2)


The Riccati equation is then integrated numerically until


II P(t) I S e, (3)

where C is some prescribed non-negative bound or until a conjugate

point intervenes (the solution becomes unbounded) O. The non-linear









nature of the Riccati equation, which makes this sort of test necessary,

also helps insure the success of the test, since it may be subject to

the phenomenon of finite escape time [K2]. That is, as opposed to the

solutions to linear differential equations which may be unbounded only

in the limit, the Riccati equation may become unbounded in a finite time.

Experience has shown that when a conjugate point does exist (2) generally

becomes unbounded quite rapidly (in terms of the number of integration

steps) and conversely when a conjugate point is not present steady-

state (3) is reached promptly.

A third possibility is that (2) has neither a conjugate point

nor a steady-state solution; that is, the solution is oscillatory.

It is clear that if (2) is oscillatory it must also be periodic and

hence the cost and the system response are periodic. This would

contradict the supposition that the proposed system design was time-

invariant and stable. Then, if a periodic solution to the Riccati

equation occurs, it must be due to numerical errors and discarded in the

same manner as a solution with a conjugate point.

A Q. which results in a solution of (2) which diverges obviously
J
must be discarded but it does not, in itself, indicate that the system

is not optimal. The procedure branches to block 8 if a conjugate

point is present and Q which is F-invariant to Q. is computed and
j+1 J
the test for conjugate points 7 is repeated. It is difficult to

conceive of an algorithmic scheme for "improving" Q. in block 18'.

In fact, if this technique were known, one could apply it iteratively

until the "best" Q is obtained and the subsequent test for conjugate

points would be a general necessary and sufficient test for the










optimality of a control law An interactive strategy is ideal, Y-

invariant Q's can be generated which not only are more likely to con-

verge but also possess a desirable structure for the specific applica-

tion.

The algorithm to be described seeks to generate anY-invariiat

Q+1 from a Q. which fails the conjugate point condition ( by shift-

ing the negative eigenvalues of Q. generally to the right while main-

taining as much simplicity as possible in Q j+1. This is done by test-
2
ing the diagonal terms q.. of Q. for 1 < i < n progressively until

a negative entry is detected which is then deleted, using equation (17)

of Chapter III.

Suppose that the minor shown below is a 3 x 3 principal minor,

containing the negative diagonal entry b.


- - ,- - - - ,
-------------------------r
-d/2 0O a 0 0

0 0o 0 b 0

0 0 1 0 0 c
'- - - r--


Because the first Q (Q 1) was diagonal and since the method proceeds

progressively down the diagonal, the remainder of the rows shown are

null except the first which may have an off-diagonal entry -d/2 placed

by the preceding iteration. By the results of the last chapter,

altering this minor as shown below:




2
As noted in the preceding chapter, the first and last terms on
the diagonal cannot be altered and invariance maintained.












- -L - - - -F
-(1/2 0 a 0 -b/2 I

0 0 0

-b/2 0 c




leaves the y of the resulting matrix invariant. Each time a Q. fails the

conjugate point test the described procedure removes an additional

negative diagonal entry until all have been eliminated. In this

manner the scheme allows k+1 iterations where k is the number of nega-

tive coefficients of (u2 ) (or the number of negative diagonal terms)

before the system is discarded. The procedure is executed iteratively,

rather than eliminating all of the negative terms in one step, with

the hope of determining as simple (diagonal) a Q as possible for which

the system is optimal.

At this point the justification for such a procedure is

probably not clear The object of the algorithm is to generate

a sequence of invariant Q's whose positive diagonal elements dominate

their respective rows. This will hopefully result in a shifting of

the eigenvalues to the right. Central to this argument is the

Gerschgorin Circle Theorem which was recorded as Lemma 3.5 in the

previous chapter.

Since the rows of any Q. generated by the described technique

have at most two off-diagonal elements, each being half of a negative

diagonal term, the eigenvalues of the modified matrices will be

increasingly restricted to be "close" to the right half-plane if the










diagonal elements are approximately of the same magnitude. There is

no hope that the matrices will eventually become positive semidefinite

with repeated iterations (this would require Y(w) O by Theorem 3.2);

however, it is reasonable to expect the likelihood of passing the

conjugate point condition to increase if the eigenvalues are shifted

to the right. An example will help to clarify the details.


Example 4.1

Consider the sequence of Q's which would be generated by this

method for

Y(w) = 7w12 6W10 5u + 3w6 + 2u 4u2 + 1.


This Y is clearly not non-negative, e.g., y(l) = 2; then there is no

possibility of constructing a positive semidefinite Q for it. The

procedure beginning with block 6 of the flow chart of Figure 4.3

must be called upon. Since there are three negative coefficients

of Y there will be at most.four Y-invarian.t, Q's generated; Figure 4.4

illustrates these iterations. The first, Q1, obviously has eigenvalues

which are simply the coefficients of t(w u ). The first iteration removes

the first negative entry on the diagonal (-4) by "splitting" it into

the two off-diagonal terms resulting in Q2. Four of the eigenvalues

remain unchanged, the one that was at -4 is moved to the origin and

the remaining two have been "smeared" by the interpretation given by

Gerschgorin's Theorem. The eigenvalue which was at 1 may now be any-

where within the interval (-1,+3) and the eigenvalue at 2 may now lie

in the region (0,+4) as shown in the plot.


Actually the right half-line, since the matrices under con-
sideration are symmetric and consequently have only real eigenvalues.











12 10 8 6 4 2
Y(U) = 7 610 5u + 3j + 2u 42 + 1


r1 0 2

-4 (") 0 0 0

2 2 0 2

3 0


-6

7


0


3 0 2.5

0 0 0

2.5 0 -6


1

0
2

Q4=


o 0 0

0 1

3 0 2.5

0 0 0 3

O 2.5 0 0 0
3 0 7


-xxx
-- -X X

--------x--x---,-F


------------0. .----


---------- -- ----v^B^C,


-10 -5


-'-x -- x---- Q


-------0- Q4

0c x-


-c Q 4


5 10

o Computed eigenvalues
x Eigenvalues by inspection
mn- Permitted eigenvalue regions


Eigenvalue Distributions

Figure 4.4 Eigenvalue Distributions for Q
.1


F1


I I -1-- -


'2









With subsequent iterations the original eigenvalues are spread

into permitted regions which are generally to the right. Due to the

structure of such matrices the actual eigenvalues are easily computed

and included in the plot. The process lives up to expectation, the

negative eigenvalues are moved distinctly to the right and the likeli-

hood of passing the conjugate point condition (esp. for Q4) is con-

siderably enhanced.

Note that the rank of Q for j > 1 is reduced by at least one.

This causes concern that some observability may be lost in the process,

with the result that an unstable plant pole becomes unobservable (a

stable plant pole becoming unobservable is, of course, inconsequential).

Although this may indeed be the case, it will be shown later that suf-

ficient safeguards exist to prevent it from passing unnoticed.

Within the framework of the process outlined there exist many

variations which may be of value in certain applications. For instance,

it may be advantageous to dispatch the negative diagonal terms of

largest modulus first rather than approaching them progressively along

the diagonal. This modification applied to Example 4.1 leads to a

somewhat greater "improvement" in the second iteration than the method

originally employed, but the results of the final iteration are, of

course, identical. The important observation to make is that no single

variation will be best in every case and that employment of good judg-

ment at this point in the scheme, rather than following a strictly

algorithmic approach, will probably be rewarded with simpler resulting

Q's. It is for this reason that the suggestion was made earlier to

utilize an interactive strategy in a computer implementation of this

procedure.









If the iterative technique of modifying Q. and testing for
J
conjugate points succeeds in obtaining a Q for which the Riccati equa-

tion does not diverge, the next step is to determine if this Q is

actually a solution to the inverse problem. This is necessary to guard

against gross numerical errors in the integration of the Riccati

equation falsely indicating the absence of conjugate points, and, as

mentioned earlier, to insure that a detrimental loss of observability

from Q has not occurred.

The steady-state solution, P, to the Riccati equation (2) is

available from the conjugate point test (block ( and provides this

check. The feedback control law,

k = Pg,

is simply the last column (and row) of P. By the structure of the

companion matrix canonical form if

n n-1
cp(s) = s + ans + ... + a s + al'


then Yk(s) = s + (a +k )s + ... + (a +k2)s + (a +k ),


where k* is the ith entry of k. The coefficients of pk computed by

substituting p. for k. can be compared to the correct (original) values.

This test is represented by block ( of Figure 4.3.

In passing, it should be noted that the process for generating

successive Q's outlined here fails to make full use of the relation

of the elements of invariant matrices (Corollary 3.3) in that only the

terms on the diagonal and second superior diagonal are affected (i.e.,

qii' i+2,i-2 and qi-2,i+2 when qii < 0). This is because the process

becomes considerably more difficult to justify as the number of altered










terms increases and because it was felt that the additional complexity

in the resulting Q was self-defeating. However, further research in

this area may ultimately lead to a necessary and sufficient condition

for optimality when Q is not positive semidefinite.

There remains but one operation which may be required to com-

plete the process of designing a performance index for which a given

closed-loop pole configuration is optimal. If the state model for the

plant (assumed completely controllable) is not in companion matrix

canonical form it may be desired to transform the Q which has been

generated into the coordinate system of the plant. This is accomplished

by the congruent transformation,


Q = T QT,


where Q is in the original coordinate system of the plant and T is the

non-singular matrix of the similarity transformation (Section 3.4),

A ~ -1
F = TFT ,


which transforms the original state matrix (F) into a companion

matrix (F).

It is necessary to show that this can be done without loss of

generality. That is, that the closed-loop poles are invariant and that

no conjugate points are introduced under this transformation.










Theorem 4.1

Optimality is preserved under similarity transfor-
mation. That is, if a completely controllable linear
plant and feedback control law are optimal with respect
to a quadratic performance index, the system and control
law subject to a non-singular coordinate transformation
minimizes the performance index in the new coordinate
system.

This theorem can be easily proven by applying the transforma-

tion required to return the system to its original form from the

companion matrix formulation to the Riccati equation. Then, by the

non-singularity of T, if the Riccati equation becomes unbounded in

one coordinate it will in the other as well.


4.3 Design by Performance Index Iteration

Once a technique, such as the one presented in the last section,

is available for generation of performance indices for optimal closed-

loop pole configurations a simple design scheme becomes evident. The

performance index could be used as a basis for improving a first attempt

at design. In terms of Figure 4.1, this would entail traversing

branches lb and 2 iteratively, each time altering the entries of Q in

a manner intended to meet additional members of the specification set.

This approach offers several advantages over competitive classical

design techniques. If the preliminary closed-loop pole configuration is

chosen so that


Y(uW) = k (jiw) 2 cpi(ju)2 1 0 for all real m,


then maintenance of the positive semidefiniteness of the resulting Q

throughout all the iterations will insure that the important stability

criteria of gain and phase margin are at least 6 db and 600, respectively.










This permits the designer to concentrate on bringing transient response

to within desired limits and later returning to the stability margins

if more stringent ones are required.

A second advantage is that a tractable number of design param-

eters are displayed in the Q matrix. The precise number can easily be


shown to be [ |) where the heavy brackets represent the greatest

integer function; i.e.,


[a] = greatest integer 5 a.


This quantity does not include the duplicate (by symmetry) terms appear-

ing across the diagonal of Q.

This means that for a third-order plant there are at most four

parameters of Q which are pertinent to the design; similarly for a

tenth order plant, at most 32. These numbers may seem rather large in

relation to the number of coefficients of the characteristic poly-

nomials; however, these parameters, as will be seen later, may be

related more or less directly to the transient response of the closed-

loop system, in contrast to the classical design parameters. Further,

far less than the maximum number given will be generally required; in

fact, usually it will only be necessary to vary two of them [H2].

It was noted in the last chapter that the first and last terms

on the diagonal of equivalent Q matrices must remain invariant. That

is, q11 must be the same among all members of an equivalence class and

similarly qnn must not change. The next theorem presents an important

new result which is a consequence of this observation.










Theorem 4.2

If Q is any member of an equivalence class of weighting
matrices (Definition 3.1) for a scalar nth order linear
system and optimal control law in companion matrix canonical
form, then
n 9 n 2
q I R TT r
11 i 1 i i=l 1

n 2 n 2
and q =n R S r
nn i=l i 1=

where r. and R., i = 1,2,3,...,n

are the plant and optimal closed-loop poles, respectively.


Proof

In the discussion of the diagonal Q case in Section 3.6, it was

shown that the coefficients of Y(u2 ) form the diagonal entries. The

optimization problem implied by this Q may have conjugate points.

Nonetheless, since the control law of the hypothesis was specified

to be optimal, there must exist at least one weighting matrix which

forms a performance index minimized by the control law and further

it must bea Y-invariant of the diagonal Q,. Then, since the first and

last entries on the diagonal of invariant matrices cannot change, these
2
entries must be exactly the first and last coefficients of Y(w ).

Consider the monic polynomial with real coefficients,

n n-1
p(s) = s + a s + ... + a s + a (4)
n-1 1 o

and its magnitude square


Ip(j |2 2n 2(n-1) 2 .
|p(j) = w + b 0( + ... + b 0 + b (5)
n-l 1 o










By definition


Y(u) = Ik(j )2I [p(ju)I2; (6)


that is, Y is the difference of two polynomials similar to (5).

Then the constant term of Y, q11, is the difference of two terms

corresponding to b in (5). Clearly,

2
b = a ,
o o

and by a well-known theorem of elementary algebra, the constant

term of a monic polynomial is (to within the sign) the product

of its zeros. Then b is the square of the product of the roots of

(4) and application of this abstraction to (6) demonstrates the

expression for q11'

2(n-1)
The coefficient of wo of y, q is the difference of
n,n
terms similar to b n- in (5). By examining the product of p(ju) and

its complex conjugate it is easily verified that


b = a 2a (7)
n-l n-1 n-2'


Again it is known from elementary algebra that for amonic poly-

nomial such as (4), a n- is the negative of the sum of the zeros of

4
(4) and an-2 is the sum of all combinations of products of the roots

of (4) taken two at a time. That is,






4
Here combination is used in the sense of combinatorial
analysis; i.e., no distinction is made between the ordered pairs
(c,d) and (d,c).











n
a r r
n-1 1 i
i=1

and

n i-1
an-2 = E Z rr where r ,i=1,2,3,...,n are the
i=2 j=l

roots of (4). The expression for a can be rewritten by forming the

sum of all the possible permutations of products of two roots (includ-

ing with themselves) and subtracting off the sum of the roots squared

and dividing by two to account for the combinations being summed twice;

i.e.

1 7n 2 n 2
S -l r. r r2
n-2 2 1 i


Then, using this expression for an_2 and substituting into (7) results

in
n 2
b r. .
-1 =l


The expression for qnn then follows directly from application of this

result to equation (6).

The interpretation of the effect of q on a system can be

greatly enhanced through the use of a simple mechanical analogy.

If the poles (r.) of a linear system are thought of as unit point

masses, the expression,

n 2
I -= r.,
i=l

would be the resulting moment of inertia with respect to a perpendic-

ular axis of rotation through the origin such a mechanical system

would possess. Then qnn represents the amount by which the "moment
nn









of inertia" of the system has increased with the addition of optimal

feedback.

If qnn is non-negative the optimal closed-loop system will

have at least one pole further removed from the origin than the plant

poles. That is, for qnn > 0 the closed-loop system will tend to be

faster in response than the open-loop, and, in general, the larger qnn

is made the faster the optimal system.

Although q 11 does not admit to analogy as well as q nn its

effect on the optimal pole locations is similar to that of qnn ; that

is, the speed of response also tends to increase with q11'

Recently there has been a resurgence of experimental investi-

gations of the effect of entries of the Q matrix on the dynamic response

of the closed-loop system [e.g., H2,R1]. Although it is hopeless to

attempt to glean a generalized design scheme from such studies, since

they are bound to specific plant configurations (i.e., a fixed number

of poles and zeros), it is possible to obtain some guidelines.

Houpis and Constantinides [H2] observed that q11 and qnn together have

primary effect on rise time (t ) and setting time (ts) and the remainder

of the entries primarily influence overshoot. These observations seem

to coincide quite well with the analysis given here.

At this point a design example would help to solidify the

preceding remarks.










Example 4.2

It is desired to compensate the unstable plant



1.0
u s(s-l)(s+2)




so that the following performance specifications are met:

1. Gain Margin: GM 12 db

2. Phase Margin: CpM 600

and in response to a step input,

3. Overshoot: Os S 5%

4. Rise time (within 90%): tr 1 sec

5. Setting time (within 1%): t \ s 2 sec.


The approach taken here will be to develop a preliminary

design, using dominant roots techniques, which is also optimal, and

to then iterate on the entire of the Q matrix until a suitable final

design is arrived at. By dealing only with positive semidefinite Q's,

phase margin, and probably gain margin as well, will be guaranteed,

leaving only the dynamic response specifications to be investigated.

Using the dominant root philosophy, the complex poles should

have a time constant of approximately 2 seconds to meet the require-

ment for ts and a damping of approximately 0.7 in order to keep max-

imum overshoot within 5% and still meet rise time specifications

[D4, Fig. 4-3, p. 91]. The third pole removed by 2.5 times the dominant

time constant should insure dominance. The resulting preliminary

design pole configuration is shown in Figure 4.5.










2j




-----c 0-l-'--- I ---x-- -x----
-5 -4 -3 -2 -1 1



0


x plant poles

O proposed closed-loop poles


Figure 4.5 Preliminary Design Pole Configuration




The algorithm of the preceding section is now applied to obtain

a Q matrix (if one exists). The system is indeed optimal and a Q matrix

which will work is


1600

Q =

0


0-

0

20


The transient response of this system was computed, using the quadratic

optimization and simulation program LQL [B7,B9]. The important char-

acteristics of the response are recorded in the first row of the table

of Figure 4.6. It is clear that response is too slow, hence, q11 is

increased; however, continued increase much beyond the value of entry 3

causes the overshoot specification to be exceeded. Reducing q33 has the

effect of reducing the damping on the dominant poles (decreasing t r) and

bringing the third pole in toward the origin and results in more rapid

settling (decreasing ts). Run number 6 meets the specifications on

transient response.



































Figure 4.6 Transient Response vs q..




The stability margins only remain to be investigated.

Since the Q for all the iterations is positive semidefinite,

GMI 6 db and CpM 600. The Nyquist diagram of Figure 4.7 shows

that the gain margin is considerably in excess of 12 db; thus

iterative number 6 meets all the design specifications.


Run 33 str Os ts
No. 11 22 33 (sec) (%) (sec)


1 1600 60 20 1.2 3.3 2.6

2 576 60 20 1.6 2.0 3.2

3 3000 60 20 1.1 4.0 2.3

4 2500 60 10 1.1 4.1 2.2

5 2500 60 4 1.1 4.0 2.1

6 3000 60 4 1.0 4.6 2.0

7 5000 60 4 0.9 5.3 1.8






















Figure 1!.7 Nyquist Plot of Final Design


Figure 4.8 Root-Locus Plot of Final Design


X Plant poles

o Closed-loop poles

o Feedback zeros




































Final design


preliminary design
















Desired response envelope


1 2 3 4 5
Time (sec)


Figure 4.9 System Responses to a Step Input Preliminary and Final Designs











The final design meets the performance criteria:


GM = 15.18 db

tM = 62.90

tp = 1.0 sec
ts = 2.0 sec

Os = 4.6%


and has the state feedback control law


54.77

k= 33.53

7.49



Although Figure 4.6 only tabulates seven iterations on the Q

matrix, fourteen iterations using program LQL were actually made,

requiring approximately 6 seconds of CPU time on an IBM 360/65 at

a cost of about $0.65.

This example illustrates two facets of this method. First,

there are no concrete rules, as such, for the manipulation of the

entries of Q to obtain desired changes in response, only general

guidelines. Secondly, the speed and low cost with which the iter-

ations can be computed make this process of practical value.

A final feature of this scheme is that the resulting design

is not bound to any specific form of compensation: this can be

regarded at times either as an advantage or a handicap However, with

the rapid advance being made in automated compensator design [e.g., Pl,

P2], the disadvantages associated with the compensator not being

prespecified are becoming largely illusionary.










4.4 Design by Explicit Performance Index Specification

If the design problem is more fully specified, a useful varia-

tion of the preceding scheme will often be profitable. For instance,

when specifications of the sort required for Example 4.2 are given and

in addition it is desired to minimize (in a mean-square sense) a spe-

cific system output, or a weighted sum of outputs, the technique of

performance index iteration takes on a particularly interesting form.

Consider the scalar input, multioutput linear system,


x = Fx + gu


y = H X,


for which it is desired to determine a feedback control law so that

a given (but at the moment arbitrary) set of classical performance

specifications are satisfied and in addition the performance index,


J = (cyTAy + u2)dt, (8)



is minimized, where c is an unspecified scalar and A a symmetric

weighting matrix. Then in the usual notation,


Q = cHAH


and cY(u) = cY(HAHi ;),


which can be easily computed using Lemma 3.4. Any closed-loop system

which minimizes (8) will have a characteristic polynomial, k (s),

such that




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs