THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM
Dy
Joseph Marcus Elder, Jr.
A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF
THE UNIVERSITY OF FLORIDA IN PARTIAL
FULFILLMENt OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1972
To my grandfather
Roy 1W. Estridge
who never doubted
and to my wife Dina
who never despaired
ACKNOWLEDGMENTS
The author wishes to express his sincere appreciation to the
members of his supervisory committee: Dr. T. E. Bullock, chairman;
Dr. W. H. Boykin, and Dr. A. E. Durling for their counsel and patience.
Special thanks are due to Dr. Bullock who gave unselfishly of his time
and energy; and abject apologies are due to his wife, Jeannette, who
endured the many late hours and missed dinners.
Thanks also are due to Dr. C. V. Shaffer who had an uncanny
knack for dispatching the multitude of small problems encountered along
the way, to Dr. L. E. Jones who was always ready to help, and to
Mr. Y. T. Parker for a great deal of food for thought.
The author is grateful to the University of Florida Graduate
School, the Department of Electrical Engineering, and the National
Aeronautics and Space Administration for financial assistance.
TABLE OF CONTENTS
Page
ACKNOWLEDGMENTS .. . . .. . . . . . . . . iii
LIST OF FIGURES . . . . . . . . ... . . vi
ABSTRACT .. . . . . . .. .... . . . vii
CHAPTER
I INTRODUCTION . . . . . . . . 1
1.1 Background . . . . . . . . . . 1
1.2 Survey of Previous Work . . . . . . 4
II THE LINEAR, QUADRATIC COST OPTIMIZATION PROBLEM . . 7
2.1 Introduction . . . . . . . . . 7
2.2 The Linear System, Quadratic Cost Problem . . 8
2.3 The Optimal Control Problem . . . . . 9
The EulerLagrange Equations . . . . 9
The Riccati Equation . ... ... . 10
The Conjugate Point Condition . . . ... 12
Sufficient Conditions Existence . . . 18
2.4 The Optimal Linear Regulator Infinite
Final Time . . . . . . . . . 20
Existence and Stability .. . . . . 20
2.5 Summary . . . . . . . .. . . 25
III THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM . 27
3.1 Introduction . . . .. . . . . . . 27
3.2 When Is a Linear Control System Optimal? . . 28
3.3 Implications of Optimality ... . . . .. 29
3.4 Companion Matrix Canonical Form . . ... 33
3.5 Characterization of the Equivalence Class of Q's 35
3.6 Resume of YInvariant 'Matrices . . . ... 57
General Structure . . . . . . 58
Spectral Factorization . . . . 59
Diagonal . . . . . . . . . 59
3.7 Summary . . . . . . ..... . 61
TABLE OF CONTENTS (Continued)
CHAPTER
IV THE INVERSE PROBLEM AND LlNEAR REGULATOR DESIC'. . .
4.1 Introduction . . . . . . . . . .
4.2 Pole Placement by Performance Index Desiratil. ..
4.3 Design by Performance Index Iteratior. . . .
4.4 Design by Explicit Performance Irdex
Specification . . . . . . . . .
4.5 SampledData Controller Design . . . . .
V CONCLUSIONS . . . . . . . . . . .
5.1 Summary of Results . . . . . . . .
5.2 Suggestions for Future Research . . . . .
APPENDICES
A ADDITIONAL PERFORMANCE INDICES ACCOMMODATED
BY THE THEORY . . . . . . . . . . .
A.1 Introduction . . . . . . .
A.2 Quadratic Performance Index with
Exponential Weighting . . .
A.3 Quadratic Performance Index with
CrossProducts . . . . . .
A.4 Quadratic Performance Indices with
Derivative Weighting . . . . .
B PROOF OF SUFFICIENCY OF THEOREM 3.1 . . .
C THE INVERSE PROBLEM NUMERICAL DETAILS . . . .
C.1 Introduction . . . .
C.2 Computation of Magnitude 
C.3 Test for Nonnegativity of
Even Polynomials . . .
C.4 Spectral Factorization .
C.5 Sample Program . . .
C.6 Subprogram Listing .
118
118
Square Polyramials
Real,
REFERENCES . . . . . . . . . . . .
Page
63
63
65
80
92
98
103
103
104
. . 107
. . 107
. . 109
. 111
148
LIST OF FIGURES
Figure Page
2.1 Conjugate Trajectory. . . . . . . 12
3.1 Return Difference . . . . . . . .... . 30
3.2 Nyquist Plot of Optimal System . . . . ... 31
3.3 System Pole Locations . . . . . . . . 38
3.4 Sparse Equivalent Matrix . . . . . . . 51
3.5 Zero Locations of Ev(F) and Od(F) . . . .... . 57
3.6 Structure of YInvariant Matrices . . . . . 58
4.1 Control System Design Procedures . . . . . 64
4.2 Proposed Closedloop Pole Configuration . . ... 66
4.3 Procedure for Specification of a Performance Index
from Proposed Pole Locations . . . . ... . 70
4.4 Eigenvalue Distributions for Q . . . . . . 76
4.5 Preliminary Design Pole Configuration . . . ... 87
4.6 Transient Response vs q. . . . . . . . . 88
4.7 Nyquist Plot of Final Design . . . . ... . 89
4.8 RootLocus Plot of Final Design . . . . ... 89
4.9 System Responses to a Step Input Preliminary
and Final Designs . . . . . . . .. . 90
A.1 Modified Plant . . . . . . . . 112
A.2 Synthesis of Optimal Control for Performance
Index with Control Derivative Weighting . . ... 114
Abstract of Dissertation Presented to the
Graduate Council of the University of Florida in Partial Fulfillment
of the Requirements for the Degree of Doctor of Philosophy
THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM
By
Joseph Marcus Elder, Jr.
March, 1972
Chairman: Dr. T. E. Bullock
Major Department: Electrical Engineering
The principal objective of this research is to extend the results
of R. E. Kalman's original analysis of the inverse optimal control prob
lem in order to create a design tool which allows the designer to employ
the powerful and sophisticated techniques of optimal control theory
when the correct choice of a performance index is not apparent. Specif
ically, methods are developed for the design of optimal linear regulators
(both continuous and discrete) which satisfy classical performance
specifications in addition to the minimality of a quadratic functional.
This permits the great computing power of optimal control theory to be
brought to bear on problems which previously could be accommodated only
by the cutandtry methods of classical design schemes. In passing,
insights into the basic nature of optimal regulators are developed in
terms of classical concepts, unifying some important notions in clas
sical and modern control theory.
The contributions of this research can be summarized as follows:
1. A complete theoretical investigation of the inverse
optimal linear regulator problem for quadratic per
formance indices with positive semidefinite state
weighting matrices and scalar input systems is reviewed
and preliminary results for the case where the weighting
matrix is allowed to be sign indefinite are given.
2. The equivalence of performance indices for scalar input
linear quadratic loss problems is resolved and a proce
dure for generating the entire equivalence class of cost
functions equivalent to a given performance index is
developed.
3. Practical numerical methods for determination of system
optimality and computation of solutions to the scalar
input inverse problem are discussed.
4. Some of the effects of specific elements of the perform
ance index on optimal system performance and pole loca
tions are determined.
5. The problem of designing optimal systems to meet classical
performance specifications is encountered and some defin
itive results obtained.
6. A solution to the problem of designing a sampleddata
controller to approximate the performance of continuous
controller is given.
CHAPTER I
INTRODUCTION
1.1 Background
In 1964 R. E. Kalman published a paper [K1]1 which has come to
have considerable impact on the theory of optimal control. It dealt
not with the conventional optimal control problem of computing a system
trajectory which extremizes a specified performance index, but rather
with the "inverse" problem of determining what performance indices, if
any, are extremized by a specified control. Kalman restricted his
attention to the case where 1he system is linear and the integral per
formance index is quadratic in the states and control. No immediate
direct application was made of Kalman's results, but they were of great
value in the application and interpretation of his earlier analysis
[K2] of what is now called the "linear quadratic loss" optimization
problem.
Quadratic functionals have long been studied in the calculus
of variations [Gl] but not until Kalman was the minimization of a func
tional quadratic in the states and input of a linear dynamical system
considered in the context of an optimal control problem. Other authors,
notably S. S. L. Chang fCl] and Newton, Gould, and Kaiser [NI],
Brackets contain reference information. A letter followed
by a numeral indicates, respectively, the first letter of the first
author's surname and the order of appearance within the given alpha
betic group of the specific reference.
had since the middle 1950's been occupied with the "analytical design" of
linear control systems through the use of a performance index quadratic
in the error (i.e., the difference between actual and desired system
responses to a specified input). Employment of Parseval's Theorem and
spectral factorization [N1] resulted in a solution strongly related to
the optimal linear filter of Wiener [Wl] of almost fifteen years earlier.
The solutions for the ISE (integral of the square of the error) problems
were very cumbersome to compute and seldom applied to systems of
greater than third order.
In the early 1960's the fledgling field of optimal control
theory underwent a metamorphosis. Difficult aerospace problems had
arisen which could only be conveniently approached as optimal control
problems. Rapidly there began appearing almost as many different
optimal control formulations as there were proponents. Superficially,
optimal control theory appeared to be a panacea; however, formidable
problems in computing and often even greater difficulties in the
implementation of some optimal designs limited their utility. In
general, the computation of optimal controls requires the use of
iterative algorithms which converge, at best, slowly. Further, optimal
controls are usually openloop in nature.
The linear quadratic loss problem suffers from neither of
these difficulties: the solution is numerically straightforward and
always results in a linear feedback control law which is easily
implemented; in addition both continuous and sampleddata controllers
can be accommodated. If the system is to operate in a noisy environment,
an extension of the quadratic loss formulation, the KalmanBucy filter,
can easily be included in the design.
Unfortunately, performance indices as a whole seldom relate
well to system design requirements which are given as classical time
and frequency.domain specifications. In this regard, the quadratic
loss problem suffers as well.
The principal objective of this research is to extend those
original results of Kalman in order to create a design tool which
allows the designer to employ the powerful and sophisticated techniques
of optimal control theory when the correct choice of a performance
index is not apparent. Specifically, methods are developed for the
design of optimal linear regulators (both continuous and discrete)
which satisfy classical performance specifications in addition to the
minimality of a quadratic functional. This would permit the great
computing power of optimal control theory to be brought to bear on
problems which previously could be accommodated only by the cutandtry
methods of classical design schemes. In passing, insights into the
basic nature of optimal regulators are developed in terms of classical
concepts, unifying some important notions in classical and modern
control theory.
The original impetus for this investigation was the requirement
to develop a method for designing a digital controller to replace an
existing continuous compensator without significantly affecting system
performance. It was believed that determination of a performance index
minimized by the continuous system would allow for performance invariant
design by computing the optimal control law for the sampleddata ver
sions of the continuous system and performance index. In Chapter IV
this scheme is discussed as a solution to a problem which will
undoubtedly occur with increasing frequency as digital controllers
become more commonplace. It was in the course of this investigation
that the potential of the inverse problem for regulator design of wider
latitude was discovered.
1.2 Survey of Previous Work
Perhaps the first attempts at relating optimal control theory
and classical design were early (c. 1950) investigations of socalled
"standard forms" (e.g., [G2]). The object was to tabulate forms of
closedloop system transfer functions which were optimal with respect
to a specified performance measure (for instance, ISE) and input.
This approach was rather restrictive and difficult to apply to problems
of interest. Its impact was nonetheless considerable and a recent
paper by Rugh [R11] indicates that the linear quadratic loss problem
has much in common with these early results.
Shortly after the linear quadratic loss problem became widely
known, the task of selecting performance indices which result in
optimal systems with desired characteristics was attacked primarily on
an experimental basis. The hope was that massive experience and some
insight into the mechanics of computing the optimal control laws would
lead to guidelines for the choice of a quadratic performance index.
One such procedure, developed by Tyler and Tuteur [Tl) consisted of
computing rootlocus plots as functions of weighting terms in the
performance index and choosing a suitable compromise.
Another study of optimal systems in classical terms was made
in 1965 by R. J. Leake [Ll]. Leake, in developing a computational
scheme for solutions to the linear quadratic loss problem, shows how
an estimate of optimal system bandwidth may be made without computing
the optimal control and comments on an intriguing geometric interpreta
tion (in the complex plane) of Kalman's criteria for optimality.
An important step in the analysis that follows will be the
determination of when two different quadratic performance indices
applied to the same plant will result in identical optimal control laws.
Kalman's paper on the inverse problem, by its nature, encounters the
problem theoretically but does not consider it in detail. A partial
practical solution to the equivalence problem appeared simultaneously
with Kalman's paper. Wonham and Johnson observed in their paper on
the linear quadratic problem with a bounded control [W2] that an arbi
trary weighting matrix in the performance index could be replaced by
a diagonal one which will result in the same optimal control. Their
result arises from the observation that when the system is expressed
in a canonical form (companion matrix form [R2]) the performance index
may be reduced by repeated integration by parts to a diagonal form;
they fail to realize, however, that the diagonalized version may no
longer possess a solution. Kalman and Englar later recognize [K3,
p. 304306] that, in the same canonical form as used by Wonham and
Johnson, certain terms may be discarded and equivalence maintained.
Beyond these early results and an occasional rediscovery of
them (e.g., Kreindler and Hedrick [K4]) the study of equivalent quad
ratic performance indices has remained dormant.
In summary:
1. A great deal of experimental work has been done in hopes of
relating optimal design to classical criteria and providing
intuition into a procedure for choosing performance indices.
Very little basic theoretical research has been invested
in this area.
2. The practical problems of determining the optimality of an
actual system and the computation of a performance index
have not been considered.
3. Some elementary equivalence relations have been developed,
but the difficulties arising from naively applying them
have been generally overlooked. Nor have the advantages of
one equivalent form over another been explored.
4. No real effort has been made to use the study of the inverse
problem as a foundation for linear regulator design.
This work begins with a review of the linear plant, quadratic cost
variational problem. The conditions for the solution to exist are dis
cussed along with techniques for computing the optimal control.
The inverse problem for a linear plant and quadratic performance
index is next considered. The conditions for optimality are developed,
the meaning of optimal control is studied in terms of classical criteria
and the equivalence of performance indices is resolved.
The fourth chapter develops computational procedures for the
determination of solutions to the inverse problem and goes on to con
sider practical techniques for the design of optimal regulators which
meet classical performance specifications.
CHAPTER II
THE LINEAR, QUADRATIC COST OPTIMIZATION PROBLEM
2.1 Introduction
As a first step toward the establishment of viable design
techniques based on the linear, quadratic cost optimal control problem,
the optimization problem itself must be reviewed in some detail. The
great power, latitude and computational elegance of this optimal control
formulation account in large part for its popularity as an object of
study. In the investigation that follows its limitations will also
have considerable impact.
This chapter will first define what is meant by the linear
plant, quadratic cost optimization problem. A method for the compu
tation of solutions which also provides a necessary and sufficient
test for existence will then be considered. The chapter closes with
a specific review of the particular subproblem which will be of
principal interest in the remainder of this work, the timeinvariant
optimal linear regulator.
2.2 The Linear System, Quadratic Cost Droblem
Consider an ndimensional linear system with state feedback
described by the state equation
dx
Fx+ Gu (1)
where u = K (t)x
and a companion performance index
t
J (xf Qx + uTRu)dt, Q = Q1 and R = RT (2)
t
where u is mdimensional; F, G, Q, and R are real, constant matrices;
and K is a real (possibly timevarying) matrix; all are of appropriate
dimensions.
It will be shown later (Appendix A) that several other
performance indices can be accommodated in the framework established
1
for this particular one. The 2 preceding the integral in (2) is for
algebraic simplicity in the present discussion and has no effect on
the result of the optimization process; it will occasionally be
deleted in the sequel without comment.
Taken together there are a variety of problems inferred by
(1) and (2). The most obvious is the optimal control problem [K2].
i.e., compute the control law K(t), if any, which minimizes the
performance index (2). A second, somewhat more obscure problem, is
the socalled "inverse problem" of optimal control theory [K1,E1].
The "inverse problem" seeks to determine what, if any, performance
index is minimized by a specific feedback control law. In the fol
lowing, both of these topics will play an important role in uncover
ing the nature of optimal systems.
2.3 The Optimal Control Problem
The EulerLagrange Equations
The necessary conditions for a control u to minimize the
performance index (2) is that the system equations (1) and the
celebrated EulerLagrange equations [Gl] be simultaneously satisfied,
i.e.,
m = (t) = O (3)
6H
u 0 (4)
where H is the Hamiltonian
I T I T T
H = x +x u Ru + X (Fx + Gu).
2 2
If R is nonsingular, equation (4) (sometimes called the "stationarity
condition" [B3]) determines a candidate for the optimal control, i.e.,
u(t) = R G X(t) (5)
The simultaneous solution of equations (1) and (3) with the substi
tution of (5) results in the twopoint boundary value problem:
d Fx GR1G x(t ) = x (given),
L]L \G T ] (6)
Q F T X(t) = .
Q rx]
The Riccati Equation
The optimal control problem for a linear system subject to
a quadratic performance index has been studied in great detail over
the past several years (e.g., [K2], [K33) and several eloquent solu
tions for the necessary conditions have been offered [Bl1. Most of
these solutions differ only in the approach taken to solve the two
point boundary value problem (6); for the purpose of this chapter the
socalled "sweep method" [Gl] is most illustrative and will be pre
sented by way of review.
Hypothesize a matrix P(t,t ) such that
X(t) = P(t,tf)x(t), (7)
then P(t,t ) would in effect provide a boundary value which is
"swept back" in time to the initial time and the initial value for
\ would simply be
X(to)= P(t ,tf )x
hence, the separation of the boundary values would be resolved.
By requiring that the EulerLagrange equations be satisfied, it may
be shown that
/dP T 1 T
+ PF + F P PGR G P + Qx = 0.
Wdt
Since x(t) is arbitrary, P(t,t ) must satisfy
dP T 1 T
dt PF F P + PGR G P Q, P(t ,t ) = (8)
which is a matrix version of the familiar Riccati equation [Dl].
1This problem is often referred to as the "optimal linear
regulator" problem in the literature; however, this designation will
be reserved for the infinite final time case here.
At first it may seem rather surprising that the solution to
a 2n order twopoint boundary value problem (6) can be obtained from
the solution of an nth order nonlinear equation (8), but recalling that
the scalar Riccati equation was solved in elementary differential
equation theory with the aid of a 2nd order linear differential equation
relates it to a familiar problem.
Finally, substitution of (7) into (5) leads to a candidate for
the optimal control (a control which satisfies the necessary condi
tions) in the form of a feedback control law, i.e.,
u = K (t)x, where K(t) = P(t,t )GR~1 (9)
and P is the solution to the Riccati equation (8).
Before proceeding any further, another necessary condition is
immediately available which has been previously ignored in this analysis.
From observation of the performance index (2), it is obvious that R
must be positive semidefinite; otherwise, a control could be hypoth
esized with sufficient high frequency content to have negligible effect
on the states of the system, hence allowing the integral (2) to become
unbounded (negatively). The requirement in the EulerLagrange equa
tions that R be nonsingular further constrains R to be positive definite.
This assumption is equivalent to the "strengthened LegendreClebsche"
necessary condition of the calculus of variations; therefore, the
requirement that R be positive definite is sometimes referred to by
that nomenclature.
In summary, a solution to the matrix Riccati equation (8)
specifies, in the form of equations (9), a control which satisfies
the necessary conditions (EulerLagrange equations) for optimality;
all that remains is to investigate sufficient conditions for the
optimality of (9). The existence of a solution to the Riccati equation
and an additional sufficient condition are related to the nonexistence
of conjugate points which are defined below.
The Conjugate Point Condition [Gl]
Definition 2.1
A point t1 is called a conjugate point to
tf if there exists a solution to the EulerLagrange
equations (6) with the boundary conditions
x(t ) = x(t ) = 0, where t S t1 < tf,
which is not zero everywhere on the interval.
Graphically, a conjugate point can be illustrated as in Figure 2.1
(cf. Figure 1 in Reference [B3]).
x(t)
2
tt
Figure 2.1 Conjugate Trajectory
Lemma 2.1 [B3]
When a conjugate point exists, both the
trivial solution (Path 1 in Figure 2.1) and the
conjugate trajectory (Path 2 in Figure 2.1)
result in a zero value for the integral
f (xTQx + uTRu)dt.
t
1
Proof
Consider the identically zero integral on the conjugate path
X (x Fx + GR G )dt = 0
t t
or SJ xTidt 1 X(Fx GR G X)dt = 0.
1 1
Applying integration by parts to the first term results in
Tx + (Fx GR1GT)dt = 0.
The first term is clearly zero and with the substitution for X from
the EulerLagrange equations (6) and for u from (5)
t
S(xTQx + uT Ru)dt = 0,
which completes the proof.
When a conjugate point does exist at tl, M(t1) must be nonzero
to be distinct from the identically zero solution to (6). If the system
is controllable, it must respond to the input (5) resulting from
X(t ) f 01 hence by continuity the conjugate path (A and x) must be non
zero for some finite interval within (t,t f). Then the conjugate path
cannot be a duplicate of the identically zero path with the addition
of a (nonzero) discontinuity; thus the presence of a conjugate point
represents more than merely misbehavior at a single point. The occur
rence of a conjugate point coincident with t, is'obviouslycdisconcerting
0
because it indicates that there are at least two candidates (that'
satisfy the necessary conditions) for an optimal control which lead
to entirely different trajectories at the same (zero) cost. When a
conjugate point occurs at t 7 to, the results are equally catastrophic
but this case does not lend itself as well to heuristic interpretation.
Theorem 2.1
For a linear, completely controllable system subject
to a quadratic performance index (2) with R positive
definite, a control u* satisfying the EulerLagrange
equations is globally optimal if and only if there exist
no conjugate points on the interval (to,t ).
A discussion of this theorem and its proof in a somewhat dif
ferent context can be found in Breakwell and Ho [B3]. The requirement
that conjugate points be nonexistent is generally referred to as the
"conjugate point condition" or occasionally in the classical calculus
of variations as the Jacobi condition [Gl].
Complete controllability requires that there exists an input,
u(t), which will drive the system
x = Fx + Gu(t) x(t ) = x
o o
from any initial state, x o, to the origin within an arbitrary time
interval; this can be shown to be equivalent to requiring that the
matrix [G,FG,...,F G] have full rank [K5]. Controllability is not
actually a severe criterion in a practical sense. A plant which is
not completely controllable can be transformed into two canonical
subsystems, one containing the completely controllable part and the
other subsystem containing the remainder of the plant dynamics [K5].
Hence, a given design problem can be thought of as two related designs
on the canonical subsystems and it is only necessary to be certain that
the noncontrollable part is stable and/or not reflected in the
performance index.
Now the solution to the linear, quadratic cost optimization
problem is essentially complete; it is merely necessary to solve the
Riccati equation for a candidate control law and to insure that the
conjugate point condition is satisfied. Although there appears to be
no convenient way to test for the presence of conjugate points, the
following theorem demonstrates that a test for the conjugate point
condition is actually implicit in the solution of the Riccati equation.
Theorem 2.2
The solution to the Riccati equation (8) for the
optimization problem specified by (1) (completely con
trollable) and (2) (with R positive definite) fails to
exist (becomes unbounded) at t t ot
if tl is a conjugate point to t .f
A formal proof can be found in Lee [L2] which is very much in
the spirit of the following heuristic justification.
From the definition of a conjugate point, x(tl) = 0; however,
X(t ) 1 0, since this could only come about in the trivial solution.
Recall that the solution to the Riccati equation was defined by (7) as
X(t) = P(t,t )x(t).
Then as t approaches tl, x approaches zero but X remains nonzero;
hence, j P(t) must correspondingly increase and finally become
unbounded at t = tl
Thus far the conjugate point condition has been presented in
a rather esoteric format with little physical interpretation. Again
consideration of the general case is difficult but the occurrence of
a conjugate point at the initial time (t ) has an interesting
interpretation. The absence of a conjugate point at t (in the linear
O
case) insures that no two system trajectories which satisfy the Euler
Lagrange equations (extremals) will ever intersect,2 that is, there will
never exist two distinct optimal trajectories ianating from a single
system state [B4].
The solution to the Riccati equation (8) also has a physical
interpretation which will be useful later.
Lemma 2.2
The value of the performance index for the
optimal control law is
o lT
J (t ,xo tf) xoP(totf)xo (10)
where P is the solution to the matrix Riccati
equation (8) which is bounded on (t ,tf), i.e.)
there exist: no conjugate points.
Proof
J = (xTQx + u Ru)dt
Substitution of the optimal control law,
1 T
u = R G Px,
results in
tf
J = xT(Q + PG 1GTP)x dt. (11)
2
Intersection here is taken so as to exclude the case of
tangential coincidence. In the general (nonlinear) case extremals
taken sufficiently close (i.e., neighboring trajectories) must
not cross.
Define
1 T
F = F GR GTP
the closedloop state matrix and k (tot) as its corresponding state
transition matrix, i.e.,
4k = F k and x(t) = kx o
Equation (11) can now be rewritten as
J = ft xTok(Q + PGR 1GP)4kx dt (12)
Substitution of the definition of Fk into the Riccati equation
results in
P= F P PFk PGRG P Q
and when employed in (12) to replace Q, (12) becomes
= T (PFk P dt
0
or
tf t
J = J o xkP4 xodt (xTPT p x + x+ TkT, x )dt.
2 o k ok o ok ko
o o
Application of integration by parts to the first integral above results
in terms which cancel the second integral, leaving
1 TT 1 T 1 T
J = TP kxo = oP(t o' t) X (t f)P(tft )x(t).
0
Since P(t,t ) = O0, this is the desired result.
This proof is considerably different from the standard one
(e.g., [Al, p. 251) in that it does not require the use of
HamiltonJacobi theory. This lemma not only provides a necessary link
in the solution of the problem considered in the next section but
provides a tie with dynamic programming, in that formulation equation (10)
is referred to as an "optimal return function" [B5].
Sufficient Conditions Existence
The conjugate point condition is not entirely satisfactory from
the standpoint of a working sufficiency condition. It is not known at
the onset whether or not the optimization problem has a solution; only
after the complete control law is computed is one assured that it was
not all for naught. What is required then, it would seem, is a suffi
cient condition somewhat more restrictive than the conjugate point
condition with the advantage that success of the optimization problem
is predetermined.
Theorem 2.3
The optimization problem of minimizing a quadratic
performance index
J = (x Qx + u Ru)dt, R = R positive definite (13)
o
subject to a completely controllable linear plant
dx
S= Fx + Gu (14)
dt
always has the global minimum
u(t) = K T (t)x(t) where K(t) = P(t,t )GR1
and = F P PF + PGR1GP Q, P(t ,t ) =0 (15)
for all finite (totf) if Q is positive semidefinite and
symmetric.
Proof
Suppose that Q is positive semidefinite and the Riccati equa
tion (15) diverges at t = t1 < tf. A solution for (15) must exist in
a sufficiently small neighborhood of tf further P(t) exists for all
t e(t +C,t ) e > 0; hence, by Lemma 2.2
J (x(t +e),t +e) = x (t +)P(t +t)x(t +) 2 0, (16)
for e < t t1
the inequality is due to the positive definiteness of the integrand
of J. As e approaches zero, some entry of P becomes unbounded. It can
be assumed without loss of generality that at least one diagonal element
of P becomes infinite; otherwise, some 2 x 2 principal ninor of P would
be negative, contradicting the positive semidefiniteness of P inferred
in (16). Let e. be a vector which is all zeros except for the ith ele
ment, a one, which corresponds to a diagonal term p.. of P which becomes
unbounded as C approaches zero; then,
J(et +e) = Pii (t1+, t'
Since J is the optimal performance index, a performance index
resulting from any arbitrary control, say u = 0, must be greater.
Then if tl + E and ei are chosen as the initial time and state of the
optimization problem, i.e., to = t + e
t
J0(e. .tl+E)=pi (t r.t95 fe eT(rt)Q5rtO)e. dT ( 7)
where (tt ) is the system state transition matrix and 0(tt )e.
o o i
is the resulting free trajectory. Clearly, the integral in (17)
remains bounded over the finite interval (t +'t ), while the left
hand side becomes unbounded as c 0. This contradicts the original
supposition of a conjugate point at tl and proves the theorem.
An interesting proof of this theorem using Lyapunov's second
method to determine the stability of the Riccati equation is given in
[K2].
Throughout the literature there appear sporadically references
to theorems such as 2.3 as necessary and sufficient conditions for a
solution of the optimization problem to exist (e.g., [D3,p. 557]);
this is patently false and numerous counterexamples exist [G3].
No necessary and sufficient condition for the nonexistence of conjugate
points (other than integrating the Riccati equation) is presently known,
although many researchers are actively pursuing these conditions and
there is reason to believe they will be found in the near future [B6].
2.4 The Optimal Linear Regulator Infinite Final Time
Existence and Stability
The EulerLagrange equations and the conjugate point condition
apply as well to the case when the final time is no longer finite.
However, as before, the conjugate point condition is not satisfactory
as an existence test; the following theorem extends Theorem 2.3 to
the infinite time case.
Theorem 2.4
If a linear plant (14) is completely controllable
and a companion quadratic performance index,
J = S (xQx + u Ru)dt, R = R positive definite (18)
T
has Q = Q positive semidefinite and if P(t,t ) is a
solution to the matrix Riccati equation (15) with
P(t ,t ) = 0, then
lim P(t,t ) = P(t) (19)
tf
exists for all t and is a solution of (15).
Proof [K2]
First, it must be shown that the limit (19) exists for all t.
Since the plant is completely controllable for every xo, there exists
a control u'(t) which transfers x to 0 by some t t l. Set u(t) = 0
for t > tl. Then
J(txo't t)=x TP(t 't )x J(t ,x t J(t ,x
1 o o 1 0 o t1) t 0 0 )
u u
is bounded for all t > t The optimal costs are also nondecreasing
1 o
as t Suppose that this were not the case, that is, suppose
Jo(t ,x 't ) > JC(t ,x ot ), for t2 > t
1 0 o 1 2 o o' 2 2 11
where u (t) is the optimal control corresponding to J1 and u (t) is
the optimal control resulting in J. Then by the positive definiteness
of the integrand of (18), use of control u2 will result in a lower cost
at time t than ul; thus contradicting the optimality of ul and
demonstrating the assertion that the optimal costs are monotonic.
The limit therefore exists for all t by the wellknown result that all
bounded monotonic sequences possess a limit.
Now it is necessary to demonstrate that (19) is a solution to
the Riccati equation. Define P(t,t f;A) as a solution of (15) with
the boundary value P(t ,t ) = A. Then, using the continuity of solu
tions of (15) with respect to its boundary values,
P(t) = lim P(t,t 2;0) = lim P(t,t IP(t t 2;0))
2 2
= P(t,t ;lim) = 't;O)) = P(t,t ;P(t1))
t 2
and P(t) is a solution of (15) for all t.
The price that is paid for guaranteeing the existence of a
solution over all time (in addition to requiring Q to be positive
semidefinite) is the complete controllability of the plant.
Corollary 2.1
The solution P(t) to the matrix Riccati equation (15)
for the optimal linear regulator problem of Theorem 2.4 is
constant and the unique positive definite solution of
PF + F PGR10P + Q = 0, (20)
the steadystate (algebraic) Riccati equation.
The stationarity of P follows from the arbitrariness of the
choice of the initial time to, since all initial times must result in
the same value of the optimal performance index. Equation (20) ensues
when the effect on P of the irrelevance of t is considered; the choice
of the positive definite solution is dictated by the positive definite
ness of (18) and uniqueness is guaranteed by the conjugate point con
dition.
It is now clear that the optimal linear regulator is the optimal
control theory analog of the classical design using state feedback.
As pointed out by Kalman [K2], the literature contains many
references where the stability of an optimal system is tacitly assumed.
For example, Letov [Zl, p. 378] equates (without proof) optimal systems
and those which are stable in the sense of Lyapunov; although probably
true for the class of optimal systems which are also stable, it is not
in general. This can be easily demonstrated with a simple example.
Let a scalar input plant,
x = Fx + gu,
have one or more eigenvalues with positive real parts and let the per
formance index to be minimized be,
t
J = S u2dt,
0
that is, Q = 0 and R = 1. The problem so defined clearly has a solu
tion (Theorem 2.3), which is
u(t) = 0.
Stability was not a result of optimization in this case primarily
because the states are not reflected in the cost (performance index);
had the plant been stable, however, the optimal system would have been
also. The final theorem in this section formalizes this observation
into a general result.
Theorem 2.5 [K11]
An optimal linear regulator problem satisfying the
conditions of Theorem 2.3 results in an asymptotically
stable control law if
i) the pair [F,H], where Q = ITI is completely
observable, i.e., [H,F H,... (F )n1H] has
full rank;
and only if
ii) the linear subspace
X, = (x 0o 1) He x 11J= 0
Ft
of the state space is null (i) or e x, x E X
is an asymptotically stable response in the
sense of Lyapunov.
Proof
i) By the assumption of observability and the positive semi
definiteness of Q, the integrand of the performance index must be posi
tive along any nonzero trajectory of the plant. Then
Jo(t,x(t)) = xT (t)x(t) > 0, x(t) 9 0 (21)
along optimal trajectories. Differentiation of the performance index,
J(t,x(t)) = (xT Qx + uTRu)dT,
results in J(t,x(t)) = (x Qx + u Ru)< 0, x(t) 1 0, which is
negative along all nontrivial trajectories. Then (21) is a Lyapunov
function [L2] and the system is asymptotically stable by Lyapunov's
Second Stability Theorem [L3, p. 37].
ii) When certain of the states are not observable in the cost,
Lyapunov's method is effectively applied only to a subsystem (i.e., the
states observed in the cost) and the more general result follows when
the stability of the remainder of the system is considered.
The requirements for stability placed on Q by Theorem 2.5 are
not unduly restrictive in that they reflect good engineering judgment;
that is, if a state has significant effect on system performance it
should influence the design process. Some authors choose to include
stability tacitly in the definition of the performance index by con
sidering only the case where Q is positive definite (e.g., [SI]).
Stability is assured since a positive definite Q must have a non
singular factor and condition (i) of Theorem 2.5 is clearly satisfied.
2.5 Summary
Over the past ten years the optimal linear regulator problem
has become one of the most widely studied optimal control problems.
Perhaps the best way to summarize this unique problem is to review the
qualities which set it apart.
1. The solution to the problem is always approached in the
same fashion regardless of the specific system or weight
ing matrices employed and in contrast to the majority of
optimal control problems, the solution is explicit.
2. In a restricted, but very large, set of weighting matrices
(i.e., Q positive semidefinite) the solution is guaranteed
to exist.
3. The solution is always in the form of a linear, constant
feedback control law, as opposed to the general optimal
control problem where a feedback formulation is not directly
26
obtainable. With another minor concession to generality
(Theorem 2.5) asymptotic stability will be assured from
the onset.
4. Numerical computation of the control laws is both straight
forward and relatively inexpensive (in terms of computing
time)[B7]; in addition, a multitude of numerical schemes
are available [Bl].
Under suitable assumptions of continuity and corresponding
definitions of controllability and observability [D2] most of these
remarks are also applicable to linear timevarying systems and non
constant weighting matrices [K2].
CHAPTER III
THE INVERSE OPTIMAL LINEAR REGULATOR PROBLEM
3.1 Introduction
Optimality in itself is not necessarily a desirable character
istic for a system to possess. For instance, suppose it is desired
to minimize the sensitivity of an existing system's response to com
ponent variations by use of an appropriate controller. A system
design which obviously has a minimal sensitivity is one that does
nothing whatever; then one candidate for an optimal controller would
be one that turns the system off. In this case, the optimal design would
probably not be satisfactory because it was not implicit in the procedure that
the resulting system should perform in some acceptable manner in addi
tion to minimizing the performance index.
The lesson is clear, optimality may be a frivolous notion
unless its ramifications are thoroughly understood in the context of
total system behavior. In the present chapter the implications of
optimality are considered for a single input linear plant subject to
a quadratic performance index to set the stage for its use as a viable
design tool.
The system to be considered in this chapter can be described
by the state equation,
dx
=t Fx + gu, (1)
and occasionally it will require an associated feedback control law,
u = k x, (2)
where F is a constant n x n matrix and g,k are constant nvectors.
In conjunction with this plant, a quadratic performance index,
J = (xQx + ru )dt, (3)
will be studied. The control weighting factor, r, will be taken with
out loss of generality as unity throughout the remainder of this work.
3.2 When Is a Linear Control System Optimal?
The original impetus to investigate the "inverse optimal control
problem" is generally considered to be a 1964 paper by R. E. Kalman,
bearing the same title as this section [K1], although it has a much
older history in the Calculus of Variations. In that paper, Kalman
considers the theoretical criterion for a linear system to be optimal
with respect to a restricted class of quadratic performance indices.
His principal result is reviewed next.
Theorem 3.1 [Kl]
Consider a completely controllable linear plant
(1) with a stable completely observable control law k
(2). Then k is an optimal control law with respect
to a quadratic performance index (3), with Q positive
semidefinite, if and only if,
11 + k (jw)gj2 2 1 (4)
for all real wu, where (s) = (sl F)1
Proof
i) Necessity: If k is an optimal control law, then by
Theorems 2.3 and 2.4
k = Pg (5)
where P is a solution to the steadystate Riccati equation,
PF + F P Pgg P + Q = 0. (6)
Substituting (5) into (6) permits it to be rewritten as
PF + FP kk + Q = 0. (7)
Adding and subtracting sP to (7) results in
P(sI F) (sI F )P kk + Q = 0.
Premultiplying and postmultiplying the above by g T (s) and (s)g,
respectively (where $(s) is as defined in equation (4)) and substitut
ing (5) where appropriate, leads to
gT T(s)k + k T(s)g + gT T T(s)kkT(s)g = g TTs)Q(s)g
which may be factored as
[1 + g (s)kll + k T(s)g] = 1 + gTT (s)Qs(s)g. (8)
Noting the semidefiniteness of Q proves the necessary condition.
ii) Sufficiency: Proof of the sufficient part of the theorem
is not particularly enlightening and requires concepts that are yet
to appear, hence it will be deferred to the appendix.
3.3 Implications of Optimality
It is somewhat surprising that the condition for optimality (4)
should be a frequency domain criteria in that all of the previous
analysis of the problem (Chapter II) was carried out in the time domain.
The study of optimal systems in the frequency domain may be responsible
for the current interest shown by Kalman [KG1 and others in realization
tl.ciry from the point of view of invariants and thus promises to have
far greater i influence in years to come.
Examination of the optimality criterion (4) reveals that the
quantity whose modulus is taken on the lefthand side of the equation,
i.e.,
f (ju) = 1 + k T(ju)g
is the "return difference" of classical feedback theory [B81]. It can
be interpreted as the difference between a unity input (in the frequency
domain) to the system and what would consequently be returned as feed
back. Graphically, this is illustrated in Figure 3.1 as the difference
between the input at node a and the feedback at node b.
Plant
a a x
0 ; (j )g
1I
b T
_T
0 k]
Feedback
Gains
fk = a b
Figure 3.1 Return Difference
The requirement of (4) that the modulus of the return differ
ence be greater than unity is the celebrated result of classical
feedback theory that the sensitivity of system response to variations
in plant parameters is reduced by the addition of feedback [B8].
Further analysis of inequality (4) reveals that it requires that the
Nyquist locus avoid a circle of unit radius about the point (1,0).
Figure 3.2 pictures a Nyquist plot of a hypothetical optimal system
constructed from a plant with two eigenvalues with positive real parts.
I
A
Figure 3.2 Nyquist Plot of Optimal System
Two additional observations may be immediately made from
Figure 3.2. First, the phase margin of an optimal system must be at
least 600, since the closest points (in phase) to the negative real
axis that the locus can cross the unity magnitude contour (points A
and A' on Figure 3.2) are displaced 600 from the axis. Second, the
system will clearly remain stable regardless of how the feedback gain
is increased. Then if gain margin is defined as that factor by which
feedback gain may be increased before system instability occurs [E2] ,
the gain margin of an optimal system may be said to be infinite.
However, if gain margin is defined as the reciprocal of the magnitude
of the loop gain at phase crossover, i.e.,
1 T
GM 1 i where Arg fk Q(jw )g] = 180,
k $(ju )g c
then an optimal system has a gain margin of at least or approximately
6 db. Kuo indicates that the two definitions given are equivalent
[K7, p. 398], but consideration of the example of Figure 3.2 reveals
that this is not the case when the plant is nonminimum phase.
A third definition, and the one that should be understood in
any references to gain margin in the sequel, is the following [T2]:
Let g. be the factor by which the feedback gain may be
increased until instability occurs and g be the factor
by which the gain may be decreased and stability main
tained, then
GI = smaller of fgi,l/g d.
This definition is more reasonable in that it reflects the actual
gain disturbance required for instability and is easily determined
from the Nyquist diagram. The gain margin for an optimal regulator,
in the sense of Theorem 3.1, then is at least 6 db.
It is instructive at this point to compare these figures of
gain and phase margin with those required in practice. Jones, Moore
and Tecosky in Truxal's classic handbook [T2, PP. 1914] recommend
phase margins of 40' to 600 and gain margins of at least 10 db for
applications typical of chemical process control. Generalities of
this sort are of course subject to severe criticism but nonetheless
it can be concluded that minimum optimal regulator specifications
compare favorably with those desired of classical designs.
3.4 Companion Matrix Canonical Form
It is well known that for any nonderogatory matrix, F, there exists
a similarity transformation, T, such that
1
F = TFT
is the companion matrix [Ml] of F. A companion matrix being a matrix
of the form:
0 1 0 ... O
0 0 1 ... O
F = (9)
al1 a2 a3'.*an
where
n n k1
cp(s) = s + E aks = det (sl F>.
k=l
That is, the last row of the companion matrix of F contains the nega
tive of the normalized coefficients of the characteristic polynomial
of F (and P), with the remainder of the matrix null save 'a superdiag
onal of ones. Some authors choose to refer to the transpose of the
matrix defined above as the companion matrix [G4] In the following
either definitionwill suffice; however, (9) will be assumed for con
sistency.
When F assumes the role of the state matrix in a completely
controllable linear system (1) the similarity transformation, T, which
transforms F to its companion matrix, places the system as a whole in
a canonical form. Specifically, if z = Tx represents the change of
basis described above and the linear completely controllable system is
x l'x + goL
T
y = x,
then FK ]]
z = Fz + gu
F = TFT the companion matrix of F
0
0
S= Tg= and H = (T1) T1.
0
1
Complete controllability allows the specified form of g to be chosen.
Since there is no loss of generality, controllable systems will be
assumed in this coordinate system and the "hat" notation will be omitted.
The problem of actually computing the required transformation
has lately occupied a good deal of the literature, e.g., [J1,R2,R3].
Computation of the transformation is conceptually not very difficult
and somp straightforward numerical solutions have been formulated
resp. R3].
The companion matrix canonical form is often referred to as the
phasevariable canonical form [S2]; this cognomen alludes to the prop
erty that each state is the derivative of the preceding state.
A second property that will find application is the fact that g has
only one nonzero element, hence, the control directly affects only
the last state (highest derivative).
3.5 Characterization of the Equivalence
Class of Q's
Employment of the coordinate system reviewed in the last section
permits additional insight to be gleaned from Theorem 3.1. First it
will be necessary to exploit a characteristic of the companion matrix
canonical form.
Lemma 3.1
For a completely controllable linear system in the
companion matrix canonical form (1)
1
S(s) = p(s)i(s)g = s2
sn1
where c(s) = (sI F)1
and tp(s) = det (sI F).
Proof
To prove the theorem it is sufficient to show that
(sIF)cp(s)(s)g = (sIF)S(s) = ((s)g.
With F in companion matrix form
s 1 0 ... 0 1 0
0 s 1 ... 0 s 0
(sIF)S(s) = =
a a a ... a +s sn1 sI s a. si1
1 2 3 ni
which is clearly Cp(s)g and the theorem is proven.
This lemma allows the simplification of equation (8) used in
the proof of Theorem 3.1 to
kTS(ju) 2 ST(jL)QSju)
(jw) = + (10)
+pju ____ 2= Jp(jwfl2
Multiplying by p(ju) 2 results in
Jcp(jw) + kSC(jwu)2 = Jp(jw)2 + S (ju)QS(ju).
With the observation that the expression inside the absolute value marks
is the closedloop characteristic polynomial (which will be denoted
p (s)) permits further simplification to
c(Ik jW)2 Icp(jw)2 = sT(jw)QS(jw). (11)
This relation (11) defines a polynomial by which the open and
closedloop characteristic equations of an optimal system must be
related to the state weighting matrix of a corresponding performance
index. This polynomial will appear frequently; consequently, it will
be convenient to refer to it with the following notation:
Y(u) = pPk (j)2 IP(jw)12 = ST(jw)QS(jw), (12)
or occasionally as
Y(Q;w) = S (ju)QS(ju)
to emphasize the functional relationship of Q.
Recalling that characteristic polynomials are invariant under
similarity transformation and the assumption of positive semidefiniteness
for Q leads to a useful corollary of Theorem 3.1.
Corollary 3.1
A completely controllable scalar linear system
(1) with a completely observable stable control law k
is optimal with respect to a quadratic performance index
(2) with Q positive semidefinite and completely observ
able if and only if
S(j 2 cp(juj)2 0 for all real w,
where cp(s) = det (sI F) and p k(s) = det (sI F+gk ),
the open and closedloop characteristic polynomials,
respectively.1
The requirement that the feedback control be completely observ
able again appears without apparent justification. Suppose that k were
not completely observable, then the pair of polynomials which comprise
the expression
T K(jw)
k (j(j)g 
will have a common factor [K5]. This can be thought of, in the clas
sical control sense, as having a zero on top of a pole which prevents
the response of the pole from being observed at the output. Let the
common factor be y(jw) and denote the polynomials with y factored with
a prime, then equation (10) becomes
2
1 + K (ju)y(ju) 1+ ST (jw)QS(jw)
cP (jw)y(jw) ICp'(juw)2ly(u)j)2
or
Jcp'(ju) + K'(jw) J2y(jmu)2 = lcp(jw)2y(jW)12+ST(ju)QS(ju).
The implication is that S (jw)QS(ju) must contain the common factor
IY(j))12 as well. Since Q is positive semidefinite, it can be factored
as Q = HHT; then by the preceding statement the vector of polynomials
formed from
Through a minor abuse of the language, the terms "completely
observable Q" and observabilityy from the cost" should be taken to
mean that the pair [F H] are completely observable when H is any factor
of Q such that Q = HH Similarly, "completely observable k" means
that the pair [F,k] are completely observable.
IH T(jU)g (13)
may have the common factor y(jw) and the Q may not be completely observ
able. The reason for the uncertainty in the observability of Q is the
ambiguity in the location of the zeros of the factor of (13) correspond
ing to y(jCu)12. Depending on the choice of H, (13) can have zeros in
the right or left halfplane which may or may not eclipse a pole of the
system. If k is completely observable, however, there will be no common
factor regardless of how H is chosen and any Q which satisfies (13) will
be completely observable. This is best illustrated with an example.
Example 3.1
Consider the second order plant,
11 2 1]
with the feedback law, k = 1
The feedback is clearly not observable and in fact cancels one of the
plant poles at 1 as shown in Figure 3.3.
Imt
x plant poles: 1, 1
0 closedloop poles: 1, 2
2 1 Re
Figure 3.3 System Pole Locations
The control law is nonetheless optimal and three performance indices
which are minimized by it have Q's,2
1 3 3 h 1
2 3 3 h2 Ll
and 3
Q3 0
The first is not observable; (13) has a zero at 1 which
coincides with a plant pole. The second is identical to Q1 except
that the zero of (13) is reflected about the axis (at + 1) and is
consequently observable. The last is nonsingular and avoids the
difficulty entirely. The important point to note is that the control
is optimal for all of these Q's. Complete observability from the cost
is not required for optimality; it is only necessary that any plant
poles which are unobservable appear in the closedloop system as well.
That is, optimal feedback cannot move system poles which are unobserv
able in the cost.
The requirement that k be completely observable is, in a sense,
an "inverse" sufficient condition to the observability of Q; if the
condition is met, then any Q for which the system is optimal must be
completely observable. If, however, k is not completely observable but
Y is nonnegative, there may still exist one or more positive semidefi
nite Q's which are completely observable for which the system is optimal.
2
These statements will be justified later.
Another corollary to Theorem 3. 1 results from consideration of the
preceding remarks.
Corollary 3.2
A completely controllable scalar linear system with
a stable feedback control law is optimal with respect to
a quadratic performance index with Q positive semidefinite if
p (jum)2 _ p(ju) 2 2 0 for all real u,
and
a) C(s) and c(pk(s) are relatively prime which
further insures that any Q for which the
control law is optimal is completely observable,
or
b) the plant is stable.
Physically, the (a) condition can be interpreted as meaning that
the aforementioned observability difficulties do not arise if the feed
back moves all the plant poles. This is the case because the common
factor in the discussion preceding the corollary does not exist. The
second (b) condition removes the ambiguity by restricting the plant
poles to the lefthalf complex plane (including the imaginary axis)
and no difficulties of the sort discussed will occur. Some necessary
and sufficient conditions similar to those of Corollary 3.2 willbe
considered coincidentally with later results.
In order to provide greater insight into the optimality condi
tion in a strictly mathematical sense, it will be convenient to consider
the requirements for Q implied by equation (11). In the same fashion
that the two preceding corollaries dealt with the lefthand side of the
equation and its relation to the system, it is desirable to consider
what the righthand side, i.e.,
Y(Q;w) = ST (jw)QS(j) ,
portends for the corresponding performance index.
Before proceeding, two wellknown lemmas on factorization will
be stated; the first concerns a factorization of real, even polynomials
and the second a factorization of positive semidefinite matrices.
Lemma 3.2
If and only if F(w) is a real, even polynomial and
r(w) 0 for all real w,
then there exists a unique "spectral" factor y(s) such
that
r(w) = y(jw)y(ju))
and y(s) has only zeros with nonpositive real parts.
Many proofs of this lemma are recorded in the literature; for
example, see Brockett [B2, p. 173 ff].
Lemma 3.3
A real, symmetric matrix Q may be factored as
Q = HH1 ,
where H is a real matrix of rank (H) = rank (Q) if
and only if Q is positive semidefinite.
This is a fundamental theorem which is encountered in the study
of quadratic forms; see [A2, p. 139].
The groundwork has now been laid to present a theorem which
concisely places the systemtheoretic results of Theorem 3.1 into a
mathematical framework.
Theorem 3.2
For every real, even polynomial r(w) of order
2(n1), there exists a real, symmetric, positive semi
definite, nth order matrix Q such that
Y(Q;x) = r(u),
(Y as defined in (12)) if and only if,
r((u) 0 for all real wu.
Proof
i) Sufficiency: If F(u) is real, even and nonnegative,
Lemma 3.2 assures that a real factor y(s) exists such that
y(jw)y(jwf)= r(w). (14)
Let h be a real vector composed of the coefficients of y(s) ordered
with the constant term first and the coefficient of the (nl)st term
last. Then, clearly,
y(ju) = h S(jm) and y(ju) = ST (j)h,
where S is as defined in equation (12) of Lemma 3.2. The product (14)
can be identified as
Y(hhT ;w) = S (ju)hhTS(jW) = y(jw)y(jW) = F(w)
and hh provides a positive semidefinite Q which satisfies the suffi
ciency part of the theorem.
ii) Necessity: It must be shown that any positive semidefi
nite Q results in a Y(Q;w) that,is a real, even, nonnegative poly
nomial. Since Q is positive semidefinite, the Hermitian form,
F(W) = Y(Q;i) = ST (jw)QS(j)),
is nonnegative and is clearly real and even. The theorem is proved.
Theorem 3.2 relates the optimality condition of Corollary 3.1 to
a realizability condition for positive semidefinite Q's. Corollary 3.1
states that if the polynomial
y(Wu) = 1k(jiW)I2 y(j)12
is nonnegative and the feedback control law is stable, the closedloop
system is optimal. Theorem 3.2 states that if an arbitrary real even
polynomial is nonnegative, then there exists a positive semidefinite
matrix Q that generates the polynomial by equation (12). If the system
is in companion matrix canonical form, the polynomial studied for
optimality and the polynomial of Theorem 3.2 are one and the same as
revealed by equation (11). Together they constitute all that is
required to determine the optimality of a given closedloop system
configuration with respect to a positive semidefinite Q and to
construct a corresponding performance index.
The proof of Theorem 3.2 provides a "recipe" for computing
at least one performance index which is minimized by a given canonical
optimal system; that is, by spectral factorization. This realization
also prevents the performance index from obscuring unstable system
poles and thus avoids the observability problem. At this point, it
would be instructive to return to Example 3.1 and verify system
optimality and the choices of Q.
Example 3.2
Consider the second order plant and feedback law of Example 3.1,
F = ,[ k = ] .
1 2 1L
The open and closedloop characteristic polynomials are
(p(s) = s2 + 2s + 1,
p (s) = s2 + 3s + 2,
and
Y(w) = Yk (jW) 2 (j) 12 = 3ju2 + 3,
is clearly nonnegative. Then since the plant is stable, the opti
mality of the feedback law follows directly from Corollary 3.2b.
Theorem 3.2 insures that a positive semidefinite Q can be found which
will form a performance index minimized by the control law,and the
proof hints at how one such Q can be constructed. Following the proof
the spectral factor of Y is computed, i.e.,
I(s) = /1(s+1) and
Y = V(jw)4(jw).
Now a vector h is constructed composed of the coefficients of 4(s)
ordered with the constant term first,
h = /3f and Q = hhT =[3
which is the choice for Q1 in Example 3.1.
In order to avoid an increasing awkwardness in notation, the
following definition is required.
Definition 3.1
A state weighting matrix Q2 is said to be equivalent
to another weighting matrix Q1 if for a given linear system (1)
the quadratic performance indices (3) formed from Q1 and Q2
are minimized by the same control law. This equivalence
will be denoted by a tilde, i.e.,
and the set of all matrices which are equivalent for a
given system and optimal control law will be referred to as
the "equivalence class of Q's" for that optimal system.
Note that this equivalence relation is correctly defined in the
strictest sense [A3], that is, it is:
i) reflective Q Q
ii) symmetric if Q, ~ Q2, then Q2 ~ Q
iii) transitive if Q1 ~ 2 and Q1 Q then Q1 Q3
The definition fails to draw a distinction between symmetric and
nonsymmetric matrices. Consistent with the remainder of this work,
symmetric weighting matrices will be tacitly assumed. There is no loss
of generality in the assumption of Q symmetric; if A is a nonsymmetric
matrix, then it is easy to see that it can be replaced in a quadratic
1 T
form by the symmetric matrix (A + A ) without altering the value of
the form fill].
The next theorem, which may be considered a central result of
this section, prescribes how all quadratic performance indices minimized
by a given system are related.
Theorem 3.3
For a scalar system, x = Fx + gu, in companion matrix
canonical form with a stable optimal control law k, a state
weighting matrix Q2 is equivalent to a weighting matrix Q1
which forms a quadratic performance index minimized by k
if and only if
a) Y(Q2;w) = Y(Q1;w),
b) the linear subspace of the state space,
T
Ft
is null or e x for x e X is an asymptotically stable
response in the sense of yapunov, and
c) the optimization problem for Q2 and the given system
has no conjugate points on t e [0,).
Proof
i) Sufficiency: By Theorem 2.1 and (c) above, the minimiza
tion of the performance index with Q2 results in a unique optimal control
law which by Theorem 2.5 and (b) is also stable; denote this control law
as k2. Suppose that k 2 k1, where k1 is the optimal control law
corresponding to Q 1. Then, since the coefficients of the closedloop
characteristic polynomials are the sum of the plant characteristic
polynomial coefficients and the entries of k1 (or k ),
Ck2 (s) k 1 (s),
2 1
and by the stability of k, and k2
k 2(ju) )2 p kkl(jO) 2
(i.e., both characteristic equations have zeros in the left halfplane).
This last inequality with the definition of Y (12) contradicts part (a)
of the hypothesis and sufficiency is demonstrated.
ii) Necessity: It must now be shown that if Q2 ~ 1, then
(a), (b), and (c) follow. Parts (b) and (c) are implicit in that they
are necessary and sufficient conditions for stability and existence,
respectively, of the optimization problem with Q 2. The first part
results from the equality of k, and k2 and the definition of Y (12).
This theorem could easily have been rewritten to include the
case where the resulting control law was not stable; however, this
complication would have no usefulness and would obscure insight
into the mechanism of equivalent Q's. Many of the results to follow
can be extended to include unstable optimal control laws but any
apparent increase in relevance is purely artificial. Hence the praxis
of considering control laws to be only stable will be continued
throughout the sequel.
As is often the case, in this theorem practicality is the price
of generality. In the discussion of Theorem 2.3 it was observed that
the conjugate point condition was not very satisfactory as a sufficiency
condition for the optimization. The same remarks apply here: no con
jug points will exist for Q2 positive semidefinite; then the conju
gatt point condition need only be examined where Q2 is not positive
semidefinite. The condition that Y be identical for equivalent Q's,
part (a), can also be simplified.
The next lemma provides the last link in reformulating the
results of Theorem 3.3 into a workable equivalence relation for
performance indices.
Lemma 3.4
For an arbitrary real, symmetric, nth order matrix Q,
the real even polynomial resulting from the quadratic form
Y(Q;W) = S (jm)QS(ju) (15)
where
s
S(s) =
.n1
can be determined from the relation
n ii
Y(Q;w) = S [q.. + 2 S (1) q i+] 1 (16)
i=l j=1
(q. = 0 for i < 1 or j > n).
Proof
Rewrite the vector S as the sum of two orthogonal vectors,
1 0
0 s
2
S(s) = a(s) + b(s) a(s) = s b(s) = 0
0 s
4
s 0
and clearly
S(s) = a(s) b(s)
since a is an even function of s and b is odd. Substitution of this
relation for S into (15) leads to
Y(Q;s) = (a(s) b(s)) Q(a(s) + b(s))
aTha bT T T
= a Qa b Qb + (a Qb b Qa).
By the symmetry of Q the bilinear terms in parentheses total zero.
Since a(s) and b(s) have odd or even elements respectively zero,
Y(Q;s) contains no terms with q. with i + j odd. The nonzero
offdiagonal terms resulting from a Qa have i and j both even, while
the nonzero offdiagonal terms from b Qb have i and j both odd.
Hence, Y may be rewritten as
a i+l 2(i1) i+j2 i+j2
Y(Q;s) = (1)i+q..s2 + 2 q..s 2 q i+j2
i=1 i,j odd i,j even
which reduces easily to (16).
This lemma is important in its own right in that it provides
an algorithm for computation of 'Y for a given Q without the necessity
of evaluating the quadratic form. Its primary value however is that
it allows the equivalence condition of Theorem 3.3 to be redefined from
the invariance of a polynomial (T) to specific algebraic constraints on
the entries of the matrices in question.
Corollary 3.3
A weighting matrix Q is equivalent to a matrix Q1 for
the system of Theorem 3.3 if and only if the entries
of the matrices are related so that the quantities
p. = ii 2q i + 2q . (17)
i = 1,2,3...,n
(q. = 0 for i < 1 or j > n)
are equal for q.. taken to be elements Q or Q and
parts (b) and (c of Theorem 3.3 are satisfied.
Proof
It must be shown that the relation (17) given in the corollary
is equivalent to part (a) of Theorem 3.3, that is, if it is satisfied,
the Y polynomials for Q1 and Q2 are identical. This is accomplished
by demonstrating that the coefficients of the respective Y polynomials
are coincident. The proof follows directly from determination of the
coefficients of Y from (16) which are then related to the p. of (17).
With additional study of equations (16) and (17) a somewhat
startling phenomenon comes to light. The only entries of a given
weighting matrix Q which influence the polynomial Y(Q;w) are of the form
qkk and qkP,k+V.
which excludes any element q.i where i+j is odd. This indicates
that approximately half of the elements of an arbitrary weighting matrix
are irrelevant (assuming that parts b and c of Theorem 3.3 are still
satisfied) with respect to the optimal control law. Kalman and Englar
(K3] noted from the structure of the companion matrix canonical form
that the q.ij elements where i+j is odd are "irrelevant" without identi
fying the underlying structural relation (17) for equivalent Q's.
Their suggestion that the "irrelevant" terms in a given weighting matrix
be nulled at the onset of the optimization procedure, in order to simplify
computation, is potentially hazardous. There is the possibility that
nulling these elements will alter the observability qualities of the
original matrix to the extent that an unstable plant pole is obscured.
Fortunately, this occurrence appears to be extremely unlikely; the
author was unable to construct a matrix which behaved in this fashion
after a very exhaustive search. It seems that an unobservable matrix
will remain so after the i+jodd terms have been struck and conversely
an observable matrix will still be observable after these terms have
been removed. A very convincing heuristic argument can be made in
support of this observation; first, however, it will be convenient to
state two lemmas.
Lemma 3.5 (Gerschgorin, 1931 [G5])
All the eigenvalues of a square matrix A = (a. .) lie in
the union of circular regions,
n
laii zj E l a I i = 1,2,3,...,n,
j=1
j/i
of the complex plane.
This is, of course, the touted Gerschgorin Circle Theorem
which has found wide application in the numerical eigenvalue problem.
A laconic proof of this famous theorem can be found in Cullen [C2,
p. 197]. Of principal immediate interest, however, will be a second
lemma which, although a corollary of Gerschgorin's Theorem, has found
prominence as a separate result.
Lemma 3.6
Diagonally dominant matrices are nonsingular. A square
matrix A is said to be diagonally dominant if
n
i a. > S a..j for i = 1,2,3,...,n (18)
j=1l
jgi
where a.. is the (i,j)th element of A; that is, if
the diagonal entries are larger in modulus than the sum
of the magnitudes of the remaining constituents of their
respective rows.
The justification for this lemma follows easily from Lemma 3.5.
If a matrix is diagonally dominant, the region of permitted eigenvalue
locations excludes the origin and the matrix is consequently nonsingular.
Consider Figure 3.4 which is a schematic drawing of a symmetric
matrix with the i + j odd terms removed and the X's representing the
remaining entries.
Q = 0 X 0 X 0 ...
X 0 X 0 X ...
Figure 3.4 Sparse Equivalent Matrix
It is clear from observation of the figure that removing the indicated
entries tends to increase the dominance of the diagonal and in a sense
makes it "more nonsingular"; thus it would be reasonable to expect an
aggrandizement of observability rather than a deterioration.
A possible second concern is that discarding elements in the
manner described may destroy the positive semidefiniteness of the
weighting matrix. A second application of Gerschgorin's Theorem,
this time in the form of Lemma 3.5, reveals that the likelihood of
degrading the positive semidefiniteness of a Q matrix by removing
the irrelevant terms is extremely small.
If the matrix is positive semidefinite, the union of circles
which form the permissible regions for eigenvalues must include parts
of the righthalf complex plane (including, perhaps, the origin); and
by a wellknown result of matrix theory [H11], the diagonal elements of
a positive semidefinite matrix, hence the centers of these circles,
must be nonnegative. Then as the offdiagonal terms are removed, as in
Figure 3.4, it is apparent from relation (18) that the radii of the
circles which may contain eigenvalues are reduced and the eigenvalues
will consequently be restricted to fall in a region that is, if anything,
more positive. The only case which is not resolved by this argument is
the one where the permissible region of the original matrix includes
part of the left halfplane and reduction of the radii does not retrieve
the region wholly into the right halfplane; thus admitting the possi
bility of a negative eigenvalue in the reduced matrix. In any case,
the migration of any of the eigenvalues of a Q matrix to the left, as
a result of the simplifying operation, appears extremely unlikely.
In general the observability and absence of conjugate points
for the sparser matrix must be tested. However, for a specific but
very important case both conditions (and hence equivalence) can be
guaranteed a priori.
Theorem 3.4
If a weighting matrix Q for a quadratic performance
index (3) operating on a scalar system in companion matrix
canonical form is unity rank, i.e.,
T
hh = Q11
where h is a vector, and the resulting optimal control law
is stable, then the matrix Q resulting from nulling the
entries q of Q for i+j od is equivalent to Q1
Proof
Since Q1 is unity rank, its elements can easily be written as
functions of the elements of the factor vector; that is,
S
Q hh T = 1h2
hIh
hh hh hh
12 13 1 4
2
h2 h2h h2h .
2
h2h3 h3 h3h4
23 3 3
where h. is the ith entry of h. The matrix Q constructed from Q by
i 2 where ij is odd is
discarding the elements qij where i +j is odd is
2
bh1
2
0 h h3 0
2
12 0 h h2h ..
2 2
0 hh O
o
It is easily verified that a rank two factor of Q2 is
h
21
0
iH2 h3
0
Q2 = H22
Part (a) of Theorem 3.3, i.e., Y(Q1 ;) = Y(Q ;w), is satisfied
by virtue of Corollary 3.3 and part (c) is fulfilled because Q2 is
shown to be positive semidefinite as a result of the existence of the
factor H2. Then all that is required is to show that no unstable
poles will be unobservable by H2 (part b) to prove Q Q1. In fact,
an even stronger condition will be shown; that is, the only plant poles
which are unobservable through Q2 are precisely those which are unobserv
able through Q This will effectively satisfy part (b) since Q1 is
required by the hypothesis to result in a stable control law and hence
cannot fail to observe any unstable plant poles
It must now be demonstrated that the only common factors of
the polynomial entries of the vector,
H S(s), (19)
are also factors of the corresponding polynomial for Q1,
h S(s) .
If H2 is thought of as defining two outputs to the system, the above
requires that the outputs not simultaneously obscure any system pole
which is observable through Q1. The polynomials formed by (19) are
2 4
hI + h3s + h5s +
3 5
and h s + h s + h s + ..
These polynomials can be recognized as the result of "separating" the
polynomial h TS(s) into even and odd functions of s. This is a common
procedure in Neicj.ork Theory in which the polynomial operators
Ev (*) and Od (*) are often defined [T31 to simplify notation.
In this notation
F(s) = h S(s) = Ev(F) + Od(F) = Ev(F) + sEv (F),
where Ev (F) is Od(F) with an s factored out. Making use of the prop
o
erty of even polynomials that their zeros lie symmetrically about the
imaginary axis and denoting the highest even power coefficient of F
as c and the highest odd power coefficient as c F(s) can be
e o
rewritten as
2 2
F(s) = c IT (s + a.) + s c 7T (s + b.).
ei=l 1 ojl=1
In the above vaS and Vb are zeros of'Ev(F) and Ev (F), and
1 3 o
the upper limits on the products must be chosen with regard to whether
the dimension of the system is even or odd. Now if Ev(F) and Ev (F)
possess a common factor, then a = b for some k and m and
k m
2 2 2
F(s) = (s2 + ak)c i=l (s +a) + c IT (s +b.)
e i= 1 0 j=1 3
igk jdm
which is clearly a factor of F(s) as well, and the theorem is proven.
This proof of the observability of the rank two Q will be use
ful in the next theorem. A simpler analysis follows easily from
consideration of the composition of the two outputs defined by H2'
Summing these outputs results in a single output identical to the one
defined by h, thus any response which is observable from h is also
observable from H2.
This theorem guarantees that if a system and an optimal control
minimize a single output in the meansquare sense (h is unity rank),
then with little effort a pair of outputs can be defined which are also
minimized. The real value of this result is, however, that it allows
enormous simplification of the unity rank performance index at the onset
of the problem, with attendant savings in numerical quality and quantity.
This result will also find application in the next chapter in another
context.
A special case of Theorem 3.4 reveals an interesting structure
which is well worth recording.
Corollary 3.4
If matrix Q1 of Theorem 3.4 is such that its factor h
forms a polynomial hTS(s) which has all zeros with non
positive real parts; then the factor H2 of matrix Q2 con
structed as described in the proof will form two polynomials,
Fl(s)
having simple zeros (except possibly at the origin) which
are restricted to lie on the imaginary axis where they
occur in conjugate pairs and alternate with each other.
Proof
The proof only requires that the root location property
described be demonstrated. The polynomials of interest are easily
recognized as
F (s) = Ev(F)
F2(s) = Od(F)
and F(s) = h S(s).
It is a wellknown theorem of Network Theory that the
Ev(F) and Od(F) functions of a Hurwitz polynomial, F, have the root
partitioning property recounted in the corollary [G6]. This phenom
enon is referred to as the Alternation [G6] or Separation [T3] property.
A complex plane diagram of the respective zero locations of a typical
case is useful to visualize this result.
o
o zeros of Ev(F)
o
x zeros of Od(F)
Re
0 F a Hurwitz polynomial
Figure 3.5 Zero Locations of Ev(F) and Od(F)
This corollary implies more than appears in the hypothesis.
By Theorem 3.2 a unity rank Q, with h S(s) a Hurwitz polynomial, can
always be constructed if Y(u) is nonnegative, then the sparse matrix
of the corollary can always be constructed as well.
3.6 Resum4 of YInvariant Matrices
Because of the necessity of investigating observability and
the testing for the absence of conjugate points much of the heuristic
appeal of the form of matrices invariant under the operator Y(Q;u)
is obscured. In order to distinguish between matrices equivalent in
the sense of Theorem 3.3 and matrices which result in identical Y's,
without resorting to cumbersome phraseology, the latter will be
referred to as "Yinvariant matrices." The conjugate point condition
is satisfied a priori for a very large class of weighting matrices
(i.e., positive semidefinite) and the observability restriction has
been shown to be a relatively innocuous constraint between Tinvariant
matrices. Then a great deal is to be gained from a review of the
structure of invariant matrices.
General Structure
Corollary 3.3 reveals that two matrices are Yinvariant if and
only if their elements are such that the ntuple
p. = q.. 2q + 2q .. i = 1,2,...,n
S 1ii il,i+l i2,i+2 
(q.. = O for i < 1 or j > n)
is equal for both matrices. For instance, in the third order case this
can be interpreted as meaning that Y will remain invariant.if a quantity
is added to q1,3 and q3,1 and twice that quantity is added to q22
In general, matrices which are Yinvariants of a given mt trix may be: con
structed by manipulating the elements on "diagonals" running from
lower left to upper right. This can be represented pictorially as in
Figure 3.6.
1 x 2 x 3
x 2 x 3
2 x 3 x
x 3 ... x nI
3 x nl x
x n1 x n
Figure 3.6 Structure of YInvariant Matrices
The integer entries of the illustration indicate which term of the
ntuple is affected by the element of a Q matrix which would be in that
location; the x's are, of course, entries which influence no term.
From this it is easy to see that matrices YinvariantAo a given matrix
can be identified by inspection.
Spectral Factorization
A special case which is of considerable interest is the Q
formed from the coefficients of the Hurwitz spectral factor of Y(w).
It is a form which can always be constructed when 'Y(u) is nonnegative
and always meets the observability and conjugate point criteria. When
the i+j odd terms are zeroed the resulting matrix is always equivalent
and possesses the root partitioning property of Corollary 3.4. This
form is theoretically straightforward to compute but in practice is
among the most difficult.
Diagonal
Another special case which is appealing in its simplicity is
the diagonal Q. This matrix is formed by placing the coefficients of
Y(2 ) along the diagonal with the constant term first. AYinvariant
diagonal matrix also can always be formed; no observability difficulties
will be encountered if the terms are all nonzero and the conjugate
point condition will be satisfied if the terms are nonnegative.
A simple example helps to demonstrate how this all fits together.
Example 3.3
Consider the real, even polynomial,
F(W) = W + 2u2 +1,
and some members of the set of (Yinvariant) matrices,
fQIY(Q;wu) = F(w))3
i) Diagonal
The diagonal Yinvariant matrix is simply,
Q1 = 0 2 :. O ,
0 0 1
which is nonsingular (thus observable) and also positive definite.
ii) Unity Rank
The roots of F(w) are sketched in the complex plane'diagram
below.
1 +1
If the left halfplane zeros are chosen, the resulting unity rank
Q = I1 is
S1 2 1 1
Q2 = 2 4 2 = 2 [1 2 11
1 2 1_
and zeroing the i+j odd terms results in
S o f' 1 o
Q = 0 4 0 = 0 2 0
J1 0 1 1 0
This matrix is clearly rank two and can easily be shown a invariant
of Q1 by adding 1 to q13 and q31 and twice that (2) to q22. The
zeros of H S(s) for Q3 are, respectively, j for the first output and 0
for the second, demonstrating the separation property of Corollary 3.4.
If the factor is composed of one zero from each halfplane, the
resulting Q is
1 0 1
Q4 = 0 0 0 ,
1 0 1
which is again clearly an invariant of Q 1
Note that in every Q matrix gener ;ed in this example the q11
and qnn entries remain fixed. From Corollary 3.3 it is obvious that
this must always be the case and that these terms are, respectively,
the first (constant term) and last coefficients of Y(w). The constancy
of these elements has a very interesting physical interpretation which
will be discussed in the next chapter.
3.7 Summary
The introduction to this chapter alluded to a concern which has
been often expressed, with varying degrees of vehemence, by those well
versed in classical design techniques; that is, the minimality of some
performance measure is seldom directly relevant to a practical design
problem. For the quadratic performance index and a linear system,
however, it was shown that resulting optimal system configurations
possess characteristics which are very much in the spirit of good
system design. The minimum optimal system stability margins of
600 in phase and 6 db in gain compare favorably with those encountered
in practice.
The problem of actually constructing a performance index which
is minimized by an optimal system was found to lead, with due consider
ation for certain cancellation and existence difficulties, to an equiv
alence class of weighting matrices which enjoy some extremely enlight
ening and useful mutual properties. When a single member of an equiva
lence class has been determined.the remainder of the class can be
constructed with relative ease.
CHAPTER IV
THE INVERSE PROBLEM AND LINEAR REGULATOR DESIGN
4.1 Introduction
The previous chapters have dealt with optimal systems, their
properties and construction. It is essential that these concepts be
well in hand if the techniques of optimal control theory are to be
applied to design problems where a relevant performance measure is not
readily discernible. In general, this is the case when a linear regu
lator is to be designed using optimal control theory. This chapter
will investigate how optimal control theory and classical techniques
may be used to complement one another in practical design problems.
For the purposes of this chapter it is helpful to think of
classical synthesis techniques for linear control systems in terms of
Figure 4.1. The problem originates with a linear system and a number
of performance specifications that the compensated system is to satisfy
(node A). The desired result is a realization (node C) which meets the
performance criteria and contains a compensator which is satisfactory
from the standpoint of practicality constraints, such as realizability
and noisy or incomplete measurements.
One way of approaching the problem is to prescribe pole loca
tions which will meet the performance specifications (node B) and then
to construct a compensator which will place the plant poles in approx
imately the desired locations while not contributing significantly
to system response (path II). This is a quite prevalent philosophy,
although it is often obscured by the specific design procedure used.
Path I can be thought of as classical synthesis and analysis procedures
used iteratively to arrive at these pole locations.
Specifications
III
Pole
Locations
Realization
Classical Design Procedure Alternatives
I. Prescribe pole locations 1. Compute performance index
a. from specifications
II. Design compensator a. from specifications
b. from pole locations
III. Direct design 2. Optimal control problem
3. Automated optimal compensator
design
Figure 4.1 Control System Design Procedures
A second approach is to determine at the onset the form of the
compensator required and manipulate its parameters to arrive directly
at a realization (path III). Wellknown variations of this approach
are Evans' root locus and leadlag design [E2]. These techniques have
the decided advantage of defining a clearcut way to proceed but it may
be difficult to accommodate noisy measurements.
The procedures of this chapter provide alternative paths in the
design scheme as illustrated in Figure 4.1 by the Arabic numbered
branches. This provides for much greater flexibility in how the problem
is attacked and employs the power of optimal control theory to relieve
some of the procedural or computational burden in some or all phases
of the design.
In some cases it will be possible to define a performance index
directly from classical specifications (path la) which will be minimized
by a control law which satisfies the specifications. The specific con
trol law can then easily be computed (path 2) or the corresponding
compensators can be designed (path 3), using KalmanBucy filter theory
[K8,K9] or one of the new techniques for automated compensator design
[P1,P2].
As indicated by path lb of the figure, it is possible to compute
a performance index corresponding to a closedloop (optimal) pole con
figuration and the remainder of the synthesis can be completed through
the use of compensator design techniques discussed in the last paragraph.
Figure 4.1 does not reveal some of the additional flexibility
allowed by these procedures. For instance, a performance index may
be specified which meets only a subset of the specifications and may
then be used as a basis for design iteration. The techniques also
provide for the design of sampleddata compensators (Section 4.4).
4.2 Pole Placement by Performance
Index Designation
If a set of closedloop system poles are proposed as a prelim
inary or final design configuration, the prescription of a performance
index which is minimized by this configuration (path Ib of Figure 4.1)
may be of considerable value in the completion or refinement of the
design. Figure 4.2 illustrates such a case.
j
x x plant poles
  n x proposed closedloop poles
x
Figure 4.2 Proposed Closedloop Pole Configuration
This configuration may have resulted from a rootlocus type
analysis, the exact scheme used is not germane to the present discus
sion; the important point is that it is not bound to any technique or
form of compensation. The construction of a performance index which
is minimized by this configuration (if possible) provides at most
2 1
approximately n /4 nominal parameters which may be varied to perturb
the design or used to design the required compensator (path 3 of
Figure 4.1).
The problem of computing a quadratic performance index which
is minimized by specified closedloop pole locations for a given
plant is basically the problem entertained in great detail in the
preceding chapter. This section seeks to deal with this problem on
a more pragmatic basis.
The plant to be considered is a completely controllable,
constant, linear system taken without loss of generality to be in
companion matrix canonical form and the performance index is the
infinite final time case of the last chapter.
The precise number of pertinent parameters will be given later.
The stable closedloop pole configuration of Figure 4.2 will be
optimal with a positive semidefinite weighting matrix Q if and only if
p(w) = Jp
where cp(s) and p k(s) are the normalized (highest coefficient is unity)
open and closedloop characteristic polynomials. Although (1) is a
succinct optimality criterion, it is not obvious how best to proceed
to test a given polynomial (1) for nonnegativity.
The necessity for testing the sign semidefiniteness of real
even polynomials arises in many other applications, for instance, tests
for positive reality in Network Theory (VI]. It can be shown by
factoring Y(w2) or through the use of Sturm's Theorem that (1) is
equivalent to requiring that Y(w 2) have no positive real roots of odd
multiplicity [Vl, p. 106].
There is still not a computationally satisfactory approach
evident. Calculation of the zeros of Y(w2) if the order is large,
is difficult numerically and best avoided as a test. Recent results
by Siljak [S3] and Karmarkar [K10] extend the interpretation of the
sign changes in the first column of the Routh table [G4] to test for
positive real zeros of odd multiplicity. This test is probably the
only alternative currently available to the computation of the zeros
of Y(w 2) as a necessary and sufficient test for nonnegativity.
Appendix C outlines a numerical implementation (in Fortran IV)
of a modification of Siljak's method, which is both accurate and effi
cient; it is believed to be the only such program in existence.
It should be reiterated that condition (1) applies only to
optimality with respect to a positive semidefinite Q; a system with
Y(u') 2 0 may well be optimal for a sign indefinite Q. In such a case
the properties conditioned on Q being positive semidefinite (Section
3.3) would not in general hold but the Q equivalence relations and
remarks concerning the observability requirements for Q (Chapter III)do.
If criterion (1) is satisfied, a positive semidefinite weighting
matrix can be generated, in fact, an entire equivalence class of posi
tive semidefinite Q's. In general the initial member of the equivalence
class must be computed by spectral factorization. This operation, like
the test for nonnegativity, is not a numerically trivial one. The
spectral factorization of Y(u), if approached naively, consists of
determining the 2(n1) roots of T, separating them by the sign of their
real parts, constructing the factor composed of the zeros with negative
real parts and multiplying it by the appropriate constant. This pro
cedure is entirely unsatisfactory. It is well known that the confidence
in approximation for root locations generally decreases drastically for
polynomials of large order; this coupled with the error induced by
constructing the spectral factor from its roots makes this procedure
numerically hazardous.
An obvious alternative is to reduce the order of Y(wu) by sub
2
stituting c = w and computing the roots of an n 1 order polynomial
which are the squares of the actual roots of interest. The roots of
Y(w) with negative real parts can be obtained directly. A refinement
of this procedure is to compute quadratic factors of Y(o) only and
from these factors, through some rather intricate logic, garner the
related left halfplane quadratic factors of Y(w). This serves to
reduce the total number of numerical manipulations, saving computing
time and decreasing the sources of error propagation. The implementa
tion of the spectral factorization algorithm discussed in the appendix
takes this approach, utilizing an efficient technique due to Bairstow
[Kll], for the approximation of quadratic factors. Bairstow's iter
ation has been shown to have a rapid rate of convergence, although it
is somewhat more sensitive to starting values than competing methods
[Kll, p. 101ff].
One objection to this procedure is that the roots of "(w) are
not directly available as a secondary test for nonnegativity. The
test for nonnegativity described earlier seems wholly adequate and
any sacrifice of computational efficiency in the spectral factorization
in favor of a redundant test does not appear justifiable.
The flow diagram of Figure 4.3 summarizes concisely the steps
to be taken in the generation of a quadratic performance index which
is minimized by a given plant and stable proposed (optimal) closedloop
pole configuration. The first step () is to compute the closedloop
and plant normalized characteris'ic polynomials (if not already known).
The magnitudesquare of the open and closedloop characteristic poly
nomials are next computed; this is another operation where the obvious
procedure, i.e., multiplication of the respective polynomials by their
complex conjugates, is not the best choice. Since the magnitudesquare
polynomials are even, half of the coefficients are zero. Thetechnique
outlined in Appendix C makes use of this structure by computing only
the nonzero terms and in a way which obviates the use of complex
Yes
k Optimal Coordinate
Yes for Transformation
Q (if required)
Figure 4.3 Procedure for Specification of a Performance
Index from Proposed Pole Locations
Numbers in circles are referred to in the text.
arithmetic. When the difference of the magnitudesquare polynomials
is taken (, the resulting Y(w) is an even polynomial of order 2(n1)
due to the normalization of the highest power coefficients of Cp and ck
to one.
A decision based on the nonnegativity of Y is made in blockG.
If Y is nonnegative the system is clearly optimal and the specifica
tion of the Q's may proceed. The unityrank Q is computed from the
left halfplane factor of Y ), using the techniques for spectral
factorization discussed earlier, which further insures the observabil
ity conditions are satisfied, and the equivalence class of positive
semidefinite Q's is immediately available.
If Y is not nonnegative the procedure is no longer straight
forward and positive results are not guaranteed. An initial Q1 is
constructed so that
Y(Q1 ;) = y(u),
and all the unstable plant poles are observable ). A reasonable
2
choice for an initial Q is the diagonal case (coefficients of Y(w )
on diagonal); this is the easiest to generate and a desirable form
for the answer. This choice of Q is inserted in the Riccati equation
for the system in companion matrix canonical form,
P(t) = FTP(t) P(t)F+ P(t)ggT P(t) Q, P(tf) )=0. (2)
The Riccati equation is then integrated numerically until
II P(t) I S e, (3)
where C is some prescribed nonnegative bound or until a conjugate
point intervenes (the solution becomes unbounded) O. The nonlinear
nature of the Riccati equation, which makes this sort of test necessary,
also helps insure the success of the test, since it may be subject to
the phenomenon of finite escape time [K2]. That is, as opposed to the
solutions to linear differential equations which may be unbounded only
in the limit, the Riccati equation may become unbounded in a finite time.
Experience has shown that when a conjugate point does exist (2) generally
becomes unbounded quite rapidly (in terms of the number of integration
steps) and conversely when a conjugate point is not present steady
state (3) is reached promptly.
A third possibility is that (2) has neither a conjugate point
nor a steadystate solution; that is, the solution is oscillatory.
It is clear that if (2) is oscillatory it must also be periodic and
hence the cost and the system response are periodic. This would
contradict the supposition that the proposed system design was time
invariant and stable. Then, if a periodic solution to the Riccati
equation occurs, it must be due to numerical errors and discarded in the
same manner as a solution with a conjugate point.
A Q. which results in a solution of (2) which diverges obviously
J
must be discarded but it does not, in itself, indicate that the system
is not optimal. The procedure branches to block 8 if a conjugate
point is present and Q which is Finvariant to Q. is computed and
j+1 J
the test for conjugate points 7 is repeated. It is difficult to
conceive of an algorithmic scheme for "improving" Q. in block 18'.
In fact, if this technique were known, one could apply it iteratively
until the "best" Q is obtained and the subsequent test for conjugate
points would be a general necessary and sufficient test for the
optimality of a control law An interactive strategy is ideal, Y
invariant Q's can be generated which not only are more likely to con
verge but also possess a desirable structure for the specific applica
tion.
The algorithm to be described seeks to generate anYinvariiat
Q+1 from a Q. which fails the conjugate point condition ( by shift
ing the negative eigenvalues of Q. generally to the right while main
taining as much simplicity as possible in Q j+1. This is done by test
2
ing the diagonal terms q.. of Q. for 1 < i < n progressively until
a negative entry is detected which is then deleted, using equation (17)
of Chapter III.
Suppose that the minor shown below is a 3 x 3 principal minor,
containing the negative diagonal entry b.
  ,    ,
r
d/2 0O a 0 0
0 0o 0 b 0
0 0 1 0 0 c
'   r
Because the first Q (Q 1) was diagonal and since the method proceeds
progressively down the diagonal, the remainder of the rows shown are
null except the first which may have an offdiagonal entry d/2 placed
by the preceding iteration. By the results of the last chapter,
altering this minor as shown below:
2
As noted in the preceding chapter, the first and last terms on
the diagonal cannot be altered and invariance maintained.
 L    F
(1/2 0 a 0 b/2 I
0 0 0
b/2 0 c
leaves the y of the resulting matrix invariant. Each time a Q. fails the
conjugate point test the described procedure removes an additional
negative diagonal entry until all have been eliminated. In this
manner the scheme allows k+1 iterations where k is the number of nega
tive coefficients of (u2 ) (or the number of negative diagonal terms)
before the system is discarded. The procedure is executed iteratively,
rather than eliminating all of the negative terms in one step, with
the hope of determining as simple (diagonal) a Q as possible for which
the system is optimal.
At this point the justification for such a procedure is
probably not clear The object of the algorithm is to generate
a sequence of invariant Q's whose positive diagonal elements dominate
their respective rows. This will hopefully result in a shifting of
the eigenvalues to the right. Central to this argument is the
Gerschgorin Circle Theorem which was recorded as Lemma 3.5 in the
previous chapter.
Since the rows of any Q. generated by the described technique
have at most two offdiagonal elements, each being half of a negative
diagonal term, the eigenvalues of the modified matrices will be
increasingly restricted to be "close" to the right halfplane if the
diagonal elements are approximately of the same magnitude. There is
no hope that the matrices will eventually become positive semidefinite
with repeated iterations (this would require Y(w) O by Theorem 3.2);
however, it is reasonable to expect the likelihood of passing the
conjugate point condition to increase if the eigenvalues are shifted
to the right. An example will help to clarify the details.
Example 4.1
Consider the sequence of Q's which would be generated by this
method for
Y(w) = 7w12 6W10 5u + 3w6 + 2u 4u2 + 1.
This Y is clearly not nonnegative, e.g., y(l) = 2; then there is no
possibility of constructing a positive semidefinite Q for it. The
procedure beginning with block 6 of the flow chart of Figure 4.3
must be called upon. Since there are three negative coefficients
of Y there will be at most.four Yinvarian.t, Q's generated; Figure 4.4
illustrates these iterations. The first, Q1, obviously has eigenvalues
which are simply the coefficients of t(w u ). The first iteration removes
the first negative entry on the diagonal (4) by "splitting" it into
the two offdiagonal terms resulting in Q2. Four of the eigenvalues
remain unchanged, the one that was at 4 is moved to the origin and
the remaining two have been "smeared" by the interpretation given by
Gerschgorin's Theorem. The eigenvalue which was at 1 may now be any
where within the interval (1,+3) and the eigenvalue at 2 may now lie
in the region (0,+4) as shown in the plot.
Actually the right halfline, since the matrices under con
sideration are symmetric and consequently have only real eigenvalues.
12 10 8 6 4 2
Y(U) = 7 610 5u + 3j + 2u 42 + 1
r1 0 2
4 (") 0 0 0
2 2 0 2
3 0
6
7
0
3 0 2.5
0 0 0
2.5 0 6
1
0
2
Q4=
o 0 0
0 1
3 0 2.5
0 0 0 3
O 2.5 0 0 0
3 0 7
xxx
 X X
xx,F
0. .
  v^B^C,
10 5
'x  x Q
0 Q4
0c x
c Q 4
5 10
o Computed eigenvalues
x Eigenvalues by inspection
mn Permitted eigenvalue regions
Eigenvalue Distributions
Figure 4.4 Eigenvalue Distributions for Q
.1
F1
I I 1 
'2
With subsequent iterations the original eigenvalues are spread
into permitted regions which are generally to the right. Due to the
structure of such matrices the actual eigenvalues are easily computed
and included in the plot. The process lives up to expectation, the
negative eigenvalues are moved distinctly to the right and the likeli
hood of passing the conjugate point condition (esp. for Q4) is con
siderably enhanced.
Note that the rank of Q for j > 1 is reduced by at least one.
This causes concern that some observability may be lost in the process,
with the result that an unstable plant pole becomes unobservable (a
stable plant pole becoming unobservable is, of course, inconsequential).
Although this may indeed be the case, it will be shown later that suf
ficient safeguards exist to prevent it from passing unnoticed.
Within the framework of the process outlined there exist many
variations which may be of value in certain applications. For instance,
it may be advantageous to dispatch the negative diagonal terms of
largest modulus first rather than approaching them progressively along
the diagonal. This modification applied to Example 4.1 leads to a
somewhat greater "improvement" in the second iteration than the method
originally employed, but the results of the final iteration are, of
course, identical. The important observation to make is that no single
variation will be best in every case and that employment of good judg
ment at this point in the scheme, rather than following a strictly
algorithmic approach, will probably be rewarded with simpler resulting
Q's. It is for this reason that the suggestion was made earlier to
utilize an interactive strategy in a computer implementation of this
procedure.
If the iterative technique of modifying Q. and testing for
J
conjugate points succeeds in obtaining a Q for which the Riccati equa
tion does not diverge, the next step is to determine if this Q is
actually a solution to the inverse problem. This is necessary to guard
against gross numerical errors in the integration of the Riccati
equation falsely indicating the absence of conjugate points, and, as
mentioned earlier, to insure that a detrimental loss of observability
from Q has not occurred.
The steadystate solution, P, to the Riccati equation (2) is
available from the conjugate point test (block ( and provides this
check. The feedback control law,
k = Pg,
is simply the last column (and row) of P. By the structure of the
companion matrix canonical form if
n n1
cp(s) = s + ans + ... + a s + al'
then Yk(s) = s + (a +k )s + ... + (a +k2)s + (a +k ),
where k* is the ith entry of k. The coefficients of pk computed by
substituting p. for k. can be compared to the correct (original) values.
This test is represented by block ( of Figure 4.3.
In passing, it should be noted that the process for generating
successive Q's outlined here fails to make full use of the relation
of the elements of invariant matrices (Corollary 3.3) in that only the
terms on the diagonal and second superior diagonal are affected (i.e.,
qii' i+2,i2 and qi2,i+2 when qii < 0). This is because the process
becomes considerably more difficult to justify as the number of altered
terms increases and because it was felt that the additional complexity
in the resulting Q was selfdefeating. However, further research in
this area may ultimately lead to a necessary and sufficient condition
for optimality when Q is not positive semidefinite.
There remains but one operation which may be required to com
plete the process of designing a performance index for which a given
closedloop pole configuration is optimal. If the state model for the
plant (assumed completely controllable) is not in companion matrix
canonical form it may be desired to transform the Q which has been
generated into the coordinate system of the plant. This is accomplished
by the congruent transformation,
Q = T QT,
where Q is in the original coordinate system of the plant and T is the
nonsingular matrix of the similarity transformation (Section 3.4),
A ~ 1
F = TFT ,
which transforms the original state matrix (F) into a companion
matrix (F).
It is necessary to show that this can be done without loss of
generality. That is, that the closedloop poles are invariant and that
no conjugate points are introduced under this transformation.
Theorem 4.1
Optimality is preserved under similarity transfor
mation. That is, if a completely controllable linear
plant and feedback control law are optimal with respect
to a quadratic performance index, the system and control
law subject to a nonsingular coordinate transformation
minimizes the performance index in the new coordinate
system.
This theorem can be easily proven by applying the transforma
tion required to return the system to its original form from the
companion matrix formulation to the Riccati equation. Then, by the
nonsingularity of T, if the Riccati equation becomes unbounded in
one coordinate it will in the other as well.
4.3 Design by Performance Index Iteration
Once a technique, such as the one presented in the last section,
is available for generation of performance indices for optimal closed
loop pole configurations a simple design scheme becomes evident. The
performance index could be used as a basis for improving a first attempt
at design. In terms of Figure 4.1, this would entail traversing
branches lb and 2 iteratively, each time altering the entries of Q in
a manner intended to meet additional members of the specification set.
This approach offers several advantages over competitive classical
design techniques. If the preliminary closedloop pole configuration is
chosen so that
Y(uW) = k (jiw) 2 cpi(ju)2 1 0 for all real m,
then maintenance of the positive semidefiniteness of the resulting Q
throughout all the iterations will insure that the important stability
criteria of gain and phase margin are at least 6 db and 600, respectively.
This permits the designer to concentrate on bringing transient response
to within desired limits and later returning to the stability margins
if more stringent ones are required.
A second advantage is that a tractable number of design param
eters are displayed in the Q matrix. The precise number can easily be
shown to be [ ) where the heavy brackets represent the greatest
integer function; i.e.,
[a] = greatest integer 5 a.
This quantity does not include the duplicate (by symmetry) terms appear
ing across the diagonal of Q.
This means that for a thirdorder plant there are at most four
parameters of Q which are pertinent to the design; similarly for a
tenth order plant, at most 32. These numbers may seem rather large in
relation to the number of coefficients of the characteristic poly
nomials; however, these parameters, as will be seen later, may be
related more or less directly to the transient response of the closed
loop system, in contrast to the classical design parameters. Further,
far less than the maximum number given will be generally required; in
fact, usually it will only be necessary to vary two of them [H2].
It was noted in the last chapter that the first and last terms
on the diagonal of equivalent Q matrices must remain invariant. That
is, q11 must be the same among all members of an equivalence class and
similarly qnn must not change. The next theorem presents an important
new result which is a consequence of this observation.
Theorem 4.2
If Q is any member of an equivalence class of weighting
matrices (Definition 3.1) for a scalar nth order linear
system and optimal control law in companion matrix canonical
form, then
n 9 n 2
q I R TT r
11 i 1 i i=l 1
n 2 n 2
and q =n R S r
nn i=l i 1=
where r. and R., i = 1,2,3,...,n
are the plant and optimal closedloop poles, respectively.
Proof
In the discussion of the diagonal Q case in Section 3.6, it was
shown that the coefficients of Y(u2 ) form the diagonal entries. The
optimization problem implied by this Q may have conjugate points.
Nonetheless, since the control law of the hypothesis was specified
to be optimal, there must exist at least one weighting matrix which
forms a performance index minimized by the control law and further
it must bea Yinvariant of the diagonal Q,. Then, since the first and
last entries on the diagonal of invariant matrices cannot change, these
2
entries must be exactly the first and last coefficients of Y(w ).
Consider the monic polynomial with real coefficients,
n n1
p(s) = s + a s + ... + a s + a (4)
n1 1 o
and its magnitude square
Ip(j 2 2n 2(n1) 2 .
p(j) = w + b 0( + ... + b 0 + b (5)
nl 1 o
By definition
Y(u) = Ik(j )2I [p(ju)I2; (6)
that is, Y is the difference of two polynomials similar to (5).
Then the constant term of Y, q11, is the difference of two terms
corresponding to b in (5). Clearly,
2
b = a ,
o o
and by a wellknown theorem of elementary algebra, the constant
term of a monic polynomial is (to within the sign) the product
of its zeros. Then b is the square of the product of the roots of
(4) and application of this abstraction to (6) demonstrates the
expression for q11'
2(n1)
The coefficient of wo of y, q is the difference of
n,n
terms similar to b n in (5). By examining the product of p(ju) and
its complex conjugate it is easily verified that
b = a 2a (7)
nl n1 n2'
Again it is known from elementary algebra that for amonic poly
nomial such as (4), a n is the negative of the sum of the zeros of
4
(4) and an2 is the sum of all combinations of products of the roots
of (4) taken two at a time. That is,
4
Here combination is used in the sense of combinatorial
analysis; i.e., no distinction is made between the ordered pairs
(c,d) and (d,c).
n
a r r
n1 1 i
i=1
and
n i1
an2 = E Z rr where r ,i=1,2,3,...,n are the
i=2 j=l
roots of (4). The expression for a can be rewritten by forming the
sum of all the possible permutations of products of two roots (includ
ing with themselves) and subtracting off the sum of the roots squared
and dividing by two to account for the combinations being summed twice;
i.e.
1 7n 2 n 2
S l r. r r2
n2 2 1 i
Then, using this expression for an_2 and substituting into (7) results
in
n 2
b r. .
1 =l
The expression for qnn then follows directly from application of this
result to equation (6).
The interpretation of the effect of q on a system can be
greatly enhanced through the use of a simple mechanical analogy.
If the poles (r.) of a linear system are thought of as unit point
masses, the expression,
n 2
I = r.,
i=l
would be the resulting moment of inertia with respect to a perpendic
ular axis of rotation through the origin such a mechanical system
would possess. Then qnn represents the amount by which the "moment
nn
of inertia" of the system has increased with the addition of optimal
feedback.
If qnn is nonnegative the optimal closedloop system will
have at least one pole further removed from the origin than the plant
poles. That is, for qnn > 0 the closedloop system will tend to be
faster in response than the openloop, and, in general, the larger qnn
is made the faster the optimal system.
Although q 11 does not admit to analogy as well as q nn its
effect on the optimal pole locations is similar to that of qnn ; that
is, the speed of response also tends to increase with q11'
Recently there has been a resurgence of experimental investi
gations of the effect of entries of the Q matrix on the dynamic response
of the closedloop system [e.g., H2,R1]. Although it is hopeless to
attempt to glean a generalized design scheme from such studies, since
they are bound to specific plant configurations (i.e., a fixed number
of poles and zeros), it is possible to obtain some guidelines.
Houpis and Constantinides [H2] observed that q11 and qnn together have
primary effect on rise time (t ) and setting time (ts) and the remainder
of the entries primarily influence overshoot. These observations seem
to coincide quite well with the analysis given here.
At this point a design example would help to solidify the
preceding remarks.
Example 4.2
It is desired to compensate the unstable plant
1.0
u s(sl)(s+2)
so that the following performance specifications are met:
1. Gain Margin: GM 12 db
2. Phase Margin: CpM 600
and in response to a step input,
3. Overshoot: Os S 5%
4. Rise time (within 90%): tr 1 sec
5. Setting time (within 1%): t \ s 2 sec.
The approach taken here will be to develop a preliminary
design, using dominant roots techniques, which is also optimal, and
to then iterate on the entire of the Q matrix until a suitable final
design is arrived at. By dealing only with positive semidefinite Q's,
phase margin, and probably gain margin as well, will be guaranteed,
leaving only the dynamic response specifications to be investigated.
Using the dominant root philosophy, the complex poles should
have a time constant of approximately 2 seconds to meet the require
ment for ts and a damping of approximately 0.7 in order to keep max
imum overshoot within 5% and still meet rise time specifications
[D4, Fig. 43, p. 91]. The third pole removed by 2.5 times the dominant
time constant should insure dominance. The resulting preliminary
design pole configuration is shown in Figure 4.5.
2j
c 0l' I x x
5 4 3 2 1 1
0
x plant poles
O proposed closedloop poles
Figure 4.5 Preliminary Design Pole Configuration
The algorithm of the preceding section is now applied to obtain
a Q matrix (if one exists). The system is indeed optimal and a Q matrix
which will work is
1600
Q =
0
0
0
20
The transient response of this system was computed, using the quadratic
optimization and simulation program LQL [B7,B9]. The important char
acteristics of the response are recorded in the first row of the table
of Figure 4.6. It is clear that response is too slow, hence, q11 is
increased; however, continued increase much beyond the value of entry 3
causes the overshoot specification to be exceeded. Reducing q33 has the
effect of reducing the damping on the dominant poles (decreasing t r) and
bringing the third pole in toward the origin and results in more rapid
settling (decreasing ts). Run number 6 meets the specifications on
transient response.
Figure 4.6 Transient Response vs q..
The stability margins only remain to be investigated.
Since the Q for all the iterations is positive semidefinite,
GMI 6 db and CpM 600. The Nyquist diagram of Figure 4.7 shows
that the gain margin is considerably in excess of 12 db; thus
iterative number 6 meets all the design specifications.
Run 33 str Os ts
No. 11 22 33 (sec) (%) (sec)
1 1600 60 20 1.2 3.3 2.6
2 576 60 20 1.6 2.0 3.2
3 3000 60 20 1.1 4.0 2.3
4 2500 60 10 1.1 4.1 2.2
5 2500 60 4 1.1 4.0 2.1
6 3000 60 4 1.0 4.6 2.0
7 5000 60 4 0.9 5.3 1.8
Figure 1!.7 Nyquist Plot of Final Design
Figure 4.8 RootLocus Plot of Final Design
X Plant poles
o Closedloop poles
o Feedback zeros
Final design
preliminary design
Desired response envelope
1 2 3 4 5
Time (sec)
Figure 4.9 System Responses to a Step Input Preliminary and Final Designs
The final design meets the performance criteria:
GM = 15.18 db
tM = 62.90
tp = 1.0 sec
ts = 2.0 sec
Os = 4.6%
and has the state feedback control law
54.77
k= 33.53
7.49
Although Figure 4.6 only tabulates seven iterations on the Q
matrix, fourteen iterations using program LQL were actually made,
requiring approximately 6 seconds of CPU time on an IBM 360/65 at
a cost of about $0.65.
This example illustrates two facets of this method. First,
there are no concrete rules, as such, for the manipulation of the
entries of Q to obtain desired changes in response, only general
guidelines. Secondly, the speed and low cost with which the iter
ations can be computed make this process of practical value.
A final feature of this scheme is that the resulting design
is not bound to any specific form of compensation: this can be
regarded at times either as an advantage or a handicap However, with
the rapid advance being made in automated compensator design [e.g., Pl,
P2], the disadvantages associated with the compensator not being
prespecified are becoming largely illusionary.
4.4 Design by Explicit Performance Index Specification
If the design problem is more fully specified, a useful varia
tion of the preceding scheme will often be profitable. For instance,
when specifications of the sort required for Example 4.2 are given and
in addition it is desired to minimize (in a meansquare sense) a spe
cific system output, or a weighted sum of outputs, the technique of
performance index iteration takes on a particularly interesting form.
Consider the scalar input, multioutput linear system,
x = Fx + gu
y = H X,
for which it is desired to determine a feedback control law so that
a given (but at the moment arbitrary) set of classical performance
specifications are satisfied and in addition the performance index,
J = (cyTAy + u2)dt, (8)
is minimized, where c is an unspecified scalar and A a symmetric
weighting matrix. Then in the usual notation,
Q = cHAH
and cY(u) = cY(HAHi ;),
which can be easily computed using Lemma 3.4. Any closedloop system
which minimizes (8) will have a characteristic polynomial, k (s),
such that
