Partial realization of covariance sequences


Material Information

Partial realization of covariance sequences
Physical Description:
v, 92 leaves : ; 28 cm.
Georgiou, Tryphon Thomas, 1956-
Publication Date:


Subjects / Keywords:
Analysis of covariance   ( lcsh )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )


Thesis (Ph. D.)--University of Florida, 1983.
Includes bibliographical references (leaves 84-91).
Statement of Responsibility:
by Tryphon Thomas Georgiou.
General Note:
General Note:

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
oclc - 11272181
System ID:

Full Text







I wish to express my gratitude to all those who have guided
my steps during my years in Gainesville.

Professor R. E. KALMAN has been a constant source of encourage-
ment and inspiration during my entire graduate work. I am
particularly grateful for his continuous effort and interest in
teaching me System Theory. Without the financial support which he
arranged for me this work would not have been possible.

I wish to express my most sincere appreciation to Professor
E. W. KAMEN, the chairman of my supervisory committee. His continuous
encouragement and support were most valuable to me in the completion
of this work.

This dissertation grew out of a joint research with Professor
P. P. KHARGONEKAR. He, as the cochairman of my supervisory committee,
has been an invaluable advisor, and as a co-worker and fellow student,
has been a dearest friend.

My warmest thanks and deepest appreciation go to Professor A. R.
TAinrEiEAUTl. His enthusiastic encouragement and confidence I will
always remember and be grateful for.

I am most grateful to Professor T. E. BULLOCK for his invaluable
guidance and support from the beginning of my graduate studies. I
also wish to express my most sincere appreciation to the other
members of my supervisory committee, Professors G. BASILE and R. L.
LONG, for many valuable discussions and help in the course of my studies.

My stay in Gainesville has been a most significant experience for
me. A doctorate is a slow educational process and in this process I
gratefully acknowledge the guidance and help of Professors A. C.

I am especially grateful to Ms. Eleanor Onoda for her kind and
untiring help in administrative affairs and together with Mrs. Patty
Osborn for their excellent typing of this dissertation.

I would also like to express my deep appreciation to my
friends Bilent, Jaime, Kameshwar, Nikos, and Theodoros, for their
help and encouragement.

Words cannot tell how much I owe my wife Efi, my parents, my
sister and especially my grandmother. To them I dedicate this

I wish to acknowledge partial financial support by the US
Army Research Grant DAAG29-81-K-0156 and US Air Force of Scientific
Research Grant AFOSR81-0238 through the Center for Mathematical
System Theory, University of Florida, Gainesville, Florida, 32611,


ACKNOWLEDGEMENTS . . ... ...... ii

ABSTRACT . . . v


I. INTRODUCTION ................... .. 1


1. Orthogonal Polynomials: An Algebraic Approach 6
2. Interpolation Theory: A Function Theoretic
Approach . .... ... 15


5. Rational Covariance Extensions and the
Dissipation Polynomial .. 26
4. Asymptotic Properties of the Spectral Zeros. .. 37


5. Covariance Extensions of Dimension s 47
6. Basic Degree Theory . .... .51
7. Dissipation Polynomials and Covariance
Extensions of Dimension s . .. 53

V. THE MATRIX CASE. ... . 59

8. The Matrix Covariance Extension Problem 59
9. The Topological Approach . 61
10. The Algebraic Approach . 67


11. The ME Method and Some General Discussion 72
12. On Pole-Zero Modeling .. ... 75


EEFREIICE . .. ..... ... 84


Abstract of Dissertation Presented to the Graduate Council
of the University of Florida in Partial Fulfillment of the Requirements
for the Degree of Doctor of Philosophy


August 1983

Chairman: Dr. E. W. Kamen
Cochairman: Dr. P. P. Khargonekar
Major Department: Electrical Engineering

This work is concerned with rational covariance extensions of
partial sequences. Certain methods of the classical interpolation
theory are exploited and a novel topological approach is developed.
An associated polynomial, that we call the "dissipation polynomial",
is found to be a free parameter for covariance extensions with
dimension bounded by the number of data. A similar result holds for
the case of matrix sequences as well.

The dissipation polynomial is found to impose an "almost
recurrence" law on the SCHUR parameters of rational covariance
sequences. This is done via a new approach to spectral factorization.
These theoretical results, placed in the context of the applied
area of spectral estimation theory, suggest some recursive procedure
for pole-zero modeling.


The elementary notion of positivity of real numbers has found
various generalizations to that of quadratic forms, operators etc.
These play a key role in, not only mathematics, but many areas of
applied science as well. The reason is that positivity is, in one
form or another, intimately related with the manifestations of
physical quantities and entities.

The motivation for this work arises from the area of stochastic
processes and identification theory. The covariance function
cs := Ey +s, s = 0, + 1, ..., of a discrete-time, zero-mean,
stationary stochastic process y T Z, is characterized by the
nonnegative definiteness of the Toeplitz quadratic forms

u v
Z E ac b
s=o t=o s s-t t'

for u, v = 0, 1, ..., and as, bt E C. This is the notion of
positivity which plays a central role in our work.

A stochastic process is an abstraction. The real object is a
realization of the process: a time-series. The probabilistic
behavior is not available. For identification and prediction,
estimates of the means and covariances of the stochastic process
have to be computed from some observation record. In this, a large
number of issues are involved. Most of them are of a statistical
nature (confidence limits, etc.). Theoretically, most of these
issues are still terra incognita.

In this work we shall not be concerned about such questions but
we shall assume as the given data a partial sequence of covariances
Cs := (ct : t = 0, 1, ..., s). This is the standing assumption for
the theoretical development of several modern spectral estimation
techniques (cf. HAYKI;l [1979]). These techniques seek a certain
extension of Cs to a covariance sequence C = (ct : t = 0, 1, ...).

The set of all covariance extensions of C can be described by
several alternative approaches that we shall recapitulate in Chapter
II. However, theoretical as well as practical interest lies with a
certain subclass, the rational ones.

A sequence C = (c : s = 0, ...) is called rational iff there
exists an integer v such that the rank of the behavior (Hankel) matrices

B := [c ]
s := t+u-lt,u=l

satisfies rank B < v for s = 1, 2, .... The smallest such v will
S --
be called the dimension of C.

Rationality is directly related to existence of finite dimensional
stochastic realizations of C. The dimension of C is then precisely
equal to the minimal dimension for the corresponding state-space
(see for example FAURRE, CLERGET and GERMAIN [1978]). Moreover, in
case C is a rational sequence there exists a finite positivity

In the case of unconstrained rational extensions of a partial
sequence, called partial realizations, a minimal dimension can be
found by testing linear dependence. In case the minimal dimension
partial realization is not unique, this set is parametrized by a
linear space. These concepts have given rise to the elegant partial
realization theory of KALMAN [1979]. In the case where the extension
is also required to satisfy the covariance property the problem becomes
substantially more involved (see KALMAN [19811).

The purpose of this dissertation is, in broad terms, to elucidate
the relation between rationality and positivity. The main goal is to
unveil smae of the issues and the nonuniqueness involved in extending
the partial data C to a rational covariance sequence C, paying
special emphasis on the dimension of these extensions. We believe that
we are reasonably successful and that certain of our results can be
profitably considered in the applied area of identification.

We now give a brief description of the contents of each chapter.
More detailed introductory remarks are provided at the beginning of
each chapter.

Chapter II is devoted to a review of certain classical techniques
and concepts that are directly related to the covariance extension

In Chapter III we use the algebraic machinery developed in Chapter
II, to study rational covariance extensions. It turns out that a
certain polynomial is intimately related with the extension and can
be chosen arbitrarily. This polynomial, that we call the dissipation
polynomial, represents the zeros of the power spectrum. We found
that the dissipation polynomial determines in a precise way the
asymptotic behavior of some important sequence of parameters that is
associated to a covariance sequence. In order to obtain this result
we developed a new technique for spectral factorization.

In Chapter IV we develop a new approach that is most suited for
describing the covariance extensions of dimension bounded by the
number of data. In point of fact, in a large number of cases this
dimension coincides with the minimal dimension. The merit of this
approach lies also with the fact that it provides a novel topological
proof of a certain classical result: the positive definiteness of
the quadratic form associated with C is sufficient for the
existence of covariance extensions. The key result of this Chapter
further shows that an essential nonuniqueness in this partial realiza-
tion problem is best described in terms of the associated dissipation

Chapter V extends some of the earlier development to the case
of matrix sequences. Finally, in Chapter VI we discuss the relevance
and potential of the above in modeling.

The initial motivation and part of this dissertation grew out of
a joint work with KHARGONEKAR (GEORGIOU and KHARGONEKAR [1982]). The
material of Chapters II and III is based upon this work.

We close this introduction with a few words on notation and
standing assumptions. Throughout this dissertation, we shall work
with the field of complex numbers C : C[z] will denote the ring
of polynomials in z with coefficients in C ; C n[z] will
denote the ring of n x n-matrix polynomials in z; -
denotes complex conjugation and if p(z) E C[z], then p(z) denotes
complex conjugation on the coefficients of p(z). We shall be dealing
with both infinite sequences C = (c : t = 0, 1, ..., ct C for
t > 1 and c R) and finite sequences C := (c : t = 0, 1, ...,
0 = s t
s, ct EC for t = 1, 2, ..., s and co R). A sequence C is
= 0 -
said to be positive (resp. nonnegative) iff the Toeplitz matrices

T = [c i s = 0, 1, ..
s t-u t,u=o'

where we define ctl := Ct, are positive (resp. nonnegative)
definite for all s. A partial sequence C is said to be positive
(resp. nonnegative) iff T is positive (resp. nonnegative) definite.
Thus, a sequence C is a sequence of covariances of a stochastic
process if and only if C is nonnegative. Finally this notion of
positivity (resp. nonnegativity) will be denoted by > 0 (resp. > 0).


The purpose of this chapter is to introduce certain mathematical
concepts and techniques that are pertinent to the covariance extension

We shall begin with some algebraic aspects of the theory of
orthogonal polynomials relative to the unit circle. Toeplitz matrices,
of the same type as the ones associated with the covariance function
of a stationary stochastic process, were classically considered to
induce an inner product on the space of polynomials. The special
Toeplitz structure can be effectively exploited by considering a
particular orthogonal basis. This gave rise to the theory of or-
thogonal polynomials of SZEGO [1939]. Since that time the theory was
progressively developed by many researchers. AKHIEZER [1965],
GERONIMUS [1954], [1961] and GRENANDER and SZEGO [1958] have given
classical expositions on the subject. It was early recognized that
the theory of orthogonal polynomials had strong connections with a
prediction problem in the theory of stochastic processes (see
GRENANDER and SZEGO [1958, p. 175] or the survey paper by KAILATH [1974]).
This opened up areas of application of the theory, notably in
stochastic problems, spectral analysis and autoregressive modeling
(see the book by HAYKIN [1979] for various applied and theoretical
aspects on these). Motivated by autoregressive modeling for multi-
variate stochastic processes, WHITTLE[ 1963], and WIGGINS and ROBINSON
[1965] laid the first pieces of a theory of orthogonal matrix
polynomials. A number of researchers have then pursued this line
of research. We mention only the most recent works of YOULA and
KAJANJIAN [19781, MORF,VIEIRA and KAILATH [1978] and especially
DELSARTE, GENIN,and KAMP [1978a] that have given a rather elegant
account of the theory of orthogonal matrix polynomials on the unit

We restrict our attention to the scalar case and in Section 2 we
give a concise exposition of those aspects of the theory that we con-
sider to be relevant to the covariance extension problem.

The theory of orthogonal polynomials is connected to certain
problems in analysis. Various aspects are discussed in AKHIEZER
[1965], AKHIEZER and KREIN [1962] and KREIN and NUDEL'MAN [1977].
In particular there is a connection with a certain interpolation
problem that is equivalent to the covariance extension problem.
Both solvability conditions and a parametrization of all solutions
can be obtained by the classical SCHUR's algorithm. The machinery
of orthogonal polynomials can be used to provide a compact
description of the solutions in terms of a linear fractional
transformation. This is the content of Section 3. We should
finally mention that a similar description of an isomorphic
problem was used by DEWILDE, VIEIRA and KAILATH [1978] and also

1. Orthogonal Polynomials: An Algebraic Approach

We consider an infinite sequence C = (cs : s = 0, 1, ...,
with c in R and c in C for s > 0). We define on the
o = s -
space C[z] of polynomials in z an inner product by

u v u v
(t- at, z b z ) := t S atb ct,
t=o t s=o s t=o s=o t s t-s

This inner product is definite if and only if C > 0. Whenever
C > O (resp, > 0) then the above inner product defines a norm
(resp., semi-norm) on C[z] that we shall denote by 11.1I. We
begin by discussing separately the two cases of interest: first
the case C > 0, and second the case C > 0 but not > 0.

We now consider C to be a positive sequence. The inner
product (-, *) is now definite. We apply the standard orthogonal-
ization procedure to the natural basis (zs : s = 0, 1, ...) of
C[z] to obtain an orthogonal (but not necessarily orthonormal)
basis of monic polynomials ( s(z) : s = 0, 1, ...). These poly-
nomials are known as the orthogonal polynomials (of the first kind)
associated to the sequence C, and are given by

o(z) = 1,


(s(z) = det

... sC

C ... -s+l
0 -s+1

... z

/det Ts1, s = 1, 2, ...

Since |IIs(z)I2 = (zs, (s(z)), it follows that

(1.2) j|s((z)l2 = det Ts/det Ts_1, s = 0, 1, ...,

where det T-1 := 1.

The special inner product structure of C[z] induces upon the set of
orthogonal polynomials certain algebraic identities and an important
parametric description. We shall now discuss these.

Let Ps(z) C[z] be of degree s. We define the reverse polynomial

SzS -1
P (z)* := z S (z-).

From (z s(z)) = 0 for t = 0, 1, ..., s -1
structure of the inner product it follows that

and the Toeplitz

(1.5) (zt, s (z)*) = 0 for t = 1, 2, ..., s, and any s > 1.

Since s (z) is a monic polynomial of degree s, we can write

(1.4) s(z)* = 1 Zt bs,t t(z),

for some scalars b From (1.5) and the above we obtain
s t"

0o -





Hence, bst = (1, zDt(z) )/lt(z) I12
gives rise to the parameters

is independent of s. This fact

rt+1 := (1, zot(z))/|lt(z) I2, t = 0, 1, ....

These parameters are known as the SCHUR parameters of the sequence
C. From (1.4) we now obtain the (well known) recurrence identities

s(1) (z) = z s1(z) ;s s-1(z)*,
(is(z)* = s -l(z)* rs z s-(z),

for s = 1, 2, .... From the first identity we obtain

lls (z2) 2 s-() 12 s-1s(z)*, zes-1(z))

and also

0 = (ze (z), es1(z)*) rs ll-1(z) 112.

Combining the two we obtain

(1.6) lks(z) 12 = (1 Irs 2) lls_(z)112,

This shows that the parameters R = (rs :

s = 1, 2, "**

s = 1, 2, ...) satisfy

(1.7) Irs < 1, s = 1, 2, .

These conditions (and co > 0) are equivalent to C being positive.
In fact starting from the parameter sequence R with Irt < 1 for
all t, and c > 0, we may construct a corresponding positive
sequence. This correspondence is bijective. Furthermore, partial

O = (1, zDt(z)) bs, t(s), Dt(z)).

positive sequences Cs = (ct : t = 0, 1, ..., s,
correspond bijectively to pairs (Co, R ) where
Rs = (rt : 1, 2, ..., s with Irtl < 1 for all
show below.

with T > 0)
c > 0 and
. This we
t). This we

The sequence of parameters of a positive sequence C is givenby

r = C /co

r T = (- 1)s det
s s



c -
. cs-l


/det Ts-l1

for s = 2, 3, ..., and they satisfy Irsj < 1, s = 1, 2, ***

Conversely, starting from the
rs < l, s = 1, 2, ..., and c
sequence C by solving (1.8) for

(19 s-1 -
(1.9) Cs = cors t9l (i rt 2)

sequence of parameters that satisfy
> 0, we obtain the corresponding

+ (cl ... Cs-) Ts2


C /

for s = 1, 2, ***
this follows from

SThe above is
Irt < 1 for

valid provided det T2 0. But
t = 1, 2, ..., and the algebraic

det T r2 det Tt-
det T t det T t-
t-1 t-2

t = 1, 2, ...,

(which arises from (1.2) and (1.6)). Hence, (1.8) and (1.9) establish
the required bijective correspondence.

We now discuss the singular case of nonnegative sequences that are
not positive. Such a sequence is called singularly nonnegative.



Assume that C f (0, 0, ...) is a singularly nonnegative sequence
and let s be the smallest (positive) integer for which

det T = 0.

The partial sequences (It(z) : t = 0, 1, ..., s) and Rs are defined
as earlier and (1.1) (1.6) hold for t = 1, ..., s. However, we now
have |s (z) I2 = 0 and R satisfies

(1.10) Irt <1, t=l, 2, ..., s -1, and Irs = 1.

The interesting feature of this singular case is that any singu-
larly nonnegative sequence C, as above, is rational and uniquely
determined by Cs (or, equivalently, by co and Rs). Also any
partial sequence C with T > 0 and det T = 0 det Ts
admits a uniquely defined singularly nonnegative extension C.
We now prove these facts.

Let C be singularly nonnegative and

,. s s-1
X(z) = z + az +... + a

be a monic polynomial of least degree that satisfies IIX(z) 2 = 0.
Evidently, s is the smallest (positive) integer such that det T = 0.
Furthermore, X(z) = 's(z).

Clearly, IztX(z) I2 = 0 for all t > 0. By the fact that T +t 0
it follows that

(0 ... 0 as ... a 1)'

is a zero-eigenvector of Tst and therefore

(1.11) c = ac ... a for t > 0.
-s-t 1-s-t+l s -t -

This shows that C is a rational sequence and is in fact uniquely
determined from the partial sequence C .

We now let Ct be such that T > 0 with det T = 0. We shall
show that there exists a nonnegative extension C of C which by the
above discussion is unique.

The partial sequence Ct defines in the obvious way a semi-norm and
an inner product on the space of polynomials of degree less than or equal
to t. We denote these by i-11t and (*, )t respectively. Consider now
s to be the smallest integer for which det T = 0 and
X(z) = z + alz + ... + as be a monic polynomial of least degree that
satisfies Ix(z) 2 = 0. Precisely as we did before we now obtain that

(1.12) c _s_ ac -s-+l- ... a u for u = 1, ..., t s.

We now extend the partial sequence Ct to an infinite one C using
(1.12) for u = t s + 1, .... We now show that C > 0.

For any a(z) in C[z] denote by a mod X the remainder of
a(z) divided by X(z). Since (zu, X(z)) = 0 for all u > 0 it
follows that

(a(z), a(z)) =(amod X, a mod X)

=(a mod, a mod X), > 0.

Therefore C > 0.

The above considerations readily solve the covariance extension
problem: Given a partial sequence Cs, there exists a nonnegative
extension C of C if and only if C > 0. In particular, the
following two cases are possible:

(a) Nondegenerate case: C > 0.
In this case the set of all nonnegative extensions of C
are in bijective correspondence with sequences of parameters
R that are either finite and of the form R = (rt: t = s + 1,
..., s + u, with Irtl < 1 for t < s + u and rs+u = 1)
or infinite satisfying Irt < 1 for t = s + 1,...

(b) Degenerate case: C > 0 but Cs 0.
In this case there exists a uniquely determined nonnegative
extension C.

We now proceed to consider certain related mathematical objects,
which will be useful in the next section in showing the connection of the
above with an interpolation problem.

Define the power series

0 t
r(z) := + 21 ctz

The function theoretic properties of P(z) will be described in the next
section. Here we view P(z) as an algebraic object. We shall now
recall the notion of partial realizations (see KALMAN [1979]) and then
introduce the so-called orthogonal polynomials of the second kind of C
by considering certain partial realizations of

(z- = + 2t c zt.
t-1 o -t

Consider a formal power series F(z ) (in negative powers of z).
A pair of coprime monic polynomials (r(z), X(z)) or, equivalently, the
rational function r(z)/X(z) is said to be a partial realization of
F(z-1) of order s iff

[F(z-1)X(z)z s-de]+ = -(z)zs-deg,

where [ ] denotes "the polynomial part of". Equivalently, the rational
function 7r(z)/X(z) is a partial realization of F(z-1 ) of order s if
and only if the Laurent series 7(z)/X(z) (with the division carried out
in the field of formal Laurent series in negative powers of z) matches
1 -S
F(z -) up to and including the coefficient of z

We now consider the power series T'(z-) and define a sequence of
polynomials by

Ps(z) := [F z-1) ),

where s (z) is the s-th orthogonal polynomial of the sequence C.
The integer s runs over either all nonnegative integers or, a
finite number of them depending on whether C is positive or non-
negative,as we discussed earlier. These polynomials are known as
the orthogonal polynomials of the second kind of the sequence C and
were introduced by GERONIMUS [1961, p. 10]. From the definition of
Ys(z) we have that Co s(z)/$s(z) (though not necessarily a coprime
representation) is a partial realization of '(z- ) of order s. In the
case where C is singularly nonnegative and s is the smallest integer
for which det Ts = 0, it follows that F(z-1) is actually equal to
c s(z)/os(z).

We now show that the orthogonal polynomials of the second kind
satisfy the following recurrence identities (which, except for a sign
change are the same as (1.5)):

Ss(z) = zs (z) + ri s (z)*,
ss-1 s s-l
s(z)* = :s-l(z)* + rszs-!(z),

for s = 1, 2, ... (or a finite index sequence in case C is singularly

We define the transformation

1 t
f: C[z] -[z]: A(z) [A(z)(c + 2t c z-]

We also define a sequence C via the relation

(o O + t ^-t 'o
(c+2 Co- z )(c +2 c2tzt) = 1,

and the transformation

f: C[z] -,C[z]: A(z) c--[A(z)(c + 2t c tzt)].
co 0 t=1-t +

A 0
It is straightforward to check that ff = ff is the identity

We now modify our earlier notation by adding a subscript to
Toeplitz inner products (and norms) that specify the defining sequence.
Thus we shall have (., *)C, (., .)", and if we define I = 1,0, 0, ...}
then we shall also write (., *) .

We denote by g* the adjoint of the transformation g. We now
have that

<(.1 -- <' j(f + f*).)I,

and also

^1^^ ^ ^.+fA
(f, f.) (., 2 f*(f + f*)f.) (, (* + f).) *).

Since if*iC = .II^ is certainly a semi-norm, it follows that C is

Since the polynomials Ys(z), s = 0, 1, ..., were defined by

Ys(z) := fs(z) s = O, 1, ..., they are the orthogonal polynomials of
the first kind of the sequence C. Therefore they satisfy relations of
the type (1.5). In order to establish the precise form (1.13) of these
relations we only need to show that

(1.14) os(o) = s(O).

From the definition of s (z) we have

is(z) = -[(2tl C_ z t)s(z)] Ds(Z).

The constant term of

0 -t
o [(2tJ1 c zt)ms(z)]+

equals (2/co)(1, s (z)) = 0. Therefore (1.14) holds and hence (1.13)

We finally want to indicate another algebraic identity that is
satisfied by the two types of orthogonal polynomials that we shall
refer to later on:

(1.15) O (z) s(z)* + D (z) s(z)* = 2zshs,

where h is a scalar given by

s 12 ) 1 ,D ( z) 11 2
(1.16) hs : :r = ls(z) |2.

This identity follows from (1.5) and (1.13) using induction on s.

2. Interpolation: A Function Theoretic Approach

Consider the following interpolation problem: We are given two
regions G and G in the complex planes of the variables z and
z w
w, and a set of pairs (za, w ) with z in G and w in Gw
for all a in a certain index set I. It is required to find function
F(z) holomorphic in G with values in G that satisfies the
z w
interpolation constraints

F(z ) = w for all a in I.
a a

When certain of the points z coincide, then the interpolation
constraints are modified so as to assign at these points
values to the successive derivatives of F(z).

This problem is classical and a number of techniques have been applied
to it. The books by WALSH [1956] and AKHIEZER [1965] give comprehensive
expositions of the classical approaches to the problem. In recent years

new functional theoretic techniques have been applied that also extend
to a more general class of interpolation problems that includes
interpolation with matrix-valued functions. These techniques have been
developed in the work of SZ.-NAGY and FOIAS [1970] and SARASON [1967].

In this section we shall consider a particular case of the problem
which is directly related to the covariance extension problem. We
discuss the so-called SCHUR's algorithm that also provides a description
of the solutions, and we tie up this approach with the material of the
previous section.

The following classical theorem states that the notion of positivity
encountered earlier is expressed in terms of a function theoretic

(2.1) THEOREM (see AKHIEZER [1965, p. 178]). The power series

r(z) := c + 2cz + ... + 2c z

converges in Jzj < 1 and has Re P(z) > 0 for all z in Izl < 1
if and only if the sequence C = (cs : s = 0, 1, ..., with
co := (c + c)/2) is nonnegative.

The functions possessing the above property form the so-called
class C. (This same property is known in the engineering literature as
positive realness. See for example BELEVITCH [1968, p. 71].)

We shall consider the following interpolation problem: Given a partial
sequence Cs = (ct : t = 0, 1, ..., s) findthenecessaryandsufficient
conditions for the existence of a function in C whose power series
expansion in z begins with c + 2c z + ... + 2c zs. Also, it is
required to describe the set of all solutions.

This is known as the CARATHEODORY problem. In viewof Theorem(2.1)
it is seen to be equivalent to the covariance extension problem.

Below we proceed to discuss the so-called SCHUR's algorithm as
applied to the CARATHEODORY problem. This technique provides a para-
metrization of the solutions in terms of functions in class C.

The main technical result needed is the following simple

(2.2) LEMMA. The function ra(z) belongs to C and has power
series expansion in z that begins with 1 + 2ca z if and only if
one of the following two conditions holds:

(a) Ic(a) < 1 and

d (z) a(z) ba(z)
b(z) c (z) (z) + a (z)
a a a

is in C where aa(z) := (1 + z)( ca)

ba(z) := (1- z)(l c a)), c(z) := (1 z)(l + c(a)

and d (z) := (1 + z)(l + c ),

1 + c(a) z
(b) Ic(a) = 1 and r (z) = 1
1 -- a (a)
1 c z

INDICATION OF THE PROOF. The set of functions S(z) that are
analytic in Jzl < 1 and satisfy IS(zo) < 1 for all z in lzl < 1
forms the so-called class S. There exists a simple relation between
functions of class C and functions of class s: r(z) is in C if
and only if

(2.5) S( z)
2 r( z) + -rgf

is in S.

For functions of class S it is easier to show an analogous
statement (see AKHIEZER [1965, p. 101]): A function Sa(z) is in S
if and only if one of the following two conditions holds:

(a') |Sa (0) < 1 and

sS a(z) S (0)
Sb(Z) :=
2 1 WTQTs(z)

is in C,

(b') Sa is constant of modulus equal to one.

Applying now (2.3) to the above statement proves the lemma. 0

In point of fact this lemma gives a description of all functions
in C in terms of certain parameters: Beginning with a function
P(z) := rl(z) = 1 + 2c( z + ..., we iterate the formula

dt(z)rP(z) bt(z)
(2.) t+1(z)=- ct(z)rt(z) + at(z) '

for t = 1, 2, ..., while Ic(t) / 1. Then, P(z) belongs to C
if and only if one of the following two cases holds:

(a) Ic(tI < 1 for all t,

(b) Ict) < 1 for t = 1, ..., s -1 and
l+c (s
Ps(z) = -s with l) = 1.
1- c( z

(See also AKHIEZER [1965, p. 103].) The parameters pt := c '
t = 1, 2, ... are called SCHUR parameters of r(z).

The above lemma readily solves the CARATHEODORY problem: Given
the partial C = (1, c, ..., c ) define ct := ct, t = 1, ..., s.
By the lemma a function in C exists having power series expansion that
(1+ 2cl) (+ 2c1) s
begins with 1 + 2c, + ... + 2c z if and only if either
-L s

(1) (1) (1) t
Ic 1) 1 and c a= (c1 ) for t = 2, ..., s


c() < 1

and there exists a C-function with power series that begins with

2c(2 z + ... + 2c(2)zs-
1+ s-1

where c(2), t = ..., s 1 are obtained via the formula

al(z)(1 + 2c(l)z + + 2c(1)zs) bl(z)

c(z)(l + 2c1)z + ... + 2c 1z) + dl(z)
(2) (2) s-i s

S1 + 2c z + ... + 2cs z + O(zs)

(where the division is carried out in the field of formal Laurent series
in positive powers of z). In this way the problem can be transformed to
an equivalent one with one interpolation constraint less. This
inductive procedure is known as the SCHUR's algorithm. Iterating the
above we obtain:

The CARATHEODORY problem is solvable in precisely the following two

(a) Nondegenerate case: Ic t) < 1 for t = 1, ..., s.

In this case the general solution is nonunique and is obtained

at(z)rPt+(z) + bt(z),
(25) t() ct(z)Pt+l(z) + dt(z)

for t = s, s 1, ..., 1 and s+1 anarbitrary function in C.

(b) Degenerate case: Ic t) < 1 for t = 1, ..., u 1, with
1 (u) (u) t
u < s, c() = 1 and cU = (c)) for t = 1, ...,
s u + 1. In this case the solution is unique and is
obtained from (2.5) iterating for t = u 1, ..., 1 with

1 + c(U) z
r (z) = 1
1 c z

The property that a function r(z) belongs to C is described in
terms of the parameters t := ct), t = 1, 2, ..., and also interms of
the parameters rt, t = 1, 2, ..., that occur in the recurrence rela-
tions of the orthogonal polynomials of a corresponding (by the Theorem
(2.1)) sequence. These two sets of parameters turn out to be
equivalent. In the rest of the section we shall show this which
gives the precise connection of the SCHUR's algorithm with the material
of the previous section.

We first prove the following:

(2.6) LETMA. Let R := (rs : IrsI < 1, s = t, t + 1, ...) be a
sequence of parameters and (It(z), st(z), s = 0, 1, ...) denote
the associated orthogonal polynomials. Define

A (z) := (z) + yt(z)*, Bt(z) := st(z) (z)*,

Ct := (t(z) t(z)*, D (z) := ( (z) + t(z)*, and

A t(z) B t(z)
s s
M(z) (:= 1

s s

algebraic identity holds:

The following

t (z)\ Pt+s (z)
(2.7) s+u ( Mt () ,
t (z) s t+s( z)

PROOF. We apply induction on u. For u = 0, (2.7)
obviously holds. Assuming that it holds for u = v we obtain

s +v+l( Z)t

where applying on
of Mt(z). But

Mt(z) simply means to apply on the entries

0 -oS) t(z) M ) ).

We finally obtain

+u+l( z) = Mt(z)
(stt+u+ th s

which completes the proof.

In the case
with Ir s < 1
for 1 < t < v,

for s
0 < s,

< v

is a finite sequence (r : s + 1, ..., v,
and r v = 1) then (2.7) still holds
u, and s + t + u < v.

Let now Pt(z) be the functions in C with Pt(0) = 1 that
correspond through (1.9) and Theorem (2.1) to the parameter sequences
Rt that we defined in the lemma. We shall show the following:

(2.8) PROPOSITION. Provided Ir I < 1 for u = 1, ..., s + t,
then the following two identities hold:

St+s t+v
\ vt+s (Z) t /



A t()* s+t(z)
(2.9) rt(z) =
C (z)*rs+t(z)

(2.10) rs+t(z) =

+ B t(z)*
+ Dt(z)*

- Bt(z)*
+ At(z)*

for all t > 1, s > 0.

PROOF. We first consider the case where R = (rs : s = 1, 2,
..., with irs < 1 for all s) is an infinite sequence. We shall
show that both sides of (2.9) have the same power series expansion in
z. (Both polynomials and power series are considered as elements in
the field of formal Laurent series in z.)

We first show that (Ct(z)*s+t (z) + Dt(z)*)-1 exists. Indeed,
the coefficient of the zero-term is

() (0) + (0)* + (0) = 2 / 0.
s s s s

From Lemma (2.6) we have that

t +(z)*
(2.11) -
sut (z)*

At(z) *s+t(z) */s+t(z)* + B (z)*
s U Ut s
C t(z) *s+t Z) */0s+t(z)* + D (z)*
s u u s

Using the above it is straightforward to show that

At(z)*Ps+U(z)+ B (z)*
s s
Ct (Z)*rs+t(z) + D (z)*
s s

t z)

Dt (z)*


for all u > 0. By the definition of the orthogonal polynomials of
the second kind we also have that

P (z)

s +(z)*

- O(z++l )

for all u > 0. This establishes (2.9).

In the case where RI is a finite sequence the above are
still valid for u < v for some maximal v such that r s+t+v =1.
But then

Pt(z) = +(z)*/Ot (z)*, and
s+v s+v

rs+t(z) = t(z)*/ t (z).
v v

Therefore, by (2.11), it follows that (2.9) holds.

The identity (2.10) follows from (2.9) when solved for
r +t(z) provided the denominator in the right hand side of (2.10)
is not identically zero. This we show below.

From the definition of the orthogonal polynomials of the
second kind we have that

(2.12) t(z)*rt(z) + D(z)* = 0(zs+).
s s

From the above and (1.15) we obtain

(2.1) ) (z) + t(z) =
s s
= t(z)t(z) + t ()* t(z))/ t(z)* + (zS+1)
s 8 s S S
t s s+l
= ht z + O(z ),

t s 2
where h := (1 Irt+2). By adding (2.12) and (2.13) we obtain

Ct(z)*rt(z) + At(z)* = h zs + 0(zs+l) / 0,

since ht 0. O

We finally show:

(2.14) PROPOSITION. If r(z) denotes a function in C with
r(0) = 1 and rt(z), p Pt(z), rt are defined as before for
t = 1, 2, ... (finite or infinite), then rt(z) = rt(z) and
Pt = rt for all t.

PROOF. We apply induction on t. By definition
rl(z) = r(z) = Pl(z) and therefore pl = rI. Suppose tr(z) = t(z)
for some t. Then t = rt and hence, at(z) = At(z)*,
t t
bt(z) = BI(z)*, ct(z) = C (z)*, and dt(z) = D (z)*, as well. In
case Iptl = Irtl = 1 then both sequences have terminated and we
are done. If ptl = Irtl < 1 then from (2.9) and (2.10) we
conclude that Ft+l(z) = r t+(z).

(2.15) REMARK. As we mentioned earlier, interpolation ideas have a
strong connection with circuit theory. For example, the celebrated
DARLINGTON synthesis procedure is the analog of the SCHUR's
algorithm for solving the general Nevanlinna-Pick interpolation
problem. Several of these connections were pointed out and shown
explicitly by DEWILDE, VIEIRA, and KAILATH [1978]. In point of
fact, in that paper they derived a compact description for the
solutions of a SCHUR interpolation problem that is similar to (2.9)
(see DEWILDE, VIEIRA, and KAILATH [1978, p. 668] and also DELSARTE,
GEIFIII and KAMP [1979, p. 40]). The various forms of interpolation
can be interpreted in a circuit theoretic framework as synthesis
with cascade connection of coupling networks. In the same framework
the linear fractional transformation (2.9) is seen to correspond to a
cascade connection terminated to a resistive network with impedance
rs+1(z) (compare also with BELEVITCH [1968, p. 110]).


In the first section we begin by applying the previously derived
interpolation results to the study of rational covariance extensions.
Certain bounds for the dimension of the various extensions are
provided by this algebraic approach.

With any rational C-function or, equivalently with any rational
covariance sequence,there is associated a certain polynomial in z
and z This polynomial we call the dissipation polynomial of
the sequence. It represents the zeros of the power spectrum or,
equivalently, the zeros of an associated stochastic realization.
This polynomial is completely determined up to a scalar factor by the
tail of the associated parameter sequence. In point of fact, the
dissipation polynomial up to a scalar factor is an invariant of the
action of "shifting and truncating" the corresponding parameter

This rather interesting result is further exploited in Section 4
in connection with asymptotic properties of rational covariance
sequences. The most complete treatment up to date of the asymptotic
and analytic properties of positive sequences and of the associated
orthogonal polynomials has unquestionably been given by GERONIMUS [1961].
We shall apply some of his results to the case of rational sequences.
A certain new aspect that emerged does not seem to have an analogue
in the general case. The dissipation polynomial determines the asymp-
totic behavior of the parameter sequence. The sequence of parameters
of a rational covariance sequence is not necessarily rational; however
it is in a certain precise sense very close to being so. We shall
call this property "almost rationality".

Another aspect of this development is a new algorithmic procedure
for spectral factorization. This is a key problem in system theory
and several approaches to it have been developed. In the discrete-
time scalar case, given a rational function 7(z)/X(z), with v(z),
X(z) in C[z], that has positive real part almost everywhere on

Izl = 1, it is required to obtain a factorization

I1 M ( z i) (z-1) gz -
2 X(z) R(z-1) X(x) X(z-1)

with r(z) E C[z], for the real part of r(z)/X(z) on jzl = 1. This
factorization amounts to factoring a nonnegative trigometric polynomial

d(z, z-1) := (z)R(z-1) + X(z)-(z-1) = i(z)n(z1)

z = e as the square of the modulus of a polynomial iT(z).

The existence of such a factorization is well known. The most common
approaches are a Riccati based approach (see FAURRE, CLERGET, and GERMAIN
[1978]) and an algorithm due to RISSANEN and KAILATH [1972]. For
different aspects of the factorization problem see ANDERSON, HITZ and
[1976], STRINZIS [1972], and YOULA [1961].

In our investigations we found a new technique. This is intimately
related with the above. However, it operates on both numerator and
denominator of a C-function v(z)/X(z) instead of simply the dissipation

The key idea is based on the invariance of the dissipation polynomial
d(z, z-1 ) under the action of "shifting and truncating" the corresponding
parameter sequence. Under this operation, certain associated rational
C-functions tend to 1, uniformly on compact subsets of Izl < 1.
Consequently, both numerator and denominator polynomials tend to the same
polynomial, which turns out to be the "stable spectral factor" of
d(z, z-1).

3. Rational Covariance Extensions and the Dissipation Polynomial

A key result in partial realization theory is that a sequence
C = (ct: t = 0, 1, ... is rational if and only if the power series

r(z) = c + 2 c zt
o t=l t

defines a rational function in z (see GATITMACHER [1959, Chapter V] and
also KALMAN, FALB, and ARBIB [1969, Chapter 10]). Moreover, if
P(z) = r(z)/X(z) with 7T(z), X(z) coprime polynomials in z, then

dim C = max (deg r(z), deg X(z))

=dim r(z).

The previously derived description for the solutions of the
CARATHEODORY problem will now be applied for the study of the rational
ones. Such a solution with data a partial sequence C will be called
a pr (positive rational) extension of Cs

In the degenerate case where C is nonnegative but not positive,
there exists (see page 10) a unique covariance extension which turns out
to be rational. In the nondegenerate case where Cs is positive,the
set of all pr-extensions is described in the following:

(3.1) THEOREM. Let C be a given partial positive sequence,
Rs = (rt: t = 1, ..., s) be the associated partial parameter sequence,
and M1(z) be the corresponding matrix polynomial defined in Lemma (2.6).
-- s
An irreducible rational function c T-(z)/Xl(z), with 1 (z), X1(z) E
C[z] and l(O) = Xl(0) = 1, is a pr-extension of C if and only if
there exists a unique irreducible rational' C-function T s+(z)/Xs+1(z)
with 7T (z), X (z) E C[z] and Tr (O) = X (0) = 1, such that
s+1 s+1 s+1 s+1

7T1(z) T s+1 (z)
(3.2) = M((z) s+1z
X1(s) s Xs+(z)

Then, R = (rs+l rs+2, ...) is the sequence of parameters associated
with s+l(z)/Xs+l(z) if and only if R = (rl, ..., rs, rs+, ...) s
the sequence of parameters associated with T1(z)/X1(z). Moreover, the
following holds:

dim s+1(z) l(Z) Ts+1(z)
dim + -7 dim ( -dim *+ s.
xs+lZ) lZ Xs+lZ

It is clear that any continuation (rt : Irtl < 1, t = s + 1, ...)
for the partial sequence R leads to a positive extension of C The
positivity is effectively characterized in terms of the associated sequence
of parameters. This is not the case with the rationality. Due to the
nonlinear transformation (2.9) between the c's and the r's, the para-
meter sequence of a rational sequence is almost never a rational one.
However, the interpolation approach shows that the rationality of a
covariance sequence is completely determined by the tail of the parameter

PROOF. From (2.9) and (2.10) we readily obtain that a solution Pr(z)
to the interpolation problem is rational if and only if the C-function
rs+ (z) associated with the continuation of the parameter sequence is
also rational.

Let rs+ (z) = Ts+(z)/Xs+l(z). The normalization conditions
T71(0) = X1(0) = 1 and Ts+ (0) = Xs+(O) = 1 are compatible due to the
fact that

S+0)* r r

s 2(l- r 1 + r
Ss s

We now derive the last inequality that provides bounds on the
dimensions of various pr-extensions. We first define the matrix

D(z) B(z)
(3.3) N (z) :=
V(z) A(z)
u u

for v = 1, 2, ..., n = ..., with A:(z), Bv(z), Cv(z), Dv(z)
u U u u
defined as in Lemma (2.6). Using (2.15) it can be shown by direct
calculation that

1 0
NV(z)* MV(z)* = zuhV
u u u 0 1


hV := v(1 Irt )
u t=v t

Applying this to our case we have

s s
h xzs+i(z) x1(z)/

Since the elements of N1(z)* are polynomials of degree s, the above
implies that

deg Xs+1(z) < deg Xl(z)

and also

deg Ts+1(z) < deg Tr(z).

From (3.2) we also have that

deg Xl(z) < deg Xs+l(z) + s

and similarly

deg 7l(z) < deg Ts+l(z) + s.

These prove the last inequality in (3.1). O

We now consider an irreducible rational C-function T(z)/X(z).
The real part of v(z)/X(z) is nonnegative for all z on Izl = 1 where
it is defined. Therefore,

r(e )X(e ) + T(e )X(ee) > 0

for all e in [ 7, r]. Following the terminology of an unpublished
report by KALMAN we shall call

d(z, z-1) := ( (z)x (z-1) + (z-1)X(z))

the dissipation polynomial of 7(z)/X(z). By slight abuse of terminology
we shall also call a dissipation polynomial any polynomial p(z, z-1)
in both z and z which for z = expje and all 8 in [-in, 7] is
nonnegative. Finally, let us define the degree of d(z, z ) as the
largest power of z.

The role of the dissipation polynomial in the context of stochastic
processes will be discussed in Remark (3.8). Herein, we shall see that
the dissipation polynomials associated to the various pr-realizations
of Cs are up to a scalar factor determined completely by the choice of
the C-function rs+1(z) = Ts+l(z)/Xs+1(z) of the previous theorem.

Let r(z) = co (z)/X1(z) be a rational C-function and
R = frs : s = 1, ...) denote the associated parameter sequence. Let
also R = (r : s = t, t + 1, ...) denote the usual "shifted and
truncated" parameter sequence and 7T(z)/Xt(z), dt(z, z-1 ) the associated
rational C-function and dissipation polynomial. We now present the

(3.4) THEOREM. In the degenerate case where

R = (rs : s = 1, 2, ..., u, with Ir s < 1 for 1 ; s s u
and Iru = 1

we have that

dt(z, z-1) 0 for t = 1, 2, ...u.

In the nondegenerate case where

R = (r : s = 1, 2, ..., with Ir s < 1 for all s)
s s-

we have that

d(z, z ) = hdt (z, z l), for t = 1, 2, ....

The behavior of the dimension of wt(z) as t increases is
described in

(3.5) PROPOSITION. In the degenerate case where u is as in Theorem
(3.4) we have that

dim = u t + for t = ..., u.

In the nondegenerate case we have that the following two cases are

mt+l(z) wt(z)
dim t+( = dim
Xxt-7t+7 I

7Tt+l(z) 7t(z)
dim -t+() =dim 1.
xt+l77 XtT

Furthermore, case

(a) is equivalent to each of the following two

wt(z) -i
(a') dim-T = d (z, z

(a") dim X-t = deg (wt(z) + Xt(z)).

We will now prove (3.4) and (3.5).

PROOF OF (3.4). In the degenerate case we have

Tt(z) = Tt+l(Z)*,


Xt(z) = -t+l( )*.

From (1.15) we obtain dt(z, z-1) = h t+l. Since ru = 1, it follows

t u
h = (1 Ir ) = 0.
u-t+l v=t-

In the nondegenerate case we obtain from (1.15) that

1 1 2-1 -1
dl(tz, z ) 2 1(z) (z-1z) + i(z l)X(z))

=2 (t(z)1(z-1) +1 l(z-l) t(z) t+1(z)Xt+1(Z-l) +

+ Tt+l(z-) Xt+z))

t -1
= hd t+(z, z-1) .

PROOF OF (3.5). In the degenerate case,

Xt(z) = ut+l(z)*,


rt(z) = _t+l(Z)*,

for t = 1, 2, ..., u. Also, |t +i(0) I= I t+l() = Ir = 1
-t+is different from zero. Therefore,
is different from zero. Therefore,

deg Xt(z) = deg vt(z) = u t + 1.

In the nondegenerate case, using (2.4) we obtain

t+(z) = 7t(z) Xt(z)) + z(wt(z) + Xt(z)))/(1 + rt),


zXt+l(z) = ft(z) 7t(z)) + z(Xt(z) + vt(z)) /(l rt).

This shows that both deg 7rt+1(Z) and deg Xt+,(z) are less
than or equal to max (deg 7t(z), deg Xt(z)). Moreover, this difference
can be at most one. Therefore (a) and (b) are the only two possibilities.

In the case where max (deg Xt(z), deg rt(z)) =
deg (vt(z) + Xt(z)), then clearly deg vt+1(z) = deg Xt+,(z) =
deg (Vt(z) + Xt(z)) = dim vt(z)/Xt(z). In the case where the above does
not hold,then both deg vt+1(z) and deg Xt+1(z) are less than
dim 7t(z)/Xt(z). This establishes the equivalence of (a) and (a").

Consider now the identity

1 -1 ) +1
Sft (z) t(z-1) + Et(z')xt(z)) = dt(z, -1)

d(S) S d(o) (s)-s
= d()z + ... + d( + ... + d z-,
t t t

where we assume that d(s) d 0. It is easy to see that
deg (7t(z) + Xt(z)) = deg dt(z, z-). This establishes the equivalence
of (a') and (a"). D

In view of the above, the dissipation polynomial of the various
pr-extensions of Cs can be arbitrarily chosen by appropriate choice
of the C-function Ys+l(z)/Xs+l(z). The set of all rational
C-functions,which have dissipation polynomial fixed up to a scalar
factor, is described in the following:

(3.6) PROPOSITION. Let d(z, z-1) be an arbitrary dissipation poly-
nomial of degree s. Then for any polynomial a(z) such that a(z)

and zSd(z, z-1) are coprime polynomials there exists a unique rational
C-function r(z) = 7(z)/X(z) with r(0) = 1 and such that

(5.7) d(z, z-1) = (7r(z)x(z-1) + (z1-l)X(z)


v(z) X(z) = za(z).

Conversely, for any rational C-function r(z) having r(0) = 1
and dissipation polynomial d(z, z-1) there exists a corresponding
polynomial a(z) as above.

PROOF. Let a(z) be any polynomial such that zSd(z, z-1) and a(z)
are coprime. Then

Sd(z, z-1) + a(z)(z-1)

is a dissipation polynomial which is zero on izl = 1. Therefore, this
polynomial factors into a product b(z)t(z" ) where b(z) is coprime
with a(z) and has no root in iz[ < 1.
Consider the function S(z) := a(z)/b(z). Clearly we have that

1 1S(z)2 4C z1 > o,
s(z)2 b(z)S(z-1)

for z = expje and 0 in [- 7~, 7]. Since b(z) has no root in
Iz < 1, it follows from the maximum modulus principle that S(z) is
in S. Therefore by (2.3)

-b(z) zaz
z = b z) + za z

is in S. It can be readily checked that P(z) has dissipation
polynomial d(z, z-1 ).
Conversely, for any r(z) = r(z)/X(z) in C where r(0) = 1
and (3.7) hold, a(z) = (r(z) X(z))/z is the required polynomial.
This is shown by reversing the previous argument. 0

The algebraic approach we followed in this section gives only rough
bounds on the dimension of the various pr-extensions of C For
example, given any dissipation polynomial of a certain degree u we
can always find a corresponding C-function of the same degree. Then,
by Theorem (3.1) we can obtain a pr-extension with dimensions between
u and s + u.

In the next chapter, a different approach will be followed. It will
be shown that, with an appropriate choice of the extension, we can
always achieve dimension equal to s. Furthermore, it will be shown
that for a large number of cases, s is the smallest possible dimension
of any pr-extension.

(3.8) REMARK. The role of the dissipation polynomial in the context
of stochastic realization will be now discussed. At the same time,
certain quantities that will be used in the next section will now be

Consider a covariance sequence C = (cs : s = 0, 1, ...). The
nonnegativity of C or, equivalently, the covariance property is a
necessary and sufficient condition for the existence of a nondecreasing
function a(9), with 9 in [-7, r] such that

1f t
ct = e-tdo.(O), t = 0, 1, _+ 2, ...

(see AKHIEZER [1965, p. 180]). This function is called spectral
distribution (of C, or of a corresponding stochastic process). The
derivative cr'() of o(e) exists almost everywhere in [- v, 7]
and is called spectral density.

The C-function P(z) associated to C as in Theorem (2.1) admits
the following integral representation (see AKHIEZER [1965, p. 1791):

(3.9) r(z) = e je z da(e)
e- z

Our interest rests in the case where C is also a rational sequence.
In this case u(a) consists of two parts:

(3.10) c(Q) = aa(e) + Tj(0),

where ca(e) is an absolutely continuous nondecreasing function and
jo.() is a nondecreasing function with finitely many points where the
function increases. Furthermore, the derivative la(e) is a rational
function in e For more details see DOOB [1953, P. 542] and
GRENANDER and SZEGO [1958, p. 5].

The decomposition (3.10) induces via (3.9) the representation

(3.11) r(z) = ( = a(z) .( z)

where a (z)/Xa(z) is a C-function with the property that Xa(zo) f 0
for all z in Izl < 1 and v.(z)/X.(z) is of the form
0 3 3

7) (z) deg Xj exp(jeu) + z
(312) ( = 1 Pu exp(jeu) z

where p are positive scalars.
Let d(z, z-1), d (z, z-1) and d.(z, z -) denote the dissipation
polynomials of the above three functions (in the obvious notation).
From (3.12) it immediately follows that d.(z, z-1) = 0. Relation (3.11)
now implies that

1 1 -1
(3.15) d(z, z-) = d (z, z )X.j(z)X(z ).

A stochastic realization of C is a dynamical system E that under
certain stochastic input and initial states generates an output process
y that realizes C via the covariance function. Any stochastic
process y can be decomposed into a superposition of two uncorrelated
stochastic processes

Yv = a, + Yj,

where ya, is the so-called purely nondeterministic part and yj
the deterministic part. (This is called the Wold decomposition. For
more information see GRENANDER and SZEGO [1958, Ch. 10] or functional
analysis literature where it has been widely used, e.g., HELSON
[1964, p. 10])]

In the case where C is a rational sequence then the above
decomposition is in correspondence with (3.10). The part y j, can
be realized by superposition of sinusoidal signals with frequencies
determined from the roots of X.(z) whereas y can be realized by
a single input system having white noise input and transfer function
Ta(z)/Xa(z) where

(3.14) da(z, z-) = a(z)a(z-1)

The relations (3.13) and (3.14) indicate the role of the dissipation
polynomial in this context.

Returning to the covariance extension problem it is natural to
consider factorizations of the form (3.13) for the dissipation poly-
nomials of both 7l(z)/Xl(z) and vs+1(z)/Xs+l(z) in Theorem (3.1).
Whenever both parts in (3.10) are present it is not necessarily true
that (in the obvious notation) X1 .(z) = Xsl ,(z). Due to this fact
it appears that formula (3.2) in Theorem (3.1) is simply a computational
tool and does not seem to have a stochastic interpretation.

4. Asymptotic Properties of the Spectral Zeros

We begin by discussing a procedure for spectral factorization. This
result will be subsequently used to elucidate the role of the dissipation
polynomial on the asymptotic behavior of the sequence of parameters.

Let P (z) = 71(z)/X1(z) be a rational Q-function with r (0) = 1
and R = rg : s = 1, 2, ...) the associated parameter sequence. We
denote by Rt the usual truncated sequences, vt(z)/Xt(z) the

associated C-functions and dt(z, z-1) the dissipation polynomials,
where t = 1, 2, .... We shall consider only the nondegenerate case
where dl(z, z-1) 0. In this case by a well known factorization
theorem (see GRENANDER and SZEGO [1958, p. 20]) there exists a polynomial
Tl(z), with Tl,(0) = 1, and a positive scalar 7l such that

(4.1) dl(z, Z-1) = 1(z)1(z-1)

If we require that nl(zo)
T1(0) = 1, then both 71
polynomial 1l(z) we call

for all z in I z < 1 and also
T~1(z) are uniquely determined. This
stable spectral factor of dl(Z z-1).
stable spectral factor of dl(z, z ).

We similarly define 7t, t((z). From Theorem (3.4) we clearly have

71(z) = nt(z), for t = 1, 2, ...,

and also

71 sl(1 r 12)t.

We now set

Vt(z) = 1 t + ... + at) ,


Xt(z) = 1 + b )z + ... + b z


(4.2) rt=(at) bt)/2,

and by (2.4)

(4.5) 7Tt+l(Z) = 1/2[(Tt(z) Xt(z))/z + (TT(z) + Xt(z))]/(1 + rt),

(4.4) Xt+l(z) = 1/2[(Xt(z) rt(z))/z + (7Tt(z) + Xt(z))]/(l rt),

for t = 1, 2, .... Iterating the above we obtain the sequence of pairs
(Tt(z), Xt(z)), t = 1, 2, ....
The following theorem states that (4.2), (4.5), and (4.4) provide an
algorithmic procedure to obtain q(z).

(4.5) THEOREM. Let R1, Tt(z)(z), t(z), and T(z) be as above. Then

im 7T(z) = Jim Xt(z) =(z).

PROOF. First we need to recall certain function theoretic results:
Let r(z) be a C-function, and for simplicity assume r(O) = 1.
Let o-(@) be the associated spectral density function that is given by

9'(0) = Re r(ej )

a.e. on [- T, v].

(4.6) STATEMENT. The following are equivalent:

(a) anca' (e) is integrable in [- TT, T],
(b) there exists a function m(z) in H2 (the usual Hardy
space; see RUDIN [1966, p. 328]) such that

a(e) = Im(eje) 12

(c) a.e. on [- i, 7T],

(d) tl(l Irt12) > 0,

(e) tl rt 2 < +m

For a proof of the above statement see GERONIMUS [1961, pages 20 and 159].
In case the above equivalent conditions hold we may find a
function m(z) that also has inverse that is analytic in iz| < 1.

h en m(z) is

m(z) = exp ej e + z ena(9)deO |z| < 1,
S-7 e z

and also

7 := m(0) 2

(4.7) = exp In fn(9)de
-T 1

= t(1 Irt 2)

(See GERONIMUS [1961, pages 20, 21, and 158].) Furthermore, if
a := det Ts_/det T s = 1, 2, ..., and a := 1, then

cp(z)* := as s(z)*, s = 0, 1, ..

converges to m(z)-1 as s m. This convergence is satisfying the
following inequality (GERONIMUS [1961, Theorem 4.10]):

(4.8) p (z)*m(z) 1_ 781 I
s1 + 7 1 Iz

for jzl < 1 and where

0< 12 1/2
5s tz. Irtl2

(Note that 5s < +m because of (4.6).)

In case r(z) = v(z)/X(z) is a rational C-function with
dissipation polynomial d(z, z -) 0, the ina'(G) is integrable and
in fact

m(z) = 71/2(z)/X(z),

where q(z), 7 satisfy (4.1) and rq(z) is the stable spectral factor
of d(z, z ).

We now apply (4.8) to the C-functions Tr(z)/Xt(z) for
t = 1, 2, ..., and for s = 0 :

mt() 1I = 71/2 1 Z

S ot 1 + Yt

Ss i i2 \1/2
where 5 < (,ii r 2 Since 7 > 0 and 5,1 < +C,
s,t \u=st-l u 1 1,1
it follows that limy = 1 and lim 6, = 0. Consequently,
t ot

1t/2 z)Xtz) 11 -40

as t -> uniformly on compact subsets of IzI < 1. Hence,

im x(z) = z(z).

Similarly we can show that im ~ t(z) = q(z). 0

Using the above we now want to study the asymptotic behavior of
the sequence of parameters of rational covariance sequences. We begin
by a motivating

(4.10) EXAMPLE. Consider the rational function

(Z) 1 + ( + r z
X(z) 1 + rz

of degree one. Necessary and sufficient conditions for 7(z)/X(z)
to be in C are that

fri <1,

jaI < 1 Ir .

The sequence of parameters (provided

r = r,

1 Jr 2

and the nonlinear recurrence law


r s+1
s+2 r s12
r(l r+l| )

Irl 1) is determined by

for s = 1, 2,....

Rewriting the above in the following form


1- rs+12

we obtain


r2 1- IrI2
r1 s+l(1 r 2

By considering the factorization of the dissipation polynomial of
T(z)/X(z) we have

d(z, z-1) = az + (1 Ir2 + J2) + z-1

= 7y(z)+(z 1)

= -(pz + (1 + |p|2) + z-l)

where j(z) = 1 + pz is the stable spectral factor of


d(z, z-1 )

Therefore, a = y7. Also by the result of GERONIMUS [1961, Theorem 8.2]

7 = T (1 Irt 12) .

Combining the above two facts with (4.10) and using (4.15) we obtain

rs+2 r2(1- r1 12)
lim -- = .
s-w rs+l r7 7


r Br
(4.14) lim s+2 rs+l = 0.
s-m rs+1

This shows that as s -> the sequence of parameters satisfies more
accurately a linear recurrence law. Since in general rs s = 1, 2, ...
might take also zero values,we consider the equivalent statement:
for all E > 0 there exists an s such that for all s > s
o -

s+2 rs+l I E max (Ir s+1, r s+21).

This motivates the following:

(4.15) DEFINITION. A sequence R = (rs : s = 1, 2, ...) is said to
be almost rational iff there exists a polynomial p(z) = 1 + plz + ...
+ p z such that for all E > 0 there exists an integer s with
u o0--
the property that

r+u + Brs l + ... + Prs < max (r
s+u 1 s+u-1 Us sgts+u t ,

for all s > s.
A polynomial p(z) with the above properties is said to be an almost
recurrence polynomial for R.

(4.16) THEOREM. Let C be a rational positive sequence, R be the
associated parameter sequence and i(z) be the stable spectral factor

of the associated dissipation polynomial. Then R is almost rational
and -(z) is an almost recurrence polynomial for R.

PROOF. Denote by 7t(z)/Xt(z) the rational
with the usual truncated parameter sequences
functions have power series expansions

C-function associated
R and let these

7t(z) 0 (t)
t(z) = 1 + 2 s t =l 2, ....

The relation between (c s = 1, 2, ... and R is given by the
following formulas:


= rt,

(t) 2 + (t) (t)
c2 rt+(1 r ) + c1 1 '

(t) *
s+1 t+s

l (1 -
= (c

+ (Cl

Irt+u 2) +

/ct) )
.. c(t) )(Tst )-1



T(z) = 1 + Plz + ... + Puzu


(z) b(t) (t) u
X = +b z + ... + z .
t 1 u

By the previous theorem -iim Xt(z) = rq(z).


(4.18) Irt+u + lrt+ul + + purtlI < Irt + b(t) r + +b(t)
""r1u -1 t+u-1 "'" u t

+ Et max +u r
tfisst+u s"

for all s > s (e) and E > 0. The polynomial Xt(z) satisfies

(t) +b t) (t) + ... + b(tct) = 0.
u+l 1 u u 1

From the above and (4.17) we now have

(4.19) Irt+u + b)rt + ... + b (t)r t. m sl ft
t+u 1 t+u-1 u t, t5st+u

where ft is a polynomial function in rt+s1, bt), (det T(t) -1
for s = 1, ..., u, that when viewed as a function of the rs's has
has zero constant term. We shall now show that

(4.20) lim f = 0.

In the case of rational positive sequences no-'(e) is
integrable and therefore by the result of GERONIMUS [1961, Theorem 8.2]
the parameter sequence R is squarely summable. Hence

(4.21) im r = 0.

In view of (4.16) it follows that

lim c( = 0.
t-^c s
s fixed

(Notethat lim c(t) is not necessarily zero.) Then
s-.), t: fixed s

(4.22) Jim det T(t = 1
-> s-1

for all s. Also by Theorem (4.15)

(4.23) im b(t) = s,
_->0o S

for s = 1, ..., u. From (4.21), (4.22), and (4.23) we conclude (4.20).
Finally (4.18), (4.19) and (4.20) imply that for all E > 0 there

exists an So(E) so that

rt+u + Ilrt+u-l+ ... + urtl ESt +urs

for all t > s (e). O

(4.21) REItMARK. In the applied literature on time-series analysis it
has been noted (see for example BOX and JEIIIIS [1970, p. 179]) that
the asymptotic behavior of the partial autocorrelation coefficients of
rational power spectra, that are precisely the SCHUR parameters of our
setting, is "dominated by damped exponentials". However, no precise
statement of this seems to have been proven. Moreover, in case the
almost recurrence polynomial has roots on Izi = 1, the above state-
ment is not absolutely correct. For example take r(z) = 1 z.
Then the sequence of parameters is given by

(- l)t
rt t + 1

The asymptotic behavior of this sequence is not dominated by exponentials.


In this chapter we develop an alternative approach to the study of
pr-extensions of C We focus our attention to pr-extensions of
dimension less than or equal to s. Our key result will be an
implicit description of this set.

In Section 5 we show that for a nonempty open subset of the
data-set of partial positive sequences C there exist no
pr-extensions of dimension strictly less than s. This result justifies
our interest in pr-extensions of dimension s.

After a brief exposition in Section 6 of some basic facts about
the topological degree, we derive in Section 7 our key result: For
any dissipation polynomial of degree less than or equal to s
there exists a corresponding pr-extension of C of dimension
at most s. We should note that according to the results of the
previous chapter this dimension could be as large as 2s.

This result further provides a novel proof of the classically
known fact that the positivity of C is a sufficient condition for
the existence of solutions to the CARATHEODORY problem.

Also, most important, this topological approach provides an
implicit description of a nonuniqueness inherent in this partial
realization problem.

5. Covariance Extensions of Dimension s

The following well known proposition gives conditions for a
rational function to belong to C.

(5.1) PROPOSITION. An irreducible rational function c r(z)/X(z),
with v(z), X(z) in C[z] and c r(0)/X(0) = c R, is in C if
and only if

-1) co -1 -
(a) d(z, z-) :=- (V(z)(z-l) + (z -)x(z))

is a dissipation polynomial, and

(b) 1 (T(z) + X(z)) has no root in z < 1.

We let Cs = (ct : t = 0, 1, ..., s) be a positive sequence and
we consider a rational function c 7(z)/X(z) with power series
expansion in z that begins with

s t
co + 2t ctz t

A rational function with this property will be called a partial
realization of Cs. Thus, a partial realization cor(z)/x(z) of
C is a pr-extension of C if and only if (a) and (b) of
s s
Proposition (5.1) hold.

If cr (z)/X(z) is a partial realization of Cs and r(z), X(z)
have degree less than or equal to s, then r(z), X(z) and
b(z) := (r(z) + X(z))/2 are related via the following nonsingular
linear transformations

(5.2) co(z) = [(co + 2clz + ... + 2cszS)X(z) ]o


(5.5) cob(z) = [(c + lz + ... +c zs)X(z)] o

where [ ] denotes truncating the powers of z outside [O, s].
We want to consider when C admits pr-extensions of dimension
strictly less than s. This is given in the following:

(5.4) LEMMA. There exists a pr-extension of C of dimension
strictly less than s if and only if there exists a polynomial b(z)
of degree less than or equal to s such that for the polynomials
X(z) and r(z) obtained through (5.2) and (5.5) the following hold:

(a') d(z, z-1) = (- ((z)R(z-1) + T(z-1)X(z))

is a dissipation polynomial,

(b') b(z) = i (7r(z) + X(z)) has no root in Iz| < 1, and

(c') r(z), X(z) have a nontrivial common factor.

The essential point is that (b') is a closed condition as
compared with (b) in Proposition (5.1).

PROOF. Suppose b(z), 7(z) and X(z) satisfy the conditions
of the lemma and let To(z), Xo(z) be coprime polynomials such that

T(z) TTo( z)

Then deg TT (z) and deg X (z) are less than s. Also
d (z, z-1) = (T (z)1(z1) + (z-1)X (z))/2 is a dissipation
polynomial and TT(z) + X (z) has no zero in Izi < 1. In order for
wo(z)/Xo(z) to be a pr-extension of Cs we only need to show that
TT (z) + X (z) has no root on Izl = 1.
Suppose v (z ) + x (z ) = 0 for some z0 with Io = 1.

IV(z ) + o() 2 = do(z z) + I (z)2 + IX (z,) = 0.

Since d (zo, z1o) > 0 it follows that ro(zo) = Xo(zo) = 0, which
contradicts the hypothesis that o (z) and Xo(z) were coprime.
Therefore T (z) + X (z) has no root in Jz| < 1 and TT (z)/X (z)
is a pr-extension of C with dimension strictly less than s.
The converse is trivial. D

Consider now Y to be the set of nonnegative sequences
C = (ct : t = 0, 1, ..., s) that for simplicity we assume co 1.

The interior YO of Y is the set of positive sequences C .
We shall show that:

(5.5) PROPOSITION. The set of partial nonnegative sequences C
that admit no pr-extension of dimension strictly less than s is
an open subset of Y.

Clearly, this would also imply:

(5.6) PROPOSITION. The set of partial positive sequences C
that admit no pr-extension of dimension strictly less than s is an
open subset of Y .

PROOF of Proposition (5.5). Denote by X the space of poly-
nomials b(z) of degree less than or equal to s with b(O) = 1.
The subset of X where (b') of Lemma (5.4) holds can be shown to be
compact. Since Y is also a compact space,it follows that the
subset of pairs

(b(z), C ) E X x Y

where (a') to (c') of Lemma (5.4) hold is also compact. The projection
onto Y being a continuous map, implies that the subset of nonnegative
sequences Cs (which by Lemma (5.4) admits a pr-extension of dimension
strictly less than s) is compact. The complement of this set is
therefore open.

The fact that this set is nonempty follows by considering the
partial sequence Cs = (1, 0, ..., 0, 1/2). Clearly, C is
positive and moreover there is not even partial realization of C
of dimension less than s. Hence, there is no pr-extension of C
with dimension less than s either. O

Given Cs, whether there exists a pr-extension of dimension
strictly less than s is a decidable question. It can be answered
by applying the decision methods developed by TARSKI [1951] and
SEIDENBERG [1954] (see also JACOBSON [1974, Chapter V ]) to the

conditions of Lemma (5.4). However, these are very involved and a
simpler criterion is lacking. In fact, in the Appendix we shall
indicate the set of conditions that needs to be tested for the first
nontrivial case.

But, the set of pr-extensions of dimension less than or equal to
s is known to be always nonempty. We focus our study on this set.
We shall use concepts of homotopy and degree theory for this. So
we now make a brief digression and introduce the essentials of
degree theory.

6. Basic Degree Theory

The "degree" of this section refers to a notion of topological
degree soon to be defined. The object of study of degree theory is
the solution set of an equation d = f(b) where f is a mapping
between two topological spaces. The main question concerns the
existence and the number of solutions for a given d.

Let S be an open subset of some topological space X, f a
continuous map from S into a topological space D, and d be a
point in D. The aim of degree theory is to define an integer valued
function deg(f, S, d), called the degree of f at d relative to
S, with the properties that

(a) deg (f, S, d) is an estimate of the number of solutions
of d = f(b) in S,

(b) deg (., ., .) be continuous in the arguments, and

(c) deg (., ., .) be additive in the domain S; i.e.,
whenever Sl S2 = then deg (f, S1U S2, d)
= deg (f, Sl, d) + deg (f, S2, d).

As usual, when S is a subset of topological space X, we denote
by 8, aS, and So the closure, the boundary, and the interior of
S respectively. The exposition below is following NAGUMO [1951],
SCHWARZ [1965, Ch. III], and LLOYD [1978, Ch. I] where we refer for
additional information and detailed proofs.

Let X and W both denote the Euclidean space R with the
usual topology. (The reason for this redundant notation will become
clear below.) The set S is assumed to be open and bounded
subset of X. The maps that we consider are continuously
differentiable in an open subset containing S. The set of such
mappings is denoted by C l() and topologized by the norm

f := sup Ift(b) I+ sup
lbSN sbtN

Is I

Given f C C(S), Zf(S) denotes the set of points
that there exists a point b in f (d) where the
is zero.

Suppose now that f
The degree of f at d

deg (f, S, d) :=

: Cl(S) and d W but d
relative to S is defined

f-d S)
(bCf'l(d) n -

d in W

Sbf(yS)U Z ().

sign Jf(b).

The definition is extended to points do that belong to Zf(S), but
do not belong to f(6S), by letting

deg (f, S, do) = deg (f, S, d)

for any d f(aS) U Zf(S) and d "sufficiently close" to do.

The fact that this is well-defined and the precise meaning of the
term "sufficiently close" are described by the following:

(6.1) THEOREM. Let f
component of W\ f(aS).
to Zf,(). Then

be as above and dl, d2 belong to the same
Suppose also that neither of them belongs

deg (f, S, dl) = deg (f, S, d2).

Jf (b)

A simple consequence of the definition of degree is

(6.2) PROPOSITION. Let d f( S). Then, deg (f, S, d) 0 implies
that d f(S).

A notion that is crucial for the development of the next section
is that of homotopy: A C -homotopy between two elements fo and
fl in Cl(B) is a function

H : S x [0, 1] -RN

such that if Hx denotes the map b i-H(b, x), then Ho = fo,
H1 = fl Hx E C (s) for all x in [0, 1] and also
IHx H yl1 0 as x -y. This last condition says that H is a
continuous function in the parameter x.

The following is a very powerful result that we shall use in the
next section.

(6.5) THEOREM. Let f f be in C (s), and H be a C -homotopy
between f and fl. If d H(_S, x) for all x in [0, i], then

deg (fo, S, d) = deg (fl, S, d).

7. Dissipation Polynomials and Covariance Extensions of Dimension s

In this section we prove the following key result.

(7.1) THEOREM. Let C = (c : t = 0, ..., s) be a partial
positive sequence and

d(z, z-) := dszs + ... + dlz + 1 + d z- + ... + dsZ-s

be a dissipation polynomial (of degree < s). Then there exists a pair
of polynomials (r(z), X(z)) with deg 7(z), deg X(z) less than or
equal to s, and a positive scalar k such that the following two
conditions hold:

(a) co (z)/X(z) is a pr-extension of Cs,

1 -1 1-- -1
(b) kd(z, zl) = ](7(z)x(z1) + r(z )X(z) )

We now elaborate on the implications of the above theorem with
two immediate corollaries.

(7.2) COROLLARY. Consider a partial positive sequence C There
exists always a pr-extension of C with dimension at most equal to s.

In this way we have circumvented the need for the algebraic
machinery of orthogonal polynomials or of interpolation theory in
order to establish that the positivity of C is a sufficient
condition for the existence of solutions to the CARATHEODORY
problem. This is essentially a problem in analysis and an approach
like ours seems to be absent. Furthermore, with this new approach
we obtain some additional information about the set of pr-extensions
of dimension s.

(7.3) COROLLARY. Consider the partial positive sequence
Cs = ct : t = 0, 1, ..., s) where not all of ct, t = ..., s
s t -1
are zero. Then, for almost any dissipation polynomial d(z, z )
of degree less than or equal to s there exists an associated
pr-extension of C with precisely dimension s.

We now proceed to the

PROOF of Theorem (7.1). We again denote by X the space of
polynomials with constant term 1 and with degree less than or equal
to s.

Any b(z) C X defines through (5.2) and (5.5) a unique pair
of polynomials (7(z), X(z)) E X2 such that c7r(z)/X(z) is a partial
realization of C i.e., has power series expansion that begins with

c + 2clz + ... + 2c z .
o 1 s

The correspondence b(z) (Tr(z), X(z)) is certainly
bijective, whereas the correspondence b(z) ->-)(z)/X(z) is
clearly not.

To any pair (7(z), X(z)) as above we associate the
polynomial in z and z :

d(z, z-z + ... + dz1 + d1 + ... +dz
c -o + 1 z

The constant term do equals

d =1 [[X(z)r(z)]s X(z-1) + [I(z )T(z -1)O X()]
o o 0o 2 s o

= [i(z-i )(c + ... + c z X(z)]
-s s

= IIx(z) I| ,

where 1 IIs denotes the norm that Cs induces on the space of
polynomial of degree less than or equal to s (see page 15).
Since X(0) = 1 and Cs > 0, then

do IIx(z) II / 0.
o s

Let W be the space of "symmetric" polynomials

-L s -1 -s
d(z z = d z + ... + d z + 1 + d z + ... + d, z

with constant term equal to 1. With d(z, z-1) as above we define
the map
PC : X ->W: (1 + b z + ... + b zs) s

d d d
O _z + ... + -- z + 1 + z- + ... +--- zs .
Sd a d d
o o o o

Both X and W are Euclidean spaces of the same dimension, and
TCs is continuously differentiable in X.

We now consider two open subsets S C X and P C W, where
S is the subset of polynomials b(z) that satisfy

b(z ) = 0 implies Izo > 1,

and P is the subset that consists of all d(z, z-1) E W that

d(z z ) > 0 for all z on z = 1.

Therefore S is the set of ("stable") polynomials b(z) such that

b(z ) = 0 implies zo > 1,

and P is the set of dissipation polynomials with constant term
equal to the identity.

The statement of the Theorem can be easily seen to be equivalent
to the following: for any d P there exists a b S such that

d = pC (b).

Therefore we need to show that TCs(n) 32 P.

Since the roots of a polynomial depend continuously on the
coefficients (see MARDEN [1966]) it follows that S is open. Also
because the roots of every b(z) in S lie in IzI > 1, it follows
that S is bounded. We shall first show that

(7.4) cpC (S) I P.

For the particular sequence Co = (1, 0, ..., 0) the map
T takes the simple form

PC O : b(z) = 1 + blz + ... + bsz s b(z)i(z- )/to bt 2.

It is straightforward to show that

(7.5) deg (PC S, d) = 1 for any d in P.

In fact the computations can easily be done for d(z, z ) = 1,
and since cp (aS) = P we use Theorem (6.1) to establish (7.5).

The set of positive partial sequences is connected (this is
obvious especially when we consider the SCHUR parametrization;
see page 11). Therefore we can follow a path within the set of
positive sequences from C to any other positive sequence
C In this way we construct a continuous homotopy H(b, x) between
PCso and PCs.
We now show that

(7.6) cpC ()S) (P = for any Cs > 0.

Suppose b(z) E 3S. Then b(z ) = 0 for some z with Iz = 1.
Therefore, if d(z, z ) = Csb(z)E then b(z) 12 = 0 implies

d(zo, z ) = 0.
0 0

Consequently, d(z, z- ) E oP and (7.6) is proven.

If H(b, x) is a homotopy as above, then

H(as, x) n P / i,

for all x in [0, 1]. By Theorem (6.3) we conclude that

deg (cCs, S, d) = 1,

for any C > 0 and any d in P. Hence, by Proposition (6.2) it
follows that

9 (S) D P

for all C > 0. By the compactness of S we also have that


We want to close this chapter with the following:

(7.7) CONJECTURE. The correspondence between dissipation
polynomials d(z, z-1) and pairs of polynomials (T(z), X(z))
with r(O) = X(0) = 1, in Theorem (7.1) is bijective.

The conjecture is certainly true for the trivial sequence
C = (1, 0, ..., 0). We were also able by direct computation of the
s 1
Jacobian to show that it holds in a neighborhood of d(z, z ) = 1.
But a proof is still lacking. We should mention that the map

~Cs is not analytic; therefore

deg (PCs, S, d) = 1

for all d in P does not imply that the cardinality of cpCs(d) ( S
is one.


8. The Matrix Covariance Extension Problem

Given an n-variate, zero-mean, stationary stochastic process
y C Z we denote by

c := EyTT+S, s = 0, 1, ...,

the covariance n x n-matrix-function of y In this chapter we shall
use "~" to denote the "complex conjugate transpose of".

The covariance sequence C = cs : s = 0, 1, ...} is characterized
by the nonnegative definiteness of the block Toeplitz matrices

T = [c ]u s = O, 1, ...,
s t-u t,u=o

where now c it := ct (See for example GIHMANN and SKOROHOD
[1974, p. 196].)

Thus, we define a matrix-sequence

C = (Cs : s = 0, 1, ..., with co Hermitian)

to be positive (resp. nonnegative) iff the associated block Toeplitz
matrices T are positive (resp. nonnegative) definite for all s.
We similarly define the partial matrix sequence
Cs = (ct : t = 0, 1, ..., s) to be positive (resp. nonnegative) iff
T is a positive (resp. nonnegative) definite matrix.

This notion of matricial positivity is again related to an
analytic property of the matrix-valued power series

r(z) := c + 2 sc z.

(8.1) THEOREM (see KOVALISHINAand POTAPOV [1982]). The power series
r(z) converges in Izi < 1, and

P(z) + r(z)

is a nonnegative definite matrix for all z in Iz| < 1 if and only
if the sequence

C = (cs : s = 0, 1, ..., with co = (c + Z)/2)

is nonnegative.

Matrix-valued functions that satisfy the above conditions will
again be said to belong to class C.

The following is now the matrix CARATHEODORY problem: Given a
partial sequence Cs, find necessary and sufficient conditions for
the existence of a matrix-valued C-function with power series that
begin with

s t
c + 2 CZ c .
o + 2tltz

The matrix CARATHEODORY problem seems to have been considered only
recently by IL'MUSKIN [1974], and KOVALISHINA [1974]. See also AROV
and KREIN [1981], DELSARTE, GENIN and KAMP [1979] and KOVALISHINA and
POTAPOV [1982]. As an interpolation problem it can also be approached
through the functional analytic techniques of SZ.-NAGY and FOIAS
[1970]. See e.g., HELTON [1980].

In the next two sections we will consider the subclass of rational
solutions and carry out some of the program followed in the scalar case.

The matrix sequence C is said to be rational iff there exists an
integer v such that for all s > v the block-behavior (Hankel)

B = [c ]
s t+u-l t,u=l

have the same rank. This integer v will again be called the dimension
of C.

The rationality of C is equivalent to P(z) defining a rational
function in z. In this case r(z) can be represented as a matrix
fraction P(z)Q(z)-1 (right) or Q(z)-P(z) (left).

Suppose that C is rational and that P(z)Q(z)-1 is a right
matrix fractional representation of P(z) where P(z) and Q(z)
are right coprime polynomial matrices, i.e. there exist A(z),
B(z) C C xn[z] such that A(z)P(z) + B(z)Q(z) = I. Then it can
be shown that the dimension of C is equal to the maximum of the
degrees of P(z) and Q(z). (In this chapter, "I" will denote
the n x n identity matrix.)

In Section 9 we present a generalization of our topological
approach for the matrix CARATHEODORY problem. We shall draw similar
conclusions as in the scalar case: (a) the positivity of the partial
sequence C is sufficient for the existence of solutions, and (b) for
almost any matrix-dissipation polynomial of degree less than or equal to s
there exists a corresponding rational solution of dimension less than or equal
to s.
In Section 10 we shall give a brief account of the basic results
that come out of the algebraic approach and SCHUR's algorithm when
applied to this matrix-interpolation problem.

9. The Topological Approach

We begin by establishing the matrix version of Proposition (5.1).

(9.1) PROPOSITION. A rational function P(z)Q(z)-1 with Q(z),
P(z) in Cnxn[z] and P(O)Q(0)-1 Hermitian positive definite
is in C if and only if

det [Q(z ) + P(z )] = 0 implies 1z 0 > 1,


D(z, z-) := Qz-)P(z) + (Z1)Q(z)}

has nonnegative definite values for all z such that Izl = 1.

As in the scalar case, we call D(z, z ) the dissipation (matrix-)
polynomial of P(z)Q(z)-1. By a slight abuse of our terminology, we

shall also call any matrix polynomial in z and z satisfying
the above property a dissipation polynomial.

PROOF. An n x n matrix-function S(z) is said to be in
class S iff it is analytic in Izl < 1, and

I S(z)STz) > 0

for all z in Izl < 1. These functions are considered in operator
theory (see SZ.-NAGY and FOIAS [1970]) where they are called
"contractive" and in circuit theory (see BELEVITCH [1968]) where
they are called "bounded". The relation between C-functions
r(z), with r(0) Hermitian positive definite, and S-functions
S(z) is given by (see for example [DELSARTE, GENIN, and KAMP
[1979, p. 39])

(9.2) s(z) = 1(r(o) r(z))(r(o) + -1,


(9.5) r(z) = (I zs(z))(I + zS(z))-lr(o).

So we let r(z) = P(z)Q(z)-l. Without loss of generality we
can assume that r(0) = I. Then we obtain

S(z) = (Q(z) P(z))(Q(z) + P(z))-1

and also that, S(z) is in S if and only if

det (Q(zo) + P(zo)) = 0 implies Izo > 1,


I S(z)S(z) > 0,

for all z in Izl < 1. By the maximum modulus principle it is
sufficient to test this on the boundary of the region of analyticity:


I S(z)S(z") = 2(Q(z-l) + P(z-1))-D(z, z1)(Q(z) +P(z))->0

for all z on Izl = 1. Clearly this holds if and only if

D(z, z"1) > 0. O

(9.4) THEOREM. Let Cs = (ct : t = O, ..., s) be any partial
--- s t
positive n x n-matrix sequence and
z-1 s d -l -S
(9.5) D(z, z-1 : d z + ... + d z + I + dz + ... + d z

be an n x n-matrix dissipation polynomial. Then there exists a
pair (P(z), Q(z)) E (C xn[z])2 with deg P(z), deg Q(z) less
than or equal to s, and a positive definite matrix K such that
the following two conditions hold:

(a) P(z)Q(z)-l is a pr-extension of Cs,

(b) K/2D(z, z-1Kl/2 = ((z-)P(z) + (z-)Q(z)}.

(With ( )1/2 we denote the "Hermitian square root of".)

This theorem establishes that when the partial sequence Cs is
positive, the matrix CARATHEODORY problem is solvable. Our technique
does not seem to be possible to extend to the singular case when
C is only nonnegative. However, it provides information about the
solutions of dimension s, precisely as it did in the scalar case:
For a generic set of dissipation polynomials of degree less than or
equal to s, we can associate pr-extensions of Cs of precisely
dimension s. For the complement of this set we can associate
pr-extensions of dimension less than s.

The idea of our proof is similar to the one we gave for the scalar
case. However, certain new features require the use of a more
sophisticated technique. The main new aspect is that, in contrast to
the scalar case, matrix polynomials with no determinental zeros in

zI < 1 and with constant term the identity matrix, do not form
a bounded subset of the space of the coefficients. For example,
the polynomial

1 0) + z 1/2 a
\0 1/ 0 1/2)

has nonvanishing determinant for all Izl < 1 and all values of
a as well. We circumvent this by considering our stability set
on a certain compact manifold.

PROOF. Let P(z)Q(z)-l, with P(z)Q(z) in nxn[z], be a
partial realization of C of dimension less than or equal to s.
Then P(z) and Q(z) are related by

(9.6) P(z) = [(co + 2c1z + ... + 2cszs)Q(z)]S.

Clearly, Q(0) is nonsingular. We now define the polynomial B(z) by

(9.7) B(z) = [(co + clz + ... + cszs)Q(z)]s

and the polynomial

1-1 -S-
(9.8) D(z, z-) = D z + ... +iz + + + ... + z-

:= (Q(z-)P(z)+ P(zl)Q(z)}
-i -

in z and z By Proposition (9.1), P(z)Q(z)- is in C if and
only if det B(z) has no roots in IzI < 1 and D(z, z- ) is
nonnegative definite for all z on Izj = 1. We notice that the
pair (Q(z), P(z)) is defined up to a right unimodular factor.
Therefore, so is B(z). Moreover, det B(O) 0.

Thus, we consider the space X of polynomials

{B(z) = B + B z + ... +
0 ]

B zs, B(z) Cn[z]}
S --

of degree less than or equal to s. In this space we consider the
subset M defined by

t=o tBt I,
B > 0, and B upper triangular.

This subset is a smooth compact manifold of real dimension 2sn2. That
M is a smooth manifold follows from the open condition det B 0 0.
Compactness follows from the fact that,by the first condition,any entry
of Bt, t = O, ..., s has modulus less than or equal to 1. (It is
also easy to show that M is orientable, but we will not need this
fact here.)

The correspondence between B(z) in M and partial
tions P(z)Q(z)-l of C with dimension less than or equal
is clearly surjective. If

Q(z) = Q + Qz + ... + Qs

then the polynomial D(z, z~1) obtained by (9.8) satisfies
then the polynomial D(z, z-) obtained by (9.8) satisfies

D= (Q= Q, ... s)

Since det Qo e 0
the following map




and C > O, it follows th
is well-defined:
is well-defined:

to s


S 0 S

at D > 0. Therefore

PC : M -, W
-1/2( -1) -1/2
B(z) 0 D o z, z -0 Y,

c -1

where W is the space of the polynomials D(z, z-1) as in (9.5);
i.e., such that D(z, z ) = (z ,-1 z) and with constant term equal
to the identity matrix.

Consider the submanifold S of M of polynomials with
determinant nonvanishing in Izl < 1, and the submanifold P of W
of polynomials D(z, z-1) that have nonnegative definite values for
all z on Izl = 1. To complete the proof of the theorem we need to
show that for all C > 0 we have


Precisely as in the scalar case it can be shown that

cp ( qs) p = P I.

Also,the set of positive partial matrix sequences C is pathwise
connected. By using the homotopy invariance property of the degree,
the proof that was given for the scalar case works in this case also.
More precisely pCs is certainly a continuous map between manifolds.

(iCs is only continuous because we require taking the Hermitian square
root.) Now MILNOR [1965] defines the degree for C -mappings between
manifolds. However, as remarked by LLOYD [1978, p. 32] the definition
immediately extends to the continuous case simply by taking C1
approximations. (An explicit argument can be found in SCHWARTZ [1965,
Chapter V] and LLOYD [1978, Chapter I].) Finally, we note that as
before the degree deg (cpCs, S, D) can be seen to be 1 by considering
the point

D(z, z-) =

and the trivial sequence

C = (I, O, ..., 0).

The proof now proceeds precisely as in the scalar case.

10. The Algebraic Approach

The description of all solutions to the matrix CARAMHEODORY
problem can be found in AROV and KREIN [1981] and KOVALISHINA and
POTAPOV [1982]. It is given in the so-called "completely nondegen-
erate case", when Cs is a positive sequence. In the general case,
when C is nonnegative but not positive,there exists no closed
form expression for the solutions. However, some standard techniques
in operator theory can be used to deal with this case (cf.
SZ.-NAGY and FOIAS [1970, p. 188]).

In this section, having presented our topological approach, we
wish to give a brief account of the basic results and ideas of the
algebraic approach, which essentially relies again on SCHUR's
algorithm. We shall apply this to the case of rational C-functions.

We begin by describing the SCHUR's algorithm for the case of
matrix C-functions. The main technical fact is given in the

(10.1) LEMMA. Let rt(z) be a matrix-valued function which has a
power series expansion around the origin that begins with I + 2rtz,
where I rtr > 0. Then r (z) is in C if and only if there
exists a C-function Pt+ (z) such that

(10.2) t(z) = [at(z)rt+l(z) + bt(z)][ct(z)Pt+l(z) + dt(z)]-1


at(z) = (I rtrt)-1/2(I rt) + z(I trt) /2 ),

bt(z) = (I rtt 1(I + rt) z(I rtrt) (I + t)

ct(z) = (I rtt)1/2(I rt) z(I trt)-/2( rt),

(z) = (I r -/2 + r) + z( -1/2 +
d (z) =(I rtrtY (I + rt) + z(I r r ) (I + rt).

PROOF. The SCHUR's recurrence relation for the matrix case
is given by (see for example DELSARTE, GENIN, and KAMP [1979, (36)])

St(z) =(I- rtrt)-1/2 (rt+ Z 1(z))(I + zrtSt+(z))-1(I rr 1/2

Assuming that I trt > O, then St(z) is in S if and only if
St+l(z) is in S. Applying the bilinear transformation (9.2) we
obtain the corresponding recurrence relations for the class
C-functions. 0

Formula (10.2) can be solved for t+,(z) in terms of rt(z) and
provides an inductive procedure for solving the CARATHEODORY
problem in the completely nondegenerate case.

In the completely nondegenerate case a matrix-version of (2.9)
can also be obtained (see for example AROV and KREIN [1981]).

Here we shall apply the lemma to rational C-functions
Pt(z)Qt(z)-, and consider the behavior of the dimension and the
dissipation polynomial under the action of "truncating" the sequence
of SCHUR parameters rt or, equivalently, as t increases. We
now have the following:

(10.3) THEOREM. Let rt(z) and rt+,(z) be in C and related
as in Lemma (10.1). Then rt(z) is rational if and only if Pt+1(z)
is rational. In this case, there exist right coprime representations
rt(z) = Pt(z)t(z)-1 and t+l(z) = Pt+1(Z)Qt+l(z)-l with Pt(z),
aQ(z), Pt+l(z), t+l(z) in Ct

(10) Pt(z) at(z) bt(z) Pt+l(Z)
(10.4) =2
t(z) ct(z) at(z)) Qt+(z)/

Also, if Dt(z, z1 )(resp. Dt+1(z, zl)) denotes the associated
dissipation polynomial

(10.5) Dt(z, z-1) = Dt+l(z, z-1

Moreover, the following are equivalent:

(a) dim Pt(z) = dim rt+l(z),

(a') dim rt(z) = deg Dt(z, z-),

(a") dim Pt(z) = deg (Pt(z) + Q(z)).

For the proof we need the following:

(10.6) LEMMA. The following identity holds:

-tn ad i e t (z) +b(z)

(t((z-1) h t c e matrzz) d(z)

PROOF. By direct computation. O

We now proceed to the

PROOF of Theorem (10.3). Suppose r,+1(z) is a rational
C-function and is equal to Pt+ (z)Qt+,(z)- where Pt+1(z),
Qt+1 (z) are right coprime matrix polynomials. Define Pt(z), Qt(z)
via (10.4). It can be checked that ct(z)Pt+1(z) + dt(z)Qt+l(z)
is invertible as a power series in z. By Lemma (10.1) it follows
that rt(z) = Pt(z)Qt(z)-1. Furthermore, it holds that Pt(z),
Qt(z) are right coprime.
Indeed, since Pt+ (z), Qt+l(z) are right coprime,there exist
polynomial matrices A(z) and B(z) such that
A(z)Pt,+(z) + B(z)Qt+1(z) = I. Hence, from Lemma (10.6) and (10.4),
we obtain that there exist polynomial matrices Al(z) and Bl(z)
such that

Al(z)Pt(z) + 3B(z)Qt(z) = z2I.

But both det Pt(O) and det Qt(0) can be checked to be
different from zero. Therefore, Pt(z) and t(z) are in fact
right coprime.
The converse follows similarly by considering the identity

(10.7) z2 t+l =
Qt+(z) (z) at(z) t() z)

that follows from Lemma (10.6).
Relation (10.5) follows by considering

D,(z, z-l) l(Qt(z-)pt(z-)) Pt)t

and applying Lemma (10.6).
Finally, the equivalence of (a), (a'), and (a") can be shown
as in the scalar case (see Proposition (3.5)). 0

Thus, the precise analogues of certain facts that were seen
to hold in the scalar case, apply to the matrix case as well. We
expect that the results of Section 5 extend to the matrix case
also, and that the matrix dissipation polynomial determines the
asymptotic behavior of the matricial SCHUR parameters.


In this final chapter we want to discuss the relevance of the
covariance extension problem to the applied area of time-series
modeling. This area involves a large number of issues that we shall
not touch upon (e.g., issues of statistical nature, see BOX and
JENKINS [1970], or, of the essential difference between prediction for
time-series and prediction for stochastic processes, see
FURSTENBERG [1960]). Instead we shall consider as our point of
departure, the knowledge of a partial (sampled) covariance sequence
Based on these data, certain schemes have been proposed that
yield a unique rational covariance extension for C These schemes
form the base of modern nonlinear methods for spectral estimation
(cf., HAYKIN [1979, pages 36 and 103]). We begin Section 11 by
considering the so-called "maximum entropy" (ME) method in the context
of our previous development.

The ME method proposes the use of a particular pr-extension of
C that has constant dissipation polynomial. The constancy of the
dissipation polynomial makes the construction of a corresponding
stochastic realization trivial (since the problem of spectral fac-
torization is avoided altogether). Moreover, this construction
turns out to be recursively updated as the data set increases. This
latest property is precisely the recurrence relation satisfied by
the orthogonal polynomials and was established in this context by
LEVINSON [1947]. Recursiveness is very important in practical appli-
cations as it provides an efficient approximation procedure. In
point of fact, this is the underlying philosophy in the ladder
structure constructions in modern digital filter design.

However, the absence of zeros in the power spectrum obtained
by the ME method gives rise, in certain cases, to undesirable
phenomena (see HERRING [1980] and the references therein). Motivated
by the need for more general pole-zero approximating techniques, for

the covariance function of stochastic processes, DEWILDE, VIEIRA,
and KAILATH [1978], and DEWILDE and DYM [1981a and 1981b] (see
also RUCKEBUSH [1978] and ROSENCHER and CLERGET [1979]) have placed
the problem in a more general context of Nevanlinna-Pick interpolation
theory. However, these investigations do not seem to illuminate
the basic partial realization problem where the data is simply
C Pole-zero modeling in the context of partial realization setting
remains "a nonlinear and implicit problem, and there is no possibility
for recursively updated realizations of increasing order" (see

In Section 12 we shall indicate that this might not be precisely
so. Certainly, as it appears from our results of Chapter III, an
essential part in obtaining pr-extensions of C with nontrivial
dissipation polynomial,is in obtaining information about the
dissipation polynomial or, equivalently, the zeros of the
corresponding power spectrum. (Our results of Chapter III, in
particular Theorem (4.15) suggests that the parameter sequence might
be used for that. This is a point that requires further investigation.)
We should note that this information is already assumed in the
approximation theories of DEWILDE and DYM [1981a and 1981b]. Now,
provided such information is available we shall indicate a way that
this can be incorporated in the modeling process in an efficient
way. Theorem (12.4) will describe a recursive construction for
pr-extensions that have approximately fixed zero-structure.

11. The ME-Method and Some General Discussion

Let C be a partial positive (scalar) covariance sequence and
assume c = 1. The simplest possible choice for an admissible extension
of the partial parameter sequence R is certainly the trivial extension
(rs+t : rs+t = 0 for t = 1, ...). This extension amounts to choosing
rs+1(z) = 1 in Theorem (3.1). The associated pr-extension of Cs
is simply

s C (z)*
X Z) T S (Z)
X~ z) ~(z)* *

This particular extension has a certain uniqueness property,
namely, it maximizes

e := tlim It (z)*112

i.e., the distance in the II Ii-norm of 1 from the closure of
the manifold spanned by positive powers of z. It is immediate
from (1.6) that in this case

lkt(z)*l2 = Il1s(z)*l2,

for all t > s, and hence,

e = -lim lDt(z)*ll2 = ls(z) *12.

We now would like to explain the importance of this quantity in
the prediction theory of stochastic processes. The inner product (.,*)
that was defined in Section 1 relative to a nonnegative sequence C,
can be extended from the space of polynomials in z to a more general
space of functions on Izl = 1. This is done by realizing this inner
product via a Stieljes integral

(a(z), b(z)) = Jf a(z)b-do(e(0), z = exp je,

with a(e j), b(eje) e L2[do(e)] the Hilbert space of squarely
integrable functions on Izl = 1 with respect to the measure dc(0)
(a(;) being the spectral distribution function of C, cf., Remark
(5.8)). Let now y, T Z, be a (zero-mean, stationary) stochastic
process having C as covariance sequence. Let also L2(y) denote
the Hilbert space generated by y with the inner product defined by


(f, g) := Efg,

f, g E L2(y), and E denoting the expectation operator. Then,
the mapping


extends to an isometry between L2[da(e)] and L2(y) (cf., GRENANDER
and SZEGO [1958, p. 175]). Via this mapping,any linear problem in
L2(y) can be translated into one in L2[dc(e)] and conversely. Let
now gs(z) be any polynomial in z with gs(0) = 1 and degree less
than or equal to s. Clearly, due to the orthogonality properties

Ils(z)*II2 = gsZ) lg (z) 12.


e = lim (z)*2 = lim inf E y E ay 2
t t- t t-oo au o u=1 -u

is the square of the variance of the prediction error of y at
an instant T = 0 based on observations in the past T < 0. And
this is maximized by the choice s+(z) = 1 over all pr-extensions
of the partial data C .

The quantity e. is an essential characteristic of a stochastic
process and describes the "predictability" of the process. One can
show that (see GERONIMUS [1961, p. 1581)

e= t= l (1 I rt)

exp 17 If in a-'()de).
27~ -r a

The stochastic process is called deterministic iff e = 0, and
nondeterministic otherwise. This quantity is directly related to a notion
of entropy rate of a stochastic process in the sense of Shannon
(see HAYKIN [1979, p. 80]), and this gives the name to the method.

On the basis of the above it has been argued that the so-obtained
pr-extension of C is maximally noncommital to the unavailable
data (see JAYNES [1968]). This is indeed so, as far as the prediction
problem is concerned. However, the prediction should be more of an
"excuse" than a "reason" (see FURSTENBERG [1960, p. 7]).
In point of fact, KALMAN [1981] argues that the
partial sequence of parameters contains certainly more information
than merely the fact that these parameters are all of modulus less
than 1. In KALMAN [1981] it is also suggested that some minimal
dimension pr-extension is perhaps the right object to consider.
Unfortunately, the description of the minimal dimension pr-extensions
seems to face intractable difficulties. In point of fact, an equi-
valent question was considered by YOULA and SAITO [1967] in a circuit
theoretic context. Currently, this problem is unsolved. In Appendix
A we shall indicate some computational difficulties that arise in
the simplest nontrivial case.

12. On Pole-Zero Modeling

We begin by assuming knowledge of a number of "influential
zeros" in the power spectrum of a stochastic process. This rather
loose term appears to have a rather definite meaning in the more
application-oriented literature, See for example MAKHOUL [1976,
p. 115]. It is also stated that pole-zero modeling is not simple
and not well-understood. We shall present a simple recursive way
to incorporate the "zero" information in the modeling process.
Let C be a partial positive sequence and d(z, z1 ) be a
given dissipation polynomial of degree less than or equal to s.
From the results of Section 7 we know that there exists an associated
pr-extension of C of degree less than or equal to s.

In principal this pr-extension can be found as a
preimage of d(z, z-) under PC. Certain techniques have been
recently developed to provide constructive algorithmic procedures
for obtaining a solution of homotopy-based existence results (see
KELLOG, LI, and YORKE [1976], and also MEYER [1968]). However, this
is very cumbersome and objectionable for almost all practical
purposes. Thus, we shall not pursue it here but instead, we shall
develop an approximate but efficient solution.

Our first tool is a new representation for partial realizations
of C of dimension less than or equal to s. Let C be a positive
s s
sequence, and Yt(z), (t(z), t = 0, 1, ..., s be the orthogonal
polynomials of C .

(12.1) LEMMA. Any rational function c 7(z)/x(z), r(z), X(z) C[z],
with 7(O) = X(O) = 1 and power series expansion that begins with

(12.2) c + 2c z + ... + 2c zs
(12.2) c 1 s

is of the form

S-~ '(z)* + a z'Y -(z)* + ... + a z? (z)*
7T(Z) s 1 s-1 s o .
Co o s0 (z)* + aze 1s(z)* + .+ a szs(z)*

This result was independently of ours obtained by KIMURA [1985]
who also argues that it provides a canonical form for partial realization
of covariances. In our work a precise use of this lemma is given in
Theorem (12.4). We should also mention that this lemma is really a
fact about partial realizations and has essentially nothing to do with
positivity. Positivity is assumed for the sake of some other properties
of this representation that we shall soon discuss.

PROOF of Lemma (12.1). The polynomials 7r(z) and x(z) are
related through

c r(z) = [X(z)(c + 2c z + ... + 2c zs)].
0 0 1 s 0

This represents a nonsingular transformation between polynomials of degree
less than or equalto s. The two sets of polynomials (z -t(z)*, t = 0,
1, ..., s) and (z -tOt(z)*, t = 0, ..., s) form bases for this
space and they are related by

cozs-tct(Z)* = [-tot(z)*(co + 2clz + ... + 2c zs)],
0 0S 0

for t = 0, 1, ..., s, as it follows from the definition
of Tt(z), t = 0, 1, .... The proof of the lemma is now immediate. 0

This lemma places a system of coordinates, in the linear space whose
points represent partial realizations of (12.2), so that the ME
solution lies at the origin. Another aspect of this representation
is shown in the following:

(12.3) LENNA. Let

T(z) = 's(z)* + alz's-1(z)* + .. + szs (z)*,


X(z) = s(z)* + a zs -,(z)* + za + zs (z)*.

The polynomial (in z and z )

d(z, z-1) = (7F(z)x(z-1) + X(z)7(z-1)

has degree t < s if and only if as = = = 0.

PROOF. This follows from the fact that the degree of

Ts(z) t(z-1) + s(z))T(z-1)

is equal to is ti. This can be shown using Lemma (2.6). o

We now proceed to our final

(12.4) THECREM. Let C = (ct : t = 0, 1, ...1 be the covariance
sequence of a nondeterministic stochastic process, and let

T(z) = 1 + alz + ... + auz

be any polynomial with roots in (Iz > 1). Then,for s sufficiently

S(s)() s(z)* + az (z) + *.. + azU -s(z)*
S(s)(z) s()* + a s- u s-u
c := C
S(s)(z) o (z)* + alze 1(z)* + ... + auz (z)
s 1 -1 u s-u

is a pr-extension of Cs, and if
spectral factor of the associated

q(s)(z) denotes the stable
dissipation polynomial, then

lim 1 ( (z) = h(z).

PROOF. Let Mt(z) be as in Theorem ( 3.1). Then

(s (s) (z)

s/-U(z)* + a zYS-U(z)* +..
Mu()* 1 u-1
Ss-u(z)* + a ZS-U(Z)* + ...
u 1 u-1

+ a zu-z)
u 0
+ az Z)
u o

where tYs-(z), 5-u(z) are the orthogonal polynomials of the
parameter sequence Rsu = (rs-u+l' .. r }.
U Ssu+1 s

Since C corresponds to a deterministic process, it follows
from the result of GERONIMUS [1961, p. 159] (see (4.6)) that the
parameters of C are squarely summable. Hence, as s -4o, both

't (z)* 1

s-UD (
et (z)* -+ 1

for t = 0, 1, ..., u. Consequently, as s -* o

s-u (s-u Us-U(z).

zu (z)* + alZoul(z)* + ... + aZ)
u 1 u-1 u 0o

tends to 1 uniformly on Jzl < 1, and the associated dissipation
polynomial tends to Ti(z)i(z-l). Applying now Theorem (3.1), the
proof is complete. O


Here we shall consider the following problem: Given a partial
covariance sequence C find (simple) necessary and sufficient con-
editions on C so that it admits a pr-extension of dimension strictly
less than s.

This question is certainly the first one in attempting to obtain
an explicit answer to the minimal dimension problem of KALMAN [1981],
and YOULA and SAITO [1967]. We shall discuss the first two simple
cases: s = 2, and s = 5. The case s = 2 is trivial as it requires
conditions for the positivity of a degree 1 polynomial in z and
z The case s = 5, that requires conditions for the positivity
of a degree 2 polynomial in z and z presents already
difficulties due to the implicit nature of the conditions that seem
to be possible to approach only with the techniques of decision
methods (see JACOBSON [1974, Ch. VDT. Although this appears to be
quite elementary, it should be noted that conditions for the
first case only exist in the current literature (see
KALMAN [1981], and also KRISHNAPRASAD [1980] in an equivalent setting).

We begin by considering the case s = 2. Unless cl = 0, the
minimal dimension partial realization of

1 + 2c z + 2c2z2

is of dimension 1 and given by

V(z) 1 + (a + c )z
X = 1 + (a cl)z

where c2 = c1(C1 O). (Here and in the sequel we use the represen-
tation introduced in Lemma (12.1).) Applying Proposition (5.2) it is
straightforward to check that r(z)/X(z) is in C if and only if

cl < 1, and
(A.1) 1
_a I l |c|.

We now consider the case s = 5. In case c1 = c2 = 0 and c / 0,
then the minimal pr-extension is of dimension 3. In case c3 = clc2,
then the minimal pr-extension is either of dimension 1 or 3 depending
on whether the minimal partial realization which is of dimension
1 is in C or not. In the generic case we consider a general
rational function 7(z)/X(z) with r(z), X(z) polynomials of degree
2 and 7(0) = X(0) = 1, that has power series that begins with

2 15
1 + 2clZ + 2c2 z2 + 2c 3.

We shall restrict our attention to the case where all the scalars
take real values, and use in the various expressions the associated
parameters (rl, r2, r } instead of ([1, c2, c ). The function
r(z)/X(z) is of the form

Y2(z)* + azY 1(z)* + pz2 o(z)*

S2(z)* + azel(z)* + pz2 o(z)*

where a and 0 satisfy

(A.2) (1 r )(1 r2)r + a(l r2)r2 + prl = 0.

The conditions

Y(z) + X(z) 0 O for all z in jzj < 1,


v(z))(zl ) + T(z )X(z) > 0 for all z on |zl = 1,

give rise to the following:

(A.5) a(l + rl) < P + (1 + rl(l- r2)

(A.4) Q(l rl) < 0 + (1 rl)(l r2)

(A. ) a(1 + r) < + (1 + rl)(l + r2)
(A.6) c(l r) < + (1- rl)(l + r2)
(A.7) < 1
and either
(A.8) 48 < ja(l r2) + B + prlr2 ,

(A.9) 2(l r2 ) r2r2 + 2apBrr( r+)
2 2
48[82 28 + (1 r)(1l r2)] 0.
In general,neither of the above conditions is redundant. In the
(a, B)-space, these conditions cut out a set that corresponds to
pr-extensions of C2 of dimension 2. This set is illustrated below
as a shaded area for the two typical cases.


r2 <1 I+ I

Ir2 1 r
r21 > 1 + Irl> -



Whether C3 admits a pr-extension of dimension 2, depends on
whether the line i given by (A.2) intersects the shaded region.
Due to (A.9) which is implicit in a and P and is of degree 4,
these conditions when expressed in terms of the original parameters
of the problem, e.g. rl, r2, and r are also implicit and
extremely involved.



[1965] The Classical Moment Problem, Oliver and Boyd Ltd.,
Edimburgh and London, 255 pages.


[1962] Some Questions in the Theory of Moments, American
Math. Society, Providence, R. I., 265 pages.


[1974] "Recursive algorithm for spectral factorization",
IEEE Trans. on Circuits and Systems, CAS-21:742-750.

D. Z. AROV and M. G. KREIN

[1981] "Problems of search of the minimum of entropy in inde-
terminate extension problems", Functional Analysis and
its Applications, 15(3):123-126.


[1968] Classical Network Theory, Holden-Day, San Fransisco,
440 pages.


[1981] "AR and ARMA identification algorithms of Levinson type:
An innovations approach", IEEE Trans. on Automatic
Control, AC-26:1243-1260.

G. E. P. BOX and G. M. JENKINS

[1970] Time Series Analysis: Forecasting and Control, Holden-
Day, San Fransisco, 555 pages.


[1978a] "Orthogonal polynomial matrices on the unit circle",
IEEE Trans. on Circuits and Systems, CAS-25:149-160.


[1978b] "A simple algorithm to spectral factorization", IEEE
Trans. on Circuits and Systems, CAS-25:943-946.

[1979] "Schur parametrization of positive definite Block-
Toeplitz systems", SIAM Journal on Applied Mathematics,


[1981a] "Schur recursions, error formulas, and convergence
of rational estimators for stationary stochastic
sequences", IEEE Trans. on Information Theory,

[1981b] "Lossless chain scattering matrices and optimum linear
prediction: the vector case", Circuit Theory and
Applications, 2:135-175.


[1978] "On a generalized Szeg6-Levinson realization algorithm
for optimal linear predictors based on a network
synthesis approach", IEEE Trans. on Circuits and
Systems, CAS-25:663-675.


[19531 Stochastic Processes, John Wiley, New York, 654 pages.


[1978] Operateurs Rationnels Positifs: Application a 1' Hypersta-
bilitd et aux Processus Aldatoirs, Dunod, Paris, 294 pages.


[1982] "A lattice algorithm for factoring the spectrum of a
moving average process", Proceedings of the Conference
on Information Sciences and Systems, Princeton, New
Jersey, pp. 5-9.


[1960] Stationary Processes and Prediction Theory, Princeton,
New Jersey, Princeton University Press, 283 pages.


[1959] The Theory of Matrices,vol. 2, Chelsea, NewYork, 276pages.


[1982] "Partial realization of covariance sequences", Center
for Mathematical System Theory, University of Florida.


[1954] "Polynomials orthogonal on a circle and their appli-
cations", American Math. Society, Translations,
1(2): 253-304.

[1961] Orthogonal Polynomials, English translation from
Russian, Consultants Bureau, New York, 242 pages.


[1974] The Theory of Stochastic Processes I, Springer-Verlag,
Berlin, 570 pages.


[1958] Toeplitz Forms and their Applications, University of
California Press, Berkeley, 245 pages.


[1979] Nonlinear Methods of Spectral Analysis, Springer-Verlag,
Berlin, 247 pages.


[1964] Lectures on Invariant Subspaces, Academic Press, New York,
130 pages.


[1980] "The distance of a function to He in the Poincare
metric; Electric power transfer", Journal of
Functional Analysis, 38:273-514.


[1980] "The cause of line splitting in Burg Maximum-Entropy
spectral analysis", IEEE Trans. on Acoustics Speech
and Signal Processing, ASSP-28:692-701.


[1974] Candidate's Dissertation, Ul'yanovsk.


[1974] Basic Algebra I, Freeman, San Fransisco.


[1968] "Prior probabilities", IEEE Trans. on Systems Science
and Cybernetics, SSC-4:227-241.


[1974] "A view of three decades of linear filtering theory",
IEEE Trans. on Information Theory, IT-20:146-181.


[1979] "On partial realizations, transfer functions, and
canonical forms", in Acta Polytechnica Scandinavica,
Mathematics and Computer Science Series, 31:9-32.

[1981] "Realization of covariance sequences", Proceedings
of the Toeplitz Memorial Conference, Tel Aviv,
pp. 331-342.


[1969] Topics in Mathematical System Theory, McGraw-Hill, NewYork,
358 pages.


[1976] "A constructive proof of the Brouwer fixed-point
theorem and computational results", SIAM Journal on
Numerical Analysis, 13:473-483.


[1983] "A canonical form for partial realization of covariance
sequences", Osaka University, Technical Report 85-01,
25 pages.


[1974] "J-expansive matrix-valued functions in the Caratheodory
problem", Doclady Akademia Nauk Arm. SSR, 9(3):129-155.


[1982] Integral Representation of Hermitian Positive Functions,
private translation by T. Ando, Hokkaido University,
Sapporo, Japan, 129 pages.


[1977] The Markov Moment Problem and Extremal Problems, American
Math. Society, Providence, R. I., 417 pages.


[1980] "On the geometry of linear passive systems", in Algebraic
and Geometric Methods in Linear Systems Theory, edited by
C. I. Byrnes and C. F. Martin, American Math. Society,
Providence, R. I., pp. 253-275.


[1947] "The Wiener RMS (Root-Mean-Square) error criterion in
filter design and prediction", Journal of Mathematical
Physics, 25:261-278.


[1978] Degree Theory, Cambridge University Press, Cambridge,
172 pages.


[1975] "Linear prediction: A tutorial review", in Proceedings
of the IEEE, 64:99-118.


[1966] The Geometry of the Zeroes of a Polynomial in a Complex
Variable, American Math. Society, Providence, R. I.,
183 pages.


[1968] "On solving nonlinear equation with a one-parameter
operator imbedding", SIAM Journal on Numerical Analysis,


[1965] Topology from the Differentiable Viewpoint, the University
Press of Virginia, Charlottesville, 64 pages.


[1978] "Covariance characterization by partial autocorrelation
matrices", Annals of Statistics, 6: 643-648.


[1951] "A theory of degree of mapping
analysis", American Journal of

based on infinitesimal
Mathematics, 73:485-496.


[1972] "Partial realization of random

systems", Automatica,


[1979] "Approximation rationelles des filtres", Annals T41ecom.,


[1978] "Sur 1' approximation rationelle des filtres", Report
35, Ecole Polytechnique, France.


[1966] Real and Complex Analysis, McGraw-Hill, New York, 412 pages.


[1976] "The factorization problem -- A survey", in Proceedings
of the IEEE, 64:90-95.


[1967] "Generalized interpolation in H ", Transactions of
the American Math. Society, 127:179-203.


[ 1965] Nonlinear Functional Analysis, Courant Institute of Mathe-
matical Sciences, New York University, New York, 246 pages.


[1954] "A new decision method for elementary geometry",
Annals of Mathematics, 60:565-374.


[1972] "A solution to the matrix factorization problem", IEEE
Trans. on Information Theory, IT-18:225-232.


[1959] Orthogonal Polynomials, American Math. Society, Colloquium
Publications, Providence, Rhode Island, vol. 23, 452 pages.


[1970] Harmonic Analysis of Operators on Hilbert Space, North-
Holland Publ. Comp., Amsterdam, 387 pages.


[1951] A Decision Method for Elementary Algebra and Geometry, Univer-
sity of California Press, Berkeley, California, 65 pages.


[1956] Interpolation and Approximation by Rational Functions in
the Complex domain, American Math. Society, Colloquium
Publications, Providence, Rhode Island, vol. 20, 382 pages.


[1965] "On the fitting of multivariate autoregressions, and the
approximate canonical factorization of a spectral density
matrix", Biometrika, 50:129-134.


[1965] "Recursive solution to the multichannel filtering problem",
Journal of Geophysical Res., 70:1885-1891.


[1961] "On the factorization of rational matrices", IRE Trans.
on Information Theory, IT-7:172-189.


[1978] "Bauer-type factorization of positive matrices and the
theory of matrix polynomials orthogonal on the unit circle",
IEEE Trans. on Circuits and Systems, CAS-25:57-69.


[1967] "Interpolation with positive-real functions", Journal
of the Franklin Institute, 284:77-108.


Tryphon Thomas GEORGIOU was born on October 18, 1956, in
Athens, GREECE, to Thomas GEORGIOU and Georgia GEORGIOU. He
obtained his Diploma in electrical and mechanical engineering
from the National Technical University of Athens, in 1979.

I certify that I have read this study and that in my opinion it
conforms to acceptable standards of scholarly presentation and is fully
adequate, in scope and quality, as a dissertation for the degree of
Doctor of Philosophy.

Dr. E. W. Kamen, Chairman
Professor of Electrical Engineering

I certify that I have read this study and that in my opinion it
conforms to acceptable standards of scholarly presentation and is fully
adequate, in scope and quality, as a dissertation for the degree of
Doctor of Philosophy.

PA/wr9 KkL^jovCe^ _-
Dr. P. P. Khargonekar, Co-Chairman
Assistant Professor of Electrical

I certify that I have read this study and that in my opinion it
conforms to acceptable standards of scholarly presentation and is fully
adequate, in scope and quality, as a dissertation for the degree of
Doctor of Philosophy.

Dr. T. E. Bullock
Professor of Electrical Engineering

I certify that I have read this study and that in my opinion it
conforms to acceptable standards of scholarly presentation and is fully
adequate, in scope and quality, as a dissertation for the degree of
Doctor of Philosophy.

Dr. R. L. Long
Assistant Professor ofqathematics

I certify that I have read this study and that in my opinion it
conforms to acceptable standards of scholarly presentation and is fully
adequate, in scope and quality, as a dissertation for the degree of
Doctor of Philosophy.

Dr. A. R. Tannenbaum
Associate Professor of Mathematics

This dissertation was submitted to the Graduate Faculty of the College
of Engineering and was accepted as partial fulfillment of the requirements
for the degree of Doctor of Philosophy.
August, 1983.

Dean, College of Engineering

Dean for Graduate Studies and Research

11 111111 lll I t5lillll ll lltil i il tIII111111 1111
3 1262 08558 9563