UFDC Home  Search all Groups  UF Institutional Repository  UF Institutional Repository  UF Theses & Dissertations   Help 
Material Information
Subjects
Notes
Record Information

Full Text 
PARTIAL REALIZATION OF COVARIANCE SEQUENCES By TRYPHON THOMAS GEORGIOU A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 1985 ACKNOWLEDGEMENTS I wish to express my gratitude to all those who have guided my steps during my years in Gainesville. Professor R. E. KALMAN has been a constant source of encourage ment and inspiration during my entire graduate work. I am particularly grateful for his continuous effort and interest in teaching me System Theory. Without the financial support which he arranged for me this work would not have been possible. I wish to express my most sincere appreciation to Professor E. W. KAMEN, the chairman of my supervisory committee. His continuous encouragement and support were most valuable to me in the completion of this work. This dissertation grew out of a joint research with Professor P. P. KHARGONEKAR. He, as the cochairman of my supervisory committee, has been an invaluable advisor, and as a coworker and fellow student, has been a dearest friend. My warmest thanks and deepest appreciation go to Professor A. R. TAinrEiEAUTl. His enthusiastic encouragement and confidence I will always remember and be grateful for. I am most grateful to Professor T. E. BULLOCK for his invaluable guidance and support from the beginning of my graduate studies. I also wish to express my most sincere appreciation to the other members of my supervisory committee, Professors G. BASILE and R. L. LONG, for many valuable discussions and help in the course of my studies. My stay in Gainesville has been a most significant experience for me. A doctorate is a slow educational process and in this process I gratefully acknowledge the guidance and help of Professors A. C. ANTOULAS, B. DICKINSON, E. EMRE, and J. HAMMER. I am especially grateful to Ms. Eleanor Onoda for her kind and untiring help in administrative affairs and together with Mrs. Patty Osborn for their excellent typing of this dissertation. I would also like to express my deep appreciation to my friends Bilent, Jaime, Kameshwar, Nikos, and Theodoros, for their help and encouragement. Words cannot tell how much I owe my wife Efi, my parents, my sister and especially my grandmother. To them I dedicate this dissertation. I wish to acknowledge partial financial support by the US Army Research Grant DAAG2981K0156 and US Air Force of Scientific Research Grant AFOSR810238 through the Center for Mathematical System Theory, University of Florida, Gainesville, Florida, 32611, USA. TABLE OF CONTENTS ACKNOWLEDGEMENTS . . ... ...... ii ABSTRACT . . . v CHAPTER I. INTRODUCTION ................... .. 1 II. CERTAIN CLASSICAL APPROACHES . 5 1. Orthogonal Polynomials: An Algebraic Approach 6 2. Interpolation Theory: A Function Theoretic Approach . .... ... 15 III. RATIONAL COVARIANCE EXTENSIONS . 25 5. Rational Covariance Extensions and the Dissipation Polynomial .. 26 4. Asymptotic Properties of the Spectral Zeros. .. 37 IV. A TOPOLOGICAL APPROACH . . 47 5. Covariance Extensions of Dimension s 47 6. Basic Degree Theory . .... .51 7. Dissipation Polynomials and Covariance Extensions of Dimension s . .. 53 V. THE MATRIX CASE. ... . 59 8. The Matrix Covariance Extension Problem 59 9. The Topological Approach . 61 10. The Algebraic Approach . 67 VI. APPLIED ASPECTS OF THE COVARIANCE EXTENSION PROBLEM 71 11. The ME Method and Some General Discussion 72 12. On PoleZero Modeling .. ... 75 APPENDIX. THE :*1!ItMAL DIMENSION PROBLEM . ... 80 EEFREIICE . .. ..... ... 84 BIOGRAPHICAL SKETCH . .. .. 92 Abstract of Dissertation Presented to the Graduate Council of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy PARTIAL REALIZATION OF COVARIANCE SEQUENCES By TRYPHON THOMAS GEORGIOU August 1983 Chairman: Dr. E. W. Kamen Cochairman: Dr. P. P. Khargonekar Major Department: Electrical Engineering This work is concerned with rational covariance extensions of partial sequences. Certain methods of the classical interpolation theory are exploited and a novel topological approach is developed. An associated polynomial, that we call the "dissipation polynomial", is found to be a free parameter for covariance extensions with dimension bounded by the number of data. A similar result holds for the case of matrix sequences as well. The dissipation polynomial is found to impose an "almost recurrence" law on the SCHUR parameters of rational covariance sequences. This is done via a new approach to spectral factorization. These theoretical results, placed in the context of the applied area of spectral estimation theory, suggest some recursive procedure for polezero modeling. CHAPTER I. INTRODUCTION The elementary notion of positivity of real numbers has found various generalizations to that of quadratic forms, operators etc. These play a key role in, not only mathematics, but many areas of applied science as well. The reason is that positivity is, in one form or another, intimately related with the manifestations of physical quantities and entities. The motivation for this work arises from the area of stochastic processes and identification theory. The covariance function cs := Ey +s, s = 0, + 1, ..., of a discretetime, zeromean, stationary stochastic process y T Z, is characterized by the nonnegative definiteness of the Toeplitz quadratic forms u v Z E ac b s=o t=o s st t' for u, v = 0, 1, ..., and as, bt E C. This is the notion of positivity which plays a central role in our work. A stochastic process is an abstraction. The real object is a realization of the process: a timeseries. The probabilistic behavior is not available. For identification and prediction, estimates of the means and covariances of the stochastic process have to be computed from some observation record. In this, a large number of issues are involved. Most of them are of a statistical nature (confidence limits, etc.). Theoretically, most of these issues are still terra incognita. In this work we shall not be concerned about such questions but we shall assume as the given data a partial sequence of covariances Cs := (ct : t = 0, 1, ..., s). This is the standing assumption for the theoretical development of several modern spectral estimation techniques (cf. HAYKI;l [1979]). These techniques seek a certain extension of Cs to a covariance sequence C = (ct : t = 0, 1, ...). The set of all covariance extensions of C can be described by several alternative approaches that we shall recapitulate in Chapter II. However, theoretical as well as practical interest lies with a certain subclass, the rational ones. A sequence C = (c : s = 0, ...) is called rational iff there exists an integer v such that the rank of the behavior (Hankel) matrices B := [c ] s := t+ult,u=l satisfies rank B < v for s = 1, 2, .... The smallest such v will S  be called the dimension of C. Rationality is directly related to existence of finite dimensional stochastic realizations of C. The dimension of C is then precisely equal to the minimal dimension for the corresponding statespace (see for example FAURRE, CLERGET and GERMAIN [1978]). Moreover, in case C is a rational sequence there exists a finite positivity test. In the case of unconstrained rational extensions of a partial sequence, called partial realizations, a minimal dimension can be found by testing linear dependence. In case the minimal dimension partial realization is not unique, this set is parametrized by a linear space. These concepts have given rise to the elegant partial realization theory of KALMAN [1979]. In the case where the extension is also required to satisfy the covariance property the problem becomes substantially more involved (see KALMAN [19811). The purpose of this dissertation is, in broad terms, to elucidate the relation between rationality and positivity. The main goal is to unveil smae of the issues and the nonuniqueness involved in extending the partial data C to a rational covariance sequence C, paying special emphasis on the dimension of these extensions. We believe that we are reasonably successful and that certain of our results can be profitably considered in the applied area of identification. We now give a brief description of the contents of each chapter. More detailed introductory remarks are provided at the beginning of each chapter. Chapter II is devoted to a review of certain classical techniques and concepts that are directly related to the covariance extension problem. In Chapter III we use the algebraic machinery developed in Chapter II, to study rational covariance extensions. It turns out that a certain polynomial is intimately related with the extension and can be chosen arbitrarily. This polynomial, that we call the dissipation polynomial, represents the zeros of the power spectrum. We found that the dissipation polynomial determines in a precise way the asymptotic behavior of some important sequence of parameters that is associated to a covariance sequence. In order to obtain this result we developed a new technique for spectral factorization. In Chapter IV we develop a new approach that is most suited for describing the covariance extensions of dimension bounded by the number of data. In point of fact, in a large number of cases this dimension coincides with the minimal dimension. The merit of this approach lies also with the fact that it provides a novel topological proof of a certain classical result: the positive definiteness of the quadratic form associated with C is sufficient for the s existence of covariance extensions. The key result of this Chapter further shows that an essential nonuniqueness in this partial realiza tion problem is best described in terms of the associated dissipation polynomials. Chapter V extends some of the earlier development to the case of matrix sequences. Finally, in Chapter VI we discuss the relevance and potential of the above in modeling. The initial motivation and part of this dissertation grew out of a joint work with KHARGONEKAR (GEORGIOU and KHARGONEKAR [1982]). The material of Chapters II and III is based upon this work. We close this introduction with a few words on notation and standing assumptions. Throughout this dissertation, we shall work with the field of complex numbers C : C[z] will denote the ring of polynomials in z with coefficients in C ; C n[z] will denote the ring of n x nmatrix polynomials in z;  denotes complex conjugation and if p(z) E C[z], then p(z) denotes complex conjugation on the coefficients of p(z). We shall be dealing with both infinite sequences C = (c : t = 0, 1, ..., ct C for t > 1 and c R) and finite sequences C := (c : t = 0, 1, ..., 0 = s t s, ct EC for t = 1, 2, ..., s and co R). A sequence C is = 0  said to be positive (resp. nonnegative) iff the Toeplitz matrices s T = [c i s = 0, 1, .. s tu t,u=o' where we define ctl := Ct, are positive (resp. nonnegative) definite for all s. A partial sequence C is said to be positive (resp. nonnegative) iff T is positive (resp. nonnegative) definite. s Thus, a sequence C is a sequence of covariances of a stochastic process if and only if C is nonnegative. Finally this notion of positivity (resp. nonnegativity) will be denoted by > 0 (resp. > 0). CHAPTER II. CERTAIII CLASSICAL APPROACHES The purpose of this chapter is to introduce certain mathematical concepts and techniques that are pertinent to the covariance extension problem. We shall begin with some algebraic aspects of the theory of orthogonal polynomials relative to the unit circle. Toeplitz matrices, of the same type as the ones associated with the covariance function of a stationary stochastic process, were classically considered to induce an inner product on the space of polynomials. The special Toeplitz structure can be effectively exploited by considering a particular orthogonal basis. This gave rise to the theory of or thogonal polynomials of SZEGO [1939]. Since that time the theory was progressively developed by many researchers. AKHIEZER [1965], GERONIMUS [1954], [1961] and GRENANDER and SZEGO [1958] have given classical expositions on the subject. It was early recognized that the theory of orthogonal polynomials had strong connections with a prediction problem in the theory of stochastic processes (see GRENANDER and SZEGO [1958, p. 175] or the survey paper by KAILATH [1974]). This opened up areas of application of the theory, notably in stochastic problems, spectral analysis and autoregressive modeling (see the book by HAYKIN [1979] for various applied and theoretical aspects on these). Motivated by autoregressive modeling for multi variate stochastic processes, WHITTLE[ 1963], and WIGGINS and ROBINSON [1965] laid the first pieces of a theory of orthogonal matrix polynomials. A number of researchers have then pursued this line of research. We mention only the most recent works of YOULA and KAJANJIAN [19781, MORF,VIEIRA and KAILATH [1978] and especially DELSARTE, GENIN,and KAMP [1978a] that have given a rather elegant account of the theory of orthogonal matrix polynomials on the unit circle. We restrict our attention to the scalar case and in Section 2 we give a concise exposition of those aspects of the theory that we con sider to be relevant to the covariance extension problem. The theory of orthogonal polynomials is connected to certain problems in analysis. Various aspects are discussed in AKHIEZER [1965], AKHIEZER and KREIN [1962] and KREIN and NUDEL'MAN [1977]. In particular there is a connection with a certain interpolation problem that is equivalent to the covariance extension problem. Both solvability conditions and a parametrization of all solutions can be obtained by the classical SCHUR's algorithm. The machinery of orthogonal polynomials can be used to provide a compact description of the solutions in terms of a linear fractional transformation. This is the content of Section 3. We should finally mention that a similar description of an isomorphic problem was used by DEWILDE, VIEIRA and KAILATH [1978] and also DELSARTE, CENINI and KAMP [19791. 1. Orthogonal Polynomials: An Algebraic Approach We consider an infinite sequence C = (cs : s = 0, 1, ..., with c in R and c in C for s > 0). We define on the o = s  space C[z] of polynomials in z an inner product by u v u v (t at, z b z ) := t S atb ct, t=o t s=o s t=o s=o t s ts This inner product is definite if and only if C > 0. Whenever C > O (resp, > 0) then the above inner product defines a norm (resp., seminorm) on C[z] that we shall denote by 11.1I. We begin by discussing separately the two cases of interest: first the case C > 0, and second the case C > 0 but not > 0. We now consider C to be a positive sequence. The inner product (, *) is now definite. We apply the standard orthogonal ization procedure to the natural basis (zs : s = 0, 1, ...) of C[z] to obtain an orthogonal (but not necessarily orthonormal) basis of monic polynomials ( s(z) : s = 0, 1, ...). These poly nomials are known as the orthogonal polynomials (of the first kind) associated to the sequence C, and are given by o(z) = 1, (1.1) (s(z) = det ... sC s C ... s+l 0 s+1 s ... z /det Ts1, s = 1, 2, ... S~'JL2 Since IIs(z)I2 = (zs, (s(z)), it follows that (1.2) js((z)l2 = det Ts/det Ts_1, s = 0, 1, ..., where det T1 := 1. The special inner product structure of C[z] induces upon the set of orthogonal polynomials certain algebraic identities and an important parametric description. We shall now discuss these. Let Ps(z) C[z] be of degree s. We define the reverse polynomial SzS 1 P (z)* := z S (z). From (z s(z)) = 0 for t = 0, 1, ..., s 1 structure of the inner product it follows that and the Toeplitz (1.5) (zt, s (z)*) = 0 for t = 1, 2, ..., s, and any s > 1. Since s (z) is a monic polynomial of degree s, we can write s1 (1.4) s(z)* = 1 Zt bs,t t(z), for some scalars b From (1.5) and the above we obtain s t" 0o  c1 Cs1 cs2 z Hence, bst = (1, zDt(z) )/lt(z) I12 gives rise to the parameters is independent of s. This fact rt+1 := (1, zot(z))/lt(z) I2, t = 0, 1, .... These parameters are known as the SCHUR parameters of the sequence C. From (1.4) we now obtain the (well known) recurrence identities s(1) (z) = z s1(z) ;s s1(z)*, (1.5) (is(z)* = s l(z)* rs z s(z), for s = 1, 2, .... From the first identity we obtain lls (z2) 2 s() 12 s1s(z)*, zes1(z)) and also 0 = (ze (z), es1(z)*) rs ll1(z) 112. Combining the two we obtain (1.6) lks(z) 12 = (1 Irs 2) lls_(z)112, This shows that the parameters R = (rs : s s = 1, 2, "** s = 1, 2, ...) satisfy (1.7) Irs < 1, s = 1, 2, . These conditions (and co > 0) are equivalent to C being positive. In fact starting from the parameter sequence R with Irt < 1 for all t, and c > 0, we may construct a corresponding positive sequence. This correspondence is bijective. Furthermore, partial O = (1, zDt(z)) bs, t(s), Dt(z)). positive sequences Cs = (ct : t = 0, 1, ..., s, correspond bijectively to pairs (Co, R ) where Rs = (rt : 1, 2, ..., s with Irtl < 1 for all show below. with T > 0) c > 0 and . This we t). This we The sequence of parameters of a positive sequence C is givenby r = C /co r T = ( 1)s det s s C c 0 c _s+2 c  s . csl Cs+. /det Tsl1 for s = 2, 3, ..., and they satisfy Irsj < 1, s = 1, 2, *** Conversely, starting from the rs < l, s = 1, 2, ..., and c sequence C by solving (1.8) for (19 s1  (1.9) Cs = cors t9l (i rt 2) sequence of parameters that satisfy > 0, we obtain the corresponding c: s + (cl ... Cs) Ts2 sc,1 C / (ccl for s = 1, 2, *** this follows from identity SThe above is Irt < 1 for valid provided det T2 0. But t = 1, 2, ..., and the algebraic det T r2 det Tt det T t det T t t1 t2 t = 1, 2, ..., (which arises from (1.2) and (1.6)). Hence, (1.8) and (1.9) establish the required bijective correspondence. We now discuss the singular case of nonnegative sequences that are not positive. Such a sequence is called singularly nonnegative. (1.8) Cl Assume that C f (0, 0, ...) is a singularly nonnegative sequence and let s be the smallest (positive) integer for which det T = 0. s The partial sequences (It(z) : t = 0, 1, ..., s) and Rs are defined as earlier and (1.1) (1.6) hold for t = 1, ..., s. However, we now have s (z) I2 = 0 and R satisfies (1.10) Irt <1, t=l, 2, ..., s 1, and Irs = 1. The interesting feature of this singular case is that any singu larly nonnegative sequence C, as above, is rational and uniquely determined by Cs (or, equivalently, by co and Rs). Also any partial sequence C with T > 0 and det T = 0 det Ts admits a uniquely defined singularly nonnegative extension C. We now prove these facts. Let C be singularly nonnegative and ,. s s1 X(z) = z + az +... + a be a monic polynomial of least degree that satisfies IIX(z) 2 = 0. Evidently, s is the smallest (positive) integer such that det T = 0. Furthermore, X(z) = 's(z). Clearly, IztX(z) I2 = 0 for all t > 0. By the fact that T +t 0 it follows that (0 ... 0 as ... a 1)' t is a zeroeigenvector of Tst and therefore (1.11) c = ac ... a for t > 0. st 1st+l s t  This shows that C is a rational sequence and is in fact uniquely determined from the partial sequence C . S We now let Ct be such that T > 0 with det T = 0. We shall show that there exists a nonnegative extension C of C which by the s above discussion is unique. The partial sequence Ct defines in the obvious way a seminorm and an inner product on the space of polynomials of degree less than or equal to t. We denote these by i11t and (*, )t respectively. Consider now s to be the smallest integer for which det T = 0 and X(z) = z + alz + ... + as be a monic polynomial of least degree that satisfies Ix(z) 2 = 0. Precisely as we did before we now obtain that (1.12) c _s_ ac s+l ... a u for u = 1, ..., t s. We now extend the partial sequence Ct to an infinite one C using (1.12) for u = t s + 1, .... We now show that C > 0. For any a(z) in C[z] denote by a mod X the remainder of a(z) divided by X(z). Since (zu, X(z)) = 0 for all u > 0 it follows that (a(z), a(z)) =(amod X, a mod X) =(a mod, a mod X), > 0. Therefore C > 0. The above considerations readily solve the covariance extension problem: Given a partial sequence Cs, there exists a nonnegative extension C of C if and only if C > 0. In particular, the following two cases are possible: (a) Nondegenerate case: C > 0. In this case the set of all nonnegative extensions of C s are in bijective correspondence with sequences of parameters R that are either finite and of the form R = (rt: t = s + 1, ..., s + u, with Irtl < 1 for t < s + u and rs+u = 1) or infinite satisfying Irt < 1 for t = s + 1,... t (b) Degenerate case: C > 0 but Cs 0. In this case there exists a uniquely determined nonnegative extension C. We now proceed to consider certain related mathematical objects, which will be useful in the next section in showing the connection of the above with an interpolation problem. Define the power series 0 t r(z) := + 21 ctz The function theoretic properties of P(z) will be described in the next section. Here we view P(z) as an algebraic object. We shall now recall the notion of partial realizations (see KALMAN [1979]) and then introduce the socalled orthogonal polynomials of the second kind of C by considering certain partial realizations of (z = + 2t c zt. t1 o t Consider a formal power series F(z ) (in negative powers of z). A pair of coprime monic polynomials (r(z), X(z)) or, equivalently, the rational function r(z)/X(z) is said to be a partial realization of F(z1) of order s iff [F(z1)X(z)z sde]+ = (z)zsdeg, where [ ] denotes "the polynomial part of". Equivalently, the rational function 7r(z)/X(z) is a partial realization of F(z1 ) of order s if and only if the Laurent series 7(z)/X(z) (with the division carried out in the field of formal Laurent series in negative powers of z) matches 1 S F(z ) up to and including the coefficient of z We now consider the power series T'(z) and define a sequence of polynomials by Ps(z) := [F z1) ), where s (z) is the sth orthogonal polynomial of the sequence C. The integer s runs over either all nonnegative integers or, a finite number of them depending on whether C is positive or non negative,as we discussed earlier. These polynomials are known as the orthogonal polynomials of the second kind of the sequence C and were introduced by GERONIMUS [1961, p. 10]. From the definition of Ys(z) we have that Co s(z)/$s(z) (though not necessarily a coprime representation) is a partial realization of '(z ) of order s. In the case where C is singularly nonnegative and s is the smallest integer for which det Ts = 0, it follows that F(z1) is actually equal to c s(z)/os(z). We now show that the orthogonal polynomials of the second kind satisfy the following recurrence identities (which, except for a sign change are the same as (1.5)): Ss(z) = zs (z) + ri s (z)*, ss1 s sl (1.13) s(z)* = :sl(z)* + rszs!(z), for s = 1, 2, ... (or a finite index sequence in case C is singularly nonnegative). We define the transformation 1 t f: C[z] [z]: A(z) [A(z)(c + 2t c z] 0 We also define a sequence C via the relation (o O + t ^t 'o (c+2 Co z )(c +2 c2tzt) = 1, and the transformation f: C[z] ,C[z]: A(z) c[A(z)(c + 2t c tzt)]. co 0 t=1t + A 0 It is straightforward to check that ff = ff is the identity transformation. We now modify our earlier notation by adding a subscript to Toeplitz inner products (and norms) that specify the defining sequence. Thus we shall have (., *)C, (., .)", and if we define I = 1,0, 0, ...} then we shall also write (., *) . We denote by g* the adjoint of the transformation g. We now have that <(.1  <' j(f + f*).)I, and also ^1^^ ^ ^.+fA (f, f.) (., 2 f*(f + f*)f.) (, (* + f).) *). Since if*iC = .II^ is certainly a seminorm, it follows that C is nonnegative. Since the polynomials Ys(z), s = 0, 1, ..., were defined by Ys(z) := fs(z) s = O, 1, ..., they are the orthogonal polynomials of the first kind of the sequence C. Therefore they satisfy relations of the type (1.5). In order to establish the precise form (1.13) of these relations we only need to show that (1.14) os(o) = s(O). From the definition of s (z) we have is(z) = [(2tl C_ z t)s(z)] Ds(Z). o The constant term of 0 t o [(2tJ1 c zt)ms(z)]+ equals (2/co)(1, s (z)) = 0. Therefore (1.14) holds and hence (1.13) follows. We finally want to indicate another algebraic identity that is satisfied by the two types of orthogonal polynomials that we shall refer to later on: (1.15) O (z) s(z)* + D (z) s(z)* = 2zshs, where h is a scalar given by s s 12 ) 1 ,D ( z) 11 2 (1.16) hs : :r = ls(z) 2. This identity follows from (1.5) and (1.13) using induction on s. 2. Interpolation: A Function Theoretic Approach Consider the following interpolation problem: We are given two regions G and G in the complex planes of the variables z and z w w, and a set of pairs (za, w ) with z in G and w in Gw for all a in a certain index set I. It is required to find function F(z) holomorphic in G with values in G that satisfies the z w interpolation constraints F(z ) = w for all a in I. a a When certain of the points z coincide, then the interpolation a constraints are modified so as to assign at these points values to the successive derivatives of F(z). This problem is classical and a number of techniques have been applied to it. The books by WALSH [1956] and AKHIEZER [1965] give comprehensive expositions of the classical approaches to the problem. In recent years new functional theoretic techniques have been applied that also extend to a more general class of interpolation problems that includes interpolation with matrixvalued functions. These techniques have been developed in the work of SZ.NAGY and FOIAS [1970] and SARASON [1967]. In this section we shall consider a particular case of the problem which is directly related to the covariance extension problem. We discuss the socalled SCHUR's algorithm that also provides a description of the solutions, and we tie up this approach with the material of the previous section. The following classical theorem states that the notion of positivity encountered earlier is expressed in terms of a function theoretic property. (2.1) THEOREM (see AKHIEZER [1965, p. 178]). The power series r(z) := c + 2cz + ... + 2c z converges in Jzj < 1 and has Re P(z) > 0 for all z in Izl < 1 if and only if the sequence C = (cs : s = 0, 1, ..., with co := (c + c)/2) is nonnegative. The functions possessing the above property form the socalled class C. (This same property is known in the engineering literature as positive realness. See for example BELEVITCH [1968, p. 71].) We shall consider the following interpolation problem: Given a partial sequence Cs = (ct : t = 0, 1, ..., s) findthenecessaryandsufficient conditions for the existence of a function in C whose power series expansion in z begins with c + 2c z + ... + 2c zs. Also, it is required to describe the set of all solutions. This is known as the CARATHEODORY problem. In viewof Theorem(2.1) it is seen to be equivalent to the covariance extension problem. Below we proceed to discuss the socalled SCHUR's algorithm as applied to the CARATHEODORY problem. This technique provides a para metrization of the solutions in terms of functions in class C. The main technical result needed is the following simple (2.2) LEMMA. The function ra(z) belongs to C and has power series expansion in z that begins with 1 + 2ca z if and only if one of the following two conditions holds: (a) Ic(a) < 1 and d (z) a(z) ba(z) b(z) c (z) (z) + a (z) a a a is in C where aa(z) := (1 + z)( ca) ba(z) := (1 z)(l c a)), c(z) := (1 z)(l + c(a) and d (z) := (1 + z)(l + c ), 1 + c(a) z (b) Ic(a) = 1 and r (z) = 1 1  a (a) 1 c z INDICATION OF THE PROOF. The set of functions S(z) that are analytic in Jzl < 1 and satisfy IS(zo) < 1 for all z in lzl < 1 forms the socalled class S. There exists a simple relation between functions of class C and functions of class s: r(z) is in C if and only if (2.5) S( z) 2 r( z) + rgf is in S. For functions of class S it is easier to show an analogous statement (see AKHIEZER [1965, p. 101]): A function Sa(z) is in S if and only if one of the following two conditions holds: (a') Sa (0) < 1 and sS a(z) S (0) Sb(Z) := 2 1 WTQTs(z) is in C, (b') Sa is constant of modulus equal to one. Applying now (2.3) to the above statement proves the lemma. 0 In point of fact this lemma gives a description of all functions in C in terms of certain parameters: Beginning with a function P(z) := rl(z) = 1 + 2c( z + ..., we iterate the formula dt(z)rP(z) bt(z) (2.) t+1(z)= ct(z)rt(z) + at(z) ' for t = 1, 2, ..., while Ic(t) / 1. Then, P(z) belongs to C if and only if one of the following two cases holds: (a) Ic(tI < 1 for all t, (b) Ict) < 1 for t = 1, ..., s 1 and (s) l+c (s Ps(z) = s with l) = 1. 1 c( z (1t) (See also AKHIEZER [1965, p. 103].) The parameters pt := c ' t = 1, 2, ... are called SCHUR parameters of r(z). The above lemma readily solves the CARATHEODORY problem: Given (1) the partial C = (1, c, ..., c ) define ct := ct, t = 1, ..., s. By the lemma a function in C exists having power series expansion that (1+ 2cl) (+ 2c1) s begins with 1 + 2c, + ... + 2c z if and only if either L s (1) (1) (1) t Ic 1) 1 and c a= (c1 ) for t = 2, ..., s or c() < 1 and there exists a Cfunction with power series that begins with 2c(2 z + ... + 2c(2)zs 1+ s1 where c(2), t = ..., s 1 are obtained via the formula al(z)(1 + 2c(l)z + + 2c(1)zs) bl(z) c(z)(l + 2c1)z + ... + 2c 1z) + dl(z) (2) (2) si s S1 + 2c z + ... + 2cs z + O(zs) (where the division is carried out in the field of formal Laurent series in positive powers of z). In this way the problem can be transformed to an equivalent one with one interpolation constraint less. This inductive procedure is known as the SCHUR's algorithm. Iterating the above we obtain: The CARATHEODORY problem is solvable in precisely the following two cases: (a) Nondegenerate case: Ic t) < 1 for t = 1, ..., s. In this case the general solution is nonunique and is obtained from at(z)rPt+(z) + bt(z), (25) t() ct(z)Pt+l(z) + dt(z) for t = s, s 1, ..., 1 and s+1 anarbitrary function in C. (b) Degenerate case: Ic t) < 1 for t = 1, ..., u 1, with 1 (u) (u) t u < s, c() = 1 and cU = (c)) for t = 1, ..., s u + 1. In this case the solution is unique and is obtained from (2.5) iterating for t = u 1, ..., 1 with 1 + c(U) z r (z) = 1 1 c z The property that a function r(z) belongs to C is described in terms of the parameters t := ct), t = 1, 2, ..., and also interms of the parameters rt, t = 1, 2, ..., that occur in the recurrence rela tions of the orthogonal polynomials of a corresponding (by the Theorem (2.1)) sequence. These two sets of parameters turn out to be equivalent. In the rest of the section we shall show this which gives the precise connection of the SCHUR's algorithm with the material of the previous section. We first prove the following: (2.6) LETMA. Let R := (rs : IrsI < 1, s = t, t + 1, ...) be a sequence of parameters and (It(z), st(z), s = 0, 1, ...) denote the associated orthogonal polynomials. Define A (z) := (z) + yt(z)*, Bt(z) := st(z) (z)*, Ct := (t(z) t(z)*, D (z) := ( (z) + t(z)*, and A t(z) B t(z) s s M(z) (:= 1 s s algebraic identity holds: The following t (z)\ Pt+s (z) (2.7) s+u ( Mt () , t (z) s t+s( z) PROOF. We apply induction on u. For u = 0, (2.7) obviously holds. Assuming that it holds for u = v we obtain s +v+l( Z)t \s+v+l(Z) where applying on of Mt(z). But 0 Mt(z) simply means to apply on the entries s 0 oS) t(z) M ) ). We finally obtain +u+l( z) = Mt(z) (stt+u+ th s \s+u+l(z) which completes the proof. In the case with Ir s < 1 for 1 < t < v, where for s 0 < s, 1 R < v o< is a finite sequence (r : s + 1, ..., v, and r v = 1) then (2.7) still holds u, and s + t + u < v. Let now Pt(z) be the functions in C with Pt(0) = 1 that correspond through (1.9) and Theorem (2.1) to the parameter sequences Rt that we defined in the lemma. We shall show the following: (2.8) PROPOSITION. Provided Ir I < 1 for u = 1, ..., s + t, then the following two identities hold: St+s t+v \ vt+s (Z) t / z)*) z)* A t()* s+t(z) (2.9) rt(z) = C (z)*rs+t(z) Dt(z)*Pt(z) (2.10) rs+t(z) = Ct(z)*rt(z) + B t(z)* s + Dt(z)* S  Bt(z)* s + At(z)* s for all t > 1, s > 0. 1 PROOF. We first consider the case where R = (rs : s = 1, 2, ..., with irs < 1 for all s) is an infinite sequence. We shall show that both sides of (2.9) have the same power series expansion in z. (Both polynomials and power series are considered as elements in the field of formal Laurent series in z.) We first show that (Ct(z)*s+t (z) + Dt(z)*)1 exists. Indeed, S S the coefficient of the zeroterm is () (0) + (0)* + (0) = 2 / 0. s s s s From Lemma (2.6) we have that t +(z)* (2.11)  sut (z)* s+u At(z) *s+t(z) */s+t(z)* + B (z)* s U Ut s C t(z) *s+t Z) */0s+t(z)* + D (z)* s u u s Using the above it is straightforward to show that At(z)*Ps+U(z)+ B (z)* s s Ct (Z)*rs+t(z) + D (z)* s s t z) Dt (z)* s+u o(s+u+l), O(z) for all u > 0. By the definition of the orthogonal polynomials of the second kind we also have that rtz) P (z) s+u s +(z)* s+u  O(z++l ) for all u > 0. This establishes (2.9). In the case where RI is a finite sequence the above are still valid for u < v for some maximal v such that r s+t+v =1. But then Pt(z) = +(z)*/Ot (z)*, and s+v s+v rs+t(z) = t(z)*/ t (z). v v Therefore, by (2.11), it follows that (2.9) holds. The identity (2.10) follows from (2.9) when solved for r +t(z) provided the denominator in the right hand side of (2.10) is not identically zero. This we show below. From the definition of the orthogonal polynomials of the second kind we have that (2.12) t(z)*rt(z) + D(z)* = 0(zs+). s s From the above and (1.15) we obtain (2.1) ) (z) + t(z) = s s = t(z)t(z) + t ()* t(z))/ t(z)* + (zS+1) s 8 s S S t s s+l = ht z + O(z ), t s 2 where h := (1 Irt+2). By adding (2.12) and (2.13) we obtain Ct(z)*rt(z) + At(z)* = h zs + 0(zs+l) / 0, since ht 0. O s We finally show: (2.14) PROPOSITION. If r(z) denotes a function in C with r(0) = 1 and rt(z), p Pt(z), rt are defined as before for t = 1, 2, ... (finite or infinite), then rt(z) = rt(z) and Pt = rt for all t. PROOF. We apply induction on t. By definition rl(z) = r(z) = Pl(z) and therefore pl = rI. Suppose tr(z) = t(z) for some t. Then t = rt and hence, at(z) = At(z)*, t t bt(z) = BI(z)*, ct(z) = C (z)*, and dt(z) = D (z)*, as well. In case Iptl = Irtl = 1 then both sequences have terminated and we are done. If ptl = Irtl < 1 then from (2.9) and (2.10) we t+l conclude that Ft+l(z) = r t+(z). (2.15) REMARK. As we mentioned earlier, interpolation ideas have a strong connection with circuit theory. For example, the celebrated DARLINGTON synthesis procedure is the analog of the SCHUR's algorithm for solving the general NevanlinnaPick interpolation problem. Several of these connections were pointed out and shown explicitly by DEWILDE, VIEIRA, and KAILATH [1978]. In point of fact, in that paper they derived a compact description for the solutions of a SCHUR interpolation problem that is similar to (2.9) (see DEWILDE, VIEIRA, and KAILATH [1978, p. 668] and also DELSARTE, GEIFIII and KAMP [1979, p. 40]). The various forms of interpolation can be interpreted in a circuit theoretic framework as synthesis with cascade connection of coupling networks. In the same framework the linear fractional transformation (2.9) is seen to correspond to a cascade connection terminated to a resistive network with impedance rs+1(z) (compare also with BELEVITCH [1968, p. 110]). CHAPTER III. RATIONAL COVARIANCE EXTENSIONS In the first section we begin by applying the previously derived interpolation results to the study of rational covariance extensions. Certain bounds for the dimension of the various extensions are provided by this algebraic approach. With any rational Cfunction or, equivalently with any rational covariance sequence,there is associated a certain polynomial in z I and z This polynomial we call the dissipation polynomial of the sequence. It represents the zeros of the power spectrum or, equivalently, the zeros of an associated stochastic realization. This polynomial is completely determined up to a scalar factor by the tail of the associated parameter sequence. In point of fact, the dissipation polynomial up to a scalar factor is an invariant of the action of "shifting and truncating" the corresponding parameter sequence. This rather interesting result is further exploited in Section 4 in connection with asymptotic properties of rational covariance sequences. The most complete treatment up to date of the asymptotic and analytic properties of positive sequences and of the associated orthogonal polynomials has unquestionably been given by GERONIMUS [1961]. We shall apply some of his results to the case of rational sequences. A certain new aspect that emerged does not seem to have an analogue in the general case. The dissipation polynomial determines the asymp totic behavior of the parameter sequence. The sequence of parameters of a rational covariance sequence is not necessarily rational; however it is in a certain precise sense very close to being so. We shall call this property "almost rationality". Another aspect of this development is a new algorithmic procedure for spectral factorization. This is a key problem in system theory and several approaches to it have been developed. In the discrete time scalar case, given a rational function 7(z)/X(z), with v(z), X(z) in C[z], that has positive real part almost everywhere on Izl = 1, it is required to obtain a factorization I1 M ( z i) (z1) gz  2 X(z) R(z1) X(x) X(z1) with r(z) E C[z], for the real part of r(z)/X(z) on jzl = 1. This factorization amounts to factoring a nonnegative trigometric polynomial d(z, z1) := (z)R(z1) + X(z)(z1) = i(z)n(z1) z = e as the square of the modulus of a polynomial iT(z). The existence of such a factorization is well known. The most common approaches are a Riccati based approach (see FAURRE, CLERGET, and GERMAIN [1978]) and an algorithm due to RISSANEN and KAILATH [1972]. For different aspects of the factorization problem see ANDERSON, HITZ and DIEM [1974], DELSARTE, GENIN and KAMP [1978b], FRIENDLANDER [1982], SAEKS [1976], STRINZIS [1972], and YOULA [1961]. In our investigations we found a new technique. This is intimately related with the above. However, it operates on both numerator and denominator of a Cfunction v(z)/X(z) instead of simply the dissipation polynomial. The key idea is based on the invariance of the dissipation polynomial d(z, z1 ) under the action of "shifting and truncating" the corresponding parameter sequence. Under this operation, certain associated rational Cfunctions tend to 1, uniformly on compact subsets of Izl < 1. Consequently, both numerator and denominator polynomials tend to the same polynomial, which turns out to be the "stable spectral factor" of d(z, z1). 3. Rational Covariance Extensions and the Dissipation Polynomial A key result in partial realization theory is that a sequence C = (ct: t = 0, 1, ... is rational if and only if the power series r(z) = c + 2 c zt o t=l t defines a rational function in z (see GATITMACHER [1959, Chapter V] and also KALMAN, FALB, and ARBIB [1969, Chapter 10]). Moreover, if P(z) = r(z)/X(z) with 7T(z), X(z) coprime polynomials in z, then dim C = max (deg r(z), deg X(z)) =dim r(z). The previously derived description for the solutions of the CARATHEODORY problem will now be applied for the study of the rational ones. Such a solution with data a partial sequence C will be called S a pr (positive rational) extension of Cs In the degenerate case where C is nonnegative but not positive, there exists (see page 10) a unique covariance extension which turns out to be rational. In the nondegenerate case where Cs is positive,the set of all prextensions is described in the following: (3.1) THEOREM. Let C be a given partial positive sequence, Rs = (rt: t = 1, ..., s) be the associated partial parameter sequence, and M1(z) be the corresponding matrix polynomial defined in Lemma (2.6).  s An irreducible rational function c T(z)/Xl(z), with 1 (z), X1(z) E C[z] and l(O) = Xl(0) = 1, is a prextension of C if and only if there exists a unique irreducible rational' Cfunction T s+(z)/Xs+1(z) with 7T (z), X (z) E C[z] and Tr (O) = X (0) = 1, such that s+1 s+1 s+1 s+1 7T1(z) T s+1 (z) (3.2) = M((z) s+1z X1(s) s Xs+(z) s+l Then, R = (rs+l rs+2, ...) is the sequence of parameters associated with s+l(z)/Xs+l(z) if and only if R = (rl, ..., rs, rs+, ...) s the sequence of parameters associated with T1(z)/X1(z). Moreover, the following holds: dim s+1(z) l(Z) Ts+1(z) dim + 7 dim ( dim *+ s. xs+lZ) lZ Xs+lZ It is clear that any continuation (rt : Irtl < 1, t = s + 1, ...) for the partial sequence R leads to a positive extension of C The S S positivity is effectively characterized in terms of the associated sequence of parameters. This is not the case with the rationality. Due to the nonlinear transformation (2.9) between the c's and the r's, the para meter sequence of a rational sequence is almost never a rational one. However, the interpolation approach shows that the rationality of a covariance sequence is completely determined by the tail of the parameter sequence. PROOF. From (2.9) and (2.10) we readily obtain that a solution Pr(z) to the interpolation problem is rational if and only if the Cfunction rs+ (z) associated with the continuation of the parameter sequence is also rational. Let rs+ (z) = Ts+(z)/Xs+l(z). The normalization conditions T71(0) = X1(0) = 1 and Ts+ (0) = Xs+(O) = 1 are compatible due to the fact that S+0)* r r s 2(l r 1 + r Ss s We now derive the last inequality that provides bounds on the dimensions of various prextensions. We first define the matrix polynomial D(z) B(z) (3.3) N (z) := V(z) A(z) u u for v = 1, 2, ..., n = ..., with A:(z), Bv(z), Cv(z), Dv(z) u U u u defined as in Lemma (2.6). Using (2.15) it can be shown by direct calculation that 1 0 NV(z)* MV(z)* = zuhV u u u 0 1 where hV := v(1 Irt ) u t=v t Applying this to our case we have s s h xzs+i(z) x1(z)/ Since the elements of N1(z)* are polynomials of degree s, the above implies that deg Xs+1(z) < deg Xl(z) and also deg Ts+1(z) < deg Tr(z). From (3.2) we also have that deg Xl(z) < deg Xs+l(z) + s and similarly deg 7l(z) < deg Ts+l(z) + s. These prove the last inequality in (3.1). O We now consider an irreducible rational Cfunction T(z)/X(z). The real part of v(z)/X(z) is nonnegative for all z on Izl = 1 where it is defined. Therefore, r(e )X(e ) + T(e )X(ee) > 0 for all e in [ 7, r]. Following the terminology of an unpublished report by KALMAN we shall call d(z, z1) := ( (z)x (z1) + (z1)X(z)) the dissipation polynomial of 7(z)/X(z). By slight abuse of terminology we shall also call a dissipation polynomial any polynomial p(z, z1) 1 in both z and z which for z = expje and all 8 in [in, 7] is nonnegative. Finally, let us define the degree of d(z, z ) as the largest power of z. The role of the dissipation polynomial in the context of stochastic processes will be discussed in Remark (3.8). Herein, we shall see that the dissipation polynomials associated to the various prrealizations of Cs are up to a scalar factor determined completely by the choice of the Cfunction rs+1(z) = Ts+l(z)/Xs+1(z) of the previous theorem. Let r(z) = co (z)/X1(z) be a rational Cfunction and R = frs : s = 1, ...) denote the associated parameter sequence. Let also R = (r : s = t, t + 1, ...) denote the usual "shifted and s1 truncated" parameter sequence and 7T(z)/Xt(z), dt(z, z1 ) the associated rational Cfunction and dissipation polynomial. We now present the following: (3.4) THEOREM. In the degenerate case where R = (rs : s = 1, 2, ..., u, with Ir s < 1 for 1 ; s s u and Iru = 1 we have that dt(z, z1) 0 for t = 1, 2, ...u. In the nondegenerate case where 1 R = (r : s = 1, 2, ..., with Ir s < 1 for all s) s s we have that d(z, z ) = hdt (z, z l), for t = 1, 2, .... The behavior of the dimension of wt(z) as t increases is XtFz) described in (3.5) PROPOSITION. In the degenerate case where u is as in Theorem (3.4) we have that nt(z) dim = u t + for t = ..., u. In the nondegenerate case we have that the following two cases are possible: mt+l(z) wt(z) dim t+( = dim Xxt7t+7 I 7Tt+l(z) 7t(z) dim t+() =dim 1. xt+l77 XtT Furthermore, case conditions: (a) is equivalent to each of the following two wt(z) i (a') dimT = d (z, z t(z) (a") dim Xt = deg (wt(z) + Xt(z)). t We will now prove (3.4) and (3.5). PROOF OF (3.4). In the degenerate case we have Tt(z) = Tt+l(Z)*, and Xt(z) = t+l( )*. From (1.15) we obtain dt(z, z1) = h t+l. Since ru = 1, it follows that t u h = (1 Ir ) = 0. ut+l v=t In the nondegenerate case we obtain from (1.15) that 1 1 21 1 dl(tz, z ) 2 1(z) (z1z) + i(z l)X(z)) =2 (t(z)1(z1) +1 l(zl) t(z) t+1(z)Xt+1(Zl) + 1 + Tt+l(z) Xt+z)) t 1 = hd t+(z, z1) . PROOF OF (3.5). In the degenerate case, Xt(z) = ut+l(z)*, and rt(z) = _t+l(Z)*, for t = 1, 2, ..., u. Also, t +i(0) I= I t+l() = Ir = 1 t+is different from zero. Therefore, is different from zero. Therefore, deg Xt(z) = deg vt(z) = u t + 1. In the nondegenerate case, using (2.4) we obtain t+(z) = 7t(z) Xt(z)) + z(wt(z) + Xt(z)))/(1 + rt), and zXt+l(z) = ft(z) 7t(z)) + z(Xt(z) + vt(z)) /(l rt). This shows that both deg 7rt+1(Z) and deg Xt+,(z) are less than or equal to max (deg 7t(z), deg Xt(z)). Moreover, this difference can be at most one. Therefore (a) and (b) are the only two possibilities. In the case where max (deg Xt(z), deg rt(z)) = deg (vt(z) + Xt(z)), then clearly deg vt+1(z) = deg Xt+,(z) = deg (Vt(z) + Xt(z)) = dim vt(z)/Xt(z). In the case where the above does not hold,then both deg vt+1(z) and deg Xt+1(z) are less than dim 7t(z)/Xt(z). This establishes the equivalence of (a) and (a"). Consider now the identity 1 1 ) +1 Sft (z) t(z1) + Et(z')xt(z)) = dt(z, 1) d(S) S d(o) (s)s = d()z + ... + d( + ... + d z, t t t where we assume that d(s) d 0. It is easy to see that l deg (7t(z) + Xt(z)) = deg dt(z, z). This establishes the equivalence of (a') and (a"). D In view of the above, the dissipation polynomial of the various prextensions of Cs can be arbitrarily chosen by appropriate choice of the Cfunction Ys+l(z)/Xs+l(z). The set of all rational Cfunctions,which have dissipation polynomial fixed up to a scalar factor, is described in the following: (3.6) PROPOSITION. Let d(z, z1) be an arbitrary dissipation poly nomial of degree s. Then for any polynomial a(z) such that a(z) and zSd(z, z1) are coprime polynomials there exists a unique rational Cfunction r(z) = 7(z)/X(z) with r(0) = 1 and such that (5.7) d(z, z1) = (7r(z)x(z1) + (z1l)X(z) and v(z) X(z) = za(z). Conversely, for any rational Cfunction r(z) having r(0) = 1 and dissipation polynomial d(z, z1) there exists a corresponding polynomial a(z) as above. PROOF. Let a(z) be any polynomial such that zSd(z, z1) and a(z) are coprime. Then Sd(z, z1) + a(z)(z1) is a dissipation polynomial which is zero on izl = 1. Therefore, this polynomial factors into a product b(z)t(z" ) where b(z) is coprime with a(z) and has no root in iz[ < 1. Consider the function S(z) := a(z)/b(z). Clearly we have that 1 1S(z)2 4C z1 > o, s(z)2 b(z)S(z1) for z = expje and 0 in [ 7~, 7]. Since b(z) has no root in Iz < 1, it follows from the maximum modulus principle that S(z) is in S. Therefore by (2.3) b(z) zaz z = b z) + za z is in S. It can be readily checked that P(z) has dissipation polynomial d(z, z1 ). Conversely, for any r(z) = r(z)/X(z) in C where r(0) = 1 and (3.7) hold, a(z) = (r(z) X(z))/z is the required polynomial. This is shown by reversing the previous argument. 0 The algebraic approach we followed in this section gives only rough bounds on the dimension of the various prextensions of C For s example, given any dissipation polynomial of a certain degree u we can always find a corresponding Cfunction of the same degree. Then, by Theorem (3.1) we can obtain a prextension with dimensions between u and s + u. In the next chapter, a different approach will be followed. It will be shown that, with an appropriate choice of the extension, we can always achieve dimension equal to s. Furthermore, it will be shown that for a large number of cases, s is the smallest possible dimension of any prextension. (3.8) REMARK. The role of the dissipation polynomial in the context of stochastic realization will be now discussed. At the same time, certain quantities that will be used in the next section will now be introduced. Consider a covariance sequence C = (cs : s = 0, 1, ...). The nonnegativity of C or, equivalently, the covariance property is a necessary and sufficient condition for the existence of a nondecreasing function a(9), with 9 in [7, r] such that 1f t ct = etdo.(O), t = 0, 1, _+ 2, ... T (see AKHIEZER [1965, p. 180]). This function is called spectral distribution (of C, or of a corresponding stochastic process). The derivative cr'() of o(e) exists almost everywhere in [ v, 7] and is called spectral density. The Cfunction P(z) associated to C as in Theorem (2.1) admits the following integral representation (see AKHIEZER [1965, p. 1791): (3.9) r(z) = e je z da(e) e z Our interest rests in the case where C is also a rational sequence. In this case u(a) consists of two parts: (3.10) c(Q) = aa(e) + Tj(0), where ca(e) is an absolutely continuous nondecreasing function and jo.() is a nondecreasing function with finitely many points where the function increases. Furthermore, the derivative la(e) is a rational function in e For more details see DOOB [1953, P. 542] and GRENANDER and SZEGO [1958, p. 5]. The decomposition (3.10) induces via (3.9) the representation (3.11) r(z) = ( = a(z) .( z) where a (z)/Xa(z) is a Cfunction with the property that Xa(zo) f 0 for all z in Izl < 1 and v.(z)/X.(z) is of the form 0 3 3 7) (z) deg Xj exp(jeu) + z (312) ( = 1 Pu exp(jeu) z where p are positive scalars. Let d(z, z1), d (z, z1) and d.(z, z ) denote the dissipation polynomials of the above three functions (in the obvious notation). From (3.12) it immediately follows that d.(z, z1) = 0. Relation (3.11) now implies that 1 1 1 (3.15) d(z, z) = d (z, z )X.j(z)X(z ). A stochastic realization of C is a dynamical system E that under certain stochastic input and initial states generates an output process y that realizes C via the covariance function. Any stochastic process y can be decomposed into a superposition of two uncorrelated stochastic processes Yv = a, + Yj, where ya, is the socalled purely nondeterministic part and yj the deterministic part. (This is called the Wold decomposition. For more information see GRENANDER and SZEGO [1958, Ch. 10] or functional analysis literature where it has been widely used, e.g., HELSON [1964, p. 10])] In the case where C is a rational sequence then the above decomposition is in correspondence with (3.10). The part y j, can be realized by superposition of sinusoidal signals with frequencies determined from the roots of X.(z) whereas y can be realized by a single input system having white noise input and transfer function Ta(z)/Xa(z) where (3.14) da(z, z) = a(z)a(z1) The relations (3.13) and (3.14) indicate the role of the dissipation polynomial in this context. Returning to the covariance extension problem it is natural to consider factorizations of the form (3.13) for the dissipation poly nomials of both 7l(z)/Xl(z) and vs+1(z)/Xs+l(z) in Theorem (3.1). Whenever both parts in (3.10) are present it is not necessarily true that (in the obvious notation) X1 .(z) = Xsl ,(z). Due to this fact it appears that formula (3.2) in Theorem (3.1) is simply a computational tool and does not seem to have a stochastic interpretation. 4. Asymptotic Properties of the Spectral Zeros We begin by discussing a procedure for spectral factorization. This result will be subsequently used to elucidate the role of the dissipation polynomial on the asymptotic behavior of the sequence of parameters. Let P (z) = 71(z)/X1(z) be a rational Qfunction with r (0) = 1 1 and R = rg : s = 1, 2, ...) the associated parameter sequence. We denote by Rt the usual truncated sequences, vt(z)/Xt(z) the associated Cfunctions and dt(z, z1) the dissipation polynomials, where t = 1, 2, .... We shall consider only the nondegenerate case where dl(z, z1) 0. In this case by a well known factorization theorem (see GRENANDER and SZEGO [1958, p. 20]) there exists a polynomial Tl(z), with Tl,(0) = 1, and a positive scalar 7l such that (4.1) dl(z, Z1) = 1(z)1(z1) If we require that nl(zo) T1(0) = 1, then both 71 polynomial 1l(z) we call for all z in I z < 1 and also T~1(z) are uniquely determined. This stable spectral factor of dl(Z z1). stable spectral factor of dl(z, z ). We similarly define 7t, t((z). From Theorem (3.4) we clearly have that 71(z) = nt(z), for t = 1, 2, ..., and also 71 sl(1 r 12)t. We now set Vt(z) = 1 t + ... + at) , and Xt(z) = 1 + b )z + ... + b z Clearly, (4.2) rt=(at) bt)/2, and by (2.4) (4.5) 7Tt+l(Z) = 1/2[(Tt(z) Xt(z))/z + (TT(z) + Xt(z))]/(1 + rt), (4.4) Xt+l(z) = 1/2[(Xt(z) rt(z))/z + (7Tt(z) + Xt(z))]/(l rt), for t = 1, 2, .... Iterating the above we obtain the sequence of pairs (Tt(z), Xt(z)), t = 1, 2, .... The following theorem states that (4.2), (4.5), and (4.4) provide an algorithmic procedure to obtain q(z). (4.5) THEOREM. Let R1, Tt(z)(z), t(z), and T(z) be as above. Then im 7T(z) = Jim Xt(z) =(z). PROOF. First we need to recall certain function theoretic results: Let r(z) be a Cfunction, and for simplicity assume r(O) = 1. Let o(@) be the associated spectral density function that is given by 9'(0) = Re r(ej ) a.e. on [ T, v]. (4.6) STATEMENT. The following are equivalent: (a) anca' (e) is integrable in [ TT, T], a (b) there exists a function m(z) in H2 (the usual Hardy space; see RUDIN [1966, p. 328]) such that a(e) = Im(eje) 12 (c) a.e. on [ i, 7T], (d) tl(l Irt12) > 0, (e) tl rt 2 < +m For a proof of the above statement see GERONIMUS [1961, pages 20 and 159]. In case the above equivalent conditions hold we may find a function m(z) that also has inverse that is analytic in iz < 1. h en m(z) is 77 m(z) = exp ej e + z ena(9)deO z < 1, S7 e z and also 7 := m(0) 2 (4.7) = exp In fn(9)de T 1 = t(1 Irt 2) (See GERONIMUS [1961, pages 20, 21, and 158].) Furthermore, if a := det Ts_/det T s = 1, 2, ..., and a := 1, then cp(z)* := as s(z)*, s = 0, 1, .. converges to m(z)1 as s m. This convergence is satisfying the following inequality (GERONIMUS [1961, Theorem 4.10]): (4.8) p (z)*m(z) 1_ 781 I s1 + 7 1 Iz for jzl < 1 and where 0< 12 1/2 5s tz. Irtl2 (Note that 5s < +m because of (4.6).) In case r(z) = v(z)/X(z) is a rational Cfunction with 1 dissipation polynomial d(z, z ) 0, the ina'(G) is integrable and a in fact m(z) = 71/2(z)/X(z), where q(z), 7 satisfy (4.1) and rq(z) is the stable spectral factor of d(z, z ). We now apply (4.8) to the Cfunctions Tr(z)/Xt(z) for t = 1, 2, ..., and for s = 0 : mt() 1I = 71/2 1 Z XmtZz S ot 1 + Yt Ss i i2 \1/2 where 5 < (,ii r 2 Since 7 > 0 and 5,1 < +C, s,t \u=stl u 1 1,1 it follows that limy = 1 and lim 6, = 0. Consequently, t ot 1t/2 z)Xtz) 11 40 as t > uniformly on compact subsets of IzI < 1. Hence, im x(z) = z(z). Similarly we can show that im ~ t(z) = q(z). 0 Using the above we now want to study the asymptotic behavior of the sequence of parameters of rational covariance sequences. We begin by a motivating (4.10) EXAMPLE. Consider the rational function (Z) 1 + ( + r z X(z) 1 + rz of degree one. Necessary and sufficient conditions for 7(z)/X(z) to be in C are that fri <1, jaI < 1 Ir . The sequence of parameters (provided r = r, (4.11) rr 1 Jr 2 and the nonlinear recurrence law (4.12) 2 r s+1 s+2 r s12 r(l r+l ) Irl 1) is determined by for s = 1, 2,.... Rewriting the above in the following form rs+l r s 1 1 rs+12 we obtain rs+2 (4.13) rs+l r2 1 IrI2 r1 s+l(1 r 2 By considering the factorization of the dissipation polynomial of T(z)/X(z) we have d(z, z1) = az + (1 Ir2 + J2) + z1 1(ZAF(zl = 7y(z)+(z 1) = (pz + (1 + p2) + zl) where j(z) = 1 + pz is the stable spectral factor of rs+2 rs+l d(z, z1 ) Therefore, a = y7. Also by the result of GERONIMUS [1961, Theorem 8.2] 7 = T (1 Irt 12) . Combining the above two facts with (4.10) and using (4.15) we obtain rs+2 r2(1 r1 12) lim  = . sw rs+l r7 7 Equivalently, r Br (4.14) lim s+2 rs+l = 0. sm rs+1 This shows that as s > the sequence of parameters satisfies more accurately a linear recurrence law. Since in general rs s = 1, 2, ... might take also zero values,we consider the equivalent statement: for all E > 0 there exists an s such that for all s > s o  s+2 rs+l I E max (Ir s+1, r s+21). This motivates the following: (4.15) DEFINITION. A sequence R = (rs : s = 1, 2, ...) is said to be almost rational iff there exists a polynomial p(z) = 1 + plz + ... + p z such that for all E > 0 there exists an integer s with u o0 the property that r+u + Brs l + ... + Prs < max (r s+u 1 s+u1 Us sgts+u t , for all s > s. o A polynomial p(z) with the above properties is said to be an almost recurrence polynomial for R. (4.16) THEOREM. Let C be a rational positive sequence, R be the associated parameter sequence and i(z) be the stable spectral factor of the associated dissipation polynomial. Then R is almost rational and (z) is an almost recurrence polynomial for R. PROOF. Denote by 7t(z)/Xt(z) the rational with the usual truncated parameter sequences functions have power series expansions Cfunction associated R and let these 7t(z) 0 (t) t(z) = 1 + 2 s t =l 2, .... The relation between (c s = 1, 2, ... and R is given by the following formulas: t) 1 = rt, (t) 2 + (t) (t) c2 rt+(1 r ) + c1 1 ' (t) * s+1 t+s s1 l (1  = (c + (Cl Irt+u 2) + /ct) ) .. c(t) )(Tst )1 C Let T(z) = 1 + Plz + ... + Puzu and (z) b(t) (t) u X = +b z + ... + z . t 1 u By the previous theorem iim Xt(z) = rq(z). Therefore (4.18) Irt+u + lrt+ul + + purtlI < Irt + b(t) r + +b(t) ""r1u 1 t+u1 "'" u t + Et max +u r tfisst+u s" for all s > s (e) and E > 0. The polynomial Xt(z) satisfies (t) +b t) (t) + ... + b(tct) = 0. u+l 1 u u 1 From the above and (4.17) we now have (4.19) Irt+u + b)rt + ... + b (t)r t. m sl ft t+u 1 t+u1 u t, t5st+u where ft is a polynomial function in rt+s1, bt), (det T(t) 1 for s = 1, ..., u, that when viewed as a function of the rs's has has zero constant term. We shall now show that (4.20) lim f = 0. In the case of rational positive sequences no'(e) is integrable and therefore by the result of GERONIMUS [1961, Theorem 8.2] the parameter sequence R is squarely summable. Hence (4.21) im r = 0. In view of (4.16) it follows that lim c( = 0. t^c s s fixed (Notethat lim c(t) is not necessarily zero.) Then s.), t: fixed s (4.22) Jim det T(t = 1 > s1 for all s. Also by Theorem (4.15) (4.23) im b(t) = s, _>0o S for s = 1, ..., u. From (4.21), (4.22), and (4.23) we conclude (4.20). Finally (4.18), (4.19) and (4.20) imply that for all E > 0 there exists an So(E) so that rt+u + Ilrt+ul+ ... + urtl ESt +urs for all t > s (e). O (4.21) REItMARK. In the applied literature on timeseries analysis it has been noted (see for example BOX and JEIIIIS [1970, p. 179]) that the asymptotic behavior of the partial autocorrelation coefficients of rational power spectra, that are precisely the SCHUR parameters of our setting, is "dominated by damped exponentials". However, no precise statement of this seems to have been proven. Moreover, in case the almost recurrence polynomial has roots on Izi = 1, the above state ment is not absolutely correct. For example take r(z) = 1 z. Then the sequence of parameters is given by ( l)t rt t + 1 The asymptotic behavior of this sequence is not dominated by exponentials. CHAPTER IV. A TOPOLOGICAL APPROACH In this chapter we develop an alternative approach to the study of prextensions of C We focus our attention to prextensions of dimension less than or equal to s. Our key result will be an implicit description of this set. In Section 5 we show that for a nonempty open subset of the dataset of partial positive sequences C there exist no prextensions of dimension strictly less than s. This result justifies our interest in prextensions of dimension s. After a brief exposition in Section 6 of some basic facts about the topological degree, we derive in Section 7 our key result: For any dissipation polynomial of degree less than or equal to s there exists a corresponding prextension of C of dimension at most s. We should note that according to the results of the previous chapter this dimension could be as large as 2s. This result further provides a novel proof of the classically known fact that the positivity of C is a sufficient condition for s the existence of solutions to the CARATHEODORY problem. Also, most important, this topological approach provides an implicit description of a nonuniqueness inherent in this partial realization problem. 5. Covariance Extensions of Dimension s The following well known proposition gives conditions for a rational function to belong to C. (5.1) PROPOSITION. An irreducible rational function c r(z)/X(z), with v(z), X(z) in C[z] and c r(0)/X(0) = c R, is in C if and only if 1) co 1  (a) d(z, z) := (V(z)(zl) + (z )x(z)) is a dissipation polynomial, and (b) 1 (T(z) + X(z)) has no root in z < 1. We let Cs = (ct : t = 0, 1, ..., s) be a positive sequence and we consider a rational function c 7(z)/X(z) with power series expansion in z that begins with s t co + 2t ctz t A rational function with this property will be called a partial realization of Cs. Thus, a partial realization cor(z)/x(z) of C is a prextension of C if and only if (a) and (b) of s s Proposition (5.1) hold. If cr (z)/X(z) is a partial realization of Cs and r(z), X(z) have degree less than or equal to s, then r(z), X(z) and b(z) := (r(z) + X(z))/2 are related via the following nonsingular linear transformations (5.2) co(z) = [(co + 2clz + ... + 2cszS)X(z) ]o and (5.5) cob(z) = [(c + lz + ... +c zs)X(z)] o s where [ ] denotes truncating the powers of z outside [O, s]. We want to consider when C admits prextensions of dimension strictly less than s. This is given in the following: (5.4) LEMMA. There exists a prextension of C of dimension strictly less than s if and only if there exists a polynomial b(z) of degree less than or equal to s such that for the polynomials X(z) and r(z) obtained through (5.2) and (5.5) the following hold: (a') d(z, z1) = ( ((z)R(z1) + T(z1)X(z)) is a dissipation polynomial, (b') b(z) = i (7r(z) + X(z)) has no root in Iz < 1, and (c') r(z), X(z) have a nontrivial common factor. The essential point is that (b') is a closed condition as compared with (b) in Proposition (5.1). PROOF. Suppose b(z), 7(z) and X(z) satisfy the conditions of the lemma and let To(z), Xo(z) be coprime polynomials such that T(z) TTo( z) Then deg TT (z) and deg X (z) are less than s. Also d (z, z1) = (T (z)1(z1) + (z1)X (z))/2 is a dissipation polynomial and TT(z) + X (z) has no zero in Izi < 1. In order for wo(z)/Xo(z) to be a prextension of Cs we only need to show that TT (z) + X (z) has no root on Izl = 1. Suppose v (z ) + x (z ) = 0 for some z0 with Io = 1. Then IV(z ) + o() 2 = do(z z) + I (z)2 + IX (z,) = 0. Since d (zo, z1o) > 0 it follows that ro(zo) = Xo(zo) = 0, which contradicts the hypothesis that o (z) and Xo(z) were coprime. Therefore T (z) + X (z) has no root in Jz < 1 and TT (z)/X (z) is a prextension of C with dimension strictly less than s. The converse is trivial. D Consider now Y to be the set of nonnegative sequences C = (ct : t = 0, 1, ..., s) that for simplicity we assume co 1. The interior YO of Y is the set of positive sequences C . s We shall show that: (5.5) PROPOSITION. The set of partial nonnegative sequences C that admit no prextension of dimension strictly less than s is an open subset of Y. Clearly, this would also imply: (5.6) PROPOSITION. The set of partial positive sequences C that admit no prextension of dimension strictly less than s is an open subset of Y . PROOF of Proposition (5.5). Denote by X the space of poly nomials b(z) of degree less than or equal to s with b(O) = 1. The subset of X where (b') of Lemma (5.4) holds can be shown to be compact. Since Y is also a compact space,it follows that the subset of pairs (b(z), C ) E X x Y where (a') to (c') of Lemma (5.4) hold is also compact. The projection onto Y being a continuous map, implies that the subset of nonnegative sequences Cs (which by Lemma (5.4) admits a prextension of dimension strictly less than s) is compact. The complement of this set is therefore open. The fact that this set is nonempty follows by considering the partial sequence Cs = (1, 0, ..., 0, 1/2). Clearly, C is positive and moreover there is not even partial realization of C of dimension less than s. Hence, there is no prextension of C with dimension less than s either. O Given Cs, whether there exists a prextension of dimension strictly less than s is a decidable question. It can be answered by applying the decision methods developed by TARSKI [1951] and SEIDENBERG [1954] (see also JACOBSON [1974, Chapter V ]) to the conditions of Lemma (5.4). However, these are very involved and a simpler criterion is lacking. In fact, in the Appendix we shall indicate the set of conditions that needs to be tested for the first nontrivial case. But, the set of prextensions of dimension less than or equal to s is known to be always nonempty. We focus our study on this set. We shall use concepts of homotopy and degree theory for this. So we now make a brief digression and introduce the essentials of degree theory. 6. Basic Degree Theory The "degree" of this section refers to a notion of topological degree soon to be defined. The object of study of degree theory is the solution set of an equation d = f(b) where f is a mapping between two topological spaces. The main question concerns the existence and the number of solutions for a given d. Let S be an open subset of some topological space X, f a continuous map from S into a topological space D, and d be a point in D. The aim of degree theory is to define an integer valued function deg(f, S, d), called the degree of f at d relative to S, with the properties that (a) deg (f, S, d) is an estimate of the number of solutions of d = f(b) in S, (b) deg (., ., .) be continuous in the arguments, and (c) deg (., ., .) be additive in the domain S; i.e., whenever Sl S2 = then deg (f, S1U S2, d) = deg (f, Sl, d) + deg (f, S2, d). As usual, when S is a subset of topological space X, we denote by 8, aS, and So the closure, the boundary, and the interior of S respectively. The exposition below is following NAGUMO [1951], SCHWARZ [1965, Ch. III], and LLOYD [1978, Ch. I] where we refer for additional information and detailed proofs. N Let X and W both denote the Euclidean space R with the usual topology. (The reason for this redundant notation will become clear below.) The set S is assumed to be open and bounded subset of X. The maps that we consider are continuously differentiable in an open subset containing S. The set of such mappings is denoted by C l() and topologized by the norm f := sup Ift(b) I+ sup lbSN sbtN Sft*(b) Is I Given f C C(S), Zf(S) denotes the set of points _1 that there exists a point b in f (d) where the is zero. Suppose now that f The degree of f at d deg (f, S, d) := : Cl(S) and d W but d relative to S is defined fd S) (bCf'l(d) n  d in W Jacobian Sbf(yS)U Z (). by sign Jf(b). The definition is extended to points do that belong to Zf(S), but do not belong to f(6S), by letting deg (f, S, do) = deg (f, S, d) for any d f(aS) U Zf(S) and d "sufficiently close" to do. The fact that this is welldefined and the precise meaning of the term "sufficiently close" are described by the following: (6.1) THEOREM. Let f component of W\ f(aS). to Zf,(). Then be as above and dl, d2 belong to the same Suppose also that neither of them belongs deg (f, S, dl) = deg (f, S, d2). such Jf (b) A simple consequence of the definition of degree is (6.2) PROPOSITION. Let d f( S). Then, deg (f, S, d) 0 implies that d f(S). A notion that is crucial for the development of the next section is that of homotopy: A C homotopy between two elements fo and fl in Cl(B) is a function H : S x [0, 1] RN such that if Hx denotes the map b iH(b, x), then Ho = fo, H1 = fl Hx E C (s) for all x in [0, 1] and also IHx H yl1 0 as x y. This last condition says that H is a continuous function in the parameter x. The following is a very powerful result that we shall use in the next section. (6.5) THEOREM. Let f f be in C (s), and H be a C homotopy between f and fl. If d H(_S, x) for all x in [0, i], then deg (fo, S, d) = deg (fl, S, d). 7. Dissipation Polynomials and Covariance Extensions of Dimension s In this section we prove the following key result. (7.1) THEOREM. Let C = (c : t = 0, ..., s) be a partial positive sequence and d(z, z) := dszs + ... + dlz + 1 + d z + ... + dsZs be a dissipation polynomial (of degree < s). Then there exists a pair of polynomials (r(z), X(z)) with deg 7(z), deg X(z) less than or equal to s, and a positive scalar k such that the following two conditions hold: (a) co (z)/X(z) is a prextension of Cs, 1 1 1 1 (b) kd(z, zl) = ](7(z)x(z1) + r(z )X(z) ) We now elaborate on the implications of the above theorem with two immediate corollaries. (7.2) COROLLARY. Consider a partial positive sequence C There exists always a prextension of C with dimension at most equal to s. In this way we have circumvented the need for the algebraic machinery of orthogonal polynomials or of interpolation theory in order to establish that the positivity of C is a sufficient s condition for the existence of solutions to the CARATHEODORY problem. This is essentially a problem in analysis and an approach like ours seems to be absent. Furthermore, with this new approach we obtain some additional information about the set of prextensions of dimension s. (7.3) COROLLARY. Consider the partial positive sequence Cs = ct : t = 0, 1, ..., s) where not all of ct, t = ..., s s t 1 are zero. Then, for almost any dissipation polynomial d(z, z ) of degree less than or equal to s there exists an associated prextension of C with precisely dimension s. s We now proceed to the PROOF of Theorem (7.1). We again denote by X the space of polynomials with constant term 1 and with degree less than or equal to s. Any b(z) C X defines through (5.2) and (5.5) a unique pair of polynomials (7(z), X(z)) E X2 such that c7r(z)/X(z) is a partial realization of C i.e., has power series expansion that begins with S s c + 2clz + ... + 2c z . o 1 s The correspondence b(z) (Tr(z), X(z)) is certainly bijective, whereas the correspondence b(z) >)(z)/X(z) is clearly not. To any pair (7(z), X(z)) as above we associate the 1 polynomial in z and z : d(z, zz + ... + dz1 + d1 + ... +dz c o + 1 z The constant term do equals d =1 [[X(z)r(z)]s X(z1) + [I(z )T(z 1)O X()] o o 0o 2 s o = [i(zi )(c + ... + c z X(z)] s s = IIx(z) I , where 1 IIs denotes the norm that Cs induces on the space of polynomial of degree less than or equal to s (see page 15). Since X(0) = 1 and Cs > 0, then do IIx(z) II / 0. o s Let W be the space of "symmetric" polynomials L s 1 s d(z z = d z + ... + d z + 1 + d z + ... + d, z with constant term equal to 1. With d(z, z1) as above we define the map PC : X >W: (1 + b z + ... + b zs) s S d d d O _z + ... +  z + 1 + z + ... + zs . Sd a d d o o o o Both X and W are Euclidean spaces of the same dimension, and TCs is continuously differentiable in X. We now consider two open subsets S C X and P C W, where S is the subset of polynomials b(z) that satisfy b(z ) = 0 implies Izo > 1, and P is the subset that consists of all d(z, z1) E W that satisfy 1 d(z z ) > 0 for all z on z = 1. Therefore S is the set of ("stable") polynomials b(z) such that b(z ) = 0 implies zo > 1, and P is the set of dissipation polynomials with constant term equal to the identity. The statement of the Theorem can be easily seen to be equivalent to the following: for any d P there exists a b S such that d = pC (b). s Therefore we need to show that TCs(n) 32 P. Since the roots of a polynomial depend continuously on the coefficients (see MARDEN [1966]) it follows that S is open. Also because the roots of every b(z) in S lie in IzI > 1, it follows that S is bounded. We shall first show that (7.4) cpC (S) I P. For the particular sequence Co = (1, 0, ..., 0) the map T takes the simple form PC O : b(z) = 1 + blz + ... + bsz s b(z)i(z )/to bt 2. It is straightforward to show that (7.5) deg (PC S, d) = 1 for any d in P. 1) In fact the computations can easily be done for d(z, z ) = 1, and since cp (aS) = P we use Theorem (6.1) to establish (7.5). The set of positive partial sequences is connected (this is obvious especially when we consider the SCHUR parametrization; see page 11). Therefore we can follow a path within the set of positive sequences from C to any other positive sequence s C In this way we construct a continuous homotopy H(b, x) between PCso and PCs. We now show that (7.6) cpC ()S) (P = for any Cs > 0. s Suppose b(z) E 3S. Then b(z ) = 0 for some z with Iz = 1. Therefore, if d(z, z ) = Csb(z)E then b(z) 12 = 0 implies 1 d(zo, z ) = 0. 0 0 Consequently, d(z, z ) E oP and (7.6) is proven. If H(b, x) is a homotopy as above, then H(as, x) n P / i, for all x in [0, 1]. By Theorem (6.3) we conclude that deg (cCs, S, d) = 1, for any C > 0 and any d in P. Hence, by Proposition (6.2) it follows that 9 (S) D P s for all C > 0. By the compactness of S we also have that s S We want to close this chapter with the following: (7.7) CONJECTURE. The correspondence between dissipation polynomials d(z, z1) and pairs of polynomials (T(z), X(z)) with r(O) = X(0) = 1, in Theorem (7.1) is bijective. The conjecture is certainly true for the trivial sequence C = (1, 0, ..., 0). We were also able by direct computation of the s 1 Jacobian to show that it holds in a neighborhood of d(z, z ) = 1. But a proof is still lacking. We should mention that the map ~Cs is not analytic; therefore deg (PCs, S, d) = 1 for all d in P does not imply that the cardinality of cpCs(d) ( S is one. CHAPTER V. 1TE MATRIX CASE 8. The Matrix Covariance Extension Problem Given an nvariate, zeromean, stationary stochastic process y C Z we denote by c := EyTT+S, s = 0, 1, ..., the covariance n x nmatrixfunction of y In this chapter we shall use "~" to denote the "complex conjugate transpose of". The covariance sequence C = cs : s = 0, 1, ...} is characterized by the nonnegative definiteness of the block Toeplitz matrices T = [c ]u s = O, 1, ..., s tu t,u=o where now c it := ct (See for example GIHMANN and SKOROHOD [1974, p. 196].) Thus, we define a matrixsequence C = (Cs : s = 0, 1, ..., with co Hermitian) to be positive (resp. nonnegative) iff the associated block Toeplitz matrices T are positive (resp. nonnegative) definite for all s. We similarly define the partial matrix sequence Cs = (ct : t = 0, 1, ..., s) to be positive (resp. nonnegative) iff T is a positive (resp. nonnegative) definite matrix. This notion of matricial positivity is again related to an analytic property of the matrixvalued power series 00 r(z) := c + 2 sc z. (8.1) THEOREM (see KOVALISHINAand POTAPOV [1982]). The power series r(z) converges in Izi < 1, and P(z) + r(z) is a nonnegative definite matrix for all z in Iz < 1 if and only if the sequence C = (cs : s = 0, 1, ..., with co = (c + Z)/2) is nonnegative. Matrixvalued functions that satisfy the above conditions will again be said to belong to class C. The following is now the matrix CARATHEODORY problem: Given a partial sequence Cs, find necessary and sufficient conditions for the existence of a matrixvalued Cfunction with power series that begin with s t c + 2 CZ c . o + 2tltz The matrix CARATHEODORY problem seems to have been considered only recently by IL'MUSKIN [1974], and KOVALISHINA [1974]. See also AROV and KREIN [1981], DELSARTE, GENIN and KAMP [1979] and KOVALISHINA and POTAPOV [1982]. As an interpolation problem it can also be approached through the functional analytic techniques of SZ.NAGY and FOIAS [1970]. See e.g., HELTON [1980]. In the next two sections we will consider the subclass of rational solutions and carry out some of the program followed in the scalar case. The matrix sequence C is said to be rational iff there exists an integer v such that for all s > v the blockbehavior (Hankel) matrices s B = [c ] s t+ul t,u=l have the same rank. This integer v will again be called the dimension of C. The rationality of C is equivalent to P(z) defining a rational function in z. In this case r(z) can be represented as a matrix fraction P(z)Q(z)1 (right) or Q(z)P(z) (left). Suppose that C is rational and that P(z)Q(z)1 is a right matrix fractional representation of P(z) where P(z) and Q(z) are right coprime polynomial matrices, i.e. there exist A(z), B(z) C C xn[z] such that A(z)P(z) + B(z)Q(z) = I. Then it can be shown that the dimension of C is equal to the maximum of the degrees of P(z) and Q(z). (In this chapter, "I" will denote the n x n identity matrix.) In Section 9 we present a generalization of our topological approach for the matrix CARATHEODORY problem. We shall draw similar conclusions as in the scalar case: (a) the positivity of the partial sequence C is sufficient for the existence of solutions, and (b) for s almost any matrixdissipation polynomial of degree less than or equal to s there exists a corresponding rational solution of dimension less than or equal to s. In Section 10 we shall give a brief account of the basic results that come out of the algebraic approach and SCHUR's algorithm when applied to this matrixinterpolation problem. 9. The Topological Approach We begin by establishing the matrix version of Proposition (5.1). (9.1) PROPOSITION. A rational function P(z)Q(z)1 with Q(z), P(z) in Cnxn[z] and P(O)Q(0)1 Hermitian positive definite is in C if and only if det [Q(z ) + P(z )] = 0 implies 1z 0 > 1, and D(z, z) := Qz)P(z) + (Z1)Q(z)} has nonnegative definite values for all z such that Izl = 1. 1 As in the scalar case, we call D(z, z ) the dissipation (matrix) polynomial of P(z)Q(z)1. By a slight abuse of our terminology, we i shall also call any matrix polynomial in z and z satisfying the above property a dissipation polynomial. PROOF. An n x n matrixfunction S(z) is said to be in class S iff it is analytic in Izl < 1, and I S(z)STz) > 0 for all z in Izl < 1. These functions are considered in operator theory (see SZ.NAGY and FOIAS [1970]) where they are called "contractive" and in circuit theory (see BELEVITCH [1968]) where they are called "bounded". The relation between Cfunctions r(z), with r(0) Hermitian positive definite, and Sfunctions S(z) is given by (see for example [DELSARTE, GENIN, and KAMP [1979, p. 39]) (9.2) s(z) = 1(r(o) r(z))(r(o) + 1, and (9.5) r(z) = (I zs(z))(I + zS(z))lr(o). So we let r(z) = P(z)Q(z)l. Without loss of generality we can assume that r(0) = I. Then we obtain S(z) = (Q(z) P(z))(Q(z) + P(z))1 and also that, S(z) is in S if and only if det (Q(zo) + P(zo)) = 0 implies Izo > 1, and I S(z)S(z) > 0, for all z in Izl < 1. By the maximum modulus principle it is sufficient to test this on the boundary of the region of analyticity: 63 I S(z)S(z") = 2(Q(zl) + P(z1))D(z, z1)(Q(z) +P(z))>0 for all z on Izl = 1. Clearly this holds if and only if D(z, z"1) > 0. O (9.4) THEOREM. Let Cs = (ct : t = O, ..., s) be any partial  s t positive n x nmatrix sequence and z1 s d l S (9.5) D(z, z1 : d z + ... + d z + I + dz + ... + d z be an n x nmatrix dissipation polynomial. Then there exists a pair (P(z), Q(z)) E (C xn[z])2 with deg P(z), deg Q(z) less than or equal to s, and a positive definite matrix K such that the following two conditions hold: (a) P(z)Q(z)l is a prextension of Cs, (b) K/2D(z, z1Kl/2 = ((z)P(z) + (z)Q(z)}. (With ( )1/2 we denote the "Hermitian square root of".) This theorem establishes that when the partial sequence Cs is positive, the matrix CARATHEODORY problem is solvable. Our technique does not seem to be possible to extend to the singular case when C is only nonnegative. However, it provides information about the S solutions of dimension s, precisely as it did in the scalar case: For a generic set of dissipation polynomials of degree less than or equal to s, we can associate prextensions of Cs of precisely dimension s. For the complement of this set we can associate prextensions of dimension less than s. The idea of our proof is similar to the one we gave for the scalar case. However, certain new features require the use of a more sophisticated technique. The main new aspect is that, in contrast to the scalar case, matrix polynomials with no determinental zeros in zI < 1 and with constant term the identity matrix, do not form a bounded subset of the space of the coefficients. For example, the polynomial 1 0) + z 1/2 a \0 1/ 0 1/2) has nonvanishing determinant for all Izl < 1 and all values of a as well. We circumvent this by considering our stability set on a certain compact manifold. PROOF. Let P(z)Q(z)l, with P(z)Q(z) in nxn[z], be a partial realization of C of dimension less than or equal to s. Then P(z) and Q(z) are related by (9.6) P(z) = [(co + 2c1z + ... + 2cszs)Q(z)]S. Clearly, Q(0) is nonsingular. We now define the polynomial B(z) by (9.7) B(z) = [(co + clz + ... + cszs)Q(z)]s and the polynomial 11 S (9.8) D(z, z) = D z + ... +iz + + + ... + z := (Q(z)P(z)+ P(zl)Q(z)} i  in z and z By Proposition (9.1), P(z)Q(z) is in C if and 1) only if det B(z) has no roots in IzI < 1 and D(z, z ) is nonnegative definite for all z on Izj = 1. We notice that the pair (Q(z), P(z)) is defined up to a right unimodular factor. Therefore, so is B(z). Moreover, det B(O) 0. Thus, we consider the space X of polynomials {B(z) = B + B z + ... + 0 ] B zs, B(z) Cn[z]} S  of degree less than or equal to s. In this space we consider the subset M defined by s t=o tBt I, (9.10) B > 0, and B upper triangular. 2 This subset is a smooth compact manifold of real dimension 2sn2. That M is a smooth manifold follows from the open condition det B 0 0. o Compactness follows from the fact that,by the first condition,any entry of Bt, t = O, ..., s has modulus less than or equal to 1. (It is also easy to show that M is orientable, but we will not need this fact here.) The correspondence between B(z) in M and partial tions P(z)Q(z)l of C with dimension less than or equal is clearly surjective. If Q(z) = Q + Qz + ... + Qs then the polynomial D(z, z~1) obtained by (9.8) satisfies then the polynomial D(z, z) obtained by (9.8) satisfies D= (Q= Q, ... s) Since det Qo e 0 the following map c C c s and C > O, it follows th is welldefined: is welldefined: realiza to s Q S 0 S at D > 0. Therefore ~o PC : M , W s 1/2( 1) 1/2 B(z) 0 D o z, z 0 Y, c 1 s1 where W is the space of the polynomials D(z, z1) as in (9.5); i.e., such that D(z, z ) = (z ,1 z) and with constant term equal to the identity matrix. Consider the submanifold S of M of polynomials with determinant nonvanishing in Izl < 1, and the submanifold P of W of polynomials D(z, z1) that have nonnegative definite values for all z on Izl = 1. To complete the proof of the theorem we need to show that for all C > 0 we have s "s Precisely as in the scalar case it can be shown that cp ( qs) p = P I. Also,the set of positive partial matrix sequences C is pathwise s connected. By using the homotopy invariance property of the degree, the proof that was given for the scalar case works in this case also. More precisely pCs is certainly a continuous map between manifolds. (iCs is only continuous because we require taking the Hermitian square root.) Now MILNOR [1965] defines the degree for C mappings between manifolds. However, as remarked by LLOYD [1978, p. 32] the definition immediately extends to the continuous case simply by taking C1 approximations. (An explicit argument can be found in SCHWARTZ [1965, Chapter V] and LLOYD [1978, Chapter I].) Finally, we note that as before the degree deg (cpCs, S, D) can be seen to be 1 by considering the point D(z, z) = and the trivial sequence C = (I, O, ..., 0). 5 The proof now proceeds precisely as in the scalar case. 10. The Algebraic Approach The description of all solutions to the matrix CARAMHEODORY problem can be found in AROV and KREIN [1981] and KOVALISHINA and POTAPOV [1982]. It is given in the socalled "completely nondegen erate case", when Cs is a positive sequence. In the general case, when C is nonnegative but not positive,there exists no closed s form expression for the solutions. However, some standard techniques in operator theory can be used to deal with this case (cf. SZ.NAGY and FOIAS [1970, p. 188]). In this section, having presented our topological approach, we wish to give a brief account of the basic results and ideas of the algebraic approach, which essentially relies again on SCHUR's algorithm. We shall apply this to the case of rational Cfunctions. We begin by describing the SCHUR's algorithm for the case of matrix Cfunctions. The main technical fact is given in the following: (10.1) LEMMA. Let rt(z) be a matrixvalued function which has a power series expansion around the origin that begins with I + 2rtz, where I rtr > 0. Then r (z) is in C if and only if there exists a Cfunction Pt+ (z) such that (10.2) t(z) = [at(z)rt+l(z) + bt(z)][ct(z)Pt+l(z) + dt(z)]1 where at(z) = (I rtrt)1/2(I rt) + z(I trt) /2 ), bt(z) = (I rtt 1(I + rt) z(I rtrt) (I + t) ct(z) = (I rtt)1/2(I rt) z(I trt)/2( rt), (z) = (I r /2 + r) + z( 1/2 + d (z) =(I rtrtY (I + rt) + z(I r r ) (I + rt). PROOF. The SCHUR's recurrence relation for the matrix case is given by (see for example DELSARTE, GENIN, and KAMP [1979, (36)]) St(z) =(I rtrt)1/2 (rt+ Z 1(z))(I + zrtSt+(z))1(I rr 1/2 Assuming that I trt > O, then St(z) is in S if and only if St+l(z) is in S. Applying the bilinear transformation (9.2) we obtain the corresponding recurrence relations for the class Cfunctions. 0 Formula (10.2) can be solved for t+,(z) in terms of rt(z) and provides an inductive procedure for solving the CARATHEODORY problem in the completely nondegenerate case. In the completely nondegenerate case a matrixversion of (2.9) can also be obtained (see for example AROV and KREIN [1981]). Here we shall apply the lemma to rational Cfunctions 1 Pt(z)Qt(z), and consider the behavior of the dimension and the dissipation polynomial under the action of "truncating" the sequence of SCHUR parameters rt or, equivalently, as t increases. We now have the following: (10.3) THEOREM. Let rt(z) and rt+,(z) be in C and related as in Lemma (10.1). Then rt(z) is rational if and only if Pt+1(z) is rational. In this case, there exist right coprime representations rt(z) = Pt(z)t(z)1 and t+l(z) = Pt+1(Z)Qt+l(z)l with Pt(z), aQ(z), Pt+l(z), t+l(z) in Ct (10) Pt(z) at(z) bt(z) Pt+l(Z) (10.4) =2 t(z) ct(z) at(z)) Qt+(z)/ Also, if Dt(z, z1 )(resp. Dt+1(z, zl)) denotes the associated dissipation polynomial (10.5) Dt(z, z1) = Dt+l(z, z1 Moreover, the following are equivalent: (a) dim Pt(z) = dim rt+l(z), (a') dim rt(z) = deg Dt(z, z), (a") dim Pt(z) = deg (Pt(z) + Q(z)). For the proof we need the following: (10.6) LEMMA. The following identity holds: tn ad i e t (z) +b(z) (t((z1) h t c e matrzz) d(z) PROOF. By direct computation. O We now proceed to the PROOF of Theorem (10.3). Suppose r,+1(z) is a rational Cfunction and is equal to Pt+ (z)Qt+,(z) where Pt+1(z), Qt+1 (z) are right coprime matrix polynomials. Define Pt(z), Qt(z) via (10.4). It can be checked that ct(z)Pt+1(z) + dt(z)Qt+l(z) is invertible as a power series in z. By Lemma (10.1) it follows that rt(z) = Pt(z)Qt(z)1. Furthermore, it holds that Pt(z), Qt(z) are right coprime. Indeed, since Pt+ (z), Qt+l(z) are right coprime,there exist polynomial matrices A(z) and B(z) such that A(z)Pt,+(z) + B(z)Qt+1(z) = I. Hence, from Lemma (10.6) and (10.4), we obtain that there exist polynomial matrices Al(z) and Bl(z) such that Al(z)Pt(z) + 3B(z)Qt(z) = z2I. But both det Pt(O) and det Qt(0) can be checked to be different from zero. Therefore, Pt(z) and t(z) are in fact right coprime. The converse follows similarly by considering the identity (10.7) z2 t+l = Qt+(z) (z) at(z) t() z) that follows from Lemma (10.6). Relation (10.5) follows by considering D,(z, zl) l(Qt(z)pt(z)) Pt)t and applying Lemma (10.6). Finally, the equivalence of (a), (a'), and (a") can be shown as in the scalar case (see Proposition (3.5)). 0 Thus, the precise analogues of certain facts that were seen to hold in the scalar case, apply to the matrix case as well. We expect that the results of Section 5 extend to the matrix case also, and that the matrix dissipation polynomial determines the asymptotic behavior of the matricial SCHUR parameters. CHAPTER VI. APPLIED ASPECTS OF THE COVARIANCE EXTENSION PROBLEM In this final chapter we want to discuss the relevance of the covariance extension problem to the applied area of timeseries modeling. This area involves a large number of issues that we shall not touch upon (e.g., issues of statistical nature, see BOX and JENKINS [1970], or, of the essential difference between prediction for timeseries and prediction for stochastic processes, see FURSTENBERG [1960]). Instead we shall consider as our point of departure, the knowledge of a partial (sampled) covariance sequence C. s Based on these data, certain schemes have been proposed that yield a unique rational covariance extension for C These schemes form the base of modern nonlinear methods for spectral estimation (cf., HAYKIN [1979, pages 36 and 103]). We begin Section 11 by considering the socalled "maximum entropy" (ME) method in the context of our previous development. The ME method proposes the use of a particular prextension of C that has constant dissipation polynomial. The constancy of the s dissipation polynomial makes the construction of a corresponding stochastic realization trivial (since the problem of spectral fac torization is avoided altogether). Moreover, this construction turns out to be recursively updated as the data set increases. This latest property is precisely the recurrence relation satisfied by the orthogonal polynomials and was established in this context by LEVINSON [1947]. Recursiveness is very important in practical appli cations as it provides an efficient approximation procedure. In point of fact, this is the underlying philosophy in the ladder structure constructions in modern digital filter design. However, the absence of zeros in the power spectrum obtained by the ME method gives rise, in certain cases, to undesirable phenomena (see HERRING [1980] and the references therein). Motivated by the need for more general polezero approximating techniques, for the covariance function of stochastic processes, DEWILDE, VIEIRA, and KAILATH [1978], and DEWILDE and DYM [1981a and 1981b] (see also RUCKEBUSH [1978] and ROSENCHER and CLERGET [1979]) have placed the problem in a more general context of NevanlinnaPick interpolation theory. However, these investigations do not seem to illuminate the basic partial realization problem where the data is simply C Polezero modeling in the context of partial realization setting remains "a nonlinear and implicit problem, and there is no possibility for recursively updated realizations of increasing order" (see BENVENISTE and CHAURRE [1981]). In Section 12 we shall indicate that this might not be precisely so. Certainly, as it appears from our results of Chapter III, an essential part in obtaining prextensions of C with nontrivial s dissipation polynomial,is in obtaining information about the dissipation polynomial or, equivalently, the zeros of the corresponding power spectrum. (Our results of Chapter III, in particular Theorem (4.15) suggests that the parameter sequence might be used for that. This is a point that requires further investigation.) We should note that this information is already assumed in the approximation theories of DEWILDE and DYM [1981a and 1981b]. Now, provided such information is available we shall indicate a way that this can be incorporated in the modeling process in an efficient way. Theorem (12.4) will describe a recursive construction for prextensions that have approximately fixed zerostructure. 11. The MEMethod and Some General Discussion Let C be a partial positive (scalar) covariance sequence and S assume c = 1. The simplest possible choice for an admissible extension of the partial parameter sequence R is certainly the trivial extension (rs+t : rs+t = 0 for t = 1, ...). This extension amounts to choosing rs+1(z) = 1 in Theorem (3.1). The associated prextension of Cs is simply s C (z)* X Z) T S (Z) X~ z) ~(z)* * This particular extension has a certain uniqueness property, namely, it maximizes e := tlim It (z)*112 i.e., the distance in the II Iinorm of 1 from the closure of the manifold spanned by positive powers of z. It is immediate from (1.6) that in this case lkt(z)*l2 = Il1s(z)*l2, for all t > s, and hence, e = lim lDt(z)*ll2 = ls(z) *12. We now would like to explain the importance of this quantity in the prediction theory of stochastic processes. The inner product (.,*) that was defined in Section 1 relative to a nonnegative sequence C, can be extended from the space of polynomials in z to a more general space of functions on Izl = 1. This is done by realizing this inner product via a Stieljes integral (a(z), b(z)) = Jf a(z)bdo(e(0), z = exp je, with a(e j), b(eje) e L2[do(e)] the Hilbert space of squarely integrable functions on Izl = 1 with respect to the measure dc(0) (a(;) being the spectral distribution function of C, cf., Remark (5.8)). Let now y, T Z, be a (zeromean, stationary) stochastic process having C as covariance sequence. Let also L2(y) denote the Hilbert space generated by y with the inner product defined by 74 (f, g) := Efg, f, g E L2(y), and E denoting the expectation operator. Then, the mapping jr0 extends to an isometry between L2[da(e)] and L2(y) (cf., GRENANDER and SZEGO [1958, p. 175]). Via this mapping,any linear problem in L2(y) can be translated into one in L2[dc(e)] and conversely. Let now gs(z) be any polynomial in z with gs(0) = 1 and degree less than or equal to s. Clearly, due to the orthogonality properties (1.3), Ils(z)*II2 = gsZ) lg (z) 12. Therefore, e = lim (z)*2 = lim inf E y E ay 2 t t t too au o u=1 u is the square of the variance of the prediction error of y at an instant T = 0 based on observations in the past T < 0. And this is maximized by the choice s+(z) = 1 over all prextensions of the partial data C . s The quantity e. is an essential characteristic of a stochastic process and describes the "predictability" of the process. One can show that (see GERONIMUS [1961, p. 1581) e= t= l (1 I rt) exp 17 If in a'()de). 27~ r a The stochastic process is called deterministic iff e = 0, and nondeterministic otherwise. This quantity is directly related to a notion of entropy rate of a stochastic process in the sense of Shannon (see HAYKIN [1979, p. 80]), and this gives the name to the method. On the basis of the above it has been argued that the soobtained prextension of C is maximally noncommital to the unavailable s data (see JAYNES [1968]). This is indeed so, as far as the prediction problem is concerned. However, the prediction should be more of an "excuse" than a "reason" (see FURSTENBERG [1960, p. 7]). In point of fact, KALMAN [1981] argues that the partial sequence of parameters contains certainly more information than merely the fact that these parameters are all of modulus less than 1. In KALMAN [1981] it is also suggested that some minimal dimension prextension is perhaps the right object to consider. Unfortunately, the description of the minimal dimension prextensions seems to face intractable difficulties. In point of fact, an equi valent question was considered by YOULA and SAITO [1967] in a circuit theoretic context. Currently, this problem is unsolved. In Appendix A we shall indicate some computational difficulties that arise in the simplest nontrivial case. 12. On PoleZero Modeling We begin by assuming knowledge of a number of "influential zeros" in the power spectrum of a stochastic process. This rather loose term appears to have a rather definite meaning in the more applicationoriented literature, See for example MAKHOUL [1976, p. 115]. It is also stated that polezero modeling is not simple and not wellunderstood. We shall present a simple recursive way to incorporate the "zero" information in the modeling process. 1 Let C be a partial positive sequence and d(z, z1 ) be a 5 given dissipation polynomial of degree less than or equal to s. From the results of Section 7 we know that there exists an associated prextension of C of degree less than or equal to s. S In principal this prextension can be found as a preimage of d(z, z) under PC. Certain techniques have been recently developed to provide constructive algorithmic procedures for obtaining a solution of homotopybased existence results (see KELLOG, LI, and YORKE [1976], and also MEYER [1968]). However, this is very cumbersome and objectionable for almost all practical purposes. Thus, we shall not pursue it here but instead, we shall develop an approximate but efficient solution. Our first tool is a new representation for partial realizations of C of dimension less than or equal to s. Let C be a positive s s sequence, and Yt(z), (t(z), t = 0, 1, ..., s be the orthogonal polynomials of C . s (12.1) LEMMA. Any rational function c 7(z)/x(z), r(z), X(z) C[z], with 7(O) = X(O) = 1 and power series expansion that begins with (12.2) c + 2c z + ... + 2c zs (12.2) c 1 s is of the form S~ '(z)* + a z'Y (z)* + ... + a z? (z)* 7T(Z) s 1 s1 s o . Co o s0 (z)* + aze 1s(z)* + .+ a szs(z)* This result was independently of ours obtained by KIMURA [1985] who also argues that it provides a canonical form for partial realization of covariances. In our work a precise use of this lemma is given in Theorem (12.4). We should also mention that this lemma is really a fact about partial realizations and has essentially nothing to do with positivity. Positivity is assumed for the sake of some other properties of this representation that we shall soon discuss. PROOF of Lemma (12.1). The polynomials 7r(z) and x(z) are related through c r(z) = [X(z)(c + 2c z + ... + 2c zs)]. 0 0 1 s 0 This represents a nonsingular transformation between polynomials of degree st less than or equalto s. The two sets of polynomials (z t(z)*, t = 0, 1, ..., s) and (z tOt(z)*, t = 0, ..., s) form bases for this space and they are related by cozstct(Z)* = [tot(z)*(co + 2clz + ... + 2c zs)], 0 0S 0 for t = 0, 1, ..., s, as it follows from the definition of Tt(z), t = 0, 1, .... The proof of the lemma is now immediate. 0 This lemma places a system of coordinates, in the linear space whose points represent partial realizations of (12.2), so that the ME solution lies at the origin. Another aspect of this representation is shown in the following: (12.3) LENNA. Let T(z) = 's(z)* + alz's1(z)* + .. + szs (z)*, and X(z) = s(z)* + a zs ,(z)* + za + zs (z)*. 1 The polynomial (in z and z ) d(z, z1) = (7F(z)x(z1) + X(z)7(z1) has degree t < s if and only if as = = = 0. PROOF. This follows from the fact that the degree of Ts(z) t(z1) + s(z))T(z1) is equal to is ti. This can be shown using Lemma (2.6). o We now proceed to our final (12.4) THECREM. Let C = (ct : t = 0, 1, ...1 be the covariance sequence of a nondeterministic stochastic process, and let T(z) = 1 + alz + ... + auz be any polynomial with roots in (Iz > 1). Then,for s sufficiently large, S(s)() s(z)* + az (z) + *.. + azU s(z)* S(s)(z) s()* + a s u su c := C S(s)(z) o (z)* + alze 1(z)* + ... + auz (z) s 1 1 u su is a prextension of Cs, and if spectral factor of the associated q(s)(z) denotes the stable dissipation polynomial, then lim 1 ( (z) = h(z). PROOF. Let Mt(z) be as in Theorem ( 3.1). Then (s (s) (z) (s)(z) s/U(z)* + a zYSU(z)* +.. Mu()* 1 u1 Msu(Z)* Ssu(z)* + a ZSU(Z)* + ... u 1 u1 + a zuz) u 0 us + az Z) u o where tYs(z), 5u(z) are the orthogonal polynomials of the su parameter sequence Rsu = (rsu+l' .. r }. U Ssu+1 s Since C corresponds to a deterministic process, it follows from the result of GERONIMUS [1961, p. 159] (see (4.6)) that the parameters of C are squarely summable. Hence, as s 4o, both su 't (z)* 1 t sUD ( et (z)* + 1 for t = 0, 1, ..., u. Consequently, as s * o su (su UsU(z). zu (z)* + alZoul(z)* + ... + aZ) u 1 u1 u 0o tends to 1 uniformly on Jzl < 1, and the associated dissipation polynomial tends to Ti(z)i(zl). Applying now Theorem (3.1), the proof is complete. O AFFEIfDIX. THE MINIMAL DIMENSION PROBLEM Here we shall consider the following problem: Given a partial covariance sequence C find (simple) necessary and sufficient con s editions on C so that it admits a prextension of dimension strictly s less than s. This question is certainly the first one in attempting to obtain an explicit answer to the minimal dimension problem of KALMAN [1981], and YOULA and SAITO [1967]. We shall discuss the first two simple cases: s = 2, and s = 5. The case s = 2 is trivial as it requires conditions for the positivity of a degree 1 polynomial in z and 1 z The case s = 5, that requires conditions for the positivity i of a degree 2 polynomial in z and z presents already difficulties due to the implicit nature of the conditions that seem to be possible to approach only with the techniques of decision methods (see JACOBSON [1974, Ch. VDT. Although this appears to be quite elementary, it should be noted that conditions for the first case only exist in the current literature (see KALMAN [1981], and also KRISHNAPRASAD [1980] in an equivalent setting). We begin by considering the case s = 2. Unless cl = 0, the minimal dimension partial realization of 1 + 2c z + 2c2z2 is of dimension 1 and given by V(z) 1 + (a + c )z X = 1 + (a cl)z where c2 = c1(C1 O). (Here and in the sequel we use the represen tation introduced in Lemma (12.1).) Applying Proposition (5.2) it is straightforward to check that r(z)/X(z) is in C if and only if cl < 1, and (A.1) 1 _a I l c. We now consider the case s = 5. In case c1 = c2 = 0 and c / 0, then the minimal prextension is of dimension 3. In case c3 = clc2, then the minimal prextension is either of dimension 1 or 3 depending on whether the minimal partial realization which is of dimension 1 is in C or not. In the generic case we consider a general rational function 7(z)/X(z) with r(z), X(z) polynomials of degree 2 and 7(0) = X(0) = 1, that has power series that begins with 2 15 1 + 2clZ + 2c2 z2 + 2c 3. We shall restrict our attention to the case where all the scalars take real values, and use in the various expressions the associated parameters (rl, r2, r } instead of ([1, c2, c ). The function r(z)/X(z) is of the form Y2(z)* + azY 1(z)* + pz2 o(z)* S2(z)* + azel(z)* + pz2 o(z)* where a and 0 satisfy (A.2) (1 r )(1 r2)r + a(l r2)r2 + prl = 0. The conditions Y(z) + X(z) 0 O for all z in jzj < 1, and v(z))(zl ) + T(z )X(z) > 0 for all z on zl = 1, give rise to the following: (A.5) a(l + rl) < P + (1 + rl(l r2) (A.4) Q(l rl) < 0 + (1 rl)(l r2) (A. ) a(1 + r) < + (1 + rl)(l + r2) (A.6) c(l r) < + (1 rl)(l + r2) (A.7) < 1 and either (A.8) 48 < ja(l r2) + B + prlr2 , or (A.9) 2(l r2 ) r2r2 + 2apBrr( r+) 2 2 48[82 28 + (1 r)(1l r2)] 0. In general,neither of the above conditions is redundant. In the (a, B)space, these conditions cut out a set that corresponds to prextensions of C2 of dimension 2. This set is illustrated below as a shaded area for the two typical cases. 1 Irli r2 <1 I+ I Ir2 1 r r21 > 1 + Irl>  r 85 Whether C3 admits a prextension of dimension 2, depends on whether the line i given by (A.2) intersects the shaded region. Due to (A.9) which is implicit in a and P and is of degree 4, these conditions when expressed in terms of the original parameters of the problem, e.g. rl, r2, and r are also implicit and extremely involved. REFERENCES N. I. AKHIEZER [1965] The Classical Moment Problem, Oliver and Boyd Ltd., Edimburgh and London, 255 pages. N. I. AKHIEZER and M. G. KREIN [1962] Some Questions in the Theory of Moments, American Math. Society, Providence, R. I., 265 pages. B. D. O. ANDERSON, K. L. HITZ, and N. D. DIEM [1974] "Recursive algorithm for spectral factorization", IEEE Trans. on Circuits and Systems, CAS21:742750. D. Z. AROV and M. G. KREIN [1981] "Problems of search of the minimum of entropy in inde terminate extension problems", Functional Analysis and its Applications, 15(3):123126. V. BELEVITCH [1968] Classical Network Theory, HoldenDay, San Fransisco, 440 pages. A. BENVENISTE and C. CHAURPE [1981] "AR and ARMA identification algorithms of Levinson type: An innovations approach", IEEE Trans. on Automatic Control, AC26:12431260. G. E. P. BOX and G. M. JENKINS [1970] Time Series Analysis: Forecasting and Control, Holden Day, San Fransisco, 555 pages. P. TELSARTE, Y. GENIN, and Y. KAMP [1978a] "Orthogonal polynomial matrices on the unit circle", IEEE Trans. on Circuits and Systems, CAS25:149160. P. DELSARTE, Y. GENIN, and Y. KAMP [1978b] "A simple algorithm to spectral factorization", IEEE Trans. on Circuits and Systems, CAS25:943946. [1979] "Schur parametrization of positive definite Block Toeplitz systems", SIAM Journal on Applied Mathematics, 36:3446. P. DEWILDE and H. DYM [1981a] "Schur recursions, error formulas, and convergence of rational estimators for stationary stochastic sequences", IEEE Trans. on Information Theory, IT27:446461. [1981b] "Lossless chain scattering matrices and optimum linear prediction: the vector case", Circuit Theory and Applications, 2:135175. P. DEWILDE, A. VIEIRA, and T. KAILATH [1978] "On a generalized Szeg6Levinson realization algorithm for optimal linear predictors based on a network synthesis approach", IEEE Trans. on Circuits and Systems, CAS25:663675. J. L. DOOB [19531 Stochastic Processes, John Wiley, New York, 654 pages. P. FAURRE, M. CLERGET, and F. GERMAIN [1978] Operateurs Rationnels Positifs: Application a 1' Hypersta bilitd et aux Processus Aldatoirs, Dunod, Paris, 294 pages. B. FRIE1IDLAJTDER [1982] "A lattice algorithm for factoring the spectrum of a moving average process", Proceedings of the Conference on Information Sciences and Systems, Princeton, New Jersey, pp. 59. H. FURSTENBERG [1960] Stationary Processes and Prediction Theory, Princeton, New Jersey, Princeton University Press, 283 pages. F. R. GANTMACHER [1959] The Theory of Matrices,vol. 2, Chelsea, NewYork, 276pages. T. T. GEORGIOU and P. P. KHARGONEKAR [1982] "Partial realization of covariance sequences", Center for Mathematical System Theory, University of Florida. Ya. L. GERONIMUS [1954] "Polynomials orthogonal on a circle and their appli cations", American Math. Society, Translations, 1(2): 253304. [1961] Orthogonal Polynomials, English translation from Russian, Consultants Bureau, New York, 242 pages. I. I. GIHMANN and A. V. SKOROHOD [1974] The Theory of Stochastic Processes I, SpringerVerlag, Berlin, 570 pages. U. GRENANDER and G. SZEGO [1958] Toeplitz Forms and their Applications, University of California Press, Berkeley, 245 pages. S. HAYKIN [1979] Nonlinear Methods of Spectral Analysis, SpringerVerlag, Berlin, 247 pages. H. HELSON [1964] Lectures on Invariant Subspaces, Academic Press, New York, 130 pages. J. W. HELTON [1980] "The distance of a function to He in the Poincare metric; Electric power transfer", Journal of Functional Analysis, 38:273514. R. HEFRIIIG [1980] "The cause of line splitting in Burg MaximumEntropy spectral analysis", IEEE Trans. on Acoustics Speech and Signal Processing, ASSP28:692701. G. M. IL'MUSHKIN [1974] Candidate's Dissertation, Ul'yanovsk. N. JACOBSON [1974] Basic Algebra I, Freeman, San Fransisco. E. T. JAYNES [1968] "Prior probabilities", IEEE Trans. on Systems Science and Cybernetics, SSC4:227241. T. KAILATH [1974] "A view of three decades of linear filtering theory", IEEE Trans. on Information Theory, IT20:146181. R. E. KALMAN [1979] "On partial realizations, transfer functions, and canonical forms", in Acta Polytechnica Scandinavica, Mathematics and Computer Science Series, 31:932. [1981] "Realization of covariance sequences", Proceedings of the Toeplitz Memorial Conference, Tel Aviv, pp. 331342. R. E. KALMAN, P. L. FALB, and M. A. ARBIB [1969] Topics in Mathematical System Theory, McGrawHill, NewYork, 358 pages. R. B. KELLOGG, T. Y. LI, and J. YORKE [1976] "A constructive proof of the Brouwer fixedpoint theorem and computational results", SIAM Journal on Numerical Analysis, 13:473483. H. KIMURA [1983] "A canonical form for partial realization of covariance sequences", Osaka University, Technical Report 8501, 25 pages. I. V. KOVALISHINA [1974] "Jexpansive matrixvalued functions in the Caratheodory problem", Doclady Akademia Nauk Arm. SSR, 9(3):129155. I. V. KOVALISHINA and V. P. POTAPOV [1982] Integral Representation of Hermitian Positive Functions, private translation by T. Ando, Hokkaido University, Sapporo, Japan, 129 pages. M. G. KREIN and A. A. NUDEL'MAN [1977] The Markov Moment Problem and Extremal Problems, American Math. Society, Providence, R. I., 417 pages. P. S. KRISHNAPRASAD [1980] "On the geometry of linear passive systems", in Algebraic and Geometric Methods in Linear Systems Theory, edited by C. I. Byrnes and C. F. Martin, American Math. Society, Providence, R. I., pp. 253275. N. LEVINSON [1947] "The Wiener RMS (RootMeanSquare) error criterion in filter design and prediction", Journal of Mathematical Physics, 25:261278. N. G. LLOYD [1978] Degree Theory, Cambridge University Press, Cambridge, 172 pages. J. MAKHOUL [1975] "Linear prediction: A tutorial review", in Proceedings of the IEEE, 64:99118. M. MARDEN [1966] The Geometry of the Zeroes of a Polynomial in a Complex Variable, American Math. Society, Providence, R. I., 183 pages. G. H. MEYER [1968] "On solving nonlinear equation with a oneparameter operator imbedding", SIAM Journal on Numerical Analysis, 5:759752. J. W. MILNOR [1965] Topology from the Differentiable Viewpoint, the University Press of Virginia, Charlottesville, 64 pages. M. MORF, A. VIEIRA, and T. KAILATH [1978] "Covariance characterization by partial autocorrelation matrices", Annals of Statistics, 6: 643648. M. NAGUMO [1951] "A theory of degree of mapping analysis", American Journal of based on infinitesimal Mathematics, 73:485496. J. RISSANEN and T. KAILATH [1972] "Partial realization of random 8:389396. systems", Automatica, E. ROSEIICHER and M. CLERGET [1979] "Approximation rationelles des filtres", Annals T41ecom., 34:439445. G. RUCKEBUSH [1978] "Sur 1' approximation rationelle des filtres", Report 35, Ecole Polytechnique, France. W. RUDIN [1966] Real and Complex Analysis, McGrawHill, New York, 412 pages. R. SAEKS [1976] "The factorization problem  A survey", in Proceedings of the IEEE, 64:9095. D. SARASON [1967] "Generalized interpolation in H ", Transactions of the American Math. Society, 127:179203. J. T. SCHWARZ [ 1965] Nonlinear Functional Analysis, Courant Institute of Mathe matical Sciences, New York University, New York, 246 pages. A. SEIDENBERG [1954] "A new decision method for elementary geometry", Annals of Mathematics, 60:565374. M. G. STRINZIS [1972] "A solution to the matrix factorization problem", IEEE Trans. on Information Theory, IT18:225232. G. SZEGO [1959] Orthogonal Polynomials, American Math. Society, Colloquium Publications, Providence, Rhode Island, vol. 23, 452 pages. B. SZ.NAGY and C. FOIAS [1970] Harmonic Analysis of Operators on Hilbert Space, North Holland Publ. Comp., Amsterdam, 387 pages. A. TARSKI [1951] A Decision Method for Elementary Algebra and Geometry, Univer sity of California Press, Berkeley, California, 65 pages. J. L. WALSH [1956] Interpolation and Approximation by Rational Functions in the Complex domain, American Math. Society, Colloquium Publications, Providence, Rhode Island, vol. 20, 382 pages. P. WHITTLE [1965] "On the fitting of multivariate autoregressions, and the approximate canonical factorization of a spectral density matrix", Biometrika, 50:129134. R. WIGGINS and E. A. ROBINSON [1965] "Recursive solution to the multichannel filtering problem", Journal of Geophysical Res., 70:18851891. D. C. YOUIA [1961] "On the factorization of rational matrices", IRE Trans. on Information Theory, IT7:172189. D. C. YOUIA and N. N. KAJANJIAN [1978] "Bauertype factorization of positive matrices and the theory of matrix polynomials orthogonal on the unit circle", IEEE Trans. on Circuits and Systems, CAS25:5769. D. C. YOULA and M. SAITO [1967] "Interpolation with positivereal functions", Journal of the Franklin Institute, 284:77108. BIOGRAPHICAL SKETCH Tryphon Thomas GEORGIOU was born on October 18, 1956, in Athens, GREECE, to Thomas GEORGIOU and Georgia GEORGIOU. He obtained his Diploma in electrical and mechanical engineering from the National Technical University of Athens, in 1979. I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Dr. E. W. Kamen, Chairman Professor of Electrical Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. PA/wr9 KkL^jovCe^ _ Dr. P. P. Khargonekar, CoChairman Assistant Professor of Electrical Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Dr. T. E. Bullock Professor of Electrical Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Dr. R. L. Long Assistant Professor ofqathematics I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Dr. A. R. Tannenbaum Associate Professor of Mathematics This dissertation was submitted to the Graduate Faculty of the College of Engineering and was accepted as partial fulfillment of the requirements for the degree of Doctor of Philosophy. August, 1983. Dean, College of Engineering Dean for Graduate Studies and Research UNIVERSITY OF FLORIDA 11 111111 lll I t5lillll ll lltil i il tIII111111 1111 3 1262 08558 9563 