UFDC Home  UF Institutional Repository  UF Theses & Dissertations  Internet Archive   Help 
Material Information
Subjects
Notes
Record Information

Table of Contents 
Title Page
Page i Dedication Page ii Acknowledgement Page iii Table of Contents Page iv Page v Page vi Abstract Page vii Page viii Chapter 1. Introduction Page 1 Page 2 Page 3 Page 4 Chapter 2. Survey of literature Page 5 Page 6 Page 7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Chapter 3. Kalman filters Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Chapter 4. Logarithmic number system Page 36 Page 37 Page 38 Page 39 Page 40 Page 41 Page 42 Page 43 Page 44 Chapter 5. Error models for LNS arithmetic Page 45 Page 46 Page 47 Page 48 Page 49 Page 50 Page 51 Page 52 Page 53 Page 54 Page 55 Page 56 Chapter 6. Logarithmic Kalman filters Page 57 Page 58 Page 59 Page 60 Page 61 Page 62 Page 63 Page 64 Page 65 Page 66 Page 67 Page 68 Page 69 Page 70 Page 71 Chapter 7. Adaptive Kalman filters Page 72 Page 73 Page 74 Page 75 Page 76 Page 77 Page 78 Page 79 Page 80 Page 81 Page 82 Page 83 Page 84 Page 85 Page 86 Page 87 Page 88 Page 89 Page 90 Page 91 Page 92 Page 93 Page 94 Page 95 Page 96 Page 97 Page 98 Page 99 Page 100 Page 101 Page 102 Page 103 Page 104 Page 105 Page 106 Page 107 Chapter 8. Implementation of Kalman filters using systolic architectures Page 108 Page 109 Page 110 Page 111 Page 112 Page 113 Page 114 Page 115 Page 116 Page 117 Page 118 Page 119 Page 120 Page 121 Page 122 Page 123 Page 124 Page 125 Page 126 Page 127 Page 128 Page 129 Page 130 Page 131 Page 132 Chapter 9. Summary and conclusions Page 133 Page 134 Page 135 Bibliography Page 136 Page 137 Page 138 Page 139 Page 140 Page 141 Page 142 Page 143 Biographical sketch Page 144 Page 145 Page 146 
Full Text 
ADAPTIVE OPTIMAL FITLERING USING THE LOGARITHMIC NUMBER SYSTEM By GEORGE MICHAEL PAPADOURAKIS A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 1986 DEDICATED TO EAEU9EPIA ACKNOWLEDGMENTS I would like to thank my advisor Dr. Fred J. Taylor for his valuable guidance and assistance during my graduate studies. All these years he has provided me the much needed support and encouragement to complete my doctoral work. I would also like to thank Dr. Donald G. Childers, Dr. Antonio A. Arroyo, Dr. George Logothetis, and Dr. Jask R. Smith for serving on my supervisory committee. In addition, I would like to thank Dr. Jose C. Principe for his useful comments. Finally, I would like to thank my fellow graduate students and especially A. Skavantzos and A. Stouraitis for their help. TABLE OF CONTENTS PAGE ACKNOWLEDGMENTS .................................. ....... iii ABSTRACT...... ............................................ vii CHAPTER ONE INTRODUCTION................ ......................... 1 TWO SURVEY OF LITERATURE........................o....... 5 Kalman Filters............ ...................... ... 5 Correlated Process and Measurement Noise.......... 5 Wiener Filter................... .................. .6 Random Sampling Times..................... ........ 6 Effect of Inaccurate Mathematical Models and Statistics...... ................................ 6 Performance (Sensitivity) Analysis............... .7 Square Root Filtering................. ........... .8 Suboptimal Filtering ................................ 8 Compensation of Linear Model Inadequacies.........9 Linear Smoothing Problem......................... 10 Observers.............. ....... .................... 10 Adaptive Estimation.............................. 11 Applications............ .......................... 11 Logarithmic Number System.......................... 14 Logarithmic Conversion........................... 15 Addition and Subtraction Algorithms.............. 16 Multiplication and Division Algorithms..........17 Table Reduction Techniques.,..................... 17 Extended Precision LNS........................... 18 Digital Filtering Using LNS....................... 18 FFT Implementation with LNS ......................19 Other Applications with LNS ..................... 19 THREE KALMAN FILTERS .................................... 21 Statement of Problem............................... 21 Least Square Estimation............................ 22 Estimation of Parameters Using Weighted LeastSquares........................... 25 Recursive Filters .................................. 27 Discrete Kalman Filters ............................28 Kalman Filters with Deterministic Inputs...........35 FOUR LOGARITHMIC NUMBER SYSTEM .......................... 36 LNS Representation ................................. 37 LNS Arithmetic Operations .......................... 38 Multiplication................................... .38 Addition and Subtraction ......................... 39 A Hybrid Floating Point Logarithmic Arithmetic Unit .................................. 40 Floating Point Format ............................ 41 (FU) Square Unit................................. .41 FIVE ERROR MODELS FOR LNS ARITHMETIC .................... 45 Input Quantization ................................. 45 Coefficient Quantization........................... 54 Quantization in Arithmetic Operations.............. 54 SIX LOGARITHMIC KALMAN FILTERS ......................... 57 LNS Kalman Filter .................................. 57 Theoretical Error Analysis in LNS ..................58 Mean Error Analysis .............................. 62 Variance Error Analysis ..........................63 Simulation Studies .................................64 SEVEN ADAPTIVE KALMAN FILTERS ............................ 72 The Effect of Erroneous Models on the Kalman filter Response .......................... 72 The Effect of Input and Output Noise Covariances on the Kalman Gains ..............................75 Adaptive Kalman Filtering ..........................84 Algorithm Implementation............................86 Kalman Filter .................................... 86 Autocorrelation.................................. 90 Search Algorithm ..................................90 Algorithm Integration ..............................94 Mehra's Adaptive Algorithm .........................94 Carew's and Balanger's Adaptive Algorithm..........97 Experimental Results ............................... 99 EIGHT IMPLEMENTATION OF KALMAN FILTERS USING SYSTOLIC ARCHITECTURES .......................... 108 Systolic Architectures ............................108 Orthogonal Systolic Arrays ........................115 Kung's Systolic Array ...........................116 Liu's and Young's Systolic array ................118 Pipelined Systolic Array ........................121 Kalman Filtering Using Systolic Arrays ............123 NINE SUMMARY AND CONCLUSIONS ...........................133 BIBLIOGRAPHY............................................... 136 BIOGRAPHICAL SKETCH ...................................... .144 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ADAPTIVE OPTIMAL FILTERING USING THE LOGARITHMIC NUMBER SYSTEM By GEORGE MICHAEL PAPADOURAKIS August 1986 Chairman: Dr. Fred J. Taylor Major Department: Electrical Engineering The Kalman filter has been one of the most widely applied techniques in the area of modern control, signal processing and communication applications. The implementation of a Kalman filter using the logarithmic number system (LNS) is considered. The choice of LNS is highly attractive in this digital filtering setting since it offers high precision, wide dynamic range, attractive architecture, and ease of arithmetic operations. A theoretical error analysis is presented concerning the mean and variance of the actual estimations due to a vii finiteprecision logarithmic implementation. Simulation studies are utilized to show how closely the analytical predictions agree with actual results. A comparison between the finiteprecision conventional Kalman filter and the proposed scheme is performed in terms of speed and accuracy. In addition, an adaptive Kalman filter is considered for which the input and output noise covariances are unknown. A new procedure is presented for the identification of the unknown covariances using the autocorrelation information contained in the innovation error sequence and a modified Fibonacci search. Realtime performance is facilitated through the use of the presented efficient high speed digital autocorrelation algorithms and search procedures. Finally, the implementation of the Kalman filter using systolic array architectures is investigated. Systolic architectures have received much attention recently as a means of achieving high data rates. Systolic designs have been used to perform matrix operations making them suitable for Kalman filtering processing. Using an orthogonal array processor, a new algorithm is presented which increases the throughput of matrix operations. This new algorithm will be compared with other systolic architectures on the implementation of Kalman filters. viii CHAPTER ONE INTRODUCTION The theory of optimal filtering has application to a broad range of meaningful problems. Within the family of known optimal algorithms, the filtering technique known as the Kalman filter has been the most visible. It has been shown to be well suited for digital computer implementation provided high throughput is not a prerequisite. This class of filter has penetrated the areas of control, communication, guidancenavigation, and data processing. To successfully design a Kalman filter, and its counterparts, the engineer/scientist must address both modeling and computation problems. The Kalman filter is intrinsically a model reference filter, and as a result, can suffer from degraded performance if the system model is improperly configured. In order to desensitize the filter to model dependent errors, an adaptive filter policy should be considered. This leads to the second problem. It has been assumed that the Kalman filter would lend itself well to digital computation due to its recursive structure. This has not always been the case. The computational burdens imposed by realtime, high speed, highprecision Kalman filters can exceed a computers throughput capabilities. In addition, there is always a controversy on whether floating point or fixed point arithmetic should be used. Furthermore there is generally the overlooked problem of burst error management of filters operating in hostile environments which must be attended to. The additional computational demands of an adaptive filter often exceed the performance capability of conventional computer architectures. This is especially true in those cases where small wordlength, limited capability processors are found (viz. the microprocessor). Rather than developing theory over a real number field, we are proposing to represent numbers over finite fields and compressed numbering systems. Recent developments in the area of logarithmic number systems (LNS) have proven it to offer some significant advantages over other number weighted systems like the fixed or floating point. The main advantages of LNS are the extended dynamic range on which they operate and the high speed of operations accompanied by a remarkably regular data flow. Another remarkable feature of the LNS architecture is its ability to be efficiently transferred into VLSI. Furthermore LNS arithmetic units are ideally suited to systolic array data processing, thus increasing throughput. The organization of the dissertation is as follows: Chapter Two contains a comprehensive survey of the literature on Kalman filters and logarithmic number system. In Chapter Three, the derivation of the discretetime (sampled data) optimal estimator, known as the Kalman filter, will be presented. Chapter Four contains a description of the logarithmic number system and its properties. In addition a logarithmic number processor, (FU)2, will be developed. Chapter Five describes the quantizationerror models used in LNS arithmetic, namely, input, coefficient and arithmetic operations. In Chapter Six, the implementation of a Kalman filter using the logarithmic number system (LNS) will be considered. A theoretical error analysis will be presented concerning the mean and variance of the actual estimation due to a finiteprecision logarithmic implementation. Simulation studies will be utilized to show how closely the analytical predictions agree with actual results. In addition comparison between the finiteprecision conventional Kalman filter and the proposed scheme will be performed in terms of speed and accuracy. In Chapter Seven, an adaptive Kalman filter will be considered for which the input and output noise covariances are unknown. A new procedure will be presented for the identification of the unknown covariances using the autocorrelation information contained in the innovation error sequence and a modified Fibonacci search. 4 In Chapter Eight, implementation of a Kalman filter using a new systolic design will be investigated and compared with other systolic architectures. Finally in Chapter Nine, by way of summarizing the proceeding, the significance of the described research will be brought out and some directions for future research will be indicated. CHAPTER TWO SURVEY OF LITERATURE Kalman Filters In 1960 Kalman [1,2] developed the linear estimation theory hereafter referred to as the Kalman filter. Others [3,4] developed the theory even further and gave more insights to the linear estimation problem. This estimation theory can be formulated in terms of either a timediscrete or time continuous model. Additional subjects on Kalman filters that have been treated in the voluminous literature will be presented. Correlated Process and Measurement Noise The white noise sequences have been assumed to be uncorrelated in the formulation of the Kalman filter. This restriction is not necessary. However, the resulting equations are considerably more complicated [57]. Wiener Filter Underlying Wiener filter design [8] is the so called WienerHorf equation, its solution through spectral factorization and the practical problems of synthesizing the theoretically optimal filter from its impulse response. In the case of timeinvariant systems models and stationary noises, the Wiener filter is shown to be equivalent to the steady state Kalman filter appropriate to the given problem [810]. Random Sampling Times The measurement data have been assumed to occur at prespecified times. It can happen that the time to which a measurement is related is random. Also, the transition matrix and/or the observation matrix might contain parameters that are random. These cases have been considered by several investigators including Tou [Ii], Rauch [12], Gunckel [13] and others. Effect of Inaccurate Mathematical Models and Statistics The validity of the estimates provided by this estimation technique require an accurate mathematical description of the dynamical and observational processes. The omission of terms that actually enter these processes can result in extremely poor estimates. If the statistics (i.e., R, Q) are not accurate descriptions of the secondorder moments of the noise processes [14], the covariance matrix P might not be a realistic measure of the accuracy of the estimate. Performance (Sensitivity) Analysis The estimates of the state are random variables. To determine the validity of the estimates and the significance of P, it is necessary to perform a Monte Carlo simulation of the physical system under consideration [15]. Unfortunately, this is a costly and time consuming process. More efficient means of generating the statistical information is the mean analysis [16] and the covariance analysis (171. One particular use of a covariance performance (sensitivity) analysis is a systematic approach to the filter tuning [1824] process. The basic objective of filter tuning is to achieve the best possible estimation performance from a filter of specified structural form, i.e., totally specified except for P0 and the time histories of Q and R. These covariances not only account for actual noises and disturbances in the physical system, but also are a means of declaring how adequately the assumed model represents the "real world" system. Once a particular filter has been tuned, an error budget [1924) can be established. Essentially, this consists of repeated covariance analyses in which the error sources in the truth model are "tuned on" individually to determine the separate effects of these sources. Square Root Filtering Measurement updating of the covariance matrix requires a rather long wordlength to maintain acceptable numerical accuracy. The difficulties encountered in converting a Kalman filter tuned over a long wordlength, to an effective algorithm on a small wordlength on line computer are well documented [25]. To circumvent these problems in numerics inherent to the Kalman filter algorithm, alternate recursion relationships [2527] have been developed to propagate and update the covariance matrix in a square root sense. The square root approach can yield twice the effective precision of the conventional filter in illconditioned problems. Another alternative is the UD covariance factorization filter which although is not actually a square root filter is closely related to it. Suboptimal Filtering The dimensions of the matrices that are involved sometimes become so large as to render their manipulation virtually impossible in a particular digital computer. Also, when the state vector is large, it is often impracticable to compute estimates for every component and it becomes necessary to consider only a subset of the components. To reduce the dimensionality of the overall system and still retain a reasonable model, methods which are not optimal in the strict sense have been devised [28]. Compensation of Linear Model Inadequacies There is always some discrepancy between the performance indication propagated by the filter and the actual performance achieved in realistic applications, because the model embodied in the filter cannot be exactly correct. Such discrepancy is termed divergence [1824,2932]. Compensation techniques have been used for model inadequacy. The first of these methods is the addition of pseudonoise to the assumed model and artificial lower bounding of error covariance matrix elements [29,30,32]. By adding such fictitious noise to the dynamic model one "tells" the filter that it should decrease its confidence in its own model. In the case in which a linear model is indeed adequate, but only for a limited length of time, then limiting of effective filter memory and overweighting most recent data [30,3335] and finite memory filtering [30,36] methods are used. Finally extended Kalman filtering [30,37] attempts to exploit linear models and methods in the case in which a substantially better depiction of the true system would be in the form of a nonlinear model. The basic idea of the extended Kalman filter is to relinearize about each state estimate once it has been computed. Linear Smoothing Problem There are three classes of smoothing problems, namely the fixedinterval, fixedpoint and fixedlag [8,38]. Fixedinterval technique is used for postexperiment data reduction to obtain refined state estimates of better quality than that provided by online filters, i.e., postflight analysis of a missile. Fixedpoint smoothing is considered when there is a certain point (or points) in time at which the value of the system state is considered critical. For example conditions at engine burnout time are critical to rocket booster problems. Finally the fixedlag technique is used when it is desirable to delay the computation of a state estimate in order to take advantage of additional information. It is particularly applicable to communications and telemetry data reduction. Observers In some estimation problems, it may be desired to reconstruct the state of a deterministic, linear dynamical system  based on exact observations of the system output. For deterministic problems of this nature, stochastic estimation concepts are not directly applicable. Luenberger [39,40] formulated the motion of an observer for reconstructing the state vector of an observable deterministic linear system from exact measurements of the output. The stochastic observer unifies the concepts of deterministic Luenberger observer theory and stochastic Kalman theory, for continuous linear systems. Analogous results have been derived for discrete systems [41,42]. Adaptive Estimation In order to improve the quality of the state estimates, it would be desirable in many instances to estimate a number of uncertain parameters in the dynamics of a measurement model simultaneously in an online fashion. This is often termed combined state estimation and system identification [4349]. In addition, one would like to readjust the assumed noise strengths in the filter's internal model, based upon information obtained in real time from the measurements becoming available, so that the filter is continually "tuned" as well as possible. Such an algorithm is often termed an adaptive or selftuning estimation algorithm [4548,50,51]. Several authors use the information found in the innovation sequence to estimate the unknown noise covariances [4548,50]. Applications The earliest applications of the Kalman filter dealt with satellite orbit determination, tracking and navigation problems. Tapley and Ingram [52] used an extended Kalman filter to estimate the state and the unmodeled accelerations acting on an orbiting space vehicles. Cambell et al. [53] described the use of the filter for orbit determination for the Voyager spacecraft during its Jupiter fly by. Dawn and Fitzgerald [54] presented the application of a Kalman filter in numerous phased array radars to track satellites, reentry vehicles and missiles. Application of Kalman filtering to spacecraft range residual prediction was studied by Madrid and Bierman (55]. They developed a UD covariance factored Kalman filter to be able to evaluate the validity of range measurements to a distant spacecraft in nearreal time so as to be able to detect receivingstation hardware problems as they occur. The Kalman filter is admirably suited to the problem of monitoring measurement consistency and the related statistics. Additional study was done on several versions of the extended Kalman filter which is used to estimate the position, velocity and other key parameters associated with maneuvering reentry vehicles [56]. The Kalman filter is also well suited for application to the problem of antiaircraft gun fire control. Berg [57] made use of the Kalman filter theory to develop an accurate, numerically efficient scheme for estimating and predicting the present and future position of maneuvering fixedwing aircraft. Application of modern estimation techniques to multisensor navigation systems began in the mid1960's. Because the errors in a typical navigation system propagate in essentially a linear manner and linear combinations of these errors can be detected from external measurements, the Kalman filter is ideally suited for their estimation [5861]. Application of Kalman filtering on ship motion prediction and position control [6264] have many features in common with the tracking and navigation problems. The modeling of ship/wave interactions requires the development of simple, but effective, models using signal processing or identification techniques. The authors are concerned with the Kalman filter performance in the context of a control system. For remote sensing applications [65,66] the Kalman filter appears as a part of a signal processor that is used for offline analysis. These applications are not concerned with realtime processing. Nonetheless, simple models are used to accomplish the processing, and the performance is based on the achievement of the general goals of the system. Estimation and control problems abound in industrial processes and power systems [6771]. These applications impose the necessity for realtime operation, generally with limited computational capability, using poorly defined models of the process. Simplicity and adaptability are basic concerns in these applications. Application of Kalman filtering has been extended to the geophysics area. A Kalman filter was developed as a whitenoise estimator for seismic data processing in oil exploration [72]. The Kalman filtering approach was used to obtain optimal smoothed estimates of the socalled reflection coefficient sequence. The sequence contains important information about subsurface geometry. Ruckebusch [73] described an application of Kalman filtering to a geophysical subsurface estimation problem. Other areas where the theory of optimal estimation has applications are tracer studies in nuclear medicine, statistical image enhancement, estimation of river flows, network and load forecasting [74,75], classification of vectorcardiograms and demographic models [76]. Logarithmic Number System In 1971 Kingsbury and Rayner [77] first outlined logic hardware and software function approximations to the addition of two positive numbers in logarithmic arithmetic. Swartzlander and Alexopoulos [78] concentrated on ROM based hardware for the fastest addition but with a wordlength limit of 12 bits. Later Lee and Edgar [79] developed efficient algorithms to implement 8 and 16 bit logarithmic arithmetic on microprocessors. They established that unlike floatingpoint arithmetic, which provides accuracy at the expense of speed, the LNS provides for both and is suitable for advanced DSP applications. Kurokawa et al. [80] applied LNS to the implementation of digital filters and demonstrated that it gives filtering performance superior to that of a floatingpoint system of equivalent wordlength and range. Similar observation was made by Swartzlander et al. [81] when applying LNS to Fast Fourier Transform processor. The general conclusion of these studies is that addition, subtraction, multiplication and division are very fast and easy to implement in LNS. The advances in semiconductor memory technology have renewed interest in the LNS over the last few years. A brief survey of the literature in the LNS will be presented. Logarithmic Conversion Multiplication and division operations in computers are usually accomplished by a series of additions, subtractions and shifts. Consequently, the time required to execute such commands is much longer than the time required to execute an add or subtract command. In an early work by Michell [82] a method of computer multiplication and division is proposed which uses binary logarithms. The logarithm of a binary number is determined approximately from the number itself by simple shifting and counting with no table lookups required. Simple add (or subtract) and shift operations are all that is required to multiply or divide. However, since the logarithms used are approximations of the actual logarithms, the operations yield errors. The paper discusses a method of finding approximate basetwo logarithms and an analysis of maximum errors that may occur as a result of approximations. A more vigorous analysis of errors in the multiplication and division operations using binary logarithms is presented by Hall et al. [83]. In this paper the authors discuss algorithms for computing approximate binary logarithms, antilogarithms and applications to digital filtering computations and establish that logarithmic techniques are well suited for parallel digital filter banks and multiplicative digital filters. Marino [84] presents a parabolic approximation method for generation of binary logarithms with an increased precision over Hall et al. [83], where a review basetwo logarithm and antilogarithm computation using a fixed number of iterations is presented. Their method is claimed to yield an improved accuracy and can be implemented as a subroutine or using a hardware peripheral device. Addition and Subtraction Algorithms Kingsbury and Rayner [77] were the first to propose two methods of adding or subtracting logarithmically encoded numbers. In the direct method of addition and subtraction, they use approximate evaluation of logarithms. However, their second method is based on table lookups using ROM which leads to potentially faster operations. The paper also indicates a possible method of table reduction. In 1975, Swartzlander and Alexopoulos [78] unaware of the previous work by Kingsbury and Rayner, proposed a sign/logarithm system along with arithmetic algorithms identical to those in [77]. Their paper suggests hardware architectures for arithmetic units and includes comparison of speeds with conventional arithmetic units. Ironically, in a paper by Lee and Edger [79], the LNS was reinvented a third time but enriched with ideas for implementation in 8 and 16 bit microcomputers. A more rigorous treatment of LNS addition and subtraction algorithms with respect to storage efficiency, accuracy, range, noise and array area is found in Lee and Edger [85]. Multiplication and Division Algorithms In LNS, multiplication and division are achieved by simply adding or subtracting logarithms and are thus straightforward operations. These algorithms along with examples and hardware implementations may be found in references 7779 and 85. Table Reduction Techniques Current algorithms for addition and subtraction operations in LNS require a table lookup operation. The size of the table is governed by the wordlength size. A 16bit arithmetic would require 64K of storage. The current state of the art in high speed ROM and RAM is such that LNS is useful for wordlengths up to 1213 bits for high speed operations and 16 bits for lower speeds. Due to the discrete coding and nature of the functions filling lookup tables, a large portion of the values is zero. The number of zeros depends on the radix used in the number system and the number of fractional bits allotted in the word format. Kingsbury and Rayner [77] suggested two ways of reducing the storage capacity by employing read and compare successive approximation process and linear interpolation technique. Lee and Edgar [85] determined a cutoff value for the address below which the table output is zero. Extended Precision LNS The aforesaid address space limitations of contemporary high speed memories limit the admissible logarithm wordlengh to 1213 bits. To overcome this problem, Taylor (86,87] suggested a modified interpolation technique to be useful in increasing the wordlength of practical logarithmic arithmetic unit without adding any significant hardware burden. Digital Filtering Using LNS An early work by Kingsbury and Rayner [77] implemented a second order recursive lowpass filter with a 16 bit logarithmic arithmetic and demonstrates the improvement in dynamic range and performance compared to 16 bit fixed point arithmetic. Kurokawa et al. [80] recently applied logarithmic arithmetic to the implementation of digital filters and present an analysis of roundoff error accumulation in the direct realization of logarithmic digital filter and establishes that LNS gives filtering performance superior to that of a floatingpoint system of equivalent wordlength and range. Statistical roundoff error behavior of a single logarithmic arithmetic operation may be found [80]. In another paper by Sicuranza [88], preliminary results on two dimensional filters implemented with LNS are presented. FFT Implementation with LNS Swartzlander et al. [81] presented a FFT implementation scheme with LNS and established that this approach resulted in an improved error performance for a given wordlength over FFT's implemented with conventional fixed or floating point arithmetic. Other Applications with LNS Swartzlander and Gilbert [89] concluded that convolution units using LNS arithmetic have a figure of merit twice that of two's complement merged arithmetic and tens of orders higher than high speed modular array. Besides, they claim that the advances in VLSI technology offer a strong potential for LNS applications. Taylor [86,87] presented an approach to a fast floating point arithmetic unit employing LNS. His approach does not 20 require exponent alignment, supports high speed addition and multiplication and admits a simple VLSI realization. CHAPTER THREE KALMAN FILTERS Statement of the Problem Application of optimal estimation is predicated on the description of a physical system under consideration by means of mathematical models. Early work in control and estimation theory involved system description and analysis in the frequency domain. Recent works have involved system descriptions in the time domain. The formulation used employs statespace notation which offers the advantage of mathematical and notational convenience. Moreover, this approach to system description is closer to physical reality than any of the frequencyoriented transform techniques. It is particularly useful in providing statistical description of system behavior. Initially, the work done in the time domain was concerned with continuous systems, but recently work was extended to the discrete case in which information is available or designed only at specified time intervals. With the use of digital computers or microprocessors, discrete systems can be simulated easily and computation burden is avoided considerably. A discrete stochastic dynamic system can be represented as xi = Fxi + wi (3.1) Z. = Hxi. + v. (3.2) Where i 0,1,2 ...... x. = nxl state vector F = nxn state transition matrix (constant) w. nxl vector of Gaussian input white noise z. mxl output vector H = mxn output matrix (constant) v. mxl vector of Gaussian measurement errors (white) The above equations are illustrated in Figure (31). Least Square Estimation The linear leastsquares problem involves using a set of measurements z, which are linearly related to the unknown quantities x, by the expression z Hx + v DISCRETE SYSTEM MEASUREMENT Figure 31. A Discrete Stochastic Dynamic System. where v is a vector of measurement "noise." The goal is to find an estimate of the unknown, denoted by x. In particular, given the vector difference z Hx (3.3) ^ we wish to find the x that minimizes the sum of the squares of the elements of zHx. The vector inner product generates the sum of squares of a vector. Thus, we wish to minimize the scalar cost function J, where J = (zHx) (zHx) (3.4) Minimization of a scalar, with respect to a vector, is obtained when S= 0 (3.5) ax and the Hessian of J is positive semidefinite. Differentiating J and setting the result to zero yields H Hx H z (3.6) It is readily shown that the second derivative of J with respect to x is positive semidefinite; and Equation T (3.5) does indeed define a minimum when H H possesses an inverse (i.e. when it has a nonzero determinant); the leastsquares estimate is T 1 T x (HTH) H z (3.7) Estimation of Parameters Using Weighted LeastSquares A more optimal way to estimate the state vector x is the weighted leastsquares estimate. For this estimate we choose x, as the value of x that minimizes the quadratic for J=.5[(xx') TM (xx')+(zHx)TRl(zHx)] (3.8) M and R are the inverse matrices of the expected values of (xx')(xx')T and (zHx)(zHx)T respectively. Note that x' is an estimate of the state before the measurements were made. With this choice of weighting matrices, the weighted leastsquares estimate is identical to the conditionalexpected value estimate assuming Gaussian distributions of x and v. To determine x, consider the differential of Equation (3.8) dJdxT[M1 (xx')HT R1 (zHx)] (3.9) In order that dJ0 for arbitrary dxT the coefficient of dxT in Equation (3.9) must vanish 1 T 1 1 T 1 (M +H R H)x=M x'+H R z (M I+HTR H)x'+HTR (zHx') or T 1 x x'+PH R (zHx') where P = MI+HTR1 H (3.10) The quantity P in Equation (3.10) is the covariance matrix of the error in the estimate x; that is, we have T P=E[(xx)(xx)1] (3.11) Since M is the error covariance matrix before measurement, it is apparent from Equation (3.10) that P, the error covariance matrix after measurement, is never larger T 1 than M, since H R H is at least a positive semidefinite matrix. Thus, the act of measurement, in the average decreases (more precisely, it never increases) the uncertainty in our knowledge of the state x. Another noteworthy property of the estimate is the fact that T E[(xx)x] = 0 that is, the estimate and the error of the estimate are uncorrelated. In this case where x and v are Gaussian, this implies that x and (xx) are independent; in other words no A A improvement in (xx) can be obtained by knowledge of x or z. Recursive Filters A recursive filter is one in which there is no need to store past measurements for the purpose of computing present estimates. Consider a discrete system, whose state at time tk is denoted by x(tk) or simply Xk, where wk is a zero mean white sequence of covariance Q. xk FXk1 + Wkl (3.12) Measurements are taken as linear combinations of the system state variables, corrupted by uncorrelated noise. The measurement equation is written in vectormatrix notation as zk = Hxk + vk (3.13) where, zk is the set of m measurements at time tk, namely, Zlk,'z2k ...... Zmk arranged in a vector form. In addition, H is the measurement matrix at time tk; it describes the linear combinations of state variables which comprise z k in the absence of noise. The dimension of the measurement matrix is mxn, corresponding to mdimensioned measurements of a ndimensioned state. The term vk is a vector of random noise quantities (zero mean, covariance R) corrupting the measurements. Given a prior estimate of the system state at time tk, denoted xk() we seek an update estimate, xk(+) based on use of the measurement, zk. In order to avoid a growing memory filter, the estimate is sought in the linear recursive form xk(+)=KXk()+KkZk (3.14) where K. and Kk are timevarying weighting matrices, as yet unspecified. Although the following deviation is for an assumed recursive, single stage filter, the result has been shown to be the solution for a more general problem. If wk, vk, are Gaussian, the filter we will find is the optimal multistage filter; a nonlinear filter cannot do better [2]. Discrete Kalman Filters It is possible to derive the Kalman filter by optimizing the assumed form of the linear estimator. An equation for the estimation error after incorporation of the measurement can be obtained from Equation (3.14) through substitution of the measurement Equation (3.13) and the defining relations Xk(+)Xk+Xk(+) (3.15) Xk()=Xk+Xk() (3.16) The result is Xk (+)=(K+KkHI)Xk+K(Xk()+KkVk (3.17) By definition E(vk]=0. Also if E[(xk()]0, this estimator will be unbiased for any given state vector xk only if the term K'+KkHI is zero. Thus we require K' = IKkH and the estimator takes the form Xk(+)=[IKkH]xk()+Kkzk or alternatively ^ ^ ^ Xk(+)=Xk()+Kk[ZkHXk()] (3.18) The corresponding estimation error is from Equations (3.13), (3.15), (3.16), and (3.18) Xk(+)=[IKkH]xk()+Kkvk (3.19) Using Equation (3.19) the expression for the change in the error covariance matrix when a measurement is employed can be derived. From the definition ~ ~ T Pk(+)=E[xk(+)xk(+)T] Equation (3.19) gives Pk+)EIKH)Xk)XkT T T T] P k(+)=E[(IK kH)x k()[x k() (IK kH) +V k K k ]+ +KkV k[xk()T(IKkH) +VkKk ]} By definition T Pk()=E[xk()Xk()] Rk=E[VkVk TI and, as a result of measurement errors being uncorrelated T T E[xk()vk l=E[vkxk()T]=0 Thus Pk(+)=[IKkH]Pk()[IKkH]T+KkRKkT (3.20) The criterion for choosing Kk is to minimize a weight scalar sum of the diagonal elements of the error covariance matrix Pk(+). Thus, for the cost function we choose Jk=E[xk(+)TSxk(+)] (3.21) where S is any positive semidefinite matrix. The optimal estimate is independent of S; hence, we may as well choose S=I, yielding Jk =Trace[Pk(+)] This is equivalent to minimizing the length of the estimation error vector. To find the value Kk which provides a minimum, it is necessary to take the partial derivative of Jk with respect to Kk and equate it to zero. Use is made of the relation for the partial derivative of the trace of the product of two matrices A and B (with B symmetric), [Trace(ABAT)] = 2AB 8A From Equations (3.20) and (3.21) the result is 2[IKkH]Pk()H T+2KkR=0 Solving for Kk, T T lP^ Rr (3.22) k =Pk()H T[HPk()H T+R]1 (3.22) which is referred to as the Kalman gain matrix. Examination of the Hessian of Jk reveals that this value of Kk does indeed minimize Jk" Substitution of Equation (3.22) into Equation (3.20) gives, after some manipulation, Pk(+)=Pk()Pk()HT [HPk()H T+R] HPk() =[IKkH]Pk() which is the optimized value of the updated estimation error covariance matrix. Thus far we have described the discontinuous state estimate and error covariance matrix behavior across a measurement. The extrapolation of these quantities between measurements is xk()=Fxkl(+) Pk()=FPkl(I(+)FT +Q The equations of the discrete Kalman filter are summarized in Table (31). Nevertheless the application of these equations requires knowledge of the input and output noise covariances statistics, Q and R respectively. Figure (32) illustrates these equations in block diagram form. The Kalman filter to be implemented appears outside the dashedline box. In the linear discrete Kalman filter, calculations of the covariance level ultimately serve to provide Kk, which is then used in the calculation of mean ^ values (i.e. the estimate xk). There is no feedback from the state equations to the covariance equations. Table 31. Summary of Discrete Kalman Filter Equations. System Model xk =Fxkl+Wk'I Wk~N(O,Q) Measurement Model z k =Hx k+V, v~N(0,R) Initial Conditions E[x0]=x0, E[(x0x0)(x0X0) T]P0 Other Assumptions E[WkVj T=0 for all j,k Extrapolation State Estimate xk ()=Fxkl(+) T Error Covariance Pk()=FPkl (+)F +Q State Estimate Update xk(+)=xk()+Kk[zkHxk()] Error Covariance Update Pk(+)=[IKkH]Pk() Kalman Gain Matrix Kk=Pk()HT[HPk()HT+R]1 MATHEMATICAL MODEL DISCRETE SYSTEM I _7 EASuFEMENT DISCRETE KALMAN FILTER xk L __J Figure 32. System Model and Discrete Kalman Filter. Kalman Filters with Deterministic Inputs When the system under observation is excited by a deterministic timevarying input, u, whether due to a control being intentionally applied or a deterministic disturbance which occurs, these known inputs must be accounted for by the optimal estimator. The dynamic system containing deterministic inputs can be described as xi Fx i1 + wi_1 + u_1 Z. = Hx. + v. In order for the estimator to be unbiased the state estimate update is given as follows: xk(+)=Fxkl(+)+ukl+Kk[ZkHFx kl(+)Hukl] (3.23) By subtracting x from x, it is observed that the resultant equations for x are precisely these obtained before. Hence, the procedures for computing P,K, etc., remain unchanged. CHAPTER FOUR LOGARITHMIC NUMBER SYSTEM Various number systems have been in use for signal processing. Traditionally, when a large dynamic range and high precision are required, data are given a floating point representation. The problem with floating point arithmetic is that operations are slow compared to their fixed point counterparts. On the other hand, fixed point arithmetic offers high speed in the expense of dynamic range and precision. Recent developments in the area of logarithmic number systems (LNS) have proven it to offer some significant advantages over other number weighted systems like the fixed or floating point. The main advantages of LNS are the extended dynamic range on which they operate and the high speed of operations accompanied by a remarkable regular data flow. LNS Representation In a floating point system, a real number X can be approximated as +e' Xm r ; i/r where r is the radix, mx is the unsigned Mbit mantissa and e' is the unsigned Ebit exponent. In an LNS environment, a x real X is represented as +e x) X+r ; exzex+x; Ax=logr(mx) where in practice X is coded as a (N+2)bit word. The first two bits represent the sign of X. The Nbit exponent ex is represented as a Nbit word consisting of F fractional bits and NF=I integer bits. By distributing the appropriate number of bits for the integer and fractional parts, both large dynamic range and high precision can be achieved. More specifically, this system is characterized by lxmax=r 2(2 l)=r2 2 F(2NI)~r2I lximin r N r 2F+l(2N_1) 2 I1+1 RA=r r where Xl max and IXlmin are the largest and smallest positive numbers in LNS, respectively. In addition, RA is the range of this system (the ratio of the largest to the smallest number). LNS Arithmetic Operations Following the previously adapted number representation, arithmetic in LNS is described below. For the fastest execution of operations like addition and subtraction, table lookup techniques have been employed to support them. Multiplication In LNS, the logarithm of the product of two numbers is given by the sum of their logarithms. Specifically, multiplication in this system is performed in the following manner. e eb e For A=r a, B=r C=r c, S=sign bit C=AB; ec4ea+ea; Sc=SaOeb where 9 denotes an "exclusiveor" operation. Overflow occurs when the magnitude of the result exceeds the largest number which can be represented for the given word format. Overflow and underflow can be easily detected, by a simple comparison with the largest and smallest numbers representable in the system. Addition and Subtraction Addition and subtraction in this number system are defined in terms of the identity A+B=A(1+B/A). In particular, addition and subtraction in LNS are performed in the following manner. ea eb ec For Ar Br ,Cr Ssign bit and assuming without loss of generality that A>B C=A+B; ec<ea+r (v); v=ebea; tv=logr(l+rV); Sc=Sa CAB; ecea+er0 (v); v=ebea; ev=logr(lrV); Sc=Sa The realization of t and e is achieved through the use of table lookup operations from highspeed, highdensity RAMs or ROMs. Here, A Nbit address would be presented to a memory unit and the value of t and 9 would be obtained as direct table lookup. Based on contemporary highspeed memory chip limitations, the wordlength of v can be estimated to be on the order of 12 bits. PLAs have been also used in place of ROMs in order to reduce table requirements. A Hybrid Floating Point Logarithmic Arithmetic Unit When a large dynamic range and high precision are required, data must often be given a floating point representation. Although standardization is on the way, there remains today many floating point formats. The problem with floating point arithmetic is that operations are slow and complex. Furthermore, floating point addition can cause a special problem. The time it takes to perform a floating point addition can be markedly dependent upon the relative values of the data to be added. As a result, developing efficient real time code in floating point is difficult. In addition, the multiplier and addition paths are sufficiently different so as to demand two separate data paths and hardware subsystems. As a result, the utilization rate of the hardware floating point unit can be as low as 50%. Below we present a variation of the floating point arithmetic unit. It incorporates the precision and dynamic range of the floating point system while eliminating the disadvantages of high overhead and reduced throughput due to the exponent alignment requirement. This new concept shall be referred to as the Florida University Floating (Point) Unit or (FU)2 [87]. Floating Point Format In a floating point environment, a real number X can be approximated by Equation (4.1). Arithmetic in the floatingpoint system is performed in stages. Multiplication is a multistep process given by 1. Multiply mantissas and add exponents. 2. Postnormalize mantissa and round result. 3. Adjust exponent if required. In a commercial floatingpoint adder/subtractor unit, up to onethird of an arithmetic cycle can be consumed in an exponent alignment process. In addition, the length of time required to complete an exponent alignment is data dependent and therefore variable. For example, in a LSI11, exponent alignment can take up to 7 us with a basic mantissa add time of 42 ps. Formally, in a floatingpoint add, the following steps are taken: 1. Align exponents and shift mantissas accordingly. 2. Add mantissas. 3. Postnormalize mantissa and round result. 4. Adjust exponent if required. 2 (FU) Unit The floating point logarithmic processor, shown in Figure (41), consists of a floating point to LNS encoder, a LNS arithmetic unit, and a LNS to floating point decoder. 2 Several key observation can be made about the (FU) 2 Unit. They are 1. No exponent alignment is required for addition. 2. The multiplier is a subset of the adder. 3. Independent of the values of X and Y, all adds will occupy the same amount of time. 2 As a result, both a fast and low overhead (FU) unit can become a reality if the mapping denoted f and Y can be mechanized efficiently as a table lookup operation. For r2, the computed address vba can be presented to a preprogrammed memory unit which responds with the value of *(v)log2(l+2V ). In order to be consistent with the addressing space limitation found in modern memory units, v is currently limited to about 12 bits of precision. Taylor was able to extend this figure using a linear interpolation scheme [86]. Lee and Edgar also introduced a data compression policy which ignored all addresses which map, under 4, into zero [79]. Along a similar line of reasoning, it can be argued that the mapping Y and T~ can also be performed as a table lookup operation. The LNS arithmetic unit contains both an adder and multiplier. This is unlike a conventional floating point system where the adder and multiplier paths are separate. Nested multiply and add operations can be interleaved among 2 several (FU) units. In a conventional floatingpoint unit with distinct multipliers and adders, a utilization factor FLOATING POINT TO LNS ENCODER 7F7 hJ LNS ARITHMETIC UNIT ADD m MULT II i x=M e' ADDITION x j PATH I FLOATING POINT TO LNS II DECODER SI I ^ TADD ADD MULTI . I ^* ~~MULTIPLICATION 'I  UI MULTY' y=m e I  I I e' Y L___ J 2 Figure 41. (FU) System Architecture. 44 2 as low as 50% can occur. The (FU) units can be made to internally switch data paths so as to support both operations (i.e. 100% utilization). This can be a very 2 important property if the (FU) is to be used to support systolic or other data path determined architectures. CHAPTER FIVE ERROR MODELS FOR LNS ARITHMETIC The implementation of filters with digital devices having finite wordlength introduces unavoidable quantization errors. There are three main sources of quantization error that can arise: input quantization, coefficient quantization and quantization in arithmetic operations. The development of quantization error models for LNS arithmetic will be presented with special emphasis on input quantization. Once a model to represent input quantization error has been developed, models to represent the other two types of error can be easily be obtained. Input Quantization A quantizer can be viewed as a nonlinear mapping from the domain of continuousamplitude inputs onto one of a countable number of possible output levels. In this error analysis the input is presented as an infinite precision floating point number and it is mapped to LNS with the use of a floating point to LNS encoder, illustrated in Figure (41). The only source of error in forming ex is the one resulting from the logarithmic mapping m x logr mx which is performed as a memory table lookup operation. The proposed error analysis model is illustrated in Figure (51). Here, the two paths associated with the formation of Ax and its finite wordlength approximation are shown as parallel paths. Upon receiving the real mantissa m x, the lower path provides the ideal mapping Ax=log rmx while the upper path consists of an input quantizer (QI, which provides a discrete value of * mx ), an ideal mapping of mx into L=logr mx and finally an output quantizer (Qo, which provides the machine version of * L, namely Ax RND(L). The input and output quantization errors can be defined as m* m* * EI=mxmx ; E =log r(m x *)[log r(m x )] where E and E are uniform white noise sequences possessing the following statistical properties: E[EI]=E[Eo]=0; E[EoEoj]=(q 2/12)8 *2 E[mxE]=E[(log (m* ))Eo]=0; E[EiEij]=(q2/12)6k NI NO Here, qI=2 and qo=2N where N and N are the numbers of bits available at the input and output of the logarithmic Figure 51. Input Quantization Error Model. mapping L. In addition we assume that mx is uniformly distributed over [l/r,l). The final error metric E is parameterized by D=EI/m x, as E = Ax Ax = logr (m x+E I)+Eologrm = logr(I+D)+EO Through the application of the theory of functions of random variables [90], the probability density function (p.d.f.) fD(D) is shown to be given by f(D)= mJf (Dmm) dm; q < D < q f D(D)= Im f ~Em (Dm x 'm x) dm X; q1 D wM 