UFDC Home myUFDC Home  |   Help
<%BANNER%>

State Estimation

Material Information

Title: State Estimation A Decision Theoretic Approach
Physical Description: 1 online resource (114 p.)
Language: english
Creator: Levinbook, Yoav Nir
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2007

Subjects

Subjects / Keywords: bayes, estimation, filter, kalman, minimax, restricted, risk, state
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The problem of state estimation with stochastic and deterministic (set membership) uncertainties in the initial state, model noise, and measurement noise is approached from a statistical decision theory point of view. The problem is initially treated within a general framework in which the state estimation problem is a special case. General existence results such as the existence of a minimax estimator and a least favorable a priori distribution are derived for the state estimation problem. Then, attention is restricted to two important cases of the state estimation problem. In the first case uncertainties in the initial state, model noise, and observation noise are considered. It is assumed that the a priori distributions of the initial state and the noises are not perfectly known, but that some a priori information may be available. The restricted risk Bayes approach, which incorporates the available a priori information, is adopted. When attention is restricted to affine estimators based on a quadratic loss function, a systematic method to derive restricted risk Bayes solutions is proposed. When the filtering problem is considered, the restricted risk Bayes approach provides us with a robust method to calibrate the Kalman filter, considering the presence of stochastic uncertainties. This method is illustrated with an example in which Bayes, minimax, and restricted risk Bayes solutions are derived and their performance is compared. In the second case only the initial state uncertainty is considered. The initial state is regarded as deterministic and unknown. It is only known that the initial state vector belongs to a specified parameter set. The (frequentist) risk is considered as the performance measure and the minimax approach is adopted. The search of estimators is done within the class of all estimators. If the parameter set is bounded, a method of finding estimators whose maximum risk is arbitrarily close to that of a minimax estimator is provided. This method is illustrated with an example in which an estimator whose maximum risk is at most 3% larger than that of a minimax estimator is derived.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Yoav Nir Levinbook.
Thesis: Thesis (Ph.D.)--University of Florida, 2007.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2007
System ID: UFE0021259:00001

Material Information

Title: State Estimation A Decision Theoretic Approach
Physical Description: 1 online resource (114 p.)
Language: english
Creator: Levinbook, Yoav Nir
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2007

Subjects

Subjects / Keywords: bayes, estimation, filter, kalman, minimax, restricted, risk, state
Electrical and Computer Engineering -- Dissertations, Academic -- UF
Genre: Electrical and Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: The problem of state estimation with stochastic and deterministic (set membership) uncertainties in the initial state, model noise, and measurement noise is approached from a statistical decision theory point of view. The problem is initially treated within a general framework in which the state estimation problem is a special case. General existence results such as the existence of a minimax estimator and a least favorable a priori distribution are derived for the state estimation problem. Then, attention is restricted to two important cases of the state estimation problem. In the first case uncertainties in the initial state, model noise, and observation noise are considered. It is assumed that the a priori distributions of the initial state and the noises are not perfectly known, but that some a priori information may be available. The restricted risk Bayes approach, which incorporates the available a priori information, is adopted. When attention is restricted to affine estimators based on a quadratic loss function, a systematic method to derive restricted risk Bayes solutions is proposed. When the filtering problem is considered, the restricted risk Bayes approach provides us with a robust method to calibrate the Kalman filter, considering the presence of stochastic uncertainties. This method is illustrated with an example in which Bayes, minimax, and restricted risk Bayes solutions are derived and their performance is compared. In the second case only the initial state uncertainty is considered. The initial state is regarded as deterministic and unknown. It is only known that the initial state vector belongs to a specified parameter set. The (frequentist) risk is considered as the performance measure and the minimax approach is adopted. The search of estimators is done within the class of all estimators. If the parameter set is bounded, a method of finding estimators whose maximum risk is arbitrarily close to that of a minimax estimator is provided. This method is illustrated with an example in which an estimator whose maximum risk is at most 3% larger than that of a minimax estimator is derived.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Yoav Nir Levinbook.
Thesis: Thesis (Ph.D.)--University of Florida, 2007.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2007
System ID: UFE0021259:00001

Full Text
xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101206_AAAADC INGEST_TIME 2010-12-06T15:49:59Z PACKAGE UFE0021259_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 44402 DFID F20101206_AABNQL ORIGIN DEPOSITOR PATH levinbook_y_Page_079.pro GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
SHA-1
d4822a2341688e0a7e2a24eb75854ce80f192b73
24542 F20101206_AABNPW levinbook_y_Page_006.jpg
69876fc1db0583acafc000daba17117a
6584 F20101206_AABNRA levinbook_y_Page_061thm.jpg
27571a1892012c398eced82bb67bf2df
a5d7a91a29cf17738d654a2d73410bc7d8681ca1
84485 F20101206_AABNQM levinbook_y_Page_060.jpg
d06752f0ddf8930ca76b290e1552aeeb
d974dff56c66e1d3998a149ed354fa98895a4456
1051889 F20101206_AABNPX levinbook_y_Page_019.jp2
674718bf064b3609877e99bcda7ac4a6
8f3549ddbe7bece17e893a099ea7274964859192
924 F20101206_AABNQN levinbook_y_Page_007.txt
c5eb8fcf67c53622fa922d1263fa08d1
3fb5dd67a10f9b68a7f6e47207e3d551db12b909
1178 F20101206_AABNPY levinbook_y_Page_072thm.jpg
b296265b2d1b6b00d6ec533334d69a17cbd025b8
115924 F20101206_AABNRB levinbook_y_Page_045.jpg
3f6c84a44e77148150a5db1bbfee732a
b5e43f5564b879442564922007114f5d282a35ce
41021 F20101206_AABNQO levinbook_y_Page_074.QC.jpg
cf080b83c72972e85efcd6c86ecb7625
81562914ac69cff7d257da30befd624feece295e
25271604 F20101206_AABNPZ levinbook_y_Page_016.tif
03e631eec4dabb6cd06a7e409345b9a1
df0de8c8497c317834345d26f67d917085b64501
16095 F20101206_AABNRC levinbook_y_Page_109.QC.jpg
2e8d04eed51a5af4941c29af10240184
b1806c8ceb2f1751ac798521f73508a4108c63bc
1051983 F20101206_AABNRD levinbook_y_Page_035.jp2
8bab6551b2dd7b2ffb4998459f50d9abe406795a
62560 F20101206_AABNQP levinbook_y_Page_083.pro
fae7afd43c43ae1ef42119bbdc444d2b
3bbbd3d47de0976f93b3be07b9e38bdb6cc9d7b5
8312 F20101206_AABNRE levinbook_y_Page_034thm.jpg
115a61f427cbdccbce5ff5e66cbfc286
29dff14810d0fbb67a44a389e53ee39a3d54884e
8215 F20101206_AABNQQ levinbook_y_Page_098thm.jpg
c3091768b31f337064c2614763456266
c11193277cb71a3e46eb4c1d3f9a751cbe26a1c8
7131 F20101206_AABNRF levinbook_y_Page_106thm.jpg
5156f9212caa2115fd75da4646dcbddd
c8bd485b86f5e470288c1cd052b4f73c92d5c431
2677 F20101206_AABNQR levinbook_y_Page_044.txt
0bdb76424071a513720ccc9e6bb83dfc
9a6ce6f9cfa9e80662a435835a025951d9a15d67
2428 F20101206_AABNRG levinbook_y_Page_062.txt
129be051ff829e1e67f0a7d141dbe2cc
2f58eeb3934c737804908efe551ef90d985bef73
31990 F20101206_AABNQS levinbook_y_Page_067.QC.jpg
c08e13f482daf868fd16c6ac745abcb25cf4d473
30109 F20101206_AABNRH levinbook_y_Page_079.QC.jpg
e6ee9eaa372a3c8a577ca7d47fb474d9
4d216c2e7359153dea6b0ec5cbb32eabe5e42fb7
59874 F20101206_AABNQT levinbook_y_Page_064.pro
8f180cb024f874841eccacc184a49217
0a7cef0cff86baa919be6f96d87ca5850ea2c3ab
37158 F20101206_AABNRI levinbook_y_Page_037.QC.jpg
f49668222a8859d44b0a033764fb10f6
5c370c80e2ce2abd15f7657019cb1a1ce9bbdb81
33774 F20101206_AABNQU levinbook_y_Page_020.QC.jpg
6f3f809729a2fdb0ba135692da4b445d
fa59d9803ed9f901228c600bf678d6fa0af9902c
21992 F20101206_AABNRJ levinbook_y_Page_007.pro
02388400399ec6b04ce1624fc0a18883
edea3e5631d48fafc5f272f217fc2413ba2115f0
38905 F20101206_AABNQV levinbook_y_Page_099.QC.jpg
9cbec60df2eb93ab984c271b04cf4241
d2c4a2954d3281856e0ac6d160177313dd096038
28234 F20101206_AABNRK levinbook_y_Page_100.QC.jpg
31b04ef4ee1db3cd9033bb6817be63a6
34200 F20101206_AABNQW levinbook_y_Page_016.jpg
cd7e25096820ef41d98c25291507c4f1
e96c2731189289d3ce192d4b087c0cd2e66d6aee
16340 F20101206_AABNRL levinbook_y_Page_016.pro
251c5b64cf4270c9e9b528913f4c3e0f
d0a1a1d264a35656e68f16686e83855716a1124d
1877 F20101206_AABNQX levinbook_y_Page_048.txt
076874d9775fbf3b878c5b935d63520e
28c7dcc558f992e63ed38618a743eba1a37d7870
1769 F20101206_AABNSA levinbook_y_Page_080.txt
eb5dca39f4f420e356f058d4b80ee348
0e842e73c6887361468dd7813cfe66dc1ede2ce4
F20101206_AABNRM levinbook_y_Page_100.txt
1c02c0556483290776d2cbce89286949
e414c5fcc85f15059ea73b8c50d7b156ac0146bd
1027978 F20101206_AABNQY levinbook_y_Page_067.jp2
8e1025d403d4c02cd9c678f5082d38f1
02fd13e620b3354095d1fb2ab663ebd8611d5926
93871 F20101206_AABNSB levinbook_y_Page_088.jpg
8a29d7049f27de63220af243bfb54a87
af10b4fc916863423bec1f71c76f841e0cb42709
7840 F20101206_AABNRN levinbook_y_Page_050thm.jpg
0f7e0ab46bc1e5736fe380ff2afa63f7
803c139aa939258ef6156b961d47abc6bb13a1df
F20101206_AABNQZ levinbook_y_Page_069.tif
b21ae8f4dbcdcd42f6cdf8155514f7c9
ea541e3800e794708a49dce372b35204dd086ae2
2152 F20101206_AABNSC levinbook_y_Page_032.txt
087fc95120fdba05df0a7ec52e7fa0f5
7a49c02bc417baf99653bf2331eba3c89c57c897
9401 F20101206_AABNRO levinbook_y_Page_083thm.jpg
e87415dc38ae1fbd16861f4099a489fb
f01b03f67f1517b44fedacf715ba496dc18e6c38
38523 F20101206_AABNSD levinbook_y_Page_086.QC.jpg
9f333677c5b060dc9dfaab0eef0937e7
67330 F20101206_AABNRP levinbook_y_Page_015.pro
96293a654d09b4616cc043f8f26da7bd
d56d1f63eb455bc15a571057507d29e80b0b9456
F20101206_AABNSE levinbook_y_Page_060.tif
0c12a0f2f9302972aafd4e2ab4739b34
6df192abe270a2fd950527dd784752e4689f92b9
3820 F20101206_AABNSF levinbook_y_Page_105thm.jpg
F20101206_AABNRQ levinbook_y_Page_057.tif
a77ee93b4ca2177601e95ba3dcce62ce
1404a956e1df50e11418c6e83c5a9a454d1257ff
69119 F20101206_AABNSG levinbook_y_Page_102.pro
d926432736fc23f770dbb897d5558a2f
9de3f83755b9d78bff7fe4558fd63d25fb5a5cf6
2891 F20101206_AABNRR levinbook_y_Page_016thm.jpg
418a0ee419bdb65bee54f6b58a8c770a
d655c6b4caf6bf8b3159c3ba004e03f78cf593fd
F20101206_AABNSH levinbook_y_Page_109.tif
e2638c114f47e9e9e0d8daa402ec201e
496fd491c1e2966fced485a5b4000bd68f4a7bf1
1053954 F20101206_AABNRS levinbook_y_Page_078.tif
cf7885b2228345c5cafb7903c3afc215
34317145af449875045b77d2a298619cb8a471b2
67683 F20101206_AABNSI levinbook_y_Page_075.pro
9db9cf53eabe6e207db0ec596f37f5e0
72f4dd804a67d9ba706fb489416f61e94ae94f2d
120673 F20101206_AABNRT levinbook_y_Page_062.jpg
2333941229cbedda7715a4d6fc5e43352b2392e5
F20101206_AABNSJ levinbook_y_Page_086.tif
fd786fba2c72ebf332648d2b85eb20ab
1a9e673c3f26d19948ee957e3dcfb4b62d23e0bf
16592 F20101206_AABNRU levinbook_y_Page_007.QC.jpg
fb7432000e6b2eaf4b14e7208ed3068b
0aaf802766bb1e570d260655dc5f2bcafb20a679
2025 F20101206_AABNSK levinbook_y_Page_059.txt
198b2c2f1b66cfdb1d330e2bbab72fff
862e422eed87cefa281f6f375aab4285b8f0d083
37173 F20101206_AABNRV levinbook_y_Page_005.QC.jpg
feb54317cb358dd57275c82a5696ba08
8a7b9f7f35ea2e661b230ff09d46bc6a079e28a8
37622 F20101206_AABNSL levinbook_y_Page_107.QC.jpg
f635995399e9c2b8a813756550d92ea3
9896ee4bfc27715e1d1984de7a83df0a3dd7d543
40894 F20101206_AABNRW levinbook_y_Page_100.pro
b58527e1f6a375cb69a743be9c5f1b20
a8ccfc47171e63724dde747dd612ef7536e74ab3
82012 F20101206_AABNTA levinbook_y_Page_022.jpg
cc063d5ba592e8e4c295e98c22af1686
1f732846dcaaf810c44d44cdb5180e287a1f6444
F20101206_AABNSM levinbook_y_Page_080.tif
3bc9468572f88d8c58ac8d1f246a5bf3
e5a6f6bb8496795d68f9b491d709e4ef85e0fb76
2634 F20101206_AABNRX levinbook_y_Page_015.txt
a395981dc4f5769685a3e7110416c216
57e0e74ab3067281752816616ae1ef924cecac21
40065 F20101206_AABNTB levinbook_y_Page_070.QC.jpg
bf3f32999207f20f21e09e9b5c86d621
881311 F20101206_AABNSN levinbook_y_Page_053.jp2
a605ba72331febf821b3cf075bf2e52c
4a0299a05912fa7a11392a3864f5f8d5e0ff3f1e
7293 F20101206_AABNRY levinbook_y_Page_060thm.jpg
b728201d6d621b19d3735b187173e7b9
7715d52366cf12bb143c8e65dc7411a08750167f
40590 F20101206_AABNTC levinbook_y_Page_022.pro
27a6251b88324bb882e7a2e016618823
7478c8a3082396e8874963814a067a0e5f072c0b
799437 F20101206_AABNSO levinbook_y_Page_007.jp2
1ca7a0316abe360d57305313de3ee158
411269a911858eac6e41edb014791ff03bd408b7
118765 F20101206_AABNRZ levinbook_y_Page_086.jpg
f819b082086e80aa1b0eee6ed5d01ac1
e066f984453f62b55a401219469f821a2cdb0204
2522 F20101206_AABNTD levinbook_y_Page_017.txt
97815b9941f4a498ba5d3b83554e64a151f5487c
132185 F20101206_AABNSP levinbook_y_Page_075.jpg
961515a9034409c6b327e67f16d956d3
c3752818012d2475daf5368796a187d0da7b9ab2
29419 F20101206_AABNTE levinbook_y_Page_096.QC.jpg
f458192dfc951ae351f8c8de4a3aa5ab
e117a67654cc8dcb04cac941aea655880e85eb68
124697 F20101206_AABNSQ levinbook_y_Page_066.jp2
f37fd77a1b79aca2a823978b77e60d59
38231 F20101206_AABNTF levinbook_y_Page_035.QC.jpg
bfac9ec8666c23a4a38e95666baa3dae
7f2bbd754ef36732a85d9ce3af4558571d2c5c3d
8441 F20101206_AABNTG levinbook_y_Page_068thm.jpg
89f6cb5c8ae983cabe902820bc5c048e
da7d4c533244dc4764152e172da90e6126c79d65
1000818 F20101206_AABNSR levinbook_y_Page_063.jp2
37238 F20101206_AABNTH levinbook_y_Page_038.QC.jpg
59740e6028c979211c35b7fd5aa12223
8925ae3c033d6f85863f1d6da836e5f408634656
60203 F20101206_AABNSS levinbook_y_Page_042.pro
25ceca3f2afef806ee8a725bdb514122
b7d6a760e867013f1bb81c4a43b76f91de5c7bc2
44950 F20101206_AABNTI levinbook_y_Page_046.pro
006de5b25622674997ff277714b86cd1
b220f0315bbab702ac9f50c86d04037091c4a5b1
2502 F20101206_AABNST levinbook_y_Page_071.txt
c1ff67b72f2950d18e5dc1b258111322
c4028b4f34018c6829dc46f6f6512b1d78a8c3ab
55569 F20101206_AABNTJ levinbook_y_Page_055.pro
28c10ce639a2ac7e14d692e0e66522ba
2e7cdefcf23a2a1a66b7e346fe9f2941eb8df599
72545 F20101206_AABNSU levinbook_y_Page_061.jpg
3424dda69e25786de279f2df7167b433
7b05414778ae7b82b741ba9180bb347a24faa58a
8850 F20101206_AABNTK levinbook_y_Page_107thm.jpg
a352e73d0601f577a0faca9117ee0520
3103057cc34673bb500a3814eb289cb822bcd549
127554 F20101206_AABNSV levinbook_y_Page_112.jpg
94fef925e52720be5b4cb1907c0357b8
93bd0ec4f9d1dab56fbde58fcf734e0ee03d1618
116255 F20101206_AABNTL levinbook_y_Page_009.jp2
708328a7e6ab759f5ecf05e3e161d2af
350b358a8c132ffa5de2e38cbfa462e759e46ee2
37689 F20101206_AABNSW levinbook_y_Page_064.QC.jpg
03ed932392b1f3c343527f0032fe2829
63d06406070ce4937e06d6454d96ce90277d731e
39336 F20101206_AABNTM levinbook_y_Page_083.QC.jpg
4101fc60e1af7fc4ddfacee545a5a133
ae71a0ce1ee1a4c98b80cf86c2f61484d782d27c
45329 F20101206_AABNSX levinbook_y_Page_063.pro
d2958020182b573d509a5a35a141fa5f
9f3bbcf7fb8b23f4b11a284f678992f085615e3a
5631 F20101206_AABNUA levinbook_y_Page_002.jp2
73c39d4a17f489145c560b8061edbd02
a7863c37b7ed55fafc2dc44c9c522abf58e9671a
2295 F20101206_AABNTN levinbook_y_Page_098.txt
4c1a57c95fc3712b7a97524fc2dc99b7
918fa38b747baa79a81f1b94b63ef332943613c8
34366 F20101206_AABNSY levinbook_y_Page_114.jpg
a84506a33e0eeaa24949b04cedf9431c
fabec7b5d2fb6982657e3efd834bd57d64bb5df6
2333 F20101206_AABNUB levinbook_y_Page_058.txt
273d266f146610affffe182cbbc18379
2af897324ce734379b33691b1d58db86043cde1d
2630 F20101206_AABNTO levinbook_y_Page_014.txt
c54fd5aa030dd018ff1644df4005b4c7
d1e6f03bace035f3a98a9b48d29acbf0bf83c26e
38312 F20101206_AABNSZ levinbook_y_Page_056.QC.jpg
352752a7dbfa1cd82a8dcf65c44eeae556809d97
88253 F20101206_AABNUC levinbook_y_Page_050.jpg
b0f5e78f5126d7818c174457dfa01e3c
809fb6f880eae4a319cb50a98ccd81106cbe7053
1800 F20101206_AABNTP levinbook_y_Page_094.txt
6c57708fca07e08f4a9e4ea86ef9ff5f30c2b271
31178 F20101206_AABNUD levinbook_y_Page_028.QC.jpg
c585f85b13653f3a3a7de62b71f1d070
c63050b9a6b344c58c63c4ec5d701e85369e3dc0
9390 F20101206_AABNTQ levinbook_y_Page_026thm.jpg
ebda5c0ba5a5f476d616a040e8de80bf
03c9afb8a86788808d144c38be66fe2fc52f86fc
39770 F20101206_AABNUE levinbook_y_Page_053.pro
1a98cd78d25142c18d4fc6877e5dbed1
a9ec83b415b0e3ed0cd0d5c7a8312ecd2c140017
7768 F20101206_AABNTR levinbook_y_Page_067thm.jpg
2aa8ac9ea387812342693b401a1bdaf36f9e55a2
1903 F20101206_AABNUF levinbook_y_Page_001thm.jpg
0969b9cc8945965ce45d9c29fca86ae582148d27
83505 F20101206_AABOAA levinbook_y_Page_080.jpg
fe867ebf9b7010404ac2698ff2909f79
4bedda60abbca471799c67e934a001e08070f4c1
3376 F20101206_AABNUG levinbook_y_Page_072.QC.jpg
341a0b45d8dc5e307d772e03c8835960
0862a441166f01b1f919df5ac1dd50d519af41f3
7917 F20101206_AABNTS levinbook_y_Page_021thm.jpg
95c0c34f8e9ba9c96086ef3f23527b12
9df962b0fedcf7562c257aee2a9b6acebbc13223
93794 F20101206_AABOAB levinbook_y_Page_081.jpg
b3428b210883f1cda48ff5902101dc93221124b5
150 F20101206_AABNUH levinbook_y_Page_072.txt
1051963 F20101206_AABNTT levinbook_y_Page_065.jp2
96ff5fea176d37e37b3efc665c407db7
ed3bb16d7976f5823a7fb97189835da4bfc51455
102322 F20101206_AABOAC levinbook_y_Page_082.jpg
2a503d09c53b49de6ef20beb607b4791
e7b819a2ec8ca718fda06241c8a91dcd48811c1f
F20101206_AABNUI levinbook_y_Page_037.tif
23ef1cacf03d399e888865d9954d1a6f
4d4e5c2898c634faf96941d44e79efcaf7fcab49
9312 F20101206_AABNTU levinbook_y_Page_041thm.jpg
6ceeaaeb9503021ff9ca9c863134f64e
1d8bb55ed56fc83e1a4a0b2c293f0113bc27e82f
a89749baee603b8dc5ae9061f520e3ab
566f7cda37f12e9d6f8edd561a498b3c4fc22948
95417 F20101206_AABNUJ levinbook_y_Page_021.jpg
ea79e5c37f3c56601ec7ef74bf25d2c4
aed5b500684b78f998852c094e8169a367ce50a6
F20101206_AABNTV levinbook_y_Page_017.tif
91e970fd4115a092755648ce2db9c4d7
173fbea3dfd8be2f56a4a7db81870f1a6e81f9c8
87698 F20101206_AABOAE levinbook_y_Page_096.jpg
482aa2b7e0c36ee5ac42cd6c51daa1c94aa52e7a
2446 F20101206_AABNUK levinbook_y_Page_009.txt
f70e49bf67137e48f641f04a533fdaf1
8529 F20101206_AABNTW levinbook_y_Page_055thm.jpg
5aa96523d5a96fdc999a6b9ac508fe8f
e79fa9aff1a1c9925fbdcde736fa82a42231bb17
114682 F20101206_AABOAF levinbook_y_Page_101.jpg
40fd61762e16f6a07ba64bd1be75d96d
68b5f1f9877a41f88fdfcfcd09bb794894c4c8ae
88152 F20101206_AABNUL levinbook_y_Page_046.jpg
e51ce48f1b2b00eeef2315e0d0dcb1eb
632cc282a7006d8299a02de7ca2cf75fd9d6c11d
7601 F20101206_AABNTX levinbook_y_Page_046thm.jpg
542e08eb77bb0119f7d22e6e73c8a871
492e2faa832e44a622cdcc1a1e592b2e078be96f
98789 F20101206_AABOAG levinbook_y_Page_106.jpg
a91b70a450216ab3977b52cb385d523c
8fbb3036c3e3f45175ffbed08902dc4fe0bfd722
7971 F20101206_AABNVA levinbook_y_Page_005thm.jpg
027a4ca2cbab84103053ebda7baa8c0b
d5a0797440a4c41552b251f3df320a6e70bf2efc
890933 F20101206_AABNUM levinbook_y_Page_097.jp2
9cd8d860ccb4e96696e3d44c5b89675b
00c0bf768c0ae068b13df3a951afd8047f242be0
34417 F20101206_AABNTY levinbook_y_Page_105.jpg
e6e8bf725f77d9417ab55de14d9c3c3c
21aca87b2c32918e73bf9717bc23142f86b1712f
21960 F20101206_AABOAH levinbook_y_Page_001.jp2
f5158e6b387e7a70686c4dfda525725e
584e75ddf7a613d3a68ccba6df584e8244aea9ae
F20101206_AABNVB levinbook_y_Page_014.tif
8ec26fbab2bd86ce2eeb0a2613b796f2
c8eb76a5e781c86c3ea02ac843fcb6996e69566d
1049193 F20101206_AABNUN levinbook_y_Page_024.jp2
5505e830645b2b8f8a759a99f166993d
031708b2e35947bdbdff78cc556b1d09415023d0
6465 F20101206_AABOAI levinbook_y_Page_003.jp2
8b69b392d51d61f70774cc681a39edb5
5a626e925bb9d4629cc07a9dc7dea8c36babd73d
F20101206_AABNVC levinbook_y_Page_064.tif
5068a22e5455d13089725349c73d87e7
379cb5850255d679bef3c88091b0d0913537c779
2115 F20101206_AABNUO levinbook_y_Page_052.txt
b1f4ec1511609e5b4e20f1332f0bd960
296360be1ba774e6e59c821304e90831e9b1f8e5
117255 F20101206_AABNTZ levinbook_y_Page_107.jpg
f048c0cc985b03916e26e87b2e5466f1
defb1fb923351328478124f5fb1f4de662d10f59
66897 F20101206_AABOAJ levinbook_y_Page_004.jp2
43c0ca450e877a4e4ec153b4294ff8b8
F20101206_AABNVD levinbook_y_Page_083.tif
dfed703fd33cf345de07992e053f0cb5
65bcef64465c11bed09f557c68251a0ec40783f3
121625 F20101206_AABNUP levinbook_y_Page_013.jpg
fe7589a9293e739eaec0f613703e555d
f26c9d92829972f15cc190fe095bc1b5a3333a62
15427 F20101206_AABOAK levinbook_y_Page_008.jp2
ac4d26cf4a888a1423e83e2c0c9cb3a4
6d2c567749cf6a004e28dbe8b467fb6d4d0fb059
112316 F20101206_AABNVE levinbook_y_Page_049.jpg
c0c0e3e3945b1c93f374983983aa497d
b3cd208e29569c1ccdf30da24e16f0b2b1b42e16
4018 F20101206_AABNUQ levinbook_y_Page_076.pro
f2cd6e8ce215883ac74357f1f043e4d4
ba84da87da320ebab967b4358b4b7ac25726da56
19869 F20101206_AABOAL levinbook_y_Page_010.jp2
a3d93509a30a3f863c429c3772d5e6d1
3c48f644441f16dc8db7b93fe9d56ffda15a76eb
F20101206_AABNVF levinbook_y_Page_047.tif
0cfc92fa2db7bceb42a3fd57a544bf1e
1852 F20101206_AABNUR levinbook_y_Page_063.txt
d47867f9a5a70c7fc0e0b97bd0df098f
e4da4f4349cefd6030bd82ae99ba561f734fb5f0
1051938 F20101206_AABOAM levinbook_y_Page_014.jp2
a8abdb6197f110b38a752115cb6fb811
bf61a9dafdd787192f240bd61bc7ef875a8fe4fa
2526 F20101206_AABNVG levinbook_y_Page_011.txt
3b327ef19f1f3050dbb65ba096d29cc7
2ee77821efe17ded0de91d654eb3502d2ce12991
F20101206_AABNUS levinbook_y_Page_085.tif
49e27350ae7cd40662e6fea37875b33d
26629 F20101206_AABOBA levinbook_y_Page_108.jp2
5221898911b9ff6e460b884c3e90149c
e8daaa3cbbb5c028b9a4305346751b419952fd2a
1051980 F20101206_AABOAN levinbook_y_Page_015.jp2
10394473e25f4c25a4a6b705759ff551
eb57dba6d010edffdcdb66999444ec3f8a739afb
8045 F20101206_AABNVH levinbook_y_Page_009thm.jpg
9af6a1819c389d4060a692b1a82ae387
501083 F20101206_AABOBB levinbook_y_Page_109.jp2
0b879639a38c5c4feebfa0d2b7c435e9
57dbd0997e09ab74e1506a6e93147a68f6727c52
927343 F20101206_AABOAO levinbook_y_Page_029.jp2
2ba4196124c18bb7046e84deea8b0f3e
bbd1422f22663002270e04ef41119660d09e7111
10784 F20101206_AABNVI levinbook_y_Page_023.pro
5a1f1aa2d8816103ee73f14b4f6461c2
27e58f3a096e0c69d8177263ac040d0a70132a28
141621 F20101206_AABNUT levinbook_y_Page_071.jp2
414bfdcfdaaf1d0bbb19cb2b78a3a800
beb276a4b196403d1a3aec16839288dc7323e46c
90361 F20101206_AABOBC levinbook_y_Page_113.jp2
782e2f306e0bed153db29a813ce6d6de
af2b4083be7f1c277dfac2905a3ffc66e226c98b
1051976 F20101206_AABOAP levinbook_y_Page_032.jp2
05ce067019896eca8940edb2ecebf801
0b872e44f8e53c9f82fdb61382900d72be972666
41434 F20101206_AABNVJ levinbook_y_Page_075.QC.jpg
c228bdef94aa96d719725254e2a7663c4027418c
F20101206_AABNUU levinbook_y_Page_091.tif
c24ce6a4242189e1f9125473a183a86f
bd25f2714ef1af3ffc699339e55492f440fba242
F20101206_AABOBD levinbook_y_Page_002.tif
50da5b0e59362faf8ff6ef2808fdca5208d44ab4
1051981 F20101206_AABOAQ levinbook_y_Page_038.jp2
9fc2636585929dfc76094dbb63241b9a
ebd0c64cb7095025575210ed00ae5d718205a7e1
40024 F20101206_AABNVK levinbook_y_Page_060.pro
507e04b9f09d388282039a3fae4c15cc
614e03f733a4f19cb3f6998b9f207aa8fa13bbf1
33664 F20101206_AABNUV levinbook_y_Page_090.QC.jpg
c243cde430c82931cdc558a2038c4f34
F20101206_AABOBE levinbook_y_Page_008.tif
c2c338772399b6a6dce823dc353570b8
4a54a7bb7429f84ca3d08d25fcf91692c388c047
1051955 F20101206_AABOAR levinbook_y_Page_040.jp2
d9b3c7fe160c2169fd47e5bf78fd72ef
c3f2e36cab2418f083e7fcb456e324644bf7f411
F20101206_AABNVL levinbook_y_Page_082.tif
3ea600bf07594073bdd863ffdbf9c339
2d5106a35269548130eef6b99bc67d972b9abd7f
54865 F20101206_AABNUW levinbook_y_Page_057.pro
107ef62a75d480583341121fa853858b
40c8159c15149fee4ebfafba6041d4b49a5702a5
F20101206_AABOBF levinbook_y_Page_010.tif
7f1d4423e34434762a77793aa184f69ba2d4a7dd
693300 F20101206_AABOAS levinbook_y_Page_047.jp2
4edaec77e620a78e494b963de232f960
034a503a9802355627899fa1b9b767264749dc54
47837 F20101206_AABNVM levinbook_y_Page_059.pro
428a60ba326cd0fbf5875779dbc761b5
6423ba36afe3d027a348423017ef5e1060e25188
57854 F20101206_AABNUX levinbook_y_Page_092.pro
24242981a4a6537a24b337a9df8b283f
41e5d136d53c422402db73a95c1259864fed3e59
F20101206_AABOBG levinbook_y_Page_013.tif
ab9b7dda3d1108b74caa9648e458deeac5efd361
8918 F20101206_AABNWA levinbook_y_Page_011thm.jpg
c5c2f282ce281dd863aa9a7b369f4203
10aa80f872c6a940bb844e3a4be055583f3884e8
1051964 F20101206_AABOAT levinbook_y_Page_056.jp2
cc01a473f645d76639699a941d9cdb22
84e18282063357206953c8dab6e476a9028c5c30
66594 F20101206_AABNVN levinbook_y_Page_047.jpg
3fae9beda536b772f6e0d4b8e0671f2f
3db567d09fdd4f0162527b018ff56bd83a01856f
53456 F20101206_AABNUY levinbook_y_Page_039.pro
aa003784fe1dab03446ed981c518b502
6d5b119a505d9d5788c6b4d32cb316fd57a95f81
F20101206_AABOBH levinbook_y_Page_019.tif
450399f520f33bc85aa7e8f41c61df2f
45338 F20101206_AABNWB levinbook_y_Page_028.pro
20a89521465213ebcc28434b10d8b203
7b54d0a4fca07588062ecdd4bf8ed01ae75a6252
1051979 F20101206_AABOAU levinbook_y_Page_057.jp2
798649 F20101206_AABNVO levinbook_y_Page_091.jp2
854cc6545097e3a7e4d294bffcdfba89
261b6c0e6878759368797179cbf23d2482eab2ef
8067 F20101206_AABNUZ levinbook_y_Page_082thm.jpg
7f1fdf4fb80578d69ce7219e0486bbd9
7bebc994ebe7e0c4152404964f01f36d1a54a4d8
F20101206_AABOBI levinbook_y_Page_024.tif
c2347f412964376206280e2910b6b8ec
262296ea1620f467240f05fc2fc9ffee8f65b338
32070 F20101206_AABNWC levinbook_y_Page_103.jpg
00149377711594948c87b6c65a976958
d424cb97e265690516704466161d97b2c3f44d68
F20101206_AABOAV levinbook_y_Page_064.jp2
8a0578b2c3bf3528e6e5fdc2fc09f7b3
413bb8c17af9bf6500343ac805eae07ff1909358
F20101206_AABNVP levinbook_y_Page_067.tif
97751997a53874e9dcc47c774a524ea4
cfbeedfe4e7728cd3dc7db1aa2b2081039ae9da0
F20101206_AABOBJ levinbook_y_Page_025.tif
03b05e78c0af0051ba30cc04474706f4
304e982c0e519fcfed11de3318fee31abeed4604
27972 F20101206_AABNWD levinbook_y_Page_097.QC.jpg
e11314d3067c5b7f35b79ef98da4dfab
6ea29b40174f123e881f04836cbb26aee6fb1107
1051950 F20101206_AABOAW levinbook_y_Page_069.jp2
c268c58b45a46f8d24a81ff82aae43cd
105855 F20101206_AABNVQ levinbook_y_Page_106.jp2
ea387587acc18fb6dfbba7ea793958434cdc4b7b
F20101206_AABOBK levinbook_y_Page_026.tif
a4bc04424b8698e04b4f7ae093025711
12230a112686922932b8e31e3f6df4c9e12bd622
1051917 F20101206_AABNWE levinbook_y_Page_037.jp2
9424d221021433b4a962f488815d8be1
90f48f1362290310ff947681d6b1054908332e16
123420 F20101206_AABNVR levinbook_y_Page_041.jpg
c7b9ab19d534b176bc0218caa33cbf3e
F20101206_AABOBL levinbook_y_Page_033.tif
cea907847357b1700ec0255128c43f65
d548d65f3115c3234f40c7b1a16e8c4b6456d705
F20101206_AABNWF levinbook_y_Page_113.tif
1818a49188a095ff5d83c7b26aeb597c
4d10187020cd178ca1dce6f24b3aa7e3279be54d
1051960 F20101206_AABOAX levinbook_y_Page_073.jp2
dd161f8d7e6c0e29cc7045d3b126f5ff0472d086
94421 F20101206_AABNVS levinbook_y_Page_059.jpg
61d3eb284a255b38d50b7302fbee2953
88f26d7c364bf3836349278fcc7bf9ac4458a2f8
67344 F20101206_AABOCA levinbook_y_Page_014.pro
0130d31ac0e9076198053678b43450e7524e036f
F20101206_AABOBM levinbook_y_Page_039.tif
91296d9172cb2e90902f0b46d005b1ce37e35801
966 F20101206_AABNWG levinbook_y_Page_109.txt
c4784ee82c8c579cd6b8b83d9168e63a
a67674412f93c5c42b02dabb1e6b3525d490778b
978193 F20101206_AABOAY levinbook_y_Page_088.jp2
b2be30118ccd1335cac4d52e4f8aa65c
d0633ed266a90760d4426753f65349c5786f32ae
104930 F20101206_AABNVT levinbook_y_Page_031.jpg
3256aed3b01cd7e1f68b3ace708a2632
c6cecff453ae35c53bb72c56d3cbb554e31820fb
27722 F20101206_AABOCB levinbook_y_Page_018.pro
b53b7069761623cf310c75296490a537
76ce9586f12800e5a592a3e72afde56c02965616
F20101206_AABOBN levinbook_y_Page_058.tif
15266f35f39ee0a0985ce9606077f477
bf5512cf6ab854ffc0531a433064419b470355ef
36291 F20101206_AABNWH levinbook_y_Page_089.pro
9249a60a817077a9a97befb9df0299b3
8efbe2607037f3c2947df8125febfa7784ea7a8d
1051984 F20101206_AABOAZ levinbook_y_Page_093.jp2
8f0bdf697ac93d90e85b8c3a7795f5e8e897dddd
58883 F20101206_AABOCC levinbook_y_Page_019.pro
b707c4d501ce2ebbc8fc3a609546b890
14e3229969e83b2a86a361f2ee4d2c3746630182
F20101206_AABOBO levinbook_y_Page_059.tif
00ce9f1a79b364cc96c4d32919c94703
db0a028684925dd4bc7fe4a47df1ec2ec1ee5154
914952 F20101206_AABNWI levinbook_y_Page_080.jp2
189a6f821f0b273796f8f14a489bbd60
7b8622280dc73629df7777e63c518db89642dcbe
6815 F20101206_AABNVU levinbook_y_Page_080thm.jpg
cdce81eb25800e130d6b0bf9363d7260
ba79875c66f37cffa7e23943ac9e2f5ee7b22049
59553 F20101206_AABOCD levinbook_y_Page_035.pro
889c202540ed52130b674fb3abb2bcbf
3edfe7fbb9bd9c410e2924a3f0854009f7a8d598
F20101206_AABOBP levinbook_y_Page_071.tif
fb960826175e2b31fabb586f9e9c12ed
92bac95b92bd92b3b2a1fdf80a1082f511e34841
2146 F20101206_AABNWJ levinbook_y_Page_040.txt
eaf002b78542f0d5984fcf3ea9331e59
c90ca0978e321945ba3eb3b78a3067455dd082ef
1289 F20101206_AABNVV levinbook_y_Page_076thm.jpg
04339882a73168d3df17b0e32f916d7b
47510 F20101206_AABOCE levinbook_y_Page_050.pro
ac18e965427688976cc224a70a9105e1
17e6a3e455133c6d21a408a181bca3058e88710e
F20101206_AABOBQ levinbook_y_Page_087.tif
fd37206e7305f1a02750a05b0b7883d5
16dbb38673bc1f25b1cfd6be683febabed9bcc00
1824 F20101206_AABNWK levinbook_y_Page_079.txt
db58a86c9ff4df80a5271abe8c4cd8b8
e460dc03cfb009c171122e5443638dda1b44d47f
F20101206_AABNVW levinbook_y_Page_009.tif
4c3ef49077ca7d4ce8d324a15af2f13e
60670 F20101206_AABOCF levinbook_y_Page_056.pro
F20101206_AABOBR levinbook_y_Page_090.tif
8d3ddbd93cde3ab06e715ff3532c4b6e
106810 F20101206_AABNWL levinbook_y_Page_090.jpg
991cfe00d53f4aa5f174e4386ab5fb31
4991a786de2abce785f020d427f01dc6cb3de4fb
74317 F20101206_AABNVX levinbook_y_Page_051.jpg
b57a79c022f379744da4b0fd03ea43aee1b92a3a
55391 F20101206_AABOCG levinbook_y_Page_073.pro
3eee1e5c4f3093c8a73d75a04684f871
55be8e73ec147f78b5c1abe6d29cbddb97e74b13
1006023 F20101206_AABNXA levinbook_y_Page_081.jp2
cce4d713eb6b22329fc6e97446bc9a3a
f0f9b5704005fa1ab2d6ab94a19fd513f846cc23
F20101206_AABOBS levinbook_y_Page_094.tif
f9a66a8dec2bcea9a3ac1fbbb4fb80d1
F20101206_AABNWM levinbook_y_Page_054.tif
89a282f094f43ace440f18d50a481514
d2ff51608ea7c0e88a53d5558f49502a2001b618
95497 F20101206_AABNVY levinbook_y_Page_043.jpg
42d512720049fefef600b13602c611b9
bcf249c9c704c295a013df4fea011aca085fb157
60234 F20101206_AABOCH levinbook_y_Page_074.pro
2d5d893c22fa959897c804a34cbfbfe8
e17373658a710686b79cbed2a95211e75a474302
F20101206_AABNXB levinbook_y_Page_005.jp2
e12c5345dd1642deb90c76299e0f1409
0acb6d187ae0169bbb52532810e5cd4a61958c52
F20101206_AABOBT levinbook_y_Page_096.tif
07ce2f8890957d54673c74922e7c6844
7e8bbdcc9f70b37a74cffb8323148a15c00bb314
8806 F20101206_AABNWN levinbook_y_Page_038thm.jpg
6031dbbf2c367a222452be1c7156942a
d627fc76fec8a32435a534a65beaa87cb767e185
F20101206_AABNVZ levinbook_y_Page_095.tif
c65b3d67833dd6d6d02750a7182e19a23cf799e4
36514 F20101206_AABOCI levinbook_y_Page_091.pro
d456f9bea9b209dab6a0d62cda7c2480
57564 F20101206_AABNXC levinbook_y_Page_009.pro
1394e89b831edbfddacf163247611f39
88e0a17b62e0145a6b612b3cdfc64f9085ac800d
F20101206_AABOBU levinbook_y_Page_097.tif
d3284527b6d29aa8bd03c907b2ee1026
efa244432197e34b8785a9aa4af38baab8cee094
34568 F20101206_AABNWO levinbook_y_Page_031.QC.jpg
be5b786b6e2a6935b06e9ffe462b2dd8
40702 F20101206_AABOCJ levinbook_y_Page_097.pro
b56bbb580519a6b51380a33cdd99664507f2e3f1
F20101206_AABNXD levinbook_y_Page_012.jp2
53a1a3f85d37e26efd8287a86381cb40
46878abfd3886da0611c834a01b776490f1f1554
F20101206_AABOBV levinbook_y_Page_102.tif
d1af43f1fa45331301454e64b3e52f0c78bfdd0a
F20101206_AABNWP levinbook_y_Page_026.jp2
cc5186c7c03a26cbb68e8ef350ce72fd
2125f27077ab9e5059e4f7292c5ee53cda935623
59061 F20101206_AABOCK levinbook_y_Page_101.pro
37d382be3c2a8afc9d070db44c06a15f
7230 F20101206_AABNXE levinbook_y_Page_100thm.jpg
8c335f594ee4a049213d93a9c3e82d23
F20101206_AABOBW levinbook_y_Page_111.tif
a78c26c9d710720a6c4876f1ecfdf6b3
a47fbaf9906cb28f4ba2bec8c6f94929dc33a296
40621 F20101206_AABNWQ levinbook_y_Page_093.QC.jpg
f253a9171c939e07acfc7c0a8d9772f51efe2bd6
51483 F20101206_AABOCL levinbook_y_Page_106.pro
1d65c8fa82e21ba6a10b304e7e253641
1051971 F20101206_AABNXF levinbook_y_Page_036.jp2
a074d91bff8e6389a3d9206fb9d726fe
ffa0ced144ce67123c03d1978cc09309d07b74cb
F20101206_AABOBX levinbook_y_Page_112.tif
17317758d947ed9fdb348e4724d5fc47
73765dd1e897b1b26cc9da2f27edbefd62e0b0d2
F20101206_AABNWR levinbook_y_Page_104.tif
dc1bc3487ac695c0ed027d3fb358a61a
0d4180b18365c976e6e5413418200c2a556676eb
1751 F20101206_AABODA levinbook_y_Page_091.txt
4e1ee873e44d08859c5055acc9ce6c7f
6fcb33c4f93e543648eba5fa909d6b4cb0e6345d
61262 F20101206_AABOCM levinbook_y_Page_107.pro
5de0245e5e23a2c73390062eae2e3b7e
4c33f64d5542d5177685774f93d5c0598a5d1169
57055 F20101206_AABNXG levinbook_y_Page_036.pro
2008d9bdd710acfb67977861072c6913
b6660df4f87447fd89f60354393656e0cccc7910
F20101206_AABOBY levinbook_y_Page_114.tif
2c3bd6006bdecedbaaa3841a9f176ef824d2e1e6
90033 F20101206_AABNWS levinbook_y_Page_085.jpg
3e6dd64151fc1ace3b0322624a334f7a
bc5a6cb60f520ed3cbf56ca988390598827cd435
419 F20101206_AABOCN levinbook_y_Page_001.txt
32ca270199358f671d18ffb5cf8d591c2c4d0133
34820 F20101206_AABNXH levinbook_y_Page_040.QC.jpg
d8e371eba79b83bd25086f977d2ab302
70276769dcbe599352d17d704cf5a9ed1b0fb17c
44528 F20101206_AABOBZ levinbook_y_Page_005.pro
c072fbca04dabd151e0d0ab5fd49af05
f7328b2f45205d5d26d3cdd7963e1bbe0897d978
F20101206_AABNWT levinbook_y_Page_052.tif
885324f0c604e6dc49858e70d4dc9c58
5ec5a270653fe9622b603d91c8dd2b2d4149469f
407 F20101206_AABODB levinbook_y_Page_095.txt
6506c4c593872192be5b96c210bf45519ae9cbfc
1308 F20101206_AABOCO levinbook_y_Page_004.txt
80d58f92977363a928e93eb29fd29750
904ed66af8a3db8b0e6ec5f04588390fc950a2d9
33675 F20101206_AABNXI levinbook_y_Page_082.QC.jpg
31fa877db546dfed321724e2441071e4
93 F20101206_AABNWU levinbook_y_Page_002.txt
5b1b1d588e03d2aa20e2d2ce57baba2d
0b95431a0a602934bca60f9a28b5905739e977bd
12297 F20101206_AABNAA levinbook_y_Page_104.pro
476bef86e8ca4bd7265566c1934db3b9
90fde4a4154778cb66b752e9d2884f134ccf2857
581 F20101206_AABODC levinbook_y_Page_105.txt
761d37e422cf06f94ff6a30663b9cbfd
5e15ae2d19b33214e30de5891477e05ba34801eb
1926 F20101206_AABOCP levinbook_y_Page_005.txt
52c9881f10bf3a9c3c6dcd481919a672
5023be1c7fde62dceb7391a5bea025131e44453f
23767 F20101206_AABNXJ levinbook_y_Page_051.QC.jpg
d352657ea8db7e58e198738698087e22
47017 F20101206_AABNAB levinbook_y_Page_087.pro
7436f8ef3647950c75ffd8640d9b69ce
07cffb79f13f23e8b5c0dfd7a64c97298c0b081f
F20101206_AABODD levinbook_y_Page_006thm.jpg
bccf4a430f44b8daf4934dd11dcabfd8
f93c1ce72737a54fcd72b949ed129251bdc4b3da
2532 F20101206_AABOCQ levinbook_y_Page_013.txt
11e2565783623a8bdc7eac0cb18a4ba142f616dd
F20101206_AABNXK levinbook_y_Page_110.tif
d69d602b7314fe3f4be409c3d0482dfd102ae11e
2367 F20101206_AABNWV levinbook_y_Page_074.txt
0d9af12cea9e70cea2c496d24e5f6bdc
c571289e096732bb3be42c351c4981d7633dcc04
118137 F20101206_AABNAC levinbook_y_Page_025.jpg
6cbe7783173893f34b40328fe925a10e9842b7c6
5711 F20101206_AABODE levinbook_y_Page_010.QC.jpg
d3e4e7bee56d5c26838e6d73c79d5506
ca3a3c5921e3da117098723d0b46af4c9732278b
2390 F20101206_AABOCR levinbook_y_Page_019.txt
dc92ab63d7eef5bc059c52e9c2057109
F20101206_AABNXL levinbook_y_Page_048.tif
80399f36d31cd331c9b8b847a1a3125d
d797f3b47604e3e193bfbee8768d16457a763881
128381 F20101206_AABNWW levinbook_y_Page_013.jp2
d6fe89a4aa09e31bbfac87c2976257b5
1bcd22610ebc0040b1b8379efd16f01cac93cf86
b36423ba0fbfb447b08bd802f6ddb710
62fcbfc93d542c4bd0834c6ec0c34311e2913b15
38367 F20101206_AABODF levinbook_y_Page_011.QC.jpg
18bf3a2a9f643970c6c70940bda6a130
c59ac233e141a95d9e4c99ea7f3bb17e1bd3730c
F20101206_AABNYA levinbook_y_Page_023.tif
36a4e8b70c73995742d00aa60179e6b3
8cbedabcc4c4cbee4bd045b5c4610272eddd8b2e
2116 F20101206_AABOCS levinbook_y_Page_039.txt
23e364e6797c4a2c38a5200a0e98829d
2195 F20101206_AABNXM levinbook_y_Page_095thm.jpg
0d75aa4b17cef04f59236e89b36747db
56526 F20101206_AABNWX levinbook_y_Page_058.pro
18b9fbd8bfca88e404a6a7346e8e44cf
9b5685a006173125a8def44781992aa62595ff3d
9092 F20101206_AABNAE levinbook_y_Page_056thm.jpg
5577c4605f80bd0844d1f6c5373d7687
9188aac1aecd5b9d42511152da01afee3f854522
40713 F20101206_AABODG levinbook_y_Page_012.QC.jpg
d05bfab1d0fd457238133874ededbc85
f667f1e9ca3c79fb85b543abf93eaf8746df1c58
F20101206_AABNYB levinbook_y_Page_063.tif
ccb34c6ddd4204bb75cac803092efa24
844cd248dd9ec68a0bf9b2120fba02830bc9d06c
2354 F20101206_AABOCT levinbook_y_Page_049.txt
4d764206c6f67d4892aeca350e94df58
c36b5c422654b07bd54decdf4e50a4327e9eef14
9268 F20101206_AABNXN levinbook_y_Page_054thm.jpg
06cce191a9cedb2b97cbe2f7a4f45c75
73472dc0ec351f1b3116a40507663318ac94d9a5
34781 F20101206_AABNWY levinbook_y_Page_069.QC.jpg
d5829c373589ab19ae15f850ff364dfc
824d0ea8aed889a7401211cbb42ecee28de05733
38220 F20101206_AABNAF levinbook_y_Page_071.QC.jpg
92382584e6fa39576b6621508668fc77
704d1d7edc5ac2f5c73fe14a098e313b35c8934a
9345 F20101206_AABODH levinbook_y_Page_014thm.jpg
fae9f36b754700c36a4c683d74514a7d
158896efd9fcb0225faf312b75530cdf3c7ab5b1
F20101206_AABNYC levinbook_y_Page_067.txt
42de8e67e67e7c50c6966995beec026f
260dbe5022b6f4c2cf3d13a678c532910c17875f
2528 F20101206_AABOCU levinbook_y_Page_054.txt
5b0502463f4d6b6a014ea7e5940a55aa
d567824d3a4fc6bf268d18082cd83e1db9a9936a
F20101206_AABNXO levinbook_y_Page_038.tif
3015d8862c4840a950f12534ff88b0dc7f64ecbb
F20101206_AABNWZ levinbook_y_Page_031.tif
4389220279da38219bb5895993454d57
fa200dd2521fd38c42ff19f5dbe7ea29dd78cda2
1051986 F20101206_AABNAG levinbook_y_Page_074.jp2
746d312bda129a64ee2a50ffa9bcb0cd
49289e5b0a62c16f374307765489b67af074c91b
11608 F20101206_AABODI levinbook_y_Page_016.QC.jpg
cc8c28c1de615ba9352dfb0fd38fcd82
82f7affcc0dd4fa34b922e728353c8b15443ef13
20779 F20101206_AABNYD levinbook_y_Page_047.QC.jpg
f562baf1ed86c394af45a422cf8f507f
4ed1177607259d1c7dc3701bb33c5741b512148a
1700 F20101206_AABOCV levinbook_y_Page_060.txt
7e5f8e6fd42482a996467e25340e10a5
36fce2b458416a8ddebfa82ae03c595c67216df8
31312 F20101206_AABNXP levinbook_y_Page_043.QC.jpg
97545 F20101206_AABNAH levinbook_y_Page_087.jpg
26470 F20101206_AABODJ levinbook_y_Page_022.QC.jpg
e6c8c16bb603285818cbeec2004ebed2
44b021be882d8ffea9c11a200352a9e51e73a931
1016442 F20101206_AABNYE levinbook_y_Page_048.jp2
e56e9e47b4d3cf49bff952921ed45383
2238 F20101206_AABOCW levinbook_y_Page_065.txt
50472b4a394d6ef3385f43806b733426025cbe3d
7245 F20101206_AABNXQ levinbook_y_Page_077thm.jpg
c33c567e1edfdb609e8c7e1a19bcaf80
9006 F20101206_AABNAI levinbook_y_Page_042thm.jpg
986999a4d024f727c9209e704136266b
217f8898341a9045b12470df895548d45c3d6749
7610 F20101206_AABODK levinbook_y_Page_023.QC.jpg
43a30405401f4fd2b7d7880bef675a1da9942788
8958 F20101206_AABOEA levinbook_y_Page_086thm.jpg
365e7a54f639455d9df333ab1d35d0cb
aa292fff1433165095e0c2ff4a4a89fceb72f077
41042 F20101206_AABNYF levinbook_y_Page_014.QC.jpg
2642 F20101206_AABOCX levinbook_y_Page_070.txt
5920a1ff126fba6b01f32719e865fb91
c3f5b945b817292d0882831f567c76340a10e8b6
9658 F20101206_AABNXR levinbook_y_Page_072.jpg
d3ac1195ed015b8f3d600ec053e38a20
0859b4f9b5618c4ede121c6fa390a7fa3bd8df4c
121844 F20101206_AABNAJ levinbook_y_Page_011.jpg
1e28c2782a04d1ffd9342f53144f4e17
f94678362100728e647e947d12a3c002931ff00d
38230 F20101206_AABODL levinbook_y_Page_025.QC.jpg
76ab67ab9aab7686f71a9d81b9322e03
995825e66eaa5bb3e2c95269c952821e3876fa1d
31548 F20101206_AABOEB levinbook_y_Page_087.QC.jpg
7448532ec4da92ebce2bf2ef6414c04e
54cda0249d3f6790c0880f03a11514a46d095ed0
F20101206_AABNYG levinbook_y_Page_007.tif
253214af3bd0e7b9f050545fec4996a82fca60da
1931 F20101206_AABOCY levinbook_y_Page_077.txt
79b16f09315f273b2375892ffc217b05
e4da842d55f2a90bb5d44f2c3c0d8a9c7a6f1527
135118 F20101206_AABNXS levinbook_y_Page_044.jpg
b4830ba3ffcaa04faeb79886412b5642
26f15a20f05b712424032309d8d75406ac000f8c
1051924 F20101206_AABNAK levinbook_y_Page_021.jp2
4ac92eaa7d223ce6a07a2af329db5f33
8522 F20101206_AABODM levinbook_y_Page_027thm.jpg
3ba74170c68a566a1a87f5b3a8a3c49aae9a6201
7852 F20101206_AABNYH levinbook_y_Page_059thm.jpg
8f65d9abab729d65e3c542a5d808e0f3
7fe6fde7b2e8b1384725a6fd557effd074d5ce61
1890 F20101206_AABOCZ levinbook_y_Page_087.txt
a7282281f388aea6a986c848452d5400
2d20b9f3e71504a765d4f0cb896fc13df561bb53
64594 F20101206_AABNXT levinbook_y_Page_111.pro
8260b307e0991e3233d66d9374632492
41130d7e52d1548b1e4a9671b61af2df83cfc1e0
F20101206_AABNAL levinbook_y_Page_072.tif
688481fc1c6d0b20fa55329ee3a7e377
10e334d85479025bce83eb6245b4ba3e664c9c51
34988 F20101206_AABODN levinbook_y_Page_033.QC.jpg
db19c1f7316b6a3079e2245382dcb802
0c5e6b0725243abc55d175f1eb4db0b78bbe430b
8266 F20101206_AABOEC levinbook_y_Page_090thm.jpg
f72d90ba5f94748914387a5276d8e248
37ccde3d310d2035ab8dbcf388513393634c5d33
38361 F20101206_AABNYI levinbook_y_Page_013.QC.jpg
d1979a0915eba865b4c35131b500da1a
0bb10a3127e446e461b00e6e5c0c52b089a1915d
8041 F20101206_AABNXU levinbook_y_Page_006.QC.jpg
0f3873bfd2410c61f18f7c0080cb693d
102115 F20101206_AABNBA levinbook_y_Page_052.jpg
517ac4ae21c2a0aac0e9268a2fa18fc4
33b7a174059df41bf140f672119cfd48b8f247f6
52833 F20101206_AABNAM levinbook_y_Page_034.pro
194408bba6953830cdbf5e4e2a03a7db
f3560a875b5fba9fe3ba3dfe9a746563f8b908a7
34945 F20101206_AABODO levinbook_y_Page_034.QC.jpg
b1c90ea90c2266211679333be05d8bc5
7792eb5b03236190002e41d1214978619cc8af5c
6266 F20101206_AABOED levinbook_y_Page_091thm.jpg
93eb7629c4e1881861361e410c50c84a
5176f64120471f6cac9a3f5f8228804e9ff5bccb
1880 F20101206_AABNYJ levinbook_y_Page_028.txt
38a07ed5cc63625fbf1b5d1ea0fb3b35
1483 F20101206_AABNXV levinbook_y_Page_003.pro
af033fbf0cbeacc2f80360bc19ed910e
49364d537819607644972b1318327d90c8ff71f1
106300 F20101206_AABNBB levinbook_y_Page_033.jpg
e4f7a481a7ccf1fd682b2318720d53db
F20101206_AABNAN levinbook_y_Page_028.tif
ffe89a035b97d42ca5d45b775ff2ea52
fcbaa904c3933cb79b26097a43e916947c0d6d86
9613 F20101206_AABODP levinbook_y_Page_044thm.jpg
027602c66b1b3f6a4377662874a7eca5
c154baef5d92086ac51b6e05bce65d98a1701598
6111 F20101206_AABOEE levinbook_y_Page_094thm.jpg
a73f158911483dfc3380525219aef867
7b3bec8dbaf8fc03bebfb1b47ce9163a333e43b2
84546 F20101206_AABNYK levinbook_y_Page_100.jpg
4844d8666008190d7fd1c9062f81f1dc6a7e0539
F20101206_AABNBC levinbook_y_Page_102.jp2
e996a1239f88a6fac1bda31a8a4100f0
d1cb2be9aac9f229a0d046d4ce08a737b274ae6d
36904 F20101206_AABNAO levinbook_y_Page_051.pro
48017a8ee86cf9958af421c87bc79333
7c7238ea53b832307fc051cdd333bbfe78195629
37008 F20101206_AABODQ levinbook_y_Page_045.QC.jpg
1b45bb271c146de17dcf746a2c0f34b4
9ab4e48539846c2b28ee46502ce136db102f2209
9273 F20101206_AABOEF levinbook_y_Page_099thm.jpg
ca20e67feaceb1e593c7f43effb4980c6b200b76
2173 F20101206_AABNYL levinbook_y_Page_090.txt
cd6d70dfa574e3678f77db16bce3ca7f
7a4ee4b6c8b636ca99ce69300f7f7e02df846f80
32561 F20101206_AABNXW levinbook_y_Page_030.pro
999dc7524de1aab640e0847774c6d300
F20101206_AABNBD levinbook_y_Page_005.tif
bdabccab98d9357a74970d0743f08d80
64062 F20101206_AABNAP levinbook_y_Page_054.pro
8cdcc23cddb2a3ac5fd3a8a04d1f1924
8098 F20101206_AABODR levinbook_y_Page_052thm.jpg
34e737ccc6c9886d08533b2db1c5065b
5dec71a8d153ac1fb4ceaecc22a0a03104aa40fd
9486 F20101206_AABOEG levinbook_y_Page_102thm.jpg
09c7671e6781500328f32eabf09e017f
76108f314c8fe4b73320a930bdb3d858b5675c58
F20101206_AABNYM levinbook_y_Page_042.tif
a93e0ff4a52a475b25e657f21f6b9fba
5cd1a367cce6fd425c47479051f37ab74ef9d15f
40893 F20101206_AABNXX levinbook_y_Page_029.pro
bf40048c4cd56d46cd6367f5c4a4d9ff
79264a4492bf6cd79275edda78fb6f0bf11efb3a
F20101206_AABNBE levinbook_y_Page_089.tif
c8ef24cb4e40f8fe6c4454742cc694cf
14580eb2619d0ddb7dd93fecae68a968a9df91f7
118113 F20101206_AABNAQ levinbook_y_Page_066.jpg
0f4118ba9c8056623555990772ddfdc6
21421016bacc19765b65b13a949c4dc7f080ba47
F20101206_AABNZA levinbook_y_Page_027.tif
f4b19b1ab8f8d85189ac121e5ba8090d
3b2e0007f655d7c2f85f883639c6d1a3eebb1f75
31836 F20101206_AABODS levinbook_y_Page_052.QC.jpg
24493035c56f3e3a7abdf06e41a8783d
051287fbd354bedd8afa05ac4d7f22460e4d52b1
31821 F20101206_AABOEH levinbook_y_Page_106.QC.jpg
d00025dca496eef07faba84c91c35c03
1041348 F20101206_AABNYN levinbook_y_Page_043.jp2
6cfb33243c6b7f69fbc275dc68db2f73
34fe486d06e074ef2319a7a56c52c3d378c3ca0e
1051204 F20101206_AABNXY levinbook_y_Page_087.jp2
8199c9922155a08440764eb3d5f8f04b
e0e35389d34bf958887de30c96069a7d91e671a4
7739 F20101206_AABNBF levinbook_y_Page_063thm.jpg
f3a3c5de5d52dc156c76dc88d38fa945
2df7f433526db517093502fd114e60a7d710face
34680 F20101206_AABNAR levinbook_y_Page_039.QC.jpg
7e7ac4e178b939526c5293b12015c5e1
9199dc76bb2276941fd26b8db1ccb1407ce0e582
30364 F20101206_AABNZB levinbook_y_Page_048.QC.jpg
e50d0b10c3f66a26d4cce7a0edb5f500
99d76831af03dce192169cce19b952e6163fbb05
30566 F20101206_AABODT levinbook_y_Page_059.QC.jpg
bc5aabff504df2ac6dce9dfc50dcd37a
24e0565f9384efcb5663bbf11c67e5583dcdd94a
3685 F20101206_AABOEI levinbook_y_Page_110thm.jpg
64e8b33b01d9722a31d5508bd9d9b93e
3857fed51dedaa8e95a1f1e773ec9ae3e3eefe6f
1955 F20101206_AABNYO levinbook_y_Page_081.txt
38457dcf89523553350fae87dbccb2dd
9e393d5aea0330ef1c23babf90dcd3ae26d255c2
52787 F20101206_AABNXZ levinbook_y_Page_040.pro
fc5b3695bd9d529b60e99af6e003a408ef18eee9
9108 F20101206_AABNBG levinbook_y_Page_049thm.jpg
8b20b26dc1ee713d9b6be467f44963c3
bd0d38bbd1004672c7cee6ee5b088ae36076d88f
12702 F20101206_AABNAS levinbook_y_Page_110.QC.jpg
de095f499cd33ee0ed39c0f3692b0001
f6f2df4376767b8a09a3fc43fcb630de2366a868
2570 F20101206_AABNZC levinbook_y_Page_012.txt
1e0e94542fa635fba443bf0a3f32d71a4ff60dfe
8994 F20101206_AABODU levinbook_y_Page_064thm.jpg
1342ab0f32dfdfd165c571f452e017e9
8915 F20101206_AABOEJ levinbook_y_Page_111thm.jpg
55d6e85bc946f62d4ff8b48459eb0289
6d30cb0a6a22b1c58b0c24b2fcd4b921358d1eca
F20101206_AABNYP levinbook_y_Page_006.tif
f0f5121c7e770d09528a3f408a123b0f
298b2424c36a7c32de82c14992ca98bccac9ca4c
8883 F20101206_AABNBH levinbook_y_Page_019thm.jpg
9f7dc8e4f5ea775c2d69aa518f8085f8
cd5dface784a9bbdd46110e43d155908919c9cc4
7371 F20101206_AABNAT levinbook_y_Page_095.QC.jpg
073c4312d4c4ecb5d2b2bf528a5eb76c
1616c4ed9fdd91a753716f1490193dd3fc70b726
F20101206_AABNZD levinbook_y_Page_070.tif
f2c8658375a24dca394b9abbe9fcdede
cd137cf7f19eec58f26d5835cf2a0df11b9a0dd0
35593 F20101206_AABODV levinbook_y_Page_065.QC.jpg
2f6c41aaacaa3c5b328ed68e8be098b3
2eff7c956ae9d05340639c792ef00bde39334dfb
37740 F20101206_AABOEK levinbook_y_Page_112.QC.jpg
c97f740b5e7f4600105c45b3cd62844d
d0d682b2a643d1a8035ba4a705ec2e967b9a5d5b
8659 F20101206_AABODW levinbook_y_Page_071thm.jpg
827659ecb542cdd9fe243c944d1f693e
9241 F20101206_AABNYQ levinbook_y_Page_025thm.jpg
03ce8f648bd934c75cb54eb7e56e744c
25401 F20101206_AABNBI levinbook_y_Page_053.QC.jpg
de31e82d3d0bc15c2e4a6e55fa9d028f45efc7c5
2452 F20101206_AABNAU levinbook_y_Page_083.txt
242f8b0aee4c6cbd69864912770d95f6
0b20b8329b762c35fbeef6e4f0f4b1dfc90d11e0
7092 F20101206_AABNZE levinbook_y_Page_053thm.jpg
58782c14ec0d14db23d66161a83bdd46
6da1a411dab97e327a4ce30a919f299802d1d139
11423 F20101206_AABOEL levinbook_y_Page_114.QC.jpg
f48cf18426b5a1f1d6684b5e29b00624
77909aeef8e145eabfa229c9b238eb87791a74e1
35192 F20101206_AABODX levinbook_y_Page_073.QC.jpg
6e2034f510b55fbdc30f6b3c1659f050
5830a537d1abd854b6a4a45c543c8e0c8781480f
118714 F20101206_AABNYR levinbook_y_Page_064.jpg
d6a15118e4bb64eda8d77995e34191b5
23bb4bbb8fd3e047da4b4664c291588aeedfaecf
42224 F20101206_AABNBJ levinbook_y_Page_102.QC.jpg
0e5d54813119271a131a23d75bf0b169
096683b49346dc30c5e8ef43e9b0c5032219a1bb
F20101206_AABNAV levinbook_y_Page_066.tif
913a32113fddbc307cbd97bd810d0758
f8f5dbf0e0cc6548cc19b9999e89e1a83da7c597
57056 F20101206_AABNZF levinbook_y_Page_033.pro
d575cb6a94a374915594746710a43f3f
cf825958fed1428fb4829638d9de5faa9def7ef7
134122 F20101206_AABOEM UFE0021259_00001.mets FULL
d9bcf7c06fe87801739aa58db6c4875c
4809d4b3c03487d50430f867b8c3118f9ffca6e1
3920 F20101206_AABODY levinbook_y_Page_076.QC.jpg
4b5721f41c7a866e4bd5a78ae845209b
fe64cbcab9ff25fd3581df89359c2f8437e7327a
122032 F20101206_AABNYS levinbook_y_Page_071.jpg
edcd15af1daf66b3aed592a5830a41c4f2fdedc9
1966 F20101206_AABNBK levinbook_y_Page_050.txt
238c573b0076c25cbc7eb3e68846c1a9
361cd8c46b28a29ca6a51270f2004cf94c144dfb
1559 F20101206_AABNAW levinbook_y_Page_030.txt
a1bf7f4cb8883e6e629a44e3e03fc720
34fb37eac26f64258d71efee197d1e7a15a51da2
1879 F20101206_AABNZG levinbook_y_Page_023thm.jpg
e392a1eee540377094a12881085c98c0
d333e3b15f220211431ae85b3af3b6969da6ffdd
7842 F20101206_AABODZ levinbook_y_Page_079thm.jpg
6f9fffd762a1bbe05ef9fae8436ec436
659926119893b468f821a08bf13201538937e48c
F20101206_AABNYT levinbook_y_Page_098.tif
affb0eec1991ddfa59509e40e0f26e07
d9929001f28c1a7c7128186195973a954ca8c5f2
659 F20101206_AABNCA levinbook_y_Page_114.txt
4b5669b7915f740ec35642733134bf78
803671 F20101206_AABNBL levinbook_y_Page_061.jp2
2a8c9bc7c66d974122550b4edc24e9ee
3e77409d37b40e8f4e5603c0a174ba4bd8408152
6033 F20101206_AABNAX levinbook_y_Page_030thm.jpg
dc8be7909cf8703dede50c4cbf17e204
7a2c7ae9d7dab6c3d8f54f416a01499c8c1bb4fd
91438 F20101206_AABNZH levinbook_y_Page_063.jpg
1a9405fcba1b94ed1f0515a8199077dd
99438d94eee24b3edc1c9de69720192c2897e4a9
2366 F20101206_AABNYU levinbook_y_Page_101.txt
1effd345fc2709855ffff993ff2dc9e8
1d2f097db554ff8e32c73fc85e70f05411b02ca7
57058 F20101206_AABNBM levinbook_y_Page_045.pro
723016155065b4b0b0ede4ce917b7a3a
e1ae174eaf585fd2433b1c287daa40c093706914
34841 F20101206_AABNAY levinbook_y_Page_009.QC.jpg
b6a43a60dc28402c8569cea331155e79
9e980afe742c9fbb41029aba001cc5db7357a8fc
98851 F20101206_AABNZI levinbook_y_Page_078.jpg
aab0f63137a6808228ff44c7e4a2aec7
c37f38ce7563908b75cc986e91d257a894827f58
57905 F20101206_AABNYV levinbook_y_Page_018.jp2
7a160cedfacb4f4a7c66fda69ffec728
fe624942cf8b4e0c9c29ceb2a3e6570d1c217139
2301 F20101206_AABNCB levinbook_y_Page_073.txt
0d84c9569c207ed4363cbb9e4aa13bd7
151829d458ef2fd93744805835104412fb11c35f
2373 F20101206_AABNBN levinbook_y_Page_086.txt
78710f7cd142a71e2b88cfb3c55ca31c
e35db7f2fe8e5670776f172928b04b3c3889a4e2
35300 F20101206_AABNAZ levinbook_y_Page_027.QC.jpg
fa1d27ef009f98196730f7c7131cc7e1
ebfccc3fb461fd0f73cbc1d8029738975670eb86
68410 F20101206_AABNZJ levinbook_y_Page_044.pro
c2ff9b6f2f91a9b346a876bbee04edf9
9f9aeb74b22805d85a6f99817afc7a7d4e4987e8
27132 F20101206_AABNYW levinbook_y_Page_080.QC.jpg
e7cb9eb17a58c993336eaf6fb49da343
c0aa954fc7130b93e2b0aa66b7eec278407edf1e
9059 F20101206_AABNCC levinbook_y_Page_070thm.jpg
254b44f14974148742e216c7f261ba71
a9858ec5e5eebc0afda6049712551f27d8e23fa9
F20101206_AABNBO levinbook_y_Page_079.tif
12a8293eb2bc9da76bd793584425e4de
d2caa62d10a173970ac5b085507d315709b374cc
21754 F20101206_AABNZK levinbook_y_Page_030.QC.jpg
d9515ca63858bff8e35d62a29864c029
60074ffe1fa40026e6e7fcc7d389b644deac291e
66699 F20101206_AABNCD levinbook_y_Page_093.pro
87de505a8fa62bbb49d5655e9e0d754c6797653d
F20101206_AABNBP levinbook_y_Page_050.tif
f0ddd185bcfa50c9d88db2a3a1912c4bcb24f7c9
928254 F20101206_AABNZL levinbook_y_Page_085.jp2
967cfafb2fa57994a5469f6262bd58f2
6fcac06c110097fc0e7fc8c4e60fb129b0e6063a
705330 F20101206_AABNCE levinbook_y_Page_030.jp2
147363a06bd820a2117e08dcf489ca0d
956612b7619364fc1a976ec315df6063382f6e28
87945 F20101206_AABNBQ levinbook_y_Page_077.jpg
64b96d227fe4cbff4aab1ff26f4ae275a94be749
30517 F20101206_AABNZM levinbook_y_Page_088.QC.jpg
dece5b68710bbde6d45002db9c94eb8d
25116cece9bb2744bf82f283bf75dba38e2d1bc6
31618 F20101206_AABNYX levinbook_y_Page_004.pro
3c2fa0bb43a8dc7ab83289bec5ff4e3c
36af7ea393218303040072621ed20db1fb67da2a
F20101206_AABNCF levinbook_y_Page_101.jp2
105d4ca8b9789d40c13f2c84c0b21f0f
b8f8606aba13aff4a71969d3c02a461160d7d2e5
1678 F20101206_AABNBR levinbook_y_Page_097.txt
9905f8d2c48f77ed942f46d107877ba154676f42
60342 F20101206_AABNZN levinbook_y_Page_086.pro
4cef88b77c62096e8fb18224479fa292
4d48776a1b386e829876e4a32ea565e201544cca
1963 F20101206_AABNYY levinbook_y_Page_021.txt
276d66f05b3a1eb737d9c7db4df5f7ec
fefd0b13bddd6596649476ffb90f5961c0a6c859
52669 F20101206_AABNCG levinbook_y_Page_068.pro
1a7335f3d2d37dbed4217c6b9a38d4cd
2690d8b593d45c37fee88619405020ac3b01ea68
36350 F20101206_AABNBS levinbook_y_Page_036.QC.jpg
585e04315b3a57a426d5712905bb54fa
173741 F20101206_AABNZO UFE0021259_00001.xml
c2acb5df08c25b1c468acac109232224
d3afc6b8b56c29e554d54625b3157062616523be
2555 F20101206_AABNYZ levinbook_y_Page_099.txt
cf5efdc651dbf2f90c271c739f2300c0
5dfdca54f3bf5c6d349cfc41c07ed91f8bd9563f
8909 F20101206_AABNCH levinbook_y_Page_066thm.jpg
a95668f3ccf439515fcf26c6510017e3
deec8344ca7b74d2964982ae736a5f189cb60b8d
F20101206_AABNBT levinbook_y_Page_052.jp2
43d927f7106210f29ac3b9d37aa389e2
847a4f9a09fdfae6d74b45b1cf156326a2171e84
F20101206_AABNCI levinbook_y_Page_105.tif
03c395a3ececca7162c33b0608d18b13
4440375ac4cafcae0ae5b5943433534e024e71e4
91027 F20101206_AABNBU levinbook_y_Page_079.jpg
fdfb986ea1a162437f86619fc5388b29
ec9b33f56f33209b419eeec352d4533478f1085c
28743 F20101206_AABNBV levinbook_y_Page_046.QC.jpg
b94844d1b4a074632e60b2bafcbc8190
408d2ecfcbd7350e9659a0f928e64f699fe6d477
5337 F20101206_AABNZR levinbook_y_Page_003.jpg
9de8901c958ae3aaae601db27bc0db51
80892cf5180d008a8515d94164f653f31983a107
2641 F20101206_AABNCJ levinbook_y_Page_075.txt
96248a1c4262475caaca95ec8f3236c4
5ce029834f69c2200c573066ccd1600e4a6d81af
41474 F20101206_AABNBW levinbook_y_Page_113.pro
6eeda2f65d3c2bbe7f2c7332cec409c0
64320 F20101206_AABNZS levinbook_y_Page_004.jpg
1051982 F20101206_AABNCK levinbook_y_Page_068.jp2
6e542e7ff0b1c154efbd088201b5bd8a
6e392b1fd75ce50998c33133e25bf47f398fa9bc
2391 F20101206_AABNBX levinbook_y_Page_064.txt
cbc2ff4977c864eb2a209739943eb93e
32cce4c0acd7cf852c76efd63e9ebb440f901993
53747 F20101206_AABNZT levinbook_y_Page_007.jpg
8bc034544b8499b76e611a8ff5f6350e
b66c2d643f4cee35217276dda978feccb47ac526
6693 F20101206_AABNDA levinbook_y_Page_113thm.jpg
de1f36eba4fd33e30c0aa323dc54a532
297 F20101206_AABNCL levinbook_y_Page_006.txt
5d5cbd0024c6b114fc3ef27ef19bcfe9
79eff079e0f9ec9448b36e31265431089186c4a6
8763 F20101206_AABNBY levinbook_y_Page_031thm.jpg
294318d77401724d85bfa2d86b29b697
dac89200cf6e42c6652930e872c89ab95ae004df
58152 F20101206_AABNZU levinbook_y_Page_018.jpg
16e8d9e1b7b033deba006f9a8844a06f
64423eff772961db813b6cd333631044ec427f04
2202 F20101206_AABNDB levinbook_y_Page_034.txt
abd87436fd4b63e83fc147e3411ab4fa
2679d8dbf55e5bbde50a6838dc62def5e301695a
F20101206_AABNCM levinbook_y_Page_051.tif
03ccc049a749bd0e8a3121442f979fff
3512325eb305d5ae537c0da3081efc6c2a70571c
1051892 F20101206_AABNBZ levinbook_y_Page_054.jp2
1d583c0486a831f07b98c02cd5b7d7db
6fb2c794e25bec7ca1cf04b1fce08c075bb0be2c
21682 F20101206_AABNZV levinbook_y_Page_023.jpg
3a4c402a5477ae31162ab82c25414358
dcac0de1e8e491e6d4abcb1cc863d4658b0b4191
F20101206_AABNCN levinbook_y_Page_044.tif
25c00f4832cd881cac03a2ac0a231247
880d1bea4780bb93b1a51468051d9d3437f461e4
65977 F20101206_AABNZW levinbook_y_Page_030.jpg
dc46a3824e75cba5a1a1205e66daa94d
4d5498a0427a1b7db1a8e123a129e2221c9d0010
67248 F20101206_AABNDC levinbook_y_Page_026.pro
b78f0669c6994cae2e69fe13c2513b89
fd3f04965452eb28c5e0dc0335f34b52939152e3
35506 F20101206_AABNCO levinbook_y_Page_032.QC.jpg
12ced59ee0fb9beb52eb6b97d00d5dd5
115750 F20101206_AABNZX levinbook_y_Page_038.jpg
0d1f3f1c424d3b19b204aa8bcca8430c
bb3d0612789a906664ef2eaf1fda69d1da19b02b
59515 F20101206_AABNDD levinbook_y_Page_025.pro
29ed9c1ab918843c1991a42c2d16e9a4
497a1f262b522df9101319af5bf5c9365486e3b4
9500 F20101206_AABNCP levinbook_y_Page_075thm.jpg
90a05b153c1fea85c7de9d0619647283
100749 F20101206_AABNDE levinbook_y_Page_098.jpg
f45091c51909a0e0d2cde560116335da
c82a5ed1ff6b20657697a76c81d36c349438025f
45413 F20101206_AABNCQ levinbook_y_Page_081.pro
c778094056a8c62394891f96ff88570d
5e6ed000474737f1e23f882c49639de505a50a16
110974 F20101206_AABNZY levinbook_y_Page_055.jpg
d177170d7dfa7efcdde939c2e26ce570
aef51182c400bd01251c4683f7188af03453f3c6
37744 F20101206_AABNDF levinbook_y_Page_019.QC.jpg
653584770a9abac4d09fafb7a25c935d
571ded18d845cc2575649b2b0186053d6d0a2be4
30169 F20101206_AABNCR levinbook_y_Page_063.QC.jpg
de705bac9a9421185b2ab4df4eca0ce4e33834e4
129181 F20101206_AABNZZ levinbook_y_Page_074.jpg
facdbe568216fdb37bf704c724c06e3e
9d7aac0440f3dca8db0d88c5c18bf7bc7b358484
25589 F20101206_AABNDG levinbook_y_Page_091.QC.jpg
db70b5fc5a595fcff0e5f9f22d5481b6cf62ce10
823629 F20101206_AABNCS levinbook_y_Page_051.jp2
184dd80f5769ed96f7a15fce635591fb
6c0b7e14b6b7ace031216020192bb3fd68e54eff
2036 F20101206_AABNDH levinbook_y_Page_084.txt
6ef4e0f0a9f1bce40624687a47ac444e
972621 F20101206_AABNCT levinbook_y_Page_050.jp2
c497f6f761cc03b034ff070647cebf94
cc1b7328a0b877f9e3f7f56384e1d55bb950519c
651 F20101206_AABNDI levinbook_y_Page_016.txt
34bcd2b6e576eeca6a0a6025b8aa20bf
7d9b18b5767303c766795ccba7c3feb453edd76b
2098 F20101206_AABNCU levinbook_y_Page_068.txt
287f7f957411790ebbf775efd84d36a5
7abdf0cab4986fece4321001c8aa295783732651
45836 F20101206_AABNDJ levinbook_y_Page_088.pro
ff9ff009ed23d69f270eb688c74909e8
bbd74a76691c61a8f99605d81bd0d78d6cea5fee
1051959 F20101206_AABNCV levinbook_y_Page_044.jp2
cd178e3d33d638f848e412cdff0fc1c6
2279 F20101206_AABNDK levinbook_y_Page_055.txt
e36c5533ca7539298d8704fb11aecd91
a0aa600727335d424dee85060d72b2c43a93c007
121495 F20101206_AABNCW levinbook_y_Page_107.jp2
8cc97fe2f85490e2b6338cfb0b42bc3a3c01e0d6
112580 F20101206_AABNEA levinbook_y_Page_034.jp2
4dd0bae7901e77835c9254ca7589c8b6
1fa5b485f861e16cb08e20c6705f1be53a17fc03
F20101206_AABNDL levinbook_y_Page_046.tif
e075e76d975dffbb877fe22823472b9b
aeb0831d9ee8299ef47a9708101bf4832c4c1d31
1024212 F20101206_AABNCX levinbook_y_Page_028.jp2
536892ea4ebe0538bdc251098260f9a0
32694072422a5e6389a115b348621e183b526d7b
F20101206_AABNEB levinbook_y_Page_077.tif
b785f93b2f65b85e9c523b8ef6e38c0c
0c213b1864783e4f21ae236e7fe2e13f882bf4bf
F20101206_AABNDM levinbook_y_Page_049.tif
7f3d532760c149258d0655740ca7b6e7
3f0200c7df0a10351a93e49d6be57c678b1c8f06
42513 F20101206_AABNCY levinbook_y_Page_044.QC.jpg
f06e99d51b358d078c9d79750402bf40
3c5ed343cf776cfe3311cf620e5d4590c0c87ebe
1051921 F20101206_AABNEC levinbook_y_Page_031.jp2
fed5164176c4f51ce5cd0357284f6538
ab662a5ec3314f95f22cc286c1a2f046310dcc60
10898 F20101206_AABNDN levinbook_y_Page_076.jpg
ce1f85afb2185972372375b7c657e76d
7aee3a975f2ae90bebededb66d296678fc42cbb7
9270 F20101206_AABNCZ levinbook_y_Page_112thm.jpg
3f57a9932a0e10477117bfc51099cd9f
91ed9f4c211a358f04c5188507bc92d309a97076
106949 F20101206_AABNDO levinbook_y_Page_034.jpg
6263aab616493941e1e31bbc210a30d8
61456 F20101206_AABNED levinbook_y_Page_062.pro
eeac9a4645638baa89858e181e36758c
678df3b1b11f8ea8e23f72bac977df1667665e64
F20101206_AABNDP levinbook_y_Page_042.jp2
d4dc0833b7a6e87fdd03efb05bf9cc68
e299ce6c1b1e9dc0dafdb09d6163ab60b98757f5
2482 F20101206_AABNEE levinbook_y_Page_107.txt
5e12037080427ec6f1084690afced670
57933e85ac406753ce7449a09a868ee808c1bf67
F20101206_AABNDQ levinbook_y_Page_055.tif
24d01015ed9100b2a81a5e8f8f022e42
495a5c477937d174b26f9094e9cb36a83a395076
7189 F20101206_AABNEF levinbook_y_Page_029thm.jpg
496623405ddc1f88ecec2ddabf6c3771
d71a567d4460df9652a7400af2065baa63753edf
1837 F20101206_AABNDR levinbook_y_Page_085.txt
147c26a7bdd7f1a4e6426afb87dfa94e
111fed73331eebf4672cd05fec033931ff429a71
15394 F20101206_AABNEG levinbook_y_Page_114.pro
3befe0a61f161d1a2c1e25ba1a9cba7b
233ed7664c209bd90355bde5484cafed18735e22
F20101206_AABNDS levinbook_y_Page_100.tif
74b756f26a08c9a5a6b0757f92a63616c2ee2b36
94450 F20101206_AABNEH levinbook_y_Page_028.jpg
1cfa5e1b512b2be628cf245e2390986d27dbe31c
62866 F20101206_AABNDT levinbook_y_Page_017.pro
856d49de664db9da6498692aed6a36ea
19b33b03033751cd1b524988cd64dd7ec73e5976
107136 F20101206_AABNEI levinbook_y_Page_039.jpg
854167f266dc42ba20b6fcec9dd75b10
f94a2a5374a021471514c106eca2624c8f84f7f4
7221 F20101206_AABNDU levinbook_y_Page_022thm.jpg
d2cb205645b54e467ea2168ef391b517c5648445
911 F20101206_AABNEJ levinbook_y_Page_002.pro
eccc8dd8125af0c770a4342a716f90f8
7ee6d9e2b0e0f6df63f0bdaf786a59fb79232fe9
63449 F20101206_AABNDV levinbook_y_Page_071.pro
29fff5aa2fc3a4612053375d40f4450d
4acb6eb55a429ba0aef2eb9a9e104104ca726cce
1051968 F20101206_AABNEK levinbook_y_Page_090.jp2
14756db811d1e91be1ec38c66f388fc0
8483 F20101206_AABNDW levinbook_y_Page_092thm.jpg
01a2c11b4c5ceb769e4be744382a547c
80888a9af3f39f8be361a425023ff0c8150823e7
15099 F20101206_AABNEL levinbook_y_Page_110.pro
a4168f5e4b5641157afd7681717fb1e7a9628e69
724 F20101206_AABNDX levinbook_y_Page_110.txt
ee5317dc0e7de6db8ff9f7fbcac90b43
c2126fab4486641e36c49605d023ff2c65ca5ae0
2351 F20101206_AABNFA levinbook_y_Page_025.txt
3429e25e010a86c2fb8fb55b986151b9e6fa9d60
2253 F20101206_AABNEM levinbook_y_Page_033.txt
2ed102a79bd721b560f93ee987c2a097
32686 F20101206_AABNDY levinbook_y_Page_078.QC.jpg
1ceb4842affbd237854904f29041bbf9
02325de6e1cfa42df7407f138f964854a22f442f
23140 F20101206_AABNFB levinbook_y_Page_061.QC.jpg
ac9f53c45c2146d4d528af0f85a4aa65
ff628cdd0a0ffc6e23797a66696309f187ecc614
2142 F20101206_AABNEN levinbook_y_Page_027.txt
ce269b41dd330a11e36690a34bca99ba
b411e1ccb4bf3c0fe241ed8dd126ca227672a89f
76245 F20101206_AABNDZ levinbook_y_Page_091.jpg
0ee720590e0eb4b0563baea85f17c845
329b9fd45aa8f971c1c7a255c617bc127173fc9e
47136 F20101206_AABNFC levinbook_y_Page_024.pro
5c805d2f63274dde2dc6c64c26af620b
7615175a5cc61632de21e0ee898efee617bb7042
62332 F20101206_AABNEO levinbook_y_Page_041.pro
c079aa67f6b0be7bb8fa5475f4d1e18e
20519 F20101206_AABNFD levinbook_y_Page_004.QC.jpg
45f961ffba44c314968ca7de0f19b822
7f959d2b2f393abb12a96dd2ae8d8401cff02349
128880 F20101206_AABNEP levinbook_y_Page_012.jpg
aac5cb5887b9010b39d4ee02c032a861
e7ffcc133835f86b0446f5eb52b02cb3d41d77b8
F20101206_AABNEQ levinbook_y_Page_081.tif
87146bc15bdd13c96a3d0020e009a98c021bb802
1051953 F20101206_AABNFE levinbook_y_Page_098.jp2
98142eb1b6fcfb202671dc071cfde5df
b39bf6166dd77184449f3da6af3cee219df7bba6
F20101206_AABNER levinbook_y_Page_061.tif
ee333a441d1f0d4ac7b92a9a3a80d526
e3e3643c47872938f1fa5dc7aa7dcfdb564c5ce7
8302 F20101206_AABNFF levinbook_y_Page_020thm.jpg
919aa546ab455decd7406c07829ce9f0
83ffa34a8ecfd45be8b482869e07b137fcb033e3
F20101206_AABNES levinbook_y_Page_001.tif
1f052faa87c5b16e041f84961af561e0
ed6277a7d742195d6e452e2e343847103dd03e6b
4132 F20101206_AABNFG levinbook_y_Page_002.jpg
9903783e472f84768dae0c2e5d5d48bb
05cf5891876186ca76eb87a4296a50c9f163ccbe
326 F20101206_AABNET levinbook_y_Page_010.txt
7962 F20101206_AABNFH levinbook_y_Page_084thm.jpg
ac47375349c6c40a6e19c48d9289600d
66b2ff5a4864048ef429aa9ab443462fe77e9e96
29077 F20101206_AABNEU levinbook_y_Page_081.QC.jpg
e09477a6640f6657cb7af3c6c8a2a1d1
5d5213c39a6537f8173791ff57cbeff4847eba45
F20101206_AABNFI levinbook_y_Page_073thm.jpg
751030925bb5fa6abda2f349bce95bb2
5b1fb87884e4e855f919b7905a3634d4cee5db28
51303 F20101206_AABNEV levinbook_y_Page_052.pro
2a76384f34ba918f81cb273c36c9835b
fb318806881ce6c7488c8d1a67a2d823b0a6e822
2382 F20101206_AABNFJ levinbook_y_Page_066.txt
d44de333f5d346ec9c238362ff273bd0
0b311e498e00be39d6bba8be3afdc811c1eb686d
F20101206_AABNEW levinbook_y_Page_043.tif
f6aec6ec00d55bec529191f313e73820
3be7864e98bc66dd9a4a3a889f0b6a3466695fe6
1997 F20101206_AABMZQ levinbook_y_Page_024.txt
ebc1d88f64757013e1d270d79d9b7d834da9e73e
F20101206_AABNFK levinbook_y_Page_101.tif
217a4d33ae74a0430c13938d5a5eb894
f7803dce5586690a10f24bb3ab5cba77c705a3f1
382967 F20101206_AABNEX levinbook_y_Page_104.jp2
2ee3f553298db8f82b4fa3f6788c9826
db2955a3d6fd2239c1c384f91ddb1d6a24e5c93c
2604 F20101206_AABMZR levinbook_y_Page_093.txt
e2e8a4f0ea39a454233a67493c14a6dd
165 F20101206_AABNGA levinbook_y_Page_076.txt
d96655ec145f4de52d0601362ac6b95814fb46fa
6509 F20101206_AABNFL levinbook_y_Page_097thm.jpg
eb6419908e6758753df86f61d78071d1
dec08ee874cca32b8b85b08c47eda542ce372015
F20101206_AABNEY levinbook_y_Page_065.tif
71b93bc7e1ac6742c52858f4d6e4e0df
9941c3bb7d9cd85c83083734fd9b124a221cf80a
9067 F20101206_AABMZS levinbook_y_Page_035thm.jpg
71a0329fb34a637ed9111efdc25250e3
5d010936bdaa918117ec8a2e2ba0d8315c05d886
42238 F20101206_AABNGB levinbook_y_Page_026.QC.jpg
64170d7e2b0b3452cf1745ed2d2a29de
9f9b61171611b7151885a68c97c1a5f7a461c187
436 F20101206_AABNFM levinbook_y_Page_023.txt
033d5fb78a584810383b1e94c64c6f1e
408d1a5fb50bc70e7478bf61e7b58ffff4b403a8
119973 F20101206_AABNEZ levinbook_y_Page_005.jpg
1dd6dca32277bd91982fae22ee8e3435
aedbb02429bb91c37cca54fcc6af2898673c8a6d
1724 F20101206_AABMZT levinbook_y_Page_029.txt
ee3895366f34f7e03389a80de09649e4
0348b622c661506255b46f99a47f1fa92e87c5c7
F20101206_AABNGC levinbook_y_Page_083.jp2
e71af72902a077cc8354fa5643f340ce
28924 F20101206_AABNFN levinbook_y_Page_089.QC.jpg
56d5cdd6c322fb11c46eb057819cbe688c640fc4
1051935 F20101206_AABMZU levinbook_y_Page_062.jp2
d1ba6326fc9f5a46371f50ec49fa398f
ea6338fc9634028a07b24ebdc4e5c260850c6e15
121100 F20101206_AABNGD levinbook_y_Page_017.jpg
0dcc7892765343323620893e87a0fe31
131001 F20101206_AABNFO levinbook_y_Page_014.jpg
4727cf0b1010e08010f392794b937e25
7388 F20101206_AABMZV levinbook_y_Page_008.pro
6c3777a16f7a9e369c7c864bb4dc35cf
566f3e2b84b917fc25753e15aedc09efb0de7109
1051985 F20101206_AABNGE levinbook_y_Page_099.jp2
f161a9f0212cedbf6678dd4c07607ddb
837ca40d113c924d674bfedc818700a070ebb3db
5786 F20101206_AABNFP levinbook_y_Page_051thm.jpg
20ca70997458f67a9ef1ddfd5d7bc258
2b659b13e2d69702be96678ffe6c0d9e5f6b9e25
101985 F20101206_AABMZW levinbook_y_Page_076.jp2
5554192e611aaf611eca58fe9ff48bee
de9ecc1acb665d00b976283f67b439b99553db4f
32501 F20101206_AABNFQ levinbook_y_Page_098.QC.jpg
8e5244bb04782e407d69ff6801053e3e
221382bee41063bf04d21219024ec63f8e2ac7b2
8365 F20101206_AABMZX levinbook_y_Page_032thm.jpg
964a95fc5bb67b0b6f1e2591979850826f391e8a
2698 F20101206_AABNGF levinbook_y_Page_102.txt
8e87685cb977e303fc3c09c70be7b7d4
41f10ba9e96bed3098f3fdf7d8d6126f8ba9e1b3
40755 F20101206_AABNFR levinbook_y_Page_080.pro
f79018d2eabeaa7e2f5f97a34aaa0dc6
125533 F20101206_AABMZY levinbook_y_Page_093.jpg
ef23b762ff9006bbb85b5cc94d85f46a
69daa1a9ef403338ac5c9cfe6eb57b4430e22e0e
1051918 F20101206_AABNGG levinbook_y_Page_045.jp2
14caa354f7b38ac764f334a9878f2b80
da9c1936475a61602d57293a28109857aa235246
2234 F20101206_AABNFS levinbook_y_Page_038.txt
7a4df50aa980c82042116b8dd049277c
6e1cb507d587bd10ba84abb8632e03ea63af102b
35436 F20101206_AABNGH levinbook_y_Page_068.QC.jpg
644ff55e8fe2c20219ed4d7cf42255a3
5f2b60270ef5bb273e39daea2b23d060ab2905be
109 F20101206_AABNFT levinbook_y_Page_003.txt
9d0a5a5c65852092fba347f910193935
de7ea9f027e65ef97a59c69caf65e63932c36ba1
79480 F20101206_AABMZZ levinbook_y_Page_053.jpg
fcfcdf74d15c5b4eff2a8962ba616567
b6102031b65191c166a13f11701c9a6a6984f352
904336 F20101206_AABNGI levinbook_y_Page_022.jp2
690b57b6fd239b8ae92082ed3962945e
06699f5d7c2b594ab5e2e5f0e37fd180d39d3d02
8781 F20101206_AABNFU levinbook_y_Page_045thm.jpg
6146b34b16c0e0e230244e03fa1866fe
52f2a639e2e554b182b8336776ef7ed7e90f6aef
83468 F20101206_AABNGJ levinbook_y_Page_113.jpg
09f153b084bfe8b02f6ba2da6ec713c8
53379 F20101206_AABNFV levinbook_y_Page_090.pro
746a05c14aa6e2794739bf07e037152a
8a8b676eed407db2fc276f82ccd8f48451e41c43
2383 F20101206_AABNGK levinbook_y_Page_069.txt
24679 F20101206_AABNFW levinbook_y_Page_047.pro
d391133fff17344d8ce607e46f28f6cb
002dff816e87029789c503a104c4bbb8fe65c025
11819 F20101206_AABNHA levinbook_y_Page_103.QC.jpg
9f3152f3bc8fc406c1bb63a55385320e
8d9c6b8e011f63225124ed240ef23603098f69ea
F20101206_AABNGL levinbook_y_Page_062.tif
5a6576aa4a36c8e3d5b84aa268a10d1f798b7dc6
F20101206_AABNFX levinbook_y_Page_018.tif
fdd968a71772376d366aa9065e6307c9
F20101206_AABNHB levinbook_y_Page_056.tif
95f0cd7d12920753c929c4a565232519
2347 F20101206_AABNGM levinbook_y_Page_092.txt
f9f52b6f3f7ebcd7df8930f3ff05644f
76ab8300b4daf4c161b9fa05b08bc97f3a22fc91
104464 F20101206_AABNFY levinbook_y_Page_040.jpg
e29ca6b53fa62b594ef2662385496a78
58313 F20101206_AABNHC levinbook_y_Page_069.pro
2e42fcc15eb1d9f34124e928bf3ffda1
632988130f53f12d2688492201616c8bf1e9300d
7680 F20101206_AABNGN levinbook_y_Page_024thm.jpg
beee7fd77b329b18e0d2db64dc0c19d755a0f6c9
63559 F20101206_AABNFZ levinbook_y_Page_112.pro
26cf83d506fd6d8fa24edec1b11d1ae84943c60d
F20101206_AABNHD levinbook_y_Page_108.tif
652a7722a5d1848d8bfc907e68ebd2f7
4142 F20101206_AABNGO levinbook_y_Page_018thm.jpg
478c7cbb251f95ac508be3ca5de0bd71
F20101206_AABNHE levinbook_y_Page_041.jp2
bd5e52f7def75c0ce37538f2ddbc5412
894027 F20101206_AABNGP levinbook_y_Page_100.jp2
779af0cb21505b3c4cda8b918881a0ec
f9392e180c9501a673e93998fb9e0dc680263f14
65676 F20101206_AABNHF levinbook_y_Page_012.pro
0e7f2db1e48113a86fbe14ba1d3503f4
fef1965dee5c1b725cc1112d55eca0c78796cca5
46202 F20101206_AABNGQ levinbook_y_Page_067.pro
6ca77d8638be588823275a396fb3317e
8075 F20101206_AABNGR levinbook_y_Page_010.pro
2aeb75eae2b3a30b080fcd579029efd2
a8dd54cf769c47785cedab57fa95fd658f4f067e
F20101206_AABNGS levinbook_y_Page_035.tif
973dd58443af99ea99224445e71310dc
5dc221261947a2e5cd671a066f0c84457b6e2d70
55248 F20101206_AABNHG levinbook_y_Page_020.pro
299a04a04c5af46e04715ac77ea5d0d4
ef5ffbb55bfa90e4f0d1c48ce75fc3f455028233
40081 F20101206_AABNGT levinbook_y_Page_054.QC.jpg
0af005304e9f263cc44d290fe0011c8b
109840 F20101206_AABNHH levinbook_y_Page_032.jpg
f804f0ee7e57b37ae9574b40641487ec
5d5c1a6278624f3a5a2c5f26c911a3a106519b9e
1106 F20101206_AABNGU levinbook_y_Page_018.txt
b7c87cfedecc1d478b851b1628ac682e
89bcb8de0b55df4e89bf96b81480f03d5291878f
103007 F20101206_AABNHI levinbook_y_Page_078.jp2
19468ebedf3c295267c1b6c4ceabfb2c
17e02fb31ff627f836d0207bdaa1d05fe80a69d5
937388 F20101206_AABNGV levinbook_y_Page_089.jp2
d3c7acf228d145c27f4bd3a216386550
90bf6903a034e7704742a6f32555152fdb18b372
F20101206_AABNHJ levinbook_y_Page_030.tif
2cc1f2fd010925ed07aa765440889ceb
1051951 F20101206_AABNGW levinbook_y_Page_020.jp2
5fbc47b624f4b7269dce15c6ac7517c2
56934 F20101206_AABNHK levinbook_y_Page_038.pro
2e45602ae15965b67c4e64df142d6647
2e1ee836767b9d72800ab9e6f01ef97873686c70
1098 F20101206_AABNGX levinbook_y_Page_047.txt
59727774574d38841de39010d1afab60
16d4871cf028e59767dc38de66743aee3fb23328
2278 F20101206_AABNIA levinbook_y_Page_045.txt
1d480e2a9268b9967ac284c9a98fc392
4182f50d731a53fb4ca7ac9c9e9c22f3e2a3e96a
11702 F20101206_AABNHL levinbook_y_Page_103.pro
da7684a6bacf987c9e57cb6e66b1bb66ecfda0c0
29865 F20101206_AABNGY levinbook_y_Page_050.QC.jpg
db61540677d13733bc0dfeb4f491bb24
fb74bc315d4c459659e63e905c0e87b6219a35d3
28686 F20101206_AABNIB levinbook_y_Page_029.QC.jpg
16b6c45de0fe80d95205bb373655df5e
da579ef2ecc660fce7725d130c3e8a4ebcfa08c0
2385 F20101206_AABNHM levinbook_y_Page_056.txt
27dc6f16f3c7f0f24e82e0d843eb1b2a
95e0d2ab5a7668648d2b3a2aeb11d1a016a2bc8b
7498 F20101206_AABNGZ levinbook_y_Page_096thm.jpg
425f8e1f7299f0df5a55f4a39cee20e0
814bc906ed210117b8f46be1fcbed52fd21b050a
35854 F20101206_AABNIC levinbook_y_Page_057.QC.jpg
f0d9d24c2d0741f29db1c9c45ebe11e9
11b7df0c803f18550e99c8cea710a1cd60095d14
372369 F20101206_AABNHN levinbook_y_Page_110.jp2
54d2123460de09bcc01038cf269b10aa
40af8d62fcc20422e4f7b432f8bffbdf96fa7390
1047327 F20101206_AABNID levinbook_y_Page_084.jp2
fb51c6a267093a78cd58ece1690a82a0
218442 F20101206_AABNHO levinbook_y_Page_095.jp2
1a1d7d8afcdb1693f90f235c28589482
61909d065e70db5008b49c63b12ba554413118af
82814 F20101206_AABNIE levinbook_y_Page_094.jpg
53a2c625cba3dd1ba0015147483220e2da431a5d
58394 F20101206_AABNHP levinbook_y_Page_049.pro
b2958f838d9ca04d7775fe3227987f1b7f93b1e1
508 F20101206_AABNIF levinbook_y_Page_002thm.jpg
963995afb082884b3889b86c23da844e
F20101206_AABNHQ levinbook_y_Page_074.tif
0ff0f4061dcbbe296e46479a7bd402ca
a1d0ac7ea3bddcd4a0cd178d00c47b025c57139a
3603 F20101206_AABNIG levinbook_y_Page_103thm.jpg
c8573e51926fd8d353d32d01250c0bb8
112113 F20101206_AABNHR levinbook_y_Page_036.jpg
4163ba7d47425a8790e460230101e04f
244060 F20101206_AABNHS levinbook_y_Page_006.jp2
3d0deed430f4589f01d7f156b9b68ffc
4539fb1d16d08bbbf9152f4359f0ab6718c0efea
348771 F20101206_AABNIH levinbook_y_Page_103.jp2
8d5a98f8da314d4580a90ed04da971ab
36a17ec2b1ba3e08d6b21ae5b5dec160801e23de
8818 F20101206_AABNHT levinbook_y_Page_069thm.jpg
b8825eabfbe7171bef039809ed2a0a3a
63d7928c860c6846fd1960d5527184fc56e58bea
11869 F20101206_AABNII levinbook_y_Page_108.pro
f8871f9360567da48a2844bb98a083fd
F20101206_AABNHU levinbook_y_Page_082.jp2
4384827da4d61f3eb75c3967a84e7771
4409526d31c9b01ba0c63811e797343235a674fb
35887 F20101206_AABNIJ levinbook_y_Page_101.QC.jpg
78a1f398dffa261394c840bffa2129e7
32522e1b3844821e3191e0cdb71964167227ba2b
1833 F20101206_AABNHV levinbook_y_Page_053.txt
6f7f0efe33286a271e72f88046acdb18
5c65883f9488696b1cc492c38c234db1f1b22f6d
1455 F20101206_AABNIK levinbook_y_Page_061.txt
d52b14f15fcef97239e9b1b3d39cd88d
7542 F20101206_AABNHW levinbook_y_Page_087thm.jpg
108e70a615c23c58b6d4bbd782f06f13
a4c036c9c7657d66ca6c98b558c4ecdda48ebc51
2240 F20101206_AABNJA levinbook_y_Page_037.txt
b23e4f0fb9ab82733525c8f79cb3bfa1
6537e2930f17a0f6a471537a0772a24fac178d6b
8172 F20101206_AABNIL levinbook_y_Page_043thm.jpg
53b683a241efd839832f2d1a3ffa70c7
43452cd6c59ae0e4281f93bcafbefcf9bb75161a
1458 F20101206_AABNHX levinbook_y_Page_008thm.jpg
3e464340d89aee6f4a4abddf5836d8dd
cb6e33144a2073fd034ccd3a0ab72baa3e185c49
2293 F20101206_AABNJB levinbook_y_Page_036.txt
3fd8b5424a1f2e535b108b2deff9298c
36019 F20101206_AABNIM levinbook_y_Page_104.jpg
5bf812755b19ab1d81d20cbf53324319
fe80d0d8af2e557334255aff7e3f1d82af59d69f
2079 F20101206_AABNHY levinbook_y_Page_088.txt
efeaf56bff89f3bb2d0443d7297fb6d1
4d9b6cdfebec354de73db6d15d5f2fa61b3fecae
7508 F20101206_AABNJC levinbook_y_Page_089thm.jpg
7d708b7b8b1389616d528c656d7bf617
16094 F20101206_AABNIN levinbook_y_Page_010.jpg
3d2ccf0a9183c2769655740ee12b1ac01107e2b2
29873 F20101206_AABNHZ levinbook_y_Page_024.QC.jpg
40e14a87b60621b3b781e9e63a2c19e3
28ab6403043c7690086bf478a55eae6f06f926f1
8252 F20101206_AABNJD levinbook_y_Page_033thm.jpg
6dcaaba127366a9a54447b650d574880
8b0096d320c21d84ab157986b268f976b32c8691
86194 F20101206_AABNIO levinbook_y_Page_029.jpg
73ca5980530158986a8e29ef4a1a2195
9cca6c80e814f01f8a8ebe04a92c0908236dc7e0
109495 F20101206_AABNJE levinbook_y_Page_009.jpg
e31a8bc476113ccd15b4a49a86e63e76
df59bc719b7e3a0c86252a85b3cc798ccaa4a751
F20101206_AABNIP levinbook_y_Page_093.tif
d4c36d3795ec60246fa8c921491b6b27
8f1b9e9de6a70d23fd21f5c6ed5c56b06a9bd81b
1051943 F20101206_AABNJF levinbook_y_Page_025.jp2
68e8804cd8226bb5eb3ed19065e6b3c1b4fb9199
153870 F20101206_AABNIQ levinbook_y_Page_070.jp2
8f3523ef8ab3cdb520cda46fd6c4dfd5
fca4e5cb63d22eba6a267f5753ea91980bde5e40
43314 F20101206_AABNJG levinbook_y_Page_096.pro
b3ef1cb4839318ac007a2dc0f09b72e7
27389 F20101206_AABNIR levinbook_y_Page_108.jpg
17193f7e2d349571667e266ebc09c193
ee4171933dd01716fedf7ee5c3809f555a1b1a9d
67658 F20101206_AABNJH levinbook_y_Page_070.pro
afc9bb0bc3d09eafa8a63effe21dd2ca
922963 F20101206_AABNIS levinbook_y_Page_077.jp2
878092e8cd786ebab9d8f6bf22ea6511
61af13334e0f71cae19c1535b6984fa4f8127d04
109126 F20101206_AABNIT levinbook_y_Page_027.jpg
1169f3b1ae8270c7276391db78d6f38e
38b7c7701287d29956ce31285b37c17e46c2b1cb
985888 F20101206_AABNJI levinbook_y_Page_079.jp2
3bae67413b2118700ba964113ef52f50
d1095052362a5dd3a6a0ae0152fbac13a6376d42
F20101206_AABNIU levinbook_y_Page_073.tif
0bb57aebdc4206e9def5d88476f45eda
cd514a8520124afa77ea02608046cddc83858270
1051920 F20101206_AABNJJ levinbook_y_Page_011.jp2
073fc48e361477b42c1f29209e2430d8
3a8d0dc0e580b532db5db96a75d3159370cccdff
F20101206_AABNIV levinbook_y_Page_004.tif
73089fe088a59e1941faae3e7de9c848
c7798ae46e33ba96b26a1dd572bf777f9e03da25
1051967 F20101206_AABNJK levinbook_y_Page_092.jp2
6d022c719725f9054a987a415084bd63
8e2d43bdf5fdcced739633fe528928935da02d16
8126 F20101206_AABNIW levinbook_y_Page_078thm.jpg
7e9a5b4957ac5efbeb22ebd3f4d75132
21b2aa3139ff9745636836138744f4e20062eb1f
7027 F20101206_AABNJL levinbook_y_Page_048thm.jpg
dc2f533f5091db9f5949e3e778101efa
584acb96cf1b40780ce5a74383115958a28f09fa
35620 F20101206_AABNIX levinbook_y_Page_055.QC.jpg
22fbc148549289452b83c84c6b4ef0ba
fc0719724332027d0c60ffe9481d2a6d5141a777
898606 F20101206_AABNKA levinbook_y_Page_060.jp2
ed27dd4e987ce0dbb054c55f52c0a227
4313cdfdd63476c0ef34ba48143ec39b51ea67fd
7194 F20101206_AABNJM levinbook_y_Page_085thm.jpg
7055d2173753d721dccbce5bb84c8ac3
7fcc10001b1c9a0aa561833c09940b8ea8413175
25120 F20101206_AABNIY levinbook_y_Page_094.QC.jpg
9882003d54dd2787ff92f061680a596a
db3dfae87c699bf3f1e1004a5a55d091d92788e6
118274 F20101206_AABNKB levinbook_y_Page_019.jpg
a90abb3092c0229a235b99a2b9468dba
98281 F20101206_AABNJN levinbook_y_Page_084.jpg
58ff0ef8b1c4c7db076e803d401da30a
635f5d2daeff18382732267643bedda8dee5345c
F20101206_AABNIZ levinbook_y_Page_032.tif
88a612f2a5a2502484ce4cb495d16d7c
68b71a2d929fb50a8b3cafc5c79c572f2cc4caed
8462 F20101206_AABNKC levinbook_y_Page_065thm.jpg
2835cdb5c72a300c3bab23ff2ffe3e95
62379 F20101206_AABNJO levinbook_y_Page_011.pro
935d4f7232b8b60328ce378251f961c1
4a1a37f6329c7a10eec31f466be49e6f5c2f2077
3892 F20101206_AABNKD levinbook_y_Page_104thm.jpg
3d9f25d96d5538102c4c4d54d945b3ea
2efd430313817c8e0f89f1f1169cf5d89fb21633
40681 F20101206_AABNJP levinbook_y_Page_077.pro
73b61789d53c6d77d43ddced2e92ce09
8b2f8507caa90ca2f8b9fa6598b459a1748ee27d
1488 F20101206_AABNKE levinbook_y_Page_003.QC.jpg
fc1557e100f3f3e5ef24515c8378a1b6
1e00eec2b19a1df69a8b4790782efe2c2a362036
50758 F20101206_AABNJQ levinbook_y_Page_098.pro
9643de0f3ca28e962f8794c2a8b3027d
186737af5bc6d0ff524ffa29b74975efda9ed179
47077 F20101206_AABNKF levinbook_y_Page_048.pro
1f134d758d93f597e6cacd30b9658290
f3c37294b620de5025873b14e10a28637b2ddcc9
54028 F20101206_AABNJR levinbook_y_Page_027.pro
f9eda60825b67980dee53d33825e4158
5309a3c04023324a6c968e89c79ac167b78006cc
367 F20101206_AABNKG levinbook_y_Page_008.txt
0cfe5a80b8b7ecf3e6161d11ec17a01b
38b00cb6df03ceb811b0d8eb4ce761a1af61ed38
2169 F20101206_AABNJS levinbook_y_Page_031.txt
9950bdc0fe5fa7f9f788c07b7e3a7366
1caffdba99be4680dfb89925dbfe59e704603913
F20101206_AABNKH levinbook_y_Page_027.jp2
3b1d0089ffeb91d3a37352af9792f121
e27a22d0abbb5502d57939568c116b161fec1a8d
9381 F20101206_AABNJT levinbook_y_Page_015thm.jpg
de2b89301201147d791a102834ca27c3
04aa47656b36036d098c5097647f48a6f0fe67d2
854491 F20101206_AABNKI levinbook_y_Page_094.jp2
81b561746b1d3815a6d1d77e47512a91
F20101206_AABNJU levinbook_y_Page_088.tif
33f5f73d2ccbd0083a703dbca213082a
eacae73e48af8dc1a66d3805ac4f90bef72d895b
6876 F20101206_AABNJV levinbook_y_Page_095.pro
05713c0e84e90b12781ff283ba06b132
f977b0fb395dc2b78d02eb924ea747a92bf2d84b
1763 F20101206_AABNKJ levinbook_y_Page_022.txt
68a6d778b7368d3abbbb26c56941d21d
23ba17491278a26f21b0350218561ffd9cb5d3d7
126456 F20101206_AABNJW levinbook_y_Page_111.jpg
d80052469e39867c52729db1f8079947fdf3aeaa
2161 F20101206_AABNKK levinbook_y_Page_108thm.jpg
234db79718b24606ece220cc9e7dd5866478689e
64665 F20101206_AABNJX levinbook_y_Page_013.pro
3754d3a445ac7dc1b482476f2ddbe7c0
b45960471c1634bf7b02ff8085f903ddb6953997
50998 F20101206_AABNLA levinbook_y_Page_082.pro
4bed724aedecaecee10debc5800a8f3a
b2b21354d21645727599a77e8bd1fd62e8519269
467 F20101206_AABNKL levinbook_y_Page_108.txt
9653f4fde930d04d8835815354db971c
70e610767100a4bdf614ce1421c3d1ae2bc24ce8
3499 F20101206_AABNJY levinbook_y_Page_072.pro
46559cc74d4d3996df94e86d0f8148da
cde127ec5b17386178e14f864730be8f16068d2c
F20101206_AABNLB levinbook_y_Page_040.tif
e694a17b9c557de872d10f4e320aa933
10d304a8393cefa57946bf0f8b63e1fd220f1b3f
36068 F20101206_AABNKM levinbook_y_Page_114.jp2
acb66d87a2fef282866f97bdf7532434
79351f383edc26314e597e6d59a6aff3e449d4e3
22014 F20101206_AABNJZ levinbook_y_Page_095.jpg
ced03498c03bd3175712b40dd3683f64
0d783158a1e24a5451d47ff3cefa946cf639821c
8820 F20101206_AABNLC levinbook_y_Page_108.QC.jpg
56a5a4da29dfa2fc8090402d345bf734
e291ca30e9ed31d66d79c75c8e6f25edfd732194
39196 F20101206_AABNKN levinbook_y_Page_017.QC.jpg
82204cdf870026339a4d959ec62a362c
6be7eeeb6798a05ae17cf1e695bf3d6441c1628c
122803 F20101206_AABNLD levinbook_y_Page_042.jpg
6e040039775b16ba0bc37d165803c76a
92885 F20101206_AABNKO levinbook_y_Page_048.jpg
45f3bebf58bca2167f225bf74f627bea
8d6d527f856455068a609154dbbc2dc2a2fa3f24
29029 F20101206_AABNLE levinbook_y_Page_077.QC.jpg
c40282715063fe47ccd97d1f18c91929
9469 F20101206_AABNKP levinbook_y_Page_012thm.jpg
5c8fe012c0855258899544bbddfd8571
8134ff4a113b87044eb07cb156b701b6b51b2353
7603 F20101206_AABNLF levinbook_y_Page_081thm.jpg
4604114aa987d265e59d1ff6d0a257d0
d7a36f6fc4b3c06343e39ab8d4fd7d0c3cffdc41
6982 F20101206_AABNKQ levinbook_y_Page_006.pro
3cca5b5e6d854e868e366b9768cbeab0
d886ac49dbf63af5784e70d91bcd1ee3834c9c11
2120 F20101206_AABNLG levinbook_y_Page_106.txt
0af1cb75486f472f0aa5291b51976ab8
2fb635289724869e3922a25afe38e1568fdae4fe
F20101206_AABNKR levinbook_y_Page_092.tif
6a0c73c88db623a6f0f4d6f6201437d1
15b4de531df3fe8515167b95c0a5932df64300d7
F20101206_AABNLH levinbook_y_Page_022.tif
13cbdb355019b4253ed6e9e680d53446
a1e9eb95b2858472de6643cbde02c3420d926749
631 F20101206_AABNKS levinbook_y_Page_003thm.jpg
a7801ca0442f94899edaf84e88ee9201
34806b1868548536b473752002408b27304a57ec
1462 F20101206_AABNLI levinbook_y_Page_051.txt
6a825f28aca5e8d424665439bb0868ee
afb8586d6c4113f5a9ca5ae75d2790e0cc62b333
115230 F20101206_AABNKT levinbook_y_Page_037.jpg
bc95e7241409714ab2d8af1004183cb9
F20101206_AABNLJ levinbook_y_Page_112.jp2
b9fd6b6d48e3545ab391355f3e41be36e286e31b
F20101206_AABNKU levinbook_y_Page_068.tif
065f15b4b585c3c62e25b2309973eea2
3b29514a247a4a2e6ec593eed94e456a78bae473
F20101206_AABNKV levinbook_y_Page_020.tif
834b4a62660ea8fe550f6c172d9c352d
e43e4e73f969717a10dab3b53436f36b11f622f3
944825 F20101206_AABNLK levinbook_y_Page_046.jp2
c9f595fcbabb9b7f08afefa5b5572691
5f3c6300a3504db9ae11f0a48779f11baff341b2
1924 F20101206_AABNKW levinbook_y_Page_096.txt
0eec24b08ef153bfe348f512aa53dd17
34a3858fbb65544e31d82c78237277d5885d4f12
47741 F20101206_AABNMA levinbook_y_Page_021.pro
f68202bda258df236cb96469c03b3fea
c7037ab1bcee417553328cd953a24aa00e2496ae
31794 F20101206_AABNLL levinbook_y_Page_084.QC.jpg
77900834d6b378a90780fc51118fe4a4
9359 F20101206_AABNKX levinbook_y_Page_093thm.jpg
945edcccc15f81dff9e361efd1cc383b70854932
63396 F20101206_AABNMB levinbook_y_Page_099.pro
040fea1dc50e082e0667cb209b82156d6917fd8d
114779 F20101206_AABNLM levinbook_y_Page_058.jpg
36b6a3dbdae366017bdfee0e9c80af49
e7c54bf086d540440d7e96d5bbf830a58d7ddae7
2608 F20101206_AABNKY levinbook_y_Page_111.txt
0fd8a8b04aa7df2e32aee4f5d62159e8
201cfb53af038415107d28e62b81e07eb773be73
F20101206_AABNMC levinbook_y_Page_106.tif
3387978c3cb0cfdc0277393cd4db5236
2a99586a3d08ea74d2e5a16eee2f6e78c063d7e2
109488 F20101206_AABNLN levinbook_y_Page_069.jpg
ece8a74c620e0b71e8a4330dcce115ea
30445 F20101206_AABNKZ levinbook_y_Page_021.QC.jpg
38e2ac7b9d123eebe23cd4f802766624
37088 F20101206_AABNMD levinbook_y_Page_058.QC.jpg
4e75ca6e4c34ac22d5da372faf11432b36d9cb6d
2168 F20101206_AABNLO levinbook_y_Page_057.txt
7e18564ba9af74c14986852f62b68289
81346f38df9beca9d110872c9c98d04ce3824ffe
33346 F20101206_AABNLP levinbook_y_Page_061.pro
26c8c2c560bddc150483abae2033b28c
bb8998f4333a8cec4d1c6a8ae7567c08112b1c56
2048 F20101206_AABNME levinbook_y_Page_082.txt
114476c691d3e928a9b550e1d0ec4106
11494 F20101206_AABNLQ levinbook_y_Page_104.QC.jpg
1e29e987b875d8b5ac3076f18813666a
bb39a3b9092e7f15ea8bf1ae0289a50eff973f5a
44919 F20101206_AABNMF levinbook_y_Page_043.pro
319409ceb8420acf2debee9b2d723ff7
F20101206_AABNLR levinbook_y_Page_055.jp2
cba2e2022fd156b4585c416644362392
ecff45b7446aa88a2ee4df17ebe1c03a21656e23
98293 F20101206_AABNMG levinbook_y_Page_067.jpg
7bc5c331192106de8a1e5af117af3e809567d721
132141 F20101206_AABNLS levinbook_y_Page_026.jpg
b7002d3eddcae5ab6bdb6ef4d1350857
8b79a2d22f5195d39e255b7e7e1283bb1f1e9c51
6265 F20101206_AABNMH levinbook_y_Page_008.QC.jpg
d858f88d386ac0c261ff4dd86bdd806e3d31da0b
24765 F20101206_AABNLT levinbook_y_Page_023.jp2
d13ddcae2c08ac6c228c6a80d0bd0629
2261 F20101206_AABNMI levinbook_y_Page_020.txt
47296be7fd591636a09761d7f8062491
2ff9048024a8a7a582f2d48a0525706469973766
F20101206_AABNLU levinbook_y_Page_076.tif
6abc29eea048c731393a13c8dec05706
39837 F20101206_AABNMJ levinbook_y_Page_042.QC.jpg
294179f8349c6f225c037b7e409f8ac6
F20101206_AABNLV levinbook_y_Page_036.tif
7bbc3484d96fcc934fd91835de4467e7
c767876c84e5f7865f7aab6398acdc9da4fabf3e
89654 F20101206_AABNMK levinbook_y_Page_089.jpg
f4a6f505ea26561cef8718fffdc9aa61
6275c092e615427d9d12d073f27bb19d7c44e5fd
2623 F20101206_AABNLW levinbook_y_Page_026.txt
c901931962d0bbb83c81e518a0fd038c3474247f
F20101206_AABNLX levinbook_y_Page_034.tif
f37619f58c478335d6a55ed20f3103c4
d6c50effa7d871e1bc8bbdcbc1c478901f73a556
53563 F20101206_AABNNA levinbook_y_Page_031.pro
938bbe0d2d63a3b29cf4203b8088092d
ff0c4576f75082198236cee4f2fe06da8dabc5c7
35632 F20101206_AABNML levinbook_y_Page_111.QC.jpg
a129b9628df5aa73028ec24957a48afc
23db19938fd701f2ed350c1b0c39a19236890472
37346 F20101206_AABNLY levinbook_y_Page_110.jpg
5eb5c962d5317847552601e0b7f8db5b
aefb4830b4ec7f00dc73c7b18f123a629e858096
40200 F20101206_AABNNB levinbook_y_Page_085.pro
50b2d0bb5d5f915bb70ce86f49c1bb4c
db2e1c0aa067346485e1fa1a19bee579de269941
F20101206_AABNMM levinbook_y_Page_033.jp2
2c4af54d0395bbcef488dbbe62b39abb
1c4df9739caa1f3f2d412fec303cb0cc8a9e2acf
8035 F20101206_AABNLZ levinbook_y_Page_039thm.jpg
a28f4153749218ea395d5339a20e1134
07432463f4d119805f274acdde32eef51886553a
8946 F20101206_AABNNC levinbook_y_Page_013thm.jpg
5910e6f8df2e0262e73991f93cb988c2
da170c80da0dea9e65756e1c7d146e63097a63fd
1521 F20101206_AABNMN levinbook_y_Page_089.txt
1eef188bb2d23146ac2b8fb485d1929e
596ed6dc6c16df37f0b86e9484b75b10a56a2f0b
F20101206_AABNND levinbook_y_Page_003.tif
a846527d52940b1e61009c331605f47a
F20101206_AABNMO levinbook_y_Page_029.tif
a1bf2353230f46907ee14e815aa3113b
8392ab570a756e0b635aba1cd31d430d97c3dd1c
F20101206_AABNNE levinbook_y_Page_107.tif
008f2fda1a511f2abcb1bfa1da784d31
538afeb292be491497296f2de75805d084c984d2
130246 F20101206_AABNMP levinbook_y_Page_015.jpg
1937d70036134ec4ae34d179dbef7b8d
7286 F20101206_AABNNF levinbook_y_Page_001.QC.jpg
cc80ba0ee5b50282230f9ecaff2c968b
ded2dc298ccc7fea8f1996bd89a7926dece860b0
1051946 F20101206_AABNMQ levinbook_y_Page_058.jp2
1f74a8b84e0bd26043c1db9ec32b1f9e
5f88aa5fb0ae717bc8392101320e4af7372fcb70
F20101206_AABNNG levinbook_y_Page_012.tif
95579cb73cdc22f2fc8f263b39716fdb
afde91d761bb0450d8e44d9c6bcaf2e92a055d56
8614 F20101206_AABNMR levinbook_y_Page_036thm.jpg
61d97f96bd1984b737cb5250b3526ab6
0a78ac687ec9e420c0aaaafb8457afa97e22c45d
1865 F20101206_AABNNH levinbook_y_Page_043.txt
59d60a04d0c41e6f8631d793755ec105
d00044419d080c1e4cb72c8320f54fe4e1f7c233
27881 F20101206_AABNMS levinbook_y_Page_060.QC.jpg
efc2e215837a224208c2f5488f9e2ae0
0257f22c58674ee28ed8205f4d9346632858cdae
54644 F20101206_AABNNI levinbook_y_Page_032.pro
a03476bce06cae8d8a85b4624e6478df
88605a3591a1585cfd3d680dccf2566fdbb605b6
122337 F20101206_AABNMT levinbook_y_Page_017.jp2
15b5d778b377f7e26e28c21032ced333
58eb47fbe8ebacdb66a985b970649079163a34a7
9322 F20101206_AABNNJ levinbook_y_Page_074thm.jpg
485b172b9fa800d5617745dc00e6ca72
4c7070cb45f93e887ffdcb36593ea605a6be30f5
26899 F20101206_AABNMU levinbook_y_Page_113.QC.jpg
84f0f2502446623882dedfa0b2a61df3
fcf03be40fbda124cf72de9bc8cca162c1689e4d
123764 F20101206_AABNNK levinbook_y_Page_083.jpg
29e4261fa8350c79661425bc47fecc0f
bbcfca75a1567e89bc8c7bb3a46b6ce852e9433c
1686 F20101206_AABNMV levinbook_y_Page_113.txt
F20101206_AABNNL levinbook_y_Page_053.tif
ddb42e66922111cdfd52c5f9477c6c37
dfcc792d937a74afc9e437e046de2fb8944abe2a
122324 F20101206_AABNMW levinbook_y_Page_099.jpg
f71670a63d61f69036e4ee9e1b7183d1
616d9c758052d7ed38f830bbcdafcd572601efc5
F20101206_AABNOA levinbook_y_Page_011.tif
4e742f8f66d0db4b87fdb262eea9484d
440675562236be5279664432b76752a034469d5e
118518 F20101206_AABNMX levinbook_y_Page_056.jpg
0fa18dc52d631d05bd93a9bc283383c8
3fe98ff04f0b8ca2e5c3c776857dc450033079e8
41177 F20101206_AABNOB levinbook_y_Page_015.QC.jpg
c10c31b0b76a01b722dea3578be20824
ece49789e1a56433c125fa360b77eb9f6bf65aa6
38898 F20101206_AABNNM levinbook_y_Page_062.QC.jpg
ef0bcc8b8274f41de1c59745cf49b554
1051933 F20101206_AABNMY levinbook_y_Page_049.jp2
75efac9dc6ed08465e7985f9871c0da79c966f4f
56472 F20101206_AABNOC levinbook_y_Page_065.pro
d9a4054ab53037d21190e69a96ef14de
2aced42dff2e4a361f9635c6ed4d0809423a5afe
36494 F20101206_AABNNN levinbook_y_Page_049.QC.jpg
b36363a713b14d5245886eaec301df54
dee55a50924aa6cffb030671d8bc5833d6c3fe9c
11482 F20101206_AABNMZ levinbook_y_Page_105.QC.jpg
1e2df62b0e44887c9f185bf33b058de5
66518ebd11a9063cba3ca723917bd6981c74edab
F20101206_AABNOD levinbook_y_Page_084.tif
89d0c127baa3f1ef965e5d16455b3fda
f7ed679afb535a57c45e774f86eaacfeb7fb56b5
8967 F20101206_AABNNO levinbook_y_Page_037thm.jpg
34caa519830779f627a611970a2456a2
97255be4611eacd7f941b1a25edaf7ef57f843fd
F20101206_AABNOE levinbook_y_Page_041.tif
dd3f2d3a0be0181b59a3a9a959d05ac5
b2a45cdb0281dfba0140f1464a0d21d80b066f10
9070 F20101206_AABNNP levinbook_y_Page_101thm.jpg
417f080d851841e824bbc7acac3dcc2d
122d8dc25bcf8464ea454e40bd9b0a6a250d3e6b
112093 F20101206_AABNOF levinbook_y_Page_065.jpg
7fbaa4a8b630f46d5070db8c3f40149f
2acb6812a213baa7a3801940d9235eb68b4084cd
F20101206_AABNNQ levinbook_y_Page_042.txt
30420f2d0a1ef18de578d6ea6e5d4eee
da604e5c7b5e9338e4b68e6f9570c4a37e57fe26
108176 F20101206_AABNOG levinbook_y_Page_020.jpg
5a4494287370f6fb094becd579362a6a
14012b9162a875d01d0d6ceecc483f5c15aae3d8
23889 F20101206_AABNNR levinbook_y_Page_001.jpg
ee44b19faea401c218aa4a71060f49b18512b07d
47998 F20101206_AABNOH levinbook_y_Page_078.pro
ef9532d2b9a8020f67729db89557c900
34c071eae8b0fa8e71b3ce1c02a190607ff99bd1
1975 F20101206_AABNNS levinbook_y_Page_078.txt
e67a57a67f2edd51b4a0b9ce15d2a075
b7a108038c23ca8de8e4e8c871d992b6dd84154b
20264 F20101206_AABNOI levinbook_y_Page_109.pro
6bb2e5e21c40e569de84fbd72bdfa77b
c535afefe2a29ce0bd50bf9122af6dda68e8c376
9216 F20101206_AABNNT levinbook_y_Page_062thm.jpg
4f9eb72a2c77b701ee2916d1070b3339
36f72d3262663e86c6f310a7039397c14a983c5b
38319 F20101206_AABNOJ levinbook_y_Page_066.QC.jpg
3e36354049303a226b13b11904591477
79127172512b9149084b27ff9ee5dd2a5cc0a085
110625 F20101206_AABNNU levinbook_y_Page_057.jpg
f3ed3fe0feebcbc47dd4b2ae1fcc1318
3521e667ec7da519253c849e99255c5df627d8ae
F20101206_AABNOK levinbook_y_Page_021.tif
478be6d254e10215b599a73d2d7df0db
f736c243539b7c2c081eeefa93b96fafd80a98f5
40010 F20101206_AABNNV levinbook_y_Page_041.QC.jpg
771cafeef7d9f42f991f619d11030a107d7ece16
1644 F20101206_AABNOL levinbook_y_Page_010thm.jpg
d26b92ac5ffc64130a2a6e7e849c905f
879bd2af1cd3d0918c0674fc285ddc484841edb4
F20101206_AABNNW levinbook_y_Page_039.jp2
2258e2b954da8483bc8b4d94b36e6255
28c18ffeb64973270e317b8bdc1fdb142e31f044
118635 F20101206_AABNOM levinbook_y_Page_035.jpg
d74c9678e9be940607bed09492c056bc
e8edb352d71a12c5fa1b3c2563ed809910d35c57
35810 F20101206_AABNNX levinbook_y_Page_094.pro
81ecf5455e115749e07ee3d80265dcf0
742472cb96b2cd33775068664f39084c13b0fdb9
402394 F20101206_AABNPA levinbook_y_Page_105.jp2
eed3759c586a448283d47279b9b99712
af4347e9a3e8fe84d5014e5264497006609bcab0
2510 F20101206_AABNNY levinbook_y_Page_114thm.jpg
e4b9348cfde10061e123bc296c35cbf3
559ff8c0dd1daf415f6a180d58daa216e1a58096
47450 F20101206_AABNPB levinbook_y_Page_084.pro
7f523e43ca46a9893962e4f6ddfcf452
b6444f3e8b4857159be80d97a6e3a6f40cc6c7bb
8633 F20101206_AABNON levinbook_y_Page_040thm.jpg
dcafc3c7af867d471c6ec25a521cb44c
1038691 F20101206_AABNNZ levinbook_y_Page_059.jp2
7380020a7a06a26af3250b4d15ed8e11a62a561b
623 F20101206_AABNPC levinbook_y_Page_103.txt
fec15902db3d6b75bb8e7d7e6597ce6f
aa0b8e61f1d31b7368705ca2bf420e58dd32cb97
F20101206_AABNOO levinbook_y_Page_075.jp2
b3ae1bf0eeb09b99262898d0302c7995
b7fac4368f23e669547fdb33d5e9c45e30f9cdc8
124300 F20101206_AABNPD levinbook_y_Page_054.jpg
11e7c44afb1d73012586d31808f063f0
6774eaf3b8a73fedc39785c694868948c04ae186
86878 F20101206_AABNOP levinbook_y_Page_072.jp2
8e954fc3a1d93df2abeb9caed0961880
907f904bd641964856791983143cac8a266521b4
19026 F20101206_AABNPE levinbook_y_Page_008.jpg
ed837275929ba68776089f2e56d3d5a3
cebd8b854b626d44238b8333356662a5f235700c
6158 F20101206_AABNOQ levinbook_y_Page_047thm.jpg
cea861ddb7d467ebbd2e6fc77acf7ae6
4975f8758e130a5b19688e6a715e7bd604b88963
760863 F20101206_AABNPF levinbook_y.pdf
d61fd4de8d0dcb4398339e3122f95c32
aeb9d97361a434876ce2ab37f0fb2728af5777d3
8835 F20101206_AABNOR levinbook_y_Page_057thm.jpg
d8172611a41a3df4e41a7c96945e3d17
96f4fb78174138e12f4ddecc9e57797986bb377d
28898 F20101206_AABNPG levinbook_y_Page_085.QC.jpg
c44c253aa4281a7d146dc75728dbe90b
2138f5c821e4698a88f14bb17175b76403bf5e7b
7919 F20101206_AABNOS levinbook_y_Page_028thm.jpg
4013647847a9b58354cefcea4ea72665
5e11b261357f4176e84681ed8944b2e425da4275
1502 F20101206_AABNPH levinbook_y_Page_002.QC.jpg
3a0ef690f8908d47db4facb05f64eb47
3b79529dd848dfb343432799653510a3c6bb9a6e
F20101206_AABNOT levinbook_y_Page_103.tif
19aea22a786bfd1c0e01a1aa14604aed02e9df57
4830 F20101206_AABNPI levinbook_y_Page_109thm.jpg
7fedbe3fdeac5f2c312b1bc0f07934db
e2a31c8ee31facf4ea216703a8fe379f1680b430
132921 F20101206_AABNOU levinbook_y_Page_102.jpg
36aa23d54e7e9b9a08c51992d9fbc15b
f1b8cc5d3970dcd649b09ec85b9eb56a6a8d6b45
2518 F20101206_AABNPJ levinbook_y_Page_041.txt
a375ab3b26edd5c86eab28757e488b86
56441f32e3c350d3e6d5a7918c7535e5dd710662
1203 F20101206_AABNOV levinbook_y_Page_104.txt
2452c4d0f31eda78a69e44366c71dff0
5fd7c1f38a11c0a50a1f0253002f6998f84e883f
F20101206_AABNPK levinbook_y_Page_045.tif
83fb152a02806545a362cb8c7bcdbbbe
fa2cf969188bc52f9c7b8ef6eb8a99c911dc85bc
109035 F20101206_AABNOW levinbook_y_Page_068.jpg
9718a6312130ed990324bc805465a5ea
d994d83107b12d4b01daa642df1feeb6476f8c26
86390 F20101206_AABNPL levinbook_y_Page_097.jpg
2f5bc936978585f24797fb7b4710f5de
a82b022ff295e957b1686bff494ab03a36ee9c5c
56659 F20101206_AABNOX levinbook_y_Page_037.pro
9f99ac6b2ca4d275796d12cb25ef2edb
d0fd66114698c3ffd04ee32f332d16dc71b20425
2364 F20101206_AABNQA levinbook_y_Page_035.txt
0af8f320a74e8592eaa62939c7f951fb
3af5e03a8894f1b0132a70cc7e011f82e5042e48
59268 F20101206_AABNPM levinbook_y_Page_066.pro
eaefbe35347c1398c72f65034ff4a6b0
6fb67b253a95d757eea0520ed8f5484837d24d75
8589 F20101206_AABNOY levinbook_y_Page_058thm.jpg
088d4b8cee4dd33fde8a82891d5832bd9ee5576e
4921 F20101206_AABNQB levinbook_y_Page_004thm.jpg
8586d1b1cf4dba8cbaf5d391fae1d979
970182 F20101206_AABNPN levinbook_y_Page_096.jp2
5cb1698b9c2cea7f99c242d4b47169889026a2ef
8936 F20101206_AABNOZ levinbook_y_Page_017thm.jpg
67088b365e9ca75c3df25f39384fef9c
3163045a09a4b5b8b6cff7508c60f34f68cbb23d
361668 F20101206_AABNQC levinbook_y_Page_016.jp2
878898be025b6e1ecc28d64e5b30e806
F20101206_AABNQD levinbook_y_Page_086.jp2
ca3a8179e5efda862f0e653690e33f52
7e52740e234823f93080b27efe86ed8251b0f8cd
50774 F20101206_AABNPO levinbook_y_Page_109.jpg
c9ab339197b8ed8d39f3ebc519aacbb5
cec9052719f63ddffa9376b2f978447257a73e60
2554 F20101206_AABNQE levinbook_y_Page_112.txt
bedcc85628974c14e7392f94136c0561e79b87a5
F20101206_AABNPP levinbook_y_Page_015.tif
7681 F20101206_AABNQF levinbook_y_Page_088thm.jpg
bbb63ac395c4f31656b47e3388a66109
cf69ab66b489e2f121dd5fec0f623d0f615ff850
107053 F20101206_AABNPQ levinbook_y_Page_073.jpg
175ee7e15d8c8643983f8fc38e3f203a
e1f7cb26f6cc67a4f12dc2677e8c0fbaccfc1169
3940 F20101206_AABNQG levinbook_y_Page_007thm.jpg
534463aef7189b27de1fcaf0a3f8813b
ebf37590e73022b799aa8c9e7178827bf180fb8d
F20101206_AABNPR levinbook_y_Page_099.tif
ea30407110cfcb43ff50141fb519d2ff
9b278b2e86c0cc16f35c5c6602ae2a302a9835da
18480 F20101206_AABNQH levinbook_y_Page_018.QC.jpg
6b847ab6f7d3bf56331a498f242d9e53
8ef57709b1b3940be4a77836bcb280097cc08414
1929 F20101206_AABNPS levinbook_y_Page_046.txt
2602f6367055afed7596c60f4e3db664
6b06978a268d806959f957a37fa0a6babf8b798d
126950 F20101206_AABNQI levinbook_y_Page_070.jpg
43e4e9ff6a1035be40bb47694c602b1d
5bf340f28beeaa2af2d53cbfe441d47ff289bf44
37321 F20101206_AABNPT levinbook_y_Page_092.QC.jpg
109c84c60ecd3d4b951de9c66f1f8543
4d6ee818b2915db04dc3d1b72269a53b8d093dbc
93016 F20101206_AABNQJ levinbook_y_Page_024.jpg
c0fa7e7777f31e7c9be51fbb9cd7d89d
312f26c54abd98455cab12847cb5f05b31bbb381
10423 F20101206_AABNPU levinbook_y_Page_105.pro
a0fcb3b8abe0b995405f3b09fc0c7245
d943f75044eeaba5dd07786e6f228fea3974904d
7445 F20101206_AABNQK levinbook_y_Page_001.pro
25ed0b3579b524f351600e1dd67917ca
3f710c528426f0a1d03e97f67981d4865d5b4c6b
131495 F20101206_AABNPV levinbook_y_Page_111.jp2
de8c7b661cec924cee93f8831b1463e7
0e5087549d2baf1342159c67c7c77b2f6f1fffe4

STATE ESTIMATION: A DECISION THEORETIC APPROACH

By
YOAV N. LEVINBOOK

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2007

@ 2007 Yoav N. Levinbook

To the memory of my father, Benjamin Levinbook

ACKNOWLEDGMENTS

I would like to thank Professor Tan F. Wong, my advisor and the chair of the supervisory

committee, for his guidance, useful advice, and, in particular, for the freedom and encouragement

he gave me for pursuing my own research interests. I feel that I significantly evolved as an

electrical engineer in the four years I worked closely with him. I have no doubt that without his

help I would not have completed this work.

I wish to take this opportunity to thank all the members of the supervisory committee for

I would also like to thank Professor Paul Robinson and Professor Sergei Shabanov from the

Department of Mathematics, who were my instructors in several courses. The knowledge I have

gained from them proved to be very valuable for this work.

I would like to express my deepest gratitude to my beloved mother and late father, which

have always supported me and encouraged me. I hope I lived up to their expectations.

Finally, I am indebted to my dear wife, Eliane, for her support, encouragement, and

patience. Without her, I could not have confronted all the difficulties of the last four years.

page

ACKNOWLEDGMENTS ......... . . .. 4

LISTOFFIGURES ............. .............. 7

LIST OF ABBREVIATIONS ......... . .. .. 8

ABSTRACT.............. ......... ...... 9

CHAPTER

1 INTRODUCTION ......... ... .. 11

2 GENERAL NOTATION AND CONVENTIONS .... .... .. 17

3 DECISION THEORETIC FORMULATION ... .. . .. 19

4 GENERAL DECISION THEORETIC RESULTS ... .. .. 24

5 THE CASE THAT THE RISK IS SPECIFIED BY A LOSS FUNCTION .. .. .. 34

6 THE CASE OF CONVEX LOSS FUNCTION .... .... .. 40

7 FINDING A MINIMAX ESTIMATOR AND THE DUAL PROBLEM .. .. .. .. 49

8 APPROXIMATING A MINIMAX ESTIMATOR .... .... .. 52

9 THE RESTRICTED RISK BAYES PROBLEM AS A MINIMAX PROBLEM .. 61

10 ESTIMATION WITH A RESTRICTION ON THE OBSERVATIONS THAT CAN
BEUSED ............. ............... 66

11 THE STATE ESTIMATION PROBLEM . ... .. 73

12 AFFINE STATE ESTIMATION BASED ON QUADRATIC LOSS FUNCTIONS .. 77

12.1 Finding a Restricted Risk Bayes Solution ... .... .. 77
12.2 Finding a Maximizer of the Risk . ... .. .. .. 84
12.3 Connection to the Kalman Filter and E-Minimax Approach . . 87
12.4 Numerical Example . .. .... .. 90

13 STATE ESTIMATION WITH INITIAL STATE UNCERTAINTY .. . 96

13.1 Conditional Mean Estimators ....... .. .. 97
13.2 Approximations to Minimax Estimators ..... .... .. 98
13.3 Numerical Example ... . ..... .. .00

14 CONCLUSIONS ......... . ... .. 106

APPENDIX

A PROOF OFLEMMA 5.2 . . . 1..07

B PROOFOFLEMMA9.1 ............. ...........109

C PROOF OFLEMMA 9.2 . . .. .... .10

REFERENCES . .. . ... .111

BIOGRAPHICAL SKETCH ......... ... .. .. 114

LIST OF FIGURES

Figure page

12-1 Achieved Bayes risk vs. maximum risk . .... .. .. 94

12-2 The maximum risk over O, of the Bayes, minimax, and restricted risk Bayes solu-
tionsvs.e ............. ............... 95

13-1 A full view of 31o', which is a finite (6, V)-dense subset of 31o .. .. .. .. .. .. 103

13-2 A zoom-in view of the bottom left comer of 31o', which is a finite (6, V)-dense subset
of31o. ........ ... ......... .......104

13-3 The risk of .i-(To), the derived -,-optimal estimator, as a function of .ro E 31o' .. .. 104

13-4 The a priori distribution -ro, which is defined on 31o' .. .. .. .. .. 105

13-5 The maximum risk of the Kalman Filter initialized with zero mean and covariance
O.2I as a function of o . ..... .. 105

LIST OF ABBREVIATIONS

CM: conditional mean ......... . ... .. 97

KF: Kalman Filter ......... . .. .. 11

LMMSE: linear minimum mean squared error . ... .. 11

MSE: mean squared error ......... . . .. 45

Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

STATE ESTIMATION: A DECISION THEORETIC APPROACH

By

Yoav N. Levinbook

August 2007

Chair: Tan F. Wong
Major: Electrical and Computer Engineering

The problem of state estimation with stochastic and deterministic (set membership)

uncertainties in the initial state, model noise, and measurement noise is approached from

a statistical decision theory point of view. The problem is initially treated within a general

framework in which the state estimation problem is a special case. General existence results such

as the existence of a minimax estimator and a least favorable a priori distribution are derived

for the state estimation problem. Then, attention is restricted to two important cases of the state

estimation problem. In the first case uncertainties in the initial state, model noise, and observation

noise are considered. It is assumed that the a priori distributions of the initial state and the noises

are not perfectly known, but that some a priori information may be available. The restricted

risk Bayes approach, which incorporates the available a priori information, is adopted. When

attention is restricted to affine estimators based on a quadratic loss function, a systematic method

to derive restricted risk Bayes solutions is proposed. When the filtering problem is considered,

the restricted risk Bayes approach provides us with a robust method to calibrate the Kalman filter,

considering the presence of stochastic uncertainties. This method is illustrated with an example

in which Bayes, minimax, and restricted risk Bayes solutions are derived and their performance

is compared. In the second case only the initial state uncertainty is considered. The initial state is

regarded as deterministic and unknown. It is only known that the initial state vector belongs to a

specified parameter set. The (frequentist) risk is considered as the performance measure and the

minimax approach is adopted. The search of estimators is done within the class of all estimators.

If the parameter set is bounded, a method of finding estimators whose maximum risk is arbitrarily

close to that of a minimax estimator is provided. This method is illustrated with an example in

which an estimator whose maximum risk is at most 3%b larger than that of a minimax estimator is

derived.

CHAPTER 1
INTRODUCTION

The problem of state estimation for linear dynamical systems has received considerable

attention in signal processing, controls, communications, econometrics, and a wealth of other

fields. The usual formulation of the problem assumes that the initial state, model noise, and

measurement noise are random vectors with perfectly known a priori distribution or at least with

known covariance and mean. It is well known that if these assumptions, together with other usual

assumptions regarding the noises, are valid, the Kalman filter (KF) [1] is the linear minimum

mean squared error (LMMSE) estimator. If in addition all the stochastic quantities are Gaussian,

the KF is the minimum mean squared error estimator.

Since the assumption of complete knowledge of the a priori distribution is seldom satisfied,

a Bayesian approach is used in practice. The a priori distributions of the initial state, model

noise, and measurement noise are learned from past experience and used as approximations of

the corresponding true distributions. Nevertheless, even if extensive past experience is available,

the estimated distributions may still deviate from the true ones. The effect of such errors in the

a priori information on the performance of the KF is studied in [2]-[4]. The effect of the errors

in the a priori information of the initial state, model noise, and measurement noise may be very

significant and a KF updated based on erroneous a priori information may perform poorly. Thus

it is necessary to consider other approaches that are robust against uncertainties in the a priori

distribution of the initial state, model noise, and measurement noise.

The state estimation literature deals extensively with the general problem of linear systems

with stochastic or deterministic uncertainties using game theory and the minimax approach (cf.

[5]-[14] and the references therein). Usually the so-called E-minimax approach is adopted. The

F-minimax approach [15] regards the parameter as random with its distribution lies in a class F.

However, the exact distribution in the class is unknown. A F-minimax estimator is an estimator

that minimizes the supremum of the Bayes risk, where the supremum is taken over all elements

of F. When the F-minimax approach is used, the class of available estimators is usually restricted

to linear or affine estimators. As a result, an element of the class 0 is specified by a first-order

statistic (mean) and second-order statistic covariancee) pair.

There are several other approaches in statistical decision theory that seem suitable in this

context. Among the most prominent approaches is the restricted risk Bayes approach. The

restricted risk Bayes approach, proposed by Hodges and Lehmann [16], is a compromise between

the Bayes approach and the minimax approach. A restricted risk Bayes estimator minimizes

the Bayes risk with respect to an a priori distribution suggested based on some past experience

subject to the restriction that the maximum risk does not exceed the minimax risk by more than a

given amount. This approach utilizes available a priori information but at the same time provides

a safeguard in case this information is not accurate. If the a priori information is fairly accurate,

a restricted risk Bayes estimator has good Bayes risk properties. Other work considering the

restricted risk Bayes approach or closely related approaches include [17]-[20]. Despite the

appealing formulation of the restricted risk Bayes approach, it has not been utilized in the context

of state estimation.

Although the problem of state estimation with stochastic uncertainties has been approached

from the F-minimax approach, we believe that approaching this problem from the restricted risk

Bayes approach also has a considerable merit. If a state estimation problem can be regarded as

a zero-sum two-person game (henceforth to be referred as a game) against a rational opponent,

then the F-minimax approach seems very attractive. However, in most applications, if we regard

the state estimation problem as a game, the game is against Nature. Using the F-minimax

approach in this case corresponds to a very pessimistic viewpoint that regards Nature as a rational

opponent that wishes to cause us the largest possible loss. The F-minimax approach may still

seem reasonable in the case that there is no a priori information that enables us to regard certain

distributions in C as more likely than others. However, in many applications, some a priori

information regarding the true distribution may be available. This a priori information may be in

the form of a nominal distribution, which is suggested based on some past experience. It is well

known that under certain conditions, a F-minimax estimator is a KF relative to a least favorable a

priori distribution. If our a priori information suggests that the true distribution is very different

from a least favorable a priori distribution, then using the F-minimax approach may result in

very conservative estimators. The restricted risk Bayes approach enables us to adopt a less

conservative but still rather robust approach, which gives us a guaranteed safeguard in terms of

the risk. We assume the nominal distribution is an approximation of the true a priori distribution

and search for an estimator that minimizes the Bayes risk relative to the nominal distribution

subject to the constraint that the Bayes risk relative to any distribution in 0 is less than a given

value. We can determine this value based on the amount of past experience that we have. The

stronger the available past experience, the more we can trust the approximated distribution and

the larger the value we can allow.

In this work, we consider the restricted risk Bayes approach. Under our formulation, the

F-minimax approach is a special case. We consider the risk, based on quadratic loss functions,

as our performance measure. In this case, we restrict ourselves to affine estimators in order

to derive estimators that are attractive in terms of computational complexity. We provide a

systematic method for solving restricted risk Bayes and E-minimax solutions. In some important

cases, this method can be easily used to calibrate the KF, considering the presence of stochastic

uncertainties.

While in most applications, the observation noise can be indeed modeled as random, or

even Gaussian (e.g., the thermal noise in communications systems), in some applications one

can argue whether the initial state can be better modeled as an unknown constant, rather than as

random, and the same may be also argued for the model noise. In this work, we also consider the

case that there is a deterministic uncertainty in the initial state as a special case of the problem

of state estimation with deterministic uncertainties. The distributions of the model noise and

measurement noise are assumed known. As mentioned above, it may be reasonable to assume

the model noise is deterministic and unknown as well. We do not pursue this approach here

since we believe that this would obscure the results that we derive for the initial state uncertainty

due to the technical difficulties of considering deterministic uncertainties in the model noise.

In addition, if we are concerned with estimation problems in which it is necessary to estimate

fast, using a small number of samples, the uncertainty in the initial state may a have a much

more significant effect than the uncertainty in the model noise. The reason is that if the estimated

signal changes slowly (which is the case in many applications), the uncertainty in the model

noise may be small whereas the uncertainty in the initial state may be very large. For instance,

in tracking problems the changes of the velocity or acceleration of the target between adjacent

samples are usually small relative to the range of their possible values at the initial time. When

deterministic uncertainty is considered in the initial state, the minimax approach may be the

preferable approach. A minimax estimator minimizes the supremum of the (frequentist) risk.

The minimax approach seems a reasonable approach when it is only known that the initial

state vector belongs to a certain parameter set. The minimax approach is especially suitable for

applications in which we may interpret the estimation problem as a game against an opponent.

This is certainly the case in many military applications.

The state estimation literature has traditionally focused on the Bayesian approach and the

F-minimax approach when dealing with uncertainty in the initial state vector. The classical

approach that regards the initial state vector as deterministic, has been mostly ignored. This is

despite the fact that in many important applications (see for example [21] and [22]), it seems

more reasonable to model the initial state vector as deterministic and unknown. One example that

falls within the classical estimation framework is Danyang and Xuanhuang [23], where the state

estimation problem was considered from a least squares viewpoint and the best linear unbiased

estimator was derived. In [24], the authors consider the problem of state estimation with initial

state uncertainty from a decision theoretic point of view. The initial state vector is regarded as

deterministic and unknown. It is only known that the initial state vector belongs to a parameter

set. The risk, based on quadratic loss functions, is considered as the performance measure. The

search for a minimax estimator is done within the class of all possible estimators. Minimax

estimators are derived for the case of unbounded parameter set and approximations of minimax

estimators are derived for the case of bounded parameter set. In this work, we will repeat some

of the results in [24] that deal with the bounded parameter set case in order to illustrate how

minimax estimators can be approximated with arbitrarily prescribed accuracy.

While we are mainly interested in the state estimation problem, a large part of this work will

be concerned with a more general estimation problem. In fact, some of the existence results such

as the existence of a minimax estimator and a restricted risk Bayes solution hold in a very general

setting and may have applicability not only in the state estimation problem.

The rest of this work is organized as follows. In Chapter 2, we present notation and

conventions that are used throughout this work. In Chapter 3, we present a general decision

theoretic formulation that is needed in order to address the problem of state estimation with

stochastic and deterministic uncertainties and also to derive other, more general, results. In

Chapter 4, we derive several general decision theoretic results, which are based on well known

results from decision theory and game theory; the applicability of these results is not limited

only to the problem of state estimation with stochastic and deterministic uncertainties. In

Chapter 5, we consider the case that the risk function is specified by a loss function. We derive

rather weak conditions that guarantee the existence of a minimax estimator and a restricted risk

Bayes solution. In Chapter 6, we restrict ourselves to convex loss functions, in general, and the

quadratic loss function, in particular. In Chapter 7, we discuss how a minimax estimator can

be found by solving the dual problem of finding a least favorable a priori distribution. Since,

in general, finding a minimax estimator may be an extremely difficult task, in Chapter 8 we

discuss how one can derive approximations to minimax estimators, where the approximation

can be made as accurate as desired. In Chapter 9, we consider the general restricted risk Bayes

estimation problem and show that this problem is equivalent to a sequence of minimax estimation

problems. In some estimation problems there are restrictions on the observations that can be

used in order to estimate the parameters. This is the case in the state estimation problem in which

each state can be estimated using only certain observations. We consider this type of estimation

problems in Chapter 10. In Chapters 11, 12, and 13, we restrict ourselves to the problem of state

estimation with uncertainties in the initial state, model noise, and measurement noise, which

is the main problem considered in this work. In Chapter 11, we derive some general existence

results that are based on the results of the previous chapters. In Chapter 12, we consider the case

of stochastic uncertainties in the initial state, model noise, and measurement noise, and restrict

ourselves to affine estimators. We propose a method that can be easily used to derive a restricted

risk Bayes solution in many important cases. In Chapter 13, we consider the case of deterministic

initial state uncertainty, and search for estimators within the class of all estimators. We conclude

this work in Chapter 14.

CHAPTER 2
GENERAL NOTATION AND CONVENTIONS

Let R"N denote the N~-dimensional Euclidean space, and let RW = RIW. Let R"~ = {a e

R"N : a(i) > 0 for i = 1, 2, ..., N}. We use RWNxM to denote the space of NV-by-M~ real

matrices. Let S" denote the space of n-by-n real symmetric matrices. Let S" denote the cone

of positive semi-definite matrices in S". Let N = {0, 1, .. .}. Let a denote an NV dimensional

vector. We use a(i) to denote the ith element of a and |a| to denote the Euclidean norm of a.

We let ||a|| = CE |a(i)|. Let Ai and B denote arbitrary matrices. We use N~(A) to denote

the nullspace of A, At to denote the Moore-Penrose pseudo inverse of A, tr(A) to denote the

trace of A, and ||IA||2 to denote the 2-norm of A. We use A > 0 (A > 0) to denote that A

is positive definite (positive semi-definite) and symmetric and A > B (A > B) to denote

that A B > 0 (A B > 0). We use A 0 B to denote the Kronecker product of A and

B. Let A denote an arbitrary set. We use | A| to denote the cardinality of A. If AC R ", we

let A = { x E R"N : x~y = 0, for ally E A}.Let A1,. ,AS be arbitrary sets. Let

A = n0 Ali, where product on sets means cartesian product. We use vri : A Ai to denote

the projection onto the ith factor. Given a s A, let a(i) = xri(a). If Al,. ,AS are subsets of

vector spaces V1, .. respectively, a' and a" are in A, and as R I, then a = a' + a" means

that a(i) = a'(i) + a"(i) for i = 1,. ., n, and a = caa' means that a(i) = caa'(i) for i = 1, .., n.

Let f : X Y and let Z be a subset of X. Then f (Z) = {f (z) : z E Z}. Let lni denote the

NVx NV identity matrix. Let ONxM/ denote the NVx M~ zero matrix. When the dimensions of the

zero matrix and identity matrix are clear from the context, they will simply be denoted 0 and I,

respectively. For any topological space X, we use B(X) to denote the o--algebra of Borel subsets

of X. Given a measurable space (X, FT), where X is a subset of R"N and FT = B(X), and a

probability measure -r defined on (X, FT), by the mean vector and autocorrelation matrix of -r, we

mean the mean vector and autocorrelation matrix of a random vector that is distributed according

to -r, i.e., the mean vector of -r is fx xd-r(x) and the autocorrelation matrix of -r is fx xx~d-r(x).

Let S denote an index set, possibly uncountable, and (Xs, F,), for each s e S, be a measurable

space. Then nses Fs, denotes the o--algebra on nses X, generated by subsets of the form

nses A,, where A, E T, for each s ES and A, = X, for all but a finite number of s E S. In

particular, given measurable spaces (X1, Fi), (X2, F2,),. ., (XN, Fy), FI~x T~ 2 x FTN

denotes the o--algebra on X1 x X2 x x XN generated by subsets of the form Al x A2 x x AN,

where Ai E Fei for i = 1,. ., NV. Given a measurable space (X, FT), let m(FT) denote the set of all

real-valued measurable functions on (X, FT) and m(FT) denote the set of all extended real-valued

measurable functions on (X, FT). Given real numbers a and b, a Vb denotes max {a, b} and a Ab

denotes mina, b}. Some of the derivations in the sequel require the use of the extended real

number system. We use the usual conventions for arithmetic operations in the extended reals.

When we take the supremum or infimum of a subset AC R we regard A as a subset of the

extended reals, i.e., the supremum and infimum always exist and may take the values +oo and

-oo, respectively.

CHAPTER 3
DECISION THEORETIC FORMULATION

Let (R, &~, Q) be a probability space, where R is the sample space of an experiment and

elements of -TA are the events of the experiment. Suppose there are given a set 8, called the set of

possible states of nature, and a family P = { Po : 0 E 8}) of probability measures defined on the

measurable space (R, &~). The probability measure Q is unknown to the experimenter; however

it is known that Q is an element of P. Suppose there is given a measurable space (Y, FTy). The

experimenter observes the value y taken by a random element (or a Y-valued random variable )

Y : R (f FTy); this value is called an observation.

In Wald's interpretation [25] of the statistical decision problem as a zero sum two-person

game, Nature chooses an element 8 E 8, called the true state of nature. The experimenter needs

to reach a certain decision based on the observation y without knowing the true state of nature.

The experimenter reaches such a decision by choosing a decision rule from a class of decision

rules. Here we are mainly interested in estimation problems, although some of the results of this

work apply also to decision problems.

In classical estimation, the experimenter estimates the true state of nature 8 E 8 or more

generally a function of 0. In this case the space (R, -T2, Q) can be taken to be (Y, FTy, Q), where

Q is the distribution of Y. In this work, we are interested in a more general case. Suppose

there is given an additional measurable space (X, 6T) and a random element X : R

(X, 8Tx). Suppose Y and X have a joint distribution defined on the product measurable space

(y x X, FTy x 5T). Suppose the experimenter wants to estimate the value x taken by X. In this

case, the space (R, -T2, Q) can be taken to be the space (y x X, FTy x 8x, Q), where Q is the

joint distribution of Y and X and is an element of P. Note that the previous mentioned case can

be regarded as a special case, in which the distribution of X assigns mass 1 to the true state of

nature 8 E 8.

Suppose the experimenter is allowed to choose estimates for x from a class D. The class

D, equipped with a topology, is referred to as the space of possible estimates. Often D coincides

with X. However, it is convenient not to make this restriction. Given a topological space A, let

My/1 denote the class of all probability measures on (A, B(A)). A nonrandomized estimator i-

is simply a ~T-measurable mapping from Y into D. A mapping :i : Y M zD, y H 9(I)

is said to be a randomized estimator if y H -(D'| y) is a 8 y-measurable function for all

D' E B(D). A randomized estimator (from now on just an estimator) can be used to determine

uniquely a procedure for choosing an estimate for :r. The estimate of :r, given y is observed

and i- is used, is an element of D selected according to the probability measure i-(-|y). A

nonrandomized estimator i- can be regarded as a randomized estimator that assigns to each

ye E a Dirac measure, i.e., a measure that assigns a mass 1 to a single point in D. Let 2'

denote a class of estimators for :r. Since we adopt a decision theoretic approach, the merit of

an estimator will be judged based on a risk function. The risk function is the expected loss

incurred to the experimenter when using an estimator :i and 8 is the true state of Nature. Let

R : 8 x X R WU {+oo} denote a risk function. In the formulation of this work, an estimation

problem may be specified by a triplet (8, 2', R).

For the convenience of the reader, we present here some basic decision-theoretic definitions

taken from [26] and [25], with slight modifications. These definitions are given for an estimation

problem (8, 2 R).

Definition 1. An estimator :i* E 2' is said to be a nzinintax estimator if

sup R(8, i-*) inf sup R(0,;i).

Definition 2. Let e > 0. An estimator :i* E X is said to be an e-nzinintax estimator if

sup R(8, i-*) < inf sup R(8, i-) +
BOe *E~ aee

Definition 3. An estintator i-' is said to be as good as an estintator i-" if R(0, i-') < R(0, i-") for

all 8 E 8. An estintator i-' is said to be better than an estintator i-" if R(0,;i') < R(0,;i") for all

8 E 8 and R(0, i-') < R(0, i-") for at least one 8 E 8. An estintator i-' is said to be equivalent to

an estintator i-" if R(0, i-') = R(0, i-") for all 8 E 8.

Definition 4. An estimator & is said to be admissible if there exists no estimator better than 2. An

Definition 5. A class A of estimators is said to be essentially complete relative to a class B of

estimators if given any estimator & in B, there exists an estimator i* E A that is as good as 2.

If AC c is essentially complete relative to X, we simply say that A is essentially

complete.

Definition 6. An estimator & is said to be an essentially unique minimax estimator if any

minimax estimator is equivalent to 2.

Let V(8, X, R) = infecx supeoe R(0, 2) be the minimax risk. The following assumptions

will be assumed throughout this work, unless mentioned otherwise:

Assumption 3.1. V(8, K, R) < +oo.

The case V(8, .[, R) = +oo is not very interesting. Clearly in this case the minimax

approach is not suitable since any estimator is a minimax estimator.

Assumption 3.2. 8 is a metric space with metric a.

Assumption 3.3. The risk function R is nonnegative.

Assumption 3.3 can be weakened to the assumption that infeee int ,x R(0, 2) > -oo, but

for the sake of presentation it is advantageous to assume that R is nonnegative.

We need additional definitions. If & is such that R(-, 2) is bounded from below and is a

B(8)-measurable function, we denote the Bayes risk relative to -r Me Zl of &

~(~,i (3J8

In this work, we consider only estimation problems (8, X, R) for which R(-, 2) is bounded from

below and is a B(8)-measurable function for all ie E Hence the Bayes risk is always defined.

Definition 7. An estimator i* E X is a Bayes solution relative to -r Me Zl ifr (-, i*)=

infEx gr (-, 2). An estimator i* E X is a Bayes solution if it is a Bayes solution relative to some

distribution -r Ms/l.

Definition 8. An estimator & is said to be an essentially unique Bayes solution relative to -r if any

Bayes solution relative to -r is equivalent to x.

Note that we may regard a point 8 E 8 as an element of Me/l by regarding it as the

probability measure that assigns mass 1 to 0. Thus 8 may be considered as a subset of Mel. With

this viewpoint, r(-, 2) is an extension of R(-, ) from 8 to Me/l, i.e., r(0, 2) = R(0, 2) for all

S8.O

We also use the notion of Bayes solution in the wide sense:

Definition 9. Let {n }r be a sequence of a priori distributions in Me/l and i* be an estimator

Then i* is a Bayes solution relative to the sequence {n }) if

lim [r(ni, i*) -- inf r(i, 2)] = 0.

An estimator i* is a Bayes solution in the wide sense if there exists a sequence {nri } EZ/e such

that i* is a Bayes solution relative to the sequence {8 }).

It iseasy to show that for each re M 2e, inf ~xr-,~: I ~,2 V(8, K, R). Thus by

Assumption 3.1, infe,x r (-, 2) < +oo, and the term r (-r, i*) infa,x r (-r, 2), in the above

definition, is well defined.

Definition 10. An a priori distribution -ro E Me/l is said to be least favorable if

1111T T, Z Sup inf r 7r, x.

Definition 11. The estimation problem (8, I R) is said to be strictly determined if

inf sup r(-r,i)= sup inf r(-r, ).

For any class AC c of estimators, we let A(M~, R) = { E A : supose R(0, 2) < M}).

Definition 12. Let -r Me Zl and Co ERIWU {+oo }. An estimator io is said to be a restricted risk

Bayes solution relative to (-r, Co) if

T T,0 111 T T I

Note that if Qo = V(8, K, R), then for any -r Me Zl, a restricted risk Bayes solution

relative to (-r, Qo) is a minimax estimator. In addition, a restricted risk Bayes solution relative to

(-r, +oo) is a Bayes solution relative to -r. Thus the problems of finding a minimax estimator and

a Bayes solution may be regarded as two extreme cases of the problem of finding a restricted risk

Bayes solution.

CHAPTER 4
GENERAL DECISION THEORETIC RESULTS

In this section, we present several decision theoretic results for an estimation problem

(8, X2, R). These results hold in a much more general case than the estimation problem

considered in this work and may be used in general decision theoretic problems. These results are

based on well known results from decision theory and game theory.

First, we note that it is well known that (cf. [26, Exercise 2.2.1])

sup R (B, i) sup r (-r, .). (4-1)

Consider the space Me/l equipped with the topology of weak convergence [27, pp. 236]. The

topology of weak convergence makes Me/l a Hausdorff space.

Lemma 4.1. Suppose 8 is compact. Then

1. Me/l is nzetrizable and compact.

2. Let~i e E be such that R(0,.i-) < +oc for each Ie 8 and R(-,.i) is continuous on 8.

Then r (-, i) is continuous on M sl.

3. Let~i e E X be such that R(0,.i-) < +oc for each Ie 8 and R(-,.i) is upper senicontinu-

ous on 8. Then r (-, i) is upper senticontinuous on M sl.

Proof 1) By the hypothesis of the lemma, 8 is a compact metric space and hence is separable

(cf. [28, Exercise 2.25]). This implies that Me/l is metrizable [29, pp. 122]. In addition, [29,

Theorem 3.1.9] furnishes that Me/l is compact.

2) By [29, Theorem 3.1.5], J, fdvr is a continuous function of -r for any bounded and

continuous f. Since R(-,.i) is a continuous real-valued function on a compact set, it is bounded.

Hence r(-,.i) is continuous on Ms/l.

3) By [29, Theorem 3.1.5], J, fdvr is an upper semicontinuous function of -r for any up-

per semicontinuous f that is bounded from above. Since R(-,.i) is an upper semicontinuous

real-valued function on a compact set, it is bounded from above. Hence r(-,.i) is upper semicon-

tinuous on Me/l for any .i E C. O

Given a set Z, let & (Z) denote the space of nonnegative extended real-valued functions on

Z equipped with the topology of pointwise convergence. For u', u" E W (Z), we write u' < u"

to denote that u'(z) < u"(z) for all z E Z. Consider the space W (8). Let A denote an arbitrary

set and f : 8 x A [0, +oo] an arbitrary function. Then for each a s A, f (-, a) E W (8). Let

A' and A" denote arbitrary sets, and let f' : 8 x A' [0, +oo] and f" : 8 x A" [0, +oo]

denote arbitrary functions. It is convenient to use the notation (A', f') -4 (A", f") to denote that

for each element a" E A", there exists an element a' E ~A' such that f'(-, a') < f"(-, a"). Let

(A', f') ~ (A", f") denote that (~A', f') -4 (A", f") and (A", f") -4 (~A', f'). Clearly ~ is an

equivalence relation. Now, consider an estimation problem (8, .[, R), and let A' and A" denote

classes of estimators (i.e., subsets of X). In this case, (A', R) -4 (A", R) simply states that the

class A' is essentially complete relative to ~A". In this case, the notation can be simplified, and we

can write A' -4 A" to denote that A' is essentially complete relative to A". Note that the relation

-4 is a preorder on the collection of all classes of estimators. Clearly A -4 A, and it can be easily

verified that A -4 B and B -4 C imply A -4 C. Similarly, if A and C are classes of estimators, we

use A ~ C instead of (A, R) ~ (C, R).

Definition 13. A subset U of W (Z) is said to be half-closed if for each a in the closure of &,

there exists a u* E U such that u* < u.

Note that given an estimation problem (8, X, R), for each is E r (-, ) E W (Mel) and

R(-, 2) E & (8). Let M(A, Me/l) = {r(-, 2) : S E A} and M(A, 8) = {R(-, ) : f E A}. The

following definition is due to LeCam [30].

Definition 14. Given an estimation problem (8, K, R), a subset A of K is said to have the

property (W) if& (A, 8) is half-closed.

We need also the following closely related definition.

Definition 15. Given an estimation problem (8, K, R), a subset A of K is said to have the

property (W*) if&(A,Me/l) is half-closed.

The preceding definition can be reformulated as follows: Given an estimation problem

(8, K, R), a subset A of .[ is said to have the property (W*) if for each net {idaeA n A,

there exists a subnet {:ib bEB and an element i-* E A such that lim inf r(-, i'b) > r( ~).

Indeed, suppose #'(A, Me/l) is half-closed. Let {:i'o,,A denote a net in A. Let U denote the set

of limit points of this net. The set U is not empty since the space ~(Mel) is compact [31]. Since

~(Mel) is compact, M(A, Me/l) is relatively compact. Thus there exists a subnet {ib bEB that

converges to a point it in the closure of d(A, Me/l). Hence lim inf R(-, ib) = lim R( *,;b) = IL.

Since '(A, Me/l) is half-closed, there exists an element t' E M(A, Me/l) such that < Is.

Hence there exists an element i-* E X such that R(-,;i*) < liminf R(-,;ib). Suppose

for each net {:i )aEA in A, there exists a subnet {:ib bEB and an element i-* E A such that

lim inf r (-, ib) > r (-,;i*). Let it* belong to the closure of s'(A, Me/l). Then there exists a net

{u,),eA Ed 8(,iZ) that converges to u*. Clearly for each element u, in this net, there is

an element i's E A such that u, = R(-,;i). Thus lim R(-, is) = u*. It follows that there

exists a subnet {:ib bEB and an element i-* E A such that lim inf R(-, i'b) > R(-,;i*), whence

u* > R(-,;i*) and M(A, Me/l) is half-closed. In entirely analogous way, A has the property (W)

if and only if for each net {:i' )aEA in A, there exists a subnet {:ib bEB and an element i-* E A

such that lim inf R(-, ib) > R(-,;i*). Since each 8 can be identified as an element in Me/l, as

discussed previously, it is clear that the property (W*) implies the property (W). The property

(W*), as formulated with nets, is closely related to Wald's weak compactness [25, pp. 53].

The only difference is that in the current definition sequences are replaced by nets. Hence the

property (W*) is weaker than Wald's weak compactness. Sufficient conditions for a subset A of

X to have the property (W) are given in [31]. A simple sufficient condition is that there exists

a Hausdorff topology J for A such that A is compact and R(0, -) is lower semicontinuous on A

for each 8 E 8. Similarly, a sufficient condition for A to have the property (W*) is that there

exists a Hausdorff topology J for A such that A is compact and r(-r, -) is lower semicontinuous

on A for each -r Me Zl. Suppose there exists a topology J for A such that A is compact. Then

if the topological space (A, J) satisfies the first axiom of countability, by Fatou's Lemma, the

requirement that r(-r, -) is a lower semicontinuous on A for each -r Me Zl can be replaced by the

requirement that R(0, -) is a lower semicontinuous on A for each 8 E 8.

The following definition appears in [31].

Definition 16. A class A of estimators is said to be subconvex if for any i', i" E A and

0 < a~ < 1, there exists E A such that caR(0, i') + (1 ~) R(0, i") > R(0, 2) for all 8 E 8.

Clearly if A is a convex subset of a certain vector space and R(0, -) is convex over A for

each 8 E 8, then A is subconvex.

We are now ready to state several theorems some of which will be used throughout this work

and some of which are presented for their own sake.

Theorem 4.1. Given an estimation problem (8, X, R). Suppose that K has the property (W).

Then there exists a minimax estimator

Proof The proof is very similar to the proof of Wald's Theorem 3.7 [25]. The main differences

is that sequences are replaced by nets. Recall that we assume that V(8, X9, R) < +oo. Let {x,}

be a sequence such that supose R(8, in) converges to infc,x supose R(0, 2). Then there exists a

subnet {idaeA and an element i* E X such that lim inf R(-, is) > R(-, i*). Since {idaeA iS a

subnet of {x,}, lim sup,, R(-, i) > lim inf R(-, fe). In addition, lim,,, supose R(8, in)

lim sup,, R(0, in) for all 8 E 8. Thus in f a supose R(0, i) > supose R(0, i*), and i* is a

minimax estimator. O

After addressing the existence of a minimax estimator, we want to address the existence of a

restricted risk Bayes solution.

Theorem 4.2. Given an estimation problem (8, X9, R). Suppose .9 has the property (W*).

Then if V(8, K, R) < Qo, there exists a restricted risk Bayes solution relative to (v, Qo).

Proof Suppose V(8, X9, R) < Q. Note that the class X9(Qo, R) is not empty even if

Qo = V(8, K, R) since by Theorem 4.1i, there exists a minimax estimator.

Let {i,} be a sequence in X(Qo, R) such that limes, r(v, 2,) = infiEX(co,R) doI, 2).

Then since .9 has the property (W*), there exists a subnet {idaeA Of (Xn} and an element

i* E K such that liminf r(-r, i) > r(-r, *) for each r E Mel. Certainly r(-r, ,) < Qofor

all n and each -r E Mo/. Since {idaeA is a subnet of {x,}, it is clear that lim inf r(-r, 2,) < Co

for each -r Me Zl, whence R(0, i*) < Co for each 8 E 8. Thus i* E X(Qo, R). It is also

clear that infier(co,,) r(v, 2) > r(v, i*). Thus i* is a restricted risk Bayes solution relative to

Putting Qo = +oo in the above theorem, we get that for each -r Me Zl, there exists a Bayes

solution relative to -r (provided that X has the property (W*)).

The following Theorem, a version of Wald's complete class theorem, appears in [31].

Theorem 4.3. Suppose X is subconvex and has the property (W). Then the class of Bayes

solutions in the wide sense is essentially complete.

The following theorem is essentially a corollary to a very well known game-theoretic result

[32, Theorem 4.2].

Theorem 4.4. Suppose 8 is compact, K is subconvex, R(-, 2) < +oo for each & E K, and

R(-, 2) is upper semicontinuous on 8 for each is K Then the estimation problem (8, X, R)

is strictly determined and there exists a least favorable a priori distribution.

Proof Fix is E Since R(-, 2) is an upper semicontinuous real-valued function on a compact

set 8, it is bounded. Thus r (-, ) is bounded on 8, and hence a real-valued function. By Lemma

4.1, Me/l is compact and r(-, 2) is upper semicontinuous on Me/l for each is E It is easy to

verify that since X is subconvex, for each i', i" E X and 0 < a~ < 1, there exists i s such

that r (-, 2) < ~r (-r, i') + (1 a~)r (-, ") for all 8 E 8. Certainly r (-, ) is concave on Me/l for

all ie E Applying [32, Theorem 4.2],

1111 Sup r 7r, x sup inf r 7r, x.

Thus (8, X, R) is strictly determined. Since inf~Ex r(-, 2) is upper semicontinuous on a

compact set Me/l, there exists -ro E Me/l such that

III T TOZ) Sup inf r (r, x).

Thus -ro is a least favorable a priori distribution. O

The assumption in Theorem 4.4 that R(-, 2) < +oo for each i s and R(-, 2) is upper

semicontinuous on 8 for each i s is obviously too strong. In the following theorem this

assumption is considerably weakened, but at the price of a stronger assumption on X. Let

& C K denote the class of Bayes solutions relative to -r Me Zl. Before stating the theorem, we

need the following lemma.

Lemma 4.2. Suppose .F has the property (W*), then

infr~~i =inf r(-r, i) for all y E Me-lo

Proof In Theorem 4.2 it is shown that if X has the property (W*), there exists a Bayes solution

relative to -r for all -r Me Zl. The lemma follows easily. O

Theorem 4.5. Suppose 8 is compact, K is subconvex and has the property (W*), R(-, 2) <

+oo for each is E and R(-, ) is upper semicontinuous on 8 for each is E Then the

estimation problem (8, X, R) is strictly determined and there exists a least favorable a priori

distribution.

Proof Let G and Go denote the convex hull of (.[, Me/l) and &(9, Me/l), respectively. It

is easy to see that each g EG is a concave upper semicontinuous real-valued function on the

compact set Me/l. Thus by [32, Theorem 4.2],

inf sup g(-r) =sup inf g(-r). (4-2)

Fix -r Me Zl. On one hand, since &(9, Me/l) C Ga,

inf g(-r) < inf gr)=if(, )

On the other hand, if we fix g E G'a, then there exist an integer n > 0, real numbers

ai1, an > 0, and elements ul, ., an e (9, Mel) such that g = CE asse and

I~ as = 1. Let 1 < i < n be such that a () < Up (7) for j = 1,. ., n. Then g(-r) > a ().

Thus

inf g(-r) > inf gr)

It follows that

inf g(r) = inf r (-, 2). (4-3)

Since G > Ga,

inf sup g (r) < inf sup g (-). (4-4)
g6G 76MenL g6GSB 76Men

Fix g E G. Then since X is subconvex, there exists i s such that r (r, i) < g(-r) for all

-r MeiZ Hence

inf sup r (-, ) < inf sup g (-). (4-5)
26X 76enL g6G 76MenL

By (4-2)-(4-5) and Lemma 4.2, (8, X, R) is strictly determined. Since info,EG g is upper

semicontinuous on the compact set Me/l, there exists -ro E Me/l such that

inf g(-ro) = sup inf g(-r).
g6GSB 76Men g6GgB

By (4-3) and Lemma 4.2, -ro is a least favorable a priori distribution. O

If the compactness of 8 is dropped in the above theorem, then there may not be a least

favorable a priori distribution. If there exists no least favorable a priori distribution, but the

estimation problem (8, X, R) is strictly determined, a minimax estimator can be found as a

Bayes solution relative to a least favorable sequence of a priori distributions, i.e., a sequence

{7r,} E Me/ that satisfies lim,,, infie g r(-r,, i) = supe~Me inf~Ec r(r, i). We are not going

to deal with the question of how such a sequence can be constructed. The following theorem is

essentially Wald's Theorem 3.9 in [25].

Theorem 4.6. Suppose the estimation problem (8, X, R) is strictly determined. Then if -r is a

least favorable a priori distribution, any minimax estimator is also a Bayes solution relative to

Proof It can be verified that the proof of [25, Theorem 3.9] applies without any changes. O

When 8 is compact, we have the following version to Wald's complete class theorem.

Theorem 4.7. Suppose 8 is compact, K is subconvex and has the property (W*), R(-, 2) is

lower semicontinuous on 8 for each is K R(-, 2) < +oo for each is E and R(-, ) is

continuous on 8 for each is E Then & is essentially complete.

Proof Since the class of Bayes solutions in the wide sense is essentially complete, we are done

if we show that any Bayes solution in the wide sense is a Bayes solution. Suppose that i* is a

Bayes solution in the wide sense. Then there exists a sequence {n}i E Me/l such that

lim [ n n 2 *)] = 0 (4-6)

By Lemma 4. 1 part 1), there exists -rs Me Z and a subsequence {4}i ) Me Zl such that -ri

converges weakly to -r. By the hypothesis of the theorem, r(-, 2) is continuous on Me/l for any

is E Thus infies r(-, 2) is upper semicontinuous on Me/l [33, Proposition 1.5.12]. By Lemma

4.2, ini~,x r(-,2) is upper semicontinuous on Me/l. Since r(-,i*) is lower semicontinuous on

M el, inf a r (-, ) r (-, *) is upper semicontinuous on Me/l and

lim sup[ inf r(i,,) Tirq,,i*)] < inf r(O,i) -- r(q0, *). (4-7)

By (4-6) and (4-7), inf~Ex r(-r, 2) > r(-r, i*), whence i* is a Bayes solution relative to Ire. O

Remark 4.1. Suppose C7 is a metric space and 8 is a subset of W. Let 8 denote the closure

of 8. Then if R(-, 2) is lower semicontinuous on 8 for each is K a minimax estimator for

(8, K R) is also a minimax estimator for (8, X R). In addition, if A is essentially complete

in the estimation problem (8, X, R), it is also essentially complete in the estimation problem

(8, K, R). Thus we can solve (8, X, R) instead of (8, X. R). Indeed, suppose A is essentially

complete in the estimation problem (8, X, R). Fix is E Then there exists i' E A such that

R(0, i') < R(0, 2) for all 8 E 8. Since 8 C 8, R(0, i') < R(0, 2) for all 8 E 8 and A is also

essentially complete in the estimation problem (8, X, R). Fix is E It is easy to show that

the lower semicontinuity of R(-, 2) on 8 implies that supeoe R(0, i) = supose R(0, 2). Since

& is arbitrary, infe, g supose R8, i) = infie g supose R(0, 2). Thus a minimax estimator for

(8, K R) is also a minimax estimator for (8, X R). In particular if C7 is a finite-dimensional

normed space and 8 is a bounded subset of 7, then, II idustI~, loss of generality, we may assume

that 8 is compact since if thri\ is not the case, we can consider the estimation problem (8, X, R)

In this work it will be sufficient to impose the following conditions:

Condition 4.1. X is subconvex.

Condition 4.2. There exists a compact metrizable space 9 and a function R* : 8 x X*

[ 0, +oo] such that (X9, R) ~ ( 9 ", R*), R* (0, -) is lower semicontinuous on 9 for each 8 E 8,

and R*(-, a) E m(B(8)) for each as K *.

Condition 4.3. R(-, 2) is lower semicontinuous on 8 for each is K R(-, 2) < +oo for each

is 9 and R(-, ) is continuous on 8 for each a E .

Condition 4.4. If~ E X is a Bayes solution relative to -r M Z/e, & is an essentially unique

Bayes solution relative to -r.

In the following corollary, we summarize all the results of this chapter that are needed in the

subsequent chapters.

Corollary 4.1. Suppose Conditions 4.1 and 4.2 hold. Then X has the property (W*), there

exists a minimax estimator and there exists a restricted risk Bayes solution relative to (-r, Co) for

each -r M Z/e and Co > V(8, X9, R). If in addition, 8 is compact and Condition 4.3 holds,

(8, K, R) is strictly determined, there exists a least favorable a priori distribution -ro E Me/l,

any minimax estimator is a Bayes solution relative to -ro, and the class of Bayes solutions is

essentially complete. If in addition, Condition 4.4 holds, then the (essentially unique) Bayes

solution relative to a least favorable a priori distribution -rs Me Z is an essentially unique

Proof Although, the results of this chapter were formulated for an estimation problem

(8, 2', R), there is no use whatsoever of the fact that .9 is a class of estimators. Therefore,

the results also hold for the triplet (8, 2'*, R*), which is, in fact, a zero sum two- person game

(cf. [25]). For the sake of simplicity, we assume that (8, 9 ', R*) is an estimation problem. The

proof can be trivially modified to the case that 9 is not a class of estimators, but an arbitrary

set. Let r*(-r, a) = je R*(0, a)d-r, where -r Me Zl and as E *. It can be verified that since

2'* is a compact metrizable space and R*(0, -) is lower semicontinuous on 2 for each 8 E 8,

{r*(-, a) : as E X} is half-closed, i.e., 2'* has the property (IT*). It can be verified that

since ( 2, R) ~ ( 2*, R*), .9 has the property (IT*). By Theorem 4.1, there exists a minimax

estimator. By Theorem 4.2, there exists a restricted risk Bayes solution relative to (-r, Qo) for

each -r M Z/e and Qo > V(8, 2', R). If, in addition, Condition 4.3 holds, it follows from the

preceding results, that (8, 2', R) is strictly determined, there exists a least favorable a priori

distribution -roE Me/l, any minimax estimator is a Bayes solution relative to -r, and the class of

Bayes solutions is essentially complete. Suppose Condition 4.4 holds as well. Let -r denote a

least favorable a priori distribution. Since any minimax estimator is a Bayes solution relative to

I-r (Theorem 4.6), the (essentially unique) Bayes solution relative to -r is an essentially unique

minimax estimator. By [26, Theorem 2.3.1], this Bayes solution is admissible. O

While sometimes it is possible to verify Conditions 4.1-4.4 directly, other times, especially

when there is no closed from expression for R, it may be difficult to check whether these

conditions hold. In the next chapter, we consider the case that R is based on a loss function L.

In this case, it is possible to derive conditions that can be more easily checked when there is no

closed form expression for R.

CHAPTER 5
THE CASE THAT THE RISK IS SPECIFIED BY A LOSS FUNCTION

A risk function R is usually chosen by first specifying a loss function. A loss function

L : E x D R WU {+oo} specifies that L(:r, d) is the loss incurred to the experimenter

when using the estimate d and :r is the true value of the parameter. Let 9 denote the class of all

(randomized) estimators. Ignoring measurability considerations for now, let L (:r, i) denote the

mapping y H D L(:r, d)d~i(dly). The risk function R : 8 x 9 R WU {+oo} is specified as
follows:

R(0,;i- = x L(:,I-)d~/o U,;r)L. (5-1)

We impose several condition on the family p, the space Y, the space D of possible esti-

mates, and the loss function L.

Condition 5.1. The space Y is a Borel subset of a separable complete metric space and

FTy = B(y), where B(Y) is meant in the sense of the relative topology.

Recall that we assume there is given a probability space (R, Fo2, Q), where R = Y x X,

Fo~ = FTy x FTx, and Q2 is the joint distribution of Y and X. The marginal distribution of X,

denoted Qx, is an element of a family ~Px = (Pox : 8 E 8} of probability measures on (X, FTx),

where Pox(A) = Po(Y x A) for each Ae F x and 8 E 8. Similarly, the marginal distribution

of Y is an element of a family ~Py = {Po' : 0 E 8} of probability measures on (Y, FTy), where

Po ~ (A = Po(A x CL) I fo eac Aey andIC V t 8. Let denote a a-subalgebra of Fo.2 Let

f denote a nonnegative random variable. Let E( f |) denote the conditional expectation of f

with respect to the a-subalgebra W. Let Fa~x denote the a-subalgebra generated by the random

element X. Let E( f |X) = E( f |Fo~x). Let Ae F o.2 The conditional probability of the event

A given the random element X is denoted Pr(A|X) and is defined as E(I;|X), where Ig is the
indicator function of A. The conditional probability of the event A given that X = :r, which is

denoted Pr(A|X = :r), is any FTx-measurable function g for which

/ Ig()dQ)() g/~(:r)ilxdrx(r)foreah Ce (5-2)

To see that such a function indeed exists, the reader is referred to [34, pp. 220]. If y is a version

of Pr(A|X), then according to [34, pp. 221], a conditional probability of the event A given that

X = :r, g(:r), can be constructed as follows: g(X(w)) = g(w) (i.e., g(:r) = g(Lo), where w is an

element in R such that X (w) = :r).

Since in this work the distribution Q is unknown, but is known to be an element of P, the

conditional probability of Ae F o~ given X and the conditional probability of A given X = :r

depend on the true state of nature 8 E 8.

Definition 17. A faction Q (- |X) : FTy x R [0, 1], (A, w) H Q (A|X) (Lo) is said to be a

regular conditional distribution of Y given X if

1. For each we E Q(-|IX) (w) is a probability measure on (f Fy).

2. For each Ae F y, Q (A|X) is a version of the conditional probability Pr (Axx | X).

Fix 0 E 8. Condition 5.1 furnishes that there exists a regular conditional distribution of Y

give X wen Q= P [34, Theorem 2.7.5]. Let Po(-|X)l denote a regular conditionall disriutionvl

of Y given X when Q2 = Pa. For each r E and AeIC F y, we defineI Po(cA|:) = Po(A|X)(w),

where w E is such that X(w) = :r. We call Po(-|:r a regular conditional distribution of Y given

X = :r when Q = Po (or 8 is the true state of nature). Note that since Po(-|X) is regular, Po(-|:r

is a probability measure for each E X. In addition, for each Ae F y, Po(A|:r) is a version of

the conditional probability Pr (Axx | X = :r). Let pv x = { Po(- |:) : xrE X, 8 E 8 }.

Condition 5.2. There exist a o--Jinite measure p on (f FTy) and for each I E 8 a regular

conditional distribution of Y given X = :r when Q = Po, denoted Po (- |:r, such that Po (- |:r is

absolutely continuous with respect to ftfor each Ie 8 and xre X.

Let p~lo| I e) denote a density of Po (-|:r) with respect to p.

Condition 5.3. The loss faction Le E (FTx x B(D)) and is nonnegative.

LICondition 5.4. ThIe mpingLYIL 1 Po(A) E m(B(8)) for each Ae F y x FTx.

Conditions 5.3 and 5.4 were added to guarantee that the integration necessary in the

calculation of the risk function are well defined. It can be verified that if Conditions 5.1- 5.4

hold, R(-,;i) is an extended real-valued nonnegative B(8)-measurable function and hence the

Bayes risk relative to any distribution TrE Me/l is also well defined. We have the following
alternative expression for the risk function:

R(0, i) =~ LS (x.:)pol/ll)dP B] dox. (5-3)

Condition 5.5. The space D is a locally compact metrizable space and is o--compact.

We need the following condition for the case that D is not compact.

Condition 5.6. If D is not compact, then for each sequence of compact sets D, such that

U" ,D, = D and each element d, Sf D, (n = 1, 2, .. .), lim inf,,, L(x, d,) = supdED L(X, d)

for all x E X.

Condition 5.7. For each x e X, L(x, -) is lower semicontinuous on D.

It is easy to see that 9~ is subconvex. Indeed, given i:', i:" E 9~ and 0 < a~ < 1, let i:*

be an element in 9 that satisfies i*(-|y) = ak~'(-|y) + (1 a~)&"(-|y) for all ye E Then

R(-, i*) = a~R(-, i') + (1 a~)R(-, i"). Thus 9 is subconvex.
The space 9 can be identified with a subset of a certain vector space; this was shown

by LeCam in [30]. It is useful to discuss the properties of this vector space. The following

discussion essentially appears in [30] and [31]. Let C,(D) denote the class of continuous real-

valued functions on D with compact support, and let ||lu|| = supdED |U(d)|I for U E Oc(D).

Let LI1 denote the Banach space of equivalence classes of integrable functions on (Y, Ty, p)

with norm || f |1 = | f |dlp Denote by Lpv x the linear subspace of EL spanned by py lx. Let

the product space Co(D) x L~pv be equipped with the norm | | (u, f) | | = | |u| | V | | fl | for

ne E (D) and fe La x~. Let # denote the vector space of bounded linear functionals on

Oc(D) x L~pv The weak*-topology turns # into a locally convex topological vector space

[33]. Call gl, g2 EE Y|Tj) pX-eyUIValent if fr, |yl g2| fdy = 0 for all f e py x. Call
two estimators it,~ 2 ~ Y|X-equivalent if for each D' E B(D), Az(D'|-) and 2(D'|-)

are Py~lx-equivalent. L~et 9 denote the class of equivalence classes so obtained. A functional

e E is said to be positive if a > 0 and f > 0 imply ~(u, f) > 0. According to LeCam if

Conditions 5.1-5.5 hold, every positive linear functional of norm 1 can be represented by an

integral Q(u f ) = @~ u(d)di.~(dl~y) f (y~d y, where is E Certainly the converse is also true, i.e.,
each is E (or more precisely the equivalence class containing 2) is a positive linear functional

on C,(D) x LIv x of norm 1. Thus the class 9 is the class of positive linear functionals of

norm 1. In the sequel, we are not going to distinguish between an estimator and the equivalence

class containing this estimator. Whether a class A of estimators refers to the estimators or the

corresponding equivalence classes can be understood from the context. Let J denote the relative

topology for 9 induced by the weak*-topology.

Lemma 5.1. Suppose Conditions 5.1-5.5 and 5. 7 hold. Then the topological space (9, J) is

metrizable and R(0, -) is lower semicontinuous on 9 for all 8 E 8. If~ in addition, D is compact,

9 is compact.

Proof Under Condition 5.5, the space C,(D) is separable [30]. Since by Condition 5.1, Y

is a separable metric space, the space LI1 is separable [35, pp. 92]. Since LIv x is a subspace

of a separable normed space, it is also separable. It follows that the space C,(D) x L1pv x is

separable. By Theorem [36, Theorem 3.16], if @oc C is weak*-compact, then @o is metrizable.

By Banach-Alaoglu Theorem [33, Theorem 2.5.2], the set Be = {~ E # : || || < 1} is

weak*-compact, where | |~ | denotes the operator norm of a E Thus Be is metrizable. Since ]

is the relative topology, 9 is metrizable.

Let R(x, o, 2) = f L(x, 2)pa (1i| I )dpl. Then by (5-3), R(0, 2) = f R(x, 0, 2)dlox (x).

Using the results of LeCam [30], it can be shown that R(x, 8, -) is lower semicontinuous on 9

for each (0, x) E 8 x X. Since 9 is metrizable, we have by Fatou's Lemma that R(0, -) is lower

semicontinuous on 9 for each 8 E 8.

Suppose D is compact. LeCam [30] showed that a class A of estimators is J-compact if it

is J-closed and if the following conditions holds: For each e > 0 and each (0, x) E 8 x X, there

exists a nuE C:(D) satisfying 0 < Ir < 1 and fy u(d)dit(d y)dYo(,,I|, > 1 e unifo~rmly for all

i E A. The preceding condition certainly holds for 9 when D is compact. Since 9 is 1-closed,

it is compact. O

The following auxiliary lemma is needed in order to prove that Condition 4.2 holds under

very weak conditions.

Lemma 5.2. Let (A, F4~) be a measurable space. Let (C, w) be a topological space that is

locally compact, o--compact, and metrizable, but not compact. Let f : A x C R WU {+oo}

be a nonnegative function. Suppose f (a, -) is lower semicontinuous on C for each a s A.

Suppose for each sequence of compact sets C, such that U" ,C, = C and each element c, ( C,

(n = 1, 2, .. .), lim inf us, f (a, en) = supeec f (a, c) for all as A.~ Let (C*, 0*) denote the

one-point compactification ofC, and let oo denote the added point. Let f : A x C* R WU {+oo},

be defined as follows: For each as A f (a, c) = f (a, c) if ce C, and f (a, 00) = supeec f (a, c).

Then C* is compact and metrizable, f is nonnegative, f (a, -) is lower semicontinuous on C*

for each as A and a subset Co of C is in B(C) if and only if it is in B(C*). In addition, if

fe m(F x B(C)), fe m(Fa x B(C*)), and if f (, c) E m(A) for each Ee C,

f (-, c) EM (F~) for each c E C*.

Proof See Appendix A. O

Theorem 5.1. Suppose Conditions 5.1-5. 7 hold. Then Condition 4.1 and 4.2 hold for (8, 9, R).

As a consequence, there exists a minimax estimator and a restricted risk Bayes solution relative

to (-r, Co) for each -r Me Zl and Co > V(8, 9, R).

Proof We already showed that Condition 4.1 holds. Let us show that Condition 4.2 holds.

Suppose D is compact. Then by Lemma 5.1, Condition 4.2 holds. Suppose D is not compact.

We will use the one-point compactification of D to prove the theorem. The idea to use the

one-point compactification of the class of possible estimates to prove results of the type of this

theorem seems to appear first in [30]. Let D* denote the one-point compactification of D, and

let oo a D* denote the point that is added to D. Let L* : E x D* R WU {+oo} be defined

as follows: For each x E X, L*(x, d) = L(x, d) if d e D, and L*(x, 00) = supdeD L(x, d).

Let 9* denote the class of all estimators with D* as their space of possible estimates. By Lemma

5.2, L* E m(FTx x B(D*)) and D* is compact. Let L*(x, 2)(y) = JD* L*(x, d)di(dly),

and let R* (0, i) = fy xx L*(x, 2) (y)dYo(y, x). Then Conditions 5.1-5.5 and 5.7 hold f~or

(8, 9*, R*). Then by Lemma 5.1, / is compact and R*(0, -) is lower semicontinuous on 9*

for each 8 E 8. It can also be verified that R*(-, a) E m(B(8)) for each as E *. Let 9*(D)

denote the class of estimators & in 9* such that 2(Dly) = 1 for all ye E We claim that

(9*(D), R*) -4 (9*, R*). Indeed, fix i' E 9* such that i'(ooly) > 0 for some ye E Let

y' = {ye Y : i'(Dly) = 0}. Clearly Y' is measurable. Let (Aly) = i'(A n Dly)/i'(Dly)

for y ( y' (A E B(D)), and let 2(-|ly) be a Dirac measure with respect to a point d' E D

for all ye Y'. It can be verified that 2(D'|y) is B(y)-measurable for each D' E B(D).

Now if ye Y', then L* (x, 2) = L* (x, d') < L* (x, 00). If y ( y', then SD* L* (x, d) di(d|y)=

JD L*(x, d)/i'(Dly)di'(dly) < x'(Dly) JD L*(x, d)/i'(Dly)di'(dly)+ (1-i'(Dly))L*(x, 00) =

JD* L*(x, d)di'(dly). Thus L*(x, i) < L*(x, i'). It follows that & is as good as i'. This proves
that ( / "(D), R*) -4 ( / R*). Since 9* (D) C / ', we have that (9* (D), R*) ~ ( / R*).

Clearly ( /, R) ~ (9*(D), R*). Thus ( /. R) ~ ( /', R*). It follows that Condition 4.2

holds. O

As a consequence of Theorem 5.1, under the rather weak Conditions 5.1-5.7, there exists a

minimax estimator and a restricted risk Bayes solution relative to (-r, Qo) for each -r Me Zl and

Qo > V(8. /. R). In order to get the stronger results when 8 is compact, namely that (8, 9, R)

is strictly determined and that there exists a least favorable a priori distribution, we need to

prove that Condition 4.3 holds. Unfortunately, this seems to require rather strong conditions

on the loss function and family P. A set of such conditions is well known for the case that the

loss function is uniformly bounded and P is dominated by a o--finite measure. However, we

are mainly interested in the case that the loss function is unbounded (e.g., the quadratic loss

function). Moreover, in many cases P is not necessarily dominated by a o--finite measure.

CHAPTER 6
THE CASE OF CONVEX LOSS FUNCTION

In this chapter we consider the special case in which D is a convex subset of a finite

dimensional normed space and L(x, -) is convex over D for each x E X. Throughout this chapter

we will assume that L is a real-valued function, Y = RA~, X = WN, Ty = a(IN), and

-T = B(RWN) even if it is not implicitly stated. We are mainly interested in the case that the loss

function is quadratic, i.e., L (x, d) = | V(x d) | 2, where Ve R N, X Nz We will need the following

conditions.

Condition 6.1. for each x E X there exists an e > 0 and a c such that L(x, d) > eld| + c for all

dE D.

Condition 6.2. The measures {PY : 0 E 8 } are mutually absolutely continuous, i.e., for each

8, 8' E 8, PY is absolutely continuous with respect to Pf.

Clearly nonrandomized estimators are more attractive than randomized estimators in terms

of implementation. In general, randomized estimators can outperform nonrandomized estimators.

However, if D is a convex subset of R"N, and if for all x E X, L(x, -) is convex over D, it may

be sufficient to consider nonrandomized estimators. Let ~D denote the class of nonrandomized

estimators. The following lemma is closely related to the Rao-Blackwell theorem [37, Theorem

1.6.4].

Lemma 6.1. Suppose D is a convex subset of R", L(x, -) is convex over D for all x E X, and

Condition 5.3 holds. Then if Conditions 6.1 and 6.2 hold, ~D -4 9.

Proof Let is E Let Oo = {0 E 8 : R(0, 2) < +oo}. If 80 is empty, any estimator in ~D is

as good as 2. Suppose then that 0o is nonempty. Clearly the lemma is proved if we can show that

there exists an estimator i' E D such that R(0, i') < R(0, 2) for each 8 E 0o. Fix 0 E 80. Since

R(0, 2) < +oo, there exists a set Ae F y x &T such that L(x, 2)(y) < +oo for each (y, x) E A

and Po(A) = 1. Let C = {ye Y : (y, x) E A}. It follows that JD Id \$dy) < +OO for 811 y 6 C.

Thus the integral JD d di(dly) is well defined for all y e C. It is well known that C E FTy. Clearly

Po(C x X) = 1. Thus Po'(C) = 1. Since the measures {Po' : 0 E 8} are mutually absolutely
continuous, Po'(C) =1 fort eac n 8~. Thu Po(C x y) 1 for each 8 E 8.

Let i' denote a nonrandomized estimator such that i'(y) = JD d di \$ly) for y 6 C

and i'(y) = 0 otherwise. That i' is indeed a nonrandomized estimator, i.e., a FTy-measurable

function, follows from the fact that C E FTy By the Jensen inequality,

L~~x,2)(y)= L~~~idy ~~'y)) onC X (6-1)

Integrlating (6-1) with respectL to Po, we have R(0, 2) > R(0, i'). Since i' E ~D, ~D -4 9. O

Since ~D -4 9, we consider in the rest of this chapter the estimation problem (8, ~D, R)

instead of the estimation problem (8, '/, R). Clearly, ~D is subconvex by Jensen inequality.

Hence Condition 4. 1 holds for (8, ~D, R). Since ~DC 9, ~D ~ 9. Hence it is clear that Condition

4.2 holds for (8, D, R) if it holds for (8, 9, R). Thus if Conditions 5.1-5.7 hold, Condition

4.2 holds for (8, D, R). In Lemma 6.2 below, we show that under weak conditions, Condition

4.4 also holds for (8, ~D, R). In order to prove that Condition 4.3 holds for (8, ~D, R), it seems

necessary to make additional assumptions on the loss function and the family P.

Sometimes it is convenient to restrict the class of estimators that are available to the experi-

menter to the class of affine estimators. An estimator x is said to be affine if it is nonrandomized

and is an affine function on y. Since we consider only the case that Y = RWN and X = RAN, & is

affine exactly when & = Ay + b for some Ae R N X N and be R AN, and the space of possible es-

timates D is then RWN. Let L denote the class of affine estimators. The space L can be identified

with the space RWNXN x RWN where (A, b) E RWNX~y x RWN, is the estimator & = Ay + b E

and vice versa. Thus the space L can be identified with a finite-dimensional vector space with the

following addition and multiplication by a scalar: If & = (A, b), i' = (A', b') and a~ is a scalar,

& + i' = (A + A', b + b') and ai~ = (a~A, a~b). Let the space be equipped with the norm

|| "'| |A- '|| + b b'|i, where & = (A, b) and i' = (A', b'). Clearly is convex.

It is easy to see that Jensen's inequality furnishes that R(0, -) is convex over L for each 8 E 8

if L(x, -) is convex over D for each x E X. Thus if L(x, -) is convex over D for each x E X, L

is subconvex and hence Condition 4. 1 holds for (8, L, R). In Theorem 6.1 below, we show that

under rather weak conditions, Condition 4.2 holds for (8, L, R). As in the estimation problem

(8, ~D, R), it seems necessary to make further assumptions on the loss function and the family

p in order to prove that Condition 4.3 holds for (8, L, R). In Lemma 6.2 below, we show that

under weak conditions, Condition 4.4 also holds for (8, L, R).

Theorem 6.1. Suppose D = RWN, and Conditions 5.3, 5.4, 5.6, and 5.7 hold. Then Condition

4.2 holds for (8, L, R). As a consequence, there exists a minimax estimator and a restricted risk

Bayes solution relative to (Qo, r) for each Qo > V (8, 9, R) and -r E eZl.

Proof Clearly if {i,} E is a sequence that converges to an element i* in the sense of the

norm |I | | |, it converges pointwise on y. Let in = (A,, b,) for n = 1, 2, ... Suppose in

converges to i* = (A*, b*). Fix x E X. By Condition 5.7, lim inf,,, L(x, 2,(y)) > L(x, i* (y))

for each ye E By Fatou's lemma, lim inf,,, R(0, 2,) > R(0, i*) for each 8 E 8. Thus

R(0, -) is lower semicontinuous on for each 8 E 8. Let us show that for each sequence of

compact subsets C, of L such that U" zC, = L and each element in Sf C, (n = 1, 2, ...),

lim inf,,, R(8, in) = supy,~ R(0, i) for all 8 E 8. Fix a sequence of compact subsets C,

of L such that U" zC, = L and a sequence {i,} E L such that in ( C, (n =1,2...)

Fix 0 E 8. Certainly lim inf,,, R(8, in) < supper R(0, 2). Thus it is left to prove that

lim inf,,, R(0, in) > supy,~ R(0, 2). By Fatou's lemma,

lim inf R(0, 2,) > lim inf L(x, 2, (y))dPo(y, x). (6-2)

Fix (y, x) E Y x X. Let D, = {i,(y) : in E C,} (n = 1, 2, .. .). We claim that D, is a

compact subset of D. Indeed, let {di} be a sequence in D,. Then there exists a sequence {ij} in

C, such that (~(y) = di. Since C, is compact, there exists a subsequence {i~ } of the sequence

{ii} and an element i'* E C, such that &~ i '*. This implies that &~ (y) i '*(y), whence

{di, } i '*(y). Since i'*(y) E D,, D, is compact. We claim that U" zD, = D. Fix d E D.
Clearly there exists i s such that 2(y) = d (e.g., & = (A, b), where A = 0 and b = d).

Since U" ,C, = C, i s C, for a sufficiently large. This implies that d e D, for a sufficiently

large. Thus U" zD, = D. By Condition 5.6, lim inf,,, L(x, d,) = supdeD L(x, d), where

d, = 2,(y). It is easy to verify that supdeD L(x, d) = sup,,, L(x, 2(y)). Thus for each i s

we have

lim inf L(x, in(y)) > L(x, 2(y)). (6-3)

Since y and x are arbitrary,

Since x is arbitrary,

lim inf R(8, in) > sup R(0, 2). (6-5)

Since is a finite dimensional normed space, it is locally compact, o--compact, and

metrizable. Let denote the one-point compactification of L and let oo denote the added point.

For each 8 E 8 let R*(0, 2) = R(0, 2) if & E L, and let R*(0, 00) = sup,,, R(0, 2). By Lemma

5.2, is compact and metrizable, R*(0, -) is lower semicontinuous on for each 8 E 8, and

R*(-, a) E m(B(8)) for each as E *. Clearly L C *. Since oo is the only element in \ L and

R*(-, 00) > R(-, 2) for each is L (, R) -4 (*, R*). Since C *, (, R) ~ (*, R*). Thus
Condition 4.2 holds. O

The following lemma is concerned with the essential uniqueness of Bayes solutions.

Lemma 6.2. Consider the estimation problem (8, K, R), where X is either ~D or L. Suppose

D is a convex subset of R"N, L(x, -) is strictly convex over D for all x E ./, Condition 5.3 holds,

and V (8, K R) < +oo. Then if Condition 6.2 holds, a Bayes solution relative to -r Me Zl is an

essentially unique Bayes solution relative to -r.

Proof We prove the lemma for the case X = ~D. The proof for the case X = L is entirely

analogous. Fix -r Me Zl. It can be verified that our assumption that V(8, ~D, R) < +oo

implies that infeez, r (-, 2) < +oo. Suppose i', i" E D are Bayes solutions relative to -r.

The lemma is proved once we show that is equivalent to i'. Since inf~Er r(-r, i) < +oo,

r (-, i') = r (-, i") < +oo. Let i* = 1/22' + 1/22". Then by Jensen inequality, L (x, i* (y)) <

1/2L(x, i'(y)) + 1/2L(x, i"(y)) with strict inequality whenever i'(y) / i"(y). It follows that

R(0, 1i*) < 1/2R(0, 1i') + 1/2R(0, i"). Then clearly r(-r, i*) < 1/2r(r, i') + 1/2r(r, i"). Since

i' and i" are Bayes solutions relative to -r, we must have r (-, i*) = 1/2r (-, i') + 1/2r (-, i").

Thus R(0, i*) = 1/2R(0, i') + 1/2R(0, i") (a.e. -r). Clearly there exists a set 0o E B(8) such

that R(0, i') < +oo and R(0, i") < +oo for each 8 E 80 and -r(80) = 1. Thus there exists

an element 8o E Oo such that R(00, i*) = 1/2R(00, i') + 1/2R(00, i"). It follows that there

exists a set Ae F y x FTx such that L(x, i*(y)) = 1/2L(x, i'(y)) + 1/2L(x, i"(y)) for each

(y, x)It EAIC an Poo(,A) =. Let C = {ye Y : (y, x) E A}. Then Ce F y, ~I = ~I" on C and

Poo(C x X) = 1. Since the measures {Po' : 0 E 8, x E X} are mutually absolutely continuous,

Po(C x X) = 1 for each 8 E 8. It follows easily that i" is equivalent to i'. O

In the rest of this chapter, we assume the loss function is quadratic, i.e., L : (x, d) H

|V(x d)|12, where Ve R N"X A. We assume VTV > 0. The extension to the case VTV > 0 is

discussed later. In this case it can be verified that Conditions 5.1, 5.3, 5.5-5.7, and 6.1 hold. Thus

if D is convex and Condition 6.2 holds, ~D -4 9. Certainly ~D is subconvex and Condition 4.1

holds. In addition, if Conditions 5.2 and 5.4 hold, Conditions 4.2 holds for (8, ~D, R). Similarly,

Condition 4.1 holds for (8, L, R), and if Conditions 5.2 and 5.4 hold, Condition 4.2 holds for

(8, L, R). Suppose, in addition, that P is a Gaussian family of distributions, i.e., Y and X are

jointly Gaussian, when 8 is the true state of nature, for each 8 E 8. Then py and pv x are also

Gaussian families of distributions. Suppose the family pylx is dominated by the Lebesgue-

Borel measure on (RW~y x B(RIWy)), which is denoted p. It can be easily verified that py is

also dominatedLL by p andC thatL sinlce, in1 addUition, for echLI t V po~y), the densitLy of1, Po:,+, 1~,., 1,,,,:,~+, ,,,,Dr/\,/. 1,~,:,, kwith

respect to p, is positive, the measures {Po' : 0 E 8} are mutually absolutely continuous. Thus
Condition 6.2 holds. We would like to check under what conditions Condition 4.3 holds. Under

the current assumptions, it is well known that there exists a regular conditional distribution of X
given Y =. y,~, when is the true state of ntreLt Po(-|y) denote thi conditional distributor,:

which is well known to be Gaus sian. Let is : Ya R Nz y H Ea(X |y), where Ea (X | y

denotes the conditional expectation of X given Y = y, when 8 is the true state of nature. That

is, is is the conditional mean estimator for x based on the observation y when Q = Pa. Let

Fe = Ea(|V(fo(Y) X)|2), i.e., ~e iS the mean squared error (MSE) matrix of is, when

Q2 = Pa. Then by the so-called orthogonality condition, it is straightforward to get the following

expression for the risk function:

R(0, 2) = tr(V~oV )+|To) 2(y|pe(y)dy.l (6-6)

In what follows 9, is the class of Bayes solutions when (8, ~D, R) is considered, i.e., for each

i* AD tereexitsren uhta i,7 inf~Ez r (i, -r). Since ~D -4 9, each element

in 9, is also a Bayes solutions when (8, 9, R) is considered. In what follows, continuity of

functions from 8 into R"N and RWNxN is meant in the sense of the Euclidean norm and 2-norm,

respectively.

Theorem 6.2. Let Z = [YT XT]T. Suppo3se H Ea(Z) and 8 H Ea(ZZT) are continuous

on 8. Then if 8 is compact, R(-, 2) is lower semicontinuous on 8 for each f e D and R(-, 2) is

bounded and continuous on 8 for each is E Hence Condition 4.3 holds for (8, ~D, R).

Proof As mentioned earlier, py is a Gaussian family. It is easy to see that by the assumption

of the theorem, 8 H Ea(Y) and 8 H Ea(YYT) are continuous on 8. Since we assume

that the family py is dominated by p, 8 po(y) is continuous on 8 for each ye E It is

well known that 20 (Y) = Ea (X) + Ea ((X Ea (X)) (Y Ea (Y))") Ae (y Ea (Y)) and

re = ro Ea((X Ea(X))(Y Ea(Y))")Ae E((Y Ea(Y))(X Ea(X))T), where As

and To are the covariance matrices of Y and X, respectively, when 8 is the true state of nature

(i.e., when Q = Po). Since 8 H Ea(Z) and 8 H Ea(ZZT) are continuous on 8, 8 H fo(y) is

continuous on 8 for each ye~ E and 8 H Fo is continuous on 8. Since Po is Gaussian, there

exist a matrix As a RWN, X and a vector be E RWN such that 20 (Y) = Aey + be. Since 8 H 20 (y)

is continuous on 8 for each ye~ Y and Y = RIWN, 8 H As and 8 H be are continuous on 8. Since

8 po(y) is continuous on 8 for each ye E it follows easily from Fatou's lemma and (6-6)

that R(-, ) is lower semicontinuous on 8 for each is ED.

Fix -r E Mo.l Let 2, denote the (essentially unique) Bayes solution relative to -r. It is well

known that when the loss function is quadratic, the conditional mean estimator with respect to

-r Me Zl is a Bayes solution relative to -r. Thus without loss of generality, we may take 2, to

be the conditional mean estimator, i.e., 2,(y) = E,(Xly), where E, denotes the expectation

operator when 8 is the value taken by a random element whose distribution is -r. Note that since

by our assumption V(8, ~D, R) < +oo, r(-r, 7) < +oo. This implies that the MSE matrix of the
estimator 7 is bounded.

Clearly E, (X |y) = E, (E, (X |y, 8) | y). Let us examine the term E, (X |y, 8). This term is the

conditional expectation of X given Y = y and 8, where 8 is the value taken by a random element

whose distribution is -r. But this is exactly Eo (X |y). Thus E, (X |y) = E, (Eo (X |y) |y). It follows

that | V(7(y) 200 (Y)) 2" = | VE, (So(Y) 200 (Y) Y) 12. Thus

I ~v(v)(Y f o.(Y) 2 < E,(|V(f o(y) iso lU)) 121

=E,(|V((Ae Aeo)y + be boo) 21v

< E,((||V||2(|| Ao Aeo 2|lay| + |bo boo 1))21Y

Since the mappings 8 H As and 8 H be are continuous on the compact set 8, they are bounded.

Thus there exist positive real numbers cl and c2 Such that

|V(:, (y) ieo(Y) 12 < E,((clly| + c2 2 y) = (Cly + C2 2

Since 8 H Fo is continuous on 8, which is compact, there exists a positive real number a such

that tr {V~oVT} < a for all 0 E 8. We have from (6-6) that

R(00, 7)
It is easy to verify that R(00, fr) < +oo. Put h(y) = (clly| + C2 2. Since 8o is arbitrary

R(0, 7) < +oo for each 8 E 8. Moreover, for each 8 E 8, |V(i,(y) is(y))|2 < h(y) and & is

a Po-integrable function. Certainly

|R(0, 17) R(00, 97)| < | tr(V(Fe Foo)VT)|I

+/4 (W(iaiii) -2,(7)| -|IooU -7U) 2 0

Let {0,} be sequence that converges to 8o. We showed that lims,, |Co. Feo | = 0. Thus

lim tr(V(Fos Foo)VT) = 0. (6-9)

By the Lebesgue dominated convergence theorem,

Certainly for each n > 0, | V(ion (y) 97(y)) 12 < h(y). Since h is Poo-integrable and 8 po(Y)

is continuous on 8 for each ye E a well known theorem on exponential families [37, Theorem
1.4. 1] furnishes that

It can be verified that since 8 po(y) is continuous on 8 for each ye E the above equation

implies that

Since |V(fos (y) fr(y))|2 y)

/ | ~ion(7)- 977))2 00 y)|d 0.(6-11)

By (6-8)-(6-11), R(-, 7) is continuous on 8. Since R(-,, ) < +oo and R(-, 7) is continuous
on 8, which is a compact set, R(-, 7) is bounded on 8. It follows that R(-, 2) is bounded and
continuous on 8 for each is 9 O

Let Mr denote the class of Bayes solutions when (8, L, R) is considered.

Theorem 6.3. Let Z = [YT XT]T. Suppo3se H Ea(Z) and 8 H Ea(ZZT) are continuous

on 8. Then if 8 is compact, R(-, 2) is lower semicontinuous on 8 for each is L R(-, i) is

bounded and continuous on 8 for each is M Hence Condition 4.3 holds for (8, L, R).

Proof Although some modifications are needed, the proof of this theorem is very similar to the

proof of Theorem 6.2 and is omitted. O

While the proofs of some of the results of this chapter clearly break down if VTV is not

positive definite, all these results are valid also in the case that VTV is not positive definite.

There is a simple method to show that this is indeed the case. Note that if VTV is not positive

definite, L(x, d) = L(x, d') whenever d d' E Ni~(V). Thus we may call d and d' in RWN

equivalent if d d' E Ni(V) and choose the space of possible estimates to be the set of

equivalence classes so obtained instead of RWN. In this case, the space of possible estimates is

equipped with the metric a (d, d') = | Vt V(d d') |, where d, c' E D, d is any element in d, and

d' is any element in d'. This choice for D is equivalent to choosing D = Ni(V)I with the usual

Euclidean norm since for any equivalence class in D there is associated a point in Ni(V)I and

vice versa. It can verified that with either one of these choices for D, the results for (8, ~D, R) are

still valid. To show that the results for (8, L, R) are still valid, it is necessary, to define a class

cL = {(N~(V)IA, Ni(V)lb) : (A, b) E }. It is obvious that L' ~ L. Certainly for each & E L',

2(y) is in the new space of possible estimates. Now, it is straightforward to show that all the
results of this chapter are still valid for (8, L', R) and hence for (8, L, R).

CHAPTER 7
FINDING A MINIMAX ESTIMATOR AND THE DUAL PROBLEM

In this section, we consider in more detail the problem of finding a minimax estimator for an

estimation problem (8, X, R). We need the following additional conditions.

Condition 7.1. For any -r M E Zl, there exists an essentially unique Bayes solution relative to -r.

We let 2, denote the (essentially unique) Bayes solution relative to -r. Let r(-r) = r(-r, 7).

Let Me/l denote the class of distributions in Me/l with finite support.

Condition 7.2. If {n } is a sequence in Me/l that converges weakly to -r Me Zl, then R(0, in)

converges to R(0, 7) uniformly on compact subsets of 8.

By Corollary 9.1i, if Conditions 4.1-4.4 hold and 8 is compact, the problem of finding a

least favorable a priori distribution is dual to the problem of finding a minimax estimator. Thus in

the rest of this chapter, we concern the problem of finding a least favorable a priori distribution.

The following theorem is essentially similar to a theorem in [18] and the iterative algorithm

proposed in [38].

Theorem 7.1. Suppose 8 is compact, R(-, i) is continuous on 8 for each is 9 and Conditions

7.1 and 7.2 hold. Construct a sequence {-ri}@, E Me/l as follows. Let -ri be any distribution in

Me/l. Having chosen -rl, ri E M, let Os E 8 be such that R(0s, in) = supeoe R(8, in). Let

ni,o = c004 + (1 a~)nr. Let asi be such that r(9i,ai) = supe[o, 1] T(ni,a) and let ni+l = 74,ai. Then
the sequence {8 }) converges weakly to a least favorable a priori distribution.

Proof The proof follows easily from the proof of [18, Theorem 2.3], with slight modifications.

The main difficulty in the algorithm described in Theorem 7.1 is in finding 04 E 8 such that

R(0s, in) = supoe, R(8, in) for i > 1. Another difficulty is due to the fact that, in general,

since the support of -r may grow as i grows, the complexity of the algorithm calculations also

grows with i. The problem of finding asi such that r"(nr,ai) = supe[o,1] "(ri~,a) is addressed
below and can be solved numerically. In some special cases, it is easy to find Os E 8 such that

R(0s, in) = supose R(8, in) for i > 1 and the complexity of the algorithm calculations remain

fixed as i grows. In these special cases, the algorithm described in Theorem 7.1 is practical in

finding minimax estimators. One such case is when 8 is a finite set. In the sequel, we show that

when we consider linear estimation with quadratic loss function, the algorithm of Theorem 7.1

can be often used to find a minimax estimator. In the more general case, this algorithm can be

used just to find e-minimax estimators, which are good approximations to minimax estimators for

e sufficiently small. We discuss the derivation of e-minimax estimators, in great detail, in the next

chapter.

The problem of finding asi E [0, 1] such that r"(40,a) = supae[o, 1] T(8i,a) is a standard

optimization problem in RW:

maximize r"(cl + (1 a~)-r) subject to as [0, 1].

Fix -r1,r BE Me/l, and let To, = 071r + (1 a~)-r for as [0, 1]. In the rest of this chapter we

consider the following optimization problem, which includes the previous one as a special case:

maximize r(-r,) subject to as [0, 1]. (7-1)

Lemma 7.1. a~ H (-r) is concave on [0, 1].

Proof That r is concave on Me/l is well known and easy to prove. As a consequence, it is easy to

verify that a~ H (-r) is concave on [0, 1]. O

Let D(a~) = r(Tis iv.) r(-r,;i'r) for as [0, 1].

Lemma 7.2. Fix c~o E (0, 1). Then r"(-r) r"(To.) < D(no)(a~ n~o) for all as [0, 1].

Proof Certainly r(-r,) r(To,,) < r(-roo 'r,) r(-r,,). It can be easily verified that r(-roo m>)

r(-r,,) = (a~ n~o)D(n~o). The lemma follows easily. O

As a consequence of this lemma, -D(n~o) is the subderivative of a~ -r(-r,) at the point

n~o E (0, 1).

Lemma 7.3. Suppose Condition 7.2 holds. Then a~ D (c) is monotonically decreasing and

continuous on [0, 1] and r(-r,,) = supe[o,11? (-r) for n~o E [0, 1] if D(n~o) = 0.

Proof Certainly To, converges weakly to -roo whenever a~ converges to cto. Since -r1 and -r2 have

finite supports, Condition 7.2 furnishes the continuity of a~ D(a~) on [0, 1]. Since -D(a~) is

the subderivative of the convex function am -r(-r,), aa D(a~) is monotonically decreasing

on (0, 1). By continuity, a~ D(a~) is monotonically decreasing on [0, 1]. Fix to E [0, 1].

Suppose D(ao) = 0. Then by lemma 7.2, r(-r,) < r(-roo) for all as [0, 1]. It follows that

r(To,) = supago,1,] r(To,).

Remark 7.1. Lemmas 7.2 and 7.3 imply that we can use relatively simple numerical al gwiduallr\

to solve (7-1) numerically. Suppose r (-ri, 27) = r (-72, 7). Then r (-r1) = r (-ro, 7) for all

as [ 0, 1]. It follows that r (-r1) > F (-r) for all as [0, 1] and hence a~ = 1 is a solution of (7-1).

Similarly, if r (-r, in ) = r (-r2, i), a~ = 0 is a solution of (7-1). Using the fact that a~ D (a)

is continuous and monotonically decreasing on [0, 1], and the fact that as [0, 1] is a solution of

(7-1) if D(a~) = 0, the optimization problem (7-1) can be solved as follows: If D(1) > 0, then

a~ = 1 is a solution of (7-1). IfD (0) < 0, then a~ = 0 is a solution of (7-1). Finally, ifD (0) > 0

and D (1) < 0, there exists as (0, 1) such that D (a) = 0 and hence a~ solves (7-1). In ;Ihi\ case,

since D(a~) = 0 and aa D(a~) is monotonically decreasing on (0, 1), a~ can be easily found

using the bisection method.

CHAPTER 8
APPROXIMATING A MINIMAX ESTIMATOR

In this chapter we approximate minimax estimators by using e-minimax estimators. The

main idea is to find a minimax estimator (or an approximation to a minimax estimator) i* for an

estimation problem (87, X, R), where 87 is a finite subset of 8, such that & is an e-minimax

estimator for the estimation problem (8, X, R). The main question is how to construct the set

87. Finding a minimax estimator for (87, .F. R), when Of is a finite set can be done using the

algorithm in Theorem 7.1. In practice, it may be necessary to find an e'-minimax estimator for

(87, K, R) such that this estimator is an e-minimax estimator for (8, X, R). In the latter case,

the algorithm in Theorem 7.1 can still be used, but it is necessary to have a condition that enables

us to check when the required precision is achieved. Checking if an estimator 7, an essentially

unique Bayes solution relative to -r, is an e-minimax estimator can be done using the following

lemma:

Lemma 8.1. Suppose Condition 7.1 holds and supose R(8, 17) F (-) < e, then 2, is an

e-mmnimax estimator

Proof Since supose R(8, ir) in a supose R(0, i) > r(-r), we have that

sup R(0, 7) < inf sup R(0, 2) + e.

Thus 2, is an e-minimax estimator. O

The condition of Lemma 8.1 is only a sufficient condition for an e-minimax estimator.

However, under certain conditions, if the numerical algorithm converges to a least favorable a

priori distribution, then there exists an integer NV such that Lemma 8. 1 is satisfied for the NVth

iteration.

Lemma 8.2. Suppose 8 is compact, ./ is subconvex and has the property (W*), R(-, i) < +oo

for each is E R(-, 2) is continuous on 8 for each is E and Conditions 7.1 and 7.2 hold.

Let {nri } EZ/e be a sequence that converges weakly to a least favorable a priori distribution

I-o EM eZl. Then there exists an integer NV such that supose R(8, i,) r (-r) < e for all i > NV.

Proof Clearly the lemma is proved if it is shown that supose R(8, in) r"(-r) 0 Since 8 is

compact and Condition 7.2 holds, it is straightforward that

sup R(0, in) sup R(0, fro). (8-1)
BEe sEe

Since R(-, fr) is continuous on 8, it can be shown, as in the proof of [38, Theorem 2 part 3],

that

F(i) r(To). (8-2)

By Theorems 4. 1 and 4.5, there exists a minimax estimator and (8, K, R) is strictly determined.

By Theorem 4.6, fro is a minimax estimator. Thus

r(-ro) = sup R(0, fro). (8-3)
BEe

By (8-1)- (8-3), supose R(8, in) r"(-r) 0 and the lemma is proved. O

The following condition is needed for some of the results of this chapter.

Condition 8.1. 8 is a subset of a normed space 7 and the metric a for 8 is induced by the

norm |I | | | of W.

A set U is said to be 6-dense in 8 (in the sense of the metric a) if for any 8 E 8 there exists

8' E U such that A(0, 8') < 6. Note that if 8 is compact, for each 6 > 0, there exists a finite

subset of 8 that is 5-dense in 8.

Theorem 8.1. Suppose X is subconvex and has the property (W), Condition 7.1 holds, and the

family {R(-, 7) : -r Me }Zl is equicontinuous on 8. Then for any e > 0, there exists a 6 > 0

such that for any finite 6-dense subset 8; of 8, the following hold:

1. For any -r M E Zl, there exists a probability measure -ro E Te, such that

R(0, fro) R(0, 7) < e for all 8 E 8.

2. There ex-ists a probabiiliy measure 7o E 7e, such that 7 (7o) = supEreve 7 (7) and f r is an

e/2-minimax estimator

Proof 1) Let p(0, 8') = supee |R(0, 7) R(0', 7)|i. The equicontinuity of {R(-, 7):

-r Me}i/o on 8 implies that for any e > 0, there exists a 6 > 0 such that p(0, 0') < e/2

whenever A(0, 8') < 6. Let Of be a finite subset of 8 that is 5-dense in 8 in the sense of a.

Then 87 is e/2-dense in 8 in the sense of p, i.e., for any 8 E 8, there exists a O' E 87 such that

p(0, 8') < e/2. Fix -r Me Zl. Consider the estimation problem (87, X, R). Let 87 be equipped

with the discrete topology. Since 87 is finite, it is compact and R(-, 2) is continuous on 87 for

each is K It can be shown, similarly to Theorem 4.7, that there exists a probability measure

I-o E Te, such that R(0, fro) < R(0, 7) for all 8 E 87. Since 87 is e/2-dense in 8 in the sense

of p, R(0, fro) < R(0, 7) + e for all 0 E 8 and part 1) is proved.

2) Clearly the results of Chapter 4 can be used for the estimation problem (87, X, R). Thus

for any finite 87, there exists a least favorable a priori distribution Tro E -re, and fro is an admis-

sible minimax estimator for (87, X, R). It follows that maxeee, R(8, fro) < maxeee, R(8, fr)

for any -r Me Zl. Since Of is e/2-dense in 8 in the sense of p, supose R(8, iro)

supeoe R(0, 7) + e/2. Thus fro is an e/2-minimax estimator for (8, X, R). O

Remark 8.1. Theorem 8. 1 part 1) is an e-complete class theorem, provided that 9J is essentially

complete (the reader is referred to[/39]for the definition of an e-complete class of decision

functions). It differs from TU 10 al~ il: 's e-complete class theorem in[/39] and has the advantage
that each element in thri\ class is admissible.

Theorem 8. 1 part 2) implies that an admissible e-minimax estimator can be found in the

following way: Partition the set 8 to disjoint sets 81,82.., O N Such that the diameter of

84 (i = 1,. ., NV) does not exceed 6. In each set 84 take a point Bi. Let 87 = {Or 82, HN *

Solve for a minimax estimator for (87 X R). If 6 is sufficiently small, then the resulting

estimator is e-minimax for (8, K, R). Theorem 8.1 does not spec~if\ how to choose 6 that would

guarantee a certain e. However it is clear from the proof that it is sufficient that 6 \aiisfit 1

|R(0, 7) R(0', 7) | < e/2 whenever -r Me Zl, 8, 8' E 8, and A(0, 8') < 6. In fact for

thri\ choice of 6, it is enough derive an e/2-minimax estimator for (87 K R) and the resulted

estimator is e-minimax for (8, X, R).

The hypothesis in Theorem 8. 1 that the family {R(-, 7) : -r Me}i/o is equicontinuous on

8 is rather strong. While it is often satisfied in the case that the risk is based on a loss function

that is uniformly bounded, it may not be satisfied in the case that the loss function is unbounded.

Another, difficulty with Theorem 8.1 is that the requirement from the set 87 is very strong. A

set 87 that is constructed according to this theorem has the property that any estimator can be

approximated by a Bayes solution relative to a measure with support in 87 with degradation of

no more than e. However, we are mainly interested in approximating a minimax estimator and

not any estimator. Thus it may be possible to choose a finite set whose cardinality is significantly

smaller than 87 in Theorem 8.1. Due to the above, we are not going to use Theorem 8.1 in the

sequel, and we are going to derive methods that do not require equicontinuity.

It is convenient to use the notion of Fr~chet differentiability. Let B(X, Y) denote the set

of bounded linear operators from a normed linear space X to a normed linear space Y. Given

f E B(X, Y), || f || denotes the operator norm of f i.e., || f || = sup,: ||2llst ||I fX|| .
Definition 18. Let X and Y be normed linear spaces, U C X open, f : X Y and x e U. The

function f is said to be Fri'chet differentiable at x if there is an element AE B (X Y) such that

||f (x h) f (x) Ah||
him = 0.
hwo ||h||

We call A the Fri'chet derivative of f at x and denote it by D f(x).

If f : X Y is twice Fr~chet differentiable at x E X, we let D2 f(x) denote the second

Frichet derivative of f at x. Note that D2 f(x) E B(X, B(X, Y)). Given is 9 X, let DRa(0) and

D2Ra(0) denote the Fr~chet derivative and the second Fr~chet derivative, respectively, of R(-, 2)
at 0.

Definition 19. Given an estimation problem (8, X9, R), let y > 0 and 80 be a subset of

8. An estimator i* E X is said to be a (y, 80)-optimal estimator if supesea R8, *

(1 + y) V (o, K R). An estimator i* e X is said to be a y-optimal estimator if it is a

(y, 8) -optimal estimator

Clearly a y-optimal estimator is an e-minimax estimator for e = y V(8, K, R). It is often

more convenient to search for a y-optimal estimator instead of an e-minimax estimator. The

reason is that by deriving y-optimal estimator, 100y gives us the maximum degradation in terms

of the maximum risk in percent (relative to the minimax risk). This is different than the case that

we derive an e-minimax estimator, in which e is not normalized by the minimax risk. Certainly

what is considered a small e in a certain problem, may be considered huge in a different problem.

Lemma 8.3. Suppose X is subconvex and has the property (W), Conditions 7.1 and 8.1 hold

and 8 is compact. Suppose for any -r Me Rl, there exists an extension of R(0, 7) from 8 to an

open convex set 8 C UC c such that R(0, 7) is Fri'chet differentiable on U and there exists

a real number M~ such that | |DR,7 (0) | | < M~ supose R8, 17) for all -rE Me/l and 8 E 8.

F~ix Y > 0 and 0 < ?' < -i, and let d = .~i~ Then if 8p is a fnite 5-dense subset of 8, a

(y', 87) -optimal estimator is a y-optimal estimator

Proof Let -ro E Me/l be such that the support of -ro is contained in 87 and xfo is a (y', 87)-

optimal estimator. Fix 8, 8' E 8 such that | |8 el'|~ |, <. Then by the mean value theorem,

|R(0, ro) R(0', r)I I 0-0 8' supe[o~, ||DR;~b(a0B+ (1 a)0')| < 6MsuposeR(8 N~,r).

TIhus supose R(B, ir)(1 bM) < supose, R(B1 fro) < V(87, K^, R)(1 + y'). Since

V(87, K, R) < V(8, K, R), we have supose R(8, fro) < V(8, K, R)(1 + y), and fro is a

y-optimal estimator. O

The cardinality of a finite subset 87 of 8 that is constructed according to Lemma 8.3 may

still be very large; this may cause the calculation of a least favorable a priori distribution for

(87, X, R) to be formidable. In a special but very important case of compact 8, it is possible to
derive a finite 6-dense subset of 8 such that 6 depends linearly on j This means that if y is

decreased by factor of say 4, 6 should be roughly decreased only by factor of 2 (assuming y
and y'
Definition 20. Let K( be a subset ofa vector space V. A nonempty set S C K( is called an

extreme set of K if a, b E K, O < t < 1, and (1 t) a + tb E S imply a, b e S.

Definition 21. Let K be a subset of a vector space V. The point p e K is an extreme point of K

if and only if a, b e K, O < t < 1, and (1 t) a + tb = p imply a = b = p.

Let A be a subset of a vector space V. Let dA denote the extremal boundary of A, which is

the set of all extreme points of A. Let co(A) denote the convex hull of A. Given a subset K of a

vector space V and a subset S of K, let & (S) be the set of all points s ES such that a, b E K,

O < t < 1, and a = (1 t)a + tb imply a, b E S. Clearly the set &K(S) is an extreme set of

Kt. Given a set A, we let 2A denote the power set of A, i.e., the collection of all subsets of A. We

consider the case that 8 is a compact convex subset of the normed space C7 and 88 is a finite set.

Note that by [36, Theorem 3.20], the set co(88) is compact and hence the Krein-Milman theorem

[36, Theorem 3.23] implies that 8 = co(88).

Lemma 8.4. Suppose 8 is a compact convex subset of the normed space 7 and 88, is a finite

set. Let 6 > 0. Then there exists a finite subset 8; of 8 such that for any nonempty A E 2ae,

87 n deo (co(A)) is a 6-dense subset of deo (co(A)). The set 87 is a 6-dense subset of 8 and

Proof Let Al, A2,..., AN denote the elements of 2ae excluding the empty set. Fix 1 < i < NV.

It is easy to see that Beo(co(Ai)) C 8. Thus Beo(co(Ai)) is relatively compact and hence totally

bounded. Thus there exists a finite subset 87,< of deo(co(Ai)) that is 5-dense in de(co(Ai)).

Let 87 = U~,Of87. Then clearly 87 is finite. Moreover, 87 n de(co(Ai)) = 87,4 and hence

Of n Be(co(Ai)) is a 6-dense subset of Be(co(Ai)). Since 8 = co(88), 8 = de(co(88))

and Of is a 6-dense subset of 8. Let 8 E 88. Then {0} E 2ae. Since 8 is an extreme point,

Be(co({0})) = {0}. Thus there exists 1 < j < NV such that 87,; is a nonempty subset of {0} and

we must have B y,; = {0}. Thus 8 E Of and Of > 88. O

Let ?" = {ae R : ||a||1 = 1}.

Lemma 8.5. Suppose 8 is a compact convex subset of the normed space 7 and 88 is a finite

set. Let Or 82 N denote the elements of 88,. Fix 0 E 8. Then there exists an index set

J C (1, 2,. Nsuch that

#(i) > 0, i EJ
pe R" n ",O (i04- (8-4)
i=1 P(i) = 0, otherwise

Proof Certainly 8 can be written as a convex combination of 81, 82, *, HN. Let 1 < M~ < N be
ther largest intege, r for whichl therem exist ye R r n 7" and an index set J such that | J| = M~,
O 2", O()s ()>0frali ,ad#i for all i ( j. Suppose (8-4) is false.

Then there~ exists R' r R"P n 7"N and a non empty index set J' / J such that 8 = 'i)s

P'(i) > 0 for all is J ', and P'(i) = 0 for all i ( J'. Let po = (4+ #'/.Thn8= N )

Po(i) > 0 for all i EJ U J', and Po(i) = 0 for all i ( J U J'. Since J' / J, | J U J'| > M~, which
is a contradiction to the definition of M~. Hence (8-4) must hold. O

Lemma 8.6. Suppose X is subconvex and has the property (W), Conditions 7.1 and 8.1 hold,

8 is a compact convex subset of the normed space 7, and 88~ is a finite set. Suppose for each

-r Me Zl, there exists an extension of R(-, 7,) from 8 to an open convex set 8 C U cC

such that R(-, 7) is twice Fri'chet difgerentiable on U and there exists a positive real number

M~ such that | |D2R~, (0) | | < M supose R(8, ir) for all -rE Me/l and 8 E 8. Let y > 0 and

0 < y' < 7. Let a = .I Suppose 87 is a finite set that latirisi 1: For any nonemnpty
A E 2ae, Of n Be,(co(A)) is 5-dense subset of de(- (co(A)). Then a (y', 87)-optimal estimator is a

y-optimal estimator

Proof Let Or, 82, N denote the elements of 88~. Let -ro E Me/l be such that the support of

-ro is contained in 87 and xro is a (y', 87)-optimal estimator. Clearly R(-, fr) is a continuous

function on a compact set 8. Thus there exists a point 0* E 8 such that supos A, iRo8~)

R(0*, fr). If 0* E 87, then the lemma clearly holds. Suppose, then, that 0* is not in 87.

By Lemma 8.5, there exists an index set J C {1, 2, .. ., N}) such that (8-4) holds for 0*. Let

A = {Os : ie J}~. Since 0* is not in 87, 0* is not an extreme point (Lemma 8.4) and | J| > 1. Let

8' E Of n Be(co(A)) be such that A(0*, 0') < 6. Then 8' = Ci, y(i)8i for some y E R"~ n 7"N

such that y(i) = 0 for i ( J. Since 8* = C (i8fosmeeR N sctht(i>0

for all i EJ and P(i) = 0 for i ( J, 0* + b(0* 8') E co(A) for sufficiently small b > 0. Let

0"1 = 0* + b(0* 8'), where b > 0 is sufficiently small so that 0"1 E co(A). Then the line segment

connecting 8' and 0"1 lies in co(A). Moreover, 0* lies in this line segment. Since 8* is a maximum

of R(-~, o) over 8 and R(-,, fr) is continuously differentiable on an open set 8 C UC C7, the
directional derivative of R(-, fr) at the point 0*, in the direction of 8', must be zero, i.e.,

DR;,, (0*) (0'- 0*) = 0.

By the Taylor theorem,

|R0'ir) R0*ir l 2|' 0*||I sup ||D2 ;~~c81+ t)8*)1
2~aE[o, 1]

By the hypothesis of the theorem,

| | 8' 0* | |~ sup | | D2R ~(c81 1 c) 8* I 2 M sup R (0, iro)
aE[o, 1] aEe

Thus supose R(H, iro,) supose, R(H 70~) < 70,~ ) R('o~~) < 2M/2 supose R(H, iro)-

It canl be verified that since supose, R(H 0) < V(87, K,~- R)(1 + y') and V(87, K,: R) <

V(8, .[, R), then supose R(8, fro) < V(8, X. R)(1 + y). Thus fro is a y-optimal estimator.

Suppose the normed space C7 in Condition 8.1 is R"N with its usual norm. In this case, if 8 is

a compact convex set and 88 is finite, 8 is said to be a convex polytope in R".

Definition 22. Let Ve R NxN denote an odopels~rnal matrix and let I denote the ith column

vector of V. Let 6 E R"N be a vector whose ith component 6(i) > 0 for i = 1,. ., NV. We say

that A' is (6, V) -dense subset ofAC R "W if for any as A there exists an a' E ~A' such that

| (a -a')TI | < 6(i)for i= 1,...,N1.

We have the following version of Lemma 8.6.

Lemma 8.7. Suppose .F is subconvex and has the property (W), Conditions 7.1 and 8.1

hold, and 8 is a convex polytope in R"N. Suppose for each -r E f.... there exists an extension

of R(-, 7) from 8 to an open convex set 8 C U cC such that R(-, 7) is twice Fri'chet

difgerentiable on U and there exists a positive definite Ao a RWNxN such that -D2R~ (8

Ao supose R(0, 7) for all -r Me o and 8 E 8. Let V be a matrix whose ith column is the ith

eigenvector of the matrix Ao (i = 1,. ., NV). Let (4 denote the ith eigenvalue of the matrix Ao

(i = 1,. .l. N. Le 7 > 0 annd 0 < 7' < 7. Letd t IWZ hR be sh that 6(i) = \i
for i = 1, ... N. Suppose that 87 is a finite set that \nat~is7 \.: For any nonempty A E 2ae,

8; n de (co(A)) is (6, V)-dense subset of de -(co(A)). Then a (y', 87)-optimal estimator is a

y-optimal estimator

Proof Similarly to the proof of Lemma 8.6, let -ro E Me/l be such that the support of -ro is

contained in 87, xfo is a (y', 87)-optimal estimator, and 0* E 8 be such that supose R(8, ) =

R(0*,2). Then 0* E deo(co(A)) for some A E 2ae. Let 8' E 87 n deo(co(A)) be such that

| (0' 0*), | < 6(i). By the Taylor theorem, we have that for some 8" E 8 in the line segment
connecting 0* and 8'

R(0, ro)= (0*, fr) + ~(0' *)'"D2; (o(, / _

Certainly there exists a a2,. .. UN 6W Such that 8' = C By the h~ypoth~esis of th~e

theorem,

-(0' *)'D2R: p (0 l _0 0* pl _O O*TAo (0' *) sulp R(0l fr) i a suip R(0-, fr).
i= 1

Since a, (8' *): a,' < fi(i)2. Thus supose n(o:. ) -1P~ supos ro)

R(0*, ~ ~ ~ i= br)-R0,fo (i)2(4/2 supose R(8 ro). It can be verified that since

supose, R(8, fro) < V(87, K, R)(1 + y') and V(87, X9, R) < V(8, X9, R), then

supeoe R(8, fro) < V(8, X9, R)(1 + y). Thus fro is a y-optimal estimator. O

CHAPTER 9
THE RESTRICTED RISK BAYES PROBLEM AS A MINIMAX PROBLEM

Let v be an element of Me/l and Qo E [0, +oo]. In this section, we show that the problem

of finding a restricted risk Bayes solution relative to (v, Qo) is equivalent to a certain minimax

problem. The main result of this chapter, Theorem 9.1i, appears as a conjecture in [16].

We impose the following additional conditions in order to derive the results of this chapter:

Condition 9.1. There exists an essentially unique Bayes solution relative to v.

Let Qo = supoe, R(8, iv), where is denotes the (essentially unique) Bayes solution relative

to v.

Condition 9.2. V(8, .[. R) < Co < +oo.

Let RP(0, 2) = p r(v, 2) + (1 p)R(0, 2) and rP(-r, 2) = fe RP(0, 2)dr, where 0 < p < 1.

Let K(i, p) = supose RP(0, 2) and let K(p) = infe, g K(i, p). Since supose R(8, ~) > W, ~)

we have that

K(i, pi) > K(i, p2) 1f 2iIp. (9-1)

Lemma 9.1. Suppose Conditions 9.1 and 9.2 hold. Then K is concave, decreat~sing, and

continuous on [ 0, 1].

Proof See Appendix B. O

Let G(A) = supose R(8, i) rv ~).

Lemma 9.2. Let 0 < pi < p2 < 1. Suppose that it is a minimax estimator for (8, ./, RP1) and

2 is a minimal eStimatOTJOT (0, X P). Th8#

3) supose R8, ~1) < supose 2(8 *~

Proof See Appendix C. O

If 0 < p < 1, there is a strong relation between the estimation problems (8, K, R) and

(8, K, RP), which stems from the definition of RP.

Lemma 9.3. If fo is a Bayes solution relative to -r Me Zl in the estimation problem (8, X, RP),

then io is a Bayes solution relative to pv + (1 p)-r in the estimation problem (8, X, R).

Proof The lemma is an immediate consequence of the fact that for any is E rP(-r, 2)=

pr~~i)+ ( ~r~,2 = ~py+ ( ~r,).O

Remark 9.1. By Corollary if Conditions 4.1 and 4.2 hold for (8, K, RP), there exists a

minimax estimator in the estimation problem (8, X, RP). If in addition, 8 is compact and

Condition 4.3 holds, there exists a least favorable a priori distribution -ro E Me/l, and any

minimax estimator is a Bayes solution relative to -ro. A minimax estimator is an essentially unique

minimax estimator and is admissible if Condition 4.4 holds as well. In fact, it can be shown that

if 0 < p < 1, each one of Conditions 4.1-4.4 holds for (8 K RP) if it holds for (8 K R).

Remark 9.2. Certainly we can use the results of Section 7 for the estimation problem

(8, K, RP), where 0 < p < 1, provided that Conditions 7.1-7.2 hold for the estimation problem

(8, K, RP). In fact, if 8 is compact, it is sufficient that Condition 7.1-7.2 would hold for the

estimation problem (8, K, R). Indeed, suppose 8 is compact and Conditions 7.1-7.2 hold for

the estimation problem (8, X, R). It follows easily from Lemma 9.3 that for any -r Me Zl, there

exists an essentially unique Bayes solution relative to -r in the estimation problem (8, X, RP),

whence Condition 7.1 holds. Let if denote the (essentially unique) Bayes solution relative to -r

in the estimation problem (8, X, RP). Let {n)g, be a sequence in Me/l that converges weakly

to To E Me/l. Let -rf = pv + (1 p)-r for all i > 0. Then {-rf} converges weakly to -rd. Since

R(0, ~If) = R(0, 4) for all i > 0, R(0, ~If) converges to R(0, ~If) uniformly on the compact set
8. Since the convergence is uniform, r (v, if ) converges to r (v, ifo). Thus RP/C (0 f ) converges

to RP (0, ~If) uniformly on compact subsets of 8 and Condition 7.2 holds.

Lemma 9.4. Suppose X is subconvex and has the property (W*) and Conditions 9.1-9.2 hold.

Let Q(p) = -yp + P, where y < G(A,) and Pe R Suppose Q(p) > K(p) for all p e [0, 1].

Then if y > G(A) whenever & is a minimax estimator for (8, K, R), there exists i' E X such

that K (i', p) < Q (p) for all 0 < p < 1.

Proof Suppose y > G(A) whenever & is a minimax estimator for (8, X, R). Certainly Q is
continuous on [0, 1] and by Lemma 9.1, K is also continuous on [0, 1]. Thus there exists e > 0

such Q(p) K(p) > e for all p e [0, 1]. It is easy to verify that {K(i, -) : S E X(Co, R)} is an

equicontinuous family on [0, 1]. Thus there exists n > 1 such that

|K(p) K(p')| < e/2 and

|K~, ) -K~, p)|< e/2 whenever |p p'| < 1/n and is E (Co, R). (9-2)

Let pi = i/n and ~i be a minimax estimator for (8, X, R^i) for i = 0, 1, n. Note
that in = is and ~i exists for all 0 < i < n 1 by Theorem 4.1 and Remark 9.1. Since

io is a minimax estimator for (8, X, R) and in = is, G(in) > y > G(fo). Thus by
Lemma 9.2 part 1), there exists 1 < m < n such that G(im) > 7 > G(im-1). By Lemma

9.2 part 3), im-1 E X(Co, R). Let rl be such that rlG(im) + (1 rl)G(im-1) = 7. Let

K,7(p = q~im, ) + 1 -)K(:im-,_l p). Since K(i, p) = G()po + sulpose R(0, i), then

K,7l(p) = -7p + 17 supose R(H:0L, i) + (1 17) supose R(H, im-1)_l. It can be verified that

K(pm) < K,7(pm) = 17K(pm) + (1 rl)K(im-1, pm).) (9-3)

Using (9-2),

qK~p) +(1 q)Kim-, p) < K~p) +(1 q)Kpm-) +e/2 K~m) e.(9-4)

By (9-3) and (9-4), |K(pm,) K,,(p,)| < e. It follows K,,(pm) < 0(pm,). Since the line K,,(p)

is parallel to Q(p), we must have that K,7(p) < Q(p) for all 0 < p < 1. Since K` is subconvex,
there exists i' E X such that supose R(0, i') < rl supose R(8, im) + (1 rl) supose R(8, im-1)

and r (v, i') < l r (v, im) + (1 -q) r(v?, im-,)l. Therefore, K (.i', p) < K~, (p) < Q (p) for all

0 < <1. O

Theorem 9.1. Suppose X is subconvex and has the property (W*), Conditions 9.1-9.2 hold,

and V(O. .F. R) < Co < Co. Then io E X is a restricted risk Bayes solution relative to (v, Co)

if and only if there exists 0 < po < 1 such that supose RPo (8, f o) = inf e,x supose RPo (0, 2) and

supose R(8, fo) = Co.

Proof Suppose there exists 0 < po < 1 and io E X such that suposeRP(8 ro 9)

inf~Ex supose RPe(0, 2) and supose R(8, fo) = Co. Suppose that io is not a restricted risk Bayes
solution1,+ ,, re at v to (v C o) T hen, there ,:,,, exists, ` an e t m t r i' such that su pose R (,E'

Co and r (v, i') < r (v, fo). This implies that supose RP0o(0, i') < supose RP0o(8, fo), which is a

contraiction.IUI HeceICC 1o is a re;strictdU 15 risk C; Bayes 1I souionrlaiZLve to (v, C/o).

Suppose that io is a restricted risk Bayes solution relative to (v, Co). LCertainly supoeO IL(0,O 1o
Co.Supoe upseR(,i) C. etQr) pr(v, fo) + (1 p)~Co. Then Q(p) > K(io, p)

for all 0 < p < 1 and Q(1) = K(io, 1). Since Co < Co, So is not a Bayes solution and

K((io, 1) > K(1). Thus Q(p) > Krp for Iv all 0 < pv < 1. Let = Co r(v, fo). Then y < G(is)

and Q(p) = -yp + Co. Let i* be a minimax estimator for (0. .F. R). Since io is a restricted risk

Bays sluionreltie t (v C) ad C >V(8, K, R), r(v, fo) < r(v, i*). Thus y > G(i*).

By Lemma 9.4, there exists i' E X such that K(i', p) < Q(p) for all 0 < p < 1. It follows

thatZ supeoe R(0\, LI ') < o and r(v, i') < r(v, fo). This contradicts the fact that io is a restricted

risk, Bayes,..:, solution reatv to, (v o) Thus~ suoeR(,,) o Suppose there exists no

0 < p < 1 such that supose RP(8, fo) = inf~Ec supose RP(0, 2). Then K(io, p) > K(p) for

all 0 < p < 1. Since supose R(8, fo) = Co, K(io, p) = Q(p) and by Lemma 9.4, there exists

i' such that supose R(8, i') < Co and r(v, i') < r(v, fo), which contradicts the fact that io is

a restricted risk Bayes solution relative to (v, Co). Therefore,, there exists 0 < p~o< 1 suc thatl IIL

supose RPe(0, fo) = infc,x supose RPe(0, 2). Finally, since V(8, K, R) < Co < Co. O

Under the hypothesis of Theorem 9.1i, finding a restricted risk Bayes solution relative to

/, (v:,.:,,,, Co) s euivlentto indng minimax, esimto for,+:,+, the~1, estmaio prole (8, C, R o

some 0 < p < 1.

Lemma 9.5. Suppose X is subconvex and has the property (W*) and Conditions 9.1-9.2

hold. Let {p,}g, be a sequence in [0, 1] that converges to po E (0, 1). Let ~i be a minimax

estimator for (8, K RPi) for i = 0, 1, .. Then if V (8, K R) < supose R(8, f o) < Co,

supeoe R(8, fi) supose R(8, fo).

Proof Let M~ = supose R(8, fo) and suppose V(8, K, R) < M~ < Co. Fix e > 0 such that

M~ e > V(8, K, R) and M~ + e < Co. By Theorem 4.2, there exist restricted risk Bayes

solutions relative to (v, M~ e) and (v, M~ + e). Therefore, Theorem 9.1 yields p' and p" in (0, 1)

such that supose R(0, *, ) = M~ e and supose R(0, x*,,) = M~ + e, where i*r is a minimax

estimator for (8, X, RP) for all p E (0, 1). By Lemma 9.2 part 3), p' < po < p". In fact, since

supeoe R(0, *,) < supose R(8, fo) < supose R(0, *,,), we must have p' < po < p". Thus

p' < pi < p" for i sufficiently large. Lemma 9.2 part 3) furnishes that | supose R(8, ii) M|1 < e

for i sufficiently large. It follows that supose R(8, ii) supose R(8, fo) O

Remark 9.3. If X is subconvex and has the property (W*), Conditions 9.1-9.2 hold, and

V(8, L, R) < Co < Co, a restricted risk Bayes solution relative to (v, Co) can be solved

in the following manner: Find po E (0, 1) such that supose R(8, fo) = Co, where io is a

minimax estimator for (8, X, RPO). The estimator io is a restricted risk Bayes solution relative

to (v Co). If a minimax estimator for (8, K RP) can be found and the supremum of its risk

can be calculated for all p e (0, 1), Theorem 9.1 and Lemma 9.2 part 3) imply that we can

find a sequence {pi } that converges to po. By Lemma 9.2 part 3), such a sequence can be found

easily using the bisection method. 1Moreover Lemma 9.5 implies that if {ps} is a sequence that

converges to po and ~i is a minimax estimator for (8, K, RPi), then supose R(8, in) is close

to Co whenever p, is sufficiently close to po. Thus for a sufficiently large, in can be used to

approximate a restricted risk Bayes solution relative to (v, Co) with any desired accuracy. Thus

we have a practical way to derive a restricted risk Bayes solution relative to (v, Co).

CHAPTER 10
ESTIMATION WITH A RESTRICTION ON THE OBSERVATIONS THAT CAN BE USED

As before suppose we are given a probability space (R, -T2, Q), where R = Y x X and

-T& = FTy x FTx, and random elements Y : R H (f FTy) and X : R H (X, FTx), which are

jointly distributed and their distribution Q is an element in a class {Po : 0 E 8} of measures

on (R, -T2). So far we considered the case that the experimenter observes an observation

ye E the value taken by Y, and based on this observation estimates the parameter x, which

is the true value of X; little was assumed on the structure of the spaces y and X. Let S, and

S, denote arbitrary sets. Suppose for each A E So there is associated a measurable space

(3x FTx ) and for each A E S, there is associated a measurable space (y x, Fy,). In addition,

suppose X = n,, s, y >es,~ Yx, E = nes. Ex,, and Fy = ns,~ FT>. Let
Y, : R (P x, FTy ) and Xx : R H (3x FTxx) be the random elements defined as follows: For

each we E Y (w) and X ( ) are the projections of Y(w) and X(w) into the spaces yi and 3x ,

respectively. Let yx and xx denote the true values of Yx and X respectively. Given a subset S

of S,, let Tis = nxas yi and FTys = n,,s -Ty,. Let Ys : OR (Ps, -Tys) denote the random

element defined as follows: For each Lce E Ys(Lo) is the projection of Y(Lc) into the space Ps.~

Let y7s denote the true value of Ys.

In many applications there are restrictions on the components of y that may be used in order

to estimate components of x.

Definition 23. Given the sets S, and S, defined above and a mapping & : S, 2sy, the triplet

(S,, S,, h) is called an estimation space. An estimator for x \subin t to (S,, S,, h) is a collection

{ix : A E S, }, where &x is an estimator for xx based on Yh( ).

Thus an estimation space completely specifies which components of the parameter x are

to be estimated and which components of the observation y can be used. Suppose (S,, S,, h) is

an estimation space and for each A E So there is associated a space D the space of possible

estimates for x a space Xx, the space of available estimators for xx based on Yh( ), and a risk

function Rx : 8 xA Rxi U {+oo}. In what follows we assume Rx is nonnegative for each

A E S,. Then in fact there are given a collection of estimation problems {(8, X9x, R ) : A E S,}.

Fix A E S, and S C S,. By an estimator &x of xx based on ys, we mean a mapping from Pis

into -rD, such that ys H x(D |ys) is FTys-measurable for all DI ae BD). Let D = ness Dr

and X = n,,s. X Then D and X are the space of possible estimates for x and the space of

available estimators for x, respectively.

If, in addition, there is given a (nonnegative) risk function R : 8 x .9 R WU {+oo}, the

problem is similar to the problem treated so far. Given & : S, 2sy, for each A E S,, let PoX

denote the joint distribution of Yh,3 and X when 8 is the true state of nature. If S is a subset

of S,, we let Po~s denote the marginal distribution of Ys, when H is the true state of nature. If

Ae S we let Pux^ denote the marginal distribution of X when O is the true state of nature.

Given a class A of estimators for x and A E S,, we use Ax to denote {ix : f E A}.

It is left to specify a risk function for (8, X9). One possibility is R(0, 2) = sup,,ss R (0, ~x).

If for each A E S,, there exists a minimax estimator for x which is denoted by if then

i* = { 1 : A E S,} is a minimax estimator for x. Indeed, it is easy to see that

inf sup R(0, 2) inf sup sup R (0, & ). (10-1)

Since for each &x E K supeoe R (0, & ) > supose Rx N, 1r),

inf~ sup sup R (0, & ) > sup sup R (0, if ) sup R(0, i*). (10-2)

By (10-1) and (10-2), i* is a minimax estimator for x. Thus with the above risk function,

minimax estimation for x subject to (S,, S,, h) is completely determined by minimax estimation

for xx (A E S,).

Let us consider another possibility for the risk function. Suppose S, C N. Let w : S,

[0 + 00). We can define the risk function

Without loss of generality, we may assume that w(A) > 0 for each A E S, since if w(A) = 0 for a

certain X, we can remove that A from S, without affecting the risk function R.

Theorem 10.1. Suppose there are given a collection of estimation problems {(8, X R ) : X E

S, }, where S, C N. Let K = H Ass. K and let R(0, 2) = C,,es R (0, & ) w(A), w (A) > 0

for each A E S,. Then

1. If Conditions 4.1 and 4.2 hold for (8, K R ) for each A E S,, then Conditions 4.1 and

4.2 hold for (8, K, R).

2. Suppose S, is finite. Then if Conditions 4.3 and 4.4 hold for (8, K R ) for each A E Sz,

then Conditions 4.3 and 4.4 hold for (8, K, R).

Proof 1) Fix i', i" E .9 and 0 < a~ < 1. Then for each A E S,, there exists if; aKX such

that R (-, x)
R(-, i*) < a~R (-, ') + (1 a~)R (-, i"), whence X is subconvex. Thus Condition 4. 1 holds for

(8, X, R).
For each A E S, let 9* be a compact metrizab~le space and RIafnto fo

into [0, +oo] such that (K R ) ~ ( X*, Rr;), Rr (0, -) is lower semicontinuous on X* for each

Se 8 and Rr;(-, a) E m(B(8)) for each as E *. Let X* = nes. X* be equipped with the

product topology. Let R*(0, 2) = C,,es Rr;(0, & )w(A). Then by Tychonoff's Theorem, 9 '
is compact. In addition, X* is metrizable since it is a countable product of metrizable spaces.

Since for each A E S, andu v e 2 RIL(0, & ) is lower semicontinuous, for each finite subset

S of S,, CExs Rr;(0, & )w(A) is lower semicontinuous. Since the pointwise supremum of

any collection of lower semicontinuous function is a lower semicontinuous function, we have that

R*(0, -) is lower semicontinuous on X* for each 8 E 8. Since the pointwise limit of a sequence

of measurable functions is a measurable function, it is easy to show that R*(-,~x E ) m(B(8)).

It is also rather straightforward to show that (X9, R) ~ (X*, R*). Thus Condition 4.2 holds for

(8, X, R).

2) The proof is rather straightforward and is omitted. O

In the case of countable S,, Theorem 10.1 implies that under rather weak conditions on

(8, K R ) (A E S,), there exists a minimax estimator for (8, X, R) and a restricted risk

Bayes solution relative to (-r, Qo) for each -r Me Zl and Qo > V(8, K, R). The theorem is

especially useful for the case of finite S, since it specifies that if certain conditions holds for each

one of the estimation problems (8, K R ) (A E S,), they hold for (8, .[, R) and hence the

results of the previous chapters are valid for (8, .[. R).

Example 10.1. Suppose the experimenter observes a sequence {y, } of observations, where

y, E R">Y, and there are given a sequence {x, } of parameters, where x, E RWN. In ;///1 case,

Ai = RWN and it is convenient to take FTy, to be B(Ai). Suppose there is given an estimation

space (S,, S,, h), where S, = N, S, C N, and & : S, 2s. The observation y is \imphl, the

sequence {y, }. For example, if S, = N and & : n E S, H {0,. ., n}, we have the so-called

casual filtering problem (assuming the index n is a time index). If h : n e S, H {0,. ., a 1},

we have the so-called one-step prediction problem. If at is a positive integer S, = {0, 1,. ., at },

and & : n E S, H {0,. ., n}, we also have a certain casual filtering problem. Also,

estimation of a continuous time process can be entered to dIri\ formulation. Put S, = [0, +oo),

S, C [ 0, +oo) and, for example, let & : a s [ 0, +oo) [ 0, a]. In ://i\ case the observation y is a

function (or a signal).

Example 10.2. Consider the following discrete-time linear stochastic system in state-space form:

Xn I = Fox, + In~,,, a > 0,

yn = He x, +v, ,

where x, E RWN" (n > 0) is the state vector y, E RWN is the system output, v, E R"~

is the measurement noise, In~,, E RWN" is the model noise, and H, and F, are matrices in

RWNXN and RWNXN, respectively. Suppose there is given an estimation space (S,, S,, h),

where S, C N, S, = N, and for each A E S,, h(A) is a finite subset of S,. This example

is certainly a special case of the previous example. Note that the filic ,rio. prediction, and

\rl ur llrings problems are all special cases of dIri\ example. In the most general case, there are

stochastic and/or deterministic uncertainties in xo, O',1, {va }, {H, }, and {F, }. For example,

in some problems the initial state vector xo can be modeled as a deterministic and unknown

vector which is known to belong to a set 31o C RAN. This is a deterministic uncertainty. Of

course, in other problems xo may be modeled as a random vector whose distribution is known

to belong to a certain class of distributions (e.g., the Gaussian distribution with zero mean and

some restriction on the covariance). This is an example of a stochastic uncertainty. The system

noise sequence {tr~,, }, for example, can be modeled as a random process whose distribution

is known to belong to a certain set (e.g., a subset of the set of joint Gaussian distributions of

a sequence of independent random vectors), but in some problems can be better modeled as

a deterministic sequence that is known to belong to a certain set (e.g., a subset of the set of

bounded sequences). The relations between the quantities xo, O',1, }, {va }, {H, }, and {F, } are

also important. For example, suppose xo, {tr~,, }, and {v, } are modeled as random with stochastic

uncertainties, then the dependence between l;
also be modeled. The most convenient way to spec~if\ 8, in ;//i\ general case, is it to assume

that (xo, ( I,,, }, (vn}, {H,}, {F,}) is a value of a random element whose joint distribution

belong to a class 8 of measures defined on the appropriate product space. There is no problem

with dIri\ formulation since even if one of l;
regarded as random with distribution that assigns probability 1 to a certain value in No0. Our

basic assumption regarding 8 was that 8 is a metric space. This is true if 8 is equipped with

the topology of weak convergence since xo, I,',,- vn, H,, and F, (n = 0, 1, ..) are defined on

separable metric spaces. Thus the results of previous chapters and dIri\ chapter may be used.

Note that in dIri\ case an element of M/e is a probability measure defined on a class of probability

measures. Luckily, in many important cases, we can use instead of the set 8 defined above a

simpler equivalence class, as illustrated in the following example.

Example 10.3. Suppose that in the previous example { H,} and { F,} are known sequences,

the sequences {v, } and {tr~,, } are uncorrelated, v, and vm are uncorrelated for n / m, and

In~,, and I,. are uncorrelated for n / m. In addition, suppose the initial state xo and v, (Ir,, )

are uncorrelated for all n > 0. Suppose also that the distributions of In~,, (v,) and I,. (vm)

are identical for each n and m and have zero mean. As in the previous example, suppose h(A)

is a finite subset of S, for each A E S,. Suppose xo is the value of a random vector whose

disvtr-ibtion belongs to a set 81 o~f measures. on (RA, B(RN.)), vu, is thep value of a randonm

vector whose distribution belongs to a set 82 Of meaSuTes On (WN aINy )) (n = 0, 1, ..),

and In~,, is the value of a random vector whose distribution belongs to a set 83 Of meaSuTes On

(RAN, B(RWN,)) (n = 0, 1, ..). Let 8 = 81 x 82 x 03, where ife Oi 4for i = 1, 2, 3,

O1x 02 x 3 is the product measure. Then 8 can be regarded as the space of state of nature

instead of the space used in the previous example. Suppose R(0, 2) = C,,es R (0, & )w(A),

R (0, & ) = fygX) xxh SDX L (Zx d )di:(\$ ?i(d |,3dr (y, x), and L (xl d) = | V(x d)|2
where Vx RN x N"XN. If~ in addition, we assume that each measure in 8 is Gaussian, the space

of states of nature can be further \imlrrllfie d. Indeed, for each 8 = Or x 02 x 83 E O, Ele i 8e

and As (0) denote the mean vector and autocorrelation matrix of Os, respectively, for i = 1, 2, 3.

For examley6, if 0 = Or x 02 X ~83 thena 11 (0) = fx ,.."'i rand At (0) = fx~o xox dO LePt

oi = {((Di(e), Ag(e)) : e 8 } for i = 1, 2, 3. Let C7 = RWN x ga, C2 IN x N, and

C73 INze xNzc Le 1~ 2 C<* ~ C~3 eranly i S a vector space with coordinate-wise

addition and multiplication by a scalar (i = 1, 2, 3). Given as WE let | |a| |4e = |a(1) | + | |a(2)1 | |2

Similarly, C7 is a vector space with coordinate-wise addition and multiplication by a scalau:

Givenz as E let ||a||v = C:= ||a(j)||4-~. Let 8 = 81 x 82 x O3. Th8# O is a subset of W.

The set 8 equipped with the norm |I | | | is clearly a metric space. Then it is more convenient to

regard 8, which is a subset of finite dimensional normed space, as the space of states of nature

instead of 8 provided that P is a Gaussian family. Note that if the class of estimators is restricted

to affine estimators, since the loss function Li is quadratic for each i E S,, then II idust~, loss of

generality, we may assume that each measure with bounded covariance in 8 is Gaussian since it

can verified that the risk function depends only on the mean vector and autocorrelation matrix of

dur \le probability measures.

In the rest of this work, we apply the general theory of previous chapters to the state

estimation problem of Example 10.3.

CHAPTER 11
THE STATE ESTIMATION PROBLEM

In this chapter we consider in more detail the state estimation problem of Example 10.3.

For the sake of clarity, let us repeat the formulation of this problem. We consider the following

discrete-time linear stochastic system in state-space form:

X,+1 = Fox, + It',,, a > 0,

Yn = H,x, + v., (11-1)

where x, E RWN" (n > 0) is the state vector, y, E RW~y is the system output, v, E RW~y is

the measurement noise, It',, E RWN" is the model noise, and H, and F, are matrices in RWNyXN

and RWN, X respectively. We assume that there is given an estimation space (S,, S,, h),

where S, C N, S, = N, and for each A E S,, h(A) is a finite subset of S,. We assume that

{H,} and {F,} are known sequences, the sequences {v,} and {tt',,} are uncorrelated, v, and

vm are uncorrelated for n / m, and it',, and w., are uncorrelated for n / m. In addition,

the initial state xo and v, (I,',,) are uncorrelated for all n > 0. We assume xo is a Gaussian

random vector and {v,} ({tt',,}) is a sequence of identical Gaussian random vectors. We

assume that the mean and covariance of xo and the covariances of v, and I,',, are unknown; it

is only known that each one of these quantities belong to a certain set. The space of states of

nature 8, in this case, is 81 x 82 x 02, where 81 is the class of possible mean vector and

autocorrelation matrix pairs for xo, 02 iS the class of possible mean vector and autocorrelation

matrix pairs for v, (n = 1, 2, ...), and 83 iS the class of possible mean vector and autocorrelation

matrix pairs for It',, (n = 1, 2, .. .). As mentioned in Example 10.3, the set 8 is a subset of

a finite dimensional space normed space C7 (see the definition of C7 in Example 10.3). For

8 = Or x 02 x 83, where 04 E 84 for i = 1, 2, 3, we let ri(0) and As(0) denote the mean

vector and autocorrelation matrix of Os, respectively, for i = 1, 2, 3. Our main assumption

regarding the space 8 is that A2(0) > 0 for each 8 E 8. We consider the risk function

R(0, 2) = C,,es R (0, & )w(A) for this problem, where w(A) > 0 for each A E Sz,

and Vx R IN"XN (the reader is referred to previous chapters for the above notation as well as

subsequent notation). Note that P is a Gaussian family of distributions.

Suppose V(8, ~D, R) < +oo and V(8, L, R) < +oo. When 8 is bounded and S, is finite, it
is obvious that V(8, ~D, R) < +oo and V(8, L, R) < +oo. In the case that S, is finite and 8 is

not compact or the case that S, is not finite, it is not necessarily the case that V(8, ~D, R) < +oo

and V(8, L, R) < +oo. It is outside the scope of this work to derive the exact conditions for

V(8, ~D, R) < +oo and V(8, L, R) < +oo since we are mainly interested in the case that S,
is finite and 8 is compact. It is sufficient to mention that these conditions will depend on system

theoretic notions such as constructibility, stabilizability, and detectability. The interested reader

is referred to [24], where such conditions are derived for the special case of uncertainties in the

initial state.

Let us show that Conditions 4.1, 4.2, and 4.4 hold for the estimation problems (8, ~D R )

and (8, L R ) for each Ae E and that Condition 4.3 holds as well if 8 is compact. Fix

Ae E Our first step is to show that Conditions 5.1-5.7 hold. Since the loss function Lx is

quadratic, it is only left to show that Conditions 5.2 and 5.4 hold (Chapter 6). Consider the

conditional distribution of Yh( ) given X = x, when 8 is the true state of nature. Let Po^(-|IX = x)

denote this conditional distribution, which is certainly Gaussian. Moreover, since A2(0) > 0 for
e~tac r a 8, thefamly,~x~ { o ( X= ) : 0 E 8, X EX } is dominated by the Lebes gue-B orel

distribution of Yh( ) given Xx = x when 8 is the true state of nature. Let Ae B (i( 3)) be such

that p l(A) = 0. Then Po^(A|X = x) = 0 for each x E X. It can be verified that this implies that
Po^(A|X = x ) = for each xx EX 3. Thus~ then family {P (-X~ =( x ) : 0 E 8, XX E3X }

is dominated by pr and Condition 5.2 holds. Put Z: = [Y~, X~ ]. L~et {0,}) be a sequence in 8

that converges to 8o E 8. Then rli(0,) i qi(80) and As(0,) A s(00), for i = 1, 2, 3. It can

be verified that this implies that Eo, (Z) Eeo (Z) and Eo, (ZZT) Eeo (ZZT). It follows

that Po\, converges weakly to Po>,. Since the sequence {0,} is arbitrary, {Po\}) converges weakly

to PoX whenever {0,t} converges to 8o, {O,z} E 8, and 8o E 8. It follows that 8 Po^(A) is

B(8)-measurable for each Ae B (Pihz)) x B(3x ) [40].Thus Condition 5.4 holds. It is also

clear that 8 H Ea(Z) and 8 H Ea(ZZT) are continuous on 8. It now follows from the results

of Chapter 6 that Conditions 4.1, 4.2, and 4.4 hold for (8, ~D R ) and (8, L R ) for each

A E S,., and Condition 4.3 holds as well if 8 is compact. By Theorem 10.1i, whether (8, ~D, R)

or (8, L, R) is considered, there exists a minimax estimator, and there exists a restricted risk

Bayes solution relative to (-r, Co) for each -r Me Zl and Co > V(8, ~D, R). Suppose in addition

that S,. is finite and 8 is compact. Then we have the following results for the estimation problem

(8, ~D, R): (8, ~D, R) is strictly determined, there exists a least favorable a priori distribution

I-o E Me/l and a conditional mean estimator relative to -ro is an essentially unique admissible

minimax estimator. Moreover, the class of conditional mean estimators is essentially complete.

Note that, in general, a conditional mean estimator relative to -r is not a LMMSE estimator since

-r may not assign mass 1 to a single point in 8. Similarly, we have the following results for the

estimation problem (8, L, R): (8, L, R) is strictly determined, there exists a least favorable

a priori distribution -ro E Me/l and a LMMSE estimator relative to -ro is an essentially unique

admissible minimax estimator. Consider the filtering problem. Then a LMMSE estimator relative

to -r is not necessarily a KF if -r does not assign mass 1 to a single point in 8. There is a special

and important case in which a LMMSE estimator with respect to -r M y/1 is a KF. We will treat

this case in the sequel.

In the following chapters we consider two special cases of the above problem. The first case

is the case of stochastic uncertainties in the initial state, model noise, and observation noise with

L as the class of available estimators. The second case is the case of deterministic uncertainties in

the initial state with ~D as the class of available estimators. In both cases we will assume that S,. is

finite and 8 is compact. Note that by Remark 4.1, the case that 8 is bounded, but not necessarily

compact is also covered. These two cases are important on their own merit, and they will also be

used to illustrate some of the general results of previous chapters. For example, the first case will

be used to illustrate the method proposed in Chapter 9 to derive a restricted risk Bayes solution

(Remark 9.3) and the second case will be used to illustrate the method proposed in Chapter 8 to

derive an approximation for minimax estimators.

CHAPTER 12
AFFINE STATE ESTIMATION BASED ON QUADRATIC LOSS FUNCTIONS

In this chapter we consider a special case of the state estimation problem of Chapter 11. We

consider the case of state estimation with stochastic uncertainties in the initial state, model noise,

and observation noise with the class of available estimators being the class of affine estimators.

Throughout this chapter, we assume 8 is compact. Thus V(8, L, R) < +oo. In this chapter,

we assume that the estimation space (S,, S,, h) is such that S, = {0, 1,..., nt}, S, = N, and

h(k) = {0, 1,. ., Gk } for each k E S,, where at and nk, for k = 0, 1,. ., nt, are nonnegative

integers. We assume that w(k) = 1 for each k E S,, i.e., R(0, 2) = C"' Rk(8, k~). Given

Co > V(8, L, R) and ve 8 our goal is to find a restricted risk Bayes solution relative to

(v, Co).

12.1 Finding a Restricted Risk Bayes Solution

First, we want to derive a closed form expression for R(0, i). Let F,, = FF _1 Fj (i >

j) and Fi~i = Fi. We will use the convention Fi~i 1 = I and Fi,y = 0 if j > i + 1. Let

0,, = H' Fo To, FoH2T n T-1,0Hlf]".Let ( = [0 HJ 1 FH2T nT-1,1HTI] =
[0 1 0, H2 F2HT'""FT1, 2HI 000H3 -,H],.. 00 I

Let I, = [1, 1, 1"]. In addition, Let I( = 0 for 1 > n. Let y" = [yoT yT yT]. Note that

Yh(k) = Unr It is easy to verify that

y"n = OnXO + E n I- + u, (12-1)

Zk Fk-,0,z 0 P -1it -1, (12-2)
i=1

where w" = [wo" wT -- (] and u = [VTo VT -- Let k, > 0 and n > 0. By (12-1) and

(12-2),
nVk-1

i=0

For each k E S, and Zk, = Ayo" + be E k, let As, = A and be, = b. Let Wk, = VkT ~, for

k = 1, .., us. By our assumptions regarding the noises and the initial state,

Rk8,k)= tr(VW(As 0,, A-1,_lo)A(0)(As 0,n FI-i,o) VkT>

nkVk-1]

i=0
+2~ 71/ ~b, WkI(As 0,~ F-1,o)917(e)+ |Kb,|2.12 (12-3)

Let r1(0) = [r1(0)T ra2) rl3(0) ]T and let A(0) be the block diagonal matrix with Az(0),

A2(0), and A3(0) in its diagonal blocks. Let Ay,4(0) = As(0) ri(0)ri(0)T for i = 1, 2, 3.

We assume that some a priori information regarding the true state of nature 8 E 8 is

available. The case that no a priori information is available is an important special case. The a

priori information is given in the form of a nominal v E 8. For simplicity, we assume rll(v) = 0.

There is no loss of generality in this assumption since if rll(v) / 0, we can translate the states

and observations and bring the problem to this form. Below, we summarize the assumptions

taken so far together with some new assumptions.

Assumption 12.1. ve 8 and rl(v) = 0.

Assumption 12.2. 81 is a compact and convex subset of~ and (-rl, A) E 81 whenever

(rl, A) E 81.

Assumption 12.3. 82 is a compact and convex subset of % and for all 8 E 8, rl2 (0) = 0 and

A2 (0) > 0.

Assumption 12.4. 83 is a compact and convex subset of % and r/3(0) = 0 for all 8 E 8.

The assumptions that 84 is compact and convex for i = 1, 2, 3 can be somewhat relaxed,

but, for the sake of clarity, it is advantageous to make these assumptions. The assumption that

(-rl, A) E 81 whenever (rl, A) E 01 is rather weak and is satisfied in many important cases. This

assumption is clearly satisfied in the case that 81 = {((9, Al + rlrl) : rl E FI, Al E 02}, where

01 = {9l : (rl, A) E 01} and 02 = {A rlrl : (rl, A) E 01}, and El is symmetric around the

point 0, i.e., rl E FI if and only if -rl E Fr. It is assumed that Assumptions 12.1- 12.4 hold in the

sequel. Note that 8 is then a compact convex subset of C?.

By (12-3), Rk k~) is both convex and concave on 8 for all Ak E k~, i.e., for any

0 < a~ < 1 and 8', 0"1 E 8,

In addition, Rk ', k) is COntinuous on 8 for all ik E k~. It follows that R(-, 2) is both convex

and concave on 8 and continuous on 8 for all i E L. Let r(-r, 2) = fe R(0, 2)dvr and let

Tk-r To ) = So k 8, k~)dTr for k = 0, 1,. ., us. Let Z be the class of all finite subsets of 8. Let

Me/l denote the space of distributions in Me/l with finite support. Let -r Me Zl. Then there exists

Z E Z and 81,...,0z2 E 8 such that Z= {01,...,0z }) and r(Z) = 1. In this case,1let r(i)

denote the mass that -r assigns to the point Of f~or i = 1,...,1 |Z. Finally, let H o r = C1 Osir(i).

Since 8 is convex, 8 o re 8 Since R(-, 2) is both convex and concave on 8 for all ie E ,

Lemma 12.1. Let -r* E Me/l. Then there exists 0* E 8 such that r(-r*, 2) = R(0*,i) for all

is L As a consequence, & is a Bayes solution relative to -r* if and only if & is a Bayes solution

relative to 0*. Similarly, ri(-r*, is) = Ri (0*, ~i) for all ~i E 4 and ~i a Bayes solution relative to

-r* if and only if is is a Bayes solution relative to 0* (i E S,).

Proof Fix is E Since 8 is compact, it is separable. Thus the space of distributions with finite

support, Me~l, is dense in Me/l in the sense of weak convergence [27, Appendix 3]. Let {74}) be a

sequence of distributions in Me/l that converges weakly to -r*. Since R(-, 2) is continuous on the

compact set 8, R(-, 2) is bounded and

/ R(0,,, 2)r im (0 )do (12-4)

Let Of = 0 o nr (i = 1, 2, ..). Since 8 is compact, there exists a 0* E 8 and a sub sequence {Of f

such that {0fm } converges to 0*. Since R(-, ) is continuous on 8,

R(0*, 2)= =limn R(Ofm ). (12-5)

Since R(-, 2) is both convex and concave on 8 and -r has finite support,

/ R(, 2du R(O ) ( = 1 2,. ..).(12-6)

By (12-4)-(12-6), r(-r*, 2) = R(0*, 2). The proof follows from the arbitrariness of 2. O

It is clear from the proof of Lemma 12.1 that if -r* e Me/, is a Bayes solution relative to

-r* if and only if & is a Bayes solution relative to 0*, where 0* = 0 0 -r*.

We want to apply the results of Chapter 9 for the estimation problem (8, L, R). We have

already shown that Conditions 4.1-4.4 hold in Chapter 11. Let us show that Conditions 7.1, 7.2,

and 9. 1 hold as well. In Chapter 11 it was shown that L has the property (W*) and Condition 4.4

holds. Thus Theorem 4.2 implies that Condition 7.1 holds. Certainly the weaker Condition 9. 1

must hold as well. It is left to prove that Condition 7.2 holds for (8, L, R). In fact, it is sufficient

to prove that Condition 7.2 holds for (8, k, k~) for each k E S,. Fix k E S,. For the sake

of clarity, we prove that Condition 7.2 holds for (8, k, k~) in the case that r11(0) = 0 for all

8 E 8. It is easy to verify that this is true also in the more general case of Assumption 12.2 but

the expressions are rather cumbersome. Assuming r11(0) = 0 for all 0 E 8 and using (12-3),

Rk 8 ~k) = tr.( /Aiyk,/1(0)A Vk ) 2 tr.( /~A~iY k,2(0) VkT)

+ tr( 7kI,3(0)Vk')+ + |Kbyl |2, (12-7)

where

7k,1 n) Az Ch(0)O(, + I,,, O A2(H ,~, y 1 A3(),
nlkAk
7k,2(B k-1,0Az(0)O(~B + Fk-1,iA3)~k)

i= 1

Recall that ||2'k Ok~ = ||A. A. || I I. b~I ~., LII~ iS a norm, making k

into a normed space.

Let it,k E k~ denote the (essentially unique) Bayes solution relative to -r Me Zl. It is not

difficult to show that

Aenr = Yk,2(r)7. Y,(7) and ban~ = 0, 128

where we extend yk~i from 8 to Me/l by defining yk~i 7r) = yk,i 8 o Tr) for all Te ME Zl (i = 1, 2).

Let {-ri} be a sequence in Me/l that converges weakly to -ro E Me/l. By Lemma 12.1i, there

exists a sequence {Os} E 8 and an element 8o E 8 such that Rk(8, 74,k) k R(8, Os,k)

(i = 0, 1, ..). Certainly yk~i is COntinuous on 8 for i = 1, 2. It follows easily from (12-8)

that the mapping 8 H O,k is COntinuous on the compact set 8 and hence uniformly continuous

and bounded. Thus there exists p > 0 such that SO,k E Bp for all 8 E 8, where B, = {ik E

Ck : k1~1~ I p}. It is easy to see that RkIS isCOntinuous on the compact set 8 x B,. Thus

{Rk~(0, -) : 0 E 8} is equicontinuous on B,. Since 8 H 20,k is UnifOrmly continuous on 8, the

family {0 Rk (8 8,~) : 8' E 8} is equicontinuous on 8 and therefore Rk (8, 0s,k~) COnverges

to R, (8, Bo,k~) UnifOrmly on 8. It follows that R, (8, 74,k) COnverges to Rk(8, To,k) UnifOrmly on

8 and Condition 7.2 holds.

Let 2, denote the (essentially unique) Bayes solution relative to -r Me Zl in the estimation

problem (8, L, R). Since 8 is compact and R(-, 2,) is continuous on 8, Co < +oo and

Condition 9.2 holds if V(8, L, R) < Co. Let AP = Ap, +(lp)r. By Lemma 9.3, AP is a Bayes

solution relative to -r in the estimation problem (8, L, RP). Let rp(,, 12) = pr(v, 2) + (1 -

p)r(-r, 2). Let r"P(-) = rP(,, AP). Since Conditions 4.2-4.4, 7.1, and 7.2 hold for the estimation

problem (8, L, R), they hold for the estimation problem (8, L, RP) for 0 < p < 1 (Remarks

9.1 and 9.2). Thus using the results of the previous chapters and Lemma 12.1i, if 0 < p < 1,

there exists 8o E 8 such that r"P(8o) = supoe, r"P(0) and "0 is an admissible, essentially unique

minimax estimator. In particular, by setting p = 0, there exists a minimax estimator for (8, L, R).

Let us consider the problem of finding a minimax estimator for the estimation problem

(8, L, RP), where p E [0, 1). As mentioned earlier, if V(8, L, R) < Co < Co, the solution of

this problem for p e (0, 1) is necessary in order to find a restricted risk Bayes solution relative

to (v, Co) using the method of Remark 9.3. The case p = 0, corresponds to minimax estimation.

Note that if Co > Co, the problem is reduced to regular Bayes estimation and is is the solution.

The case Co = V(8, L, R) corresponds to minimax estimation.

Let 0o = {0 E 8 : r1(0) = 0}. Let us show that 0o is a convex and compact subset of

8. Fix 0 and 8' in 80 and 0 < a~ < 1. Let On = caO + (1 a~)0'. Since 8 is convex, On E O.

Clearly r1(8a) = 0. Thus On E 0o and 80 is convex. Let {Os} be a sequence in 0o that converges

to 8o E 8. Then by Assumptions 12.2-12.4, r1(8o) = 0. Thus 80 is closed. Since 8 is compact,

80 is compact. Let to = {ie E : by, = 0 for k = 0, 1,..., nt}.

Lemma 12.2. Let 0 < p < 1 and consider the estimation problem (8, L, RP).

Consider the following algorithm:

Step 1: Choose 01 E 80 and let i = 1.
Step 2: Findl Of a 80 sucrh that R(Of i ) = supoeaO R(8, if ).

Step~ ~ ~~~V 3: IfR(s i )=R(f, ) then stop; the distribution Os is a least favorable a priori
distribution.

Step 4: Let 80,i = cli + (1 a)04 for asE [0, 1]. Find to E [0, 1] such that r"P(0a,,i)

supoe [o, 1] TP(H0,i).

Step 5: Put 8i41 = 0o,~i, let i= i+1, and return to step 2.

Then the sequence {Os } is in Oo, it converges weakly to a least favorable a priori distribution

8o E 0o, and ifo, is a minimax estimator 1Moreover the sequence {if } is in to and RP (0, if )

converges uniformly on 8 to RP (0, fo), >

Proof We claim that for any 8 E 8, there exists 0* E 80 such that r"P(0) < rP(0*). Indeed,

fix 0 E 8. Let 8' be such that A(0') = A(0) and r1(0') = -r1(0). It can be verified that since

rl(v) = 0, r"P(0) = rP(0'). Let 0* = ( + '). Then 0* E 0o. By Lemma 7.1, ?"(0) < ?"(0*).
Thus there exists a least favorable a priori distribution 80 that is in 0o. Certainly 8o is also a

least favorable a priori distribution in the estimation problem (0o Pp). Since 0 < p < 1,

RP(0*, 2) = supose RP(0, 2) if and only if R(0*, 2) = supoe, R(0, i). Since 0o is a compact
subset of 8, we may apply Theorem 7.1 for the estimation problem (0o Pp) and derive the

above algorithm. Note that the algorithm is simplified with the help of Lemma 12.1 since we

need to consider only distributions with support of a single point, i.e., elements of 8. Since we

have considered the estimation problem (0o Pp), the sequence {04}@, is in 0o. By Theorem

7.1, the sequence converges weakly to a least favorable a priori distribution 8o E 0o. Thus "0 is

a minimax estimator. Since r1(04) = 0 and rl(v) = 0, if a Lo for all i > 0. Since Condition 7.2

holds, RP (0, ') converges uniformly on 8 to RP(0, Fo).

The main steps of the algorithm of Lemma 12.2 are Steps 2 and 4. In Step 2, we need to

solve the problem of finding a maximizer of R(-, 2) over 0o. We will address this problem in

Section 12.2. Step 4 can be done using numerical methods as described in Remark 7.1.

Suppose V(8, L, R) < Co < Co. Then by Theorem 4.2, there exists a restricted risk Bayes

solution relative to (v, Co). If a minimax estimator for (8, L, RP) and the supremum of its risk

can be calculated for all p e (0, 1), the method discussed in Remark 9.3 can be used to find a

restricted risk Bayes solution relative to (v, Co). By Lemma 12.2, we can assume that a minimax

estimator for (8, L, RP) is in Lo for all p E (0, 1). It can be verified, using (12-3), that if & e to,

there exists 0* E 80 such that R(0*, i) = supose R(0, 2). In general, given an estimator is to~

it may be very difficult to calculate suppose R(0, 2) and the complexity of this calculation may

vary significantly according to 8. Nevertheless, in Section 12.2, we address this problem and are

able to solve it for some important cases of 8.

12.2 Finding a Maximizer of the Risk

In this section, we consider the problem of finding a maximizer of R(-, 2) over 0o, where

is L o, i.e., given~ i Lo, we want to find an element 0* E 80 such that R(0*, 2) =

suppose R(0, 2). This problem is important since we encounter it in step 2 of the algorithm

of Lemma 12.2 and in the method to find a restricted risk Bayes solution, which is discussed

in Remark 9.3. We consider specific cases of the parameter set 0o. There is one immediate

case in which this problem has a simple solution. The definition of an extreme point is needed

(Definition 21). Recall that given a set A, we use dA to denote the extremal boundary of A,

which is the set of all extreme points of A. Since 0o is convex and compact and R(-, 2) is convex

and continuous on 8, by Bauer's minimum principle [41],

sup R(0, i) sup R(0, i).
8680 B6880

If 880 is finite, suppose R(8, i) = maxceaeo R(0, 2). Thus suppose R(0, i) can be easily

calculated.

Fix is L o. By (12-3), R(0, i) = f, tr(As(0t)We) f~or some 91 E SN. 942 6 YN' and

93 E Nzi. Let #4 = {As(0) : 0e 8}0 for i = 1, 2, 3. Let # = {(A(018,n(), A2(0), A3 e E O -

It follows from the definition of 8 that # = GI x #2 x 3. Thus

sup R(0, i)= sup tr I(AsW, I ) sup tr(As~i).
8680 (Ayh,AgA)E6 AgEs

Therefore, we are left with the following optimization problem:

Given NV > 0, a matr~vixr We S"nd a convex compact subse~t A o S,nr

maximize tr(AW) subject to A E A. (12-9)

Consider the important case in which A = { Ae S : ft (A) < ., fr (A) < 0 }, where

fl,. ., frv are convex (real-valued) functions such that A is compact and convex. Then (12-9) is

equivalent to the following convex optimization problem with generalized inequality constraints:

minimize tr(AW)

subject to f()<,i ,.,

A > 0.

Convex optimization problems with generalized inequality constraints can be often solved

numerically as easily as ordinary convex optimization problems [42, pp. 167]. Thus in many

important cases, (12-9) can be solved numerically. In the rest of this section, we consider several
cases in which (12-9) has an analytical solution.

Let Ao a Si" and De R WX" be a nonsingular matrix. Let Wr = (D- ) W~D-l and

el >ea 02 > N denote the eigenvalues of W. Let Be S and At >X~ A2 N be the

singular values of B. Then by a trace inequality of von Neumann [43],

tr(B t)~ <: Agg (12-10)
j= 1

We consider the following possibilities for A:

1) A = {A E S" : tr(DADT) < 1}. Fix AE A. LetX At X~> 2 XN
denote the eigenlvalues of DAD"7. By (12-10), tr(AWrI) =-- trDD 9)
tr AW) < tr DADT Q1. Since A E A, tr A) < Q1. Let A* = D-1UUT D- ), where fi is the

eigenvector of W corresponding to go. Then A* E A and tr(A*W) = gi, whence A* maximizes

tr(AW) over A.

2) A = { Ae S : | |D(A Ao)DTII | |, 1, where |I | || is the Frobenius norm.
Let At >X~ A2 N denote the singular values of D(A Ao)D By (12-10) and
Cauchy-Schw arz inequaity, tr((A Ao) < EAl {g.Tu

tr(AW) < ||D(A Ao)DT||y||W||y + tr(AoW). Since A E A, tr(AW) < ||9||,l + tr(Ao ).

Suppose A* E A and tr(A*W) = ||9|| + tr(AoW). Then

tr (A*n,~ Ao)D = 1.

Since the assignment (A|B) = tr(AB) yields an inner product on the space of real-valued

NV-by-NV matrices and ||D(A* Ao)DT||, < 1, we have by the Cauchy-Schwartz inequality

that D)(A* Ao)D7 = ~iT'hus A* maximizes tr(AWI) over A if and only if A* =
Alo + D1'I(D 1)T

3) A = {A E S" : ||D(A Ao)DT||2 < 1}. Fix A E A. Let A1>X~ A2 N
denote the singular values of D(A Ao)DT. By (12-10), tr((A Ao) ') < X1 CE= gy. Thus

tr(AW) < tr(W)||ID(A Ao)DT||2 + tr(AoW). Since A E A, tr(AW) < tr(W) + tr(AoW). Let
A* = Ao + D-I(D-1)T. Then A* E A and tr(A*W) = tr(W) + tr(AoW), whence A* maximizes

tr(AW) over A.

4) A = {eAo + (1 e)A : ||DADT||y < 1}, where 0 < e < 1. Certainly A*=

eilo + (1 e) D-1 1)T maximizes tr(AW) over A(. We canl replace | | || in the definition of A

by tr(-) or || ||2~ and have analogous results.

Let us consider an example in which we find a maximizer of tr(At Wi) over Gr.

Example 12.1. Suppose the set 31o is compact in the sense of the usual Euclidean norm and

symmetric around the point 0, i.e, xo E 310 if and only if -xo E 31o. Let MZ/xo denote the set of all

probability measures on (31o, a(31o)). Suppose it is known that the true distribution of the initial

state belongs to MZ/xo. While MZ/xo has elements that are not Gaussian, for each such element

there corresponds a Gaussian distribution with the same mean vector and autocorrelation matrix.

Since the risk functions depends only on the mean vector and autocorrelation matrix, the results

of thi\ chanpter can be used~. LePt 81 = {((9, A) : 17 = fxoxody, A = fxoxrox~r, reMxo
Since 31o is compact, MZ/xo is weakly compact. Let {-ri}@, be a sequence in MZ/xo that converges

weakly to -ro E MZ/xo. Then clearly {((qi, As) }g, converges to (rlo, Ao), where rli and As are the

mean vector and autocorrelation matrix of -ri (i = 0, 1, ..), respectively. Since MZ/xo is compact,

81 is compact. Since MZ/xo is convex in the usual sense, 81 is convex. Fix (rl, A) E 81. Let

r E MZ/xo be such that rl and A are the mean vector and autocorrelation matrix of -r, respectively.

Let -' be defined as follows: for every Ce B (3o), -r'(C) = -r(-C), where -C = {-xo : xo E C)

and is in B(31o). Ct, icirrh -' E MZ/xo, r' = -rl, and A' = A, where rl' and A' denote the

mean vector and autocorrelation matrix of 7', respectively. Thus (-rl, A) E 81. Recall that

GI = {At (0) : 0 E 8 }. We want to find a maximizer oftr(ARl) over Gr.

Suppose 3o = {xo E RWN : x"DTDxo < 1}, where D E RWN~xN is nonsingular

Fix Ae t r. Then there exi'Fs ~tse Mxo such? that AZ = fXoxoxz~cdr. Th~us tr(DADT')=

fxo tr(Dxox"D..TT)d~r < 1. Let Ae S N.1 and suppose tr(DADT) < 1 and A f 0. Then
C As < 1, where As is the eigenvalue of DADT corresponding to the ith eigenvector ui.
Certainly DADT = C Asn.LT Thus A = C Asibil', where As = Ag/(C Ay) and fig =

CjDI14 for i = 1, N. Since, ifDTZ)i = C As < 1, ~ig E Xo and -~ig E Xo

for i = 1, .. ., Nz., Let -r* be the distribution that assigns mass Ag/2 to fig and Ag/2 to -fig

(i=1, I. N). Thenr r* E MxoLs, has zero mean, and A = x xox~d~rdr*. Thus Ae E 1 It follows
that At = {A E SN. : tr(DADT) < 1}. This case was alreadyv treated in rlhi\ section.

Suppose 31o is a convex polytope that is symmetric around the point 0. Then there ex-
ist a finite number of points Xo,1, X0,2, 0 X,N E 30 Such that xo,i and -xo,i are extreme

pointsf of Zofor i: = 1, 2, ..., N. Certain~ly sup~,,l tr(A~l) = suprerxo So tr(xrox Wi~)dv <
suzeotr/xo -W'1). By Bauer's minimum principle [41], supzoexo tr/xo "W'1) = supzoeaxo tr(xox ~).

Since 31o has a finite number of extreme points, sup,,,, tr (A"
Since xoi,~ix E 01 for i = 1, ..., NV, suphE4 tr(A~l) = Inaxli~aN tr(xo,ix,~i 'Y) andU therelt

exists 1 < i < NV such that xo~ix", maximizes tr (Al) over Gr.

12.3 Connection to the Kalman Filter and E-Minimax Approach

Let igln denote an estimator for xm based on the observations II,,. Let F(0, iml,) denote the

MSE matrix of the estimator iml, when 8 is the true state of nature. Let 2,(0) and in,_-1(0) be

defined by the well known update equations of the KF:

Bo -1(e) = r11(0), (12-11)

Fo-()= A,~,(0), (12-12)

K,~(0) = Ps,_n1(0)HI[Ay,2( + H,F4,l_l(0)H ]- (12-13)

8,(0)=e>10 ,0)[ -Hi,10] (12-14)

r,(0) = [I K,(0)H,]F, (0)(8, (12-15)

in1()= F,2,(0), (12-16)

I', ,l(0) = Ay,~3(H) + Elnful0)Fu (12-17)

where 0,(0) = F(0, 2,(0)), r,,,_,(0) = r(0, l4,,_,(0)), and K,(0) denotes the Kalman gain.

Consider the filtering problem specified by the estimation space (S,, S,, h), where S,=

{0,. ., 71<}, S, = N, and h(i) = {0,. ., i} for each i E S,. Then is = (o (8), 11(8), in, (8>>
and is referred to as the KF relative to 0. Since A2(0) > 0, the existence of the KF is guaranteed

[44]. It is easy to verify that Rk (8, k (8 )) = trW1~B k8, k8 ))).

Using the results of the previous sections, there exists a 0 E 8 such that the KF relative to

8 is a restricted risk Bayes solution relative to (v, Co) for all Co > V(8, L, R). In fact, there is

another interesting property regarding the KF. Using Corollary 9.1 and Lemma 12.1i, we have

that the class of KFs relative to H E 8 is essentially complete. Thus as long as the performance

is judged solely based on the risk function, if the choice of estimators is restricted to affine
estimators, then no matter what optimality criterion is used, one may consider only the class of

KFs relative to H E 8.

Our next step is to derive more convenient expression for Rk(8, k~(0')) in the case that 8 and

8' are in 0o. Let Uk 8, 8/ k (,~ (0')). Using (12-13)-(12-17), it can be shown that

Fo 1(, ')= A (0)

', (0, H') = K, (0') A2 0 K, (0') + [I K, (0') H,] 4, (0, H') [I K, (0')H,]T

Since R, (8, k ,8 )) = tr( k k(0, 8')), we have a more efficient way to calculate the risk

than through (12-3). Recall that if H and H' are in 0o, then R(0, fe,) = C,_ tr(A4i(0) We(0)).

Our goal is to find an expression for We(0'), for i = 1, 2, 3, since it is needed in order to find a

maximizer of R(-, is,) over 0o

Let F,(8/) = F,-Fx,t(BI)H,, 1et Fi,j(0') = i(0')Ni_,(0') Fj(0') (i > j) and Fi,i(0') =

Fi(0'). We will use the convention E-_l1,4(0') =ILe rm0)=(I-Kt()r)-,m)

(0 < m < n, O < n), let Dn,m(8/) = Gr,m+1(8/)F, >Ka(0) (n > m > 0) and D,~,(0') = K,(8/)

(n > 0). Note that C+1~,m = (I K,+1H,t+1)Exs,m and Cr,-1 = Gr,mF,-1. Then

Gro(0')Az(0)Gro(0') + Dnm(0')A2(0)Dnm(0') + Grm(0')A3 8 rtm 8
m= o m= 1

r,(e, e')

Let Ti(0') = CE" >= ,~i(BI) 914zGi(0') for i = 0, 1, .., us. Then

It follows that

n=0

n=0 m=0

m=1 m=0

n=l m =l m =

In general, the calculation of W2(0') and 93 0 ) TCCJUifeS the storage of Ko(0'), .., K,t (8'),

which may be problematic for large us. This is due to the fact that Ti can be updated based on

Ti+l but not vice versa. Nevertheless, in the important case that R, is invertible for 0 < i <

us 1, the calculation of Wi, 92, and 93 may be done in such a way that there is only a need of

a fixed storage place that does not depend on us. First, let us show that since A2 0 ) iS invertible,

I K,(0')H, is invertible. By (12-13),

I K(0')H = I 0,1(0')H [nA28) HnF4,_1(0)H ]- Hn

Thus by the matrix inversion lemma, I K(,(0')H1, is invertible and

(I K,(0')H,)-1 = I + 04,_-1(0')If"A2 pl) -1H,.

Since Fi is invertible, T441 = [(I K Hi)- F- ] T4I KeH ]- F-' (F- ) H'sF- In this

case, Ti41 can be updated based on Ti and only a fixed storage place is needed.

We now discuss the connection between the restricted risk Bayes approach and the 0-

minimax approach and illustrate that the F-minimax approach can be regarded as a special case

of the restricted risk Bayes approach. The class 0 of a priori distributions in the F-minimax

approach coincides with the class 8 of the states of nature in the restricted risk Bayes approach.

By setting Qo = V(8, L, R), we have that for any -r Me Zl, a restricted risk a Bayes solution

relative to (-r, Qo) is a minimax estimator for (8, L, R). Since 8 = 0, the risk R(0, x) is, in fact,

the Bayes risk relative to a certain distribution in 0 if we adopt the F-minimax formulation. Thus

a minimax estimator for (8, L, R) is a F-minimax estimator. Therefore, the results of this work

can be used to find a F-minimax estimator. We note that if some a priori information is available,

the restricted risk Bayes approach is preferable to the F-minimax approach since it utilizes this

information. However, if no a priori information is available, the most reasonable choice for C6

seems to be V(8, L, R) and hence we are left with F-minimax estimation.

12.4 Numerical Example

To illustrate the theory of the previous sections, we consider the following simplified

problem as an example. Suppose a target is moving in the one-dimensional space. We assume

the three-state track model [45]. Let x, E RW3, where x,(1) denotes the position of the target,

x,(2) denotes the velocity of the target, and x,(3) denotes the acceleration of the target. We

also assume a radar measures the position of the target. Hence the state space model in (11-1) is

specified with

1 a A2/2

Fu = 0 1 a

00 1

Ir,, is a zero-mean random vector with nominal covariance matrix

000

G~o = 0 0 0

H, = [1 0 0], and v, is a zero-mean random variable with nominal variance Xo. We assume

xo E 31o, where 31o = {xo E RW3 : x DTDlxo < 1} and that a nominal distribution for xo

is available. The nominal distributions of xo, In~,,. and v, are available from past experience but

are not assumed to be the exact distributions. The exact distribution of xo belongs to MZ/xo, the

space of distributions defined on (31o, a(31o)). The exact distribution of the measurement noise is

known to belong to the class of zero-mean distributions whose variance A satisfies | A Xo I I x

for some rx > 0. Similarly, the exact distribution of the model noise is known to belong to a class

of zero-mean distributions whose covariance Q satisfies ||ID(Q Qo)DT||y < r, for some rg >
0and De R N"XN. By Example 12.1i, Pi = {AE : tr(D1ADT ) <' 1}. In addition, it s ler

that #2~ = {AES : ~ ||D2( 0) || < "1} and #.3 = {Q ES :- ||D3 0D|y }

where D2 = --1/2 and D31 r--1/2D. In this example, we consider the filtering problem specified

by the estimation space (S,, S,, h), where S, = {0,..., nt}, S, = N, and h(i) = {0,..., i} for

each i E S,. We choose the following values: a = 0.01, at = 120, Ao = 1 x 104, g0 = 100,

rx = 1000, r, = 30,

5 x10-6 0 0

DI = 0 0.002 0

0 0 0.01

2/A2 0 0

0 0 1

We assume that the nominal distribution of xo has zero mean and diagonal covariance with

diagonal entries (1 x 106, 1000, 10). We consider the risk based on quadratic loss functions

as our performance measure and let the weight matrix W,, for each 0 < n < us, have all zero

elements, except only for the unity element in its upper left hand corner. This choice of the

weight matrix implies that we are only interested in estimating the position of the target and we

regard the velocity and acceleration of the target as nuisance parameters.

Clearly Assumption 12.1 holds in this example. We have shown that Assumption 12.2 holds

in Example 12.1. It can be easily verified that Assumptions 12.3 and 12.4 hold as well. Our

first step is to choose Co. In order to choose Co, we need to calculate V(8, L, R) and Co. Once

these quantities are available, we can choose Co based on the amount of a priori information

we have. Let &;r denote the (essentially unique) minimax estimator in the estimation problem

(8, L, RP). We can get an insight as to how to choose Co by plotting the Bayes risk r (v, & )

versus the maximum risk supoe, R(0, & ). We use the algorithm of Lemma 12.2 to calculate
A* for per [0, 1) and then calculate r(v, & ) and supose R(0,~* & .Fig 12-1 showsr the plot of

the Bayes risk r(v, &* ) versus the maxIimumlll risk~ suose p U~aCI ,u,,,,,IU~vr R(0, &* ) Note that byI Theorem1 9.1,

this figure, in fact, shows the Bayes risk achieved by a restricted risk Bayes solution relative to

(v, Co) versus Co. Hence this figure tells us the tradeoff between the penalty on the Bayes risk

and the safeguard on the maximum risk by employing restricted risk Bayes estimation. In this

example, V = 1.225 x 10s and Co = 6.474 x 106. We can see from the figure that on one

hand, it would make little sense to choose Co > 8 x 10s since the improvement in the Bayes risk

would be minor and the degradation in terms of the maximum risk would be very significant. On

the other hand, if we choose Co to be very small, we would have only a minor improvement in

terms of the maximum risk and a significant degradation in terms of the Bayes risk. Therefore,

it seems that in most cases, except maybe the case of complete lack of a priori information, we

would choose 1.5 x 105 < Co < 8 x 105. Again, the exact choice of Co depends on the amount

of a priori information we have and therefore is done heuristically. For illustration, we choose

Co = 5 x 105. We calculate the restricted risk Bayes solution relative to (v, Co) using the method

discussed in Remark 9.3 and the algorithm of Lemma 12.2. It turns out that the restricted risk

Bayes solution relative to (v, Co) is a minimax estimator for the estimation problem (8, L, RP"),

where po = 0.986. With this choice of Co, the restricted risk Bayes approach can reduce the

maximum risk to about 1/13th of that of a Bayes solution while suffering only minimally (about

4%) on the Bayes risk.

Fig. 12-1 illustrates the behavior of the Bayes, minimax, and restricted risk Bayes solutions

in two extreme cases: The case that v is the true state of nature and the case that the true state of

nature is the worst case choice for each one of these estimators, respectively. However, Fig. 12-1

does not illustrate the behavior of these estimators for other values of the true state of nature.

It is important to evaluate the performance of these estimators relative to all 8 E 8 that are

likely to be the true state of nature based on our a priori information. In order to do that, we

need to make some assumptions regarding the set of 8 E 8 that are likely to be the true state

of nature. We assume that this set is 8, = {eov + (1 co)8 : e < co < 1, 8 E 8},

where 0 < e < 1, i.e., the true state of nature is likely to be an co-mixture between v and

an unknown 8 E 8 for e < co < 1. The more a priori information we have, the more

likely it is that the true state of nature is close to v and hence the larger e is. Note that for any

:i E L, supose, R(0, i-) = er(v, i-) + (1 e) supose R(0, i-). Let i-* denote the (essentially

unique) minimax estimator and recall that :i-, denotes the (essentially unique) restricted risk

Bayes solution relative to (v, Co). It is easy to verify that supose R(, 'e) = e?"(v) + (1 e)Co,

supeoe, R (0, i*) = er (v, -*) + (1-e) V(8, L, R), and suppose R(, )~ = er (vI, i*) + (1- e) Co

Let us assume that there is a sufficient amount of a priori information so that e > 0.95. Fig. 12-2

shows the maximum risk over 8, of the Bayes, minimax, and restricted risk Bayes solutions for

e E [0.95, 1]. This figure shows the worst case performance of these estimators relative to values

in 8 that are likely to be the true state of nature. The maximum risk over 8, of the restricted

risk Bayes solution is less than that of the Bayes solution for almost the entire range of e. In

the interval [0.95, 0.98], the maximum risk over 8, of the restricted risk Bayes solution is in

fact significantly less than that of the Bayes solution. The maximum risk over 8, of the Bayes

solution is less than that of the restricted risk Bayes solution only for e > 0.9995 and even in

this case, the difference is very small. It can also be seen that even a relatively small uncertainty

in the true state of nature may lead to undesirable performance of a Bayes solution relative to v.

For example, in the case e = 0.98, using the restricted risk Bayes solution instead of the Bayes

solution leads to 50I' improvement in terms of the maximum risk over 8,. In addition, it can be

seen that the performance of the restricted risk Bayes solution is superior to that of the minimax

estimator for all e > 0.95. This illustrates that if a priori information is available, the restricted

risk Bayes approach is preferable to the somewhat conservative minimax approach.

105x104

o Minimax Solution
03

95-
06

9-"
85-
S09

75- FO~
0 975 Unrestricted
Bayes Solution

0 999

650135

Ssupp R(9, f) x lo'

Figure 12-1. Achieved Bayes risk vs. maximum risk

x 105
47

Bayes Solution
Minimax Solution
Restricted Risk Bayes Solution

3 5

3-

25-

15-

05
095

0 955 096 0 965 097 0 975 098 0 985 099 0 995

Figure 12-2. The maximum risk over 8, of the Bayes, minimax, and restricted risk Bayes
solutions vs. e

CHAPTER 13
STATE ESTIMATION WITH INITIAL STATE UNCERTAINTY

In this chapter we consider another special case of the state estimation problem of Chapter

11. We consider the case of deterministic uncertainties in the initial state. We assume that the

sequences {v,} and {tt',,}), which have zero mean, satisfy for n, m > 0

G~v~v0, otherwise

0 otherwise
E(vow )= 0, (13-1)

where E denotes the expectation operator. This means that 82 and 83 COntain only one element.

We restrict ourselves to the case A2 > 0, which is usually the case in well modeled problems

[44]. It is left to specify the initial state vector xo. We model xo as a deterministic (unknown)

parameter belonging to a parameter set No0. We assume 31o is a compact subset of RWN. Note

that the current formulation deviates from the standard KF assumptions in that the initial state is

not modeled as random with known statistics. Since we assume deterministic uncertainty in the

initial state vector, 81 is the collection {(xo, xox ) : xo E o}0 of mean and second moment

pairs. Note that since 31o is compact, 8 is compact.

We are interested the filtering problem that is specified by the estimation space (S,, S,, h),

where S, = { 0,. ., at }, S, = N, and h(i) = { 0,. ., i } for each i E S,. We let w (i) = 1 for

each i E S,, i.e., the risk function is simply given by

n= o

Since there is a one-to-one correspondence between 8 and 31o, we slightly abuse the notation and

assume that the space of states of nature is 31o, i.e., in this chapter 8 = Zo0. We also use R(xo, 2)

instead of R(0, 2), R,(xo, 2) instead of R,(0, 2), for n = 0, ..., us, and etc.

13.1 Conditional Mean Estimators

Recall that rI,, = s yTU T]". Given -r Me Zl, let 2,(-r) denote the mapping

y"n H Exo(X,|y"n), i.e., 2,(-r) is the conditional mean (CM) estimator with respect to -r when

xo is the true state of nature. Let F,(-r) denote the MSE matrix of 2,(-r) with respect to -r. Note

that we may regard a point xo E 31o as an element of M /xo by regarding it as the probability

measure that assigns mass 1 to xo. Thus 2,(xo) and F,(xo) are well defined. Let 2(-r) =

(f-o(T),., i In,(7)j). Recall that r(-r, ) = fxo R(xro, 2)dr. In addition, we use the following

simplified notation: r(-r) = r(-r, 2(-r)). It is easy to verify that r(-r) = CE,o tr(W,F,(-r)), where

We = Vs"v,.
Let o dennote the conditional mean estimator whel~n xo = and let I`o denote the MSE

matrix of fo when xo = 0. It is clear that io and yo can be carllclated usingT the well kmnown KF~

recursions [44] by initializing the KF with zero mean and covariance. It is straightforward to

verify that F,(XO) = 1o for all o r Zo. Lt n 3+ POn FI K = l- n TA

H,o,_ znlHI] -1for n > 0,and ~o= 0. Let F,= F,- F,,KoH,,1 et FL~3 = FF_1--- Fy (i > j)
and Fi~i = Fi. We will use the convention Fi_,i = I.

The problem of state estimation with a random initial state whose distribution in not

necessarily Gaussian is considered by Lainiotis et al [46] and the CM estimator for this problem

is derived. Let io,n(-r) denote the CM estimator with respect to -r for xo based on ti,,, i.e.,

io Sxo") =~~)~ n e o.()=foEx (onr -x)i~) -x)) dr. Using

the results of [46], it is not difficult to get the following set of equations for n > 0:

2,(r) = ain ~ ) (13-2)

C,=( o,),1,1=(I KoH,)F,-1,o, a > 0, (13-3)

Co = I, (13-4)

F()= Fo + CnCo,n('r)C,, (13-5)

U, H AgHo+ i 1,~-,oH~, (Her _llZH, + A2 -1 He e1,o, (13-6)
i= 1

tn (Y") = Ho Ag2 Yo + i 1,oH, (Her _zH,7 + A2 -1 s A _z (13-7)
i= 1
p" (y") oc exp [T U,o- 2xT ty" gy) (13-8)

where the term go is a fu~nction of y1 and its exact calcu~latin on isnnecessarry for our purpose

Thus 2,(-r), the CM estimator with respect to -r for x,, is the sum of two terms; one term is

io, the CM estimator for x, when the system has zero initial conditions, and the second term

is completely specified by io,n('r), the CM estimator with respect to -r for xo. An alternative

interpretation is to regard 2,(-r) as the sum of two terms, where the first term is an estimator of

x, for known zero initial condition and the second term accounts for the effect of the unknown

initial state. A conditional mean estimator with respect to -r is an essentially unique Bayes

solution relative to -r. The expression for the conditional mean estimator is important since an

essentially unique minimax estimator is the conditional mean estimator with respect to a least

favorable a priori distribution. In addition, the class of conditional mean estimators with respect

to -r M i/xo is essentially complete. Let ~D = { (7) : re xo iZ1X)

13.2 Approximations to Minimax Estimators

In order to find the (essentially unique) minimax estimator, we need to derive a conditional

mean estimator relative to a least favorable a priori distribution. Hence we are left with the

following dual problem: Find 7o E Mxo, such that r(7o) =sup,eMso r(7). This dual problem

is conceptually easier than finding a minimax estimator directly by the definition. Nevertheless,

solving the dual problem may still be a difficult task. Thus it may be necessary to search for

sub-optimal estimators in terms of the maximum risk as suggested in Chapter 8. Our goal is then

to derive suboptimal estimators that can give maximum risk arbitrarily close to that of a minimax

estimator. The following Lemmas are derived in [24].

Lemma 13.1. Suppose 31o is compact. Then if {8 } is a sequence in MZ/xo that converges weakly

to To E MZ/xo, then R(xo, Sh))> converges to R(xo, 2(To)) uniformly on No0.

Lemma 13.2. Let f e D Then there exists an extension of R(-, 2) from No0 to an open convex

set 31o C Uc RWN such that R(-, 2) is twice difgerentiable on U. 1Moreover for all xo E 310

D2 2 0o) < 2U,, sup R(xo, 2). (13-9)
zo6Xo

Because of Lemmas 13.1 and 13.2 and the fact that 31o is a subset of finite dimensional

normed space, we can use the results of Chapter 8 to find a y-optimal estimator. In particular if

31o is a convex polytope, Lemma 8.7 can be used to find a y-optimal estimator. In order to find

a y-optimal estimator, we need to set a 0 < y' < y and construct a sufficiently dense finite

subset 31o' of 31o according to the lemma. We then need to find a (y', 3lo')-optimal estimator.

The lemma tells us that this estimator is a y-optimal estimator. A (y', 31o')-optimal estimator

can be found using, for example, the algorithm that is proposed in [38] and that also appears

in [18] and Theorem 7.1. Using Theorem 7.1, it is not difficult to verify that all the necessary

conditions for this algorithm to be valid hold. It worths mentioning a technical difficulty in this

approach. For us to use the algorithm of Lemma 7.1, we need to calculate matxroeXof R(xo, NT)),
which means that we have to calculate the risk R(xo,2(-r)) for each xo E 31ol. However, an

analytical expression for R(xo, (7)), for xo t Xof and 7 e Mzoy, is not available, in general.
Hence R(xo,2(-r)) must be calculated numerically for all xo E 31o'. Thus the resulting numerical

algorithm involves intensive multi-dimensional integration at each step. Due to this difficulty, it

is desirable to use efficient numerical integration methods. In [24] there is a discussion regarding

how the numerical integration can be done efficiently. Also, in [24] an alternative algorithm to

the one of Theorem 7.1 is proposed. It is argued that this algorithm may be preferable to the one

of Theorem 7.1, in terms of the computation burden, since it may require less calculations of the

risk function.

The computational complexity of a y-optimal estimator derived according to Lemma 8.7

depends linearly on the support of -ro. It may be rather high, and it is certainly higher than the

complexity of affine estimators. In Chapter 12 we discussed the problem of restricted risk Bayes

estimation when the class of estimators is restricted to affine estimators. When the class of

estimators is restricted to affine estimators, the minimax problem of this section can be regarded

as a special case of the problem considered in Chapter 12. If we can find an affine estimator that

is also a y-optimal estimator, we would probably prefer the affine estimator. If such an affine

estimator exists, the results of this section are still important since they give us tight lower and

upper bounds on the maximum risk and hence enable us to evaluate the performance of the

affine estimator with the best possible performance in the sense of the maximum risk. We further

illustrate this in the following numerical example.

13.3 Numerical Example

We consider the following problem as an example. Suppose a target is moving in the one-

dimensional space. We assume the following simplified model of motion. Let .r,z E R2, where

.r, (1) denotes the position of the target and .r, (2) denotes the velocity of the target. We assume

r,w+1 = Fr,z + 1,',,, (13-10)

where

and I,',, is zero mean Gaussian random vector with covariance matrix

00 ,

We assume xo E 31o, where 31o is a closed rectangle in RW2. We also assume a radar measures the

position of the target and the observations from the radar obey the following equation

yn = Hx, + v,, (13-11)

where H = [1 0] and v, is a zero-mean Gaussian random variable with variance A2. NOte that

this example is a simplified version of many real applications in which the KF is used [21].

In this example, we assume that 31o = [-1000, 1000] x [-50, 50], a = 0.01, A2 = 2500,

q = 25, at = 10, W, = 0 for n < 5, and W, =, for n = 5,. ., us, where W, = V,TV,.

Our goal is to derive a y-optimal estimator for y = 0.03. This means that the degradation,

when using the derived estimator instead of an exact minimax estimator, is at most 3%b of the

maximum risk. The admissible y-optimal estimator that we derive is a CM estimator with respect

to a discrete a priori distribution.

Let V be a matrix such that the ith column of V is the ith eigenvector of 2Uto. Let 6i denote

the ith eigenvalue of 2Uto. Let y = 0.03 and y' = 0.0023. The above choices of y' results from a

certain tradeoff. The choice of y' determines how dense the finite subset 31o' of 31o and specifies

h~ow close the resulted estimator is to a minimax estimator for (No', D,;: R). If we choose y' very
close to zero, we would decrease |Xo| but we woumld need more iteratinon to find a (y', Ro)-

If we choose y' too large, the dimension of 31o' would grow and hence also the computational

complexity. Let b6() = f--i- Ior i = 1, 2. We construct a finite set Xo' such that Xo' is

(6, V)-dense subset of 31o, o', n 31,b is (6, V)-dense subset of 31,b, where 31,b denotes the boundary

of Ro, and the points (-50, 1000), (50,! 1000), (-50, -1000), and (50, 1000) belong to No'f. The

set 31o' is plotted in Figs. 13-1 and 13-2. Due to the eigen-structure of the matrix Uto, the resulted

set 31o' is not a standard grid. The points in the set 3lof are much denser in the direction of the first

eigenvector than in the direction of the second eigenvector.

Next, we solve for an (y', 31o')-optimal estimator. We set the initial a priori distribution to

be the uniform distribution on 31o'. We then update the a priori distribution using a variant of the

projected gradient method as discussed in [24]. As mentioned earlier, it is possible to use also the

algorithm of Theorem 7.1. Each update increases the Bayes risk. We stop the algorithm when the

required tolerance of y' is achieved. Since a (y', Ro')~-optimal estimator is an e-minimax estimator

for (N ,~ 1D, R) with~ c = V/(o D, R), it is possible to check< whether th~e required tolerance is

achieved using Lemma 8.1i. Let -ro denote the resulted a priori distribution; the resulted estimator

is then 2(-ro), which is a y-optimal estimator. Fig. 13-3 shows the risk R(xo, 2(-o)) for xo E 31o .

The maximum risk on the set 3lof is 1864.6. Due to the construction of 31o', the maximum risk on

31o is less than 1916.1i. In Fig. 13-4 we show the resulted a priori distribution Tro.

It seems unreasonable to compare between the proposed y-optimal estimator with an

estimator that is derived using the F-minimax approach. The reason is that by assuming that

xo is deterministic and belongs to 31o, there is no systematic way as to how to choose a class 0

of distributions. Of course, one can choose E to be MZ/xo, but then a F-minimax estimator can

be shown to be a minimax estimator (i.e., the two problems are equivalent). It may be the case

that a properly initialized KF can have a very close performance to that of a minimax estimator.

If one adopts the formulation of this work, then a rigorous way to initialize the KF is to derive

a linear (or affine) minimax estimator, which is a KF with respect to a least favorable a priori

distribution. In practice, the KF is often initialized with zero mean and covariance a2I, where

e2 iS chosen heuristically. We wish to illustrate that this heuristic method can sometimes lead to

undesirable performance. In Fig. 13-5, the maximum risk of the KF initialized with zero mean

and covariance a21 iS plotted as a function of a e [150, 450]. It is apparent that choosing a

too small leads to a very poor performance. In addition, choosing a too large may also lead to

undesirable performance. The best choice of a in terms of the maximum risk is a = 289.7 and

the maximum risk in this case is 2042.8. Thus for the best choice of a, we have a degradation of

at least 6.5% and at most 10%b in terms of the maximum risk relative to the estimator 2(-ro). This

is not a significant degradation and when complexity is taken into account, it may be preferable

to use a KF that is initialized with a = 289.7. In this example, it is clear that the degradation if

a linear minimax estimator is used instead of a minimax estimator is at most 10%b and may be in

fact much less. This should not imply that the whole construction of i(Tro) is unnecessary. It is

this construction and the calculation of maxsoexo; R(xo, 2(7o)) that enabled us to establish that a

linear minimax estimator has a very good performance in this case.

S20-

10 -0

-0-

-1000 -800 -600 -400 -200 0 200 400 600 800 1000

xo (1) (initial position [m])

Figure 13-1. A full view of Ro',f which is a finite (6, V/)-dense subset of Zo

-46-

S-47-

~-47 5-

S-48-

;-48 5-

-49-

-49 5-

-50- *

-1000 -950 -900 -850

xo (1) (initial position [m])

Figure 13-2. A zoom-in view of the bottom left corner of No', which is a finite (6, V/)-dense

subset of 31o

1900 x

1800-

1700 m

HC1600 m

1500 m

1400
50

1000
0 500

-500
xo(2) (initial velocity [m/s]) -so -1000 xo(1) (initial position [m])

Figure 13-3. The risk of x(ro), the derived y-optimal estimator, as a function of xo E o,~

x 10

1000

-500
xo(2) (initial velocity [m/s]) -so -1000 xo(1) (initial position [m])

Figure 13-4. The a priori distribution -ro, which is defined on 31o

2900-

2800-

2700-

6 2600-

S2500-

S2300-

2200-

2100-

2000-

1900-
150 200 250 300 350 400 450

The maximum risk of the Kalman Filter initialized with zero mean and covariance
O.2I as a function of o-

Figure 13-5.

CHAPTER 14
CONCLUSIONS

We considered the problem of state estimation with stochastic and deterministic uncer-

tainties in the initial state, model noise, and measurement noise using a decision theoretic point

of view. We considered both the case that the class of available estimators is the class of all

estimators and the case that the class of available estimators is the class of affine estimators. We

showed that a minimax estimator and a restricted risk Bayes solution exist when the risk is based

on quadratic loss functions. Under further conditions, a minimax estimator can be found as a con-

ditional mean estimator or a LMMSE estimator relative to a least favorable a priori distribution.

In the case that all the uncertainties are stochastic, we adopted the restricted risk Bayes approach,

which incorporates the use of a priori information to derive estimators that are robust to devia-

tions of the model from the nominal assumed model. We derived a general method to obtain a

restricted risk Bayes solution. When the class of estimators is restricted to affine estimators, this

method can be easily used provided that a maximizer of the risk function can be calculated. We

considered several important cases in which a maximizer of the risk function could be calculated

analytically and showed that in many other cases it could be calculated numerically. Thus we

provided a systematic way to derive restricted risk Bayes solutions. When the filtering problem

is considered, the restricted risk Bayes approach provides us with a robust method to calibrate

the Kalman filter. We also considered the problem of state estimation with deterministic initial

state uncertainty. In this case, we proposed a numerical method to derive an approximation for

a minimax estimator with the possibility to make the approximation as accurate as desired. This

method seems to be especially attractive in the case that the parameter set is a convex polytope in

APPENDIX A
PROOF OF LEMMA 5.2

Let w' be the system of subsets B C w* such that oo a B and C* \ B is a closed, compact

subset of C in the sense of the topology w. The topology w* is such that each set in w* is a union

of a set from w and a set of w'. It is well known that w is, in fact, the relative topology on C. Since

C is locally compact, o--compact, and metrizable, C* is compact and metrizable.

Let us show that the f(a, -) is lower semicontinuous on C* for each a s A. Fix a point

c* E C. Let {c,} be a sequence that converges to c* in the sense of w*. Then {c,} is eventually

in any w*-neighborhood of c*. Since (C*, 0*) is Hausdorff, there exist disjoint open sets A*

and B* such that A* is a neighborhood of c* and B* is a neighborhood of 00. Since B* is

open, it is the union of an element of w' and an element w, and we can assume without loss

of generality that it is in w'. Let A be an open w-neighborhood of c*. Then A U A* is an w-

neighborhood of c*. Since {c,} is eventually in A U B* and eventually in A*, {c,} is eventually

in A. Thus {c,} converges to c* in the sense of the topology w. By the hypothesis of the lemma,

lim inf,,, f (a, c,) > f(a, c*). It follows that lim inf,,, f (a, c,) > f(a, c*). Let {c,} be

a sequence in C that converges to oo. Then for each A E w', {c,} is eventually in A. Thus for

each w-compact subset C' of C, there exists NV > 0 such that c, ( C' for n > NV. Let {C,} be

a sequence of compact subsets of C such that U" zC, and c, Sf C, for (n = 1, 2, ..). Since

lim inf,,, f (a, c,) = supec f (a, c), lim inf,,, f (a, c,) = f (a, 00). It follows that f (a, -)

is lower semicontinuous on C* for each a s A. Clearly f is nonnegative. Since w is the relative

topology, each Co C C that is in B(C*) is in B(C). Let Co a w. Then since Co U co E w* and

C* \0 oo E w*, Co aB(C*). If Co aB(C), it is a countable union and intersection of elements of w;

hence a countable union and intersection of elements of B(C*). Thus Co aB(C*). It follows that

if Co c C, Co aB(C) if and only if Co E a(C*).

Suppose f(-, c) E m(FA~) for each c e C. Then clearly f(-, c) E m(FA~) for each c e C.

Since f (-,oo) = supeec f (-, c), f (-, 0) E m(FA~). Thus f (-, c) E m(FA~) for each c e C*.

Suppose fe E (FA x B(C)). Let A be a Borel set of the extend reals. The set f-1(A) =

{(a, c) E Ax C* : f (a, c) E A}. Then f -1(A) = {(a, c) E Ax C : f (a, c) E A} U {(a, 00):
a s A, f (a, 00) E A}. Clearly f -1(A) = {(a, c) E Ax C : f (a, c) E A} is in FA~ x B(C). Hence
it is in FA~ x B(C*). Since f (-, 0) E m(FA~), the set f -l(-,oo)(A) = {a EA : f (a, 00) E A}
is in FA~. Thus {(a, 00) : a s A, f (a, 00) E A} = f -l(-,oo)(A) x {oo}, and hence is in

FA~ x B(C*). It follows that f"-1(A) E Fg~ x B(C*). Thus fe m(FA x B(C*)).

APPENDIX B
PROOF OF LEMMA 9. 1

Note that for any~i 6 E O < pi, p2 < 1, and 0 < rl < 1,

The concavity of K follows from the fact that for any 0 < rl < 1,

K~vyt +(1 77p2)= inf .K(:i, rlpt + (1 rl)p2)

= rlK(pl) + (1 rl)K(p2 m

where the inequality above is a direct result of (B-1). Eqn. (9-1) implies that K(pl) > K(p2)

for any 0 < pi < p2 < 1. Since K is decreasing and concave on [0, 1], it is continuous except

perhaps at the point p = 1. Certainly K(:i,., 1) = K(1) and K(:i,., p) > K(p) for all 0 < p < 1.

Thus for any 0 < p < 1,

| K(p () |< (f e, ) -K :fe.,1)= ( -p)sup R (0, :f e) r (v, :f e.)]< (1 p) Co

where the last inequality results from our assumption that R is nonnegative. This proves that K is

continuous at p = 1 and thus on [0, 1].

APPENDIX C
PROOF OF LEMMA 9.2

1) It is easy to see that

K(2 2) = K(p2) < K(ix, p2) < K(ix, pt) = K(pt) < K(2 1

(C-l)

where the second inequality is a direct consequence of (9-1).

This implies that K(2, 1) K(2, 2) > K(ix, pi) K(ix, p2). It easily follows that

2) By (C-1), K(ix, p2) K(2, 2) > 0. Thus

(o, 12 2 1-P)[G(2) G(ii)] < I~ )

By part 1) of the lemma, r (v, 2) TU 1 -~l

3) By (C-1), K(2, 1i) K(A pi) > 0. Thus

sup R(0, f t) + pl [G(2) G(ii)] < sup R(0, 2 -
BEe sEe

By part 1) of the lemma, supose R(8, 1) < supeoe R(8, 2 -

REFERENCES

[1] R. E. Kalman, "A new approach to linear filtering and prediction problems," ASIME
Transactions, Journal of Basic Engineering, vol. 82, pp. 34-45, Mar. 1960.

[2] J. Heffes, "The effect of erronous models on the Kalman filter response," IEEE Trans.
Automat. Contr, vol. AC-11, pp. 541-543, Apr. 1966.

[3] T. Nishimura, "On the a priori information in sequential estimation problems," IEEE Trans.
Automat. Contr, vol. AC-11, pp. 197-204, Apr. 1966.

[4] T. Nishimura, "Error bounds of continuous Kalman filters and the application to orbit
determination problems," IEEE Trans. Automat. Contr, vol. AC-12, pp. 268-275, Jun.
1967.

[5] M. Mintz, "A note on minimax estimation and Kalman filtering," IEEE Trans. Automat.
Contr, vol. AC-14, pp. 588-590, Oct. 1969.

[6] J. M. Morris, "The Kalman filter: A robust estimator for some classes of linear quadratic
problems," IEEE Trans. Inform. Theory, vol. IT-22, pp. 526-534, Sep. 1976.

[7] V. Poor and D. P. Looze, "Minimax state estimation for linear stochastic systems with noise
uncertainty," IEEE Trans. Automat. Contr, vol. AC-26, pp. 902-906, Aug. 1981.

[8] C. J. Martin and M. Mintz, "Robust filtering and prediction for linear systems with
uncertain dynamics: A game-theoretic approach," IEEE Trans. Automat. Contr, vol. AC-28,
pp. 888-896, Sep. 1983.

[9] S. Verdli and H. V. Poor, "On minimax robustness: A general approach and applications,"
IEEE Trans. Inform. Theory, vol. IT-30, pp. 328-340, Mar. 1984.

[10] S. Verdli and H. V. Poor, "Minimax linear observers and regulators for stochastic systems
with uncertain second-order statistics," IEEE Trans. Automant. Contr, vol. AC-29, pp.
499-511, Jun. 1984.

[11] J. C. Darragh and D. P. Looze, "Noncausal minimax linear state estimation for systems with
uncertain second-order statistics," IEEE Trans. Automat. Contr, vol. AC-29, pp. 555-557,
Jun. 1984.

[12] B. I. Anan'ev, "On minimax state estimates for multistage statistically uncertain systems,"
Problems of Control and Information Theory, vol. 18, pp. 27-41, 1989.

[13] Y. L. Chen and B. S. Chen, "Minimax robust deconvolution filters under stochastic
parametric and noise uncertainties," IEEE Trans. Signal Processing, vol. 42, pp. 32-45, Jan.
1994.

[14] B. I. Anan'ev, "Minimax estimation of statistically uncertain systems under the choice of a
feedback parameter," Journal oflMathematical Systems, Estimation, and Control, vol. 5, pp.
1-17, 1995.

[15] J. O. Berger, Statistical Decision Theory and Ba! edition, 1985.

[16] J. L. Hodges, Jr. and E. L. Lehmann, "The use of previous experience in reaching statistical
decisions," Ann. 1Math. Stat., vol. 23, pp. 396-407, Sep. 1952.

[17] B. Efron and C. Morris, "Limiting the risk of Bayes and empirical Bayes estimators part
1: The Bayes case," I. Amer Statist. Assoc., vol. 66, pp. 807-815, Dec. 1971.

[18] P. J. Kempthomne, "Numerical specification of discrete least favorable prior distributions,"
SIAIM J Sci. Statist. Comput., vol. 8, pp. 171-184, Mar. 1987.

[19] P. J. Kempthomne, "Controlling risks under different loss functions: The compromise
decision problem," Ann. Statist., vol. 16, pp. 1594-1608, Dec. 1988.

[20] I. M. Johnstone, "On minimax estimation of a sparse normal mean vector," Ann. Statist.,
vol. 22, pp. 271-289, Mar. 1994.

[21] R. F. Berg, "Estimation and prediction for maneuvering target trajectories," IEEE Trans.
Automat. Contr, vol. AC-28, pp. 294-304, Mar. 1983.

[22] M. H. Kao and D. H. Eller, "Multiconfiguration Kalman filter design for high-performance
GPS navigation," IEEE Trans. Automat. Contr, vol. AC-28, pp. 304-314, Mar. 1983.

[23] L. Danyang and L. Xuanhuang, "Optimal state estimation without the requirement of a
priori statistics information of the initial state," IEE Trans. Automant. Contr, vol. 39, pp.
2087-2091, Oct. 1994.

[24] Y. Levinbook and T. F Wong, "State estimation with initial state uncertainty,"
IEEE Transactions on Information Theory, 2005, Submitted for publication. URL:
http://wireless.ece.ufl.edu/~twong/Preprinsmnmxpf

[25] A. Wald, Statistical Decision Functions, John Wiley and Sons, New York, 1950.

[ 26] T. S. Ferguson, 1Mathematical Statistics: A Decision Theoretic Approach, Academic Press,
New York, 1967.

[27] P. Billingsley, Convergence of Probability 1Measures, John Wiley & Sons, New York, 1968.

[28] W. Rudin, Principles of Mathematicalnrrtab \i\. McGraw-Hill, New York, 3rd edition,
1976.

[29] D. W. Stroock, Probability Theory, An Analytic View, Cambridge University Press, New
York, 1993.

[30] L. LeCam, "An extension of Wald's theory of statistical decision functions," Ann. 1Math.
Stat., vol. 26, pp. 69-81, Mar. 1955.

[31] H. Kudo, "On the property (W) of the class of statistical decision functions," Ann. 1Math.
Stat., vol. 37, pp. 1631-1642, Dec. 1966.

[32] M. Sion, "On general minimax theorems," Pacific J. 1Math., vol. 8, pp. 171-176, 1958.

[33] G. K. Pedersen, Arab\ \i\ Now, Springer-Verlag, New York, 1989.

[34] A. N. Shiryaev, Probability, Springer-Verlag, New York, 2nd edition, 1989.

[35] J. L. Doob, 1Measure Theory, Springer-Verlag, New York, 1994.

[36] W. Rudin, Functional Analysis, McGraw-Hill, New York, 2nd edition, 1991.

[37] E. L. Lehmann, Theory of Point Estimation, Wiley, New York, 1983.

[38] W. Nelson, "Minimax solution of statistical decision problems by iteration," Ann. 1Math.
Stat., vol. 37, pp. 1643-1657, Dec. 1966.

[39] J. Wolfowitz, "On e-complete class of decision functions," Ann. 1Math. Stat., vol. 22, pp.
461-465, Sep. 1951.

[40] C. D. Aliprantis and K. C. Border, Infinite DimensionalAnahn \i\.: A Hitchhiker's Guide,
Springer-Verlag, Berlin, 1994.

[41] N. M. Roy, "Extreme points of convex sets in infinite dimensional spaces," Amer 1Math.
1Monthly, vol. 94, pp. 409-422, May 1987.

[42] S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press,
Cambridge, 2004.

[43] L. Mirsky, "A trace inequality of John von Neumann," Maonashe~tfire flrMathematik, vol.
79, pp. 303-306, 1975.

[44] T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation, Prentice Hall, NJ, 2000.

[ 45] S. S. Blackman, 1Multiple Target Tracking with RadarApplications, Artech House,
Washington, DC, 1986.

[46] D. G. Lainiotis, S. K. Park, and R. Krishnaiah, "Optimal state-vector estimation for non-
Gaussian initial state-vector," IEEE Trans. Automat. Contr, vol. AC-16, pp. 197-198, Apr.
1971.

BIOGRAPHICAL SKETCH

Yoav N. Levinbook was born in Tel Aviv, Israel, on October 30, 1974. He received the B.S.

degree Magnaa com laude) from Tel Aviv University, Israel, in 2000 and the M.S. and Ph.D.

degrees in electrical and computer engineering from the University of Florida, Gainesville, in

2006 and 2007, respectively.

He was with the Motorola Semiconductor, Herzliya, Israel, and Smartlink, Netanya,

Israel, as an electrical engineer. His research interests include statistical decision theory, state

estimation, signal processing for communications, and sensor networks.

PAGE 1

1

PAGE 2

2

PAGE 3

3

PAGE 4

PAGE 5

page ACKNOWLEDGMENTS .................................... 4 LISTOFFIGURES ....................................... 7 LISTOFABBREVIATIONS .................................. 8 ABSTRACT ........................................... 9 CHAPTER 1INTRODUCTION .................................... 11 2GENERALNOTATIONANDCONVENTIONS .................... 17 3DECISIONTHEORETICFORMULATION ....................... 19 4GENERALDECISIONTHEORETICRESULTS .................... 24 5THECASETHATTHERISKISSPECIFIEDBYALOSSFUNCTION ........ 34 6THECASEOFCONVEXLOSSFUNCTION ...................... 40 7FINDINGAMINIMAXESTIMATORANDTHEDUALPROBLEM ......... 49 8APPROXIMATINGAMINIMAXESTIMATOR .................... 52 9THERESTRICTEDRISKBAYESPROBLEMASAMINIMAXPROBLEM .... 61 10ESTIMATIONWITHARESTRICTIONONTHEOBSERVATIONSTHATCANBEUSED ......................................... 66 11THESTATEESTIMATIONPROBLEM ......................... 73 12AFFINESTATEESTIMATIONBASEDONQUADRATICLOSSFUNCTIONS ... 77 12.1FindingaRestrictedRiskBayesSolution ..................... 77 12.2FindingaMaximizeroftheRisk .......................... 84 12.3ConnectiontotheKalmanFilterand-MinimaxApproach ............ 87 12.4NumericalExample ................................. 90 13STATEESTIMATIONWITHINITIALSTATEUNCERTAINTY ........... 96 13.1ConditionalMeanEstimators ............................ 97 13.2ApproximationstoMinimaxEstimators ...................... 98 13.3NumericalExample ................................. 100 14CONCLUSIONS ..................................... 106 APPENDIX 5

PAGE 6

................................. 107 BPROOFOFLEMMA9.1 ................................. 109 CPROOFOFLEMMA9.2 ................................. 110 REFERENCES ......................................... 111 BIOGRAPHICALSKETCH .................................. 114 6

PAGE 7

Figure page 12-1AchievedBayesriskvs.maximumrisk .......................... 94 12-2ThemaximumriskoveroftheBayes,minimax,andrestrictedriskBayessolu-tionsvs. 95 13-1AfullviewofXf0,whichisanite(;V)-densesubsetofX0 103 13-2Azoom-inviewofthebottomleftcornerofXf0,whichisanite(;V)-densesubsetofX0 104 13-3Theriskof^x(0),thederived-optimalestimator,asafunctionofx02Xf0 104 13-4Theaprioridistribution0,whichisdenedonXf0 105 13-5ThemaximumriskoftheKalmanFilterinitializedwithzeromeanandcovariance2Iasafunctionof 105 7

PAGE 8

CM:conditionalmean......................................97KF:KalmanFilter........................................11LMMSE:linearminimummeansquarederror.........................11MSE:meansquarederror....................................45 8

PAGE 9

9

PAGE 10

10

PAGE 11

1 ]isthelinearminimummeansquarederror(LMMSE)estimator.IfinadditionallthestochasticquantitiesareGaussian,theKFistheminimummeansquarederrorestimator.Sincetheassumptionofcompleteknowledgeoftheaprioridistributionisseldomsatised,aBayesianapproachisusedinpractice.Theaprioridistributionsoftheinitialstate,modelnoise,andmeasurementnoisearelearnedfrompastexperienceandusedasapproximationsofthecorrespondingtruedistributions.Nevertheless,evenifextensivepastexperienceisavailable,theestimateddistributionsmaystilldeviatefromthetrueones.TheeffectofsucherrorsintheaprioriinformationontheperformanceoftheKFisstudiedin[ 2 ][ 4 ].Theeffectoftheerrorsintheaprioriinformationoftheinitialstate,modelnoise,andmeasurementnoisemaybeverysignicantandaKFupdatedbasedonerroneousaprioriinformationmayperformpoorly.Thusitisnecessarytoconsiderotherapproachesthatarerobustagainstuncertaintiesintheaprioridistributionoftheinitialstate,modelnoise,andmeasurementnoise.Thestateestimationliteraturedealsextensivelywiththegeneralproblemoflinearsystemswithstochasticordeterministicuncertaintiesusinggametheoryandtheminimaxapproach(cf.[ 5 ][ 14 ]andthereferencestherein).Usuallytheso-called-minimaxapproachisadopted.The-minimaxapproach[ 15 ]regardstheparameterasrandomwithitsdistributionliesinaclass.However,theexactdistributionintheclassisunknown.A-minimaxestimatorisanestimatorthatminimizesthesupremumoftheBayesrisk,wherethesupremumistakenoverallelementsof.Whenthe-minimaxapproachisused,theclassofavailableestimatorsisusuallyrestricted 11

PAGE 12

16 ],isacompromisebetweentheBayesapproachandtheminimaxapproach.ArestrictedriskBayesestimatorminimizestheBayesriskwithrespecttoanaprioridistributionsuggestedbasedonsomepastexperiencesubjecttotherestrictionthatthemaximumriskdoesnotexceedtheminimaxriskbymorethanagivenamount.Thisapproachutilizesavailableaprioriinformationbutatthesametimeprovidesasafeguardincasethisinformationisnotaccurate.Iftheaprioriinformationisfairlyaccurate,arestrictedriskBayesestimatorhasgoodBayesriskproperties.OtherworkconsideringtherestrictedriskBayesapproachorcloselyrelatedapproachesinclude[ 17 ][ 20 ].DespitetheappealingformulationoftherestrictedriskBayesapproach,ithasnotbeenutilizedinthecontextofstateestimation.Althoughtheproblemofstateestimationwithstochasticuncertaintieshasbeenapproachedfromthe-minimaxapproach,webelievethatapproachingthisproblemfromtherestrictedriskBayesapproachalsohasaconsiderablemerit.Ifastateestimationproblemcanberegardedasazero-sumtwo-persongame(henceforthtobereferredasagame)againstarationalopponent,thenthe-minimaxapproachseemsveryattractive.However,inmostapplications,ifweregardthestateestimationproblemasagame,thegameisagainstNature.Usingthe-minimaxapproachinthiscasecorrespondstoaverypessimisticviewpointthatregardsNatureasarationalopponentthatwishestocauseusthelargestpossibleloss.The-minimaxapproachmaystillseemreasonableinthecasethatthereisnoaprioriinformationthatenablesustoregardcertaindistributionsinasmorelikelythanothers.However,inmanyapplications,someaprioriinformationregardingthetruedistributionmaybeavailable.Thisaprioriinformationmaybeintheformofanominaldistribution,whichissuggestedbasedonsomepastexperience.Itiswellknownthatundercertainconditions,a-minimaxestimatorisaKFrelativetoaleastfavorablea

PAGE 13

13

PAGE 14

PAGE 15

24 ]thatdealwiththeboundedparametersetcaseinordertoillustratehowminimaxestimatorscanbeapproximatedwitharbitrarilyprescribedaccuracy.Whilewearemainlyinterestedinthestateestimationproblem,alargepartofthisworkwillbeconcernedwithamoregeneralestimationproblem.Infact,someoftheexistenceresultssuchastheexistenceofaminimaxestimatorandarestrictedriskBayessolutionholdinaverygeneralsettingandmayhaveapplicabilitynotonlyinthestateestimationproblem.Therestofthisworkisorganizedasfollows.InChapter 2 ,wepresentnotationandconventionsthatareusedthroughoutthiswork.InChapter 3 ,wepresentageneraldecisiontheoreticformulationthatisneededinordertoaddresstheproblemofstateestimationwithstochasticanddeterministicuncertaintiesandalsotoderiveother,moregeneral,results.InChapter 4 ,wederiveseveralgeneraldecisiontheoreticresults,whicharebasedonwellknownresultsfromdecisiontheoryandgametheory;theapplicabilityoftheseresultsisnotlimitedonlytotheproblemofstateestimationwithstochasticanddeterministicuncertainties.InChapter 5 ,weconsiderthecasethattheriskfunctionisspeciedbyalossfunction.WederiveratherweakconditionsthatguaranteetheexistenceofaminimaxestimatorandarestrictedriskBayessolution.InChapter 6 ,werestrictourselvestoconvexlossfunctions,ingeneral,andthequadraticlossfunction,inparticular.InChapter 7 ,wediscusshowaminimaxestimatorcanbefoundbysolvingthedualproblemofndingaleastfavorableaprioridistribution.Since,ingeneral,ndingaminimaxestimatormaybeanextremelydifculttask,inChapter 8 wediscusshowonecanderiveapproximationstominimaxestimators,wheretheapproximationcanbemadeasaccurateasdesired.InChapter 9 ,weconsiderthegeneralrestrictedriskBayesestimationproblemandshowthatthisproblemisequivalenttoasequenceofminimaxestimationproblems.Insomeestimationproblemstherearerestrictionsontheobservationsthatcanbeusedinordertoestimatetheparameters.Thisisthecaseinthestateestimationprobleminwhicheachstatecanbeestimatedusingonlycertainobservations.WeconsiderthistypeofestimationproblemsinChapter 10 .InChapters 11 12 ,and 13 ,werestrictourselvestotheproblemofstateestimationwithuncertaintiesintheinitialstate,modelnoise,andmeasurementnoise,which 15

PAGE 16

11 ,wederivesomegeneralexistenceresultsthatarebasedontheresultsofthepreviouschapters.InChapter 12 ,weconsiderthecaseofstochasticuncertaintiesintheinitialstate,modelnoise,andmeasurementnoise,andrestrictourselvestoafneestimators.WeproposeamethodthatcanbeeasilyusedtoderivearestrictedriskBayessolutioninmanyimportantcases.InChapter 13 ,weconsiderthecaseofdeterministicinitialstateuncertainty,andsearchforestimatorswithintheclassofallestimators.WeconcludethisworkinChapter 14 16

PAGE 17

17

PAGE 18

18

PAGE 19

PAGE 20

26 ]and[ 25 ],withslightmodications.Thesedenitionsaregivenforanestimationproblem(;X;R).

PAGE 21

PAGE 22

3.1 ,inf^x2Xr(;^x)<+1,andthetermr(i;^x)inf^x2Xr(i;^x),intheabovedenition,iswelldened.

PAGE 23

23

PAGE 24

26 ,Exercise2.2.1]) ConsiderthespaceMequippedwiththetopologyofweakconvergence[ 27 ,pp.236].ThetopologyofweakconvergencemakesMaHausdorffspace. 1. 2. Let^x2XbesuchthatR(;^x)<+1foreach2andR(;^x)iscontinuouson.Thenr(;^x)iscontinuousonM. 3. Let^x2XbesuchthatR(;^x)<+1foreach2andR(;^x)isuppersemicontinu-ouson.Thenr(;^x)isuppersemicontinuousonM. Proof. 28 ,Exercise2.25]).ThisimpliesthatMismetrizable[ 29 ,pp.122].Inaddition,[ 29 ,Theorem3.1.9]furnishesthatMiscompact.2)By[ 29 ,Theorem3.1.5],Rfdisacontinuousfunctionofforanyboundedandcontinuousf.SinceR(;^x)isacontinuousreal-valuedfunctiononacompactset,itisbounded.Hencer(;^x)iscontinuousonM.3)By[ 29 ,Theorem3.1.5],Rfdisanuppersemicontinuousfunctionofforanyup-persemicontinuousfthatisboundedfromabove.SinceR(;^x)isanuppersemicontinuousreal-valuedfunctiononacompactset,itisboundedfromabove.Hencer(;^x)isuppersemicon-tinuousonMforany^x2C. 24

PAGE 25

30 ]. 25

PAGE 26

31 ].SinceU(M)iscompact,R(A;M)isrelativelycompact.Thusthereexistsasubnetf^xbgb2BthatconvergestoapointuintheclosureofR(A;M).HenceliminfR(;^xb)=limR(;^xb)=u.SinceR(A;M)ishalf-closed,thereexistsanelementv2R(A;M)suchthatvu.Hencethereexistsanelement^x2XsuchthatR(;^x)liminfR(;^xb).Supposeforeachnetf^xaga2AinA,thereexistsasubnetf^xbgb2Bandanelement^x2Asuchthatliminfr(;^xb)r(;^x).LetubelongtotheclosureofR(A;M).Thenthereexistsanetfuaga2A2R(A;M)thatconvergestou.Clearlyforeachelementuainthisnet,thereisanelement^xa2Asuchthatua=R(;^xa).ThuslimR(;^xa)=u.Itfollowsthatthereexistsasubnetf^xbgb2Bandanelement^x2AsuchthatliminfR(;^xb)R(;^x),whenceuR(;^x)andR(A;M)ishalf-closed.Inentirelyanalogousway,Ahastheproperty(W)ifandonlyifforeachnetf^xaga2AinA,thereexistsasubnetf^xbgb2Bandanelement^x2AsuchthatliminfR(;^xb)R(;^x).SinceeachcanbeidentiedasanelementinM,asdiscussedpreviously,itisclearthattheproperty(W)impliestheproperty(W).Theproperty(W),asformulatedwithnets,iscloselyrelatedtoWald'sweakcompactness[ 25 ,pp.53].Theonlydifferenceisthatinthecurrentdenitionsequencesarereplacedbynets.Hencetheproperty(W)isweakerthanWald'sweakcompactness.SufcientconditionsforasubsetAofXtohavetheproperty(W)aregivenin[ 31 ].AsimplesufcientconditionisthatthereexistsaHausdorfftopologyJforAsuchthatAiscompactandR(;)islowersemicontinuousonAforeach2.Similarly,asufcientconditionforAtohavetheproperty(W)isthatthereexistsaHausdorfftopologyJforAsuchthatAiscompactandr(;)islowersemicontinuousonAforeach2M.SupposethereexistsatopologyJforAsuchthatAiscompact.Thenifthetopologicalspace(A;J)satisestherstaxiomofcountability,byFatou'sLemma,therequirementthatr(;)isalowersemicontinuousonAforeach2McanbereplacedbytherequirementthatR(;)isalowersemicontinuousonAforeach2. 26

PAGE 27

PAGE 28

PuttingC0=+1intheabovetheorem,wegetthatforeach2M,thereexistsaBayessolutionrelativeto(providedthatXhastheproperty(W)).ThefollowingTheorem,aversionofWald'scompleteclasstheorem,appearsin[ 31 ]. 32 ,Theorem4.2]. Proof. 4.1 ,Miscompactandr(;^x)isuppersemicontinuousonMforeach^x2X.ItiseasytoverifythatsinceXissubconvex,foreach^x0;^x002Xand0<<1,thereexists^x2Xsuchthatr(;^x)r(;^x0)+(1)r(;^x00)forall2.Certainlyr(;^x)isconcaveonMforall^x2X.Applying[ 32 ,Theorem4.2],inf^x2Xsup2Mr(;^x)=sup2Minf^x2Xr(;^x):Thus(;X;R)isstrictlydetermined.Sinceinf^x2Xr(;^x)isuppersemicontinuousonacompactsetM,thereexists02Msuchthatinf^x2Xr(0;^x)=sup2Minf^x2Xr(;^x):Thus0isaleastfavorableaprioridistribution. 28

PAGE 29

4.4 thatR(;^x)<+1foreach^x2XandR(;^x)isuppersemicontinuousonforeach^x2Xisobviouslytoostrong.Inthefollowingtheoremthisassumptionisconsiderablyweakened,butatthepriceofastrongerassumptiononX.LetBXdenotetheclassofBayessolutionsrelativeto2M.Beforestatingthetheorem,weneedthefollowinglemma. 4.2 itisshownthatifXhastheproperty(W),thereexistsaBayessolutionrelativetoforall2M.Thelemmafollowseasily. Proof. 32 ,Theorem4.2], Fix2M.Ononehand,sinceR(B;M)GB,infg2GBg()infg2R(B;M)g()=inf^x2Br(;^x):Ontheotherhand,ifwexg2GB,thenthereexistanintegern>0,realnumbers1;:::;n>0,andelementsu1;:::;un2R(B;M)suchthatg=Pni=1iuiandPni=1i=1.Let1inbesuchthatui()uj()forj=1;:::;n.Theng()ui(). 29

PAGE 30

SinceGGB, Fixg2G.ThensinceXissubconvex,thereexists^x2Xsuchthatr(;^x)g()forall2M.Hence By( 4 )( 4 )andLemma 4.2 ,(;X;R)isstrictlydetermined.Sinceinfg2GBgisuppersemicontinuousonthecompactsetM,thereexists02Msuchthatinfg2GBg(0)=sup2Minfg2GBg():By( 4 )andLemma 4.2 ,0isaleastfavorableaprioridistribution. Ifthecompactnessofisdroppedintheabovetheorem,thentheremaynotbealeastfavorableaprioridistribution.Ifthereexistsnoleastfavorableaprioridistribution,buttheestimationproblem(;X;R)isstrictlydetermined,aminimaxestimatorcanbefoundasaBayessolutionrelativetoaleastfavorablesequenceofaprioridistributions,i.e.,asequencefng2Mthatsatiseslimn!1inf^x2Xr(n;^x)=sup2Minf^x2Xr(;^x).Wearenotgoingtodealwiththequestionofhowsuchasequencecanbeconstructed.ThefollowingtheoremisessentiallyWald'sTheorem3.9in[ 25 ]. 30

PAGE 31

Proof. 25 ,Theorem3.9]applieswithoutanychanges. Wheniscompact,wehavethefollowingversiontoWald'scompleteclasstheorem. Proof. ByLemma 4.1 part1),thereexists02Mandasubsequencefikg2Msuchthatikconvergesweaklyto0.Bythehypothesisofthetheorem,r(;^x)iscontinuousonMforany^x2B.Thusinf^x2Br(;^x)isuppersemicontinuousonM[ 33 ,Proposition1.5.12].ByLemma 4.2 ,inf^x2Xr(;^x)isuppersemicontinuousonM.Sincer(;^x)islowersemicontinuousonM,inf^x2Xr(;^x)r(;^x)isuppersemicontinuousonMand By( 4 )and( 4 ),inf^x2Xr(0;^x)r(0;^x),whence^xisaBayessolutionrelativeto0.

PAGE 32

PAGE 33

Proof. 25 ]).Forthesakeofsimplicity,weassumethat(;X;R)isanestimationproblem.TheproofcanbetriviallymodiedtothecasethatXisnotaclassofestimators,butanarbitraryset.Letr(;a)=RR(;a)d,where2Manda2X.ItcanbeveriedthatsinceXisacompactmetrizablespaceandR(;)islowersemicontinuousonXforeach2,fr(;a):a2Xgishalf-closed,i.e.,Xhastheproperty(W).Itcanbeveriedthatsince(X;R)(X;R),Xhastheproperty(W).ByTheorem 4.1 ,thereexistsaminimaxestimator.ByTheorem 4.2 ,thereexistsarestrictedriskBayessolutionrelativeto(;C0)foreach2MandC0V(;X;R).If,inaddition,Condition 4.3 holds,itfollowsfromtheprecedingresults,that(;X;R)isstrictlydetermined,thereexistsaleastfavorableaprioridistribution02M,anyminimaxestimatorisaBayessolutionrelativeto0,andtheclassofBayessolutionsisessentiallycomplete.SupposeCondition 4.4 holdsaswell.Let0denotealeastfavorableaprioridistribution.SinceanyminimaxestimatorisaBayessolutionrelativeto0(Theorem 4.6 ),the(essentiallyunique)Bayessolutionrelativeto0isanessentiallyuniqueminimaxestimator.By[ 26 ,Theorem2.3.1],thisBayessolutionisadmissible. WhilesometimesitispossibletoverifyConditions 4.1 4.4 directly,othertimes,especiallywhenthereisnoclosedfromexpressionforR,itmaybedifculttocheckwhethertheseconditionshold.Inthenextchapter,weconsiderthecasethatRisbasedonalossfunctionL.Inthiscase,itispossibletoderiveconditionsthatcanbemoreeasilycheckedwhenthereisnoclosedformexpressionforR. 33

PAGE 34

WeimposeseveralconditiononthefamilyP,thespaceY,thespaceDofpossibleesti-mates,andthelossfunctionL. 34

PAGE 35

34 ,pp.220].If~gisaversionofPr(AjX),thenaccordingto[ 34 ,pp.221],aconditionalprobabilityoftheeventAgiventhatX=x,g(x),canbeconstructedasfollows:g(X(!))=~g(!)(i.e.,g(x)=~g(!),where!isanelementinsuchthatX(!)=x).SinceinthisworkthedistributionQisunknown,butisknowntobeanelementofP,theconditionalprobabilityofA2FgivenXandtheconditionalprobabilityofAgivenX=xdependonthetruestateofnature2. 1. Foreach!2,Q(jX)(!)isaprobabilitymeasureon(Y;FY). 2. ForeachA2FY,Q(AjX)isaversionoftheconditionalprobabilityPr(AXjX).Fix2.Condition 5.1 furnishesthatthereexistsaregularconditionaldistributionofYgivenXwhenQ=P[ 34 ,Theorem2.7.5].LetP(jX)denotearegularconditionaldistributionofYgivenXwhenQ=P.Foreachx2XandA2FY,wedeneP(Ajx)=P(AjX)(!),where!2issuchthatX(!)=x.WecallP(jx)aregularconditionaldistributionofYgivenX=xwhenQ=P(oristhetruestateofnature).NotethatsinceP(jX)isregular,P(jx)isaprobabilitymeasureforeachx2X.Inaddition,foreachA2FY,P(Ajx)isaversionoftheconditionalprobabilityPr(AXjX=x).LetPYjX=fP(jx):x2X;2g. 5.3 and 5.4 wereaddedtoguaranteethattheintegrationsnecessaryinthecalculationoftheriskfunctionarewelldened.ItcanbeveriedthatifConditions 5.1 5.4 hold,R(;^x)isanextendedreal-valuednonnegativeB()-measurablefunctionandhencethe 35

PAGE 36

30 ].Itisusefultodiscussthepropertiesofthisvectorspace.Thefollowingdiscussionessentiallyappearsin[ 30 ]and[ 31 ].LetCc(D)denotetheclassofcontinuousreal-valuedfunctionsonDwithcompactsupport,andletjjujj=supd2Dju(d)jforu2CC(D).LetL1denotetheBanachspaceofequivalenceclassesofintegrablefunctionson(Y;FY;)withnormjjfjj1=RYjfjd.DenotebyLPYjXthelinearsubspaceofL1spannedbyPYjX.LettheproductspaceCC(D)LPYjXbeequippedwiththenormjj(u;f)jj=jjujj_jjfjj1foru2Cc(D)andf2LPYjX.LetdenotethevectorspaceofboundedlinearfunctionalsonCC(D)LPYjX.Theweak-topologyturnsintoalocallyconvextopologicalvectorspace[ 33 ].Callg1;g22m(FY)PYjX-equivalentifRYjg1g2jfd=0forallf2PYjX.Calltwoestimators^x1;^x22DPYjX-equivalentifforeachD02B(D),^x1(D0j)and^x2(D0j)arePYjX-equivalent.Let~Ddenotetheclassofequivalenceclassessoobtained.Afunctional2issaidtobepositiveifu0andf0imply(u;f)0.AccordingtoLeCamifConditions 5.1 5.5 hold,everypositivelinearfunctionalofnorm1canberepresentedbyan 36

PAGE 37

5.1 5.5 and 5.7 hold.Thenthetopologicalspace(D;J)ismetrizableandR(;)islowersemicontinuousonDforall2.If,inaddition,Discompact,Discompact. Proof. 5.5 ,thespaceCc(D)isseparable[ 30 ].SincebyCondition 5.1 ,Yisaseparablemetricspace,thespaceL1isseparable[ 35 ,pp.92].SinceLPYjXisasubspaceofaseparablenormedspace,itisalsoseparable.ItfollowsthatthespaceCc(D)LPYjXisseparable.ByTheorem[ 36 ,Theorem3.16],if0isweak-compact,then0ismetrizable.ByBanach-AlaogluTheorem[ 33 ,Theorem2.5.2],thesetB=f2:jjjj1gisweak-compact,wherejjjjdenotestheoperatornormof2.ThusBismetrizable.SinceJistherelativetopology,Dismetrizable.LetR(x;;^x)=RYL(x;^x)p(yjx)d.Thenby( 5 ),R(;^x)=RXR(x;;^x)dPX(x).UsingtheresultsofLeCam[ 30 ],itcanbeshownthatR(x;;)islowersemicontinuousonDforeach(;x)2X.SinceDismetrizable,wehavebyFatou'sLemmathatR(;)islowersemicontinuousonDforeach2.SupposeDiscompact.LeCam[ 30 ]showedthataclassAofestimatorsisJ-compactifitisJ-closedandifthefollowingconditionsholds:Foreach>0andeach(;x)2X,thereexistsau2Cc(D)satisfying0u1andRYu(d)d^x(djy)dP(yjx)1uniformlyforall^x2A.TheprecedingconditioncertainlyholdsforDwhenDiscompact.SinceDisJ-closed,itiscompact. 37

PAGE 38

4.2 holdsunderveryweakconditions. Proof. A 5.1 5.7 hold.ThenCondition 4.1 and 4.2 holdfor(;D;R).Asaconsequence,thereexistsaminimaxestimatorandarestrictedriskBayessolutionrelativeto(;C0)foreach2MandC0V(;D;R). Proof. 4.1 holds.LetusshowthatCondition 4.2 holds.SupposeDiscompact.ThenbyLemma 5.1 ,Condition 4.2 holds.SupposeDisnotcompact.Wewillusetheone-pointcompacticationofDtoprovethetheorem.Theideatousetheone-pointcompacticationoftheclassofpossibleestimatestoproveresultsofthetypeofthistheoremseemstoappearrstin[ 30 ].LetDdenotetheone-pointcompacticationofD,andlet12DdenotethepointthatisaddedtoD.LetL:XD!R[f+1gbedenedasfollows:Foreachx2X,L(x;d)=L(x;d)ifd2D,andL(x;1)=supd2DL(x;d).LetDdenotetheclassofallestimatorswithDastheirspaceofpossibleestimates.ByLemma 5.2 ,L2m(FXB(D))andDiscompact.LetL(x;^x)(y)=RDL(x;d)d^x(djy), 38

PAGE 39

5.1 5.5 and 5.7 holdfor(;D;R).ThenbyLemma 5.1 ,DiscompactandR(;)islowersemicontinuousonDforeach2.ItcanalsobeveriedthatR(;a)2m(B())foreacha2X.LetD(D)denotetheclassofestimators^xinDsuchthat^x(Djy)=1forally2Y.Weclaimthat(D(D);R)(D;R).Indeed,x^x02Dsuchthat^x0(1jy)>0forsomey2Y.LetY0=fy2Y:^x0(Djy)=0g.ClearlyY0ismeasurable.Let^x(Ajy)=^x0(A\Djy)=^x0(Djy)fory=2Y0(A2B(D)),andlet^x(jy)beaDiracmeasurewithrespecttoapointd02Dforally2Y0.Itcanbeveriedthat^x(D0jy)isB(Y)-measurableforeachD02B(D).Nowify2Y0,thenL(x;^x)=L(x;d0)L(x;1).Ify=2Y0,thenRDL(x;d)d^x(djy)=RDL(x;d)=^x0(Djy)d^x0(djy)x0(Djy)RDL(x;d)=^x0(Djy)d^x0(djy)+(1^x0(Djy))L(x;1)=RDL(x;d)d^x0(djy).ThusL(x;^x)L(x;^x0).Itfollowsthat^xisasgoodas^x0.Thisprovesthat(D(D);R)(D;R).SinceD(D)D,wehavethat(D(D);R)(D;R).Clearly(D;R)(D(D);R).Thus(D;R)(D;R).ItfollowsthatCondition 4.2 holds. AsaconsequenceofTheorem 5.1 ,undertheratherweakConditions 5.1 5.7 ,thereexistsaminimaxestimatorandarestrictedriskBayessolutionrelativeto(;C0)foreach2MandC0V(;D;R).Inordertogetthestrongerresultswheniscompact,namelythat(;D;R)isstrictlydeterminedandthatthereexistsaleastfavorableaprioridistribution,weneedtoprovethatCondition 4.3 holds.Unfortunately,thisseemstorequireratherstrongconditionsonthelossfunctionandfamilyP.AsetofsuchconditionsiswellknownforthecasethatthelossfunctionisuniformlyboundedandPisdominatedbya-nitemeasure.However,wearemainlyinterestedinthecasethatthelossfunctionisunbounded(e.g.,thequadraticlossfunction).Moreover,inmanycasesPisnotnecessarilydominatedbya-nitemeasure. 39

PAGE 40

37 ,Theorem1.6.4]. 5.3 holds.ThenifConditions 6.1 and 6.2 hold,DD. Proof. 40

PAGE 41

Integrating( 6 )withrespecttoP,wehaveR(;^x)R(;^x0).Since^x02D,DD. SinceDD,weconsiderintherestofthischaptertheestimationproblem(;D;R)insteadoftheestimationproblem(;D;R).Clearly,DissubconvexbyJenseninequality.HenceCondition 4.1 holdsfor(;D;R).SinceDD,DD.HenceitisclearthatCondition 4.2 holdsfor(;D;R)ifitholdsfor(;D;R).ThusifConditions 5.1 5.7 hold,Condition 4.2 holdsfor(;D;R).InLemma 6.2 below,weshowthatunderweakconditions,Condition 4.4 alsoholdsfor(;D;R).InordertoprovethatCondition 4.3 holdsfor(;D;R),itseemsnecessarytomakeadditionalassumptionsonthelossfunctionandthefamilyP.Sometimesitisconvenienttorestricttheclassofestimatorsthatareavailabletotheexperi-mentertotheclassofafneestimators.Anestimator^xissaidtobeafneifitisnonrandomizedandisanafnefunctiononY.SinceweconsideronlythecasethatY=RNyandX=RNx,^xisafneexactlywhen^x=Ay+bforsomeA2RNxNyandb2RNx,andthespaceofpossiblees-timatesDisthenRNx.LetLdenotetheclassofafneestimators.ThespaceLcanbeidentiedwiththespaceRNxNyRNxwhere(A;b)2RNxNyRNxistheestimator^x=Ay+b2Landviceversa.ThusthespaceLcanbeidentiedwithanite-dimensionalvectorspacewiththefollowingadditionandmultiplicationbyascalar:If^x=(A;b),^x0=(A0;b0)andisascalar,^x+^x0=(A+A0;b+b0)and^x=(A;b).LetthespaceLbeequippedwiththenormjj^x^x0jjL=jjAA0jj2+jbb0j,where^x=(A;b)and^x0=(A0;b0).ClearlyLisconvex.ItiseasytoseethatJensen'sinequalityfurnishesthatR(;)isconvexoverLforeach2ifL(x;)isconvexoverDforeachx2X.ThusifL(x;)isconvexoverDforeachx2X,L

PAGE 42

4.1 holdsfor(;L;R).InTheorem 6.1 below,weshowthatunderratherweakconditions,Condition 4.2 holdsfor(;L;R).Asintheestimationproblem(;D;R),itseemsnecessarytomakefurtherassumptionsonthelossfunctionandthefamilyPinordertoprovethatCondition 4.3 holdsfor(;L;R).InLemma 6.2 below,weshowthatunderweakconditions,Condition 4.4 alsoholdsfor(;L;R). 5.3 5.4 5.6 ,and 5.7 hold.ThenCondition 4.2 holdsfor(;L;R).Asaconsequence,thereexistsaminimaxestimatorandarestrictedriskBayessolutionrelativeto(C0;)foreachC0V(;D;R)and2M. Proof. 5.7 ,liminfn!1L(x;^xn(y))L(x;^x(y))foreachy2Y.ByFatou'slemma,liminfn!1R(;^xn)R(;^x)foreach2.ThusR(;)islowersemicontinuousonLforeach2.LetusshowthatforeachsequenceofcompactsubsetsCnofLsuchthat[1n=1Cn=Landeachelement^xn=2Cn(n=1;2;:::),liminfn!1R(;^xn)=sup^x2LR(;^x)forall2.FixasequenceofcompactsubsetsCnofLsuchthat[1n=1Cn=Landasequencef^xng2Lsuchthat^xn=2Cn(n=1;2;:::).Fix2.Certainlyliminfn!1R(;^xn)sup^x2LR(;^x).Thusitislefttoprovethatliminfn!1R(;^xn)sup^x2LR(;^x).ByFatou'slemma, Fix(y;x)2YX.LetDn=f^xn(y):^xn2Cng(n=1;2;:::).WeclaimthatDnisacompactsubsetofD.Indeed,letfdigbeasequenceinDn.Thenthereexistsasequencef^x0iginCnsuchthat^x0i(y)=di.SinceCniscompact,thereexistsasubsequencef^x0ijgofthesequencef^x0igandanelement^x02Cnsuchthat^x0ij!^x0.Thisimpliesthat^x0ij(y)!^x0(y),whencefdijg!^x0(y).Since^x0(y)2Dn,Dniscompact.Weclaimthat[1n=1Dn=D.Fixd2D.Clearlythereexists^x2Lsuchthat^x(y)=d(e.g.,^x=(A;b),whereA=0andb=d).Since[1n=1Cn=C,^x2Cnfornsufcientlylarge.Thisimpliesthatd2Dnfornsufciently 42

PAGE 43

5.6 ,liminfn!1L(x;dn)=supd2DL(x;d),wheredn=^xn(y).Itiseasytoverifythatsupd2DL(x;d)=sup^x2LL(x;^x(y)).Thusforeach^x2Lwehave Sinceyandxarearbitrary, Since^xisarbitrary, SinceLisanitedimensionalnormedspace,itislocallycompact,-compact,andmetrizable.LetLdenotetheone-pointcompacticationofLandlet1denotetheaddedpoint.Foreach2letR(;^x)=R(;^x)if^x2L,andletR(;1)=sup^x2LR(;^x).ByLemma 5.2 ,Liscompactandmetrizable,R(;)islowersemicontinuousonLforeach2,andR(;a)2m(B())foreacha2L.ClearlyLL.Since1istheonlyelementinLnLandR(;1)R(;^x)foreach^x2L,(L;R)(L;R).SinceLL,(L;R)(L;R).ThusCondition 4.2 holds. ThefollowinglemmaisconcernedwiththeessentialuniquenessofBayessolutions. 5.3 holds,andV(;X;R)<+1.ThenifCondition 6.2 holds,aBayessolutionrelativeto2MisanessentiallyuniqueBayessolutionrelativeto. Proof. 43

PAGE 44

Intherestofthischapter,weassumethelossfunctionisquadratic,i.e.,L:(x;d)7!jV(xd)j2,whereV2RNxNx.WeassumeVTV>0.TheextensiontothecaseVTV0isdiscussedlater.InthiscaseitcanbeveriedthatConditions 5.1 5.3 5.5 5.7 ,and 6.1 hold.ThusifDisconvexandCondition 6.2 holds,DD.CertainlyDissubconvexandCondition 4.1 holds.Inaddition,ifConditions 5.2 and 5.4 hold,Conditions 4.2 holdsfor(;D;R).Similarly,Condition 4.1 holdsfor(;L;R),andifConditions 5.2 and 5.4 hold,Condition 4.2 holdsfor(;L;R).Suppose,inaddition,thatPisaGaussianfamilyofdistributions,i.e.,YandXarejointlyGaussian,whenisthetruestateofnature,foreach2.ThenPYandPYjXarealsoGaussianfamiliesofdistributions.SupposethefamilyPYjXisdominatedbytheLebesgue-Borelmeasureon(RNyB(RNy)),whichisdenoted.ItcanbeeasilyveriedthatPYisalsodominatedbyandthatsince,inaddition,foreach2,p(y),thedensityofPwithrespectto,ispositive,themeasuresfPY:2garemutuallyabsolutelycontinuous.ThusCondition 6.2 holds.WewouldliketocheckunderwhatconditionsCondition 4.3 holds.Underthecurrentassumptions,itiswellknownthatthereexistsaregularconditionaldistributionofXgivenY=y,whenisthetruestateofnature.LetP(jy)denotethisconditionaldistribution,whichiswellknowntobeGaussian.Let^x:Y7!RNx;y7!E(Xjy),whereE(Xjy)

PAGE 45

InwhatfollowsBDistheclassofBayessolutionswhen(;D;R)isconsidered,i.e.,foreach^x2BD,thereexists2Msuchthatr(^x;)=inf^x2Dr(^x;).SinceDD,eachelementinBDisalsoaBayessolutionswhen(;D;R)isconsidered.Inwhatfollows,continuityoffunctionsfromintoRNandRNNismeantinthesenseoftheEuclideannormand2-norm,respectively. 4.3 holdsfor(;D;R). Proof. 6 )thatR(;^x)islowersemicontinuousonforeach^x2D. 45

PAGE 46

6 )that ItiseasytoverifythatR(0;^x)<+1.Puth(y)=(c1jyj+c2)2.Since0isarbitraryR(;^x)<+1foreach2.Moreover,foreach2,jV(^x(y)^x(y))j2h(y)andhis 46

PAGE 47

Letfngbesequencethatconvergesto0.Weshowedthatlimn!1jn0j=0.Thus BytheLebesguedominatedconvergencetheorem, Certainlyforeachn>0,jV(^xn(y)^x(y))j2h(y).SincehisP0-integrableand7!p(y)iscontinuousonforeachy2Y,awellknowntheoremonexponentialfamilies[ 37 ,Theorem1.4.1]furnishesthatZYh(y)pn(y)d!ZYh(y)p0(y)d:Itcanbeveriedthatsince7!p(y)iscontinuousonforeachy2Y,theaboveequationimpliesthatZYh(y)jpn(y)p0(y)jd!0:SincejV(^xn(y)^x(y))j2h(y), By( 6 )( 6 ),R(;^x)iscontinuouson.SinceR(;^x)<+1andR(;^x)iscontinuouson,whichisacompactset,R(;^x)isboundedon.ItfollowsthatR(;^x)isboundedandcontinuousonforeach^x2BD. 47

PAGE 48

4.3 holdsfor(;L;R). Proof. 6.2 andisomitted. WhiletheproofsofsomeoftheresultsofthischapterclearlybreakdownifVTVisnotpositivedenite,alltheseresultsarevalidalsointhecasethatVTVisnotpositivedenite.Thereisasimplemethodtoshowthatthisisindeedthecase.NotethatifVTVisnotpositivedenite,L(x;d)=L(x;d0)wheneverdd02N(V).Thuswemaycalldandd0inRNxequivalentifdd02N(V)andchoosethespaceofpossibleestimatestobethesetofequivalenceclassessoobtainedinsteadofRNx.Inthiscase,thespaceofpossibleestimatesisequippedwiththemetric(~d;~d0)=jVyV(dd0)j,where~d;~d02D,disanyelementin~d,andd0isanyelementin~d0.ThischoiceforDisequivalenttochoosingD=N(V)?withtheusualEuclideannormsinceforanyequivalenceclassinDthereisassociatedapointinN(V)?andviceversa.ItcanveriedthatwitheitheroneofthesechoicesforD,theresultsfor(;D;R)arestillvalid.Toshowthattheresultsfor(;L;R)arestillvalid,itisnecessary,todeneaclassL0=f(N(V)?A;N(V)?b):(A;b)2Lg.ItisobviousthatL0L.Certainlyforeach^x2L0,^x(y)isinthenewspaceofpossibleestimates.Now,itisstraightforwardtoshowthatalltheresultsofthischapterarestillvalidfor(;L0;R)andhencefor(;L;R). 48

PAGE 49

9.1 ,ifConditions 4.1 4.4 holdandiscompact,theproblemofndingaleastfavorableaprioridistributionisdualtotheproblemofndingaminimaxestimator.Thusintherestofthischapter,weconcerntheproblemofndingaleastfavorableaprioridistribution.Thefollowingtheoremisessentiallysimilartoatheoremin[ 18 ]andtheiterativealgorithmproposedin[ 38 ]. 7.1 and 7.2 hold.Constructasequencefig1i=12~Masfollows.Let1beanydistributionin~M.Havingchosen1;:::;i2~M,leti2besuchthatR(i;^xi)=sup2R(;^xi).Leti;=i+(1)i.Letibesuchthat~r(i;i)=sup2[0;1]~r(i;)andleti+1=i;i.Thenthesequencefigconvergesweaklytoaleastfavorableaprioridistribution. Proof. 18 ,Theorem2.3],withslightmodications. ThemaindifcultyinthealgorithmdescribedinTheorem 7.1 isinndingi2suchthatR(i;^xi)=sup2R(;^xi)fori1.Anotherdifcultyisduetothefactthat,ingeneral,sincethesupportofimaygrowasigrows,thecomplexityofthealgorithmcalculationsalsogrowswithi.Theproblemofndingisuchthat~r(i;i)=sup2[0;1]~r(i;)isaddressedbelowandcanbesolvednumerically.Insomespecialcases,itiseasytondi2suchthatR(i;^xi)=sup2R(;^xi)fori1andthecomplexityofthealgorithmcalculationsremain 49

PAGE 50

7.1 ispracticalinndingminimaxestimators.Onesuchcaseiswhenisaniteset.Inthesequel,weshowthatwhenweconsiderlinearestimationwithquadraticlossfunction,thealgorithmofTheorem 7.1 canbeoftenusedtondaminimaxestimator.Inthemoregeneralcase,thisalgorithmcanbeusedjusttond-minimaxestimators,whicharegoodapproximationstominimaxestimatorsforsufcientlysmall.Wediscussthederivationof-minimaxestimators,ingreatdetail,inthenextchapter.Theproblemofndingi2[0;1]suchthat~r(i;i)=sup2[0;1]~r(i;)isastandardoptimizationprobleminR:maximize~r(i+(1)i)subjectto2[0;1]:Fix1;22~M,andlet=1+(1)2for2[0;1].Intherestofthischapterweconsiderthefollowingoptimizationproblem,whichincludesthepreviousoneasaspecialcase: maximize~r()subjectto2[0;1]: Proof. LetD()=r(1;^x)r(2;^x)for2[0;1]. Proof. Asaconsequenceofthislemma,D(0)isthesubderivativeof7!~r()atthepoint02(0;1). 7.2 holds.Then7!D()ismonotonicallydecreasingandcontinuouson[0;1]and~r(0)=sup2[0;1]~r()for02[0;1]ifD(0)=0.

PAGE 51

7.2 furnishesthecontinuityof7!D()on[0;1].SinceD()isthesubderivativeoftheconvexfunction7!~r(),7!D()ismonotonicallydecreasingon(0;1).Bycontinuity,7!D()ismonotonicallydecreasingon[0;1].Fix02[0;1].SupposeD(0)=0.Thenbylemma 7.2 ,~r()~r(0)forall2[0;1].Itfollowsthat~r(0)=sup2[0;1]~r(). 7.2 and 7.3 implythatwecanuserelativelysimplenumericalalgorithmstosolve( 7 )numerically.Supposer(1;^x1)=r(2;^x1).Then~r(1)=r(;^x1)forall2[0;1].Itfollowsthat~r(1)~r()forall2[0;1]andhence=1isasolutionof( 7 ).Similarly,ifr(1;^x2)=r(2;^x2),=0isasolutionof( 7 ).Usingthefactthat7!D()iscontinuousandmonotonicallydecreasingon[0;1],andthefactthat2[0;1]isasolutionof( 7 )ifD()=0,theoptimizationproblem( 7 )canbesolvedasfollows:IfD(1)0,then=1isasolutionof( 7 ).IfD(0)0,then=0isasolutionof( 7 ).Finally,ifD(0)>0andD(1)<0,thereexists2(0;1)suchthatD()=0andhencesolves( 7 ).Inthiscase,sinceD()=0and7!D()ismonotonicallydecreasingon(0;1),canbeeasilyfoundusingthebisectionmethod.

PAGE 52

7.1 .Inpractice,itmaybenecessarytondan0-minimaxestimatorfor(f;X;R)suchthatthisestimatorisan-minimaxestimatorfor(;X;R).Inthelattercase,thealgorithminTheorem 7.1 canstillbeused,butitisnecessarytohaveaconditionthatenablesustocheckwhentherequiredprecisionisachieved.Checkingifanestimator^x,anessentiallyuniqueBayessolutionrelativeto,isan-minimaxestimatorcanbedoneusingthefollowinglemma: 7.1 holdsandsup2R(;^x)~r(),then^xisan-minimaxestimator. Proof. TheconditionofLemma 8.1 isonlyasufcientconditionforan-minimaxestimator.However,undercertainconditions,ifthenumericalalgorithmconvergestoaleastfavorableaprioridistribution,thenthereexistsanintegerNsuchthatLemma 8.1 issatisedfortheNthiteration. 7.1 and 7.2 hold.Letfig2Mbeasequencethatconvergesweaklytoaleastfavorableaprioridistribution02M.ThenthereexistsanintegerNsuchthatsup2R(;^xi)~r(i)foralliN.

PAGE 53

7.2 holds,itisstraightforwardthat SinceR(;^x0)iscontinuouson,itcanbeshown,asintheproofof[ 38 ,Theorem2part3],that ByTheorems 4.1 and 4.5 ,thereexistsaminimaxestimatorand(;X;R)isstrictlydetermined.ByTheorem 4.6 ,^x0isaminimaxestimator.Thus By( 8 )( 8 ),sup2R(;^xi)~r(i)!0andthelemmaisproved. Thefollowingconditionisneededforsomeoftheresultsofthischapter. 7.1 holds,andthefamilyfR(;^x):2Mgisequicontinuouson.Thenforany>0,thereexistsa>0suchthatforanynite-densesubsetfof,thefollowinghold: 1. Forany2M,thereexistsaprobabilitymeasure02fsuchthatR(;^x0)R(;^x)
PAGE 54

PAGE 55

8.1 thatthefamilyfR(;^x):2Mgisequicontinuousonisratherstrong.Whileitisoftensatisedinthecasethattheriskisbasedonalossfunctionthatisuniformlybounded,itmaynotbesatisedinthecasethatthelossfunctionisunbounded.Another,difcultywithTheorem 8.1 isthattherequirementfromthesetfisverystrong.AsetfthatisconstructedaccordingtothistheoremhasthepropertythatanyestimatorcanbeapproximatedbyaBayessolutionrelativetoameasurewithsupportinfwithdegradationofnomorethan.However,wearemainlyinterestedinapproximatingaminimaxestimatorandnotanyestimator.ThusitmaybepossibletochooseanitesetwhosecardinalityissignicantlysmallerthanfinTheorem 8.1 .Duetotheabove,wearenotgoingtouseTheorem 8.1 inthesequel,andwearegoingtoderivemethodsthatdonotrequireequicontinuity.ItisconvenienttousethenotionofFrechetdifferentiability.LetB(X;Y)denotethesetofboundedlinearoperatorsfromanormedlinearspaceXtoanormedlinearspaceY.Givenf2B(X;Y),jjfjjdenotestheoperatornormoff,i.e.,jjfjj=supx:jjxjj1jjfxjj. jjhjj=0:WecallAtheFrechetderivativeoffatxanddenoteitbyDf(x).Iff:X!YistwiceFrechetdifferentiableatx2X,weletD2f(x)denotethesecondFrechetderivativeoffatx.NotethatD2f(x)2B(X;B(X;Y)).Given^x2X,letDR^x()andD2R^x()denotetheFrechetderivativeandthesecondFrechetderivative,respectively,ofR(;^x)at.

PAGE 56

7.1 and 8.1 holdandiscompact.Supposeforany2~M,thereexistsanextensionofR(;^x)fromtoanopenconvexsetUCsuchthatR(;^x)isFrechetdifferentiableonUandthereexistsarealnumberMsuchthatjjDR^x()jjMsup2R(;^x)forall2~Mand2.Fix>0and0<0<,andlet=0 Proof. ThecardinalityofanitesubsetfofthatisconstructedaccordingtoLemma 8.3 maystillbeverylarge;thismaycausethecalculationofaleastfavorableaprioridistributionfor(f;X;R)tobeformidable.Inaspecialbutveryimportantcaseofcompact,itispossibletoderiveanite-densesubsetofsuchthatdependslinearlyonq 8.3

PAGE 57

36 ,Theorem3.20],thesetco(@)iscompactandhencetheKrein-Milmantheorem[ 36 ,Theorem3.23]impliesthat=co(@). Proof. LetSN=fa2RN:jjajj1=1g.

PAGE 58

8 )isfalse.Thenthereexists02RN+\SNandanonemptyindexsetJ06=Jsuchthat=PNi=10(i)i,0(i)>0foralli2J0,and0(i)=0foralli=2J0.Let0=(+0)=2.Then=PNi=10(i)i,0(i)>0foralli2J[J0,and0(i)=0foralli=2J[J0.SinceJ06=J,jJ[J0j>M,whichisacontradictiontothedenitionofM.Hence( 8 )musthold. 7.1 and 8.1 hold,isacompactconvexsubsetofthenormedspaceC,and@isaniteset.Supposeforeach2~M,thereexistsanextensionofR(;^x)fromtoanopenconvexsetUCsuchthatR(;^x)istwiceFrechetdifferentiableonUandthereexistsapositiverealnumberMsuchthatjjD2R^x()jjMsup2R(;^x)forall2~Mand2.Let>0and0<0<.Let=q Proof. 8.5 ,thereexistsanindexsetJf1;2;:::;Ngsuchthat( 8 )holdsfor.LetA=fi:i2Jg.Sinceisnotinf,isnotanextremepoint(Lemma 8.4 )andjJj>1.Let02f\E(co(A))besuchthat(;0)<.Then0=Pi2J(i)iforsome2RN+\SNsuchthat(i)=0fori=2J.Since=Pi2J(i)iforsome2RN+\SNsuchthat(i)>0

PAGE 59

2jj0jj2Csup2[0;1]jjD2R^x0(0+(1))jj:Bythehypothesisofthetheorem,jj0jj2Csup2[0;1]jjD2R^x0(0+(1)jj2Msup2R(;^x0):Thussup2R(;^x0)sup2fR(;^x0)R(;^x0)R(0;^x0)2M=2sup2R(;^x0).Itcanbeveriedthatsincesup2fR(;^x0)V(f;X;R)(1+0)andV(f;X;R)V(;X;R),thensup2R(;^x0)V(;X;R)(1+).Thus^x0isa-optimalestimator. SupposethenormedspaceCinCondition 8.1 isRNwithitsusualnorm.Inthiscase,ifisacompactconvexsetand@isnite,issaidtobeaconvexpolytopeinRN. 8.6 7.1 and 8.1 hold,andisaconvexpolytopeinRN.Supposeforeach2~M,thereexistsanextensionofR(;^x)fromtoanopenconvexsetUCsuchthatR(;^x)istwiceFrechet

PAGE 60

Proof. 8.6 ,let02Mbesuchthatthesupportof0iscontainedinf,x0isa(0;f)-optimalestimator,and2besuchthatsup2R(;^x)=R(;^x).Then2E(co(A))forsomeA22@.Let02f\E(co(A))besuchthatj(0)vij(i).BytheTaylortheorem,wehavethatforsome~2inthelinesegmentconnectingand0R(0;^x0)=R(;^x0)+1 2(0)TD2R^x0(~)(0):Certainlythereexistsa1;a2;:::;aN2Rsuchthat0=PNi=1aivi.Bythehypothesisofthetheorem,(0)TD2R^x0(~)(0)(0)TA0(0)sup2R(;^x0)=NXi=1a2iisup2R(;^x0):Sinceai=(0)vi,a2i(i)2.Thussup2R(;^x0)sup2fR(;^x0)R(;^x0)R(0;^x0)PNi=1(i)2i=2sup2R(;^x0).Itcanbeveriedthatsincesup2fR(;^x0)V(f;X;R)(1+0)andV(f;X;R)V(;X;R),thensup2R(;^x0)V(;X;R)(1+).Thus^x0isa-optimalestimator. 60

PAGE 61

9.1 ,appearsasaconjecturein[ 16 ].Weimposethefollowingadditionalconditionsinordertoderivetheresultsofthischapter: C0<+1.LetR(;^x)=r(;^x)+(1)R(;^x)andr(;^x)=RR(;^x)d,where01.LetK(^x;)=sup2R(;^x)andletK()=inf^x2XK(^x;).Sincesup2R(;^x)r(;^x),wehavethat 9.1 and 9.2 hold.ThenKisconcave,decreasing,andcontinuouson[0;1]. Proof. B LetG(^x)=sup2R(;^x)r(;^x). Proof. C 61

PAGE 62

Proof. 4.1 and 4.2 holdfor(;X;R),thereexistsaminimaxestimatorintheestimationproblem(;X;R).If,inaddition,iscompactandCondition 4.3 holds,thereexistsaleastfavorableaprioridistribution02M,andanyminimaxestimatorisaBayessolutionrelativeto0.AminimaxestimatorisanessentiallyuniqueminimaxestimatorandisadmissibleifCondition 4.4 holdsaswell.Infact,itcanbeshownthatif0<1,eachoneofConditions 4.1 4.4 holdsfor(;X;R)ifitholdsfor(;X;R). 7 fortheestimationproblem(;X;R),where0<1,providedthatConditions 7.1 7.2 holdfortheestimationproblem(;X;R).Infact,ifiscompact,itissufcientthatCondition 7.1 7.2 wouldholdfortheestimationproblem(;X;R).Indeed,supposeiscompactandConditions 7.1 7.2 holdfortheestimationproblem(;X;R).ItfollowseasilyfromLemma 9.3 thatforany2M,thereexistsanessentiallyuniqueBayessolutionrelativetointheestimationproblem(;X;R),whenceCondition 7.1 holds.Let^xdenotethe(essentiallyunique)Bayessolutionrelativetointheestimationproblem(;X;R).Letfig1i=1beasequenceinMthatconvergesweaklyto02M.Let0i=+(1)iforalli0.Thenf0igconvergesweaklyto00.SinceR(;^xi)=R(;^x0i)foralli0,R(;^xi)convergestoR(;^x0)uniformlyonthecompactset.Sincetheconvergenceisuniform,r(;^xi)convergestor(;^x0).ThusR(;^xi)convergestoR(;^x0)uniformlyoncompactsubsetsofandCondition 7.2 holds. 9.1 9.2 hold.LetQ()=+,whereG(^x)and2R.SupposeQ()>K()forall2[0;1].

PAGE 63

Proof. 9.1 ,Kisalsocontinuouson[0;1].Thusthereexists>0suchQ()K()>forall2[0;1].ItiseasytoverifythatfK(^x;):^x2X( Leti=i=nand^xibeaminimaxestimatorfor(;X;Ri)fori=0;1;:::;n.Notethat^xn=^xand^xiexistsforall0in1byTheorem 4.1 andRemark 9.1 .Since^x0isaminimaxestimatorfor(;X;R)and^xn=^x,G(^xn)G(^x0).ThusbyLemma 9.2 part1),thereexists1mnsuchthatG(^xm)G(^xm1).ByLemma 9.2 part3),^xm12X( Using( 9 ), By( 9 )and( 9 ),jK(m)~K(m)j<.Itfollows~K(m)
PAGE 64

9.1 9.2 hold,andV(;X;R)K(1).ThusQ()>K()forall01.Let=C0r(;^x0).ThenG(^x)andQ()=+C0.Let^xbeaminimaxestimatorfor(;X;R).Since^x0isarestrictedriskBayessolutionrelativeto(;C0)andC0>V(;X;R),r(;^x0)r(;^x).Thus>G(^x).ByLemma 9.4 ,thereexists^x02XsuchthatK(^x0;)K()forall01.Sincesup2R(;^x0)=C0,K(^x0;)=Q()andbyLemma 9.4 ,thereexists^x0suchthatsup2R(;^x0)
PAGE 65

9.1 9.2 hold.Letfng1i=1beasequencein[0;1]thatconvergesto02(0;1).Let^xibeaminimaxestimatorfor(;X;Ri)fori=0;1;:::.ThenifV(;X;R)0suchthatM>V(;X;R)andM+< C0.ByTheorem 4.2 ,thereexistrestrictedriskBayessolutionsrelativeto(;M)and(;M+).Therefore,Theorem 9.1 yields0and00in(0;1)suchthatsup2R(;^x0)=Mandsup2R(;^x00)=M+,where^xisaminimaxestimatorfor(;X;R)forall2(0;1).ByLemma 9.2 part3),0000.Infact,sincesup2R(;^x0)
PAGE 66

66

PAGE 67

Sinceforeach^x2X,sup2R(;^x)sup2R(;^x), By( 10 )and( 10 ),^xisaminimaxestimatorforx.Thuswiththeaboveriskfunction,minimaxestimationforxsubjectto(Sx;Sy;h)iscompletelydeterminedbyminimaxestimationforx(2Sx).Letusconsideranotherpossibilityfortheriskfunction.SupposeSxN.Letw:Sx![0+1).WecandenetheriskfunctionR(;^x)=X2SxR(;^x)w():

PAGE 68

1. IfConditions 4.1 and 4.2 holdfor(;X;R)foreach2Sx,thenConditions 4.1 and 4.2 holdfor(;X;R). 2. SupposeSxisnite.ThenifConditions 4.3 and 4.4 holdfor(;X;R)foreach2Sx,thenConditions 4.3 and 4.4 holdfor(;X;R). Proof. 4.1 holdsfor(;X;R).Foreach2SxletXbeacompactmetrizablespaceandRafunctionfromXinto[0;+1]suchthat(X;R)(X;R),R(;)islowersemicontinuousonXforeach2,andR(;a)2m(B())foreacha2X.LetX=Q2SxXbeequippedwiththeproducttopology.LetR(;^x)=P2SxR(;^x)w().ThenbyTychonoff'sTheorem,Xiscompact.Inaddition,Xismetrizablesinceitisacountableproductofmetrizablespaces.Sinceforeach2Sxand2,^x7!R(;^x)islowersemicontinuous,foreachnitesubsetSofSx,^x7!P2SR(;^x)w()islowersemicontinuous.Sincethepointwisesupremumofanycollectionoflowersemicontinuousfunctionisalowersemicontinuousfunction,wehavethatR(;)islowersemicontinuousonXforeach2.Sincethepointwiselimitofasequenceofmeasurablefunctionsisameasurablefunction,itiseasytoshowthatR(;^x)2m(B()).Itisalsoratherstraightforwardtoshowthat(X;R)(X;R).ThusCondition 4.2 holdsfor(;X;R).2)Theproofisratherstraightforwardandisomitted. 68

PAGE 69

10.1 impliesthatunderratherweakconditionson(;X;R)(2Sx),thereexistsaminimaxestimatorfor(;X;R)andarestrictedriskBayessolutionrelativeto(;C0)foreach2MandC0V(;X;R).ThetheoremisespeciallyusefulforthecaseofniteSxsinceitspeciesthatifcertainconditionsholdsforeachoneoftheestimationproblems(;X;R)(2Sx),theyholdfor(;X;R)andhencetheresultsofthepreviouschaptersarevalidfor(;X;R).

PAGE 72

10.3 72

PAGE 73

10.3 .Forthesakeofclarity,letusrepeattheformulationofthisproblem.Weconsiderthefollowingdiscrete-timelinearstochasticsysteminstate-spaceform: wherexn2RNx(n0)isthestatevector,yn2RNyisthesystemoutput,vn2RNyisthemeasurementnoise,wn2RNxisthemodelnoise,andHnandFnarematricesinRNyNxandRNxNx,respectively.Weassumethatthereisgivenanestimationspace(Sx;Sy;h),whereSxN,Sy=N,andforeach2Sx,h()isanitesubsetofSx.WeassumethatfHngandfFngareknownsequences,thesequencesfvngandfwngareuncorrelated,vnandvmareuncorrelatedforn6=m,andwnandwmareuncorrelatedforn6=m.Inaddition,theinitialstatex0andvn(wn)areuncorrelatedforalln0.Weassumex0isaGaussianrandomvectorandfvng(fwng)isasequenceofidenticalGaussianrandomvectors.Weassumethatthemeanandcovarianceofx0andthecovariancesofvnandwnareunknown;itisonlyknownthateachoneofthesequantitiesbelongtoacertainset.Thespaceofstatesofnature,inthiscase,is122,where1istheclassofpossiblemeanvectorandautocorrelationmatrixpairsforx0,2istheclassofpossiblemeanvectorandautocorrelationmatrixpairsforvn(n=1;2;:::),and3istheclassofpossiblemeanvectorandautocorrelationmatrixpairsforwn(n=1;2;:::).AsmentionedinExample 10.3 ,thesetisasubsetofanitedimensionalspacenormedspaceC(seethedenitionofCinExample 10.3 ).For=123,wherei2ifori=1;2;3,weleti()andi()denotethemeanvectorandautocorrelationmatrixofi,respectively,fori=1;2;3.Ourmainassumptionregardingthespaceisthat2()>0foreach2.WeconsidertheriskfunctionR(;^x)=P2SxR(;^x)w()forthisproblem,wherew()>0foreach2Sx, 73

PAGE 74

24 ],wheresuchconditionsarederivedforthespecialcaseofuncertaintiesintheinitialstate.LetusshowthatConditions 4.1 4.2 ,and 4.4 holdfortheestimationproblems(;D;R)and(;L;R)foreach2Sx,andthatCondition 4.3 holdsaswellifiscompact.Fix2Sx.OurrststepistoshowthatConditions 5.1 5.7 hold.SincethelossfunctionLisquadratic,itisonlylefttoshowthatConditions 5.2 and 5.4 hold(Chapter 6 ).ConsidertheconditionaldistributionofYh()givenX=x,whenisthetruestateofnature.LetP(jX=x)denotethisconditionaldistribution,whichiscertainlyGaussian.Moreover,since2()>0foreach2,thefamilyfP(jX=x):2;x2XgisdominatedbytheLebesgue-Borelmeasureon(Yh();B(Yh())),whichisdenoted.LetP(jX=x)denotetheconditionaldistributionofYh()givenX=x,whenisthetruestateofnature.LetA2B(Yh())besuchthat(A)=0.ThenP(AjX=x)=0foreachx2X.ItcanbeveriedthatthisimpliesthatP(AjX=x)=0foreachx2X.ThusthefamilyfP(jX=x):2;x2XgisdominatedbyandCondition 5.2 holds.PutZ=[YTh()XT]T.Letfngbeasequenceinthatconvergesto02.Theni(n)!i(0)andi(n)!i(0),fori=1;2;3.ItcanbeveriedthatthisimpliesthatEn(Z)!E0(Z)andEn(ZZT)!E0(ZZT).ItfollowsthatPnconvergesweaklytoP0.Sincethesequencefngisarbitrary,fPngconvergesweakly 74

PAGE 75

40 ].ThusCondition 5.4 holds.Itisalsoclearthat7!E(Z)and7!E(ZZT)arecontinuouson.ItnowfollowsfromtheresultsofChapter 6 thatConditions 4.1 4.2 ,and 4.4 holdfor(;D;R)and(;L;R)foreach2Sx,andCondition 4.3 holdsaswellifiscompact.ByTheorem 10.1 ,whether(;D;R)or(;L;R)isconsidered,thereexistsaminimaxestimator,andthereexistsarestrictedriskBayessolutionrelativeto(;C0)foreach2MandC0V(;D;R).SupposeinadditionthatSxisniteandiscompact.Thenwehavethefollowingresultsfortheestimationproblem(;D;R):(;D;R)isstrictlydetermined,thereexistsaleastfavorableaprioridistribution02Mandaconditionalmeanestimatorrelativeto0isanessentiallyuniqueadmissibleminimaxestimator.Moreover,theclassofconditionalmeanestimatorsisessentiallycomplete.Notethat,ingeneral,aconditionalmeanestimatorrelativetoisnotaLMMSEestimatorsincemaynotassignmass1toasinglepointin.Similarly,wehavethefollowingresultsfortheestimationproblem(;L;R):(;L;R)isstrictlydetermined,thereexistsaleastfavorableaprioridistribution02MandaLMMSEestimatorrelativeto0isanessentiallyuniqueadmissibleminimaxestimator.Considerthelteringproblem.ThenaLMMSEestimatorrelativetoisnotnecessarilyaKFifdoesnotassignmass1toasinglepointin.ThereisaspecialandimportantcaseinwhichaLMMSEestimatorwithrespectto2MisaKF.Wewilltreatthiscaseinthesequel.Inthefollowingchaptersweconsidertwospecialcasesoftheaboveproblem.Therstcaseisthecaseofstochasticuncertaintiesintheinitialstate,modelnoise,andobservationnoisewithLastheclassofavailableestimators.ThesecondcaseisthecaseofdeterministicuncertaintiesintheinitialstatewithDastheclassofavailableestimators.InbothcaseswewillassumethatSxisniteandiscompact.NotethatbyRemark 4.1 ,thecasethatisbounded,butnotnecessarilycompactisalsocovered.Thesetwocasesareimportantontheirownmerit,andtheywillalsobeusedtoillustratesomeofthegeneralresultsofpreviouschapters.Forexample,therstcasewillbeusedtoillustratethemethodproposedinChapter 9 toderivearestrictedriskBayessolution 75

PAGE 76

9.3 )andthesecondcasewillbeusedtoillustratethemethodproposedinChapter 8 toderiveanapproximationforminimaxestimators. 76

PAGE 77

11 .Weconsiderthecaseofstateestimationwithstochasticuncertaintiesintheinitialstate,modelnoise,andobservationnoisewiththeclassofavailableestimatorsbeingtheclassofafneestimators.Throughoutthischapter,weassumeiscompact.ThusV(;L;R)<+1.Inthischapter,weassumethattheestimationspace(Sx;Sy;h)issuchthatSx=f0;1;:::;ntg,Sy=N,andh(k)=f0;1;:::;nkgforeachk2Sx,wherentandnk,fork=0;1;:::;nt,arenonnegativeintegers.Weassumethatw(k)=1foreachk2Sx,i.e.,R(;^x)=Pntk=0Rk(;^xk).GivenC0V(;L;R)and2,ourgoalistondarestrictedriskBayessolutionrelativeto(;C0). wherewn0=[wT0wT1wTn]Tandvn0=[vT0vT1vTn]T.Letk0andn0.By( 12 )and( 12 ),Ayn0+bxk=(AOnFk1;0)x0+n_k1Xi=0(ATi+1nFk1;i+1)wi+Avn0+b:

PAGE 78

Let()=[1()T2()T3()T]Tandlet()betheblockdiagonalmatrixwith1(),2(),and3()initsdiagonalblocks.Let;i()=i()i()i()Tfori=1;2;3.Weassumethatsomeaprioriinformationregardingthetruestateofnature2isavailable.Thecasethatnoaprioriinformationisavailableisanimportantspecialcase.Theaprioriinformationisgivenintheformofanominal2.Forsimplicity,weassume1()=0.Thereisnolossofgeneralityinthisassumptionsinceif1()6=0,wecantranslatethestatesandobservationsandbringtheproblemtothisform.Below,wesummarizetheassumptionstakensofartogetherwithsomenewassumptions. 78

PAGE 79

12.1 12.4 holdinthesequel.NotethatisthenacompactconvexsubsetofC.By( 12 ),Rk(;^xk)isbothconvexandconcaveonforall^xk2Lk,i.e.,forany0<<1and0;002,Rk(0+(1)00;^xk)=Rk(0;^xk)+(1)Rk(00;^xk):Inaddition,Rk(;^xk)iscontinuousonforall^xk2Lk.ItfollowsthatR(;^x)isbothconvexandconcaveonandcontinuousonforall^x2L.Letr(;^x)=RR(;^x)dandletrk(;^xk)=RRk(;^xk)dfork=0;1;:::;nt.LetZbetheclassofallnitesubsetsof.Let~MdenotethespaceofdistributionsinMwithnitesupport.Let2~M.ThenthereexistsZ2Zand1;:::;jZj2suchthatZ=f1;:::;jZjgand(Z)=1.Inthiscase,let(i)denotethemassthatassignstothepointifori=1;:::;jZj.Finally,let=PjZji=1i(i).Sinceisconvex,2.SinceR(;^x)isbothconvexandconcaveonforall^x2L,r(;^x)=R(;^x)forall^x2L. Proof. 27 ,Appendix3].Letfigbeasequenceofdistributionsin~Mthatconvergesweaklyto.SinceR(;^x)iscontinuousonthecompactset,R(;^x)isboundedand 79

PAGE 80

SinceR(;^x)isbothconvexandconcaveonandihasnitesupport, By( 12 )( 12 ),r(;^x)=R(;^x).Theprooffollowsfromthearbitrarinessof^x. ItisclearfromtheproofofLemma 12.1 thatif2~M,^xisaBayessolutionrelativetoifandonlyif^xisaBayessolutionrelativeto,where=.WewanttoapplytheresultsofChapter 9 fortheestimationproblem(;L;R).WehavealreadyshownthatConditions 4.1 4.4 holdinChapter 11 .LetusshowthatConditions 7.1 7.2 ,and 9.1 holdaswell.InChapter 11 itwasshownthatLhastheproperty(W)andCondition 4.4 holds.ThusTheorem 4.2 impliesthatCondition 7.1 holds.CertainlytheweakerCondition 9.1 mustholdaswell.ItislefttoprovethatCondition 7.2 holdsfor(;L;R).Infact,itissufcienttoprovethatCondition 7.2 holdsfor(;Lk;Rk)foreachk2Sx.Fixk2Sx.Forthesakeofclarity,weprovethatCondition 7.2 holdsfor(;Lk;Rk)inthecasethat1()=0forall2.ItiseasytoverifythatthisistruealsointhemoregeneralcaseofAssumption 12.2 buttheexpressionsarerathercumbersome.Assuming1()=0forall2andusing( 12 ), 80

PAGE 81

whereweextendk;ifromto~Mbydeningk;i()=k;i()forall2~M(i=1;2).LetfigbeasequenceinMthatconvergesweaklyto02M.ByLemma 12.1 ,thereexistsasequencefig2andanelement02suchthatRk(;^xi;k)=Rk(;^xi;k)(i=0;1;:::).Certainlyk;iiscontinuousonfori=1;2.Itfollowseasilyfrom( 12 )thatthemapping7!^x;kiscontinuousonthecompactsetandhenceuniformlycontinuousandbounded.Thusthereexists>0suchthat^x;k2Bforall2,whereB=f^xk2Lk:jj^xkjjLkg.ItiseasytoseethatRkiscontinuousonthecompactsetB.ThusfRk(;):2gisequicontinuousonB.Since7!^x;kisuniformlycontinuouson,thefamilyf7!Rk(0;^x;k):02gisequicontinuousonandthereforeRk(;^xi;k)convergestoRk(;^x0;k)uniformlyon.ItfollowsthatRk(;^xi;k)convergestoRk(;^x0;k)uniformlyonandCondition 7.2 holds.Let^xdenotethe(essentiallyunique)Bayessolutionrelativeto2Mintheestimationproblem(;L;R).SinceiscompactandR(;^x)iscontinuouson, 9.2 holdsifV(;L;R)< C0.Let^x=^x+(1).ByLemma 9.3 ,^xisaBayessolutionrelativetointheestimationproblem(;L;R).Letr(;^x)=r(;^x)+(1

PAGE 82

4.2 4.4 7.1 ,and 7.2 holdfortheestimationproblem(;L;R),theyholdfortheestimationproblem(;L;R)for0<1(Remarks 9.1 and 9.2 ).ThususingtheresultsofthepreviouschaptersandLemma 12.1 ,if0<1,thereexists02suchthat~r(0)=sup2~r()and^x0isanadmissible,essentiallyuniqueminimaxestimator.Inparticular,bysetting=0,thereexistsaminimaxestimatorfor(;L;R).Letusconsidertheproblemofndingaminimaxestimatorfortheestimationproblem(;L;R),where2[0;1).Asmentionedearlier,ifV(;L;R)
PAGE 83

Proof. 2(+0).Then20.ByLemma 7.1 ,~r()~r().Thusthereexistsaleastfavorableaprioridistribution0thatisin0.Certainly0isalsoaleastfavorableaprioridistributionintheestimationproblem(0;L;R).Since0<1,R(;^x)=sup2R(;^x)ifandonlyifR(;^x)=sup2R(;^x).Since0isacompactsubsetof,wemayapplyTheorem 7.1 fortheestimationproblem(0;L;R)andderivetheabovealgorithm.NotethatthealgorithmissimpliedwiththehelpofLemma 12.1 sinceweneedtoconsideronlydistributionswithsupportofasinglepoint,i.e.,elementsof.Sincewehaveconsideredtheestimationproblem(0;L;R),thesequencefig1i=1isin0.ByTheorem 7.1 ,thesequenceconvergesweaklytoaleastfavorableaprioridistribution020.Thus^x0isaminimaxestimator.Since(i)=0and()=0,^xi2L0foralli0.SinceCondition 7.2 holds,R(;^xi)convergesuniformlyontoR(;^x0). ThemainstepsofthealgorithmofLemma 12.2 areSteps2and4.InStep2,weneedtosolvetheproblemofndingamaximizerofR(;^x)over0.WewilladdressthisprobleminSection 12.2 .Step4canbedoneusingnumericalmethodsasdescribedinRemark 7.1 .SupposeV(;L;R)
PAGE 84

12.2 ,weaddressthisproblemandareabletosolveitforsomeimportantcasesof. 12.2 andinthemethodtondarestrictedriskBayessolution,whichisdiscussedinRemark 9.3 .Weconsiderspeciccasesoftheparameterset0.Thereisoneimmediatecaseinwhichthisproblemhasasimplesolution.Thedenitionofanextremepointisneeded(Denition 21 ).RecallthatgivenasetA,weuse@AtodenotetheextremalboundaryofA,whichisthesetofallextremepointsofA.Since0isconvexandcompactandR(;^x)isconvexandcontinuouson,byBauer'sminimumprinciple[ 41 ],sup20R(;^x)=sup2@0R(;^x):If@0isnite,sup20R(;^x)=max2@0R(;^x).Thussup20R(;^x)canbeeasilycalculated.Fix^x2L0.By( 12 ),R(;^x)=P3i=1tr(i()i)forsome12SNx+,22SNy+,and32SNx+.Leti=fi():2gfori=1;2;3.Let=f(1();2();3()):2g.Itfollowsfromthedenitionofthat=123.Thussup20R(;^x)=sup(1;2;3)23Xi=1tr(ii)=3Xi=1supi2itr(ii):Therefore,weareleftwiththefollowingoptimizationproblem:GivenN>0,amatrix2SN+,andaconvexcompactsubsetAofSN+, maximizetr(A)subjecttoA2A: ConsidertheimportantcaseinwhichA=fA2SN+:f1(A)0;:::;fN(A)0g,wheref1;:::;fNareconvex(real-valued)functionssuchthatAiscompactandconvex.Then( 12 )is 84

PAGE 85

PAGE 86

PAGE 87

PAGE 88

wheren()=(;^xn()),njn1()=(;^xnjn1()),andKn()denotestheKalmangain.Considerthelteringproblemspeciedbytheestimationspace(Sx;Sy;h),whereSx=f0;:::;ntg,Sy=N,andh(i)=f0;:::;igforeachi2Sx.Then^x=(^x0();^x1();:::;^xnt())andisreferredtoastheKFrelativeto.Since2()>0,theexistenceoftheKFisguaranteed[ 44 ].ItiseasytoverifythatRk(;^xk(0))=tr(Wk(;^xk(0))).Usingtheresultsoftheprevioussections,thereexistsa2suchthattheKFrelativetoisarestrictedriskBayessolutionrelativeto(;C0)forallC0V(;L;R).Infact,thereisanotherinterestingpropertyregardingtheKF.UsingCorollary 9.1 andLemma 12.1 ,wehavethattheclassofKFsrelativeto2isessentiallycomplete.Thusaslongastheperformanceisjudgedsolelybasedontheriskfunction,ifthechoiceofestimatorsisrestrictedtoafneestimators,thennomatterwhatoptimalitycriterionisused,onemayconsideronlytheclassofKFsrelativeto2.OurnextstepistoderivemoreconvenientexpressionforRk(;^xk(0))inthecasethatand0arein0.Letk(;0)=(;^xk(0)).Using( 12 )( 12 ),itcanbeshownthat0j1(;0)=1()n(;0)=Kn(0)2()Kn(0)T+[IKn(0)Hn]njn1(;0)[IKn(0)Hn]Tn+1jn(;0)=3()+Fnn(;0)FTn:

PAGE 89

12 ).Recallthatifand0arein0,thenR(;^x0)=P3i=1tr(i()i(0)).Ourgoalistondanexpressionfori(0),fori=1;2;3,sinceitisneededinordertondamaximizerofR(;^x0)over0.Let~Fn(0)=FnFnKn(0)Hn,let~Fi;j(0)=~Fi(0)~Fi1(0)~Fj(0)(i>j)and~Fi;i(0)=~Fi(0).Wewillusetheconvention~Fi1;i(0)=I.LetCn;m(0)=(IKn(0)Hn)~Fn1;m(0)(0mn;0n),letDn;m(0)=Cn;m+1(0)FmKm(0)(n>m0)andDn;n(0)=Kn(0)(n0).NotethatCn+1;m=(IKn+1Hn+1)FnCn;mandCn;m1=Cn;m~Fm1.Thenn(;0)=Cn;0(0)1()Cn;0(0)T+nXm=0Dn;m(0)2()Dn;m(0)T+nXm=1Cn;m(0)3()Cn;m(0)T:Leti(0)=Pntn=iCn;i(0)TWnCn;i(0),fori=0;1;:::;nt.Theni=~FTii+1~Fi+(IKiHi)TWi(IKiHi):Itfollowsthat1(0)=ntXn=0Cn;0(0)TWnCn;0(0)=0(0);2(0)=ntXn=0nXm=0Dn;m(0)TWnDn;m(0)=ntXm=1Km1(0)TFTm1m(0)Fm1Km1(0)+ntXm=0Km(0)TWmKm(0)and;3(0)=ntXn=1nXm=1Cn;m(0)TWnCn;m(0)=ntXm=1m(0):Ingeneral,thecalculationof2(0)and3(0)requiresthestorageofK0(0);:::;Knt(0),whichmaybeproblematicforlargent.Thisisduetothefactthaticanbeupdatedbasedoni+1butnotviceversa.Nevertheless,intheimportantcasethatFiisinvertiblefor0int1,thecalculationof1,2,and3maybedoneinsuchawaythatthereisonlyaneedofaxedstorageplacethatdoesnotdependonnt.First,letusshowthatsince2(0)isinvertible, 89

PAGE 90

PAGE 91

PAGE 92

12.1 holdsinthisexample.WehaveshownthatAssumption 12.2 holdsinExample 12.1 .ItcanbeeasilyveriedthatAssumptions 12.3 and 12.4 holdaswell.OurrststepistochooseC0.InordertochooseC0,weneedtocalculateV(;L;R)and 12.2 tocalculate^xfor2[0;1)andthencalculater(;^x)andsup2R(;^x).Fig. 12-1 showstheplotoftheBayesriskr(;^x)versusthemaximumrisksup2R(;^x).NotethatbyTheorem 9.1 ,thisgure,infact,showstheBayesriskachievedbyarestrictedriskBayessolutionrelativeto(;C0)versusC0.HencethisguretellsusthetradeoffbetweenthepenaltyontheBayesriskandthesafeguardonthemaximumriskbyemployingrestrictedriskBayesestimation.Inthisexample,V=1:225105and 92

PAGE 93

PAGE 94

AchievedBayesriskvs.maximumrisk 94

PAGE 95

ThemaximumriskoveroftheBayes,minimax,andrestrictedriskBayessolutionsvs.

PAGE 96

PAGE 97

44 ]byinitializingtheKFwithzeromeanandcovariance.Itisstraightforwardtoverifythatn(x0)=0nforallx02X0.Let0n+1jn=3+Fn0nFTn,K0n=0njn1HTn[2+Hn0njn1HTn]1forn>0,andK00=0.Let~Fn=FnFnK0nHn,let~Fi;j=~Fi~Fi1~Fj(i>j)and~Fi;i=~Fi.Wewillusetheconvention~Fi1;i=I.TheproblemofstateestimationwitharandominitialstatewhosedistributioninnotnecessarilyGaussianisconsideredbyLainiotisetal[ 46 ]andtheCMestimatorforthisproblemisderived.Let^x0;n()denotetheCMestimatorwithrespecttoforx0basedonyn0,i.e.,^x0;n()=RX0x0pnx0(yn0)d 97

PAGE 98

PAGE 99

8 .Ourgoalisthentoderivesuboptimalestimatorsthatcangivemaximumriskarbitrarilyclosetothatofaminimaxestimator.ThefollowingLemmasarederivedin[ 24 ]. BecauseofLemmas 13.1 and 13.2 andthefactthatX0isasubsetofnitedimensionalnormedspace,wecanusetheresultsofChapter 8 tonda-optimalestimator.InparticularifX0isaconvexpolytope,Lemma 8.7 canbeusedtonda-optimalestimator.Inordertonda-optimalestimator,weneedtoseta0<0
PAGE 100

7.1 ,intermsofthecomputationburden,sinceitmayrequirelesscalculationsoftheriskfunction.Thecomputationalcomplexityofa-optimalestimatorderivedaccordingtoLemma 8.7 dependslinearlyonthesupportof0.Itmayberatherhigh,anditiscertainlyhigherthanthecomplexityofafneestimators.InChapter 12 wediscussedtheproblemofrestrictedriskBayesestimationwhentheclassofestimatorsisrestrictedtoafneestimators.Whentheclassofestimatorsisrestrictedtoafneestimators,theminimaxproblemofthissectioncanberegardedasaspecialcaseoftheproblemconsideredinChapter 12 .Ifwecanndanafneestimatorthatisalsoa-optimalestimator,wewouldprobablyprefertheafneestimator.Ifsuchanafneestimatorexists,theresultsofthissectionarestillimportantsincetheygiveustightlowerandupperboundsonthemaximumriskandhenceenableustoevaluatetheperformanceoftheafneestimatorwiththebestpossibleperformanceinthesenseofthemaximumrisk.Wefurtherillustratethisinthefollowingnumericalexample. whereF=264101375andwniszeromeanGaussianrandomvectorwithcovariancematrix3=264000q375:

PAGE 101

PAGE 102

PAGE 103

AfullviewofXf0,whichisanite(;V)-densesubsetofX0

PAGE 104

Azoom-inviewofthebottomleftcornerofXf0,whichisanite(;V)-densesubsetofX0 Theriskof^x(0),thederived-optimalestimator,asafunctionofx02Xf0

PAGE 105

Theaprioridistribution0,whichisdenedonXf0 ThemaximumriskoftheKalmanFilterinitializedwithzeromeanandcovariance2Iasafunctionof

PAGE 106

106

PAGE 108

108

PAGE 109

TheconcavityofKfollowsfromthefactthatforany0<<1,K(1+(1)2)=inf^x2XK(^x;1+(1)2)inf^x2XK(^x;1)+(1)inf^x2XK(^x;2)=K(1)+(1)K(2);wheretheinequalityaboveisadirectresultof( B ).Eqn.( 9 )impliesthatK(1)K(2)forany01<21.SinceKisdecreasingandconcaveon[0;1],itiscontinuousexceptperhapsatthepoint=1.CertainlyK(^x;1)=K(1)andK(^x;)K()forall01.Thusforany0<1,jK()K(1)jK(^x;)K(^x;1)=(1)sup2R(;^x)r(;^x)(1) 109

PAGE 110

wherethesecondinequalityisadirectconsequenceof( 9 ).ThisimpliesthatK(^x2;1)K(^x2;2)K(^x1;1)K(^x1;2).IteasilyfollowsthatG(^x1)G(^x2).2)By( C ),K(^x1;2)K(^x2;2)0.Thusr(;^x2)+(12)[G(^x2)G(^x1)]r(;^x1):Bypart1)ofthelemma,r(;^x2)r(;^x1).3)By( C ),K(^x2;1)K(^x1;1)0.Thussup2R(;^x1)+1[G(^x2)G(^x1)]sup2R(;^x2):Bypart1)ofthelemma,sup2R(;^x1)sup2R(;^x2). 110

PAGE 111

[1] R.E.Kalman,Anewapproachtolinearlteringandpredictionproblems,ASMETransactions,JournalofBasicEngineering,vol.82,pp.34,Mar.1960. [2] J.Heffes,TheeffectoferronousmodelsontheKalmanlterresponse,IEEETrans.Automat.Contr.,vol.AC-11,pp.541,Apr.1966. [3] T.Nishimura,Ontheaprioriinformationinsequentialestimationproblems,IEEETrans.Automat.Contr.,vol.AC-11,pp.197,Apr.1966. [4] T.Nishimura,ErrorboundsofcontinuousKalmanltersandtheapplicationtoorbitdeterminationproblems,IEEETrans.Automat.Contr.,vol.AC-12,pp.268,Jun.1967. [5] M.Mintz,AnoteonminimaxestimationandKalmanltering,IEEETrans.Automat.Contr.,vol.AC-14,pp.588,Oct.1969. [6] J.M.Morris,TheKalmanlter:Arobustestimatorforsomeclassesoflinearquadraticproblems,IEEETrans.Inform.Theory,vol.IT-22,pp.526,Sep.1976. [7] V.PoorandD.P.Looze,Minimaxstateestimationforlinearstochasticsystemswithnoiseuncertainty,IEEETrans.Automat.Contr.,vol.AC-26,pp.902,Aug.1981. [8] C.J.MartinandM.Mintz,Robustlteringandpredictionforlinearsystemswithuncertaindynamics:Agame-theoreticapproach,IEEETrans.Automat.Contr.,vol.AC-28,pp.888,Sep.1983. [9] S.VerduandH.V.Poor,Onminimaxrobustness:Ageneralapproachandapplications,IEEETrans.Inform.Theory,vol.IT-30,pp.328,Mar.1984. [10] S.VerduandH.V.Poor,Minimaxlinearobserversandregulatorsforstochasticsystemswithuncertainsecond-orderstatistics,IEEETrans.Automat.Contr.,vol.AC-29,pp.499,Jun.1984. [11] J.C.DarraghandD.P.Looze,Noncausalminimaxlinearstateestimationforsystemswithuncertainsecond-orderstatistics,IEEETrans.Automat.Contr.,vol.AC-29,pp.555,Jun.1984. [12] B.I.Anan'ev,Onminimaxstateestimatesformultistagestatisticallyuncertainsystems,ProblemsofControlandInformationTheory,vol.18,pp.27,1989. [13] Y.L.ChenandB.S.Chen,Minimaxrobustdeconvolutionltersunderstochasticparametricandnoiseuncertainties,IEEETrans.SignalProcessing,vol.42,pp.32,Jan.1994. [14] B.I.Anan'ev,Minimaxestimationofstatisticallyuncertainsystemsunderthechoiceofafeedbackparameter,JournalofMathematicalSystems,Estimation,andControl,vol.5,pp.1,1995. 111

PAGE 112

[15] J.O.Berger,StatisticalDecisionTheoryandBayesianAnalysis,Springer,NewYork,2ndedition,1985. [16] J.L.Hodges,Jr.andE.L.Lehmann,Theuseofpreviousexperienceinreachingstatisticaldecisions,Ann.Math.Stat.,vol.23,pp.396,Sep.1952. [17] B.EfronandC.Morris,LimitingtheriskofBayesandempiricalBayesestimatorspart1:TheBayescase,J.Amer.Statist.Assoc.,vol.66,pp.807,Dec.1971. [18] P.J.Kempthorne,Numericalspecicationofdiscreteleastfavorablepriordistributions,SIAMJ.Sci.Statist.Comput.,vol.8,pp.171,Mar.1987. [19] P.J.Kempthorne,Controllingrisksunderdifferentlossfunctions:Thecompromisedecisionproblem,Ann.Statist.,vol.16,pp.1594,Dec.1988. [20] I.M.Johnstone,Onminimaxestimationofasparsenormalmeanvector,Ann.Statist.,vol.22,pp.271,Mar.1994. [21] R.F.Berg,Estimationandpredictionformaneuveringtargettrajectories,IEEETrans.Automat.Contr.,vol.AC-28,pp.294,Mar.1983. [22] M.H.KaoandD.H.Eller,MulticongurationKalmanlterdesignforhigh-performanceGPSnavigation,IEEETrans.Automat.Contr.,vol.AC-28,pp.304,Mar.1983. [23] L.DanyangandL.Xuanhuang,Optimalstateestimationwithouttherequirementofaprioristatisticsinformationoftheinitalstate,IEEETrans.Automat.Contr.,vol.39,pp.2087,Oct.1994. [24] Y.LevinbookandT.F.Wong,Stateestimationwithinitialstateuncertainty,IEEETransactionsonInformationTheory,2005,Submittedforpublication.URL: [25] A.Wald,StatisticalDecisionFunctions,JohnWileyandSons,NewYork,1950. [26] T.S.Ferguson,MathematicalStatistics:ADecisionTheoreticApproach,AcademicPress,NewYork,1967. [27] P.Billingsley,ConvergenceofProbabilityMeasures,JohnWiley&Sons,NewYork,1968. [28] W.Rudin,PrinciplesofMathematicalAnalysis,McGraw-Hill,NewYork,3rdedition,1976. [29] D.W.Stroock,ProbabilityTheory,AnAnalyticView,CambridgeUniversityPress,NewYork,1993. [30] L.LeCam,AnextensionofWald'stheoryofstatisticaldecisionfunctions,Ann.Math.Stat.,vol.26,pp.69,Mar.1955. [31] H.Kudo,Ontheproperty(W)oftheclassofstatisticaldecisionfunctions,Ann.Math.Stat.,vol.37,pp.1631,Dec.1966.

PAGE 113

[32] M.Sion,Ongeneralminimaxtheorems,PacicJ.Math.,vol.8,pp.171,1958. [33] G.K.Pedersen,AnalysisNow,Springer-Verlag,NewYork,1989. [34] A.N.Shiryaev,Probability,Springer-Verlag,NewYork,2ndedition,1989. [35] J.L.Doob,MeasureTheory,Springer-Verlag,NewYork,1994. [36] W.Rudin,FunctionalAnalysis,McGraw-Hill,NewYork,2ndedition,1991. [37] E.L.Lehmann,TheoryofPointEstimation,Wiley,NewYork,1983. [38] W.Nelson,Minimaxsolutionofstatisticaldecisionproblemsbyiteration,Ann.Math.Stat.,vol.37,pp.1643,Dec.1966. [39] J.Wolfowitz,On-completeclassofdecisionfunctions,Ann.Math.Stat.,vol.22,pp.461,Sep.1951. [40] C.D.AliprantisandK.C.Border,InniteDimensionalAnalysis:AHitchhiker'sGuide,Springer-Verlag,Berlin,1994. [41] N.M.Roy,Extremepointsofconvexsetsininnitedimensionalspaces,Amer.Math.Monthly,vol.94,pp.409,May1987. [42] S.BoydandL.Vandenberghe,ConvexOptimization,CambridgeUniversityPress,Cambridge,2004. [43] L.Mirsky,AtraceinequalityofJohnvonNeumann,MonatsheftefurMathematik,vol.79,pp.303,1975. [44] T.Kailath,A.H.Sayed,andB.Hassibi,LinearEstimation,PrenticeHall,NJ,2000. [45] S.S.Blackman,MultipleTargetTrackingwithRadarApplications,ArtechHouse,Washington,DC,1986. [46] D.G.Lainiotis,S.K.Park,andR.Krishnaiah,Optimalstate-vectorestimationfornon-Gaussianinitialstate-vector,IEEETrans.Automat.Contr.,vol.AC-16,pp.197,Apr.1971.

PAGE 114