<%BANNER%>

Gaussian Mixture Model Based System Identification and Control

xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20110217_AAAADY INGEST_TIME 2011-02-18T03:09:04Z PACKAGE UFE0014640_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 1436 DFID F20110217_AACZCN ORIGIN DEPOSITOR PATH lan_j_Page_044.txt GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
e13287d1240f473c566c6ec949ec3138
SHA-1
fb0ce3c12ec88797ebad11be4506a97374566710
41942 F20110217_AACYXH lan_j_Page_061.pro
52c986b581d9754d4e31be989b978a91
18a157ef1fc956fdda8f18dd0633efdce77af88f
104 F20110217_AACZBY lan_j_Page_002.txt
96708e49d1bea321fd7f7c1d2d59c804
70cbaff0e852908ed8cb8e64bbaf5f73f537216a
8423998 F20110217_AACYWS lan_j_Page_023.tif
c5f938df46fef419e9c739e12e641613
9a09ccbe3b30a75e94d5f992605d2a58ba1663d5
1411 F20110217_AACZDB lan_j_Page_064.txt
2eebb0efe5bcf7d6f8f3d9fc98384418
108ed791279c4090f5de55c7b6ee6de206633b8e
1017 F20110217_AACZCO lan_j_Page_045.txt
4d72ad6c018bbed1b3aa1c53677961ec
9d658ac0315aac65d234f6dc4824a4c5546cf600
980 F20110217_AACYXI lan_j_Page_062.txt
960bb3ae58ae80f82a50d7dbb29d4b97
49c06d07cf53a4d5f60cc013cff029eefccc30f1
2035 F20110217_AACZBZ lan_j_Page_005.txt
fbc1833de48f4a48b9ca81d12c2e1544
6e3c6c350ba995594a3431889c24ded8cd050b06
F20110217_AACYWT lan_j_Page_072.tif
25e714068c14e0c30dc239817fe88412
833dee67e0e39945a2a17c62d030f727404661fe
1102 F20110217_AACZDC lan_j_Page_070.txt
a0f63e0e6bb7c7fb6a56297b6be2c2f0
a17165c3d62efa67ee5dc1d4810d5e23addc1a9b
1862 F20110217_AACZCP lan_j_Page_047.txt
7c9c790240f80728d922ed357736f28f
30c701ef37266cdfcb1ac1c869f580b17f4228eb
23150 F20110217_AACYXJ lan_j_Page_039.pro
3ad4fb9bc71710a89917f76d3d87ebd8
65ee65db34f06fd3387fc5ccf726b539cfd63f96
51353 F20110217_AACYWU lan_j_Page_085.jpg
a42a47a5bee3dc1334c8810fa76a021d
6ced8467e41d2677a5c6eb81d89f4fdf1780daf6
1530 F20110217_AACZDD lan_j_Page_071.txt
f61f697b4beb3c640257327cd92a0148
b9837ed3af12a338b22772111714959da699adfe
1445 F20110217_AACZCQ lan_j_Page_048.txt
c27354c0728763a9a4018c58b1a3727d
9b9a8c9278d04c97ad05d9143f6bdb20c0023e50
20126 F20110217_AACYXK lan_j_Page_080.QC.jpg
5f4ca3c48033454f952215e2eb430f0d
ecdfbe190d291ae947e8c85062deb2712accf959
1431 F20110217_AACYWV lan_j_Page_105.txt
6127f7967e07ec20cb06a870f6ad9837
b4d9b82896103e5b72988230ed97b1db5982c3c4
1848 F20110217_AACZDE lan_j_Page_072.txt
77415534bebb1b25793b2fbf1f354c07
95aa0e7af2e97b1829e06b765b2f16ea53c2cf7d
1652 F20110217_AACZCR lan_j_Page_049.txt
731cb60fbc244264f9177e5eddd98827
9aff768e4f219ee1bcd5934a245c8939c9a16411
810112 F20110217_AACYXL lan_j_Page_059.jp2
8866f3895865f474c3da6ad8cd67bbb8
21dca6e5bc740b4996229f4e8665f1227e48ccda
65267 F20110217_AACYWW lan_j_Page_098.jpg
c1a7630787c912a5d2de73df47d56c7a
1ed645d40fb004288198238095b3bfae4723dd6c
1335 F20110217_AACZDF lan_j_Page_073.txt
a791442a522d7ca689de206b28a85c28
b382d0224e6cbd6b6094c345505cba131f9ace97
7674 F20110217_AACYYA lan_j_Page_001.pro
1f3e4dc905812350eb9c750086fddfd7
41266d5eed21fb75443650851b39c715e78c6c40
1476 F20110217_AACZCS lan_j_Page_051.txt
4088f7bd92a227e8fad98447676b6753
73176881200720caa62f44d6a53f688ce96df4ca
6361 F20110217_AACYXM lan_j_Page_059thm.jpg
0dc8a7f03bd0bbcc23f8e5802aaffa6c
540f6f20f95281624fe504429d1f59ffd007947d
11389 F20110217_AACYWX lan_j_Page_128.QC.jpg
d93728aa37673a1a38b484246dccae62
0ecec945d2425dacdb6a425a87b9863c3e5f26fd
1788 F20110217_AACZDG lan_j_Page_074.txt
7a58a37c5b301a1de42b5e4fe4a23cd0
0e1b1e06646cafd2a494d8cecb414ab982e821be
7237 F20110217_AACYYB lan_j_Page_026thm.jpg
46997ecfb93b2b4a3225b1823fa5ae20
cdd9d1cfc53fce98e6c986d25c383310fcc49a13
1334 F20110217_AACZCT lan_j_Page_052.txt
a30a915a9eff300f2317bae793d3c80e
a7e8dd5009c0e9f1e90fb1108439e68c84ac0a66
715497 F20110217_AACYXN lan_j_Page_058.jp2
1c11d3633e38dfa52b0a4d04d49db1a8
ba7495878be580b58e8230752fdec0200758397c
1486 F20110217_AACYWY lan_j_Page_053.txt
1e725c1054a2bdbe4fad020b72c85696
281fd4dc9f936f9ef83d34b4efd2ac1b5ea12afb
1168 F20110217_AACZDH lan_j_Page_078.txt
98c8b2d0e529a8f3d118aa2332dcfaa3
2e2a956fe748be2b0d16c7a919b9ce5aec63353a
F20110217_AACYYC lan_j_Page_021.tif
65e29003dbae9f59c74b7e5e762754cb
1b9b8f79c44cd1e1be7e54a855fc532e2c7d68c5
1595 F20110217_AACZCU lan_j_Page_055.txt
4c96a33ed45e35cf30ab2c832790fe04
d881a21530a090c950e46ab4b404d65131a78b9e
1011 F20110217_AACYWZ lan_j_Page_098.txt
267a39d69881898d5e1957bb4a120f85
124409205b21e45712c07c1517bd5f4ca7ebcc62
1136 F20110217_AACZDI lan_j_Page_080.txt
2fac316abd9db4928c5414669a60ee9f
ed2e93a49dc6717fcedc2b3bae434e70dbab385e
49933 F20110217_AACYYD lan_j_Page_050.pro
0eb8a27df983eaddfefb8223491c86e1
91c038afd00082e3a5f2f31d120fa079aa8c4b96
1311 F20110217_AACZCV lan_j_Page_056.txt
99b808409375c5071061c3d8a0e28ce1
8e50842b996dd1781e2acd7a25bf1604d9d8d8d8
91076 F20110217_AACYXO lan_j_Page_047.jpg
b3ec29331def93dbe7b8786bde8cb4b5
2ff53959189458a42945e2e880a3a92ef998b467
1620 F20110217_AACZDJ lan_j_Page_082.txt
95797160ed6831e8eb16378ae47d8758
f2499753a0cf28764415f0ac5462773ec0331f1e
74663 F20110217_AACYYE lan_j_Page_021.jpg
1b5e78f97f16f0d427e7b63ade23152c
bd55ad730352fde9b6a46855148feefa3b4c622f
1024 F20110217_AACZCW lan_j_Page_057.txt
5dd573e00bca79a082081cadfe85576f
d84a347c5e4abae0b18ea3604c58f9b897c4630e
5844 F20110217_AACYXP lan_j_Page_057thm.jpg
1713949cb37c194073d43e731c6031f6
7ee02d96f1f5c74824968396f9d0c1e44867904c
1247 F20110217_AACZDK lan_j_Page_085.txt
24411b451a885843882db0cb5ef7cbcb
be73fb8d72deb7af20efc299fe9176ed4f2d3e44
23489 F20110217_AACYYF lan_j_Page_029.QC.jpg
63231936cd2460ee47c0a2a9eb344fb5
d90030ecb4af42be1a0eb287dc4d8ffcde454a1a
1394 F20110217_AACZCX lan_j_Page_058.txt
c154b575ec919d539749265a230a9b64
dfa74b1e2be7343c5556745f6864bf7ec0b9c966
1051951 F20110217_AACYXQ lan_j_Page_125.jp2
4db55781ef182faf365e3b10066a78d8
37090269ab501d204a843116782146372fc66815
1817 F20110217_AACZDL lan_j_Page_088.txt
cf838effcf8b8ad56eadd59c374a230c
084c16a477db2732df8c2f79e095d250aa28e216
22381 F20110217_AACYYG lan_j_Page_084.QC.jpg
17987cbb43ed617afa1059fa7f429013
783102a7c463296227a6fce581186bc6c080849c
52596 F20110217_AACYXR lan_j_Page_065.jpg
cfc77c0cce7369c64718c653ed7b5185
87f2321bec648f79172fd826ffa20182f7e34594
787 F20110217_AACZEA lan_j_Page_119.txt
a1271b525fb69a8b0d19aa1fb1c2d304
a39b71ff514cffea87f628ebd8062d2d203ea014
1490 F20110217_AACZDM lan_j_Page_092.txt
7dc54612b24b5d8a3612b08784b7bf68
d6f0f3eff480ed99f690fe4b1486c93499995985
2529 F20110217_AACYYH lan_j_Page_122.txt
6bd2ce622fe30377ba5bdc2c310894a1
9659f25030292b773bd65c77b2f76311ac3c85dc
1600 F20110217_AACZCY lan_j_Page_059.txt
cf6eb809f40a93e3bc43856c41cfa31f
39a4fd18d8b84cc455c194b54892964546fb287d
20197 F20110217_AACYXS lan_j_Page_079.QC.jpg
b73968640e43b66a62a506937fc96d72
fd80c0dd3e48b7c184e3b133e24e442a0c9a65f9
323 F20110217_AACZEB lan_j_Page_120.txt
1896f7cb35accd042f3b05bebf29eeb0
c7279c6c0c047af3e69adcd65a3d7308b2cf6f7b
1473 F20110217_AACZDN lan_j_Page_093.txt
fd4182c973657e3a8b6919671c61b6e9
1a4398054e62dbf891a789f9ec7d7bc0266b1ac2
23901 F20110217_AACYYI lan_j_Page_028.QC.jpg
c620fd31df4d796578dd3f2d9558e679
fb066480c359758eb5530731e7e12acf96f26cea
1783 F20110217_AACZCZ lan_j_Page_060.txt
4ab5d477643640540f9a064a6d9e5e39
9198d2e95a7155425ea2592a877587dd8467d413
5596 F20110217_AACYXT lan_j_Page_120.QC.jpg
0c4e7adb4765e4e716a618256bfb8dba
91b0a1352a7147660d0d48059d785a34d419be50
2443 F20110217_AACZEC lan_j_Page_124.txt
34efcee193efacc05398c897e73a110c
7bd83283edc3a6527d366ba2a1fae61481ba9b07
1608 F20110217_AACZDO lan_j_Page_095.txt
a6f5523adc38b98be2552d4643be8c5b
e74951edab530c049fc0c6921f8abc3866d7c5c2
F20110217_AACYYJ lan_j_Page_073.tif
08e811d8a5c6e7a236c5bb3c30b9bd81
9ee7c09831117320935593f6e2c976c50a3ce1f7
26510 F20110217_AACYXU lan_j_Page_100.pro
3e77ee54f77bf12067e659e18b03d67f
13ac0cb50eb99469dbacc685f1a8815edcf95fd5
2565 F20110217_AACZED lan_j_Page_126.txt
58ebacb9f4b2bd089147e8e13473057c
dd733a32dd11fb04e5dc5ada6fbcef16cb7780e7
1896 F20110217_AACZDP lan_j_Page_096.txt
2c1e7a574edcae8864d9077d8b7c1ab1
abadf95ede35f7b44d0e10b32a565bbb280f361d
23308 F20110217_AACYYK lan_j_Page_127.QC.jpg
15cf90ecea3117d5d46fcf88dd797e54
40a150a12defb4eb8567a0b92b51ac3287645a01
24819 F20110217_AACYXV lan_j_Page_113.QC.jpg
687db1c65f5b10924f75e4d78e6f0a6f
bd8fe28738bf9903988f3a06644054f905213101
1747 F20110217_AACZEE lan_j_Page_127.txt
0d843135bfd3262d226de06211985f4e
080f93889733702b3689263a8c85f95de5f1e1ee
1586 F20110217_AACZDQ lan_j_Page_097.txt
9294464abbeddd2d2b154c14edde30c5
90d6a8c8fdf71c62984e40dbd863133800488335
662 F20110217_AACYYL lan_j_Page_034.txt
c8ef9e3d205441f233ef1b20bccc00f4
77d06c70b5c872cbce86ebc635385036c6d6d8ec
845583 F20110217_AACYXW lan_j_Page_092.jp2
d770e3b36e02302a5e9b9c4747eea76a
0233c0114c073ad2953e1f55877dbfcdd9643403
619 F20110217_AACZEF lan_j_Page_003.pro
6e4b921cfb26654159f17d09e0176647
0da9604b83fedf25c939eda7192da4026da6b10e
1531 F20110217_AACZDR lan_j_Page_102.txt
8ce32c1d7a69b4586adb3b7faf3a14ce
985a6210005c0f620fa80fc495f1f20e48039d6e
7074 F20110217_AACYYM lan_j_Page_121thm.jpg
77c7accb22a805f3a0e928f07fb4ddf6
6f3759e1c698604daa7d79f9014e926102aedffd
926301 F20110217_AACYXX lan_j_Page_061.jp2
c847597f538cb0808ef53cc3f17108c0
988b54e75f6a3b5e7df68324df3e52c7e5c1bff2
28010 F20110217_AACZEG lan_j_Page_004.pro
9ff4cdcce21a10b23c84c648a6c148f9
605c8fe94d06071da1d1427425769a4abd8824bc
32886 F20110217_AACYZA lan_j_Page_091.pro
63cfe25599875169abbd24dbe446f26b
821e206696f81d6423eaee7ec35b180e438ab715
1295 F20110217_AACZDS lan_j_Page_106.txt
d110a074ff0b3504898eb4430c46a2b0
c661a869192bb8e8e3598889d06707888378118f
44229 F20110217_AACYYN lan_j_Page_072.pro
51d365feef381c4fa1423a02ac966f59
2e7a1018297fdcef461e19341fee3d64f1c2f7c5
6326 F20110217_AACYXY lan_j_Page_118thm.jpg
15e2958803e26e5bd10adf2083186b20
639891e3d850b8848e42ceba66c36571b94e06f6
31001 F20110217_AACZEH lan_j_Page_006.pro
79e098df6f46240555d5335aabd3a810
f98a927bc700b991e8f4b5a7c7eef12be3c511a1
F20110217_AACYZB lan_j_Page_109.tif
c29fba1220c6cffad77f0813146e23f0
b5d1f61111a935a0b3555b8bfc288c9f60464a3f
1001 F20110217_AACZDT lan_j_Page_110.txt
6e46f66e424a5116aaec7fd80264309a
f579e08218482754d19a73a076f0fce930bfff87
16600 F20110217_AACYYO lan_j_Page_128.pro
3b3af8ef27e8a5817510e685aa008139
48c1087699a52029c0b6250e8fde94d294248068
F20110217_AACYXZ lan_j_Page_099.tif
f5ec4e50b0d27afd70b2ed2e74ddd042
298c23a5a395374f2a3ee6d1e19e8031b463e8e4
15246 F20110217_AACZEI lan_j_Page_007.pro
3523e3ff8bc5aca54c3375a476cb85ab
b3438c0526bb441a2c5697ae9acd3b1ce277c58b
54667 F20110217_AACYZC lan_j_Page_119.jpg
e2429eff81d4c18fab9a81a3e30b439f
3a76edcab372fcccb73dd1d3cf3e19ef6f6ee2f2
1757 F20110217_AACZDU lan_j_Page_111.txt
2cc2535ce1ca4e12497202e525222a0a
c1cb00925be51af10ace21061e86a29a605e2e9e
4170 F20110217_AACZEJ lan_j_Page_011.pro
7e63a098230cd6cce6df783dd08921ac
a13a99d547837562c5d88d2e2838b55a2092941a
1634 F20110217_AACYZD lan_j_Page_120thm.jpg
1405fc73f64e9ff4200a377a143463a7
f03de830a6163079a2703a990d91126d9ef83616
1504 F20110217_AACZDV lan_j_Page_112.txt
2b127ac4e986a338c658721f207ae60e
e93dd4001c4d3b344719b5d2416e7c4a9eb41b7a
17474 F20110217_AACYYP lan_j_Page_045.pro
26e259eb5876c26845b273ad20003a1b
a19d87f9b1e10355dcf77e64801d02f253d86e62
49242 F20110217_AACZEK lan_j_Page_014.pro
6eb66d993b74ac1a267fffe1156e88d6
2dfccdc6127333e098f6f0e3f21fc9170ef2d2c1
21148 F20110217_AACYZE lan_j_Page_052.QC.jpg
13f7552cffe82403b605cd853f0f3b85
a6c748b2e0dbf72e10f1414874ffb9156392071a
167 F20110217_AACZDW lan_j_Page_114.txt
601821e69ff3211b2393d5e0d8a95b07
6118e1a2140393293eb59225d5a13666db17b0fc
72889 F20110217_AACYYQ lan_j_Page_025.jpg
9edffdde318397909cf24809453c9bae
5c2106cde91e0bafd11500667619d94c14af6568
30794 F20110217_AACZEL lan_j_Page_016.pro
0be1aaa193e385b7b1912b4f290f79fd
9b52c0d2b4178aa116befa465bd0866080a609b4
F20110217_AACYZF lan_j_Page_079.tif
a70b215f7be0701d187bb7ed38469196
6e73a68955dd539166928dcebd83730a28c2a4cb
1395 F20110217_AACZDX lan_j_Page_116.txt
cdcfb1373fd9ae3f6de1c5a544cef690
30933866f9472fa66b0edfe7b9f1f2a637dc0ce3
812671 F20110217_AACYYR lan_j_Page_084.jp2
4c9f9d4a3107f1f37245711658eff729
0178ef48252390393daafd25f81ba712081fee50
24039 F20110217_AACZFA lan_j_Page_057.pro
35d4edef9cbdb374e3ef83d916aaf6a4
2ab30eeabb8b20230d09cddc7fb2c93e473245ca
50106 F20110217_AACZEM lan_j_Page_019.pro
e50c3b833a25d8ddf0a17abb1efb41b1
93bbe97931b65e08a995654fba14a06ed8e39451
23697 F20110217_AACYZG lan_j_Page_016.QC.jpg
40dfaae649e0a6d8f87072eea05c9ef4
be25641621dc2b9c7428807df16cf17c04925684
1478 F20110217_AACZDY lan_j_Page_117.txt
6345a429c07eae227b3b372772a1d1b2
517b609d45c101a4dde1734079e98d29bd383c94
1051945 F20110217_AACYYS lan_j_Page_017.jp2
430c90f9a82ff84f3f8e038f1ad5d071
097aa89d2557cab26f8559ff70469543b4e4ea4a
36582 F20110217_AACZFB lan_j_Page_059.pro
a206287b3bbad4ffc4ffb2b379e2fa8c
38fc018ddbf775a870f3bacd6b100f646fa5e7a1
50178 F20110217_AACZEN lan_j_Page_024.pro
3892dca165ffbffccb968d0a8b67e992
f41ddd29ee3126245671e7f263b9968f08d40ac3
40445 F20110217_AACYZH lan_j_Page_010.pro
6adc53851e08b77205ab1ad633ec3e83
9441b9d5898801b5dcba9966776020c8c83dfbb9
6077 F20110217_AACYYT lan_j_Page_064thm.jpg
772a7ff4127d4b09af53fc6b47508f36
65fde3f309135f128d1b179685955c93268ec427
23739 F20110217_AACZFC lan_j_Page_065.pro
4f00eb534ce1041e138be6e50f2824f3
c5b4bc1d7e007ae72cfc4f6e8caf080a53683c53
46219 F20110217_AACZEO lan_j_Page_027.pro
2b25c771db1e0048956383025e7578be
3d600a0d5649eb50330b27f5be5d167e1574aa1c
F20110217_AACYZI lan_j_Page_120.tif
8184eb03a58d1469541eeac9e6413d76
01f8a8172b96cac104f4825d73f2fd0c131217ce
1516 F20110217_AACZDZ lan_j_Page_118.txt
aed5b2ba14203cda631ffb97ab9dc366
7e66e8dbc54e24c3681634eb4352c2139df394b3
F20110217_AACYYU lan_j_Page_112.tif
7e3b663018be7de4b1088b9d11d76c9f
65d844e3b38f770bc75c398e15aa1188f3d06476
30410 F20110217_AACZFD lan_j_Page_066.pro
38e85c10faa607c4119757478751bbab
fe9eadb5fffa6c42e8687f3f0dddf5d56cdf5050
44468 F20110217_AACZEP lan_j_Page_031.pro
fca85ab46abc9c75ffd9b9371af01d6e
7ca57294eb76b17cfa847307a9ad9f7d186b3295
733116 F20110217_AACYZJ lan_j_Page_054.jp2
21fb6036960b795216694e776fe11abd
3f67d91608a8339c3a9536f5beed2db30a79e32e
F20110217_AACYYV lan_j_Page_077.tif
65ebb8b0fb922771b6087b636aacc220
d454b8dc23181216196e84693cd1c10d288d5f70
40157 F20110217_AACZFE lan_j_Page_069.pro
8b300f0190e2b6304df05392377ffdde
3fd793dbe81ce5f845511e250c71cb941c862c28
43230 F20110217_AACZEQ lan_j_Page_032.pro
4a55d229094f9addc60ddeb5cd4319d0
79b3bf469b5cdd767f875f778a236d0f3f25115f
1353 F20110217_AACYZK lan_j_Page_084.txt
51d664efc607eb868230c5469b42be40
e88f123cb693378b0400f3359811377a63dc99a6
F20110217_AACYYW lan_j_Page_098.tif
b27f384f0ebaa8737e0c2f639135552b
167bbad97398a7a6dee2cd84d046427b20dd5cb7
33019 F20110217_AACZFF lan_j_Page_071.pro
68f1394df192adc09cfa3138395c93d1
c9703ad3c21389964e7cad449ea39a8b1a03fe38
16518 F20110217_AACZER lan_j_Page_034.pro
5a4d87eeac1a263420b5962178b61a29
cb20776996e565671d7dfc62be864f438dafe513
14804 F20110217_AACYZL lan_j_Page_104.pro
4ad56fce89f51dcc86d499a24f0dcae2
1cffb43bb0adb76df714f0f6af7d7d6eea192024
33446 F20110217_AACYYX lan_j_Page_073.pro
cb5ba02864610c9fa2b2a06646e73e4e
d5e0dacaacf339c97dd10e3b307b337937b76e98
37600 F20110217_AACZFG lan_j_Page_076.pro
7f608237c70902dcc13c5b22a4c9aef9
7981130e65edfb6a108d0d188f5ef1c798bfc153
44007 F20110217_AACZES lan_j_Page_035.pro
3fa5fef32125f692eb09a77b7e1c7aa2
1a27ad5ef9e8b0e40393cd828f71a5de38a175e1
31827 F20110217_AACYZM lan_j_Page_092.pro
d347c289cef1e01033c764400ff7bc6a
dfdd97fe5fb72826133a52fed74ed4923e257dad
30312 F20110217_AACYYY lan_j_Page_050.QC.jpg
7eb74f2ab7b7d5774605ba14a45c273d
2e9c8c77e197aa383cd052cf1f5cbc907d3d3562
24794 F20110217_AACZFH lan_j_Page_079.pro
3137a1a597ad211e2c4238fdfc5990af
dd74724992a4b039b76af0b4a853ab596b189005
47034 F20110217_AACZET lan_j_Page_038.pro
9e52b0ede8fc0ccbdde053703c779a6b
750b31b21216c0b29c332494eac7a4bc4fa31f25
18991 F20110217_AACYZN lan_j_Page_098.QC.jpg
102f8d8b7909bc837fe024bd6c8089a1
c84727c70edf1aa152d5d0e9760da36734c7a099
F20110217_AACYYZ lan_j_Page_018.tif
de01a721e1a243558afcb710c02f629a
4cfc7711f98a4a342113db5c1bde0994130dd51b
24374 F20110217_AACZFI lan_j_Page_080.pro
7f773a122e70151f2e0a84468b862e89
aa0e6a8b3403608f14c33c478e6e62edad15b0b3
30982 F20110217_AACZEU lan_j_Page_044.pro
b3f01e4497b449e12bf6a005b4c302f3
530924306d91ec8ab1b7a25ee239311cce027951
681229 F20110217_AACYZO lan_j_Page_066.jp2
6643ae435c04463f3d1e54cce906a62d
13cc385d43e7047740fd6bdfe274b3feff03aed0
34972 F20110217_AACZFJ lan_j_Page_081.pro
56f5f50e49df5bd03f80a689817a637a
ccbfb175771ef4de2da7a13bb41b3a9d9741b5ff
23781 F20110217_AACZEV lan_j_Page_046.pro
2bd0e36d1dc8cebdd1560862899cb814
c92b54cc15dec51467ec50e3679df5d6b6115e72
89678 F20110217_AACYZP lan_j_Page_027.jpg
efe120cded56c3f2d880585a710e5c52
fd87e9d76338d4b17339f717d1c4aac7f45bb56e
30358 F20110217_AACZFK lan_j_Page_084.pro
30f914d7a656b3704f3659e1b14e8947
f3eda71f250a21ae32fce5d8584bf82b9d9d545a
45899 F20110217_AACZEW lan_j_Page_047.pro
64c253d5061b667226cea5a3c2a5a471
512b758d67f71a27418e64a00d6617b9872ed461
16804 F20110217_AACZFL lan_j_Page_085.pro
57b3c62a4cf2fdb7ceb8e7b94246fe3c
612705105ed3d326785e834de71c21669b014b0e
32239 F20110217_AACZEX lan_j_Page_052.pro
7e59759f950fbe1f7b50896b31944b09
da3c11cec20c5004c60fe9dd7e386200e9ee682b
27462 F20110217_AACYZQ lan_j_Page_115.QC.jpg
8ca5e2b52433577251e4ee854c782aa4
8f021633871e81e30bc14954faf643c1333bf213
40836 F20110217_AACZFM lan_j_Page_088.pro
4d443494501faaf07900a1ca527e1ec4
cdafd176fc5d6049db63c61bcc357b4eb6268efc
39905 F20110217_AACZEY lan_j_Page_055.pro
ec0f2a31253482ec31e9fedd45f822db
58586fdaae6032bc4220349870d086a3390296e5
81175 F20110217_AACYZR lan_j_Page_063.jpg
31cc449c82c61b8473a88d28451ed185
bb6c35ef8d275fbf86e7ac0fcf59d962e8e2471f
32988 F20110217_AACZGA lan_j_Page_116.pro
f9fca21de2fcb1eeb40b847eb051a5c4
a44943a87257f2df1a7c013b54b33e0f837fc738
32148 F20110217_AACZFN lan_j_Page_090.pro
387d912cd8b4b48ce571f536809031cc
80d8ae930cb213cce5d2d9977e2e9fb66c8853cc
32044 F20110217_AACZEZ lan_j_Page_056.pro
c7bde1676181f0c6c1f5ff453cf7e07e
05140295779b25209461c0391d28f8c0c2643382
1143 F20110217_AACYZS lan_j_Page_065.txt
2e401d50ec82cff7db73436a8d5c78aa
646e3fe7dcaf5b1fb55cd044a09dff60d6a7967d
37114 F20110217_AACZGB lan_j_Page_118.pro
8b0a9a8f225fe6410f86fc1cc16f30ab
c6299d214fe1897a5a1d09fb51d790440dada341
33900 F20110217_AACZFO lan_j_Page_094.pro
a47666bdd22ed5e1edcbf90da4fba5ec
98b2982504ab8fc19db54535d01247108c659611
31285 F20110217_AACYZT lan_j_Page_013.QC.jpg
83779a188ba3bd11eb31b7cc3357ea42
8bb849f5188f700d767a81bc9dc424570387e8c5
18496 F20110217_AACZGC lan_j_Page_119.pro
e2e46979697116eb2656a117b4012689
0ca38cad0e6e5906f110f06f96e3f0de96e42609
47849 F20110217_AACZFP lan_j_Page_096.pro
efb666692aba686cb973e16a45775f2b
48beff5c6fc3d246c7c1274d579cb04eeb4c6511
6602 F20110217_AACYZU lan_j_Page_095thm.jpg
4cdbf5324b5800f828c6a421569893c0
671b5898ba3740f9e8349f76b79e4dc443ee2033
7092 F20110217_AACZGD lan_j_Page_120.pro
f3ee5d91fe8a64d4d58ecd9bd354d586
f593f638c66f028deb6b4ec3e18d3437f9491c67
35921 F20110217_AACZFQ lan_j_Page_097.pro
2aaf8d702d85219cba1d0318e6676f73
e1d22f32b15691c33f41abea59ed4ccb440c8294
39036 F20110217_AACYZV lan_j_Page_023.pro
c35f98cd160e194ba07e02bb55b32984
389adb3c3dfcea29b9b64abe5c9319e3f8c2ec59
57653 F20110217_AACZGE lan_j_Page_121.pro
d92e8d6ef34164d87cacae3d65c41d98
1d66a3e735975dc8aa840f838badb8825f5644d2
37077 F20110217_AACZFR lan_j_Page_102.pro
abe8cb477406acfe0bd8d32f8e6810d1
e96bde7b61c518732b761101ac955ec6a699acf5
149056 F20110217_AACYZW UFE0014640_00001.mets FULL
8dc1f2cea15a72de35d3bcd4c1b36745
985a1a19847e794f2a42847023d30294e8a611d3
65878 F20110217_AACZGF lan_j_Page_123.pro
feb07cef15f918500aeed4c326699619
3952696a2aa2d9950089f7e0f047e812e70473fa
36498 F20110217_AACZFS lan_j_Page_103.pro
78a2d87c5e1554c86fc24c44fbb41be2
75d6fa1f01a4cc339115dd824ed97bf5faba3ab4
63561 F20110217_AACZGG lan_j_Page_126.pro
7a07d6b8319410fc1345794f9816f41e
4eac6457e60e0f7179d9134c2c352a4de2660271
17724 F20110217_AACZFT lan_j_Page_105.pro
d552418312b508332c44f7316672dae5
c9e462ab128c8cd7acca5124850f2be9da03ddda
42786 F20110217_AACZGH lan_j_Page_127.pro
2b776b1526634476484695b9505d4bba
a1a3d4cc294848d618c6eacb329b5d0a9658e579
18135 F20110217_AACZFU lan_j_Page_106.pro
f2e26b2ae4aad0181fb83c71b3c3cdbf
ec83b658e31c4a2cb36ae4a4a1edb517d8c23c89
F20110217_AACYZZ lan_j_Page_001.tif
9f62059fee82c41b1d46d7d99776a09b
6ef4204ab9f026dd01e60e9b4beab9be846d0c4f
4093 F20110217_AACZGI lan_j_Page_002.jpg
696d370783f0ab3d436e10257dfdba98
be579eab2af5b7d836fb3142a3b02084bd28dcf9
14306 F20110217_AACZFV lan_j_Page_107.pro
34b4fb7d2db56d16b75a0299e4eb4c05
a38d4ecf6927210de0e57c39d249ece82fd03694
3079 F20110217_AACZGJ lan_j_Page_003.jpg
dafef69a3103c021113005c021dfec85
7cb5fa957f7883b049994c763d4f2d410fd1c1dc
37732 F20110217_AACZFW lan_j_Page_108.pro
d789642cba4a889b241fbd4ad32c93bd
f6bbb809c738c1df660d757f7f68a87dea876b5d
60602 F20110217_AACZGK lan_j_Page_004.jpg
7af7f2056c8f325d6c146b529032acd5
3011c0a186a49f62386cecd4bd15b8f7abb2e80b
50847 F20110217_AACZFX lan_j_Page_109.pro
500d9f111d66e5b22b828a6d28b9bbee
adfecb7545af7f6064eb82d0016c2ff5c6e5b347
83534 F20110217_AACZGL lan_j_Page_005.jpg
8db96415499f3c1ba43798b1f7a5594f
1ef8160e7038747c235a43d1328fca648548f05f
33199 F20110217_AACZFY lan_j_Page_112.pro
f659f01b31ab60384bf54814e07e1a60
6318c641422a5ec496b0cb15c50751315f79dc0d
41680 F20110217_AACZHA lan_j_Page_042.jpg
d93b96739d21c62a7bbba6ec9c4673c6
e03e6bf1b3f0a573c9dbc1c8b32f86a20f0c1bd2
78070 F20110217_AACZGM lan_j_Page_008.jpg
5ffec0ae26dce342b3e8b071ee615e09
4ded318951c39153b997f7f3b28f42f2ab0f7706
3136 F20110217_AACZFZ lan_j_Page_114.pro
428f701e3eebdf191336cb6a9e74a09e
7afa3a6f0075dbe50f0d90f49455ec89218c49f8
69845 F20110217_AACZHB lan_j_Page_054.jpg
e5a42e04f99f46ce9cdc07512f3fad7b
a6183399a6da44bca98c8ae64439bf4e611ac0d4
81431 F20110217_AACZGN lan_j_Page_010.jpg
6b6c4d6ef627c540659e30fcde2c588a
cd751df17949bccad18e00c9ecd0ac3b90e71f0a
80590 F20110217_AACZHC lan_j_Page_055.jpg
74aed86fa3f0d3b3799b399f59334d16
10cf0f7863a1e8ccbd06e1238360d19f57597023
86561 F20110217_AACZGO lan_j_Page_012.jpg
31c11edc79de87fcd5b62b0b8a21d09a
ff87f9c3f9f45e422a60e00a79c5c759618314e4
67816 F20110217_AACZHD lan_j_Page_057.jpg
52d4c846a8542e81cfa76949867d61ce
baeecef1e723dabc4b263f98f385f3335cc4fd96
94434 F20110217_AACZGP lan_j_Page_018.jpg
1a1b3af18ea4d1cf09e319f4a583a4e8
0df6173727f22b177b85b57e4172d75125a2446a
65438 F20110217_AACZHE lan_j_Page_058.jpg
953a16b3da3c2a0c4118ba6615e39384
a5dccd55b350fb0a84cd0a63bb3fddc0f3493dbd
98601 F20110217_AACZGQ lan_j_Page_019.jpg
a5671f26bf822ee7fa1596369912f907
a9921503f5e94e4d6e74e11fcecd0055d1196e61
57839 F20110217_AACZHF lan_j_Page_060.jpg
3812a44fcc4552082c4996b876e04c8a
7f2144f97ebf484578bc86ccb119a55fa83cf4d5
99280 F20110217_AACZGR lan_j_Page_024.jpg
7de2fd273849749a8bd28a3cbc4676bf
edc10602a53315190720a2d6b9909e0a8ace285b
82445 F20110217_AACZHG lan_j_Page_061.jpg
5063be5488e3b0bd8f0c0e06803f12aa
6b807984427c6c5cf9de94f27a1ffb5bee5cd16a
91620 F20110217_AACZGS lan_j_Page_026.jpg
d681285bc5567e2786d417eaf51e9876
19c35b3bd8d41b1ade6d1eaafa5df711909da7ab
55712 F20110217_AACZHH lan_j_Page_062.jpg
a3aa007fa83c0dd4dd3192ad78594c7f
a531825e5f7a0d483282c9cbb818eacf0a38c6f3
72792 F20110217_AACZGT lan_j_Page_028.jpg
2387a5864cf0e71afb79b7e29ad5c38f
739c63a4461206be32bbe4c6c7129dadccc7e73e
50630 F20110217_AACZHI lan_j_Page_067.jpg
f5ed0919c0fc129ee77044275e5c4cdf
c68fbdce6f86a8f8b210203837819595fdfcd88e
73506 F20110217_AACZGU lan_j_Page_029.jpg
80e1073f0938101c034edf8096ca70ec
fc231b1b416e84e4765e700a2bf223004c697019
66245 F20110217_AACZHJ lan_j_Page_071.jpg
aff04ea0d44bcc9616801c42c1cecf50
3b8555db2c9f534f6a2be87c810bea7d30c4edad
87819 F20110217_AACZGV lan_j_Page_031.jpg
e76c05fa670dc83fa6f8c1f76f05e2f1
f4a15b264501a11825e924abc351304fecc30bcf
67401 F20110217_AACZHK lan_j_Page_073.jpg
e3fad0c57dae7e906df838c7bbe2a517
05265a46fd2af486bb1261ede6280d3acb8bbfe8
86202 F20110217_AACZGW lan_j_Page_032.jpg
f6fa774d4a6eb62be827d3976b634102
8f6c83bcab8127c86010ba3465a8a7d2d4c95c11
76154 F20110217_AACZHL lan_j_Page_076.jpg
a547abc917e4ab8421f0911f80869c7a
297317f75be9b2582d5e637ef1ae617ef371c510
51987 F20110217_AACZGX lan_j_Page_039.jpg
18935f6e1f973c025f4bee9e163105ff
d4213779788e94e1cf165f0f5f1edc116cc2aeaa
80869 F20110217_AACZHM lan_j_Page_078.jpg
bd243cd5bdc14c60f0482094b1fe90fc
7ba7592a94d7e59f30c232ac908c01d8355d28b2
83491 F20110217_AACZGY lan_j_Page_040.jpg
5e0f6860c3aed723e2d7cb11d83f0d40
a03fe38398f7d33b81387f4d960692feeb31dfbf
65283 F20110217_AACZIA lan_j_Page_107.jpg
e552113fc083a0f164c4bdecdbbe91bb
c6db35e7391e44d6a99ad68323ad70395b4e0d1b
90269 F20110217_AACZHN lan_j_Page_082.jpg
8e4552041f94ecd6a2a92084793923aa
86e877834c3862a184b2daed403031c488d887b7
79980 F20110217_AACZGZ lan_j_Page_041.jpg
544f8fbb5d937a7e6d4daffa94e98f66
2881be8d571015e44135f04781e92bfd84614e1e
76164 F20110217_AACZIB lan_j_Page_108.jpg
655a45dd81fc6e6325056976165a3bfa
15612e30adca4a30ad65c5a9309c4f4a9e6cfa5e
93288 F20110217_AACZHO lan_j_Page_083.jpg
10af239ea13a428a2bdede921cd79694
cf775c91fcc0913241f6db180cd44c1d8e8e88bc
99726 F20110217_AACZIC lan_j_Page_109.jpg
4efb85311a44646c75fd7ddcd0fa194b
ce13c38cf5d9ea3967567f42d5f14e90e245d834
72297 F20110217_AACZHP lan_j_Page_084.jpg
026355debae0faf77523c8a55004b637
ba9cc572f0a247f0a82127a5e50432c8aaacb9c6
49779 F20110217_AACZID lan_j_Page_110.jpg
b8ef5bf11004247708381f8dffe0dd39
d2f7724dad8d4cbc8fe30a9667cf1cff3b34c700
83117 F20110217_AACZHQ lan_j_Page_088.jpg
c60ffb9fd4bba9e379b7ea331cb2bf9e
b33c64092348bc087a631cc78705188898cd7bcc
84772 F20110217_AACZIE lan_j_Page_115.jpg
55ee2d5e397b52d26e8b932e033fe975
e4d704bdb376aa25e79536c2632428bae7b446e8
81845 F20110217_AACZHR lan_j_Page_091.jpg
43ea9e3619428ac23e90a2894249251d
21f9549a5e3266ad08eb4b3312ffd5a00d05835c
77534 F20110217_AACZIF lan_j_Page_116.jpg
70bf8d295296275b43275f3fb2f04e2b
75b6cdc34e4e9088d427fbc7b941a142efcf13d6
77729 F20110217_AACZHS lan_j_Page_092.jpg
5a5608787ee601b7c83ff95b9c886985
1e1d666137c66f94ca3089f62a6399b4f8a5c690
75853 F20110217_AACZIG lan_j_Page_118.jpg
bc258d8d092a1ad0ba383ac47b251658
64712c19135e04cb68ce91b19d8d22abeadbb1c7
76225 F20110217_AACZHT lan_j_Page_093.jpg
d71b724f4762427634c1acb849061088
8ffbb49b08130522a0854e4ebff81b95d6cd774a
16803 F20110217_AACZIH lan_j_Page_120.jpg
f276313f1af9e5465265cda7ce36dc4c
b83076dff6500e9155110c5aef91f116e679b138
77968 F20110217_AACZHU lan_j_Page_094.jpg
7dd400076914657a89f713d6bf495461
22480ae96163c6bdd1480ada96eb1327744a0a76
108052 F20110217_AACZII lan_j_Page_121.jpg
68780726cdc70c0795ec91d5b20013b2
bcd75cc00098e72ca3839870ff65c986c70fc227
78081 F20110217_AACZHV lan_j_Page_097.jpg
1f3dfdcdd5f6d40ba26dac9993805c39
1ca3c1b7f581bd6d7298d05509cb6e44b0447146
122040 F20110217_AACZIJ lan_j_Page_123.jpg
0592f4f0d5f8e2aa18f71d1457de089e
bc64d483794c7a95d007949718da25bff0cd682b
98189 F20110217_AACZHW lan_j_Page_099.jpg
a1f5e3af174e6715935cd45e65eb2203
fb658b454c4f436096e33910d1b702be407525b4
84848 F20110217_AACZIK lan_j_Page_127.jpg
7ca62c632598b6c16e9a7db7b4c32be7
07a4a2056f675dd5bc68d9333a55e56528b852b6
83425 F20110217_AACZHX lan_j_Page_100.jpg
012f706a05e1acf4ed55ead7217d9109
30ddcf1a725a896c4803b47689de9503f9719449
226438 F20110217_AACZIL lan_j_Page_001.jp2
331e4b328f0c5ba3d81310e97893599d
f756f5831b01cc1c27333f759f6c45bfa09d96cd
93047 F20110217_AACZHY lan_j_Page_101.jpg
fa31b5e28522496806fb967f250c5904
4edbc185af99df85e146adda5bac4d79eb766915
981037 F20110217_AACZJA lan_j_Page_031.jp2
1bfb24f889291c6b2bc1b1845838a50b
565072736d3fe2ae56ad4947761b1f5372892f5f
750936 F20110217_AACZIM lan_j_Page_005.jp2
b55dfba81da14c4c28d70268802a6f17
0edb73e456e5801a6dccf4c6d2b1b17522a59386
59596 F20110217_AACZHZ lan_j_Page_105.jpg
3345798f36b6372957e740a8a1508f2d
45156fdd001a898ca4ebe6050226abdd24d2e394
956800 F20110217_AACZJB lan_j_Page_032.jp2
41a90506d1b309d39ed2130053e006aa
708cb7147f8df1a1c46bed6e3d3568f033d5671d
519867 F20110217_AACZIN lan_j_Page_006.jp2
6ba5fd0d4b0513bda02939baf392e175
2ce58d5ca0df6a8f6470591077508b0319833010
1042590 F20110217_AACZJC lan_j_Page_033.jp2
797cf7ec4e6a9ad8a4022fbe2e6f042b
25c0507c7f6537d1ba3ae03d55cafeaea115572b
969057 F20110217_AACZIO lan_j_Page_012.jp2
99ae7f99129d422dd91aa67e2e573426
1664a6eea36075af6946eda01ecdaaddf80b1c39
364335 F20110217_AACZJD lan_j_Page_034.jp2
3d2c1790ca808bd8abe3393ece177039
03e86ea2c42b620e4118c568057a40705490dd6d
1051977 F20110217_AACZIP lan_j_Page_013.jp2
23e6d1eaa0293c280c18bd306fe192c1
b5bca1795a40edf91e2cdbc5670f8ccc5fd65590
992545 F20110217_AACZJE lan_j_Page_036.jp2
f9714e55c9051b7098688cb2965e02a1
130da9b356ca27bc54534219b51ea12331890b9a
1051986 F20110217_AACZIQ lan_j_Page_014.jp2
9ee33028fa78bc2d9a3e38a1099b9e63
3b3b77d53253c78fa74b54839e1990b4224ad725
1039707 F20110217_AACZJF lan_j_Page_038.jp2
6c810b3e1a0edeac6b4b8f1dc7d0a983
d17d94d4701d4bfdae9de4af5478928cc9a6b38d
784785 F20110217_AACZIR lan_j_Page_016.jp2
5a8e61283594bfcc67244b1405f787a3
0cf45b52921d02b3e26af690abb4da9bccddd6c3
578765 F20110217_AACZJG lan_j_Page_039.jp2
f67e3338ed43b387d95cb80586bacbd0
5f7ef25dbc4eb6b3353a1598437246326af911d8
1047189 F20110217_AACZIS lan_j_Page_018.jp2
95c1b1a412684509d584f367b9220aa2
a41fa90f7a267d8389f77a55d7f259110d5f2db3
945657 F20110217_AACZJH lan_j_Page_040.jp2
fb1a2a2c7d5a2f0ff55d655194fb6392
3155ba9a592929f5c8a9e6451a1e2e37467fac8a
703440 F20110217_AACZIT lan_j_Page_020.jp2
f00a015ae36313fb03f51a23a7dc2ff9
043dce32701d89615b926b679a2d131249fcfa14
670323 F20110217_AACZJI lan_j_Page_043.jp2
233a0acac1b7a66e06c92cfea0e4950a
ea7727eaabf0464d7a3cea8fe76722de1384a161
824208 F20110217_AACZIU lan_j_Page_021.jp2
e0136ff53f98e7188cafc1df7a4803b1
7757e40c93a322c51723c7a51ea87604ea0103d2
457994 F20110217_AACZJJ lan_j_Page_045.jp2
730577396467b4ae60f5f7b0b5d38f45
653a4378226b2d119f3c55b76beac6b6c5ca418d
927048 F20110217_AACZIV lan_j_Page_022.jp2
cb11f028f6c36faef8f27a1f1d246ebe
d24d205df2269d3e58446d27d333de7a6bf3a65b
575735 F20110217_AACZJK lan_j_Page_046.jp2
9ff295c82995aee23f45b48734cb8272
f2d6f9b45f1959209efd6c72635f6f1889b4bedd
814778 F20110217_AACZIW lan_j_Page_025.jp2
7c72e2d8040185d25300ce54a07b1eb6
125a81dfdd2f898a0c2c83baa8c291efd25a8ffb
852467 F20110217_AACZJL lan_j_Page_048.jp2
18b7d6bfe793010722c369942644d629
7eb917e4d39733e3369d6a19034787224c4b94f5
1024109 F20110217_AACZIX lan_j_Page_026.jp2
2558ab8bc3ec49ea1a41b14a20c0be26
8b25f76b7b91fbf629de9e8ec0f8145a9846efe5
1051918 F20110217_AACZKA lan_j_Page_082.jp2
0a16178a861ab2df489487bd7b2e950d
cc682dcc09e941c760248036175c5309aa90d654
964715 F20110217_AACZJM lan_j_Page_049.jp2
70e359efb3824d1a0872079bdbf64dee
2ffdbc68a961fcbe19689a07ed16e83cbd770736
997974 F20110217_AACZIY lan_j_Page_027.jp2
ec486930a68ed204e69a08b75566f762
61d9eddf596252f7688e204919bfc572d265d786
1051962 F20110217_AACZKB lan_j_Page_083.jp2
1acb13241699b913ee7da39a89fb574f
ded4343a701bf0e4632496515a456d7a192dfe58
930320 F20110217_AACZJN lan_j_Page_051.jp2
d5aa3cb92e33921d7c79ae574a1d072d
d3ccd15ae331a41d73be10463e1a506eb1ffd038
810314 F20110217_AACZIZ lan_j_Page_030.jp2
57a1970045b2d7439662b5bec99617c9
67a62b8fce14ac1fee2fd27666bc5d75377b2ccf
915821 F20110217_AACYHA lan_j_Page_010.jp2
79f947f080bd5da2f6d69a6c85255a3b
c776aeea57912b989767e098b0350f44c21cde7e
551073 F20110217_AACZKC lan_j_Page_085.jp2
02d2c6936d64d565b8fa50a99e1d67fa
ef047d62bc84273ea7b835300579e43524055b63
727603 F20110217_AACZJO lan_j_Page_053.jp2
48905990d5018e770d4a0fcea6bcd5b2
26e83efea15d55aa4035a702b6635627b4eb81ad
5551 F20110217_AACYHB lan_j_Page_119thm.jpg
4879cbb36dd79bb40781dd651cfaecf5
02cc74cc4d5dc4a6c33f69dd59a077cf68df6e5c
894247 F20110217_AACZKD lan_j_Page_086.jp2
e3130bb909764865108758d7a95a8a48
8bca67c59de0ee4f91b9c62bacb92e213b7b6f3b
629950 F20110217_AACZJP lan_j_Page_060.jp2
fb4459b5805af8c6e5a1b03877341334
f403bf4b1c7ea007570617fe1275dbae5235c5b0
1599 F20110217_AACYHC lan_j_Page_108.txt
3bbdd172126f6e044c83e55f453f5d8d
ef27830a43f54b1945212eab9f91670d68d80475
876876 F20110217_AACZKE lan_j_Page_087.jp2
a72d56bd21760f5bb9307b955d67baea
a61a6cabcb7793dfc81b69db80f97d065a0dc5ca
857910 F20110217_AACZJQ lan_j_Page_063.jp2
d7cbcebe96e6eafccc362f02ba0a1097
e228d5c89590618099dd29fb0b832a2d3e3f3689
34087 F20110217_AACYHD lan_j_Page_058.pro
3c1b6ab360ef569fc73b169602565a4e
20896e5f4eb0fcdb01431487ea971f6314db9e68
705833 F20110217_AACZKF lan_j_Page_089.jp2
204383ab32cea61f8241344a3c81ff77
c300f628ee306323e7bd6abaf0b3dea728579873
1683 F20110217_AACYGP lan_j_Page_063.txt
b89cb32ac1e3b6c402e73201317267b8
6bf504868b6587031ee78ca76c90b8a3dc950235
724115 F20110217_AACZJR lan_j_Page_064.jp2
1d88347729937d6bb6020b161e7edcbb
29033386ab694db8f6a7cae43b2c1c038619a4fc
31406 F20110217_AACYHE lan_j_Page_007.jpg
b0602d10ba66c0a8d662d28ddea16b54
76a0192d51413111386a2e8a6a0ad20acdbfd046
940951 F20110217_AACZKG lan_j_Page_091.jp2
97d17f9e63a68f5f72f6e01d278ee4d7
5c0c9dc3b1c2490694f23e2ae95594600831208b
5942 F20110217_AACYGQ lan_j_Page_016thm.jpg
195af3a3b08cdba15d88801fff094586
002667956d32ed5a8cddd3f46449aadfa56174a1
530360 F20110217_AACZJS lan_j_Page_067.jp2
f0297d36da50158f6b629ad67ae9d79a
e4ee0211e4342863416aac9610241998a7d16acb
F20110217_AACYHF lan_j_Page_107.tif
cdbf2e428d2d33d71460bae2027345d1
d392e7573726ec38200e698487159ac306e039f6
744715 F20110217_AACZKH lan_j_Page_098.jp2
afa7b1ba71853806b6b2f332e482b888
d6bc91a16c67f6edb7e9d6008b45888e2c4ade81
17262 F20110217_AACYGR lan_j_Page_067.QC.jpg
d00fadaf2193e8395248869e1d96d6c2
456412b2c208e2c20c8c11bcd4d30399408cea2f
595920 F20110217_AACZJT lan_j_Page_068.jp2
fc8411274eae6c09489700c712ea8227
6d448397f7811e22096cf083d770db2b1b34bf51
897439 F20110217_AACYHG lan_j_Page_097.jp2
67101baa11918e94e12163ea54a798e2
3daf43879a282699555ae833ab19b225a8468de8
F20110217_AACZKI lan_j_Page_099.jp2
95e25d9c8721050384567e5a6395b912
b5a4ffcdfde0e65656c564178733529369bff2e2
86 F20110217_AACYGS lan_j_Page_003.txt
477a9373c3dfaf24ea3874282f666aad
613c738147dc9196b8023148d1c3cee7947c2284
893939 F20110217_AACZJU lan_j_Page_069.jp2
e074531b60e2895e127f41de68309634
8ee0e6e90ae2a73574d7c7c3450f378432209ab0
7978 F20110217_AACYHH lan_j_Page_123thm.jpg
9ee64779f28f3a340639ab577ab92e35
e4e58df028dc1584b8aa5c32db14c635a3ae1ea5
513462 F20110217_AACZKJ lan_j_Page_104.jp2
7a291104a82597778790b8088d6b5327
86d8622f155acdaad88671fd41c3c2d13e652afe
2055 F20110217_AACYGT lan_j_Page_109.txt
70e050f060c0244e989ebdf19f969b82
19cb8f4661384a32e770022a562319f9b53669cc
810932 F20110217_AACZJV lan_j_Page_070.jp2
b8eb3a312b95223480ecfdc42d9957aa
15f134ce6d74c26476967241fc13960286bfccba
15025 F20110217_AACYHI lan_j_Page_042.pro
46015cdaf7ddce3247e5e8137e1af408
d1fa881eb28284918093961bbb8588a0591646f7
865987 F20110217_AACZKK lan_j_Page_108.jp2
f5fc39a91ca03f52033066e7676d9bb8
dbf4fedc9eee70dadb93fa44a6769f1ffa95f23c
37280 F20110217_AACYGU lan_j_Page_029.pro
0e0dae6c72e219e3c5baf08426c98613
f222de0517f7c30155c5ab75210c97113f5ea249
948560 F20110217_AACZJW lan_j_Page_075.jp2
f36c875d6def4f5a9003489c7490bd3a
b62cd897baa4bf63e37033f29cdabb71f4b0808f
29934 F20110217_AACZLA lan_j_Page_014.QC.jpg
8ac3c528b432beb9970053e3c9b24ab4
c181d5b2dedf168e7af1beb94830b4c893514a23
53342 F20110217_AACYHJ lan_j_Page_015.pro
61a0d425ff4e5798c0f3efc88e082ec5
a872661ab6ddc2a7bc656a2e36112e2948b6310d
1051950 F20110217_AACZKL lan_j_Page_109.jp2
d64b46dc4e311ab7fec13ca3aceec174
edc296c6cec833b02319a81f9958a39a3bf6cb4f
7364 F20110217_AACYGV lan_j_Page_014thm.jpg
0278983d475869666bd569a9a7828128
76822651c69c5ef83a68b57e920c621b4cb77565
846579 F20110217_AACZJX lan_j_Page_076.jp2
b8e3915b4a0feaead8859b18dd528c83
4a12d698fdfc277a495f61b306106ae6fbd7b7c7
24777 F20110217_AACYHK lan_j_Page_009.QC.jpg
65c5705ad5679bad901cb0c2ea78a3c5
5712dc7707d007c82cd8133015ce81c24697461e
532843 F20110217_AACZKM lan_j_Page_110.jp2
8159a935514fd73ec7b77446fd882c6f
aa176dedc9bc2389c6ee715d07ce833d06cb15d5
37673 F20110217_AACYGW lan_j_Page_095.pro
dabb85caf5bfbdba5239f9f2ed95d969
c3352eba14cba5c0a02631bbb1e080a43a315378
1038675 F20110217_AACZJY lan_j_Page_077.jp2
060778992bbd6aca5597d21378431c3d
47d2cb8e24ec14d3d4ea283b75b6877305889142
21290 F20110217_AACZLB lan_j_Page_053.QC.jpg
d3b600c127f80934c5f5118ce76f2d47
d260f4c51c2aad75fde997b32c362f7f863c148c
89786 F20110217_AACYHL lan_j_Page_035.jpg
f229a0db7f02aa48cd16a7369ff5dbb6
f7f6b4196bb4f5d0f851b7bf2ffb1cc1e9bef44a
944896 F20110217_AACZKN lan_j_Page_111.jp2
db35de117715f38c149791c4a3107f29
791280b7d19bdb8afc62b1d12dff70297082d7ce
68692 F20110217_AACYGX lan_j_Page_080.jpg
cc4b51e7b550895cde6eccc7dbc126bb
0c462a3a9cea78d7703a22c494eaab88ffebe21e
765213 F20110217_AACZJZ lan_j_Page_080.jp2
313a823ede28e8518bfc26615d2e7869
3b2f162d16644464a01b432e28c772e7a7796e06
21339 F20110217_AACZLC lan_j_Page_044.QC.jpg
23f65109bffbaad3941628876a996982
cfd24718ec5aa686d2813cf26eae99639d7001c3
72028 F20110217_AACZKO lan_j_Page_114.jp2
64aa7f97ea776a0aca2246e8cb9522a1
c3eb55ecc0ac86b5b94c42becc47ce4e4de80d2c
F20110217_AACYGY lan_j_Page_069.tif
5da760e39254acf61bb7cb55e4df7c54
3cb6b5559b62aabe9f60c28288fea6d914e2b818
42438 F20110217_AACYIA lan_j_Page_115.pro
a6582455421c040cd9cef02bbbace608
e6732c7ad06a1cdd376f6cc570906ea09a8583bf
6856 F20110217_AACYHM lan_j_Page_069thm.jpg
19842aba7198385792b030f8ed4c6bfb
1fa575832f958643ab937acaddcb9096df104400
5695 F20110217_AACZLD lan_j_Page_053thm.jpg
4757b2ea87ef076f66c33199747241f6
04d63ddce6320d76cabc5d14bd2757e6b3db1dea
954169 F20110217_AACZKP lan_j_Page_115.jp2
973560cdf815abe70e5b3cc7d6c3946c
ada72444d05c507d53ec30810b6a649268745d5d
2398 F20110217_AACYGZ lan_j_Page_125.txt
d0da7cc16150a0db35a2d585cc14e2fb
be8cec462ea1f25b163cf8f1a71be45f9195ed81
F20110217_AACYIB lan_j_Page_050.tif
181dc29ef20b998a3891c6d90ad20527
1afa08e5237305ad071e12c0c74f94c884e017cb
F20110217_AACYHN lan_j_Page_032.tif
4a0f5a231c3e0afa3062db6776a88247
193a1d2aeb0f667ed03edb8c5e596991101adc83
23909 F20110217_AACZLE lan_j_Page_103.QC.jpg
3f0c39ba366bdc47aad1c874bb646122
1cede41d406ad1d0849796a9dc297112bf5114f6
831008 F20110217_AACZKQ lan_j_Page_116.jp2
adb24219386e04574ab51e6e0dd6450e
89ce2d979bfabd08d813eb6dc21b3bcf959ad46f
6152 F20110217_AACYIC lan_j_Page_071thm.jpg
b9634cecbd661e1566cc641558879f60
a62ee20c21cd61d249435cbbe93e040089a6c90f
27941 F20110217_AACYHO lan_j_Page_102.QC.jpg
bc9a2eb34a833a68aa5bd2d6134407da
4da5df82f3f1ada0d85a98d2bd63fd3d7e60b1f9
26849 F20110217_AACZLF lan_j_Page_077.QC.jpg
2f090f4e4b2b62976990cf318ed6f3eb
d04918c13038219236eb1420dea91a9166490bff
782025 F20110217_AACZKR lan_j_Page_117.jp2
932c38bc060f9eacacc7bdba14c54a2a
6e16145cb587484c7cb0427d232fad779365eb62
15044 F20110217_AACYID lan_j_Page_045.QC.jpg
e45036a7716790a4101fb813cc7c5065
4d9cb0d70b9297a90cc321942be2bfbf045d61bb
35010 F20110217_AACYHP lan_j_Page_051.pro
f1067b5b06946ce3171abddb7cba9b98
55fc65eb5338f41c3af450780028ff8b1c929151
6900 F20110217_AACZLG lan_j_Page_031thm.jpg
053b6adb510e95a6b4c1e70465696a22
1f5813501b9f81f42165c5de3ef6fbe5e95df02f
829068 F20110217_AACZKS lan_j_Page_118.jp2
3058d2d3df1d557bae473dc362929c6b
2a9f83f2c2d7ad3400e02f684b75c5e84f58d319
6540 F20110217_AACYIE lan_j_Page_113thm.jpg
029116d32a3ad9a605db37995af07b67
26acac7eb019ba2e3c95af50910dbefb650a2bf4
857695 F20110217_AACYHQ lan_j_Page_113.jp2
ad2c4e9d46627126976663dc77c6f15c
585b0b1ffa6981248d220b5a266dc3b49a28e179
7213 F20110217_AACZLH lan_j_Page_077thm.jpg
53038d590378cde19f171607f122cfc9
6dce61b465f212e9b2ca8e1a301577ec462fbdce
570844 F20110217_AACZKT lan_j_Page_119.jp2
88defdc40f6e40dd9cfa7311fe80e80c
c7e63803728eb8750da88e18bd42ed62e028fd9f
966452 F20110217_AACYIF lan_j_Page_072.jp2
7b081866fe5c2dcf081048c843ec7f5c
5268bd8c80417573c0e013cf13380ec1e7aa91c9
6610 F20110217_AACYHR lan_j_Page_097thm.jpg
e0e40f386dceac097c056f7aadcf1ec9
0b5c4f74c7dca381728b12e0b828d8c141e83af0
24417 F20110217_AACZLI lan_j_Page_091.QC.jpg
eeb4d91b6aba79a2c39095235bd4dd8f
d1f83a03313843df2b74608c63e3eef2b1ddc40d
1051969 F20110217_AACZKU lan_j_Page_123.jp2
9981725c4d77d5794056995e9ff8a496
ac45ef3b5c7f151cb22172edd5fe25a54f10db42
F20110217_AACYIG lan_j_Page_111.tif
5317f8e6c5965542ddaf229118eec4bc
8f669cb2ef26b22bb497dbb48357bb9e235a3fd1
59332 F20110217_AACYHS lan_j_Page_125.pro
b077b8fd604355a49acd8969ef5b7abe
c1c1d294ffc244a9e52b81cae2a80c268eddf912
22934 F20110217_AACZLJ lan_j_Page_056.QC.jpg
83462c79a4f7a8d29215dd98c0d43058
83afe83f7b6cc381f2444950bdf6dfea6c164c01
952501 F20110217_AACZKV lan_j_Page_127.jp2
8cbffa8737e81fd30faba686620c18d4
e81b61f8d080655877353c64fffdcd45ce7f658a
75600 F20110217_AACYIH lan_j_Page_112.jpg
d1a781700a0ee9939d7cb1af2556a172
372dc9dced1a877369396a16ff957e39e294dc87
21064 F20110217_AACYHT lan_j_Page_057.QC.jpg
3bb2afd86262679c7f028a99638c9aa7
962060d0475d8ba42cbed9295784a62169715ac1
25569 F20110217_AACZLK lan_j_Page_088.QC.jpg
a830ff9f2d845110f4692c8adc4d920f
91cee44716410c41d51d4133d021b76fac5f4fd4
2200974 F20110217_AACZKW lan_j.pdf
b1417b08c9e871bfe0109b2d7f7213d5
d7346f59f4667b722878fd776904923cc137602b
F20110217_AACYII lan_j_Page_039.tif
75fe5ed0d839e6dc080ea5de139ea7f9
bcd7aab970584ace935a80e9372e03d4cf078ba5
664 F20110217_AACYHU lan_j_Page_007.txt
71d09c0aebaf41671d5dd707f713cc74
1a4ee8fcea09fc273faa557aa5b3d861c7e62a6d
25316 F20110217_AACZMA lan_j_Page_022.QC.jpg
95eae3c23b34e1ad74e24a246c1bbb06
625ad131956e73609c83b6ff557d66375a6b2058
30526 F20110217_AACZLL lan_j_Page_096.QC.jpg
a47c35411a0bf2c8a2b6b720589755d8
f56b129198576a9c97bbf610f5fae40fdb71fedc
21360 F20110217_AACZKX lan_j_Page_073.QC.jpg
c26ad362a0b035af41c1fafdae95d9a7
484c9bd337fd4744875e65f99f20291c83130205
23106 F20110217_AACYIJ lan_j_Page_093.QC.jpg
6e2d32a32ecedf1c9ae3d227d80952cd
fbdb53cd31029c24793d62ffb905421e17dc8899
24977 F20110217_AACYHV lan_j_Page_041.QC.jpg
cd51510a618a2b59284b1bcf6c33ac87
d20fe1f9879c132144214afac737e3101711d1e3
22831 F20110217_AACZMB lan_j_Page_071.QC.jpg
18577ef94e247b2d69494c5278f99ff2
6ae0e03af9bdb6b10d2aa45af30430d2474d6048
6301 F20110217_AACZLM lan_j_Page_029thm.jpg
9cd13cf356d06eaaf62e30109ac9255e
53aa50a92b1a1a79f81f50cc36544e8cca79fbfd
8165 F20110217_AACZKY lan_j_Page_122thm.jpg
bfe0f23c6ae2f5b499211f7ddfa40a02
4a6cbf9acbe12f74368e609368fe6b02ca11d46b
828177 F20110217_AACYIK lan_j_Page_029.jp2
32e7f274e992d3c946b365566213c34c
0415d71b2e5b84c71cabed3c34168584d81177aa
33105 F20110217_AACYHW lan_j_Page_124.QC.jpg
fe31115d9ff7b87dcc0c62e519183074
623de35a6ef264000f60b01175ca30cb837a513a
5236 F20110217_AACZLN lan_j_Page_008thm.jpg
60092dd209771c60c29759eeee3bf4ca
40548e98207da3f1178300fab204129411d67c12
6619 F20110217_AACZKZ lan_j_Page_055thm.jpg
0b1c9e94b93e1395c5895e0c6c0961c0
a70d5d945675ce5717f68121ef0b082de764e6e0
5160 F20110217_AACYIL lan_j_Page_046thm.jpg
6d0c3f74991ef317cb198e1f23948fe7
faed13ce59c68ad53b78c600de4118888556a979
1686 F20110217_AACYHX lan_j_Page_086.txt
8923bb3931e1ab6cfcf858dc977cb4b1
03cbb724c0b405d775f9958fb269d0e03e68223d
25647 F20110217_AACZMC lan_j_Page_063.QC.jpg
2130adaaf141e53f7e6bac42fa58b167
d113d1bea633f7326e5cb9e4e50bc1ca6d2777cc
23273 F20110217_AACZLO lan_j_Page_025.QC.jpg
02cab34f29bd1e36bfdfab4e9f913633
249f6c57a1ebe49d0a89a5c68220e430f19f3bea
33383 F20110217_AACYJA lan_j_Page_077.pro
8d89ae096fd04f746a6e5f3194dc59da
7609692f84835849731366769cb86541ea221712
52204 F20110217_AACYIM lan_j_Page_013.pro
c3018b8658cea0ffeb691c50590c9ffa
33c293ef32cc4fd5445b77294bfa66d5084a2938
36857 F20110217_AACYHY lan_j_Page_128.jpg
bf1769f4584d06af76cb733560764b56
8ff96a675fdc220f2cc1b83683232021bde136bd
25466 F20110217_AACZMD lan_j_Page_010.QC.jpg
547338a34c46b9acf519228d4747b82d
bdd690ba51669e871de88f09d59ee6b4b64308ce
14762 F20110217_AACZLP lan_j_Page_085.QC.jpg
8fbf08b6a536423f0d60487966a4da73
384a032d3a919ddc9f7c83857ce8f559a6331b35
F20110217_AACYJB lan_j_Page_090.tif
903947a1e1c6ab24dd4de0e0bc719ffd
d7cff49be4112533edc57ca099305178d60e0d3e
F20110217_AACYIN lan_j_Page_110.tif
f1ac0e88017c27d29d16719b78e4a0c5
90a81ca56539a9eb54cf8e1d8d277b5b33eef7a5
93443 F20110217_AACYHZ lan_j_Page_038.jpg
4798505a6ff51eb4a5fb1988f1f65a4a
3721ef84b282e160803d4c21fa31fa48291472f5
24424 F20110217_AACZME lan_j_Page_023.QC.jpg
3476cb1cd60c1402b4263c5edce95538
0b29190ea455be947f06e66a56898041619ce35f
30381 F20110217_AACZLQ lan_j_Page_017.QC.jpg
9af1c6a163419dbb0d4a7af97e49e623
36efe0be77b03d537de0459b7fd7e7fde0ffee38
F20110217_AACYJC lan_j_Page_091.txt
8746f10643a87d4376b73b4eec1b99be
7b83495553014f23f6635c79b367a0de81bf31c0
3965 F20110217_AACYIO lan_j_Page_110thm.jpg
fb0bfc964c0f5c192a496e3ced047798
04d9a64b2c42004d96551261be384ce61e6abf37
204027 F20110217_AACZMF UFE0014640_00001.xml
92d1433a6d541520edf79c15b1790084
38b09fd10d3368e76b31f1e5086d0ea671435964
7126 F20110217_AACZLR lan_j_Page_036thm.jpg
db0700fce757233588d563fb205653bd
51addd6a0fab34ef09e6d02d53418ccd01351b37
6526 F20110217_AACYJD lan_j_Page_012thm.jpg
d7ad537cfb4b256073c9b66148fe1b9b
89f7f00593da6ff4dc5500c172c4099c16b14c0c
1809 F20110217_AACYIP lan_j_Page_012.txt
a84dacb1c14e1d7e96f43d6bc98dc584
edcabbbb6c2addcf1d8c31535b30813b9ea6fdfc
23665 F20110217_AACZMG lan_j_Page_005.QC.jpg
cd53ef7beecebf0ce3c78d738a9690a0
29dcc2aed0e41325e96205814c5accbff5cb81cd
6109 F20110217_AACZLS lan_j_Page_075thm.jpg
b113a7a08b0ac2c5f6cdd9d39519b654
da7b7661918bf5e8e3e15da6be3ba90fe0cdda5a
1375 F20110217_AACYJE lan_j_Page_043.txt
00b49bc056579ed242d5364d23880c52
3f800dd30ee8ba25b2d08cfab98a43963f9db5a4
961958 F20110217_AACYIQ lan_j_Page_074.jp2
4f28b3009b28380231b5380d87923214
6c2468d5542c4a02341b2183b4c0987b84af25ce
9814 F20110217_AACZMH lan_j_Page_007.QC.jpg
39a82c7969c73801bad3493cdbe9d51d
9a237ae1e179906f357f8514d0625670db9513cf
7516 F20110217_AACZLT lan_j_Page_083thm.jpg
5850a123c30bcb647d4be1bdbc3ed096
6f1c5158939a002f6be836b79c5aa083de14be87
84890 F20110217_AACYJF lan_j_Page_074.jpg
b365ec28f6a3e80ef9542e7ac4c2d619
1307f9b9b9954aa387dfa100a6a06f665dfae1c7
81624 F20110217_AACYIR lan_j_Page_022.jpg
29db2c0ca44015e560ad56c28fed7edb
a63b02c32cfb411e8384c534b995adc052d1e143
23668 F20110217_AACZMI lan_j_Page_008.QC.jpg
315e94e5612ebda0b3afee6dda4b9ff2
78c5ff80dd6c0a978e402276b6ceace93b5efc87
29787 F20110217_AACZLU lan_j_Page_121.QC.jpg
21ea5349e1f630925b4a2cc7c335d25f
f8ea2d50a3ba7648f01897a7a42a5f2f47a859e2
55563 F20110217_AACYJG lan_j_Page_046.jpg
3ccfa0e5dcb4af150e64afa458e90c92
7687c5a275fa6cfe2a9669318b217bd03278dd4c
45855 F20110217_AACYIS lan_j_Page_009.pro
6682f6867f1b94a4b9de6e613faee984
b6263dd1f1ea9dc1c5c25af6de9e0a359167a349
30597 F20110217_AACZMJ lan_j_Page_024.QC.jpg
79bb93e53139ec259b1309e5e495f8e4
ead4bd2b47987846da3aa70e329044cda92b7dae
6706 F20110217_AACZLV lan_j_Page_116thm.jpg
602a022e05ae1832201563b8fbc2dcb7
8d615ec875652dbd9f0fbcfa0a97073b3be8ed41
6323 F20110217_AACYJH lan_j_Page_076thm.jpg
462d32409e8a57dba2db6ddd959762d6
b5ff5b48e4ff9b38ac5fe3c6d4127d30718ac6ef
F20110217_AACYIT lan_j_Page_091.tif
4276147396a3b3ddf74ed0f9bf022bdd
0c01472dc605ecab72cb0d25dfd02dd411a372b7
28495 F20110217_AACZMK lan_j_Page_026.QC.jpg
3de7d5e5d60f6d6b54deb2123ae55c96
6a50f50d1d997635f55bcdc0a12aba322ba7ab17
6237 F20110217_AACZLW lan_j_Page_108thm.jpg
988446c5bd2240f365960639f2275bdf
390a653fadb1b0cce20d434087a0c43ef614f79b
42866 F20110217_AACYJI lan_j_Page_083.pro
6059647ac1060b97dd702490c2ebcb00
777dbeb84ac0fe052943583c1f549c879a4758e9
81705 F20110217_AACYIU lan_j_Page_069.jpg
282d85780adcfd2b9a490b72b301d2f1
5047cf0ca78aa174575d76474f13f031bfc564b7
22137 F20110217_AACZNA lan_j_Page_075.QC.jpg
bdac90377fa86834ac29e561e534ceb1
a0bfe9e4cbbe71c0f34b5bf43d2652b9eab9e3d4
27040 F20110217_AACZML lan_j_Page_030.QC.jpg
bdd35a6010fe2aa845b4ccf767410e84
9a393ed13d31aab5f559d81031493d87deca06e2
34091 F20110217_AACZLX lan_j_Page_123.QC.jpg
a81bb64b4373b44be8f0b3dd73bf0919
00bca15c4f92dfa0115ad65af4e22cd0bee0510a
24056 F20110217_AACYJJ lan_j_Page_112.QC.jpg
a73591973c70322ff392b119d1db79c1
85aafb0120503dd96d1eeb7c72a23fcce105ec58
5200 F20110217_AACYIV lan_j_Page_068thm.jpg
0df8df176348d8e154d5cfb03212338e
db7fc8dbb568c1092ad292e172ad279f2545ca4a
23001 F20110217_AACZNB lan_j_Page_078.QC.jpg
08f40f019c0283f66248da1bc66df5d9
f9473e6760eef05f06029863dbd089184cdd59fe
26704 F20110217_AACZMM lan_j_Page_032.QC.jpg
7b09baf43caa15e2786478628f0377dc
9515708c60c9fc042fab53b8d34e3e060d158c7c
13651 F20110217_AACZLY lan_j_Page_042.QC.jpg
826a967dc0f5bfbfbd8577c7e1fcbc46
1a3d33dc0f1a939e38b4b5ae07d752b8ab3151f9
1576 F20110217_AACYJK lan_j_Page_101.txt
ee495e4b2bd3c69c1fdb32a950f1dc73
059b57b19ecb1cc8ecf7d3ecf99ef07ff9fbdb39
78374 F20110217_AACYIW lan_j_Page_048.jpg
5da8f2ea0109d530faeb969de27666a1
0d68785c4ffa7f54b1e6ac650511154039e1d2ed
21752 F20110217_AACZNC lan_j_Page_081.QC.jpg
0cdb9105f4a8ed06c9423470223551d6
a2b5c7755ef42b6b67dd1ea3b6351014d0542f48
28564 F20110217_AACZMN lan_j_Page_033.QC.jpg
3acaad2fd2fce11ec98dbbc761d6c262
c1ffea9d7eb816725feac03a00f96875cd4ccc48
6928 F20110217_AACZLZ lan_j_Page_001.QC.jpg
c2b9689217d9a2fd6cc363740cd9f9ea
f92d2e164e23ac8fa4b98e12b12fc042e7a7da88
5400 F20110217_AACYJL lan_j_Page_044thm.jpg
9a8e8cabd9674e4b08e3fa4fad1a8d29
b5e87efcbbd7a1ef4bb1af0403c2b61e9decce08
81067 F20110217_AACYIX lan_j_Page_051.jpg
8067e339cb4973176068b7c881eca2c0
1c188062345cec6cfa5935c6bb62bdcd3607b6d6
14876 F20110217_AACYKA lan_j_Page_089.pro
4b7bbb9f38180234a77781d3ba361da5
983dd6632eb345844befba1c1f5f502807da3c0b
28257 F20110217_AACZMO lan_j_Page_035.QC.jpg
a2bdde4279bee7d701333183630f6cde
879c898b6a27681d17857b3332320d36566d6194
24301 F20110217_AACYJM lan_j_Page_076.QC.jpg
b1ec7f8a16bb90b3b21ead4a74484921
283be0d7d6268b6e4f748178d9cd4ad3fb533229
58987 F20110217_AACYIY lan_j_Page_106.jpg
11c00a0f4dc8c7e46b8beb4bc23341a3
e9bb3c7fe5c3e1c1f038b495bd9882c9da2f4d22
26444 F20110217_AACZND lan_j_Page_082.QC.jpg
aaa48628a43a97252c3f539a2c09c48a
779a8d6fe0f712897fea80bf7d364d870d04fe06
27488 F20110217_AACZMP lan_j_Page_036.QC.jpg
835cf4768549a845fb8278a6d62f87d0
f87305bab5bfe6e09bae34eb226d992c97e9278b
39592 F20110217_AACYJN lan_j_Page_063.pro
183b53d592189f7239bc59b364bf105b
3b4fe0e0c6d66d1b0c7941368a09b4474a3d61ff
F20110217_AACYIZ lan_j_Page_124.jp2
49091590d0369e90ebb0033b564c9223
19cf18188e32da53f23fc639a0e73ab628a98dbb
17479 F20110217_AACYKB lan_j_Page_062.QC.jpg
cf2d20cdd577b1c6770c1e0f59be0c6c
6eabf9898c9b05045ad33bddc4bff08e2375d610
23249 F20110217_AACZNE lan_j_Page_087.QC.jpg
f9ebdf6118ed8840663451a04a98c469
1dde0c5ca87aa52ffbe35d0b77871130f6258a11
16581 F20110217_AACZMQ lan_j_Page_037.QC.jpg
c00cf04141988dd372ff870fe66cd76f
ee225caad2ddfc4e5de682037940a9ba8814886c
17033 F20110217_AACYJO lan_j_Page_039.QC.jpg
735da6f6c112b6f154da854b82b33e92
fd674b97da4481196c30ed554dc29be896a81e9a
35021 F20110217_AACYKC lan_j_Page_082.pro
24284c906768b911987d39b1e96aaf78
b8234fe890d57aed231a3d411ae378c3a78d17d9
19441 F20110217_AACZNF lan_j_Page_089.QC.jpg
5f1ceb298c88bcf3608730488db14d73
39a79becc76e390812f15af24f71c9cf10bf2722
18851 F20110217_AACZMR lan_j_Page_046.QC.jpg
df5a67c4ad952286a803867e0d320bc5
a6335d8bd1679a7578489c151df4c05c9f3145a9
7586 F20110217_AACYJP lan_j_Page_027thm.jpg
00043ca6b95638dbd8c0d875bf0a488b
59fdfacb38f14587539e97e79e768dacdd037b1b
F20110217_AACYKD lan_j_Page_006.tif
f80cbeaff11f80845a3ea15e8750e438
52920d7467139b5a4714fe62d99e32a672352983
26678 F20110217_AACZNG lan_j_Page_090.QC.jpg
ca9e0d8ca9e1aab0a1827d00fe823dcb
bb60875cbf7a539d9515001dc07621c1a11e7eda
28048 F20110217_AACZMS lan_j_Page_047.QC.jpg
dfe10a7b3d7c3da8d35de351884a27d2
443c7919b965de71d5ec360029099d8a640f77b0
1196 F20110217_AACYJQ lan_j_Page_054.txt
201b497949d40a7b6d7bff979b3922c3
133f07e83f89f49b7e929c2b56411b7d1b4569b9
18596 F20110217_AACYKE lan_j_Page_119.QC.jpg
9786435fd94516dc0ccd526bbba5bce7
5dffea6728ebe0553605ba5307d8be8fc7b1742f
24063 F20110217_AACZNH lan_j_Page_092.QC.jpg
207f5f19fe41bdb62aaee963b2f44328
06b4d9aae3e6adf3422db752dc34cef6976dce07
29291 F20110217_AACZMT lan_j_Page_049.QC.jpg
9cf2306ee35fd5b180f1424646da322b
913f5ecc0e722a148e58df8b9410167bc4932d46
5701 F20110217_AACYJR lan_j_Page_098thm.jpg
7d3046b366bc229bb3239b8fcb1d1775
96b390c3f545844de4434b33c92ed830ecaa35da
7104 F20110217_AACYKF lan_j_Page_038thm.jpg
889f70c65cff58f0747ed46480f3be71
50bb6c6c2ed6b8cfaa34d417754917a55cf9d51b
26936 F20110217_AACZNI lan_j_Page_095.QC.jpg
9e75116fa9ec9c648479e268bf94d537
11af427e55f92b4b03b1af0bcbf7c1939d4665b1
23968 F20110217_AACZMU lan_j_Page_051.QC.jpg
e71d72557b3b59b722d9fbe4e125a8fe
a8b5d671ddb4bcae29f39eddf265a2d7e7df6475
18105 F20110217_AACYJS lan_j_Page_004.QC.jpg
b448b912827a478a0b27427cb51baad1
bfb80235efaf70c2015ea8b5dfbb683fca9f652b
53801 F20110217_AACYKG lan_j_Page_104.jpg
0e3be26e0f1286b1741495e55bc8d259
d39b5a97c3fc589737db5eb58e3ee0f26f19ac7c
25203 F20110217_AACZNJ lan_j_Page_097.QC.jpg
ffb1921f60d6a93e3884da87de9c79a3
bd458f8f0ac8aed60e0a26b850c5a97a41122576
24128 F20110217_AACZMV lan_j_Page_059.QC.jpg
013c228cc6a08857b569a71fcb6539e2
3283530af9492b54170f922d6842deebbc51b0a7
17583 F20110217_AACYJT lan_j_Page_065.QC.jpg
0ad1bebb278a4ba496d5b3f13b7ce39b
9cab1e07b942698fe67e8ad5a16a123c65eb8829
683116 F20110217_AACYKH lan_j_Page_079.jp2
337490d8570123465f680cd6b1883905
9008c2960e910c6a05463038d13db8ec5434abad
17838 F20110217_AACZNK lan_j_Page_106.QC.jpg
6daa5644fe9512113f2df69acbf02202
49b1aad68c69599636544f951820f15b50151ea6
26821 F20110217_AACZMW lan_j_Page_061.QC.jpg
73e602bceac5f5766477f3669c6d2e34
f27b078c611e6bc6a373aa9d13ff69f21acbcc15
1776 F20110217_AACYJU lan_j_Page_010.txt
9045328d3431274fa504394c30a29c1e
9537965c241f45363fcfc3a9c55323671842e052
1737 F20110217_AACYKI lan_j_Page_115.txt
a3bed9339ab656405f085840eb8266b5
e3d6c4ad489147af713e5b434d3e28393c6e6f31
5263 F20110217_AACZOA lan_j_Page_020thm.jpg
dad0368f6af2139c54a4fc3a180505c2
a8e294742c12f0f048ca5f597def5e71663b2c95
19373 F20110217_AACZNL lan_j_Page_107.QC.jpg
c5f5b197366cc4c96fd458d86d19af0b
68fc26ba93624230c3da9ac460cd3a4265775cf2
20759 F20110217_AACZMX lan_j_Page_066.QC.jpg
fcb637299a303a103d859cc0bef72f45
ee444832ef98c4952ea0d5cb90d221fcf16f1c8a
41270 F20110217_AACYJV lan_j_Page_040.pro
7edd67a0d4924c4ab901eb0a710d46b3
3bf174d7697d673833ebce87ea38ff3049b1e13d
62445 F20110217_AACYKJ lan_j_Page_079.jpg
4683d4e35788533b1d8a4cfda6b99b84
933352424f70a0219971112c6456b65a4da51575
7305 F20110217_AACZOB lan_j_Page_033thm.jpg
1ec1574af3fef6b54d321fdb19bf55ee
cb47f653ef64623c85759fd5a6114d545b1d52c5
24156 F20110217_AACZNM lan_j_Page_108.QC.jpg
1a8dbf4f0c9f8ffc3c267bc63867a019
5398a7f659f9a9facd9179b3fcdf25807e6aaa42
18464 F20110217_AACZMY lan_j_Page_068.QC.jpg
f42110e551d6e985747223e1bf198566
50e145e672285667da81596819f8b1d98db15897
25064 F20110217_AACYJW lan_j_Page_001.jpg
20590d0dfc04e69e5b7155f4511a592d
9dc6bcdf64759444b9880c8ae8c02fd4ad12a1b3
F20110217_AACYKK lan_j_Page_122.tif
e69670e32bed0a87f2cde9f289c5c070
f79ac0a781ae638f41708a23b619840e1d9888fd
2794 F20110217_AACZOC lan_j_Page_034thm.jpg
70c59932efcfcd57e912fd5d98d73267
ae0437958c7cd494e869dbc076aa4746f7b66bb3
29580 F20110217_AACZNN lan_j_Page_109.QC.jpg
eef8d1fd56e27ed2b19ea36bb49ec061
ccf78d6691642e24a39b31a5e9edae25b85b1831
25424 F20110217_AACZMZ lan_j_Page_069.QC.jpg
5a4c69cea7948077d06e49dd66e59c40
9689212a2f5c41ceb2195d9b08e39b25d4c0c583
1805 F20110217_AACYJX lan_j_Page_035.txt
387883ce256baf1ac931ac7fd38ea322
7a9f07c7be74f09bb21a7c6b3a20e9162896755a
48508 F20110217_AACYLA lan_j_Page_017.pro
5927b22adab4c52c22ab2043bfb8adfe
960c14311f9da26f555518b78c06be08dfcd28bd
F20110217_AACYKL lan_j_Page_087.txt
63d91c6c312ce54cfe89ded6a939664f
3b2ecd420ba7b71da5f793569c307ac549babbc2
7152 F20110217_AACZOD lan_j_Page_035thm.jpg
d0afd3488ef35e09de6d8654186a094c
370b04d749e866fd2695621bd9e139c992e9f172
15544 F20110217_AACZNO lan_j_Page_110.QC.jpg
7727c9d95bb1db4405eb21243375f1e9
7681999398fa08b7c54456f2e1d2bcb0b4b52606
6789 F20110217_AACYJY lan_j_Page_022thm.jpg
4e12bcf1ac7abeb9d245a80d7fd5d2a2
45b190d76951c13c6fb1b026adfb5b6d69d6b491
1051984 F20110217_AACYLB lan_j_Page_096.jp2
d8a2f88edb4ff0eeb3760b4771a1867b
54d4466c49582c2183022cad779995002fa5fe2e
F20110217_AACYKM lan_j_Page_076.tif
99f53457e62a76c3d94518d6dc287835
904fb6e317f9624dbd7971daf2db844f8102d089
25999 F20110217_AACZNP lan_j_Page_111.QC.jpg
ce3390582fc5200c6519d128fe257f6d
fd7244cedff9da9cad8325770d19b40a8fc784d2
6435 F20110217_AACYJZ lan_j_Page_041thm.jpg
6d6402bde168b4c67ac8c0b12647da21
9d31ed65c62d2395ef76b3cf8d39bba31037f8cf
F20110217_AACYKN lan_j_Page_065.tif
18ab099acd64d1bfbac8baab472ed8d5
1fe384283acbf90ed6ad69b35dfe134ed385f13f
4754 F20110217_AACZOE lan_j_Page_037thm.jpg
cdd2ce1db08d72276ee3de41a61413b5
7f22fe70357662339e658b4945511280e2802ea9
2893 F20110217_AACZNQ lan_j_Page_114.QC.jpg
b9a2e215fdd681a178fdc3fe019b8771
d1f8619301a3d80de26c086e02b2ad0d490ef119
F20110217_AACYLC lan_j_Page_013.tif
ea911e2b2cdf5deb4d1985d25ad832f6
928857c0c72103ca7039c5fc1c0cd6a82c195bf3
6529 F20110217_AACYKO lan_j_Page_048thm.jpg
6554489902892b90694e6e0d5fb4e273
40e56ba9cc8cbe5d687a2663b85348f7ee7847c1
3762 F20110217_AACZOF lan_j_Page_042thm.jpg
2f24d33435cf20e22833b7b0e777f107
fa258f2d518722895fabc7beee8abe15109e7bf6
24586 F20110217_AACZNR lan_j_Page_116.QC.jpg
e0ece58a3127338f7046c07978d62599
2e4efa0abc9c10062c1b1d50a52dddd5982665a9
75408 F20110217_AACYLD lan_j_Page_056.jpg
fbe4ca9d5f0cc1db2140320bb233378a
e195a7371b95f301c010e2ea672a081103e2e3c4
287451 F20110217_AACYKP lan_j_Page_007.jp2
0d0b479bdaadc7dedf90d86d09382f7b
bb4916898c97ba56c584c704efbf681ca0bf5c1c
5350 F20110217_AACZOG lan_j_Page_043thm.jpg
48131f40295295c9e375d50d011aea3d
27692a27304f32419f45b2bda833f617c5fc69d9
23246 F20110217_AACZNS lan_j_Page_117.QC.jpg
ee7749ad493ea2f950cedc1efa16d936
231359ddcbd8134870ba14ee1be1033cf4916fca
7694 F20110217_AACYLE lan_j_Page_125thm.jpg
005d152710d1cb8f78abc850eef1f2f2
4f56e60ccb10f63eae1adf6150e006e3c48ba480
38803 F20110217_AACYKQ lan_j_Page_113.pro
29c3b1e83d2ec0b576b0fccde8df4e62
03e76c038582cf69b4061d67a1d937e748af84e8
6462 F20110217_AACZOH lan_j_Page_063thm.jpg
ed45933d49e0ba605e6737dfdeeca7d5
2ebc691450ca24550800afc39b2aa6f881b05974
31161 F20110217_AACZNT lan_j_Page_125.QC.jpg
e3e4ede61fcf391e046d0fdceed4029e
b05d03d6198005c374b3672196b7197d28d8d757
29702 F20110217_AACYLF lan_j_Page_038.QC.jpg
896bcc43e8e9e56e2017ca210ba01fee
af33f5e049672e4067d8342dd2534f6be8f4fea7
588089 F20110217_AACYKR lan_j_Page_062.jp2
21be0673ddbb566ea394a0b598a260ce
a0aac253e622fbf28731f88c5f48beb0a2f74465
4731 F20110217_AACZOI lan_j_Page_067thm.jpg
37e1e14bc76a3442871f058254d88a5a
cc8365e0a41af1c7600ca3577aad67e62eb88b97
33642 F20110217_AACZNU lan_j_Page_126.QC.jpg
f10ed195f4dedd29f6e49997f1609d45
201e1d142fcf26617008820f7642655105d4cc6f
F20110217_AACYLG lan_j_Page_066.txt
ac8a990e33184294318b98e9d711e714
1483f2e69259af24efff020a504b2ecf12a4052b
711231 F20110217_AACYKS lan_j_Page_044.jp2
d1984936e30594b54698d5004c4f297e
813c38ed464a00a83b06029da209820856d49ab7
6114 F20110217_AACZOJ lan_j_Page_070thm.jpg
071ae3fdc682ba35bb5093f0eab0a2a5
2878fec3f96c916054bc80bd229667899ff94a1a
383 F20110217_AACZNV lan_j_Page_003thm.jpg
b1d95f85ffb8389d6b8f04ea21a04a30
f74dcc38d9d663a7d6231a08fc47a6b4ddd0c3e0
27027 F20110217_AACYLH lan_j_Page_074.QC.jpg
5c7984c63f779b32c8af2862c4f91f6a
08ab5d1ac10f3cc68b6647f1d1cc8293a25f9431
93627 F20110217_AACYKT lan_j_Page_033.jpg
fdceae94af7666a5b2c26970dc9d16b6
ffff0716c0e0f907f51e9d9d0b49a13ac970acc3
6762 F20110217_AACZOK lan_j_Page_072thm.jpg
1f629a059162b9b18608b31ed9f2138e
e61b72643bb2a30a3996a0ee1a2320175d14eb64
4952 F20110217_AACZNW lan_j_Page_005thm.jpg
ec66a3bcced35b4164e1e8063ca42df9
090516af7199b1b39b418e0d4e9e14e2e93b224b
64634 F20110217_AACYLI lan_j_Page_044.jpg
d0c0f1f50375237e5005f9890689691e
4df9c76815bdedd68cd1375e965a3f08a91f9c16
7788 F20110217_AACYKU lan_j_Page_050thm.jpg
9e8e3929b6edac71ada65f8cd9560208
53dcfef2e17f2e01fc5c2ccf51c2125773e3d051
5009 F20110217_AACZPA lan_j_Page_106thm.jpg
dbb350e8daf0f4a4ec9ebdf4f030aef3
a8f015137f62133649febb2ff5024cbd2ce93bc0
5373 F20110217_AACZOL lan_j_Page_073thm.jpg
35690b565e7e320d2ed4033011cf22fb
84ece7a793de0cf7a767b9fa137f5a48d41ed91e
2446 F20110217_AACZNX lan_j_Page_007thm.jpg
45ee529e541827fb1b12405205e7c960
61e4ac8152ae2e6f24caed714bb0e44a119f3304
6201 F20110217_AACYLJ lan_j_Page_117thm.jpg
2848a86badb86a560d127223018bb084
02e430a345282251af3c9f6853b367c140965c7b
77205 F20110217_AACYKV lan_j_Page_103.jpg
4f7ada9f4388dcbfb27c2d2432fdae6f
4885772fd2361776e0c6eb757a7e46b3f25c5b7d
5527 F20110217_AACZPB lan_j_Page_107thm.jpg
937cf1b88e1ed0614dc25e811cb94c91
a5008e268f0ece314a101ac9b09700efcb548fc7
5806 F20110217_AACZOM lan_j_Page_079thm.jpg
52808b1ae9057d860f8b5fa1541613c3
0b47e7ae3aa263c0a5b3eb3a06190bbc10295a5b
7101 F20110217_AACZNY lan_j_Page_018thm.jpg
2e65582c446d148ad6215cb55845e1c5
dd849724fd3d4cefdfde2ddb4066f7ec65eea7f6
43063 F20110217_AACYLK lan_j_Page_074.pro
06397d9b957aba88e0c1ab947da5547d
1d99e3ee72e580ec1829e60ed928492f39d25014
6027 F20110217_AACYKW lan_j_Page_087thm.jpg
be3dd1552d05f10f50863322dc84465f
40bfbabbdc66b277d47d226d1ae43823d694440a
7459 F20110217_AACZPC lan_j_Page_109thm.jpg
a310fdf915c689f72e498b12e9df072d
aa81e8d8be8b955b695c9f0a52090e81302eb16c
5177 F20110217_AACZON lan_j_Page_080thm.jpg
01020a904f4e91da71ad6f033f2dcc65
bea6fd98c6baaf6ebaf5ad10345141f67e2f906f
7394 F20110217_AACZNZ lan_j_Page_019thm.jpg
177aafd87d4858bf855892754cdcbbfb
75d60fc46d41de90b0d8d07ef023bdb604c64e9f
6291 F20110217_AACYMA lan_j_Page_091thm.jpg
24b7fd82d1edcee1f03cac7467742098
20434c7faee9f9a650cb5d26f35ca11ad3c0489e
6785 F20110217_AACYLL lan_j_Page_049thm.jpg
3f43d5831e872b35a5a12fd600fbdd69
269de96fa7d6d95de1c3126071fb3895ec1973f5
10551 F20110217_AACYKX lan_j_Page_011.jpg
fbd1aac1a8b729149ec1bfd86ed4dd50
5ba9e2e76cc6f819f73b6b1c52ebddf9fa872983
882 F20110217_AACZPD lan_j_Page_114thm.jpg
2e16ff0f2e19a3616791200fe09c4954
3b1a69d0cc7cdc7fc80f5d62bbafb5ef236fb65c
6093 F20110217_AACZOO lan_j_Page_081thm.jpg
9476f3fb2cb5908d18438d96654dd725
095df06afbb9a6f3174e1520c231ef03c0f0f7cb
24116 F20110217_AACYMB lan_j_Page_100.QC.jpg
434a7f0c74df5f0996d3584616e7ba24
209446ee05ea0f40ad6e6a5e7f38843b48d44a9a
F20110217_AACYLM lan_j_Page_113.tif
a4e7c4aa1d6f861e8c5578707c8f08dd
9cb3ed43bf0d849083b42b5111ded9c8367b0bc8
F20110217_AACYKY lan_j_Page_092.tif
355132d6d642d7382783bc33664c877c
e7d85f1a167b4027592cc396db480d7086f8047e
7748 F20110217_AACZPE lan_j_Page_126thm.jpg
8d5ec61d521e54de96ba8974620cf93c
37316d92dc5ad5e80763967343b913c6439c5d18
6899 F20110217_AACZOP lan_j_Page_082thm.jpg
2401f82a5e65c33ff82a7d6ca5e2941d
3f2e56eaad4da65d035a726a459c49af89f85696
2328 F20110217_AACYMC lan_j_Page_121.txt
5cde78dba1586524c30edc4ac3d552b7
92d7ea2793cf1aa8d645a6dcc4551e45c9be47ff
62524 F20110217_AACYLN lan_j_Page_066.jpg
fe33cecf4f584b05d1f5280c4f5d4ab4
0424ae6de16ec3d33b5b5eb84bbaa413199bf016
962581 F20110217_AACYKZ lan_j_Page_095.jp2
80ba33b5e17ffcb67390db7c352cfdb2
cc3a9bd1bdc23aea31c11dea94f2a68823a2dc6c
4157 F20110217_AACZOQ lan_j_Page_085thm.jpg
9fe0ce0a80afde9cdbacb61e8e53d43b
91c2f84776bcbe66ea0e44d5aebbffa059fd4ac4
16120 F20110217_AACYLO lan_j_Page_104.QC.jpg
56239255d89215cf53940d02bb9caf8a
885828c7d23d16d9a2cdec0c9938a43895b04c5e
5420 F20110217_AACZPF lan_j_Page_127thm.jpg
03743ce20619bbcc91c0506ecb8da05f
848a69aeaf575380bde7751b61e65d87a3157951
7002 F20110217_AACZOR lan_j_Page_088thm.jpg
e5f7c83a0246169e374de5e24212b006
3d710a0941ac2336433adbe3600341de707ff70b
1496 F20110217_AACYMD lan_j_Page_016.txt
69d4f4cf3db17320d601a4402ef6c18b
e6bd0d617403f179172ce7cd1f68be0005e6b636
21212 F20110217_AACYLP lan_j_Page_058.QC.jpg
08bb6c80be7bae4632a4822b39a39f74
4b4ee1e552c3798f5c640c7e2d2a2f592e6750bb
7046 F20110217_AACZOS lan_j_Page_090thm.jpg
27256a139b0109440ab07f75bc50b92a
9f5db08fbcce30e4cd735db8ee82aca7099ab45e
31498 F20110217_AACYME lan_j_Page_020.pro
0f4e256935f317d7d978c241b50f20e5
64757957a5645cfa049ccd99aedcfb9a1fb9a80a
2647 F20110217_AACYLQ lan_j_Page_123.txt
de603584a182105036f05adf3e68333f
12be93adb2f884a85240c7218f8c2592c3493748
6084 F20110217_AACZOT lan_j_Page_092thm.jpg
74de7f31223daddfe1e8204e21cd34a3
e2362ee04c3d7500e6cf4022115fe876f836c1a3
22827 F20110217_AACYMF lan_j_Page_054.QC.jpg
7efc02b84bea4d46b4ce3da87ed6904a
743156611833c675beae7a6b6d3e97a222764ba1
23414 F20110217_AACYLR lan_j_Page_070.QC.jpg
74b679507d8cf709a86d77581a3314b7
8b86d31daac30f16f727b87dd783c08fbf146ae7
5837 F20110217_AACZOU lan_j_Page_093thm.jpg
593c2ba2cf7c4e94073b36d67bb5ed03
d32f9f144306c052a685f9e7b7a15a06c87c6f1e
1051878 F20110217_AACYMG lan_j_Page_122.jp2
a609008ae8aa4f7891ac9c5192ebf9fc
a64c959f557d67a18dfc27a737869c76ef4305b0
1051983 F20110217_AACYLS lan_j_Page_121.jp2
f631c1ee09d54ff1a32b75705574eec2
06ce8b544c6af55ce9e4d0920e365211bb331420
7692 F20110217_AACZOV lan_j_Page_099thm.jpg
356df4cace4d4050cceb79c5694d9ff6
f7d213729f8a417cb9dfe90bf5de410bdcf6687c
6150 F20110217_AACYMH lan_j_Page_010thm.jpg
b2bc869a3d7982df93c79e5c20f13f74
0aac256359769c9495f4dc1bcc3aa95294fc9948
2001 F20110217_AACYLT lan_j_Page_019.txt
f1f198c3a4f94d91b212dd3304ee1c01
5415d1a093c1cbc0171d2bcc2308b0e44742b3bf
7008 F20110217_AACZOW lan_j_Page_100thm.jpg
eb891b8bdd70d33079cb22938455dcaa
955e6e1d56eaf55b37acc090a845f3a70b99628e
3830 F20110217_AACYMI lan_j_Page_011.QC.jpg
747822f328af07b6e67eea5d55b3198f
455f14cb18674edefe5cd2010bde8a12d026cd4b
162625 F20110217_AACYLU lan_j_Page_120.jp2
2e8b4d91a5346c9ca73d8cc787b9e355
2628f67439e0d84c730c0119631dbea2d6763947
7913 F20110217_AACZOX lan_j_Page_101thm.jpg
2bf3fdbc8a73dab576207934008de5bd
f4a5c1c446eb87c6c6129ce72b5e7696fcc8c799
12197 F20110217_AACYLV lan_j_Page_003.jp2
b26a253946493f649e7142e643ebf8ad
50b32dec8bbab3df5fccea6f2bb76fd32996bdc1
710 F20110217_AACYMJ lan_j_Page_128.txt
4dd5fc6bbee2d714e7cba31aafabae2f
949019ef18a5ce62baa4b863c312eacf8c8b3483
6782 F20110217_AACZOY lan_j_Page_102thm.jpg
73d61774bfa62f0af7ff536cb58152cd
a5e70828fa80910c5ad7150bad929b93e2a232ec
79028 F20110217_AACYLW lan_j_Page_009.jpg
8676dff006ae8fb0ef7d4eb200c49097
7ea0b4b43bd1031ef2c10b88ed0a81e6ef0f3542
987 F20110217_AACYMK lan_j_Page_002.pro
30855a35ee1dba2acff7581c30610b99
2a2e080eb82907500e2d53ca5ce528930781e498
5226 F20110217_AACZOZ lan_j_Page_105thm.jpg
ed461d908eff5ea57d2fd6e9cb355ca1
85e9bcf7f137c1bea3da97479f20f62703c56325
7736 F20110217_AACYLX lan_j_Page_024thm.jpg
5bc0f3e612a84196009487fdc23ebb20
3895eabe953d4c3483a463a936c16175933085d9
F20110217_AACYNA lan_j_Page_084.tif
bee2ec0db16a781f06bd17057aecc9b0
19b97503b44f127d5c568c32db6e0136ca622dc3
1469 F20110217_AACYML lan_j_Page_006.txt
4504e6686260af109176f112b7a60ff2
bddf3eabce2c3ea32fd05be20d269f3c527bd893
4821 F20110217_AACYLY lan_j_Page_039thm.jpg
f0a8310c7940bed68d6742a4d8b76733
2f7cbcaa77d4f4d01db756e50ed65c774ebb2d95
118665 F20110217_AACYNB lan_j_Page_126.jpg
5c8ec1e4424993940bdb106708f5b83b
2a607d4c5712931c033a8d6952401c40263bc9bb
F20110217_AACYMM lan_j_Page_066.tif
4a44c726d80c2bdb7047d227e99694b9
b147bae251716c8fd6a1e22129353f1ae3a13e36
5322 F20110217_AACYLZ lan_j_Page_058thm.jpg
449e4e3b1606849bc7afa91b6f16139e
d9d578cdd2db989886481b3ba0794ef7a458df30
6707 F20110217_AACYNC lan_j_Page_032thm.jpg
a8a46b8f7e217057af095360733d9c90
6c3e05c86872e7c5f24da0379bcd5d2a95869b32
856240 F20110217_AACYMN lan_j_Page_009.jp2
b1b129e6e8df706a80b82ec8411d1756
a259fdeeb827c4bb0e0232c88d521897f7565f62
171 F20110217_AACYND lan_j_Page_011.txt
fa6f3464479209831568c9a4a1fbd869
81c787150fd26353b90fe8044f82dd0885d419e9
660232 F20110217_AACYMO lan_j_Page_107.jp2
5d34b812d67232eb39da7f6e8ce80643
623445794954e7f6c2315f5a11628bbc00eac7e1
92959 F20110217_AACYMP lan_j_Page_049.jpg
447a4ceb955d635d0da8bcdfe782c6a2
e7673a77e329e72a5e1a3c3f675170a3c59b2e33
896 F20110217_AACYNE lan_j_Page_028.txt
a872b08ee2cd19b9614963e40a47b4af
d7d0a545d439da034c3347c02f2c321d8cc764a6
5956 F20110217_AACYMQ lan_j_Page_054thm.jpg
720a23808bb690137118b4170c65c045
69135169ba2343b7ecb851d5858f0dc8b53915a1
8066 F20110217_AACYNF lan_j_Page_114.jpg
16c72fc7839ef101c98d3330b8aa1a6f
3a0d7511797626f9d648d07991d6ce0211f9fdce
29582 F20110217_AACYMR lan_j_Page_075.pro
da39420831e5e9d5c0bd8105c5731115
9c6cc3be5aef995606868f77bc0be17ba7e611fa
922092 F20110217_AACYNG lan_j_Page_102.jp2
bae5d93817af15c17ca26feb0f740ded
72b70117eacff12a7f82ec772faa78b728839d8e
6310 F20110217_AACYMS lan_j_Page_023thm.jpg
5ba55995f67787eea371956d4cc3cfae
2943571f1df3929c1ed696c906b7b7df58e9e6b9
28645 F20110217_AACYNH lan_j_Page_083.QC.jpg
dc48d6f5d85aaeadd21924f081e0d746
7f77df224edb176d76f2b02d1f8ba09bd7556589
33350 F20110217_AACYMT lan_j_Page_122.QC.jpg
f99543c51fdbe5e1925f5c871b10980d
74b048b2b428e16e3e8f8c18c8ca70716a75cded
6871 F20110217_AACYNI lan_j_Page_040thm.jpg
58b4f24b54f110327f54a894634558b6
56ef98ebca539feae7f0aad87ea554640f093c89
561570 F20110217_AACYMU lan_j_Page_065.jp2
69f699a1eff32c2dbced27bf89652340
84b1e41f6fca32da8f83bb4e6822efc900020893
F20110217_AACYNJ lan_j_Page_042.tif
e00c3e69b26fe182fcbdc3f3588388b8
6dffdf917f9be158defa7216e52475fb15995cd5
46312 F20110217_AACYMV lan_j_Page_026.pro
8bb6930bc5c6741d50a5505ab30d6ab6
26b5e8d2ab071c0774695064c70f46488efbc1d5
30149 F20110217_AACYNK lan_j_Page_101.QC.jpg
fdaa9b5273c25f64262aeca19b1eb40a
9594c6b4c3f207a447cf3e303b26ca1518a5d543
4583 F20110217_AACYMW lan_j_Page_004thm.jpg
b401651d0941a62b259f86373858956e
2e2ca56f78bfa0e24539e9ec5b4f238257a5dae3
4373 F20110217_AACYOA lan_j_Page_045thm.jpg
ce4c9ac31d7efa16651ea4d4ff846c41
9d83876cc9511fb7f406446f727e87e05779dbf2
1914 F20110217_AACYNL lan_j_Page_001thm.jpg
9c0386ee74d0cc9641def4df7e0163f1
423c4d36bdbde20ead3724a91620aaf32fd4e7fe
F20110217_AACYMX lan_j_Page_040.tif
601727d4a34d3916e6f6506286b9aa78
86a7101bdf2331db0e1758807ab0ea3d8f87f939
F20110217_AACYOB lan_j_Page_094.tif
275e58683f4a04942fc447ab2941194d
60cf3e3e5319122b93d279fae3fb149b1f53ac4b
47196 F20110217_AACYNM lan_j_Page_033.pro
3c025553c26c4271d7f34958bf535077
7ed8afbcc378b35bb3a8354e4ab1c3753cb3908f
828923 F20110217_AACYMY lan_j_Page_103.jp2
53b4bdd3e386c4a26273bdb562d5cb43
eed325602079ecea32471560797fca7a051aa67e
F20110217_AACYOC lan_j_Page_019.tif
0c6935feaebfb5a600f515b4b10a74d8
c1fd91b126906d93cc2790fad13054c5b34957ab
F20110217_AACYNN lan_j_Page_047.tif
dda28715ba94bdf3e62c7351f6a6a09a
f04515e7bfc391cb9bb54443e604a77304ad8195
1884 F20110217_AACYMZ lan_j_Page_083.txt
5fd3c1d05c209147e0b8332ee82a9ed2
21a7527f1e6e51d18c791aa9c290728a7f64b2a5
4869 F20110217_AACYOD lan_j_Page_052thm.jpg
a0dbd524e33c34b9e5d06e832a442775
28beb21049e642395789b72ab843be02e7a98b3f
26074 F20110217_AACYNO lan_j_Page_043.pro
0c76d406232e7ef44d0450f49a47b086
b89b66d9c7fe4739388cad439506d74ec1afc920
6581 F20110217_AACYOE lan_j_Page_115thm.jpg
2d4e2acffcd8259c41f128a2bf0c7be2
c868e5950aa67921568c52dd518d2c958632ae86
746749 F20110217_AACYNP lan_j_Page_073.jp2
ab9e85066d1bc844f9a470be3d025d7a
587b798502015dd8bf4f7380f69c471cba331ae5
19365 F20110217_AACYNQ lan_j_Page_037.pro
0f54a90df80d87091fe76f7b79063af8
7e047d52b12253b937a61dfa34018c539b44ada4
76671 F20110217_AACYOF lan_j_Page_113.jpg
8f9a297af474eb8b627266e2fce41ce8
259202081c706f814bbf0201ac792584e396d2f8
27433 F20110217_AACYNR lan_j_Page_072.QC.jpg
7a89c4f625595c0a2b1eb6f9c6f4845a
65a08ec66ddbb8920a5ca925102573cd5bd49431
2051 F20110217_AACYOG lan_j_Page_013.txt
fa4c64ae12f7c33dfbae1715a057de83
f0d0122c92fb29849797add4d3752fae2058db12
23506 F20110217_AACYNS lan_j_Page_067.pro
267b5bef3438138003d2ef1c1b202e3c
843c31b3a7cb8640e4470df922658eb06e439d1a
F20110217_AACYOH lan_j_Page_029.tif
99c41e6097e326ddd3feb18195d0d096
b7934e729d1face40ace65e85f35be46fea2b595
1110 F20110217_AACYNT lan_j_Page_042.txt
f880ae680c2cf52a4e73dfcb063d5c64
34c7f5367e9b7b8c4ffb4a7fa4a45c614e6242d0
34985 F20110217_AACYOI lan_j_Page_101.pro
d5f9f79a1cae2e272b635bb565e93d53
fe01a24ce8e8fbfba74bc7866ecdefdaf73b31b6
24082 F20110217_AACYNU lan_j_Page_118.QC.jpg
6c7dbec5121972baa2a88800f81186b3
d526d9d640fb05d1b2a5ed193340ea16f11cdd2b
F20110217_AACYOJ lan_j_Page_125.tif
9e3c69d35c6411f1a5d2dd8f4bbe5888
58a2be890bf1d0c3bfa2515079c441a4fcde011a
6721 F20110217_AACYNV lan_j_Page_086thm.jpg
d00d60f55d4578c2319dad0675a5d492
32e8fb917b1b87fd13a781d1aa8309c7672b43a0
95913 F20110217_AACYOK lan_j_Page_096.jpg
0fdc92d5c7839e099432146cafab00ed
fe3b769252884abd98a49debc7346315ea27a08f
967685 F20110217_AACYNW lan_j_Page_100.jp2
f4c3feabb0d5db298c1997555e82c622
c1256e3e2d3bf2d633eb323241f7e6f816187fd0
F20110217_AACYOL lan_j_Page_068.tif
4a8e4100767bac04e3777dd07082bda9
c01cb3a541cbdcf55f0544a308cedf7bd7213c47
1051978 F20110217_AACYNX lan_j_Page_090.jp2
52131bb3fd9fa60b93e1f41cf7bf6867
8c975cf4a97ecaa2362c21e8b5ae2f76c54f3523
1047 F20110217_AACYPA lan_j_Page_011thm.jpg
8bd25876466c0c9eeed590edc47a9199
181f095825832297a4ce8d84ed338d56980023de
42843 F20110217_AACYOM lan_j_Page_111.pro
e60e1cb8ef051606cda7d4d77a7d6134
eb8642d9602b1d71182a00e653de5ba1da373ab4
1166 F20110217_AACYNY lan_j_Page_004.txt
a43df219f83199f3c96f1a04de5737f2
e724d3caaed7ca15c052c436b25bc640e8e38425
53234 F20110217_AACYPB lan_j_Page_037.jpg
2de8995cf1422dae1d3817d6930412eb
8a8bc83796acf401136e4c1fc01d624da1eab902
1863 F20110217_AACYON lan_j_Page_038.txt
7652b33a13413af1b30ef52e5eac46ec
f6d41be504f8a721a5ddd105b051e92d0e82a153
F20110217_AACYNZ lan_j_Page_011.tif
d0f00c9817f6f6443f227f2686d5547e
69d2209dd556e4368183398c92af66c25c472b08
800870 F20110217_AACYPC lan_j_Page_028.jp2
6a1205e8900d5c11768c8da1d6ea3eb9
08c34573340cfd1ac52ef0edf5cd7cfd4bc2959d
28870 F20110217_AACYOO lan_j_Page_060.pro
8815983e78a531b32bf0bd8c5cd7ae92
b22ca81a1310f7fb0e93c0adc8b1deaf20f72fd1
78882 F20110217_AACYPD lan_j_Page_075.jpg
f0088545878d85f7971c2cabfa8b16f0
d064bb509a34c4439978ddc5ae5bb60a11351ba3
60571 F20110217_AACYOP lan_j_Page_124.pro
44e9ae59aa09a92bc711e410d493ba89
4ff27764c17627f3f20b818231ea4c65122dd58e
F20110217_AACYPE lan_j_Page_015.jp2
e5660c8015cdd7fbab6660489838783a
4ca4b1f6311a729a1e38b95210f19689d9c10492
78026 F20110217_AACYOQ lan_j_Page_023.jpg
26edb9933514b202084d1ed0bce9981f
f7a4d12d1637bde6eefaa4873f79c481a41982e6
22126 F20110217_AACYPF lan_j_Page_002.jp2
b8148d3743bdbed3b3f040929b8b2df1
ea2011432e3a69aa2391b60d15a854df460a5f22
24890 F20110217_AACYOR lan_j_Page_048.QC.jpg
6ab38557e681eebf8312ba84e5b0ff65
5cdb4467f5fd6a89ae7a6aa1a75410538a11331e
1191 F20110217_AACYOS lan_j_Page_067.txt
138226ed159fec4d7f8a08e1162bcb40
083fb4d92d3fa02cb7d0e6b6c4241d8880b21950
20644 F20110217_AACYPG lan_j_Page_064.QC.jpg
310f87d22a759859e0fab10774e309aa
d534f6cd50485062a9e72d51b1da57b4428bc20d
F20110217_AACYOT lan_j_Page_119.tif
e4a44532b6460cca600d23373c9da400
8e3a634ba8d231657ca3aef8f64ff395cd0904f8
6037 F20110217_AACYPH lan_j_Page_021thm.jpg
ec34e3d33361c6608c38f56906179011
754e81030caab7445f7b7cd04dfd697a7121a44e
822942 F20110217_AACYOU lan_j_Page_056.jp2
6eb9bc41328ac3b46cb6c4d4b7ff379c
58a3aaa038e10e3128075c2a6d157d761baa0ac9
F20110217_AACYPI lan_j_Page_124.tif
46ae41452ef3d1e274ccc2ac2b1db0b3
8449b575d527b9209a1ef6db3dd57740f76dcda6
6380 F20110217_AACYOV lan_j_Page_028thm.jpg
4a96987a403770a729f99fa40f914988
c70ec2a68494bba8606f611a3d140d29c6e52eff
27650 F20110217_AACYPJ lan_j_Page_031.QC.jpg
29b819c120540e00982b82a1f11d5b52
da0cee0715b18e89373c900ed7e1b262fc14a125
7433 F20110217_AACYOW lan_j_Page_015thm.jpg
d69a559e444c0f05c906054a08d98eba
c438c55856860039ff24bb25aa753b86b62f88b1
19547 F20110217_AACYPK lan_j_Page_043.QC.jpg
ae9d9d7fdb2fedb6b7f48b8b955ec55f
e433aaef6c359fdba62720fe2ae9fbb1a93bbc68
1628 F20110217_AACYOX lan_j_Page_075.txt
0f26c89348ab1b5cdddffd944e99355f
f93f50cef1214f8fe2c1f82f623ecdc00d724385
1291 F20110217_AACYQA lan_j_Page_046.txt
70fa6622a8bb5c53d63bfe483bcdec2f
c6e4478b0da7ce00be730c99ff9b482d6d3ca5b7
7615 F20110217_AACYPL lan_j_Page_096thm.jpg
56dde434df1d374d7c10cedbe665ceb8
c3adc7b35ad5173ffb4d7282dbe69e4fe6578913
712211 F20110217_AACYOY lan_j_Page_071.jp2
06d40c65ce503ddd8f8b88c87d68c6b3
09e293bfcfe9ab2392e42cd3d231fd96c25539ce
61689 F20110217_AACYQB lan_j_Page_043.jpg
2cd6db46230c73ca27145e765a0936aa
065e5066512f19ff2e19313557782889827cbd5c
20011 F20110217_AACYPM lan_j_Page_020.QC.jpg
fd78027c238dbdffad441ea03d3efe39
1dcb583ca7fec93e19d4ff0d4538f2f0991b50dc
37972 F20110217_AACYOZ lan_j_Page_049.pro
3384ddab5502657369798d7fef5ec5f0
a53a72774b91722d00bfdada26ad8690e75084e7
F20110217_AACYQC lan_j_Page_025.tif
3d8111bc8515fbe2de34d5f254561113
15837ff4d4cb4b1c0a9629f0b1919edbef3c4b8e
64287 F20110217_AACYPN lan_j_Page_020.jpg
4883a1e95d63c6030b0c0f7ff475f112
bcd30120cf70194eb1137b0d7e9ad3cb00465aa9
1051963 F20110217_AACYQD lan_j_Page_019.jp2
ff5f636b43c5977afb36bd51a2028307
1f2e711718a3386f655e9d95d6f419ae82251619
5993 F20110217_AACYPO lan_j_Page_051thm.jpg
23e098e04578edffc219aaee2e95820b
64a07d495ece32aa752cb4bd934dc0a946bea4f5
112720 F20110217_AACYQE lan_j_Page_125.jpg
6112ef6668de2bd4ea5b547ff3af3c9e
313910c45d44a41854b78cced3be5cff0b5928e9
36732 F20110217_AACYPP lan_j_Page_021.pro
cfd731194e2b9850c26e663535eec462
d64798acf5687c8190aa113345c97ee49b6958d0
1731 F20110217_AACYQF lan_j_Page_094.txt
f3c917970f8230cbaaa63a4bf31936ff
09db00b399f52ea49716e22e488336ee2568d354
906 F20110217_AACYPQ lan_j_Page_003.QC.jpg
bb8a3e730639a72a115378356ab8c338
0ca046c38902f00139d16cdf02afd00af3fb58a1
F20110217_AACYQG lan_j_Page_027.tif
c2154fa7ea734e834970b557028b95c0
9b9e7ce011bd9ee75d450117b6fd1a0f6dae825a
33896 F20110217_AACYPR lan_j_Page_034.jpg
4d7a14c2db5c4b9a80a6962a8db84d7f
fc68f60708761b13fcabec384736b02da570b179
61117 F20110217_AACYPS lan_j_Page_089.jpg
2f67e7265182e379290e26c82dd07941
f7cfc1ca968706ac45800fbb3320b940f82df636
F20110217_AACYQH lan_j_Page_087.tif
9ed86a7cfd9dd8652c01844daa371b4a
f20320496d9706037c9baf1c382b254b148b4d10
F20110217_AACYPT lan_j_Page_034.tif
562821d758ed2ee9969016db33897eb8
f83838be4b079817d6e9ad4260eed8c9c42a6d27
7705 F20110217_AACYQI lan_j_Page_013thm.jpg
74837b79d6c1186ad51a9dd11097f9d8
32b84b5628549211aa964e4938f3917e3ea08ef8
5548 F20110217_AACYPU lan_j_Page_089thm.jpg
1f8c7eb5fd813d9596f7cfafe94f017d
f7579777827dbbd5cb006f041ac174930ec32e2d
940211 F20110217_AACYQJ lan_j_Page_088.jp2
aa812acbefd8df38320582169ea05d3d
85e8de2c9b460c74055ec983c4aa65ee708c38ad
F20110217_AACYPV lan_j_Page_075.tif
146a4ab54f367e3884490e9cfe7df2e0
26006308a8b66138577bd2fc5f009aa783876b53
27013 F20110217_AACYQK lan_j_Page_070.pro
ecfc11ba822e501169ac7de8b11efca3
19ab59adb5bca6967b098e27db49362400a78ab5
83665 F20110217_AACYPW lan_j_Page_111.jpg
b0683920834dbaef577f4db0d807dba1
ed173b074dd7561431e9bfc06cdd5186a2f4c111
904853 F20110217_AACYRA lan_j_Page_041.jp2
d54a56800b61764cd1bb6f7aa2733012
e0cc14dba7c1e5f1c57bf3d7160c2d1e3b8f07cd
19094 F20110217_AACYQL lan_j_Page_060.QC.jpg
04d01b552e424be04b4a853c14af1306
bd79a57cf1a57605917d5818f69e9d7eb0e43e75
1998 F20110217_AACYPX lan_j_Page_024.txt
9361b7d44905a5615406a5e3609c02ac
5cbdf771166b4965e20e5f4050eba7e87f0cabfe
89631 F20110217_AACYRB lan_j_Page_036.jpg
b552f3361d96e7a62b64d478275ea217
2d2e98b15e3de79e0e4bf6ebbbf0e04e415c125c
1267 F20110217_AACYQM lan_j_Page_008.txt
705b3a62289440af58a9ba6b8a0585c5
32edb9be894f48603858fb3e43174fdb85dd3c3f
F20110217_AACYPY lan_j_Page_080.tif
5fac3acb3ebc458eac92a131f5e49ceb
87d4635fa2c84ae758972938d7866a05b86e32f5
F20110217_AACYRC lan_j_Page_121.tif
afabaac3dcf48eaee757530c9458c9cc
b5b25b08b0604a477b4a0d1bdbff709fe4e1a6d1
F20110217_AACYQN lan_j_Page_004.tif
e63ae23cd7c665659bce77cbea47496d
7c0fb6a73a4d2c92800eb1e1874b8b2442b3d22e
80505 F20110217_AACYPZ lan_j_Page_030.jpg
0d673c7c0f9c327b71ea455c08ef89db
f1fd561b31e74b69645adb0ad7858dfca2cca084
771595 F20110217_AACYRD lan_j_Page_008.jp2
2b8a490e0873ba7c72ee6ebcc9e181fb
030fd2154a03cf4b0744aeb2982d36a7dffd381d
1051929 F20110217_AACYQO lan_j_Page_024.jp2
52deccc9fee5ae03a6fb554773d55630
e0e20fb57e131e7b7c23ad1f386cdc76c2c43f9e
30848 F20110217_AACYRE lan_j_Page_099.QC.jpg
1683e387057bb87916b04fe4004e861f
2d58313405455f6ad8b08b2a5e632d7351b0c5b0
74592 F20110217_AACYQP lan_j_Page_059.jpg
82d52edca21bf6cbb640131fe6ac1533
f21fb1681fa7a6ac4a91917c98e1a2daa415aeae
F20110217_AACYRF lan_j_Page_106.tif
ab1d9a5be7ab113bd549bb48aa06f1ca
76b359080626ff1c044a51777a00a2d691e66d25
1829 F20110217_AACYQQ lan_j_Page_026.txt
42b97da702fc8339aedd5cb283929dc7
c0eab9725a7e973b30eda269fd43250cff165200
44151 F20110217_AACYRG lan_j_Page_036.pro
219454afbbb1f070b267edbaf8aaebb0
34735b50383839101b4cc3eb7cd9caddf180de34
F20110217_AACYQR lan_j_Page_127.tif
2ae6f08b70217111daf6e8bc0854944d
5e4fb1425bb2393c961d81152ea75de521264ff5
742 F20110217_AACYQS lan_j_Page_089.txt
59b32682afb178a7501afdd9b7d6138f
4258bb89d12937276bc07b2cff51136a26137227
F20110217_AACYRH lan_j_Page_009.tif
5bdbbe4b814a02d1c975b568ee3729e5
f6151dd49214407dd11025428640a5836374b333
F20110217_AACYQT lan_j_Page_050.jp2
b7d1964ea345ae846d50ae39c90dac76
0503c95564aef6f640bb491179a2dc3528bf8396
5408 F20110217_AACYQU lan_j_Page_009thm.jpg
37827bb303fc71b07b2116a3a582c7b0
401fb4a0fe31d794ae1976efa2a97f5f40c95c81
1671 F20110217_AACYRI lan_j_Page_023.txt
c7a6a1b51a43f4f3b06dcf50ddc1f774
f10ed23f5c136855b31580f1aa3426678b8eda92
42203 F20110217_AACYQV lan_j_Page_022.pro
57444213950b9af685610111741b5fce
6676d62c3ee0cf9c5e630160d44ea33782b52c3c
423145 F20110217_AACYRJ lan_j_Page_042.jp2
48b226df1a16e103db1e5c73660f7ae7
fe5c8bd2c70a32622e37daf3532b93c13609c8d2
992472 F20110217_AACYQW lan_j_Page_035.jp2
4dfa7ac661b6279726c5c5c2cbda48c9
401c7948bf3d0653ab485a4e765a767858c1384c
908592 F20110217_AACYRK lan_j_Page_094.jp2
bce6da1f54276b1c5baa3fe677052764
3c61889b686b95e2e2a016fe8c87dc1283621c2d
6830 F20110217_AACYQX lan_j_Page_030thm.jpg
4e29e3dcc2aff9de136b90bee9b7d063
5633b6bc11ea2fa3b3fc1d656d7ccd56d98840c4
95886 F20110217_AACYSA lan_j_Page_011.jp2
bc54edf8c25cc68e6ff1bab21204f901
be831b9cf2c20cb4627feee611d14e34cd5f15fe
F20110217_AACYRL lan_j_Page_010.tif
c2d67c7b19b54e6c45d67c021adc3177
b0bd10dc4f26976ccac9387c5bab80775808dd79
F20110217_AACYQY lan_j_Page_116.tif
c7addd9565fe99648968afe2882ec98a
90d914a6fd3029b234d3bd720dfe422b0c275f6e
1407 F20110217_AACYSB lan_j_Page_077.txt
5acfbaab8f44961b65922ce167b9bb0b
cc3fc26921c3d778654fb4d9c21134a190cf5dc7
25604 F20110217_AACYRM lan_j_Page_086.QC.jpg
02840fda66b02e0a6a278dbfcfb08e43
9f8e70f23a585fc9b4daba0fb017c7fb5bd83b31
4822 F20110217_AACYQZ lan_j_Page_065thm.jpg
a030ec30e02d92441e5ec0544afbc5c7
6ba92f0076dd014c6e1ef68b2aabaf413a25edc2
1107 F20110217_AACYSC lan_j_Page_068.txt
7e7dec140ba42e44010d0427f700cd67
201786606f9379145e5273a7bf88a32876593c8a
F20110217_AACYRN lan_j_Page_031.tif
41f8b2e675ccbee8b9de8bca9983f981
719fcb7c6bc2fc944e2f8396fa4484d2d3996d7b
F20110217_AACYSD lan_j_Page_105.tif
67e1264455096983f8d50be03b5a5418
2466639e89937326126b094b833b53917eacd5ac
30432 F20110217_AACYRO lan_j_Page_087.pro
f19d11e9a1039fd7b0e0431ded719391
171ab89d7bafa169a529c7c46399616692476141
2875 F20110217_AACYSE lan_j_Page_128thm.jpg
baf6e30062b93866fe48dbb14257b74d
1b1c4d9c4c2589a16173470cd3c0761d7aee24dc
39059 F20110217_AACYRP lan_j_Page_041.pro
b0adea4f0622bd48c2e6cf711fb8ad75
d94c6752cd8501ad0e51a612edf3a3aa7c9abb43
6631 F20110217_AACYSF lan_j_Page_074thm.jpg
fc2e2a55ae2eb16532b44a74ba8b6c9c
b55ac53436bf889204596c8e18ee00810f973475
F20110217_AACYRQ lan_j_Page_082.tif
0f0951441834b8eafca6eb36abdf094a
70c6fc9c34a2bd0d69fb4c28ca98cdd2aff49863
76611 F20110217_AACYSG lan_j_Page_070.jpg
b80ade32aeac2fd399c2b1647f5a6588
7bac8b6e03f7c0f0b22c1703a3c96a9aa7fd5aa1
F20110217_AACYRR lan_j_Page_020.tif
c833a75790f3613c9c2e97343d1036ed
c520cec1777f88c43a6e9b09ff0cf465d1e8abfe
5346 F20110217_AACYSH lan_j_Page_062thm.jpg
2c91c7e3b9098c6970097155102c0a5f
cf3b2a8abfc3e05ccf0e1dce529f7afaabfb3ee3
1242 F20110217_AACYRS lan_j_Page_100.txt
f58b9d35f3f96521a10c51ebcef49659
d4c0cece7739cb46ef5bfc6296deba37567fa74d
7555 F20110217_AACYSI lan_j_Page_017thm.jpg
e10575d64713778e5555ce794f0a6643
195874b6f2b8b06bd18553df69fead357db1f311
2118 F20110217_AACYRT lan_j_Page_015.txt
1a7e445250aae17ceeac2c77152e2f87
47630e0044ec359a53dcc827c1f6313cb884f0b7
86761 F20110217_AACYRU lan_j_Page_095.jpg
1cf5678893fa953f297f999d38bb9c54
28b720e8cf9933a79d6bf42773ef604140307e2a
F20110217_AACYSJ lan_j_Page_045.tif
98bb9c2b6ce19d46530ab2b25ab84b1f
c5e0d0b89b9b8bccfd553a07fe1f325f36fef478
1715 F20110217_AACYRV lan_j_Page_113.txt
6eb5ff544302d41d5bca4207ab9ed206
42c02fedde26e2c8cbbe4ebf7c5e614d2829f622
F20110217_AACYSK lan_j_Page_056.tif
6dcee8ecc7a35e72981aa7c9e9d56b26
e0e38ea0d761ca0f05b5600ac4243820d2afd999
1214 F20110217_AACYRW lan_j_Page_002.QC.jpg
ba9617fa339d1207cebc7e025f41a37e
6f6431d0dfbaac49e32a0379cdf1253a48ebcdc9
752908 F20110217_AACYTA lan_j_Page_081.jp2
19575ab38918cfb7527eabdd86ad0541
3c7912fdb97c61dac8a53546e9064313b8cf333f
34505 F20110217_AACYSL lan_j_Page_117.pro
fc6a905bdf3da145802013ab40e1d609
82a22782949397929faba90757fc5de0dd360d7a
33136 F20110217_AACYRX lan_j_Page_053.pro
888f2036eef4e45020816cc1976c5f69
04fe5c0aa1f02db9ce4731874769b285df528823
74460 F20110217_AACYTB lan_j_Page_016.jpg
779098de1de816f1501998f26f5937a8
79a0299b483baa94f6b3c1e3ead1242c269b3fa0
7776 F20110217_AACYSM lan_j_Page_124thm.jpg
0fd1951430896d330d69bdc2cdae88a1
fb6ef0fb2cfdfc50808b0d027029c6b14bdd71d7
F20110217_AACYRY lan_j_Page_104.tif
fb21d4b565d30e5d0f474f1cf47a626b
24fc67ae6129e7cf7e15b8bf333ce1ac64418625
F20110217_AACYTC lan_j_Page_097.tif
08c49b351a9829becff052e7bf99d0c1
6e173885e04f3aa8f7f0deea0ebfae8d2eaf65b7
952284 F20110217_AACYSN lan_j_Page_078.jp2
2fa72fdb91ec59858ee9c8a63a8a3077
b2d919ba2d037bad738b8f82ac93ddede871aac5
1051919 F20110217_AACYRZ lan_j_Page_126.jp2
e6b6b9e689c13ff1272f64aee6c1cb0a
553b9d7f23dcde5aeb9e0efa805cdacb1c39ab05
75820 F20110217_AACYTD lan_j_Page_087.jpg
ebd46025f0ed3a0f94f052648dc00fa0
211056868de11e428d19761911e4e4f145dffec5
40270 F20110217_AACYSO lan_j_Page_086.pro
3fa710c70b1e3346d790babf39598ae6
4bad527a79e4d1563858188c2955e43165e1cf94
1528 F20110217_AACYTE lan_j_Page_076.txt
421f6f2d836c11fd4dfdfabbad25fc19
3f999c747179c8acb5fda70bb44fac95ed7932fd
768 F20110217_AACYSP lan_j_Page_104.txt
ccb4cadef6926f38412bc0c3d3337bbd
c7f343b7c92a7661da698a2a90885f180c9b471a
6893 F20110217_AACYTF lan_j_Page_111thm.jpg
4f7397b1b8bc7ed787eae00d2b695493
3a6a3c8d9a69ff6695659c2f99434cd47258aa61
F20110217_AACYSQ lan_j_Page_049.tif
c6b33022591a6e53f14c1ceff7d9c6d4
839279cf1e937ec461f52bf8173b3d79e81ff3de
101178 F20110217_AACYTG lan_j_Page_015.jpg
d6ee4ef01f6e5af28d69b74c9a5b9f40
c5b8e1d8262292ee1d62c45326911e94d4268093
1643 F20110217_AACYSR lan_j_Page_030.txt
cb88642076a7d01eacbbbbad969681ec
e865445c5ae30bb390e50a7bcdf5fd130f1a6e82
580016 F20110217_AACYTH lan_j_Page_105.jp2
88f6f93fc5ed3227d92da62ac0be2ab5
06e595391932a97c7cc8cca1ce84ac1abcb9d8b6
26571 F20110217_AACYSS lan_j_Page_040.QC.jpg
66c2ded1956959da105015cfc7a7c194
dd1922d7a6d21f200415a1ea42207cea1455176a
43807 F20110217_AACYTI lan_j_Page_012.pro
094713c1939eb701de9851ff63760079
0cd48563a18796993b9f02f591ca10a1e3fa29b5
28541 F20110217_AACYST lan_j_Page_054.pro
8fc49e6a8408137d0396e8f9dc38890a
194f0a5bf24a89752293d21aa8abe4018e4b3626
17779 F20110217_AACYTJ lan_j_Page_006.QC.jpg
674ce1bf1a4b779424788687b5891ca6
fb2d1dd43b268106aa322fdac2d64da9d4cfdfad
62888 F20110217_AACYSU lan_j_Page_122.pro
a29e145fb1c30faea1e245193ba98ab0
350084fb519c15ba47761f58e556686d6fb33735
786071 F20110217_AACYSV lan_j_Page_112.jp2
2a6387dd2a50b15a7e8943f9c90a589a
a8eb043f0517620b36a5f16b52fd671e9982b861
F20110217_AACYTK lan_j_Page_057.tif
0154253ee9a2e647683e13f9c22882d1
74da6265565163c1b4c753b3ae6912a60f9416d6
1750 F20110217_AACYSW lan_j_Page_032.txt
dd69ac225caa96f2c3e0ad7edc5ce151
3d55ece3c1a0d8e32cef1f3e5adf42855724bc80
1008730 F20110217_AACYTL lan_j_Page_101.jp2
716de8e9b2b5684700f10ee1f106f62f
a30bbd2ab1c5ea5f7be8f8df66af57b4bf823b79
542218 F20110217_AACYSX lan_j_Page_037.jp2
9cf76377cbb334df55e726187d3e221a
965deeba72585972e1b706f0039ff699abd4da03
F20110217_AACYUA lan_j_Page_037.tif
68d219e3eba87142f4506b64cca022e3
25bf1ca4c7cc3a5fecc2df26cab80dbf1acc9ab9
24906 F20110217_AACYTM lan_j_Page_094.QC.jpg
5ee3f59b6cd0eb819b42207a8a71c3f9
5795b7eca135d824246788688f5eecce0c79e0b8
116783 F20110217_AACYSY lan_j_Page_124.jpg
256c30b93d16c98973fd42a15da97610
e291e9a6a7a63dd228e4bcaf1402b28520266c00
22236 F20110217_AACYUB lan_j_Page_098.pro
a7c3da1f657290012566b8cbfdbf0bb1
7f48ac5357fb105a05f25d1b24fec58e38dbe725
46382 F20110217_AACYTN lan_j_Page_018.pro
399f66f621d5cc2920c8f9b695843417
d794f5bc7a27e3f3a0c93a9245ef1cafb9737d55
68558 F20110217_AACYSZ lan_j_Page_081.jpg
9e841239e28590e230dae134cdf78e17
44ccc2f12ef9af5242fa655e6ebf90f83ade8192
F20110217_AACYUC lan_j_Page_123.tif
10bde30c80079b87abb6c05508149fc8
d95ddd8c325b2a5a69647894b769b856268bbf0e
1154 F20110217_AACYTO lan_j_Page_079.txt
d40863066427bed3239b97933770ee0f
27677aa631878bf534f32b14bd950c82c4530253
6401 F20110217_AACYUD lan_j_Page_025thm.jpg
0bd94e6b7c1903b449703544f65b17f0
5fcbfc3377034ae67d445de6e782046ea5f5fc7b
1985 F20110217_AACYTP lan_j_Page_099.txt
0e00dad68410a0ff1a2854683bcb8396
602da7b126cc912659333e4375b94e77a894693d
67159 F20110217_AACYUE lan_j_Page_053.jpg
dcc60f6a004166d19b346766a1f4e16b
7f33a55a8b20f0649408547c48e702980e338b97
34340 F20110217_AACYTQ lan_j_Page_030.pro
2e1caf51a6e13ce330b61c35703a67f1
94765a9562d0d629de97a0ca6766617854887561
F20110217_AACYUF lan_j_Page_043.tif
8a04fe190d5fe4136c71fb4fa5312e8d
252ee6eb8651ad2712d8d8407a473e33f5383d4b
F20110217_AACYTR lan_j_Page_060.tif
75ca9efe41a45a935b0a1de924ff4c30
1f67f1edf6239ea3d5298efef0fc46a52d4e1a08
F20110217_AACZAA lan_j_Page_002.tif
93a593a3cd20a804bf8b9feae0939ba6
f2dca39bb56f4fb93efa9273934e89ec05f43e64
87766 F20110217_AACYUG lan_j_Page_072.jpg
6179781bfd60c2d32ffff939c8facd8d
24a65ea93729c6e5f70b99c1892f5f1a313799d6
579543 F20110217_AACYTS lan_j_Page_106.jp2
1d559c0287f736ac89e8eedd1c0a5e83
5eb562422fb2e4f5d8abfd82265d85c391f04669
F20110217_AACZAB lan_j_Page_003.tif
bd81266f6d53cdc28217857c52f9958b
58eadeb818119c326d9689b3e97cc32b4c46ca1a
31533 F20110217_AACYUH lan_j_Page_064.pro
2e0c81fdfcf12a0cdca4aaab1647964a
5eb989a697f1e972d270b8af865108fe7689385e
F20110217_AACYTT lan_j_Page_063.tif
f2b85309267eb1b362d7dbb26b77181e
76af1a6dc1d8fc94933899604371889a7360e7de
F20110217_AACZAC lan_j_Page_005.tif
08b150f59229c9764f4941536988375d
b997b0d804efa5e88a6b4556e5670d338633b1fc
68396 F20110217_AACYUI lan_j_Page_052.jpg
08349e99048711e6c657a5fcc0cf9b20
4146c38b264caab6ea746a40e78e2dfa973c6db6
F20110217_AACYTU lan_j_Page_024.tif
59f844a28305bec72b52634c3fcaf117
30a167ba725487b2841ac37d98aa41013f327543
F20110217_AACZAD lan_j_Page_007.tif
41b47a4d735678bb423813db1c79f9d4
c3748cc518aa7fd6f4748f16d861583c12925f68
6119 F20110217_AACYUJ lan_j_Page_103thm.jpg
d0c0698a8c9723b3f7b016bf306c498b
0dd19761fd57a6d9f3ac6b06827150d16f42fa11
5385 F20110217_AACYTV lan_j_Page_060thm.jpg
4a8394e1f37cb61190cfb0bf4327a842
8f9387e378c3ae66ffa6971f06a1ae7ccb104da9
F20110217_AACZAE lan_j_Page_008.tif
f9339b809589131f817ea05ba3c26aee
9f182e43c23d40739895f70833445ff6c462bdd3
F20110217_AACYUK lan_j_Page_089.tif
d36ca31fefcadeed022fd5f9fa0aff8d
98c6ae8193b59fea7b632fa87705d8e3320ee027
23639 F20110217_AACYTW lan_j_Page_110.pro
855765cce02e760d4f833f5626309c88
7d58b2a9a6c72002d0da80c497e6228d35a24dfc
F20110217_AACZAF lan_j_Page_012.tif
eb3034a63c28e2e8ce19e384a80f6b2a
7195e55baa803b8e48a6148ca32af87e8ca460ff
96452 F20110217_AACYTX lan_j_Page_014.jpg
0ae4303fdbcbefd64bea3bbc4fb0b3b7
e34e93f4ce9813c1b102d0418613adcb1df2e771
F20110217_AACZAG lan_j_Page_015.tif
7f27737bc006e44c7e327e9f0b2dc9c8
5727ac9b121cc8b15fa4f9879ed9e4f1f61bdc89
523 F20110217_AACYVA lan_j_Page_002thm.jpg
bb5ecac531da8613c10206e089736fe1
b951b0b5e9a58027b6ff74a22024ff34160c5f1d
82075 F20110217_AACYUL lan_j_Page_086.jpg
257a7b2c414451d455dd3e5eeebe1b3f
d8d9d9e7384d2b994d301e7274c77f69601fbdfc
19183 F20110217_AACYTY lan_j_Page_062.pro
40a45348f2c847435fadfdb2e3fe19e4
26f46cf852f90e4cddc373e52f4a6a5351b25688
F20110217_AACZAH lan_j_Page_016.tif
37bfd89e5b72ec03e36a297cf9dac0ee
64e7fd39cb883d5ed22be4a22062afa3448015fd
F20110217_AACYVB lan_j_Page_061.tif
54478a69b9d706c6968f3ca351850698
11cde34d54354b30e41b36f3cd952a92314d631b
5771 F20110217_AACYUM lan_j_Page_078thm.jpg
c8d1e8606e4fef9e0fdc35d68c89b73a
c1a12c8c0a9ce214daa4a9633c3c692d3b6d3e61
11030 F20110217_AACYTZ lan_j_Page_034.QC.jpg
efb5df27710c789e388f94588af370e0
c573ef918b560c3f080bc039d24c3c356df9f5dc
F20110217_AACZAI lan_j_Page_017.tif
13f825871cacdb7b580adbbec12e40b7
1c410b8f9bca31d6a473159f1614fc366a07f28b
631596 F20110217_AACYVC lan_j_Page_004.jp2
5c89dd78a4ac45dce57755d2b665dbdc
46822611534511379b0026775cf65648deda2094
29208 F20110217_AACYUN lan_j_Page_078.pro
34f8c62e6093c7414a719ed80413c3d4
a358ffae5a5f7abc5ef07892093d7d78c8a40847
F20110217_AACZAJ lan_j_Page_022.tif
f06f7fcc9b46fd2034797606b9856b64
18b797ff624a2ff9d38809399c1aae16f8ce0afc
7263 F20110217_AACYVD lan_j_Page_047thm.jpg
9083983707b43911fd272ef8a3596139
fd73fdf1a9661dc36f5c3140b31ab8a93318c0e6
F20110217_AACYUO lan_j_Page_055.tif
a9ba3a5e45db7c093090af4b1f9da08e
0910f272f29277a5ff11b06987ae10f93db88239
F20110217_AACZAK lan_j_Page_026.tif
4d1c5038f81badd049278d0f01daf7ad
740f821e1907caa7ddb147d7177c90c4f15b73f2
50266 F20110217_AACYVE lan_j_Page_099.pro
b6ce73858c06121551e77271c7b93b3d
e408e53b4943af60cb030e303ca110394f8081a9
F20110217_AACYUP lan_j_Page_033.tif
c8158c9628324d7e35eb1cccbff47989
1cc6d9c4935f28c98b41bbb8da5d11d18cf82e59
F20110217_AACZAL lan_j_Page_028.tif
8dcc5dc74ca18bf506660841525183ce
5a127d9e3ad3799b739d76e92ca275a66bc6ae58
37237 F20110217_AACYVF lan_j_Page_025.pro
ed3de0df4939379bfa45299aed1955c9
ca0d0ff3928e5413d10c755b93604fa829f6524f
1538 F20110217_AACYUQ lan_j_Page_025.txt
ad1a35bfb69502621c5e951a4d7f77cb
56068f4e54b1731f1a7f85db9b65261fc9b5309f
F20110217_AACZAM lan_j_Page_030.tif
77eb1031389363e7bae268bec281fb28
6f723656330ea31af4bb852594c9979038bd07af
6883 F20110217_AACYVG lan_j_Page_061thm.jpg
b5f4e30429209619cda9fcb1a13b3f01
6deb8a3144a7325dc8258e3c5c08075e88dea3be
5161 F20110217_AACYUR lan_j_Page_066thm.jpg
fb4fa96b9c7761f92c6575d28bfe5b24
11a1db3433e968f4d796bcf8b269e77713e20595
F20110217_AACZBA lan_j_Page_064.tif
caf4a20864d6a5d15714770082f94a4b
29c55db8a0d726e88d2e0e5592194af6bc0b1518
F20110217_AACZAN lan_j_Page_035.tif
0c9087d7156eb3311a1e54c7e132b424
bb998e0aab3e391a0fe410b813837d1481e7d68a
881509 F20110217_AACYVH lan_j_Page_023.jp2
1439c65489c017a9e08304f0c390e891
7167f872c4c2459b2c27f27715783eb9b7490d85
102262 F20110217_AACYUS lan_j_Page_013.jpg
12c5d8131151abf19bb061c0c8f665ec
4077daf3ce6772f3c656519a8f4df84daa5618f1
F20110217_AACZBB lan_j_Page_070.tif
2c885fe42b1029b7018cf910aa09aa4b
d82c802c76f30d83cc79ee4b05ffd758da1eec2e
F20110217_AACZAO lan_j_Page_036.tif
576f5df6709c9807fea3950aa632ec9d
62a7baa49b57b2e42dd95a1819d620901e0c2a2c
29738 F20110217_AACYVI lan_j_Page_027.QC.jpg
c3bf27711b5dd99f69c66239d332c56c
d363c79ff4144d2f4a0b71d141527b2d65d37ec3
798114 F20110217_AACYUT lan_j_Page_093.jp2
fee11a5fe8545f35b62cf98ee70f75a1
877554d6a3003763240fa52ad7701fa1f3fd266a
F20110217_AACZBC lan_j_Page_071.tif
4075bf75ec75978df36211c66f51f09f
488ff02a6007127e45b08bb33ad5c777d7a72355
F20110217_AACZAP lan_j_Page_041.tif
2690d77221aa0a24c1aa33fd91cbd1e4
bb00861849bc294308f272e60c24751787307872
17936 F20110217_AACYVJ lan_j_Page_105.QC.jpg
ce32c6cee942268fe17a01d650462af5
26d4931a806d1c809e1ea2673d815661286fd3b1
901596 F20110217_AACYUU lan_j_Page_055.jp2
7381f3d8b479e38f63d62e8b9cd87111
b2331b26443e00f5807263fb259ffa2aaec08eee
F20110217_AACZBD lan_j_Page_074.tif
f79fdfd491252f1e1b56ca9d0582aad4
9d012f60a9307854d0c63fc4aec73a000725d38f
F20110217_AACZAQ lan_j_Page_044.tif
a9e4f506487572234a7618a9cb31ed6a
442ec75086b0cdf7362084e14c92cc68b087b1d7
6381 F20110217_AACYVK lan_j_Page_112thm.jpg
7e91dc37747f3f2f9d1e4ee9e0876d23
7ea8b43e021e1c42bbeb98d2efb76bcd5cb2ac05
86142 F20110217_AACYUV lan_j_Page_102.jpg
4193fdf75569012bd5229d9c237062db
b8b3b7d9148f2d2fc372cf350838c7ca252c4738
F20110217_AACZBE lan_j_Page_078.tif
ba0942fb916cdb8970f9d95a791c4e0d
650fdee7ead71076f9ff15b43da607f8854b9e56
F20110217_AACZAR lan_j_Page_046.tif
ff8a0e37bfdbfc86c053940e5446651e
0430bca58711655a188fb5c74a605a2c96588559
1802 F20110217_AACYVL lan_j_Page_090.txt
6bf0a2e8e43276fa7a94a1f64f5d2fd6
e51db523bfd7adc2fb44a8ad748bed227e1b63e1
72748 F20110217_AACYUW lan_j_Page_117.jpg
005189c9d86e85286b5bfdfeb7000463
b96b1d6bffc1e4dc5a1f4bf4118dd8197795f156
F20110217_AACZBF lan_j_Page_081.tif
21f1c0a68466af0b3a788050817b989c
56a5d458125cabf7c2c6017ff9141799fd293bfa
1968 F20110217_AACYWA lan_j_Page_050.txt
4410a97574f8801baecc8aa7e621e2ea
e5848f6f67d7a5927faea79f879dfb3321437b77
F20110217_AACZAS lan_j_Page_048.tif
bdf75567a98381e3282a79a10d109c40
1da2b3d56df610f2c483dde879e742d255fa71ad
F20110217_AACYUX lan_j_Page_093.tif
f39acd738416eddf70f9911eac33970b
ecbca1fb212a78e04e8b66cd0790f3e4443326f0
F20110217_AACZBG lan_j_Page_083.tif
05a1defa6879a3c8992e76eef85aa9f4
a9fcc69c86f29f926072bf33b3348cc4d1d804aa
F20110217_AACYWB lan_j_Page_014.tif
36ad5efb59ee8da539d1c42c35b9816b
a7746cc589fe0a2cf250056982170ab5ecabfb4d
F20110217_AACZAT lan_j_Page_051.tif
26a7a594696eba114e1194f17f524e31
8302144890a2ac1cb833df2762371ad684192ade
F20110217_AACYVM lan_j_Page_067.tif
514ab912db50023ab262bc50e5e89163
36f42171780aa3fde7db75cced57f4e9b750c499
56594 F20110217_AACYUY lan_j_Page_068.jpg
74617601fb95f02634e67c29891d3782
e7b646349f8ad99cc404ba95b1fc0971f9fb7f86
F20110217_AACZBH lan_j_Page_085.tif
a32bfd630062635ae824b5576f9648d3
d5504c722ecd89eb705bd3472eafc9a8180999f4
96277 F20110217_AACYWC lan_j_Page_017.jpg
389257c53e6770449b73756e58c0e429
c710baec25a5013d3de9009c83056bfe46fa2f94
F20110217_AACZAU lan_j_Page_052.tif
af730ec91b61ae6bc1ff37f87edab798
7777ebe18340f7199c7cd47eafca1bff76c87b6c
857 F20110217_AACYVN lan_j_Page_107.txt
f3b656cd73743b1eba3a7a570c09828e
c29bbde5ef4dd88d1c7805b082f10d701dc2b9e4
1535 F20110217_AACYUZ lan_j_Page_103.txt
821c7ab7e7f6bbfd16ac91f9a0e82520
cef787affa7fe8e1112b859e00a8e75cf0223adb
F20110217_AACZBI lan_j_Page_086.tif
c4da5c3679f234744ff75009a53005ad
80f5d3b0fbcdd2a9c502f918ecaf288c796f8c6d
98876 F20110217_AACYWD lan_j_Page_050.jpg
589c1bc007d62ac7c29d2e0e79792094
6325d98d4b895d43aa6943aa66c596d7bbbf4565
F20110217_AACZAV lan_j_Page_053.tif
24b853731d76f5c3611918050fe5b2ba
7afd4a90dc036c360b0e962e3ac865fae7592597
1236 F20110217_AACYVO lan_j_Page_037.txt
3a55c44b33fcd507a8b03f8c415a6044
625f1a16c67abe097fd29149a672ad883ba16b16
F20110217_AACZBJ lan_j_Page_088.tif
3268ced12a239ec278f08cfcc84c409e
c3a698aaa1d0247112322b9207489f5c127e15a2
704092 F20110217_AACYVP lan_j_Page_057.jp2
3077751fa052c97411fdeac96ecee900
209770231f6366c0f4796797cd7bcb5571f6800a
F20110217_AACZBK lan_j_Page_095.tif
2815bc9ec5a727056cf673f5d4a0e87e
c5aaba1e056a42e8b31cd85601bd71d81bc62802
370612 F20110217_AACYWE lan_j_Page_128.jp2
e09cbe8e10b2e1f6635f1fe5e13906e1
634295884b02cf73d7a0454e22aea7bbc225499a
F20110217_AACZAW lan_j_Page_054.tif
4961a4bc56a01987d94dab326aed5e42
0bff367f2cb0900e199595d95f2c816e50946869
1012126 F20110217_AACYVQ lan_j_Page_047.jp2
a1979f9b1188819b7b9c4d25eee8f69c
c12bdaa39db9ca4b632081623e774d92e3d130b0
F20110217_AACZBL lan_j_Page_096.tif
67e9881bdb289807059b7667ecba74f2
f62dd98af704c18ea24269e6349057fa07a6321a
3525 F20110217_AACYWF lan_j_Page_006thm.jpg
157c3f4bca0cec17601e9fc8e7adc739
3d4dbb91416646dc4560faf3577fa56a51034e18
F20110217_AACZAX lan_j_Page_058.tif
2ebcf3c6fce904391cca0acccd2e6b32
8a2d02de4d1a662ed59af75dc1b677a342f9d2ce
30850 F20110217_AACYVR lan_j_Page_008.pro
60754873f718fc1813f75e612005a861
91f5b9ae391325658e7815ba9efcc51aa11ab9b1
1910 F20110217_AACZCA lan_j_Page_009.txt
19314a32b481913d04831f81bb474045
afceb10b3959cb975b5707da4229118786148e0c
F20110217_AACZBM lan_j_Page_100.tif
f221c858bd1827769ca85e8ce1024c8a
e473c1c26c947a27b11d1217a904d1af147d619c
1895 F20110217_AACYWG lan_j_Page_033.txt
630b1dfee4fb5da8ac6aee2e91fe86de
e95a4d951ce562942814e3a95e3788d21f25c670
F20110217_AACZAY lan_j_Page_059.tif
a4f7c421ccab66373fd54ab9989017b9
3043d8573fa473a3599fc23be1d694d8160da78c
23368 F20110217_AACYVS lan_j_Page_021.QC.jpg
e77136e3208eb4156e59a43a98e41863
0cfc9bc049652072351a740b2ebd2b2d54af6b5b
1949 F20110217_AACZCB lan_j_Page_014.txt
dc85ea3bb3ccfed4c62330f20c838ce5
be52616daeabb7d60f0770c87ffe3552beeb2602
F20110217_AACZBN lan_j_Page_101.tif
c15b850950bf322c4d251349366c7c5a
bce72aa8d18ddefd2c58a2c3b4905510baf5089a
740788 F20110217_AACYWH lan_j_Page_052.jp2
a9b9c2468785649d87c7580ee4c65bcb
16c96b2b574634e1781ed19e3d4cce970cc8e5a2
F20110217_AACZAZ lan_j_Page_062.tif
6c2635f2818bac7da3732824d47b5938
58cce222e37d00986671ca3b2dff371f594220c5
64977 F20110217_AACYVT lan_j_Page_064.jpg
e4bb84a4d4ed37f269c9f17f91f6ec61
01fb2b4145652fdc094f8ad86017a3bb5e22f835
1925 F20110217_AACZCC lan_j_Page_017.txt
8ce1db530cdb2624c99eb8bcd0be3bdd
265c8628a45353db79a331e44abcae309a8cf995
F20110217_AACZBO lan_j_Page_102.tif
b20a4c42b6be125f1fabaaec7935a05d
175d1851224ff31bfd3b7d4aab462027acd6d3e5
26194 F20110217_AACYWI lan_j_Page_055.QC.jpg
f9305d098c5193cad43160a1841235b3
ccafd1afb129dcb82ff04cf4cb1d93bbe36a7fe2
1527 F20110217_AACYVU lan_j_Page_081.txt
8e2df62efad63d808fe109c0d4a1f52c
3914e08d92fbcd3d73a10202c07514c8099a3f38
1836 F20110217_AACZCD lan_j_Page_018.txt
a44093059345f44aa4f450667e3c0527
f7bbff3f88a12b4ad90ecfc299e0c6a7eb0d4bd4
F20110217_AACZBP lan_j_Page_103.tif
3ea4b85c785a1b7e5584ecea1a0f2af1
edfae3a4c8206d058e649dbb4b98e6b3617a651d
43812 F20110217_AACYWJ lan_j_Page_005.pro
753187ec7a6ef43a7a05cd143f28b764
135c26d8b1090d51ad9fef34c0f0d70e5cb6e49a
F20110217_AACYVV lan_j_Page_038.tif
c7fb68b70f2ccaecfa11a100dbb25786
9b50a642c0a531892d1fa9988dd3f7c5de0fea64
1607 F20110217_AACZCE lan_j_Page_021.txt
9a6a9b304e376bc2046c5c734881eeda
effa582402a81af81ab74a00403c13df2976371e
F20110217_AACZBQ lan_j_Page_108.tif
d2722f9d9778172b3f208d60d5124171
d931e42767c9cfe4b08733512d43294b37b47a6c
34800 F20110217_AACYWK lan_j_Page_048.pro
6e5e0d40f0f756fe41e9663c193eed31
8e23a1d079bef50c7a2ce0dc842fc027ce932c5e
6023 F20110217_AACYVW lan_j_Page_084thm.jpg
e5b9647b4531a8ae65fc2ed1b2f561d2
aaff135f5def55336ba18af5c7b0d9937edd2247
1727 F20110217_AACZCF lan_j_Page_022.txt
9ae7ba49eae429a0f0afb7e27bceb66d
d71a9990e0896e3faf85e72dbad137d475d8d307
F20110217_AACZBR lan_j_Page_114.tif
2fa5fd8b3dbebf005477f3957e7685ff
4cde372f63dd28fe9a2673f0a36b0c995082b3fd
29751 F20110217_AACYWL lan_j_Page_018.QC.jpg
3fc04da72a52a642f747ae31a74fbaf3
02e650f3071d4357438adfd94bce082b0bad9446
1665 F20110217_AACYVX lan_j_Page_069.txt
62bd87290dc4f9c3c1d1ae3e1e9a19e9
826234103ab02e0b8f1e3e56c412acd771dfd568
1936 F20110217_AACZCG lan_j_Page_027.txt
a35252d826105732f0e5d8ab7aabfee2
0cd592a0aed1d1058db38415eb64a2c87e5bab12
6589 F20110217_AACYXA lan_j_Page_094thm.jpg
b0b7fa49ddec95c623ff6fdf6654fdad
8c93a2141108f8983356d3f0e7d44b1c23383137
F20110217_AACZBS lan_j_Page_115.tif
a2f2d317a78edd94915f3b14cf8710c8
729afddcef8136ba6db52b538826a0ce12dee614
117207 F20110217_AACYWM lan_j_Page_122.jpg
e9d6221cd6b0fa9f485678dd6fd85d8d
feb8770514a35549d37a8484600609e9732f65d2
30602 F20110217_AACYVY lan_j_Page_019.QC.jpg
cc4ac96c06227c95dcab647813a939d5
a5cc415ddec1aa29fb78b49227eb3a6dfb96ef4b
1553 F20110217_AACZCH lan_j_Page_029.txt
9e94459e9fa4bdc810912e72a1b82a00
1e53edeef7bf51c6dc79d3e2ef46de7f807ba397
5864 F20110217_AACYXB lan_j_Page_056thm.jpg
68e44886f5877713e32fcc72984af332
37c6ca0acd57686d307c96275c8a4c47988d7d8e
F20110217_AACZBT lan_j_Page_117.tif
a6fef68d6e0dbd2da186d1e3de4c7049
882b10007c0d25527fc3a69588238ad23775c064
35246 F20110217_AACYVZ lan_j_Page_093.pro
d49ddfcd57cb386d3c26885024a818e3
f92c8f234e9aadba7172089498dfb1abf02e32c3
1822 F20110217_AACZCI lan_j_Page_031.txt
57b879b6e49fd4dfee4a8a38c8338412
b4670452914631e77dd2cf3599af7574877c22a8
31867 F20110217_AACYXC lan_j_Page_015.QC.jpg
e2157233fc04f6eeb36eb1fecf2cafa7
ba89c19848b393dfb4dc693ee95fdbcc1081f8b9
F20110217_AACZBU lan_j_Page_118.tif
21d9c4ee0bf773a8c21dab0bcca7e747
3bb11619f4af6ea36a9d2ab769324aea0b10fc55
90770 F20110217_AACYWN lan_j_Page_090.jpg
e536aa9ea263b1539000511fe55783f7
aa77f0e8eef237a95c580da74701ff59c5675db4
1846 F20110217_AACZCJ lan_j_Page_036.txt
48e081d4eae9c830caca6ef34c1fd43b
e71986b5c60cb0a58aa6662c57146fa415bbceef
18138 F20110217_AACYXD lan_j_Page_028.pro
1f8dfea7e28dd51ed1fe1ef7a2e0b90d
6735d0d3672c957393b45da4218bc5736985d806
F20110217_AACZBV lan_j_Page_126.tif
1b12f9aa66fee90145cc1e953ad97f1f
f416ac6e8f6cbebb6cf575a072c9218284506839
1272 F20110217_AACYWO lan_j_Page_020.txt
8e0c52f123d1ca703b0e10a706465fb8
8168208644fba61e4a74c61362e7f58d3cf00fa8
1060 F20110217_AACZCK lan_j_Page_039.txt
d697e84f5b0af3a45747a1c396e1efaa
bc8a6235b2661dac9556ff3f96fb59aa0271d145
27191 F20110217_AACYXE lan_j_Page_012.QC.jpg
6d23963184b2e2a51c979c67b7aac7c6
df5432658e0e818029abfc40977398cf3e481216
F20110217_AACZBW lan_j_Page_128.tif
8f44d49d22808cc69a8898f34f79e73e
bf6c9f5bb9b32397606b5cbac1490115c47c8d4d
4763 F20110217_AACYWP lan_j_Page_104thm.jpg
6a54aa45830be5c5d4f20cb66651ae58
4fe442487c0987e1ed952078477649f975687651
1735 F20110217_AACZCL lan_j_Page_040.txt
e75433b7b4f110f201bed1fe74202a4a
0405466039bb80106ee4b19fd2d43b8a8f5eae57
91460 F20110217_AACYXF lan_j_Page_077.jpg
d6d1a89771e05fbff6013c364cdb6592
91e2d99cc4db58948a57478be4adc56f5946b340
59213 F20110217_AACYWQ lan_j_Page_006.jpg
b71ff921ba106dc42cc6a44e77905623
f9d50e113b43eaad854802c724f15d3480143adf
1828 F20110217_AACZDA lan_j_Page_061.txt
cda99aa18e3a650e85e6a410a0914a42
8d4f2a759fa8bef2bfc5344776f0b466762f02ed
1632 F20110217_AACZCM lan_j_Page_041.txt
958f2838021a6ac6af379cedf8fe6027
c5f421769d969e5ee3b20512b1718fb7f30dce62
45907 F20110217_AACYXG lan_j_Page_045.jpg
d36c5011ce19d5e23b3b4820ba31e758
d250edbe7d27f6e14a571138cef86037cfe6c1fe
441 F20110217_AACZBX lan_j_Page_001.txt
8875651004c20903626c82a387718379
85780b3d07571e9e56360e4c41e7fe41e2b453d1
24737 F20110217_AACYWR lan_j_Page_068.pro
a3e2c957fa2525cc060efa450fcfc5ca
6429d5eb6eb3efa2ea5f06957555889ebfcd3c0c



PAGE 4

Iwouldliketoexpressmysinceregratitudetomyadvisor,Dr.JoseC.Principe,forhiscontinuousguidance,support,andhelpthroughoutthepastveyearsofmyPh.D.study.Withouthisinvaluableencouragementandpatience,thisworkwouldhavebeenimpossible.HegavemetheprivilegetoworkintheComputationalNeuroEngineeringLabandallowedmethefreedomtoexploretheunknownworld,andwasalwaysattentiveandpertinentincriticalmoments.MydeepappreciationisalsoextendedtoDr.JohnG.Harris,Dr.JohnM.Shea,andDr.LocVu-Quocfortheirinterestsandparticipationonmysupervisorycommittee.MydeeprecognitiongoestoDr.DenizErdogmusandDr.MarkA.MotterfortheirhelpandinformativecommentsduringmystayinCNEL.IalsoowemanythankstoallmycolleaguesintheCNELgroupfortheirdiscussionofideasandfriendship.Mywife,YunZhu,hasmydeepestappreciationforhergreatloveanduncondi-tionalsupport.Finally,Iamgratefultomyparentsandmybrotherfortheirlove. iv

PAGE 5

page ACKNOWLEDGMENTS ............................. iv LISTOFTABLES ................................. vii LISTOFFIGURES ................................ viii ABSTRACT .................................... x CHAPTER 1INTRODUCTION .............................. 1 1.1MissionofSystemIdenticationandControl ............. 1 1.2LocalModelingwithImprovedGaussianMixtureModel ...... 5 1.3DissertationOutline .......................... 8 2NONLINEARDYNAMICALSYSTEMMODELING ........... 10 2.1NonlinearDynamicalSystems ..................... 10 2.2TimeEmbeddingStateSpaceReconstruction ............ 11 2.3GlobalDynamicalSystemModeling .................. 13 2.3.1PolynomialModeling ...................... 14 2.3.2EchoStateNetwork ....................... 15 2.4LocalDynamicModeling ........................ 20 2.5Summary ................................ 22 3GAUSSIANMIXTUREMODELSANDLOCALLINEARMODELS .. 24 3.1Introduction ............................... 24 3.2MaximumLikelihoodEstimationTraining{EMAlgorithm ..... 29 3.3EstimationofLocalLinearModel ................... 32 3.4ImprovementonGMM ......................... 36 3.5ComparisonoftheGMMwiththeSOM ............... 38 3.6Summary ................................ 41 4GAUSSIANMIXTUREMODELBASEDCONTROLSYSTEM ..... 42 4.1IntroductionofMixtureModelController ............... 43 4.2InverseErrorDynamicControllerUsingGMM ............ 46 4.3PIDController ............................. 50 4.4OptimalRobustController ....................... 52 v

PAGE 6

........... 55 4.4.2LLM-basedOptimalTrackingController ........... 59 4.5Summary ................................ 61 5APPLICATIONSANDDISCUSSIONS ................... 63 5.1ChaoticSystems ............................ 63 5.1.1TheLorenzSystem ....................... 64 5.1.2TheDungOscillatorSystem ................. 70 5.2SISOSystem .............................. 75 5.2.1LinearSISOMass-SpringSystem ............... 75 5.2.2NonlinearSISOMissileSystem ................. 78 5.3NonlinearMIMOSystem:LoFLYTESystem ............. 85 5.3.1SystemIntroduction ....................... 85 5.3.2SystemIdenticationandControlexperiment ........ 88 6CONCLUSIONS ............................... 97 6.1Summary ................................ 97 6.2FutureWork ............................... 98 6.3Conclusion ................................ 99 APPENDIX AOPTIMIZINGPARAMETERSOFESN .................. 100 BESN-BASEDDACCONTROLLER .................... 104 REFERENCES ................................... 110 BIOGRAPHICALSKETCH ............................ 117 vi

PAGE 7

Table page 3{1Performanceon32-DLoFLYTEdata .................... 38 5{1PerformanceontheLorenzsystem ..................... 67 5{2PerformanceontheDungoscillator .................... 71 5{3Performanceonthelinearmasssystem ................... 76 5{4Performanceonthemissilemodel ...................... 80 5{5Controlperformancecomparisononmissilesystemconsideringnoise ... 84 5{6PerformanceontheLoFLYTEsimulationdata ............... 91 5{7ControlperformancecomparisonontheLoFlytesystemconsideringnoise 92 vii

PAGE 8

Figure page 1{11-DfunctionestimationbyGMM-LLM. .................. 5 2{1BlockdiagramoftheESN. .......................... 17 2{2DistributionofESNinternalweightswithmaximumspectralradiusbeing0.9,withunitcircleasareference. ..................... 19 3{1StructureofGMMwithLLM. ........................ 26 3{2ExplanationofG-SOMalgorithm. ...................... 37 3{3SoftcombinationregionusingGMM(betweendashline) ......... 40 4{1StructureofControlSystembasedonGMM-LLM ............. 43 4{2T-Sfuzzylogicmembershipfunction .................... 45 4{3StructureofInverseControlSystem ..................... 46 4{4pole-placementPIDcontrolsystem. ..................... 51 4{5Optimalrobustcontrolsystemschematicdiagram. ............ 59 5{1DynamicsoftheLorenzsystem ....................... 64 5{2Systemidenticationperformanceonthe\x"oftheLorenzsystem. ... 66 5{3DistributionoftheLorenzsystem(dashline)andGMMweights(star) 67 5{4Weightschangeof8GMMkernels; ..................... 68 5{5PerformancecomparisonbetweenSOMandGMM ............. 69 5{6Compensationperformance ......................... 69 5{7DynamicsoftheDungoscillatorasanautonomoussystem. ....... 71 5{8SystemidenticationperformanceontheDungsystem ......... 72 5{9Sampleofweightschangeof8GMMkernels; ................ 73 5{10ControlperformanceofdierentcontrollersontheDungsystem .... 74 5{11Dynamicsofthemasssystem. ........................ 76 viii

PAGE 9

.... 77 5{13Sampleofweightschangeof6GMMkernelsformasssystem ....... 78 5{14Set-pointcontrolperformanceonthemasssystem. ............ 79 5{15Missilesystemdynamics. ........................... 79 5{16Sampleofsystemidenticationperformanceonthemissilesystem. ... 80 5{17Sampleofweightschangeof8GMMkernelsformissilesystem ...... 81 5{18PerformanceofPIDcontrollerwithdierentmodelingapproaches. .... 82 5{19GMM-LLMbasedcontrolperformanceonthemissilesystem. ...... 83 5{20ESNcontrolperformanceonthemissilesystem. .............. 84 5{21Generaldescriptionofanaircraft ...................... 86 5{22DynamicsoftheLoFLYTEsystem. ..................... 87 5{23SystemIdenticationperformanceontheLoFLYTEsystem. ....... 89 5{24SampleofweightschangeofGMMkernelsfortheLoFLYTEsystem ... 90 5{25ControlperformanceontheLoFLYTEsystem(set-point,noisefree). .. 93 5{26ControlperformanceontheLoFLYTEsystem(tracking,noisefree). ... 94 5{27RobustcontrolperformanceontheLoFLYTEsystem(set-point,30dBSNRnoise). .................................. 95 5{28RobustcontrolperformanceontheLoFLYTEsystem(tracking,30dBSNRnoise). .................................. 96 A{1DistributionofimprovedESNinternalweightswithmaximumspectralradiusis0.9,withunitcircleasareference. ................ 101 B{1DemonstrationsystemwithESNreferencemodelandcontroller ..... 105 B{2SchematicdiagramofcontrolsystemwithESNascontrollerbasedonGaussianmodels ............................... 108 ix

PAGE 10

Inthisdissertation,wepresentamethodologyofcombininganimprovedGaussianmixturemodels(GMM)withlocallinearmodels(LLM)fordynamicalsystemidenti-cationandrobustpredictivemodelcontrol.Inordertounderstandtheadvantageofthemixturemodel,itsstructureandtrainingarediscussedindetail.Agrowingself-organizingmapisutilizedtoimprovetherandominitializationofmixturemodels,whichmakestheGMMconvergencemorestable.Toincreaselocalmodelingcapa-bilityanddecreasemodelingerror,locallinearmodelsaretrainedbasedonGMMasone-steppredictors.Followingthelocalmodelingapproach,aseriesofcontrollersaredesignedtorealizeatrackingapplication,amongwhichtheoptimalrobustcontrolshowsbetterrobustnessoverothercontrollers.Fiveapplicationsystemswithdier-entdynamicsaresimulatedinordertoverifythemodelingandcontrolcapabilityoftheimprovedGaussianmixturemodel.Throughexperimentsandcomparisonwithself-organizingmaps,radialbasisfunctions,andothermethodologies,itisshownthattheimprovedGMM-LLMapproachisamoreexiblemodelingapproachwithhigher x

PAGE 11

xi

PAGE 12

1 ].Inparticular,theanalysisoflinearidenticationandcontrolfornonlineardynamicalsystemshasbeenstudiedbymanyresearchersleadingtoabetterunderstandingofvariousmechanismswhichincludesobservability,controllability,stability,androbust-ness[ 1 2 3 ].Thebestdevelopedaspectofthetheorytreatssystemsdenedbylinearoperatorsusingwellestablishedtechniquesbasedonlinearalgebra,andthetheoryoflineardierentialequations.Nevertheless,academicresearchhasbeenconcentratingaroundproblemsinvolvingtheidentication,stabilization,andcontrolofnonlineardynamicalsystemsinthelastseveraldecades.Theseeortshavenowmaturedintoabroadgrouptheoriesonnonlineardynamicalsystemsapplications.Initialeortsinthisareapursuedparametricapproached,inspiredbytheestablishedlinearsystemtheory,inwhichthedynamicalequationsofthesystemareassumedtobeknownfromphysicalprinciples,withsomeuncertaintyincertainparameters.Inthisframework,thesystemidenticationandsystemcontrolproblemsaredecoupled.Andthereforetheycanbesolvedsequentially.Recently,adaptivesystemidenticationandcontrolmethodologieshavebeeninvestigatedtoo,whichleadstoagoodunderstandingoftheadaptationinlinearsystemsandasatisfactorilygeneralinsighttoadaptationinnonlinearcontrolsystems[ 4 5 6 7 ]. 1

PAGE 13

Controltheorydealswiththesystemmanipulationofthebehaviorofdynamicalsystemstosatisfycertaindesiredoutputsconstraints[ 8 9 ].Aclassicalparametricdesignprocedurefollowsthesystemmodeling(identication)andcontrollerselectionstages.Inthecaseoftraditionalidenticationbasedonmodelderivedfromphysicalprinciples,thedataareusedtoestimatetheunknownparameters,withphysicallymotivatedsystem,whereasmodernapproachesstemmingfromtheadvancesinneuralnetworksmodelingintroduceblack-boxfunctionapproximationschemes[ 4 ].Theneuralnetworkmodelingcapabilitiesmaybefurtherenhancedusingmultiplesub-modelsinthecontextofswitchingbetweenmultipleadaptivemodelstoachieveclose-loopsystemthatenhancetransientbehaviorandcopewithmodelinguncertaintiesbetterthansinglemodel.Followingthesystemidenticationstage,dependingonthechosenmodelingapproach,thecontrollerisdesignedtypicallyusingclassicaltechniquesbasedonlinearsystemtheoryaswellasclassicalorneural-networks-basednonlineartechniques[ 1 10 ]. Modelingtechniquescanbeclassiedintoglobalandlocal.Globalmodelingusesasingleparametricsystemandtsitalloverthestatespace.Theglobalmodelingmethodsestimatethedynamicsofsysteminoneframework.Usingsinglekernelormultiplekernels,themodelistrainedwiththewholetrainingdataset.Themostdistinguishingfeatureforglobalmodelingisthatitistrainedwithallthedatapoints,andinthesystemidenticationphase,allkernelswilltakeeectoneverystepofprediction.Goodexamplesofglobalmodelingarethemulti-layerperceptron(MLP)andtheradial-basis-function(RBF),eventhoughthelastoneuseslocalkernels.Whenestimatingthedistributionintheinputspace,eachkernelintheRBF'shiddenlayeracceptsallthedatafromthetrainingdataset.Recurrentneuralnetworks

PAGE 14

(RNN)areanotherkindofpowerfulcomputationalstructureforglobalmodeling[ 3 ].Butbecauseoftheircomplexinnerstructure,ecienttrainingalgorithmsarenecessarytoachievesuccessfullearningforthegiventask.Anotherdicultyforglobalmodelingiswhendynamicalsystemcharacteristicsconsiderablyvaryovertheoperatingregime,eectivelybringingtheissueoftimevaryingnonlinearparametersintothemodeldesign.Becauseofthecomplexityandpossiblytime-variabilityofthenonlineardynamicalsystem,globalmodelinganddesigningcorrespondingcontrollersisadicultandtime-consumingtask. Localmodeling,whichutilizesadivide-and-conquerapproachtosolvecompli-catedproblems,breaksthettingofthesystemintosmallerandeasierpieceswhichcanbemanagedbysimplertopologies.Usingmultiplekernels,thelocalmodelingistrainedinawaythatakernelonlygetstraininginformationfromthosedatawhicharelocatedwithintheregionthatkerneldened.Insystemtrainingphase,everyinputdataislocatedrst,andonlythe\winner"kernelwilllatertakeeectoniden-tication.Tobetterunderstandthispoint,letustaketheself-organizingmap(SOM)[ 11 ]asanexamplewithitswinner-takes-allcriterion.Thereforethelocalmodelingtechniquecanberegardedasapiece-wisemodelingapproach,andthepiecescanbegroupedtogetherwhenitisnecessarytoapproximateglobaldynamics.Specicallywheneachmodelislinear,theresultingmodelisapiece-wiselineardynamicalap-proximationtothegloballynonlineardynamicalsystem.Theadvantagesofsuchapartitioningapproacharethefollowing: 1.systemidenticationcomplexitycanbereducedsignicantlyduetothescalingdownoftheestimationproblemfromonelargetasktomultiplesmallandsimpletask;

PAGE 15

2.thelocalmodelscaneasilyscaleuptoencompassmorevolumeinthestatespacebyaddingnewmodelsasnewdataportionisacquired,whichisespeciallyofbenetwhentrainingdatacannotcoverthewholestatespace; 3.thedesignofacontrolsystemforthenonlinearsystemcanbereducedtothedesignofmultiplesimplelocalcontrollersamongwhichcooperationispossibletogenerateasinglecontrollertothecurrentplant. TheGaussianmixturemodelwithlocallinearmodels(GMM-LLM)isalocalmodelingapproach,yetGMMtrainingbasicallyfollowstheglobalmodelingtrain-ingcriterionbecauseofthedistributingpropertyofGaussiandensityfunction.IntheGMMapproach,themodelisgeneratedbyamulti-dimensionaljointmixtureofGaussiandistributions[ 12 ].Inotherwords,thedistributionofstatevectorsisdescribedbyasoftcombinationofseveralGaussiandistributions.Andthosecombi-nationcoecientswillbedeterminedaccordingtotheGaussianposteriorprobabilityofcurrentstatevectorgivingcertainGaussiandistribution.FollowingtheGaussianposteriorprobabilityproperty,everyGaussianistrainedbyallthetrainingdataset.YetduetothenatureofthenonlinearGaussiandistributionfunction,theweightswillbelargewhenthedataisclosetothecentersofsomeGaussian,whichmaketheGaussiankernel'strainingemphasizemoreontheirneighbordatathanthedatafaraway.Inthepredictionpart,itbecomesclearthatGaussianmodelistakingeectlocally.ThebenetfromthisnatureisthattheGMMisgloballywelltrained,andlocaldynamicsareaddressedaswell.Thelatterisalsothereasonwhywecalltheapproachlocalmodeling. Theproblemofcontrollerdesigncanbefurthereasedwiththeselectionoflocalmodeling.Inthiscase,theglobalnonlinearcontrollercanbetransformedintomulti-plepiece-wiselinearcontrollersfornonlinearsystems,forwhichmanystrongdesigntoolsareavailable[ 8 9 13 14 ].

PAGE 16

Figure1{1. 1-DfunctionestimationbyGMM-LLM.Top:Nonlinearfunctionanditslinearapproximation;Bottom:GMM-LLMpairs 1{1 .TheGMMusestheGaussianfamilytoreconstructthedistributionofthestatespacevectorsapplyingasoft-combinationinsteadofawinner-takes-allcriterion.EachGaussiancomponentrepresentsacertaincharacteristicintoaspeciclocationofsystemdynamicspace.SincetheGaussiandistributionneedsonlytwoparameters,meanandcovariance,thecomputationeciencycanbelargelyincreasedinGMMtraining.Becauseofthismodelingproperty,GMMshavebeenwidelyappliedinspeechrecovery,spectralanalysis,etc.SchonerappliedGMMforfunctionapproximationusingsupervisedtraining[ 15 ].InitializingtheGaussiankernels'weightsfromgrowingself-organizingmaps(GSOM),amorestableglobalmaximumconvergenceisachieved.

PAGE 17

LocalmodelingisusuallybasedontheNearest-neighborscriterion.AclusteringalgorithmdividesthetrainingdatasetintoagroupofsmallersetsbasedontheEuclideandistanceorothersimilarcriteria,andamodelistrainedcorrespondingtoeachdataset.AsFarmerandSidorowich[ 16 ]havealreadyshown,locallinearmodelsprovideaneectiveandaccurateapproximationdespitetheirsimplicity.Locallinearmodelshaveshownverygoodperformanceincomparativestudiesontimeseriespredictionproblemsandinmanycaseshavegeneratedmoreaccuratepredictionthanglobalapproaches[ 16 17 ]. Buildinglocalmappingsinreconstructedstatespaceisatimeandmemoryconsumingprocesseventhoughtheperformancecanbesatisfactory.Asimpliedapproachistoquantizethestatespaceandbuildlocalmappingsonlyinpositionswhereprototypevectorsarelocated.TheimprovementutilizedintheSelf-OrganizingMaps(SOM)hasbeenappliedbyPrincipeetal.tomodelautonomousandnon-autonomoussystems[ 18 19 ].MotteralsoutilizedSOMbasedmodelingtothedesignofswitchingcontrollers[ 20 ]. OnepossibleproblemtheSOM-basedmodelingcannotavoidisthequantizationerrorinthemodelingphase,duetotheEuclideandistancebetweentheprototypevectorandthedataclosesttothevector.TheSOM-basedapproachassumestheprototypevectorscanrepresentthedistributionofthewholestatespaceandthedistanceisnegligible,yetthatistrueonlywhentheamountofprototypevectorsisverylarge.Meanwhile,largeamountofprototypevectorswillleadtofurtherrequirementsoncomputationalresourceandtrainingtime. ThestructureofGMMconsistsofasetofGaussianmodelsandagatingfunction.Theweightingcoecientforacertaindatapoint~xcorrespondingtotheGaussian

PAGE 18

kernelsisdescribedbythegatingfunction.Asthedistancebetweenthestateandkernelsdeterminesthegatingfunctionvalue,itmakesmodelingerrordecreaseandproducessmoothswitchingcomparedwiththeSOMmethod. TheparametersoftheGaussianmixtureareinitiallyunknown,andwillbees-timatedbasedonincompletedatathroughtheGrowing-SOMmethod.Thisimpliesthatparametersareupdatedinacooperativeratherthancompetitiveway.Thepro-cedureisanupdatedweightingofthecontributionofalldatapointsgivenaparticu-larGaussiankernel.ThetrainingofGMMisbasedonmaximumlikelihoodcriterionthroughtheexpectation-maximization(EM)algorithm.Section3.2discussestheEMtrainingalgorithmindepth.FollowingtheGaussiandistributiondensityfunction,onelocallinearmodelisgeneratedfromlocaldataforeachGaussiankernel.Thankstothegatingrule,themixtureofGaussiansframeworkisabletocontinuouslyvarytheinfrastructureofeachpiece-wiselinearlocalmodel. WhentheGMM-LLMiscomparedwithRBF,theadvantageofGMM-LLMisthemodelingoflocaldynamics.EventhoughboththeGMM-LLMandRBFusealinearcombinationinthenalstage,themechanismofhowtheydescribethelocaldynamicsisdierent.RBFmodelingusesasinglexedweightedcombinationofagroupofnonlinearoutputsfromGaussiankernels,thecentersoftheGaussiansareadapted,butnotthecovariance.AlthoughtheGMM-LLMusesalinearmodeltorepresentthelocalsystemdynamics,whichmakestheoutputchangesinalinearmannerinsteadofanonlinearway.Severalsuchlocalmodelsexistdistributedthroughoutthestatespace,andtheyareweightedbythegatingfunctiontoincreaseperformance.

PAGE 19

BasedonthemixtureGaussianreferencemodel,wedevelopasetofcontrollersfornon-autonomousandnonlinearplants.StartingfromtheEMtraining,thesys-temmodelcanberepresentedbyGMM-LLMandthemodelingerrorisbounded.Severalcontrollersuchasdynamicinverseandpole-placementPIDcontrollers,op-timalrobustcontroller,echostateneuralcontrollerarerealizedandemployedusingthemixturemodelforseveralsystemswithdierentdynamics.Weemphasizetheoptimalcontrollerandtheechostateneuralcontroller.Wewillshowthattheopti-malcontrollerexhibitsexcellentrobustnessagainstmodeluncertaintyandexternaldisturbance,andtheESNcontrollerhasbettergeneralizationcapability. Thedissertationisdividedintoveparts.InChapter2,somegeneraldescriptionanddenitionsofnonlineardynamicsystemsandlocalmodeling,time-embeddingde-laysystemreconstruction,anddynamicsystemidenticationaregiven.Thecriterion

PAGE 20

toselectembeddingdimensionwillbediscussed.Echostatenetwork,anewkindofrecurrentneuralnetworks,ispresentforthelateruseoncontrollerdesign. InChapter3,GMMtrainingmethodologyanalysisispresented,whichalsoin-cludesimprovedGMMtrainingthroughGSOMinitialization,thegenerationoflocallinearmodels(LLM),andthecomparisonofGMM-basedapproachtoSOM-basedapproachinordertoaddressthecomputationaleciencyofGaussianmodeling. InChapter4,thedescriptionsforasetofcontrollerdesignapproachesbasedonGMM,thecontrollers'mathematicalbackgrounds,andtheirstabilityissuearediscussedindetail. Chapter5givesapplicationexamplesonasetofrealsystemmodel,andmainresultsofGMM-basedapplicationsarepresentedwithcomparisonsanddiscussedindetails.Fordierentmodelingmethods,themodelingperformancewillbecomparedbasedonsignaltoerrorratiocriterion.Forcontrollers,thecontrolperformancewillbecomparedbasedoncriteriaofrising(falling)time,settlingtime,overshot,andthecovarianceofcontrolerror. Chapter6givesconclusion,contributionandbrieysummarizesfuturework.

PAGE 21

Dynamicsdescribehowonesystemstatedevelopsintoanotherstateoverthecourseoftime.Technically,adynamicalsystemisasmoothfunctionoftherealsortheintegersonanotherobject(usuallyamanifold).\Whentherealsareacting,thesystemiscalledacontinuousdynamicalsystem,andwhentheintegersareacting,thesystemiscalledadiscretedynamicalsystem"[ 21 ].Adynamicalsystemexpressedasadiscretetimeevolutionrulecanbedescribedfromtheobservationofitsoutputs.Thedynamicofthesysteminstatespacecanbedenedas (2{1) wherex(t+1)isthenextstepstatefordiscrete-timesystems.x(t)arethesystemstates,f()isanonlineartransferfunctionandisusuallyreferredasthevectoreld, 10

PAGE 22

andh()istheoutputfunction.Ifthevectoreldf()isalinearfunctionofthesystemstates,theunderlyingsystemislinear;otherwise,thesystemisnon-linear.Itisalsoassumedthatthesystemsherearedeterministic.Thusthetransitionfromthesystem'sstatex(t1)attimet1tothestatex(t2)attimet2isgovernedbydeterministicrules. 2{1 )describesasystemthatfollowstheinternalgoverningequationsf()givenacertaininitialcondition.Andthesystem'sdynamicisfullycharacterizedbyitsmanifoldencodedinthefunctionf().BecauseofTakensTheorem,transform-ingthetimeseriesintoastatespaceformwillresultinadynamicalequivalenttotheoriginalspace[ 22 ].Sincethemathematicaldescriptionoff()isunavailableformostreal-worldsystems,withtheformulationofthetimeembeddingtheorem,Takenspre-sentedamethodologytorecoveranequivalentdescriptionofffromobservationofthesystem: 23 ].Therstisaprocedurethatyieldsaglobalmeasureofphase-spaceutilizationfor

PAGE 23

(quasi)periodicandstrangeattractorsandleadstoamaximumseparationoftrajec-torieswithinthephasespace.Theseconddescribesthelocaldynamicalbehaviorofpointsontheattractorandgivesameasureofhomogeneityofthelocalow.GaoandZhengproposedthelocalexponentialdivergenceplotmethodforachaotictimese-ries,basedonwhichtheembeddingdimension,delaytime,andthelargestLyapunovexponentareestimated[ 24 ]. WedeterminetheminimumtimeembeddingdimensionusingLipschitzquotient[ 25 26 ].Ifthefunctionisassumedtodependonn1inputsanditactuallydependsonninputs,thedatasetmaycontainseveralpointsthatareverycloseinthespacespannedbythen1inputsbutdiersignicantlyinthenthinput.Assumingthesystemfunctionissmooth,ifthe(n1)-Dinputsareclose,itisexpectedthattheoutputsfromthemarealsoclose.Ifoneorseveralrelevantinputsaremissing,outputscorrespondingtodierentinputwilltakedierentvalues.Inthiscase,itcanbeconcludedthat(n1)-Dinputisnotlongenough. TheLipschitzquotientsin1-Dcasearedenedas j(i)(j)j;i;j2
PAGE 24

Asdiscussedbefore,ifnisnotbigenough,theLipschitzindexwillbelarge.Ifallrelevantinputsareincluded,theLipschitzindexwillstaynearconstant. Severalapproacheshavebeenstudiedforglobalmodeling.Onerepresentativeofglobalnonlinearmodelingisthetimedelayneuralnetwork(TDNN)whichwasrstdevelopedbyWeibelandLangin1987[ 27 ].Usingtimedelaysegmentofstateasinputdata,theTDNNisstructurallycloserelatedtothemulti-layerperceptron(MLP).TheTDNNhasbeenshowntobeauniversalmapperinmultidimensionalfunctionalspaceswithnitememory(especiallymyopicfunctionalspace)[ 28 ].An-othermethodisthepolynomialmodeling.Polynomialmodelingisappliedasthebasisfortherealizationofnonlinearmapping.Fuhrmannusesitingeometriccontroltheory[ 29 ].Recurrentneuralnetworks(RNN)representsakindofnonlinearmodel-ingmethodology.ResearchersuseRNNwhenregularmathematicalequationscannotgeneratesatisfyingresults[ 30 31 ]. Inthefollowingsections,wewillintroducepolynomialmodelingbrieyaswewilluseitsrstorderexpansionasalinearmodelinchapter3.Andwewilldiscuss

PAGE 25

theechostatenetwork(ESN)withmodieddesignmethodasanewglobalmodelingapproach. ~y(n)=b1u(n1)++bDu(nD)a1y(n1)aDy(nD)(2{6) ThepolynomialModelingcanbeextendedtoanonlinearARMAX(NARMAX)modelbyreplacingthelinearmapperin( 2{6 )withnonlinearfunctionf() ~y(n)=f(u(n1);:::;u(nD);y(n1);:::;y(nD))(2{7) Amorestraightforwardwaytoutilizepolynomialsfornonlinearsystemidenti-cationistoapplyapolynomialforapproximationoftheone-steppredictionfunction( 2{7 )withloworderexpansion,whichiscalledKolmogorov-Gaborpolynomialmod-els.Andthebenetfrompolynomialmodelisthatallthenonlinearcoecientscalculationsareavoided.AsimpliedversionoftheKolmogorov-Gaborpolynomialmodeluseslesshigh-orderinformation,alsocalledVolterraSeriesmodel,describesthenonlinearityonlyfortheinput ~y(n)=f[u(n1);:::;u(nD)]a1y(n1)aDy(nD)(2{8) Thusthesecondorderseconddegreepolynomialfunctioncanbeexpressedas (2{9) +6u2(n1)+7u2(n2)+8u(n1)u(n2)

PAGE 26

Withthissimplicationthenumberofregressorsisreduced,thusthecomputationofpredictionissimplied.ThepricepaidhereisthenecessaryincreaseoforderDinordertodescribethenonlinearityoftheoriginalsystem.Forlinearmodel,therewillbenohighorderexpansion,andtheembeddinglengthwillbeincreasedwhennecessary. 32 33 34 ].Comparingwithothernonlinearneuralnetworks(recurrentnetworksespecially),theESNhaslessamountofparame-tersneedtobetrained.Moreover,theparametersarelinearandtheycanbetrainedthroughtheWiener-Hopfapproach[ 35 ]. Heterogeneousorderednetworksor,morespecically,recurrentneuralnetworks(RNN)areconvenientandexiblecomputationalstructuresapplicabletoabroadspectrumofproblemsinsystemmodelingandcontrol.Therearemanywaystomakethisstatementmoreprecise.Forinstance,RNNscanbecastasrepresentationsofthevectoreldofadierentialsystems,orbetrainedtoembodysomedesiredattractordynamics[ 31 ].Inthoseapplications,systemswithoutinput(autonomous)aremodeled.Sincewewishtomodelinput-drivensystems,weadoptastandardperspectiveofsystemtheoryandviewadeterministic,discrete-timedynamicalsystem

PAGE 27

asafunction,whichyieldsthenextsystemoutputasaprediction,giventheinputandoutputhistory[ 36 ]. AsaspecialcaseofRNN,theESNwasrstproposedbyJaeger[ 36 37 ].TheESNshaveinterconnectedandrecurrenttopologyofnonlinearPEswhichconstitutea\reservoirofrichdynamics".TheinterestingpropertyofESNisthatonlythememorylessreadoutofESNneedstraining,whereastheinternalrecurrenttopologyhasxedconnection.ThiskeypropertydramaticallyreducesthecomplexityofRNNtrainingtosimplelinearregressionwhilepreservingsomeofthecomputationalpowerofRNN. Weconsidertherecurrentdiscrete-timeneuralnetworkasgiveningure 2{1 withMinputunits,Nprocessingunits,andLoutputunits.ThevalueoftheinputunitattimenisX(n)=[X1(n);X2(n);;XM(n)]T,thatofinter-nalunitsiss(n)=[s1(n);s2(n);;sN(n)]T,andthatofoutputunitsisy(n)=[y1(n);y2(n);;yL(n)]T.ThesystemtransferfunctionofESNcanbeexpressedas where,formostofthecase,nonlinearoutputfunctionfout()iseliminated,andonlylinearoutputisused. ThecriticaldierencebetweenESNandRNNliesintheadaptationoftherecur-rentweights.ESNcompensatesfornottrainingtherecurrentconnectionweightsbyincreasingthedimensionalityofthehiddenlayer.Inthisway,ESNcreatesadiversityofdynamicswithalargenumberofPEsinitsreservoirandthereadoutissimplytrainedtopickupdynamicalcomponentsrequiredforagivendesiredoutputsignal.Hence,itiscrucialfortheoperationoftheESNtoincludeareservoirwith\enough

PAGE 28

Figure2{1. BlockdiagramoftheESN. richness"ofdynamics.AndinmostengineeringapplicationsofESN,e.g.systemidentication,theoptimallinearmapperisobtainedthroughrecursivealgorithmswhichminimizemeansquareerror(MSE)orhigherordererrormomentum. IntheESN,theechostatesformbasisfunctionsthatareanonlinearlylteredversionoftheinputsignals[ 35 38 ].ThelinearindependencyoftheechostatesisalmostalwayssatisedbecauseofthenonlinearPEsexceptrarepathologicalcaseswhenWandWinhassymmetricalentrieswithrespecttoanytwoPEsintherecur-rentnetwork.Notethat,theechostatesarenotorthogonaltoeachothersincethereareconnectionsbetweenPEsmakingthestatescorrelated.Withthisinterpretation,allwhatthelinearread-outisdoingistocombinethebasisfunctionsderivedfromtheinputtocreateagivendesiredsignal.ThespanoftheESNcanbedenedasthesetofallfunctionsdenedby( 2{10 ),wherefoutislinear,forallvaluesofWout.Here,thecriticalquestionishowbigthespanoftheechostatesis.ThisisdirectlyrelatedtotheselectionoftheweightsoftheESNreservoir.Currently,therearetwoways

PAGE 29

togenerateW.TherstmethodselectsWrandomlyfromasparsematrixwiththehelpoftheechostateconditionkWk<1[ 36 37 ],wherekkisthespectralradius(SR).Forthesecondmethod,theWisselectedsuchthatthemaximumentropyisachieved[ 35 ].Theechostateconditionisastabilityconditionthatguaranteesthatthestatesareonlycontrolledbytheinputwhereasthesparsenessisrequiredtoavoidmuchcorrelationbetweenthestatesbyminimizingtheconnectionsamongthem. Assumingunknowndynamicalsystemtransferfunctionisg(),theobjectiveofsystemidenticationistogenerateanestimationfunction~g()suchthattheerrorpredictionissmall, Letthelinearmapper(assumeasingleoutput)bedenotedbytheweightvec-torWout.ConsideringthecostcriterionisMSE,thereadoutvectorcanbeeasilycomputedthroughWiener-Hopffunctionas whereR=E[sTs],P=E[sTd],andsisESNinternalstatestrainedfrominputsequence,discorrespondingdesiresequencefromtrainingdata. Wenowpresentamoregeneralformulationofthetrainingprocedureforthecasewithnoinputfeedforwardandoutputfeedback.

PAGE 30

Figure2{2. DistributionofESNinternalweightswithmaximumspectralradiusbeing0.9,withunitcircleasareference. 2{2 2{10 )toobtaininternalstatesss.Thendismissfromthisrunasuitablylonginitialtransientperiod.

PAGE 31

Localmodelingapproaches,ontheotherway,dividethedynamicspaceintoseveralparts.Eachlocalmodelistrainedttingonepartofthedatafromthestatespace.Thelocallinearapproximationcanbeconsideredasimpliedalternativetononlinearmodeling.TheapproachoflocallylinearARXhasbeenwidelydiscussedinthetimeseriesliterature.Itisastate-dependentsequentialtypeofrecursivealgo-rithmforidentifyingdynamicmodelsusingsimplelinearregressionorinterpolation[ 2 ].Undertheassumptionthatf()issmoothinthevicinityofx(n),f()canbeapproximatedbytherstordertermofitsTaylorseriesexpansion, (2{13) =~aTnx(n)+~bn

PAGE 32

especiallyforsystemswithcomplexbehavior.Casdaglialsostressedthatthegreatadvantageoflocalmodelingistheexibilityinbuildingthemodel[ 39 ]. Asthecentermissionofmodeling,thepredictivemappingf:RD!Rcanbeexpressedthroughtimeembeddingas wheref()isadeterministicARMAmodelwithtime-embeddedinputvectorx(n)=[x(n);x(n1);:::;x(nD+1)].Withthemethodoflocalmodelingthestatespaceispartitionedintosmallregionsandalocalmodelisestablishedtodescribethedynamicsindierentregion.Thuslocalmodelsvaryfromregiontoregion. TheapproachoflocallyARMAmodelshasbeendiscussedinthetimeserieslit-eraturebyPriestley[ 40 ].ImprovedapproachwasproposedbyFarmerandSidorowich[ 16 ],whoalsoextendedARMAmodelstolocallypolynomialapproximationsofhigherorder.Therstorderexpansion,whichisalinearmodel,isaneectivelocalmodelingmethod.Andtheapproximationwithhigh-orderpolynomialsarenotsignicantlybetterthanthoseobtainedwithrstorder.Inourwork,wewillusetherstorderexpansion. Inmanylocalmodelapplications,theself-organizingmap(SOM)hasbeenuti-lizedtodividetheoperatingregionintosmallparts,whereonelocalmodelistrainedcorrespondingtoonepartofthestate[ 5 11 18 19 ].TheSOMisappropriateforswitchingnonlinearrelationshipsofhigh-dimensionaldataintosimplegeomet-ricrelationshipsthatpreservethetopologyinthestatespace[ 41 ].PrincipeandWangmodeledachaoticsystemwithSOM-basedlocalmodelingmethod[ 19 ].Cho

PAGE 33

18 20 ]. TheroleofGMMinsystemidenticationistoestimatethedistributionofstatespace.AccordingtothedistributiondensityofeachGaussiankernelwithinGMM,thecurrentstateofthesystemisdescribedbyoneorseveralGaussianmodels.Tonishthepredictivemodeling,asetoflocalmodelsalsotrainedbasedonGMM.FollowingtheGMM'scriterion,onelocalmodelislinkedtooneGaussiankernel.Andthesetwomakeuptheidenticationmodelpair.ThedetailofimprovedGMMconstructionandtrainingandcorrespondingLLMtrainingwillbediscussedlaterinchapter3. FromthenatureofESN,wecanseeoneadvantageofESNisitsless-trainingconstructionprocedureeventhoughitisnotlinearmodelinganymore.Sinceitsinternalstatesarerandomlygenerated,theonlypartwhichneedstrainingisitsreadoutvector,andthereisnotrainingthatisnecessaryforthestates.Andsincethereadoutisoflinearform,thetrainingforthereadoutvectorcanbeeasilydone

PAGE 34

throughWiener-ltersimilarmethods.Thusthroughthefunctionalityof\internalstates",theESNwillrequirelesstrainingcomputationcomparewithotherrecurrentnetwork.AnotheradvantageofESNisitsinputstructureovertime-embeddingdatastructure.SinceESNhasa\reservoir"ofinternalstatesandpre-denedfeedback,theseinternalstatescanbedeemedan\internaldelay"toinputforacertainsystem,ESNcanrecordityettime-embeddingmayneedlongerdatastructure.WewillalsoshowthattheESNwithmaximumentropycanaccomplishsystemidenticationworkwithminimumMSE.

PAGE 35

42 ],howeveritisbasedonawelldenedstatisticalframeworkfordensityestimation.TheGMMisalsosimilartotherstlayeroftheRadialBasisFunction(RBF)networkusedinnonlinearsystemapproximation.InfactthekernelsintheRBFarealsoGaussianfunctions,whicharespreadoverthedataspaceusingaclusteringalgorithm[ 42 43 ].NormallythecovarianceoftheseGaussiankernelsarenottrainedfromthedata.TheRBFoutputlayercombinestheseinternalstateslinearlyintoasingleprojectorthatistrainedwiththedesiredresponsetoextracttheinformationinthejointspace.TheGMM-LLMapproachcombinesthesetwobasicstrategies,bycreatingalocalpartitionoftheinputspaceasinthePAalgorithm,andemploysmultiplelocallinearmodelsattachedtoeach 24

PAGE 36

partition,insteadofnonlinearcombiningtheintermediatenonlinearstatesasintheRBF. Figure 3{1 depictstheelementsoftheGMM-LLMarchitecture.TheinputdataismodeledbyamixtureofGaussian'smodel,andtheinuenceofeachGaussianisevaluatedbyaweightthatrepresentsthelikelihoodofthedatabeinggeneratedbythecorrespondingGaussianmode.DuringtrainingthedatafromeachGaussianclusterisusedtotraintheLLMs.Duringtesting,theinputisdirectlysenttoalltheLLMsandtheiroutputweightedwiththeprobabilitythatthesampleisgeneratedbyeachmodelbeforebeingcombinedbytheoutputadder.Notethatduringtestingtheseweightschangewiththedataprovidingagreaterexibilitywhencomparedwithotherlocalmodelingalternatives.Thisarchitectureisprincipledusingwellestablishedstatisticalmethods,veryexible,andstillverysimplebecauseitusesonlylinearmodels. TheGMM-LLMapproachisconceptuallyanalternativetotheSelfOrganizingMap(SOM)-LLMreviewedinchapter2.TheSOMclustersthesystemstatespaceintoagroupofsmallregionsusingawinner-takes-alloperator[ 42 ].Eachsmallinputregion,whichisalsocalledVoronoitessellation,isassociatedtoonelinearmodelthatistrainedwiththedatafromthetessellationandthatattemptstoapproximatethedesiredresponsewheninstantiated.Globallythemodelingequationcouldbedescribedas Usingsoft-combinationinsteadofwinner-takes-allcriterion,GMMdoesthelocalmodelinginacooperativemanner.Eventhoughitstillfollowsthedescriptionofthe

PAGE 37

Figure3{1. StructureofGMMwithLLM.Top:Trainingschematic;Bottom:Testingschematic. rstpartof( 3{1 ),itdoesn'tlimitthecoecientstobe\1"or\0".Consequently,theGMM-LLMapproachcanreachsimilarSIperformancewithlessPEcomparedwiththeSOM-LLMapproach.Moreover,soft-combinationcangeneratesmooth-switchingamongdierentmodels,andcanalleviatethedicultyfacedbywinner-takes-all,whenitchoosesthewrongwinner,wherethecorrespondingLLMwillgenerateworseSIresult.

PAGE 38

AnothercompetitortotheGMM-LLMisthefuzzylogicapproach.ThemostpopularfuzzylogicapproachistheTakagi-Sugeno(T-S)fuzzymodel[ 44 ]whichisalsoalocalmodelingmethod.SimilartotheGMM-LLM,T-SfuzzymodelusesmultiplelinearmodelsforSIandsoft-combinationoflinearmembershipfunctionstoweighttheoutputsfromlinearmodels.TheT-Sfuzzymodeldividesthestatespaceintoseveraloverlappedregionsusingasetofsimple\if-then"rules,onerulecorrespondstoonelinearmembershipfunction.Besidestheorderofmembershipfunction,thedierencebetweentheT-SfuzzymodelandtheGMM-LLMliesinthetrainingmethod.TheT-Sfuzzymodelneedspre-knowledgeaboutthesystemforbothmembershipfunctiontrainingandlinearmodeltraining.UnliketheunsupervisedtraininginGMM,theT-Sfuzzymodeltrainingisinasupervisedmannerwhichincreasesthecomplexityofmodeling. Assumingthetrainingdatasetcoversthestatespaceofthetargetsystem,thestatespacecouldbedescribedbytheembedded(reconstructed)dataintheappro-priatedimensionalspacewithanoptimaldelay.ThegoalistouseasetofGaussiankernelstoestimatethestructureoftheeventuallycomplexdatadistributioninre-constructedspace.SincetheonlynecessaryparametersforaGaussiandistributionareitsmeanandcovariancewhichcansavetimeandcomputationonmodeltraining.ThissimplicityisonereasonwhyGaussiandensityfunctionischosenasthekernelfunction. Settingp(~x)asthedistributiondensityofthecurrentstate~x,p(~x)canbeex-pandedinasumoverGaussianclustersck.AlltheGaussianclusterscoverthewholestatespacedenedby(k,k).Theprobabilitydensityofeachclusterreectsthe

PAGE 39

domainofinuenceofeachcluster. (3{2) =KXk=1p(ck)p(~xjck)=KXk=1p(ck)g(~xjck) whereg()indicatesthatweexplicitlystatetheformoftheconditionalGaussiandistributiondensities (2)D=2jkj1=2e(~xk)T1k(~xk)(3{3) where~xiscurrentstate,kandkaremeanvectorandcovariancematrixofclusterckinthefeaturespace.Alsothesummationoveralltheclustersisnormalizedbytheclusterprobabilitiesp(ck)tosatisfy Kk=1p(ck)=1(3{4) Iftheelementsofinput~xareindependentofeachotherorthecrosscorrelationisnegligiblysmall,thecovariancematrixcouldbereducedtoadiagonalmatrix,whichinturnsimpliesthecomputationofGaussiandensityfunction. 22d;k(3{5) whereDistheinputdimension. Function( 3{3 )and( 3{5 )showthatGMMusesthelinearcombinationofnon-linearestimationmodelssoastobuildapowerfulglobalsystemmodel.Meanwhile,

PAGE 40

GMMdescribesthelocaldatafeaturewithrelativelysimplemodelswhichsatisfydescriptionof( 3{2 ). 45 ].Wedescribethemaximumlog-likelihoodfunctionas (3{6) =NXn=1ln[KXk=1p(ck)g(~xnjck)] whereNandKarethetotalamountofdataandGaussianclustersrespectively. SincetheGaussianmodelparametersarecalculatedfromtheobserveddatasetanddescribethelocaldistributionofdatapoints,arecursivesearchwillbenecessarytondtheoptimalGaussianmodelparameters.Duringthesearch,themaximumlikelihoodfunctionisusedasanoptimizationcriterion,sotherewillbetwoestimatorscombinedineachjointupdate,onefortheestimationoflikelihood,andonefordistributionofeachGaussiankernel.TheExpectation-Maximization(EM)algorithm[ 46 ]isappliedtosearchfortheoptimalparametersoftheGaussiankernels.TheEMalgorithmhasgainedwideappreciationasanecientalgorithmforprobabilisticnetworkstraining.TheEMassumesthattheknownstates(theobserveddata)andtheunknownstates(theGaussianparameters)characterizethewholesystemmodel.

PAGE 41

Asanunsupervisedapproach,theEMtrainingbasedGMMonlyneedsthenumberofGaussiankernelsK.Allotherparametersareobtainedthroughtraining. TherecursiveprocessofEMalgorithmstartswiththeE-step(Expectationcalculation),duringwhichtheclusterparametersareassumedcorrect.Expectationofthelog-likelihood( 3{6 )ofthecompleteconditionalprobabilitybytheobservedsamplesiscomputed.AccordingtoBayesianTheorem,theposteriorprobabilitiesareproportionaltothelikelihoodfunction. Soequivalently,theposteriorprobabilities,whichrelateeachGaussianclustertoeachdatapointintheconditionalprobabilityformp(ckj~x),canbeevaluatedas (3{7) =p(~xjck)p(ck) Ki=1p(~xjci)p(ci) ThesuminthedominatornormalizesamongtheGaussianclusters,andthenumeratordeterminesthelargestpossibledensityfunctionforeachdatapoint.Thusthemaximumlikelihoodofthewholedatasetwillmonotonicallyincrease. IntheM-step(Maximizationcalculation),thedistributiondensityfunctionofeachdatapoint,whichdescribesthelikelihoodofdatagivencertaincluster,istemporarilyxed.TheupdateimprovestheparametersofeachGaussiancluster:unconditionalclusterprobabilityp(ck),meanvectorandcovariancematrix(k;k).Thusthelikelihoodwillbemaximizedrecursively.Inequation( 3{8 ),consideringthattheamountofdatapointsarelimitedandthevaluesarediscreteinreality,the

PAGE 42

integralisreplacedbythesumoveralltrainingdata. ThemeanvectorandcovariancematrixofeachGaussianclusterareupdatedaccordingtothestationarypointsofthelikelihoodtotheunknowns(k;k).Thederivativeequationoflikelihoodtothemeanis @kNXn=1ln[p(~xn)]=0 (3{9) @kNXn=1ln[KXi=1p(ci)g(~xnjci)]=0NXn=1@ @kln[KXi=1p(ck)g(~xnjci)]=0NXn=1p(ck)g(~xnjck) 2k(1)=0 Theanswertoupdatethemeanwillbeobtainedthroughmaximumlikelihoodcriterion:

PAGE 43

Thesameresultisachievedthroughtheintegrationmethodas: =Z~xp(~x)p(ckj~x) Similarly,theGaussianclustercovarianceis: k;ij=Z(~xik;i)(~xjk;j)p(ckj~x)d~x Thebasicmodelestimationprocedurecanbesummarizedintovephases: 1.Pickinitialconditions; 2.Estimatethedatadistributiondensityp(~xjck); 3.Estimatetheposteriorprobabilityofeachclusterforeachdatap(ckj~x); 4.Estimatetheclusterprobabilityp(ck),Gaussianclusterparametersmeankandcovariancek; 5.Gobacktostep(2)untiltheincrementoflikelihoodfunction( 3{6 )iswithincertainrange.

PAGE 44

estimationofthecurrentsystem'sdynamic,thedynamicestimationcanbeextendedtoaone-steplinearmapping~y(n+1)=~f(~x(n))=~LTopt~xn,where~fstandsforacertainreferencemodel.And~LoptistheoptimizedcurrentlocalmodelbasedonGaussiangatingfunction.LLMisthemostsimpliedreferencemodeloftherealsys-temanditwillsimplifythedesignprocedureofcontroller[ 16 ].Fromequations( 3{2 ),( 3{7 ),and( 3{10 )-( 3{12 ),wethuscanexpressone-steplinearpredictionfunctionas ~yn+1=KXk=1p(~xnjck)p(ck) =KXk=1p(ckj~xn)~LTk~xn=~LTopt~xn Equation( 3{13 )canbeswitchedtomatrixforminordertosimplifythecalcu-lationoflocallinearmodel~Lks. ~yn+1=PTnLT~xn(3{14) where

PAGE 45

squareerrorcriterion(MSE) minL(J)=1 =1 @~Lk=2 =)@J @L=26666642 (3{17) From( 3{17 ),usingtheresultfrom( 3{13 ),eachrowof@J @~Lcanbeexpandedas From( 3{18 ),weobtaintheMSE-basedoptimizationfunctionforlocallinearmodel~Lks.Thedierencebetweenwinner-takes-allsolutionandthissolutionisthatPk;nwillnotbealways0or1,anditdetermineswhereandhowmuchclusterkwilltakeeect.

PAGE 46

Inordertoobtainthevaluesofallthelocalmodels,wecanexpandequation( 3{18 )tomatrixformattosimplifythenotations =2666664PNn=1P1;nPr;n~xn~xTn:::PNn=1PK;nPr;n~xn~xTn3777775T26666666664~L1~L2...~LK37777777775=Rr~L Repeat( 318 )foralltheGaussianclusters,thelocallinearmodelcanthenbedescribedas =)~L=R1~P 1.AccomplishstatemodelingusingtheEMalgorithm,obtaindistributionpara-meters(k;k)foreveryGaussiancluster,conditionalprobabilityp(~xnjck),andtheclusterprobabilityp(ck); 2.Computetheposteriorprobabilityp(ckj~x)throughtheBayesianfunctionforeachcluster;

PAGE 47

3.Computethelocallinearmodelsusingdistributioninformationfromprevioustwostepsandleastsquarecriterion; 4.Inthetestingphase,re-calculatethep(~xnjck)andp(ckj~x)givennewdatapoint,thenestimatethesystemoutputthroughlocallinearmodels. 47 48 ].InordertooptimizetheGMMinitialconditions,shortentheEMtrainingtime,andstabilizethenalGMMper-formance,aGrowing-Self-OrganizingMaps(G-SOM)algorithmisapplied.AnotherbenetfromG-SOMisthatitsuppliesaexiblestructuretochangethenumberofGaussiankernelsintheGMM.VlassisandLikassetupacriteriontodetermineoptimalnumberofGaussiankernels[ 49 ]. TheinitialtopologyoftheG-SOMnetworkisatwo-dimensionalstructurewith3PEs,whichwillbeusedtodescribeclustersofpatternsintheD-dimensionaldatavector
PAGE 48

(a)(b) Figure3{2. ExplanationofG-SOMalgorithm.Left:TrainingofG-SOM;Right:GrowingofG-SOM 4.Decreaseallsignals'scountersbyasmallvaluesothatthemeanofwinningfrequencydoesn'tchangetoomuch; 5.Afterfeedinginalldata,onenewPEisaddedbetweenthePEwiththehighestsanditsfarthestneighboruntilthepre-denedPEcountisreached,andallthewinningfrequenciesaresettozero. Aftertraining,theG-SOMPEs'weights~wareutilizedasGMM'smeans'ini-tialvaluewithbalancedwinningfrequency.TheGMM'scovariancematricesareinitiallysettoalargevaluetomakeeveryGaussiankernelcoverthelargestpos-siblearea,andtheoptimalvalueofcovariancematrixwillbetrainedthroughEMalgorithm.TheimprovedalgorithmwasextensivelytestedontheLoFLYTEdatathatwillbefullypresentedinChapter5.TheLoFLYTEsimulationuses32-Dtimeembeddingspace,andthedierenceofperformancebetweentheconventionalGMMandtheGMMwithaGSOMinitializationisshownintable 3{1 .EverySERresultisanaverageofveindependentexperiments.Overall,theG-SOMinitialweightedGMMmakessystemidenticationperformance2:4dBbetterthanrandominitializedGMMwithsmallervarianceanddecreasestheEMtrainingiterations.

PAGE 49

Table3{1. Performanceon32-DLoFLYTEdata GMM GMMwithG-SOM 40dBSNRnoise SER(dB) Variance SER(dB) Variance p 23.637 0.4338 23.405 0.2589 q 26.012 0.2849 24.396 0.3738 r 24.663 0.3859 22.553 0.1753 u 21.016 0.3574 26.352 0.2877 v 22.682 0.2723 28.544 0.2065 w 22.829 0.3404 25.668 0.1926 Average 0.3458 25.153 0.2491 17 4 0noise p 25.514 0.1968 24.721 0.2173 q 27.828 0.2189 25.838 0.3036 r 22.053 0.3631 25.696 0.1301 u 19.065 0.2817 27.123 0.2085 v 26.530 0.1977 29.444 0.1447 w 23.315 0.2661 25.853 0.1350 Average 0.2523 26.446 0.1899 14 2 YetthemajorproblemsthatSOMcannotovercomeare:theproliferationofprototypevectorsinparticularinhighdimensionalspaces,thesensitivitytonoise,

PAGE 50

thenon-smoothswitchingbetweenmodelsbecauseofthewinner-takes-allmecha-nism,andthelocalminimaentrapmentduringSOMtraining[ 48 50 ].Nothingcanbedonefortherstproblemexceptincreasingthenetworksize,whichbringsalsoslowtraining,requirementoflargeamountsofdata,andmanylocallinearmodels.AlthoughtheSOMisinsensitivetonoisebelowthequantizationproducedbythevoronoitessellation,whenthenoiseishigherthewrongmodelcanbeinstantiated.TheSOMpreservesneighborhoodssothedierenceisnotdrastic,butcontributestomodelingerror.BecauseofthedierencebetweentheLLMs,whenthemodelsswitchtheremaybetransientsintheresponsethatalsoaddtothemodelingerrorformorethanonetimestep. TheGMMapproachdividesstatespaceincoarserregions,whichimprovesthepoorscalingupoftheSOMwithdimension,andalsoimprovesthenoiserobustness.However,eachregionmaynotbeaswellmodeledbyalinearmodelasinthecaseoftheSOM.However,onlyexperimentalevidencecanjudgethistrade-o.Insteadofusingwinner-takes-allcriterion,GMM-LLMcombinesmultimodelsintothecurrentdata.InthecasetheEuclideandistancebetweendataandtheclosestGaussiankernelislarge,maximumlikelihoodcriterionmakessuremorethanonekerneltakeseect.Thisphenomenonisshowningure 3{3 .Also,gure 3{3 showsthatcomparedwithotherlinearmembershipfunctions,theuseofnonlinearGaussiandistributioncanguaranteesmoothswitchingbetweenmodels.AnimportantcharacteristicoftheGMM-LLMisthattheweightingischangingduringtesting,unliketheSOM-LLM.Thiswillhelpintheoverallmodelingaccuracy. ThelocalminimumintheSOMtrainingisdiculttocontrolduringthean-nealingprocess.AlsotheSOMcannotcontrolthewinningfrequencywhichmay

PAGE 51

Figure3{3. SoftcombinationregionusingGMM(betweendashline) alsoprovideaddeddicultiesintheclustering.TheGMMisalsosensitivetolocalmaxima.However,inordertomakeGMMconvergetoglobaloptimal,inamorestablemanner,G-SOMmakestheweightssuchthateveryclusterisgeneratedwithsimilaramountofwinningfrequency[ 51 48 ].Thismeansthatforthewholetrainingdataset,eachGaussiankerneloccupiesaboutthesameportionasawinnerwithlargeoutputweight.Fritzke,Chinrungrueng,andSequinprovedtheweightsgrowingcriterionandtheoptimalvalidityrespectively[ 52 53 ]. SincetheSOMalgorithmisderivedfromheuristicideasandthisleadstoanumberoflimitations,someoftheadvantagesofGMMoverSOMarelistedbelow. 1.GMMdenesanexplicitprobabilitydensityfunctionindataspacetoestimatethedistributionofthestate.Incontrast,SOMuseswinner-takes-allcrite-rionbasedontheEuclideandistance[ 18 ].GMM'ssoft-combinationmechanismmakessmoothswitchingpossible. 2.InGMM,theneighborhood-preservingnatureofthemappingisanautomaticconsequenceofthechoiceofGaussianfunctionswhichimprovesGMM'snoiserobustness.Smoothneighborhood-preservationisnotguaranteedbytheSOMprocedure.

PAGE 52

3.ThroughtherelativewinningfrequencycriterionfromG-SOMconstruction,theGMMtrainingavoidsthelocalminimaentrapmentwhichSOMalonecannotovercome. 4.GMMimprovesthescalabilityofSOMthroughacoarserclusteringofthestatespace.Forthesamereason,thetrainingeciencyofGMMisincreased.

PAGE 53

Withintheframeworkofpredictivecontrol,theproposedGMMbasedlocallinearmodelingapproachsimpliesthedesignofcontrolsystemsfornonlinearplants.Itisdiculttodesigngloballystablenonlinearcontrollerswithsatisfactoryperformanceateverypointinthestatespaceoftheclosedloopsystem,whichisespeciallytrueinthecaseofunknownplantdynamics.However,theGMMlocallinearmodelingapproachpresentedabove,coupledwithstrongcontrollerdesigntechniques[ 54 ]andrecenttheoreticalresultsonswitchingcontrolsystems[ 55 56 ],achievesthecontrolgoal.Thetopologyoflocalcontrollersthatnaturallyarisefromthelocallinearmodel-ingapproachisillustratedingure 4{1 ,wherethelocalcontrollersaredirectlylinkedtoeachlocalmodel.WithintheframeworkofGMM-LLM,thesystemdynamicscanbedescribedpiece-wise-linearlyas ~yn+1=XkkLTkXn =LTOPTXn=A0yn+A1yn1++ADynD+B0un+B1un1++BDunD 42

PAGE 54

Figure4{1. StructureofControlSystembasedonGMM-LLM Insection4.1,abriefreviewonmixturemodelbasedcontrollerdesignwillbegiven.Insection4.2,theGMM-baseddynamicalinversecontrollerisexplainedindetail,whichislaterdemonstratedfeasibleonnonlinearsysteminchapter5.Apole-placementPIDcontrolapproachwillbediscussedinsection4.3. Consideringmodelingerror,measurementnoise,andmodeluncertainty,robustapproachestocontrolbecomeveryappropriateforreal-worldapplications.Thereforevariousapproacheshavebeenproposedtosolvetherobustcontrolproblem,includingtheH1approach[ 57 ],Lyapunovapproach[ 58 ],geometricapproach[ 59 ],etc.Lin,Brandt,andSunproposedanoptimalcontrolbasedrobustcontrolscheme[ 60 61 ],andwewillmodifyitandtitintoourGMM-LLMframework. 62 63 64 ].Indeed,the

PAGE 55

modelconstitutesthevitallinkbetweenthephysicalprobleminwhichtheoptimalcontrolwillbeused,andthemathematicalrealminwhichitneedstobedesigned.Theclassicalapproachtothecontroldesignproblemassumesaknownmodel,evenifdramaticallyreduced.Inthecasethatthemodelcomplexityisreducedunreal-istically,theconsequencewillbethegenerationofineectivecontrols.Forglobalnonlinearmodelingcontrolapproach,whichismoredicultcomparedwithlocalorlinearmodelingcontrol,neuralnetworksbasedcontrolschemehavebeenstudiedextensively[ 65 54 66 ]. UsingthePartitionAlgorithm(PA),Lainiotisrstintroducedthemulti-modelpartition(MMP)methodologyforthedesignofadaptive/intelligentsystems,wherethecontrolproblemisdecomposedintoasetofeasierderivedsubproblems[ 64 67 ].Givenasetoflocalmodels,MMPdesignstheadaptivecontrolas where^uk(n)istheoptimalcontrolcorrespondingtokthsub-model. Fuzzycontrolisanotherpopulardesignmethodologywhichattractswideinterestthesedays[ 68 ].Amongthem,Takagi-Sugeno(T-S)fuzzylogicmodel,whichappliesfuzzyrulesandmembershipfunction,isawidelystudiedandappliedfuzzymodelingandcontrolmethodology.IntheT-Sfuzzymodel,thesystemisdescribedbyasetofif-thenruleswhichhavelocalmodelsintheconsequentparts.Theithruleinthemodelisoftheform:Ifz1(t)isMi;1,and:::,andzP(t)isMi;P,then

PAGE 56

Figure4{2. T-Sfuzzylogicmembershipfunction Theeectiveregionsofif-thenruleareoverlappedwitheachother,andtheweightsofeachruleismeasuredbylinearmembershipfunctionwhichisshowningure 4{2 .Thecontrolsignalforfuzzylogiccontrolstaketheformof( 4{2 ),wherekiscomingfrommembershipfunctiontoo.Tanaka,Iwasaki,andWanguseT-SlogicswitchingfuzzylogicmodeltocontrolofanR/Chovercraft[ 69 ]. Basedontheideaofagroupofsimplecontrollersinsteadofasinglecomplexcontroller,switchingcontrolhasbeenappliedinrecentyears.Aswitchingcontrolconsistsofahighleveldiscrete-eventsupervisorandafamilyoflowlevelfeedbackcontrollers[ 70 7 ].Followingthelocalmodelingtheorem,multiplecontrollersareappliedtocertainregionsofdata.MotterusedaSOMbasedsystemidenticationmodeltodesignaswitchingcontrollerforhighspeedwindtunnelcontrol[ 20 ].LiandLeeuseaSOMstructuretoclusterthestatespace,andthenapplyfuzzyrulestodesignthecontrolsignals[ 41 ].Choetal.usesaSOMandslidingmodecontrollertorealizerobustcontrol[ 71 72 ].Basedonthoseapproaches,ourresearchwilltry

PAGE 57

Figure4{3. StructureofInverseControlSystem toimprovetheperformanceofmultiplecontrollerneartheintersectionofdierentclusters. 4{3

PAGE 58

Theprinciplebehindinverseerrordynamiccontrol(IEDC)istheasymptoticconvergenceofsystemerror.Thecontrolmethodologycanbedescribedasxingasetofstablepolesforthetrackingerrorsignaldynamics.Ifattimenthedesireplantoutputisdenotedasdn,andtheactualplantoutputisyn,theinstantaneouserrorisdenedasen=dnyn.Andthecontrolgoalistoguaranteethattheerrorsignalfollowsthel-Derrordynamicfunction: theparametervector~=[1;:::;l]Tisselectedsuchthattherootsofthepolynomial1+1x++lxl=0areallwithintheunitcircleandthustheerrordynamicsisstable. Iftheorderofpolynomialiszero,theerrorfunctionissimpliedasen+1=0.Theerrorfunctioncorrespondstoanexacttrackingcontroller,whichdeterminesthenecessarycontrolinputforthenexterrorvaluetobe\0".Thisstrategyusesonlycurrentmodelinformationandcurrentcontrolerror,soitsresponsewillbeinstanta-neous.Andobviouslyitisnotrobusttomodelingerrorsorexternaldisturbancesasthereisnofurthermodelanddynamicinformationtosmooththedisturbancefrommodeluncertainty.Through2ndordererrordynamicalfunction,theerrorsignalis

PAGE 59

lteredrst,andthelocalmodelbasedcontrolsignalcanbeobtainedas wheremodel[~Ln;x~Ln;u]T=~LTcurrent=(PTn~L)Tiscomingfrom( 3{14 )and(1en+2en1)isfromweightederrorsignalsoflasttwosteps. Fordierentchoicesorderin( 4{3 ),thecontrollawwilldrivethetrackingclosed-loopsysteminvariousways,meanwhiletheerrorsignalwillalwayssatisfythelineardierenceequationgivenin( 4{3 ).Aslongasthelocationofpolesarewithintheunitcircle,thecontrollerwillguaranteethestableconvergenceofthedynamics.ConsideringthatthemodelingerrorfromtheGMM-LLMwillaecttheactualpolelocations,thepolesin( 4{4 )cannotbechosenclosetooneinordertoguaranteestability. Nowwediscussthestabilityissueforthecontroller.From( 4{4 ),isareco-ecientstobedesignedsuchthattwopolespisarewithintheunitcirclewherep1+p2=1andp1p2=2.SincetheLLMsaretrainedbasedonMSEcriterion,themodelingerrorsateachstepcanbedeemedaboundedi.i.d.randomvariablewithzeromean.ItcanalsobeassumedthatE[e2M;n]<1,whichisreasonableifthesystemidenticationisdoneproperly,andthemeasurementnoiselevelsarenite.

PAGE 60

Fromtheexperimentalresultsinthesimulationsection,wewillshowthatthemod-elingerrorpowerisnite.Underthoseconditionsandassumptionsaboutnonlinearuncertaintiesonens,theerrordynamicfunctionischangedto whereeM;isarethemodelingerrorsateachstep. Wedenetherightsideofequation( 4{5 )as.Consideringtheworstcasewhenmodelingerrorsreachtheirbound,issettoitsupperboundu(jjjujanduisaniteconstant).TheexpressionofencanbeobtainedthroughZtransformandinverse-Ztransformtoyield Asn!1,themeanofthetrackingerrorisobtainedas limn!1E[en]=limn!1E[u (4{7) Asforthepoweroftrackingerror,theexpressioncanbewrittenas limn!1E[e2k]=limn!1E[2u (4{8)

PAGE 61

Undertheassumptionthatthemodelingerror(andmeasurementnoise)con-tributionseM;narewidesensestationaryrandomvariables,wecancomputetheasymptotic(nite)trackingerrorintermsofthepowerspectraldensityofeM;nandthecoecientsi.Asdiscussedbefore,themeanofmodelingerroriszeroorsmallvalue.Consequently,themodelingerrorwilltakeeectontrackingperformanceandthepole-placementproblemmustbesolvedconsideringthetradeobetweenconvergencespeed(fasterpoles,widerpass-band)versusbetterdisturbancerejection(slowerpoles,narrowerpass-band). 73 ].Lately,Skoczowskietal.proposedanewmethodbasedontwo-loopmodelfollowingcontrol(MFC)toimproverobustness[ 74 ].ThemixturemodelbasedPIDcontrolschemeisshowningure 4{4 Simplifyingthemodelfunction( 4{1 )toasecondordertime-embeddedSISOform ~yn+1=a1yn+a2yn1+b1un+b2un1 z2a1za2

PAGE 62

Figure4{4. pole-placementPIDcontrolsystem. DeningthedesiresignalandcontrollertransferfunctionasD(z)andG(z)=D(z)=C(z)respectively.Theclose-loopcontrolsignaltransferfunctionU(z)isgivenby Fromthedenitionofpole-placement,thegoalofthisapproachistomapthecoecientsofclose-loopsystemfunctionfromopenloopcoecients(a1;a2;b1;b2)topredesignedcontrollercoecients(ci;di),whosecharacteristicpolynomialcouldbedescribedas Theclose-loopsystemtransferfunctionisthen1 [1+zA(z)C(z)=B(z)D(z)],andthesystemcharacteristicpolynomialbecomes

PAGE 63

Usingthecharacteristicpolynomialforthesystemmodelas( 4{10 ),alongwithasecondorderobserverpolynomialtermz2,thedesignequationforpole-placementPIDisgivenas (4{12) wherea(z1)factorisaddedtothedenominatorofcontrollerfunctionC(z)inordertolterthelow-frequencynoise.Thisequationcouldbesolvedforcisanddis.Consequentlythecontrolsignalisobtainedfrom( 4{12 )as GiventheLLMonthecurrenttimestep,thePIDcontrollercouldbeeasilyobtainedthrough( 4{13 ).Itsstabilitywillbeassuredaswediscussedinprevioussectionforthedynamicinversecontroller.OnepossibleproblemcausedunderthisframeworkisthatthePIDcoecientsneedtobecalculatedoneverystepasthesystem'slocalmodelandthegatingweightsfromGMMarechanged. 60 61 ].

PAGE 64

_x=A(x)+B(x)N(x)+B(x)k(x)(4{14) isgloballyasymptoticallystableforalladmissibleperturbationsN(x). 75 ]gives minu[N2max(x)+xTx+uTu+VTx(x)(A(x)+B(x)u)]=0

PAGE 65

whereVx(x)=@V(x)=@x.Thus,ifu=k(x)isthesolutiontotheoptimalcontrolproblem,then (4{15) SinceV(x)satises Also,_V(x)=dV(x)=dt<0forx6=0,because _V=(@V(x)=@x)T(dx=dt)=VTx(x)[A(x)+B(x)N(x)+B(x)k(x)]=VTx(x)[A(x)+B(x)k(x)]+VTx(x)B(x)N(x)=N2max(x)xTxk(x)Tk(x)+VTx(x)B(x)N(x)=N2max(x)xTxk(x)Tk(x)2k(x)TN(x)=N2max(x)+N(x)TN(x)(N(x)+k(x))T(N(x)+k(x))xTx(N2max(x)N(x)TN(x))xTxxTx<0 ThustheconditionoftheLyapunovlocalstabilitytheoryaresatised.Theequi-libriumx=0of( 4{14 )isgloballyasymptoticallystableforallpossibleuncertaintyN(x),i.e.u=k(x)isasolutiontotherobustcontrolproblem.Consequently,thereexistsaneighborhoodN=fx:kxk0suchthatifx(t)entersN,

PAGE 66

then limt!1x(t)=0(4{16) Butx(t)cannotremainforeveroutsideN,otherwisekx(t)kcforallt0.There-fore, 4{16 )istruenomatterwherethetrajectorybegins[ 60 ].Consequentlythestabilityisassuredwithoutconsideringtheinitialcondition.

PAGE 67

Thecomputationaltechnique,whichwasoriginallyproposedbyYashikiandMatsumoto[ 76 ]onSISOsingledelaysystemandlaterexpandedbyKleinandRamirezetal.[ 77 ]onMIMOmultipledelayssystem,needstobetransformedtoourone-steppredictionmodelintostatevariableformatforthepurposeofcontrollabilitychecking.ThegeneralcaseofMIMOsystemsexhibitingmultipledelaysiswrittenas( 4{1 ).AndthecorrespondingstatespacemodelforMIMOsystemcanbeexpressedas =0BBBBBBBBB@00A1............00ADIIA01CCCCCCCCCA0BBBBBBBBB@y1;n...yD;nyn1CCCCCCCCCA+0BBBBBBBBB@B1...BDB01CCCCCCCCCAUn

PAGE 68

Amainadvantageofthistransformationisforthecontrollabilitytest.Dening~Aand~Basthecoecientmatrixandvectorin( 4{17 ) ~A=0BBBBBBBBB@00A1............00ADIIA01CCCCCCCCCA;~B=0BBBBBBBBB@B1...BDB01CCCCCCCCCA rank(S)=rank[~B~A~B~AD+Pdi1~B] Inourexperiments,alltheLLMgivenon-zeroresults,andsincetherankofSisrelatedtothedimensionofLLMisrelativelylow(S=[~B;~A~B]for4-DLLMcase),thecontrollabilityisassured.OnlyinthecasewhenLLMhavealargepartofclose-to-zerocoecients,whichalsomeansLLMarenotwelltrained,thestatematrixwillnotbefullrank. Torealizethedesignofrobustcontrolsignal,westartthecostfunctionJasthebasicdesignrule: Consideringthemodelinguncertaintyandmeasurementnoise,( 4{19 )canbemodiedas (4{20) =Xn(XTnQ1Xn+UTnRUn)

PAGE 69

Herethemodelinguncertaintyandmeasurementnoisearecombinedwiththestateitself.Inotherwords,thephysicaldynamicalchangesbecauseofthemodeluncertaintycanberecognizedastheincrementofsystemdynamicswhichwillinturncostmoreenergyandlargercontrolsignaltoovercomethem.Thisisareasonableexplanationasmodelingerrorappearsasawarpingofsystemstates.Thesolutionfromthiskindoffunctionformsthetraditionallinearquadraticregulator(LQR)problemandcanbeobtainedthroughthesolutionfromthediscretealgebraicRiccatiequation(DARE) TherehavebeenseveralmethodstocomputetheuniquestabilizingnumericalsolutionP=Xofthediscrete-timealgebraicRiccatiequation[ 77 ],fromwhichthegainvectorandconsequentlythecontrolsignalarecomputedas whereXnisthere-formulatedcurrentinputwithpseudostateelements,AandBarefromstatespaceformofrealsystemmodelor~Aand~BfromstatespaceformofLLM.Theschematicdiagramoftheoptimalrobustcontrolsystemisshowningure 4{5 Onepossibledisadvantagefortheoptimalrobustcontrolleristhattherelatedcomputationislargerthanthoseofinversecontroller,PIDcontroller,etc.Especiallyundertheconditionofmixturemodel,theoreticallytherewillalwaysbeachangeonthelocalmodelassystemdynamicsarechanging.SinceDAREcomputationneeds

PAGE 70

Figure4{5. Optimalrobustcontrolsystemschematicdiagram. tobenishedrightafterthegenerationofGaussianmixturemodel,consequentlytheDARE( 4{21 )andthecontrolgainmatrix( 4{22 )needtobere-calculatedateverytimestep,whichmightbeachallengeundersomeon-linetrainingenvironment.Necessarytrade-osneedtobemadewhenresourcesarelimited. 4{20 )needstobefurthermodied

PAGE 71

forstateerrorsandchangeofcontrolsignals (4{24) =Xn(ETnQ1En+dUTnRdUn) whereEnisthedynamicalerrorofsystemstates,dUnisthedierenceofcurrentcontrolsignalsandtheirpreviousrecord,QandRaresymmetricandpositivesemi-deniteweightingmatrix.Again,weassumethemodelinguncertaintyandnoisecanbecombinedwithstateerrorssinceerrorscanbedescribedasdistancebetweenstatesandnon-zeroequilibrium.Correspondingly,thetransfercoecientsforerrorsignalandcontrolsignalsneedtobechanged.Startingfromthesystemstatetransferfunctionin( 3{14 ),theerrortransferfunctionscanbedescribedas ~yn+1=a1xn+a2xn1+b1un+b2un1en+1=dn+1yn+1dn+1~yn+1=dn+1(a1xn+a2xn1+b1un+b2un1)=dn+1a1(dnen)a2(dn1en1)b1unb2un1=(dn+1a1dna2dn1b2un1)+a1en+a2en1b1un=a1en+a2en1b1(un1 (4{25) Dening^un=1

PAGE 72

Consideringthemodelinguncertainty,thenone-zerosetpointcontrolandtrack-ingcontrolproblemscanberealizedthroughoptimalcontrollaw( 4{24 ).ThenewdiscreteRiccatiequationcouldbecalculatedinthesameformwithmodiedcoef-cients(A;B)in( 4{26 ).WhenthegainmatrixfromdiscreteRiccatiequationiscalculated,thecurrentstepcontrolsignalcanbeexpressedas In( 4{24 )and( 4{27 ),weusetheupperboundofmodelingerrortodescribethemodeluncertainty,andtherealthemodeluncertaintyandnoisewillbearan-domvalueswhichonaveragearesmallerthanthat.Consequentlyundersmallnoiseandmodeluncertaintysituations,thetransientperformancefromoptimalcontrollers,comparewithothercontrolapproaches,mightnotbethebest.However,sinceopti-malcontrollerisconsideringthelong-termeectofdisturbance,itsperformanceonrobustnesswilloutrunalltheothercompetitorsinlargenoisecases.Inchapter5,wewillsimulateanddiscusstheperformanceofoptimalcontrollerondierentrealsystemswithdisturbance-freecaseandthecasewithnoiseand/ormodeluncertaintypresents.Neglectingthecomputationalcost,optimalrobustcontrolleristheoptimalchoiceformodelbasedpredictioncontroller.

PAGE 73

systemdynamicsasreferencemodel.UndertheconditionthattheGaussianmodel-ingsystemsarestable,allcontrollerscangiveastableresultwhereoptionalrobustcontrollerwilltheoreticallygivethebesttrackingperformance.Aplantmodelisrarelyafullyaccuratedescriptionoftherealplant.Neglecteddynamics,approx-imatelyknownparameters,andchangesinoperatingconditionsallcontributetoplant-modelingerrorsthatcanjeopardizecontrollerperformance.Inthefollowingchapter,wewillanswertheproblemsincludingtheperformanceofthecontrollersinrealsystems,whichincludesaccuracyandrobustness,performancecurvevs.numberofPE,andperformancecomparisonofGMM-LLMwithothermodelingapproaches. Amongalltheoptions,ESNcontrollerisquitedierentcomparedwithothers,whichwillbediscussedindetailsinappendix.ManypapersmentionedthatthetrainingofADCand/orBPTTalgorithmswillsometimeinevitablyleadtounstablecontrolresultsbecauseofitscomputationalcomplexity.Besidestheneuralnetworks(ESNhere),BPTTlengthandthechangeoflearningratewillalsotakeeectonthenalperformance.Wewillstartfromsomenonlinearsystemswith\simple"setup,useitforset-pointcontrol.

PAGE 74

TheGMMbasedlocallinearmodelingandcontroltechniqueswhichhavebeenpresentedinpreviouschaptersneedtobetestedonvarioussysteminordertoprovetheiradvantagesincludingfeasibility,simplicity,androbustness,etc.Asaninde-pendentcandidate,ESNwillbeusedforsystemidenticationandpartlybeusedforsystemcontrol.Inthischapter,dierentGMM-basedcontrolapproacheswillbetestedonasetofsystemincludingchaotictimeseriesprediction,syntheticSISOlinearsystemidentication,SISOnonlinearsystemidenticationandcontrol,NASALoFLYTEwave-rideraircraftidenticationandcontrol.Acompletesetofsimula-tionandexperimentalresultswillbeprovidedfromtheapplicationoftheseprinciplestothelistedproblems.Amongallthesystems,theLorenzsystem,asatraditionalchaoticsystemwithhighordernonlineardynamics,andthemissilesystem,asatra-ditionalSISOcontrolsystem,willbediscussedindetailsaboutsystemidenticationperformanceandcontrolperformancerespectively.Forallthesamplesystems,wewillassumethatthesystemtransferfunctionsareunknown,andthereferencemodelswillbeconstructedbasedonI/Omeasurements. 63

PAGE 75

(a)(b) Figure5{1. DynamicsoftheLorenzsystemLeft:x(top),y(middle),andz(bottom);Right:3-Ddisplay. canimprovesystempredictabilityandavoidfatiguefailureofthesystem.Severalmethodologieshavebeenappliedfortheidenticationandsynchronizationofavari-etyofchaoticsystems.Forexample,ageneralizedsynchronizationofchaosthroughlineartransformation[ 78 ],adaptivecontrolandsynchronizationofchaosusingLya-punovtheory[ 79 ],andanadaptivevariablestructurecontrolsystemforthetrackingodperiodicorbits[ 80 ].ThecontributionofourworkliesinthedesignofdierentcontrolsystemsbasedonGMM-LLMapproach. 81 ].TheLorenzsystemcanbedescribedas: _x=(yx)_y=xyxz _z=z+xy

PAGE 76

wherex,y,andzarethemeasurementsofuidvelocityandhorizontalandverticaltemperaturevariations.TheLorenzsystemcanexhibitquitecomplexdynamicsdependingontheparameters.Inourexperiment,parametersaresetas=10,=28,and=10 3toobtainchaoticdynamicsandtherststatexissetastheoutput.Using4thorderRunge-Kuttaintegrationwithtimestep0.01,9000samplesweregeneratedforanalysis.UsingtheLipschitzindexanalysistheembeddingdimensionisdeterminedtobe3andtheembeddingdelay=2.TheGMMtrainingdataisconstructedas Therst8000samplesfromthetime-seriesareusedtocreatethereconstructedstatespacethrougheight-kernelGMMalgorithm,andlocallinearmodelsaretrainedforone-stepprediction.Another1000samplesareusedforsignaltoerror(SER)performancetest 5.1.1 showsthattheestimatedoutputisquiteclosetotheoriginalLorenzsignal.Andgure 5{3 showstheoptimaldistributionofGaussianker-nelamongthestatespace.InordertofurthercomparetheidenticationperformancebetweenGMMandotherapproaches,weplottheguretoverifytheperformanceincrementofGMM-LLMasnumberofPEincrease.Fromgure 5{5 andtable 5{1 ,wendthat\soft-combination"enableGMMtousefewerPEs.Thetotalamountof

PAGE 77

(a)(b) Figure5{2. Systemidenticationperformanceonthe\x"oftheLorenzsystem.Left:usingGMM:Originalandidentiedsignals(top)andcorrespondingerror(bot-tom);Right:usingESNmodel(top)andcorrespondingerror(bottom). parametersinSOM-LLMandGMM-LLMapproachescanbeexpressedas Inordertomakeacompletecomparison,alongwithtraditionalmethodsRBFandTDNNastworeferences,two300-PEESNsareconstructed.OneESNisdesignedwithevendistributionofspectralradius[ 36 37 ],theothercomeswithoptimizedde-signcriterionwherelargerpartofinternalweightsdistributedalongthemaximumspectralradius.BothusethesameMSEtrainingcriterion.Intable 5{1 weseethatESNcanmakebetterone-steppredictioncomparedwiththeotherapproaches.Since

PAGE 78

Table5{1. PerformanceontheLorenzsystem Approaches(#ofPEs) #ofparameters SER(dB) RBF(50) 203 31.05 TDNN(50) 250 29.23 SOM(1010) 600 33.67 GMM(8) 80 29.16 ESN1(300,1Dinput) 300 35.05 ESN1(300,3Dinput) 300 43.40 ESN2(300,1Dinput) 300 36.54 ESN2(300,3Dinput) 300 45.20 Figure5{3. DistributionoftheLorenzsystem(dashline)andGMMweights(star) theESNtrainingisfocusedonthelinearreadoutvector,thetrainingforlargersizeofESNcanstillberealizedwithinreasonableamountoftime.Aninterestingphenom-enonintheresultisthatESNrelylessontheembeddingdimensioncomparewithotherapproaches,asonedimensionalinputalreadygeneratecomparablepredictionperformance.Andthatcomplieswithourtheoreticalexplanationinchapter2and4. OncetheGMM-LLMstructureisoptimizedaccordingtotheproposedmethod-ology,thecontrollerdesigncanbeaccomplishedusingstandardlinearfeedbackcon-trollerdesigntechniques.SincetheLorenzsystemisoriginallyanautonomoussystem,

PAGE 79

Figure5{4. Weightschangeof8GMMkernels; thecontrolcouldberealizedfordynamiccompensation.Givenadesiredsignaldn,thecontrollerneedstondacontrolsignalunsuchthatthesystemoutputerrorconvergestozero.Undertheconditionthatmodelpredictioniscloseenoughtotherealdynamics,theLorenzsystemwillfollowthecontrolsignal(s)totheequilibriumpoint. limn!1jdnynj=0(5{3) FortheLorenzsystem,the3-dimensionalcontrolsignalvectoris (5{4) whereonecontrolsignalcorrespondstoonestatevariable,andthepredictedstatevector~xn+1isobtainedfromGMM-LLMprediction.Tistheerrorcompensationcoecientbetween[-1,1],whichconrmstheerrorconvergestozero.Followingthe

PAGE 80

Figure5{5. PerformancecomparisonbetweenSOMandGMM (a)(b) Figure5{6. CompensationperformanceLeft:stabilizingx(top),y(middle),andz(bottom);Right:synchronizingx,y,andz(upperthree).Controlsignalsstartfromdatapoint1000forbothcases. conditionofdesiresignalsasequilibriumpointsoratotallydierentdynamics,wecanrealizeastabilizingandsynchronizingcontrol.Figure 5{6 showsstabilizingandsynchronizingperformancewherethecontrollertakeseectatpoint1000.Wefoundthatforbothcases,thecontrollercancompensatethedynamicerrorveryfastandguaranteestheconvergence.OnefactorneedstobeemphasizedhereisthattheLorenzsystem,beingachaoticsystem,isverysensitivetoappropriatemodeling.

PAGE 81

Otherwisethecontrollerwillnotconvergethedynamicstoequilibriumincasethesystemidenticationperformanceisnotcloseenoughtotherealvalues. x+_x+(x3!0x)=cos(!t+)(5{5) Dependingontheparameterschosen,theequationcantakeanumberofspecialforms.Forexample,withnodampingandnoforcing,==0andtakingtheplussign,theequationbecomes x+!0x+x3=0 Thisequationcandisplaychaoticbehavior.For>0,theequationrepresentsa\hardspring",andfor<0,itrepresentsa\softspring".If<0,thephaseportraitcurvesareclosed[ 21 ]. Withoutlosinggenerality,wesimplifythedungoscillatorwithcontrolsignal.Thesystemfunctioncanbere-writtenintostatespaceformatas (5{6) withcorrespondingconstantcoecients[;!0;;]=[0:4;1:1;1:0;1:8]and!=1:8[ 82 25 21 ],andthecontrolsignalissetasjuj5. Using4thorderRunge-Kuttaintegrationwithtimestep0.2s,8000samplesweregeneratedforanalysis.TheGMM-LLMistrainedwiththerst5000points,andthe

PAGE 82

Figure5{7. DynamicsoftheDungoscillatorasanautonomoussystem.Left:Statespacedescription;Right:Timeseries. Table5{2. PerformanceontheDungoscillator MODEL(#ofPEs) #ofparameters SER(dB) RBF(50) 254 27.94 SOM(64) 512 28.06 ESN(300) 300 30.75 GMM(8) 104 26.46 remainingisusedformodelingtestingandcontroltrackingtesting.TheembeddingdimensionisdeterminedasXk=[yk;yk1;uk;uk1]T.Fromexperiments,thesizeofGaussianmodelsissetaseightPEs.ForESNpart,thesameamountofdataisusedforbothtrainingandtesting.Thedataembeddingissetas\1"dimension(Xk=[yk;uk]T),and300internalweightsareusedforechostates. Theidenticationperformanceondungsystemisshowningure 5{8 ,andsam-pleofGaussiandistributionisshowningure 5{9 .TheirSERperformancelongwithcomparisonwithsinglemodelperformancearelistedintable 5{2 .Consideringthestructuraldierence,wewillmainlycompareGMM-LLMwithRBFandSOMbasedmodel.WendthatGMMisusingfarlessamountofmodels,whichmeansasimplerstructurecomparewithothermodelingparadigm,andthepredictionperformanceis

PAGE 83

Figure5{8. SystemidenticationperformanceontheDungsystem Left:GMM-LLMresults;Right:ESNresults. stillverygood.Figure 5{9 showsthatforalargepartofthetesting,soft-combinationistakingeectamongtheGaussianmodels,whichcomplieswithourexplanationinchapter3.ESN,ontheotherway,isusinglessinputdimension,andthepredictionisoneofthebest.ModiedESN,fromnowon,willbeourprimaryoptionfortheechostatesetupasitisproducingbetterresultsthanitspreviousdesign. Next,wetrytorealizetwocontrolmissionsbasedonGaussianmodels.Therstsetssystemdynamicstothezeroequilibriumpoint;thesecondmakesthesys-temtrackadierentdynamics.Astheoriginaldynamicissetas[;!0;;]=[0:4;1:1;1:0;1:8]and!=1:8,thenewoneissetas[;!0;;]=[0:41;1:1;1:0;2:0]and!=1:9. Forthesecontrolmissions,weusethePIDcontroller,slidingmodecontroller,andoptimalrobustcontroller.Consideringthesignicanceoftheproblem,therobustnesscomparisonofdierentcontrollerwillbediscussedinthenonlinearSISOsystem.ForPIDcontroller,thepolesaresetasp1;2=0:6toensurestabilityandgoodconvergingspeed.Theparametersforslidingmodecontrolleraresetasc1=1;c2=2:5;eT=0:0001;qT=0:3[ 83 84 85 ].Fortheoptimalcontroller,thegainmatrixisdesigned

PAGE 84

Figure5{9. Sampleofweightschangeof8GMMkernels; as Wecomparetheperformanceofcontrollersusing200strajectoryregardingfallingtime,settlingtime,andsteadystateerror.BecauseoftherelativelylowSERsystemidenticationperformance,noneofthecontrollercanmaketheDungsystemdy-namicconvergetodesiretrackperfectly,asshowningure 5{10 where0s10sofdynamicsisshown.Roughlyspeaking,thecontrolperformanceofallthreecontrollersareclosetoeachother.Inthesetpointcontrolmission,however,optimalrobustcon-trollerandslidingmodecontrolclearlyhaveshortersettlingtimeandsmallersteadystateerroroverPIDcontroller.Regardlessrising(falling)timeandsettlingtimeinthesecondgure,thePIDcontrollershowslargeosetwheresystemdynamicsreachhighlynonlinearrange,e.g.atthestartandneartheextrema.Sincethemodelingfor

PAGE 85

Figure5{10. ControlperformanceofdierentcontrollersontheDungsystemTop:Setpointcontrol,systemdynamicstartsfrom1;bottom:Trackingcontrol,newtrackcomesfromadierentsystem. theDungsystemdoesnotconsidernoise,optimalrobustcontrollerdoesnotshowuniqueadvantagehere.Consideringtherobustnessofoptimalcontrolleroverdistur-bance,wecanexpectthatitwilloutperformothercompetitorsunderthesituationwheremodelinguncertaintyandnoisearebothpresent.

PAGE 86

5.2.1LinearSISOMass-SpringSystem _x1=1 _x2=1 86 ].Intheexperimenthereuislimitedwithin0:5,otherparametersaresetasm=5,k=100,andb=0:1.Thesampleofsystemdynamicsisshowningure 5{11 TheamountofGaussianmodelsischosenas6fromexperiments.The6-kernelGMMwastrainedwith5000samplesofinput-outputpairsobtainedfromequation( 5{7 )using4thorderRunge-KuttaintegrationwithtimestepofTs=0:05s,withubeingi.i.d.uniformlydistributedrandomsignal.Thesystemidenticationdatawasconstructedwithembeddingdimensionof4,accordingtoLipschitzindex,forinputandoutputbothbeing2.Thus~X(n)=[x1(n);x1(n1);u(n);u(n1)].ForESNmodeling,theinputdimensionisagaindecreasedto~X(n)=[x1(n);u(n)]. Theidentiedmodelsweretestedonoriginal2000samplesdata,generatedusinganewsequenceofrandominputsignal.Theactualplantoutput,themodelpredicted

PAGE 87

Figure5{11. Dynamicsofthemasssystem.top:X1;middle:X2;bottom:controlsignal. Table5{3. Performanceonthelinearmasssystem Model(#ofPE) #ofparameters SER(dB) SOM-based(88=64) 512 50.75 RBF(64) 324 47.30 GMM-based(6) 78 47.24 ESN(200) 200 49.05 outputandtheestimationerrorfortheGMM-basedLLMmodelsareprovidedingure 5{12 .Figure 5{13 showsthatcooperativemodelingistakingeectduringtesting.TheSERperformanceis47.2388dB.ThesamedataisappliedtoSOMandRBFtomakeacomparison.Table 5{3 showsthatGMM-LLMdoesnotgeneratethebestresult.YetconsideringtheamountofPEusedinGMM-LLM,GMM-LLMisahighlyecientmodelingmethod. Thecontrolmissionfortheplantcanbeaccomplishedthroughaseriesofcon-trollers.Thezeroorderinversecontrollerwilldirectlyreplacenextsteppredictionwithdesiresignal.ThePIDcontrollerdesigniscarriedoutinthewayofstandardpole-placementtechnique.ThecorrespondingPIDcoecientsaredeterminedto

PAGE 88

Figure5{12. SampleofsystemidenticationperformanceontheMasssystem.Left:GMM-LLMresults;Right:ESNresults. bringtheclose-loopresponsepolesto0:05+i0:3and0:05i0:3inordertodecreasethetransientovershot,whichareallveriedinthesimulationofcross-validation.Theslidingmodecontroller,asastrongcandidatefortherobustcontrol,isconsideredasareferencewithparametersc1=1;c2=2:5;eT=0:0002;qT=0:5,whicharene-tunedresultsfromsimulation.Fortheoptimalrobustcontroller,thegainmatrixaredesignedas Theset-pointcontrolmissionissetas:systemstartsatavalueof0.5,anddesiresignalis0for3second,thenthedesiresignalischangedto0.3.Inordertoidentifytherobustnessofdierentcontrollers,anothersetofexperimentwith30dBSNRnoiseisaddedtothesystemdynamicsasmodelinguncertainty.Thenalperformanceisshowningure 5{14 .Wecanseethatinthenoisefreecase,allcontrollershavesimilarsettlingtimeandsmallsteadystateerror.PIDcontrollerhasbigovershot,shortfallingtime.Slidingmodecontrollerhaslongerfallingtimeyetsmallovershot.Optimalcontrollerhasbothshortfallingtimeandsmallovershot.In30dBnoisecase,

PAGE 89

Figure5{13. Sampleofweightschangeof6GMMkernelsformasssystem allthreecontrollersdonothavemuchdierenceonfalling(rising)time,settlingtime,andovershot.Theoptimalcontrollerobviouslyoutperformothertwoonthecriterionofsteadystateerrorastheerrorvariancefromtheoptimalcontrollerissmallerthanthosefromothercontrollers. _x1=x20:1cos(x1)(5x14x31+x51)0:5cos(x1)u _x2=65x1+50x3115x51x2100uy=x1

PAGE 90

Figure5{14. Set-pointcontrolperformanceonthemasssystem.Left:noise-freecase;Right:30dBSNRnoisecase. Figure5{15. Missilesystemdynamics.Left:statespacedescription;Right:X1,X2,andcontrol(toptobottom). TheGMMwastrainedwith6000samplesofinput-outputpairsobtainedfromequation( 5{8 )using4thorderRunge-KuttaintegrationwithtimestepofTs=0:05s,whichcorrespondsto300secondsofighttime.Inordertoexcitearichvarietyofdynamicalmodesintheplant,thesystemidenticationdatawasgeneratedusingauniformlydistributedrandominputsignalwithinthespeciedlimits[ 2 ].GMM-basedLLMapproachdevelopedaeight-modemixturemodels,resultingineightcooperativelinearmodels.Theembeddingdimension,accordingtoLipschitzindex,forinputandoutputwerebothselectedtobe2,resultingin4-coecientlocalmodels.This

PAGE 91

Figure5{16. Sampleofsystemidenticationperformanceonthemissilesystem.Left:GMM-LLMresults;Right:ESNresults. Table5{4. Performanceonthemissilemodel Model(#ofPE) #ofparameters SER(dB) SOM-based(88=64) 512 31.70 RBF(100) 504 33.03 GMM-based(8) 104 31.00 ESN(300) 300 36.25 embeddingdelaydimensionwasalsochoseninaccordancewiththedimensionalityofthestatedynamics. Theidentiedmodelswerealsotestedonoriginal50s-lengthdata(1000samples),generatedusinganewsequenceofrandominputsignal.Theactualplantoutput,themodelpredictedoutputandtheestimationerrorfortheGMM-basedLLMmodelsareprovidedingure 5{16 .TheESNisconstructedwith300PEs,andinputissettostatewithouttimeembedding.TheSERperformance,comparedwith31.7dBfrom64-PESOMintable 5{4 ,is31dB[ 18 ]. Thecontrolmissionfortheplantcanbeaccomplishedthroughaseriesofcon-trollers.Thezeroorderinversecontrollerwilldirectlyreplacethenextstepprediction

PAGE 92

Figure5{17. Sampleofweightschangeof8GMMkernelsformissilesystem withthedesiredsignal.ThePIDcontrollerdesigniscarriedoutinthewayofstan-dardpole-placementtechnique.ThecorrespondingPIDcoecientsaredeterminedtobringtheclose-loopresponsepolesfromtheplantoutputto0:5+i0:1,and0:5i0:1.Theslidingmodecontroller,asastrongcandidatefortherobustcontrol,isconsideredasareferencewithparametersc1=1;c2=1:85;eT=0:0001;qT=0:3[ 83 84 85 ].Fortheoptimalrobustcontroller,thegainmatrixaredesignedas Aswediscussedinpreviouschapter,controlperformancewillbedeterminedbymodelingperformanceaswellascontrollerperformance.Undertheconditionthatthecontrollerisstable,thecontrolperformancewilleventuallyconvergetotheminimumofmodelingerror.Inordertodelivertheresultsmoreclearly,weshowtheguresin

PAGE 93

Figure5{18. PerformanceofPIDcontrollerwithdierentmodelingapproaches. thefollowingsequence:weshowthedierent-model-same-controllerresultsrsttoverifytheeectfrommodelingpart,thenthesame-model-dierent-controllerresults. Figure 5{18 displays,undernoise-freecondition,theset-pointcontrolperfor-manceofPIDcontrollerwithdierentmodelingapproaching,plusTDNNcontrollerasareference.Recallingthesystemfunction( 5{8 ),theregionaround-1.5iswherenonlineardynamicsistakingthelargesteect.Andgure 5{18 clearlyshowsthat,1).thesinglemodelPID(PID-MA)haslongersettlingtimeandthelargestvibra-tionbeforetheconvergencebecauseofitsmodelingperformance;2).TDNN,duetoitsstructuraldierence,hastheshortestfallingtimeyetthelongestsettlingtime;3).theGaussianmixturemodelPIDcontroller(MPID-GMM)hasverycloseper-formancetotheSOMmultiplemodelPIDcontroller(MPID-SOM);4).MPID-SOMgivesthemoststableperformanceanditneedsmoretraining.Fornowwecancon-cludethatGMM-LLM,whencomparedwithothermodelingapproaches,cansavemuchcomputationalcostandthenalcontrolperformancetoalargeextent.

PAGE 94

(a)(b) (c)(d) Figure5{19. GMM-LLMbasedcontrolperformanceonthemissilesystem.(a):Stepresponsewithoutnoise(outputanddesire);(b):Stepresponsewithnoise(outputanddesire);(c):Trackingresponsewithoutnoise(outputanddesire);(d):Trackingresponsewithnoise(outputanddesire). Nowwetesttheperformanceofcontrollersundertheconditionthatmodelinguncertaintyandmeasurementnoiseispresent.Theclosed-loopnonlinearsystemistestedintwocases:onewithvariousstepchangestothedesireoutputandonewithasmoothlychangingdesiredoutput,whichisthesumofseveralsinusoidswithdierentamplitudes,phasesandfrequencies.Tosimulatetheworstcondition,thedisturbanceisconsideredmodelinguncertainty.Aftergeneratingthetrainingdata,weadd30dBSNRnoisetothesystemmodelduringtesting.Inthatcase,modeling

PAGE 95

Table5{5. Controlperformancecomparisononmissilesystemconsideringnoise Set-point Set-point Tracking Tracking ErrorMean ErrorVariance ErrorMean ErrorVariance MPID-GMM -0.0289 0.0272 0.0070 0.0108 Optimal -0.0222 0.0186 0.0048 0.0072 Slidingmode -0.0272 0.0202 -0.0024 0.0101 (a)(b) Figure5{20. ESNcontrolperformanceonthemissilesystem.(a):Noisefreeset-pointperformance;(b):Set-pointperformancewith30dBnoise. eectisnegligibleandcontrolperformanceisemphasized.Fromgure 5{19 itshowsthatthestepandtrackingperformanceofallthedesignednonlinearcontrolschemesinclose-loopoperationwithactualplantmodelisverysatisfactory.GMM-LLMmodelingerrorisinsignicantinthenalperformance.Comparingwithothertwoapproaches,PIDneedstheleastcalculationforcoecients.Consideringthedetailofgure 5{19 ,asintable 5{5 ,thedierenceamongthoseapproachesarenoticeable,inwhichtheoptimalrobustcontrollerdemonstratesthesuperiorperformancefordisturbancerejectioncomparedwithothers. Finally,wetrytheESNcontrollerforset-pointcontrolinordertoverifythefeasibilityofESNcontroller.Followingthestructureinchapter4,weconstructa300-PEESNcontrollerwithrandomreadoutvector.Theinputtothecontrolleristhecurrentsystemstateswithouttimeembedding,andthemodelinformationis

PAGE 96

derivedfromtheGaussianmixturereferencemodel.BPTTlengthissetto100step,whichcorrespondsto5secondinrealsystem.Thewholesystemisrecursivelytrainedin15iterations.Thenalcontrolperformanceisshowingure 5{20 .Inthegure,wecanseethattheESNcontrollercanaccomplishtheregularcontrolmission,yetitsperformanceisnotcomparabletotheoptimalrobustcontroller.SincetheadvantageofESNisitsgeneralizationcapacity,simplicityfortraining,anditsindependencyoftimeembeddinginput,furtherinvestigationneedstobemadeforESNbasedrobustcontroller.

PAGE 97

Figure5{21. Generaldescriptionofanaircraft ghtagainsttheshockwaves.ThewaveridershapeimprovesfuelconsumptionbydecreasingairresistanceatspeedsgreaterthanMachone.Thisfullscaleaircraftwilltakeohorizontally,thenitwilluseair-breathingenginestoacceleratetoacruisingspeedofMachveataveryhighaltitude.Anditwillenditsightbylandingonaconventionalrunway.ThetaskhereistodevelopmodelingandcontrolstrategiesforLoFLYTEbasedsolelyoninput-outputdata. Accordingtoclassicalaerodynamics,theightdynamicsofanyrigidbodyaredeterminedbymovementsalongandaroundthreeaxes:roll,pitch(longitudinalmotion),andyaw(lateralmotion).Besidesthestrongcouplingbetweensystemstates,themaincontroleectcanstillbeclassied.Theelevatoreisthemaineectforcontrollingthelongitudinalmotionstatevariables(pitchangle,,andpitchrate,q).Therudderrisprimarilycontrolsthelateralmotionstatevariables(yawangle,psi,andyawrate,r).Theaileronamainlycontrolstherollmotionstatevariables(rollangle,phi,androllrate,p).Finally,thethrottletlargelycontrolstheaircraft'slongitudinalspeed,andforsomeplanes,deectablethrustvectorsmightallowyaw

PAGE 98

(a)(b) Figure5{22. DynamicsoftheLoFLYTEsystem.Left:p(top),r(bottom);Right:Controlsignal,aileron(top),rudder(bottom). androllcontributionsfromtheenginepowerinsomecase.Undercertainsymmetryassumptionsfortheaircraftbody,thestatedynamicsoftherigid-bodyaircraftcanbedescribedasfollows _u=(wqvr)gsin+Fx=m _v=(urwp)+gcossin+Fy=m_w=(vp+uq)+gcoscos+Fz=m_p=[(IyyIzz)qr+Ixz(qrpq)+L]=Ixx_q=[(IzzIxx)rp+Ixz(r2p2)+M]=Iyy_r=[(IxxIyy)pq+Ixz(pqqr)+N]=Izz_=p+qsintan+rcostan_=qcosrsin_=qsinsec+rcossec

PAGE 99

whereu,v,andwarethespeedcomponentsoftheaircraftalongitsbodyaxesx,y,andzrespectively.p,q,andraretheangularspeedsalongthoseaxes.And,,andaretheEuleranglesthatdenetherotationmatrixbetweenthebodycoordinateframeandtheinertialcoordinateframe.Thegravitygisalongthedowndirectionoftheinertialcoordinate.Thethrottle,enginepower,andaerodynamiceectsgeneratetheforcesFx,Fy,andFzaswellasthemomentsL,M,andN.Them,Ixx,Iyy,andIzzaretheaircraftmassandmomentofinertiarespectivelywhicharedeterminedbytheaircraft'sgeometry. TheLoFLYTEprogramisanactiveighttestprogramattheAirForceFlightTestCenteratEdwardsairforcebase.TheLoFLYTEaircraftissimulatedusingasoftware,C++versionorMatlabversion,byACCandisassumedtobeclosetothetrueplant.Withoutlosingthegenerality,thethrottleisassumedconstantandthestatevariablesp,q,r,u,v,wareavailableforexternalmeasurement.Thegoalofthesystemidenticationandcontrolproblemistodeterminelocallinearmodelsfromfourinputs(aileron,elevator,rudder,andthrottle)tosixstatevariablesandtocontroltheminordertotrackadesiredtrajectoryofight. Inordertosimplifytheproblem,thelongitudinalmotionofthesystem,axisx(forward)andz(down)andpitchrateq,areconsidereddecoupledwithlateralmotions,consistingtheaxisy(right),rollratep,andyawrater.Insteadofdealing

PAGE 100

(a)(b) (c)(d) Figure5{23. SystemIdenticationperformanceontheLoFLYTEsystem. (a):desiredpandGMM-LLMprediction(top),error(bottom);(b):desiredrandGMM-LLMprediction(top),error(bottom);(c):ESNonp(top),error(bottom);(d):ESNonr(top),error(bottom). withsixvariableatthesametime,thewholesystemisbrokenupintotwoparts.AndtheproblemissimpliedtomodelingtheLoFLYTE'spandrdynamicsusinginput-outputdatawhichincludesaileron,rudder,p,andr.Figure 5{22 showsthedynamicsoftheLoFLYTEusingthecorrespondingcontrolsignals.Andlocallinearformatofsystemtransferfunctioncanbeexpressedas

PAGE 101

SampleofweightschangeofGMMkernelsfortheLoFLYTEsystem wherexn=[pn;rn]T,un=[a;n;r;n]T,Ai,BicomefromLLM~Wn.Thusthetrainingdataisconstructedas ThroughtheLoFlyteMatlabmodel,totally6000dataaregenerated.4000ofthedataareusedastrainingdata,2000areusedforsystemidenticationtesting,andthelast1000oftestingdataisusedasdesiresignalsforcontrollertesting.A10-clusterGMMistrainedbasedonthetrainingdataset,twosetsofLLMarecalculatedfromdatareferringpandrnext-stepsignalasdesiresignals.Correspondingly,one300-PEESNisconstructedforsignal\p"and\r"withcouplinginputandwithouttimeembedding.Themodelingperformanceislistedinthetable 5{6 .ComparingwithRBF,GMMisstillacomparableandcomputationecientmethod.Againweemphasizeherethatbecauseofitscomputationaleciency,GMM-LLMwillbringasimple,piece-wiselinearplatformfornonlinearrobustcontrolapproach.

PAGE 102

Table5{6. PerformanceontheLoFLYTEsimulationdata MODEL(#ofPEs) #ofParameters SERonp(dB) SERonr(dB) RBF(100) 908each 41.33 49.98 SOM(64) 1536 42.17 50.74 GMM(10) 330 40.09 50.58 ESN(300) 600 44.55 53.62 WenowstudytherobustcontrolproblemwithGMM-LLMlocalmodeling.BasedonGLL-LLM,wewillapplysecondorderinversecontrollerandoptimalrobustcontroller.Whenwedesignthecontroller,wewillassumetheminimumcouplingbe-tweenthelateralandlongitudinalmovement.Hereweperformasimulationtocontroltheroll-rate(p)andtheyaw-rate(r)oftheLoFlytebyaileronandrudder,settingtheelevatortozeroandthrottletocertainconstant.Thedesignofoptimalrobustcontrollerwillfollowthedetailinperviouschapter.Fortheinversecontroller,oncetheLLMsareavailableforpandr,andtheirdesirevalues,thesecondorderdynamicinversecontrollercanbeexpressedas

PAGE 103

error.Thecostfunctionsfortheoptimalrobustcontrollerforbothdimensionsaredesignedas Table5{7. ControlperformancecomparisonontheLoFlytesystemconsideringnoise Set-point Set-point Tracking Tracking ErrorVar.(p) ErrorVar.(r) ErrorVar.(p) ErrorVar.(r) Inverse 0.0802 0.0026 0.1833 0.0060 Optimal 0.0144 0.0002 0.0322 0.0018 Thecontrolmissionisdividedintoset-pointregulationwhichsetthesystemdy-namicstoasetofxedvalues,andarbitraryoutputtracking,whichcomesfrompartofthedatacollectioninprevioussection.Consideringtherobustnessperformance,30dBSNRnoiseisaddedtothedynamicsofthesystemasmodelinguncertainty.Thecomparisonofcontrolperformanceareshowningure 5{25 5{26 5{27 ,and 5{28 ,thedetailisshownintable 5{7 .Itisclearthatinthenoise-freecase,bothcontrollershavesimilartransientandlong-termperformance.Intherstveseconds,wendstrongcouplingistakingeectsuchthattwoparametersareconvergingataboutthesametime.Inthecaseofnoise,however,theoptimalrobustcontrollerhasobviousadvantageoversecondorderinversecontroller.Asshowninthecomparisongures 5{27 and 5{28 ,controlresultfromoptimalrobustcontrollerobviouslyhassmallererrorvariancewhen30dBnoisepresenttobothdimensionsasmodelinguncertainties.

PAGE 104

(a) (b) Figure5{25. ControlperformanceontheLoFLYTEsystem(set-point,noisefree).(a)Inversecontrollerperformance;(b)Optimalrobustcontrollerperformance.(trackofp(top-left),trackofr(top-right),controlsignala(bottom-left),andcontrolsignalr(bottom-right))

PAGE 105

(a) (b) Figure5{26. ControlperformanceontheLoFLYTEsystem(tracking,noisefree).(a)Inversecontrollerperformance;(b)Optimalrobustcontrollerperformance.(trackofp(top-left),trackofr(top-right),controlsignala(bottom-left),andcontrolsignalr(bottom-right))

PAGE 106

(a) (b) Figure5{27. RobustcontrolperformanceontheLoFLYTEsystem(set-point,30dBSNRnoise).(a)Inversecontrollerperformance;(b)Optimalrobustcontrollerperformance.(trackofp(top-left),trackofr(top-right),controlsignala(bottom-left),andcontrolsignalr(bottom-right))

PAGE 107

(a) (b) Figure5{28. RobustcontrolperformanceontheLoFLYTEsystem(tracking,30dBSNRnoise).(a)Inversecontrollerperformance;(b)Optimalrobustcontrollerperformance.(trackofp(top-left),trackofr(top-right),controlsignala(bottom-left),andcontrolsignalr(bottom-right))

PAGE 108

Localmodelingapproacheshaveattractedlargeinterestbecausetheyhavetheabilitytoexplainlocaldynamicsofanonlinearsystem,whichisdicultespeciallyinthecasewhennonlineardynamicalsystemcharacteristicsvaryfastthroughoutthestatespace.Inanumberofcases,especiallyundertheblack-boxapproach,localmodelinghasbeenproventobeamoreeectiveapproximationmethodologythanglobalapproaches. 97

PAGE 109

AnotherimportantfactorofthisschemeistheusageofGrowing-SOMsimilaralgorithmfortheinitialdistributioncalculationofGMM'sweights.WiththehelpofGrowing-SOM,therstbenetisthattheconvergetimeandspeedaregloballyoptimized.Andsecondly,sincetheGrowing-SOM'sweightsandGMM'smeanholdingthesamephysicalmeaning,GMMisprovidedaexibleclustergrowingframeworkwhichisextremelyusefulwhenaddingmoreGaussianclustersisnecessary. AmongseveralLLM-basedcontrollers,optimalrobustcontrollerisoneofthebestresultswecanobtain.Theproblemforreferencemodelbasedpredictioncontrolisthatwhenreferencemodelhaserror,thecontrolwillhaveerrorstoo.Especiallywhenmodelinguncertaintyandmeasurementnoisepresent,thecontrolperformancewilldecreasedramatically.Optimalrobustcontrollerconsidersthedisturbanceeectsinalong-termsense,thusdecreasethenoiseeectinthelargestway. 1.BesidestheconclusiononprecisionandrobustnessanalysisforcontrollerbasedonGMM-LLM,adaptivedisturbancecancelationapproachesshouldbefurtherexploredasmodelprecisionisalwaysthepriorconcerninourmodeling-controlframework. 2.ESNmodelingdoesnotneedembeddingatinput.YetESNcontrollertrainingisstillnotastraightforwardsolutionformanyproblems.StableconvergenceofESNcontrollertrainingwillbeaninterestingquestiontoask. 3.Optimallocalmodeling.GMMcandecreasequantizationerror,yetLLMstillneedtomakeatradeobetweentheeectiveregionandthecurseofdimension.

PAGE 110

Nonlinearmodelingneedmorecomplextrainingeortandithasmodeexiblemodelingcapability. 4.Othercontrollerdesignmethodology.Theaccomplishedcontrollerdesignfol-lowsthesamesoft-combinationcriterionfromGaussianlocalmodeling,whichistrainedthroughinputspace.Aninnovativeattemptcouldbemadetodesignthecontrollerthroughinput-outputspace.

PAGE 111

ThecurrentparameterdesignprocedureforESNfollowsJaeger'sstabilityen-hancedwithOzturk'sentropycriterion,whichsetsspectralradiuslessthanoneandmaximizeentropyfortheinternalweights.Theadvantageoftheabovemethodisitssimplicity,theonlymajorfactorswecancontrolare:thenumberofinternalweightsandthemaximumspectralradius.BasedonOzturk'sapproach,theeigenvaluesofESNwillbeevenlydistributedwithinthecirclewhichisdenedbymaximumspec-tralradius.Sincewedonothaveanyfurthercontrolontheinternalweights,theoptimalESNperformanceislargelylimitedtoitssizeanddistribution.Consideringthecomputationalcostonreadoutweightsandpossiblememorylimitoncertainex-perimentenvironment,theESN'ssizecannotbeincreasedbeyondreasonablelimits.Undertheconditionofstability,themaximumspectralradiusalsocannotbesettovaluelargerthan\1".ThusthereisonepossiblemodicationcanbemadetotheESNdesign,thedistributionofitsspectralradius. Accordingtothedenition,themaximumspectralradiusofESNcorrespondstoitsmemorylength.WhentheembeddinglengthrequirementislargerthantheESNmemorylength,itstartsto\forget"theinput.WhatwewanttoimproveistoincreasetheaveragememorylengthofESNandnottoincreasetherelatedtrainingcost. Fromgure 2{2 ,theweightswithlargestspectralradiuswithinESNonlyaccountforasmallportionofthewholeset.Wecanalsounderstandthoselocationsaspoles 100

PAGE 112

FigureA{1. DistributionofimprovedESNinternalweightswithmaximumspectralradiusis0.9,withunitcircleasareference. ofthelinearizedsystem.Sincethepolesofthetargetsystemareunknown,wefoundinexperimentsthatthemodelingperformancewilldecreasewhenthereisnot\reasonable"amountofESNpoleslocatednearthesystempoles.Weproposeherea\dominantpole"approximationtoESNdesign,i.e.wewillplacemostofthepolesnearthelargestspectralradius.Consideringthetargetsystemmighthaveseveralpoles(eigenvalue)distributedalongthemaximumvalue,andthemaximumpolesdominatetheinternalembeddinglength,largeramountofpolesclosetomaximumeigenvaluecouldbeagoodanswerforsystemidentication.Also,wefoundoutthatwhenalargeportionofESNpolesareclosetothemaximumspectralradius,theweightsentropywillnotdecreaseappreciablycomparedwiththemaximumentropyfromevenlydistributedpoles.Meanwhile,incasewherethedominantpoleisareasonableapproximation,ESNmodelingperformanceisalsoimproved.Comparing

PAGE 113

gure 2{2 andgure A{1 ,wecanndthattherearemorethan80%ofpolesingure A{1 equaltothemaximumspectralradius. TherealizationofsuchoptimizedESNweightsmatrixcouldbeeasilyachievedthroughmultiplerootdesign.First,wedesignaregularESN,thencalculatethepolynomialofitseigenvalues. Settingai=0,forasetofi(i>d)inordertomakethepolynomialhavemultiplecomplexrootswithidenticalmagnitudeanddierentphase.Andthenewpolynomialsn+a1sn1++adsnd=0willdeterminehowmanypolesarecomplexandtheirmagnitudesareclosetomaximumeigenvalue.InordertokeepthepolesofthenewpolynomialasthepolesofESN,were-arrangetheweightsmatrixas A{1 .Thenewmatrixwillholdthesameeigenvaluesfromthepolynomial,andsincewecandesigntherootsofpolynomial,we\design"theESN'sinternalweights.OnepossibleproblemforthemodiedESNdesignisthatitmaynotbeappropriateforalldynamicsystemmodelingproblems.Incaseswherethedominantpoleapproximationisvalidshouldyieldbetterperformance.Forinstance,whenthetaskislimitedbylongterminternalmemory,thenewcriterionisappropriate.Incaseswherethedesignrequiresmodeling

PAGE 114

ofdierenttimeconstants,theproposedESN'smodelingcapacitymightdecreaseversustheoldapproach.

PAGE 115

WehavediscussedthemodelingpropertyofESNinchapter2.ConsideringitsRNN-relatedmodelingcapability,ESNcanactasbothmodelapproximatorandcontrollerasshowningure B{1 .WhenESNisdesignedforsystemidentication,aswewillshowinthefollowingsections,itcanmakeshorttermpredictioneasierthanGMM-LLMframework.AndESN'smodelingperformancewillnotbeworsethanthoseofGaussianmodelingbecauseofitsstructuraladvantages.Intermsofmodeldescriptionsimplicity,ESNisnotanoptimalchoiceforsystemidentication.Underthelocalmodelingcontrolframework,LLMgivesthesimplestlineardescrip-tionofsystemdynamics.ThroughESNmodeling,however,itwilltakemuchmoreeorttoback-propagatethemodelinformationfromESNinternalweights.Insomecases,back-propagatedinformationmaydivergefromtheactualsystemdynamics,andthereferencemodellosesitslocalmodelingcapability.Mostofthetime,lowerorderestimationofthenonlinearmodelwillnotsatisfytheprecisionrequirement.Consequently,wewillfocusonESN-basedcontrollerdesigninthissection. CurrentlytherearelimitedpapersdiscussingapplicationsofESNtocontrol[ 32 33 ].Structurally,ESNcontrolcanbegroupedtoasetofneuralcontrol.Inbothpapers,theESNcontrolleristrainedwhendesirecontrolsignalisavailable,therebyreducingthecontrolproblemsettingtothatofthestandardmodelingproblem.Inotherwords,ESNistryingtomodelothercontrollerinsteadofcontrolthetarget 104

PAGE 116

FigureB{1. DemonstrationsystemwithESNreferencemodelandcontroller system.Thoughinsomeapplicationsinferenceofdesirecontrolsignalmightbedone,inmanyotherssuchinferenceiseitherinfeasibleorunreliable[ 34 ]. ModelbasedcontrolisarealchallengeforESN.ThemaindicultycomesfromtherequirementthatdesirecontrolsbeknownasusedforESNtraining.Thedesirecontrolvaluescansometimesbeinferredfromthedesireoutputsifatransformationfromthecontrolstotheoutputshasasingle-valuedinverseandtherelativedegreeone.However,theplantmodelmightnotbeinvertedanalytically.Furthermore,calculatingdesirecontrolsignalnumericallymightbetimeconsuming,brittleandnallyconvergetolocalminima.Generallyspeaking,determiningdesirecontrolsignalsshouldbereplacedbytheuseofback-propagationthroughtime(BPTT)tocomputethesensitivitiesofplantmodeloutputstothechangesofitscontrolinputs.TheincurredadditionalcomputationalcomplexitymayallbutnullifythemainadvantageoftheESN. ThealternativeschemeofindirectcontrolwithESNwouldplaceESNunderthemethodofderivativeadaptivecritics(DAC)suchasdualheuristicprogramming

PAGE 117

(DHP).DACisderivedfromadaptivecriticdesigns(ACD)whichincludesheuris-ticdynamicprogramming(HDP),DHP,andglobalizeddualheuristicprogramming(GDHP),etc.Prokhorovin[ 3 ]showedthatthereexistssimilaritybetweenaparticu-larformofBPTTandthepopularDACformulationforthegeneralneuralnetworkscase.ThesimilaritiesenableustocreateahybridBPTTandESNbasedDACinwhichDACprovidesderivativesofestimationfromthefuturetimesteps. WesketchanalternativeschemeofindirectcontrolwithESN.Withoutlineariz-ingthesystemfunction,manycontrolsystemscanbeexpressedincontrolsun: whereHisthevectorfunction,Gisthecontrolmatrix.Settingtheinstantaneousperformancemeasurementas( 4{19 )withQ>0;R>0,thenimplementablepara-meterizationofthecontrollermaybeexpressedusingESNwithoutputvectorasthefollowingproduct Thediscreteclose-loopsystemwillrunforward,intheBPTTway,withthesequenceofcontrolsignalsfungn=nmaxn=n0providedbytheESNcontrollerwithconstantinitialvectorWout.InordertotraintheESN'sreadoutvector,weemploythedualequationsasdesireESNoutput

PAGE 118

bygoingbackwardsfromn=nmax,wherewesettheendpointdesire;n+1,tothestartingpointn=n0.TheESNreadoutvectortrainingerrorscanbederivedfromESNoutputas TheerrorisfeedintoMSEtrainingcriteriontoupdatetheESNreadoutweights.Thewholeprocedureissetas:runningthesimulationforwardtocollectdata(systemoutputsandESNoutputs);goingbackwardtoupdateWoutwitherrors( B{3 )and( B{4 ),whichisrepeatedtilltheconvergeofWout.Preliminaryexperimentsindi-catethattheESNbasedderivativeadaptivecriticisviable,especiallyforcontrolofdistributedparametersystems. AsforthemodelinformationofHandG,ESNbasedapproachhastwooptions.OneisthewaylikeothercontrollerswediscussedinprevioussectionsinwhichGMM-LLMsuppliespiece-wiselocallinearestimation.TheGMM-LLMpartwillbringadditionalcomputationrequirement,yetthewholeframeworkwillbedesignedbasedonamoretraditionalstyleasshowingure B{2 .AnotheroptionisusingoneESNforbothsystemidenticationandcontrol.InthiswaytheESNwillhavesinglecolumnofinputandtwoindependentreadoutweightmatrixcorrespondingtostateprediction~yn+1andcontrolcostpredictionn+1.Forthestatepredictionpart,ESNmodelfunctioncanbere-writtenfrom( 2{10 ) ~yn+1=WToutf[WinX(n)+Ws(n)](B{5)

PAGE 119

FigureB{2. SchematicdiagramofcontrolsystemwithESNascontrollerbasedonGaussianmodels ConsideringthesmallrandomvaluesofinputweightsWin,thestatepredictioncouldbelocallymodiedasrstorderTaylor-expansion ~yn+1=WTout[f(Ws(n))+Win(n)_f(Ws(n))X(n)](B{6) ConsideringtheX(n)=[x(n);u(n)]T,and ThelocalmodelinformationofHandGcanbederivedfrom( B{6 )as [H;G]T=WToutWin=cosh2(Ws(n)) Wenoticethatthecalculationforthelocalmodelinformation[H;G]ismainlybasedontheESNparameters.Firstorderestimationcannotguaranteetheprecision

PAGE 120

ofthereferencemorelyetthatkindofcalculationwillnotbringtoomuchcompu-tationalcost.ConsideringthehighmodelingeciencyofGMM-LLMframework,wewillmakeuseofmodelinformationfromGMM-LLMforESNcontrollerinthefollowingchapter.

PAGE 121

[1] K.S.NarendraandK.Parthasarathy,\Identicationandcontrolofdynamicalsystemsusingneuralnetworks,"IEEEJournalonNeuralNetworks,vol.1(1),pp.698{706,March1990. [2] J.Lan,J.Cho,D.Erdogmus,J.C.Principe,M.Motter,andJ.Xu,\Locallinearpidcontrollersfornonlinearcontrol,"InternationalJournalofCISSpecialIssueNonlinearAdaptivePIDControl,vol.33(1),pp.26{35,January2005. [3] J.Si,A.G.Barto,W.B.Powell,andD.C.Wunsch,LearningandApproximateDynamicProgramming,JohnWiley&Sons,NewYork,2004. [4] D.Erdogmus,J.Cho,J.Lan,M.Motter,andJ.C.Principe,\Adaptivelo-callinearmodelingandcontrolofnonlineardynamicalsystems,"inIntelligentControlSystemUsingComputationalIntelligenceTechniques,A.Ruano,Ed.,pp.121{152.IEE,2004. [5] J.C.Principe,L.Wang,andM.A.Motter,\Localdynamicmodelingwithself-organizingmapsandapplicationstononlinearsystemidenticationandcontrol,"inProceedingsofIEEE,1998,pp.2240{2258. [6] J.ZhangandA.J.Morris,\Recurrentneuro-fuzzynetworksfornonlinearprocessmodeling,"IEEETransactionsonNeuralNetworks,vol.10(2),pp.313{326,March1999. [7] M.ZhangandT.Tarn,\Ahybridswitchingcontrolstrategyfornonlinearandunderactedmechanicalsystem,"IEEETransactionsonAutomaticControl,vol.48(10),pp.1777{1782,October2003. [8] J.D.BoskovicandK.S.Narendra,\Comparisonoflinear,nonlinear,andneuralnetworkbasedadaptivecontrollersforaclassoffed-batchfermentationprocess,"Automatica,vol.31(6),pp.817{840,1995. [9] C.T.Chen,IntroductiontoLinearSystemTheory,Holt,Rinehart,andWinston,NewYork,1970. [10] W.T.Miller,\Real-timeneuralnetworkcontrolofabipedwalkingrobot,"IEEEControlsystemMagazine,vol.14(1),pp.41{48,February1994. [11] T.Kohonen,Self-OrganizingMaps,Springer,NewYork,1995. [12] S.C.Douglas,ExactExpectationAnalysisoftheLMSAdaptiveFilterforCor-relatedGaussianInputData,Ph.D.dissertation,MIT,Cambridge,MA,2000. 110

PAGE 122

[13] R.C.DorfandR.H.Bishop,ModernControlSystems,Addison,Wesley,NewYork,NY,1998. [14] K.Ogata,ModernControlEngineering,PrenticeHall,NewYork,NY,2001. [15] B.Schoner,\Probabilisticcharacterizationandsynthsisofcomplexdrivensys-tems,"inProceedingsIEEEInternationalConferenceonAcoustics,SpeechandSignalProcessing,Minneapolis,MN,1993,pp.519{522. [16] J.D.FarmerandJ.J.Sidorowich,\Predictingchaotictimeseries,"PhysicalReviewLetters,vol.59(8),pp.845{848,1987. [17] T.Sauer,\Timeseriespredictionbyusingdelaycoordinateembedding,"inSantaFeInstituteStudiesintheSciencesofComplexity,pp.175{193.Addison-Wesley,Boston,MA,1997. [18] J.Cho,J.Lan,G.Thampi,J.C.Principe,andM.A.Motter,\Identicationaofaircraftdynamicsusingasomandlocallinearmodels,"inIEEEInternationalConferenceMWSCAS,Tulsa,OK,2002,pp.148{151. [19] J.C.PrincipeandL.Wang,\Nonlineartimeseriesmodelingwithself-organizingfeaturemaps,"inProceedingsofNeuralNetworksforSignalProcessing,Cam-bridge,MA,1995,pp.11{20. [20] M.A.Motter,ControloftheNASALangley16-footTransonicTunnelwiththeSelf-OrganizingFeatureMap,Ph.D.dissertation,UniversityofFlorida,Gainesville,FL,1997. [21] E.W.Weisstein,\Dynamicalsystem,"MathWorldwebsite,2006. [22] F.Takens,\Detectingstrangeattractorsinturbulence,"DynamicalSystemsandTurbulence,1981,SpringerLectureNotesinMathematics. [23] T.BuzugandG.Pster,\Optimaldelaytimeandembeddingdimensionfordelay-timecoordinatesbyanalysisoftheglobalstaticandlocaldynamicalbe-haviorofstrangeattractors,"PhysicalReview,vol.10(15),pp.7073{7084,May1992. [24] J.GaoandZ.Zheng,\Directdynamicaltestfordeterministicchaosandoptimalembeddingofachaotictimeseries,"PhysicalReview,vol.5,pp.3807{3814,May1994. [25] X.HeandH.Asada,\Anewmethodforidentifyingordersofinput-outputmodelsfornonlineardynamicsystems,"inProceedingofAmericanControlConference,SanFrancisco,CA,1999,pp.2520{25223. [26] A.M.FraserandH.L.Swinney,\Independentcoordinatesforstrangeattrac-torsfrommutualinformation,"PhysicalReviewA,vol.33(2),pp.1134{1140,February1986.

PAGE 123

[27] A.Weibel,T.Hanazawa,G.Hinton,K.Shikano,andK.J.Lang,\Phonemerecognitionusingtimedelayneuralnetworks,"IEEETransactionsonAcoustics,Speech,SignalProcessing,vol.37(3),pp.328{339,July1989. [28] I.W.SandbergandL.Xu,\Uniformapproximationofmulti-dimensionalmyopicmaps,"IEEETransactionsonCircuitsandSystems,vol.44(6),pp.477{485,June1997. [29] P.Fuhrmann,\Dualityinpolynomialmodelswithsomeapplicationstogeomet-riccontroltheory,"IEEETransactionsonAutomaticControl,vol.26(1),pp.284{295,February1981. [30] J.HeandO.P.Malik,\Anadaptivepowersystemstabilizerbasedonrecurrentneuralnetworks,"IEEETransactionsonEnergyConversion. [31] M.KimuraandR.Nakano,\Learningdynamicalsystemsbyrecurrentneuralnetworksfromorbitz,"NeuralNetworks,vol.11(9),pp.1589{1600,1998. [32] J.Hertzberg,H.Jaeger,andF.Schonherr,\Learningtogroundfactysmbolsinbehavior-basedrobots,"inProceedingofthe15thEuropeanConferenceonArticialIntelligence,Lyon,France,July2002,pp.708{712. [33] P.JoshiandW.Waass,\Movementgenerationwithcircuitsofspikingneurons,"NeuralComputation,vol.17(8),pp.1715{1738,2005. [34] D.Prokhorov,\Echostatenetworks:Appealandchallenges,"inIEEEInter-nationalJointConferenceonNeuralNetworks,2005,pp.1463{1466. [35] M.C.Ozturk,D.Xu,andJ.C.Principe,\Analysisanddesignofechostatenetworks,"NeuralComputation,vol.inpress,2006. [36] H.Jaeger,\The\echostate"approachtoanalysingandtrainingrecurrentneuralnetworks,"Tech.Rep.TechnicalreportGMDreport148,GermanNationalResearchCenterforInformationTechnology,http://www.faculty.iu-bremen.de/hjaeger/pubs.html,June2001. [37] H.Jaeger,\Shorttermmemoryinechostatenetworks,"Tech.Rep.Techni-calreportGMDreport152,GermanNationalResearchCenterforInformationTechnology,http://www.faculty.iu-bremen.de/hjaeger/pubs.html,August2001. [38] Y.N.Rao,S.P.Kim,J.C.Sanchez,D.Erdogmus,J.C.Principe,J.M.Carmena,M.A.Lebedev,andM.A.Nicolelis,\Learningmappinginbrainmachineinter-faceswithechostatenetworks,"inIEEEInternationalConferenceonAcoustics,Speech,andSignalProcessing,2005,pp.233{236. [39] M.Casdagli,\Nonlinearpredictionofchaotictimeseries,"PhysicalReviewD,vol.35,pp.335{356,1989.

PAGE 124

[40] M.B.Priestley,\State-dependentmodels:Ageneralapproachtononlineartimeseriesanalysis,"JournalofTimeSeriesAnalysis,vol.1(1),pp.47{71,1980. [41] C.LiandC.Lee,\Self-organizingneuro-fuzzysystemforcontrolofunknownplants,"IEEETransactionsonFuzzySystems,vol.11(1),pp.135{150,February2003. [42] S.Haykin,NeuralNetowrks,PrenticeHall,NewJersey,July1998. [43] J.Minko,SignalProcessingFundamentalsandApplicationsforCommunica-tionsandSensingSystems,ArtechHouse,Boston,MA,2002. [44] T.TakagiandM.Sugeno,\Fuzzyidenticationofsystemsanditsapplicationtomodelingandcontrol,"IEEETransactionsonSystem,Man,andCybernetics,vol.15(1),pp.116{132,1985. [45] M.A.F.FigueiredoandA.K.Jain,\Unsupervisedlearningofnitemixturemodels,"IEEETransactionsonPatternAnalysisandMachineIntelligence,vol.24(3),pp.381{396,March2002. [46] A.Dempster,N.Laird,andD.Rubin,\Maximum-likelihoodfromincompletedataviatheemalgorithm,"J.RoyalStatisticalSoc.,vol.SeriesB39(1),pp.1{38,1977. [47] C.M.BishopandM.Svensen,\GTM:Thegenerativetopographicmapping,"NeuralComputation,vol.10(1),pp.215{234,1998. [48] G.PataneandM.Russo,\TheenhancedLBGalgorithm,"IEEETransactionsonNeuralNetworks,vol.14(9),pp.1219{1237,November2001. [49] N.VlassisandA.Likas,\Akurtosis-baseddynamicapproachtogaussianmix-turemodeling,"IEEETransactionsonSystem,Man,andCybernetics-PartA:SystemandHumans,vol.29(4),pp.393{399,July1999. [50] Y.Linde,A.Buzo,andR.Gray,\Analgorithmforvectorquantizerdesign,"IEEETransactionsonCommunications,vol.28(1),pp.84{94,January1980. [51] A.Gersho,\Asymptoticallyoptimalblockquantization,"IEEETransactionsonInformationTheory,vol.25(7),pp.373{380,July1979. [52] C.ChinrungruengandC.Sequin,\Optimaladaptivek-meansalgorithmwithdynamicadjustmentoflearningrate,"IEEETransactiononNeuralNetworks,vol.6(1),pp.157{169,1995. [53] B.Fritzke,\Growingcellstructures{aself-organizingnetworkforsupervisedandunsupervisedlearning,"IEEETransactionsonNeuralNetworks,vol.7(9),pp.1441{1460,1994.

PAGE 125

[54] M.M.Polycarpou,\Stableadaptiveneuralcontrolschemefornonlinearsys-tems,"IEEETransactionsonAutomaticControl,vol.41(3),pp.447{451,March1996. [55] K.S.NarendraandC.Xiang,\Adaptivecontrolofdiscrete-timesystemsusingmultiplemodels,"IEEEJournalonAutomaticControl,vol.45(9),pp.1669{1686,September2000. [56] F.Delmotte,L.Dubois,andP.Borne,\Adaptivemulti-modelcontrollerusingtrust,"inIEEEInternationalConferenceonSystems,ManandCybernetics,1995,pp.4155{4160. [57] B.Boulet,B.A.Francis,P.C.Hughes,andT.Hong,\Uncertaintymodelingandexperimentsinh1controloflargeexiblespacestructure,"IEEETransactionsonControlSystemsTechnology,vol.51(5),pp.504{519,September1997. [58] D.HylandandD.Bernstein,\Themajorantlyapunovequation:Anonnegativematrixequationforrobuststabilityandperformanceoflargescalesystems,"IEEETransactionsonAutomaticControl,vol.32(11),pp.1005{1013,November1978. [59] G.ConteandA.Perdon,\Ageometricapproachtothetheoryof2-dsystems,"IEEETransactionsonAutomaticControl,vol.33(10),pp.946{950,October1988. [60] F.Lin,R.D.Brand,andJ.Sun,\Robustcontrolofnonlinearsystems:Compen-satingforuncertainty,"IEEEInt.JournalonControl,vol.56(6),pp.1453{1495,1992. [61] F.LinandR.D.Brand,\Anoptimalcontrolapproachtorobustcontrolofrobotmanipulators,"IEEETransactionsonRoboticsandAutomation,vol.14(1),pp.69{77,February1998. [62] L.ChenandK.S.Narendra,\Intelligentcontrolusingneuralnetworksandmultiplemodels,"inProceedingsofthe41stIEEEConferenceonDecisionandControl,2002,pp.1357{1362. [63] D.G.Lainiotis,\Partitioning:Aunifyingframeworkofadaptivesystems,i:Estimation,ii:Control,"ProceedingsofIEEE,vol.64(8),pp.1126{1142,August1976. [64] D.G.Lainiotis,\Multi-modelpartitioningthemulti-modelevolutionaryframe-workforintelligentcontrol,"inProceedingsofthe2000IEEEInternationalSymposium,July2000,pp.15{20. [65] K.S.Narendra,\Neuralnetworksforcontrol:Theoryandpractice,"ProceedingsofIEEE,vol.84(10),pp.1385{1406,1996.

PAGE 126

[66] D.A.WhiteandD.A.Sofge,HandbookofIntelligentControl:Neural,Fuzzy,andAdaptiveApproaches,VanNostrandandReinhold,NewYork,1993. [67] B.Eulrich,D.Andrisani,andD.G.Lainiotis,\Partitioningidenticational-gorithms,"IEEETransactionsonAutomaticControl,vol.25(3),pp.521{528,June1980. [68] T.YinandC.S.G.Lee,\Fuzzymodel-referenceadaptivecontrol,"IEEETransactionsonSystems,Man,andCybernetics,vol.25(12),pp.1606{1615,December1995. [69] K.Tanaka,M.Iwasaki,andH.O.Wang,\Switchingcontrolofanr/chovercraft:Stabilizationandsmoothswitch,"IEEETransactionsonSystem,Man,andCybernetics{PartB:Cybernetics,vol.31(6),pp.853{863,December2001. [70] J.P.Hespanha,\Stabilizationofnonholonomicintegratorvialogic-basedswitch-ing,"Automatica,vol.35,pp.385{393,March1999. [71] J.Cho,LocalModelingParadigmBasedonSelf-OrganizngMapforNonlinearNonautonomousSystem,Ph.D.dissertation,UniversityofFlorida,Gainesville,FL,December2004. [72] J.Y.Hung,W.Gao,andJ.C.Hung,\Variablestructurecontrol:Asurvey,"IEEETransactionsonIndustrialElectronics,vol.40(1),pp.2{22,February1993. [73] R.E.Brown,G.N.Maliotis,andJ.A.Gibby,\PIDself-tuningcontrollerforaluminumrollingmill,"IEEETrans.IndustryApplications,vol.29(3),pp.578{583,1993. [74] S.Skoczowski,S.Domek,K.Pietrusewicz,andB.Broel-Plater,\AmethodforimprovingtherobustnessofPIDcontrol,"IEEETransactionsonIndustrialElectronics,vol.52(6),pp.1669{1676,December2005. [75] J.N.Tsitsiklis,\Ecientalgorithmsforgloballyoptimaltrajectories,"IEEETransactionsonAutomaticControl,vol.40(9),pp.1528{1538,September1995. [76] S.YashikiandN.Matsumoto,\Stabilizationandoptimalregulatorproblemfortime-delaysystemsbasedonthe2driccatimatrixequation,"ElectronicsandCommunicationsinJapanPart3,vol.81(1),pp.1{12,January1998. [77] E.J.KleinandW.F.Ramirez,\Statecontrollabilityandoptimalregulatorcontroloftime-delayedsystems,"INT.JournalControl,vol.74(3),pp.281{289,2001. [78] T.YangandL.O.Chua,\Generalizedsynchronizationofchaosvialineartrans-formations,"Int.JournalofBifurcationandChaos,vol.9(1),pp.215{219,February1999.

PAGE 127

[79] T.Yang,C.Yang,andL.Yang,\Detailedstudyofadaptivecontrolofchaoticsystemswithunknownparameters,"DynamicsandControl,vol.8(3),pp.255{267,March1998. [80] X.Yu,\Trackinginherentperiodicorbitsinchaoticdynamicsystemsviaadap-tivevariabletime-delayedselfcontrol,"IEEETransactionsonCircuitsandSystemsI,vol.46(11),pp.1408{1411,November1999. [81] Y.TianandX.Yu,\Adaptivecontrolofchaoticdynamicalsystemsusinginvari-antmanifoldapproach,"IEEETransactionsonCircuitsandSystemsI:Funda-mentalTheoryandApplications,vol.47(10),pp.1537{1542,October2000. [82] J.Gonzalez,R.Femat,J.Alvarez-Ramirez,R.Aguilar,andM.Barron,\Adiscreteapproachtothecontrolandsynchronizationofaclassofchaoticoscilla-tors,"IEEETransactionsonCircuitsandSystems-I,vol.46(9),pp.1139{1144,September1999. [83] A.Bartoszewicz,\Discrete-timequasi-sliding-modecontrolstrategies,"IEEETransactionsonIndustrialElectronics,vol.45(4),pp.633{637,August1998. [84] W.Gao,Y.Wang,andA.Homaifa,\Discrete-timevariablestructurecontrolsystems,"IEEETransactionsonIndustrialElectronics,vol.42(2),pp.117{122,April1995. [85] L.C.Westphal,\Lessonsfromanexamplein\onthestabilityofdiscrete-timeslidingmodecontrolsystems","IEEETransactionsonAutomaticControl,vol.44(7),pp.1444{1445,July1999. [86] E.A.OotenandW.Singhose,\Commandgenerationwithslidingmodecontrolforexiblesystems,"inProceedings.6thInternationalWorkshoponAdvancedMotionControl,Honolulu,HI,2000,pp.64{68.

PAGE 128

JingLanwasborninBeijing,People'sRepublicofChina,onJune16,1973.HereceivedhisB.S.degreefromBeijingUniversityofAeronauticsandAstronautics,majoringinautomaticcontrol,in1996.HereceivedhisM.S.degreeatTsinghuaUniversity,majoringinautomation,in1999.Since2001,hehasbeenwiththeCom-putationalNeuroEngineeringLaboratoryattheUniversityofFloridatopursuehisPh.D.degree.Hisresearchinterestsincludenon-linearsystemidentication,Gaussianmixturemodelprediction,andneuralnetworksandadaptiveltersonsignalprocess-ing. 117


Permanent Link: http://ufdc.ufl.edu/UFE0014640/00001

Material Information

Title: Gaussian Mixture Model Based System Identification and Control
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0014640:00001

Permanent Link: http://ufdc.ufl.edu/UFE0014640/00001

Material Information

Title: Gaussian Mixture Model Based System Identification and Control
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0014640:00001


This item has the following downloads:


Full Text










GAUSSIAN MIXTURE MODEL BASED SYSTEM IDENTIFICATION AND
CONTROL
















By
JING LAN


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2006
































Copyright 2006

by

Jing Lan





































To my family.
















ACKNOWLEDGMENTS

I would like to express my sincere gratitude to my advisor, Dr. .Josii C. Principe,

for his continuous guidance, support, and help throughout the past five years of

my Ph.D. study. Without his invaluable encouragement and patience, this work

would have been impossible. He gave me the privilege to work in the Computational

NeuroEngfineeringf Lab and allowed me the freedom to explore the unknown world,

and was akr- 0-< attentive and pertinent in critical moments. My deep appreciation is

also extended to Dr. .John G. Harris, Dr. .John AI. Shea, and Dr. Loc Vu-Quoc for

their interests and participation on my supervisory coninittee. My deep recognition

goes to Dr. Deniz Erdognius and Dr. Mark A. Alotter for their help and informative

coninents during my stay in CNEL. I also owe many thanks to all my colleagues in

the CNEL group for their discussion of ideas and friendship.

My wife, Yun Zhu, has my deepest appreciation for her great love and uncondi-

tional support. Finally, I am grateful to my parents and my brother for their love.


















TABLE OF CONTENTS

page

ACK(NOWLEDGMENTS ......... .. iv

LIST OF TABLES ......... .. .. vii

LIST OF FIGURES ......... . .. viii

ABSTRACT ......... .... .. x

CHAPTER

1 INTRODUCTION . ...... ... .. 1

1.1 Mission of System Identification and Control .. .. .. 1
1.2 Local Modeling with Improved Gaussian Mixture Model .. .. 5
1.3 Dissertation Outline ......... .. 8

2 NONLINEAR DYNAMICAL SYSTEM MODELING .. .. .. 10

2.1 Nonlinear Dynamical Systems ...... .... 10
2.2 Time Embedding State Space Reconstruction .. .. 11
2.3 Global Dynamical System Modeling .... .... 13
2.3.1 Polynomial Modeling . .... .. 14
2.3.2 Echo State Network . .... .. 15
2.4 Local Dynamic Modeling . .... .. 20
2.5 Summary ........ ... .. 22

3 GAUSSIAN MIXTURE MODELS AND LOCAL LINEAR MODELS 24

3.1 Introduction . . .. .. .. .. 24
3.2 Maximum Likelihood Estimation Training -EM Algorithm .. .. 29
3.3 Estimation of Local Linear Model ... .. .. .. 32
3.4 Improvement on GMM . ... .. .. 36
3.5 Comparison of the GMM with the SOM .. .. .. .. 38
3.6 Summary ........ ... .. 41

4 GAUSSIAN MIXTURE MODEL BASED CONTROL SYSTEM .. .. 42

4. 1 Introduction of Mixture Model Controller ... .. .. .. 43
4.2 Inverse Error Dynamic Controller Using GMM .. .. .. .. 46
4.3 PID Controller ........ ... 50
4.4 Optimal Robust Controller . .... .. 52











4.4. 1 LLM-based Optimal Set-Point Controller .. .. .. .. 55
4.4.2 LLM-based Optimal Tracking Controller .. .. .. 59
4.5 Summary ........ ... .. 61

5 APPLICATIONS AND DISCUSSIONS ... .. .. .. 63

5.1 C'!s Istic Systems ......... ... 63
5.1.1 The Lorenz System . .. .. .. 64
5.1.2 The Duffing Oscillator System .... .... .. 70
5.2 SISO System ......... .. .. 75
5.2.1 Linear SISO Mass-Spring System ... .. .. 75
5.2.2 Nonlinear SISO Missile System .... .. .. 78
5.3 Nonlinear MIMO System: LoFLYTE System .. .. .. 85
5.3.1 System Introduction . .... .. 85
5.3.2 System Identification and Control experiment .. .. .. 88

6 CONCLUSIONS ......... .. .. 97

6.1 Summary ......... .. .. 97
6.2 Future Work ......... .. 98
6.3 Conclusion ......... . 99

APPENDIX

A OPTIMIZING PARAMETERS OF ESN .... .... .. 100

B ESN-BASED DAC CONTROLLER ..... ... .. 104

REFERENCES ......... . .. .. 110

BIOGRAPHICAL SK(ETCH ......... .. .. 117

















LIST OF TABLES

Table page

:31 Performance on :32-D LoFLYTE data .... .... :38

5-1 Performance on the Lorenz system ..... .. 67

5-2 Performance on the Duffing oscillator ..... .... 71

5-3 Performance on the linear mass system .... .. 76

5-4 Performance on the missile model ...... .... 80

5-5 Control performance comparison on missile system considering noise .. 84

5-6 Performance on the LoFLYTE simulation data .. ... 91

5-7 Control performance comparison on the LoFlyte system considering noise 92

















Figfure

1-1 1-D function estimation by GMM-LLM. .........

2-1 Block diagram of the ESN. ..........

2-2 Distribution of ESN internal weights with maximum spectral radius being
0.9, with unit circle as a reference.


pagfe

5

17


Structure of GMM with LLM.

Explanation of G-SOM algorithm..

Soft combination region using GMM (between dash

Structure of Control System hased on GMM-LLM.

T-S fuzzy logic membership function

Structure of Inverse Control System .

pole-placement PID control system..

Optimal robust control system schematic diagram.

Dynamics of the Lorenz system


line)


system.


)


5-2 System identification performance on the "x" of the Lorenz


-3 Distribution of the Lorenz system (dash line) and GMM weights (star

-4Weights change of 8 GMM kernels;

-5 Performance comparison between SOM and GMM.

-6 Compensation performance

-7 Dynamics of the Duffing oscillator as an autonomous system..

-8 System identification performance on the Duffing system.

-9 Sample of weights change of 8 GMM kernels;.

-10 Control performance of different controllers on the Duffing system.

-11 Dynamics of the mass system.


LIST OF FIGURES










5-12 Sample of system identification performance on the Mass system. .. 77

5-13 Sample of weights change of 6 GMM kernels for mass system .. .. .. 78

5-14 Set-point control performance on the mass system. .. .. .. 79

5-15 Missile system dynamics. ......... ... 79

5-16 Sample of system identification performance on the missile system. .. 80

5-17 Sample of weights change of 8 GMM kernels for missile system .. .. 81

5-18 Performance of PID controller with different modeling approaches. .. 82

5-19 GMM-LLM hased control performance on the missile system. .. .. 8:3

5-20 ESN control performance on the missile system. .. .. .. 84

5-21 General description of an aircraft ...... .. 86

5-22 Dynamics of the LoFLYTE system. ..... .. 87

5-23 System Identification performance on the LoFLYTE system. .. .. .. 89

5-24 Sample of weights change of GMM kernels for the LoFLYTE system .. 90

5-25 Control performance on the LoFLYTE system (set-point, noise free). 9:3

5-26 Control performance on the LoFLYTE system (tracking, noise free). .. 94

5-27 Robust control performance on the LoFLYTE system (set-point, :30 dB
SNR noise). ......... . 95

5-28 Robust control performance on the LoFLYTE system (tracking, :30 dB
SNR noise). ......... . 96

A-1 Distribution of improved ESN internal weights with maximum spectral
radius is 0.9, with unit circle as a reference. ... .. .. .. 101

B-1 Demonstration system with ESN reference model and controller .. .. 105

B-2 Schematic diagram of control system with ESN as controller based on
Gaussian models ......... . .. 108















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

GAUSSIAN MIXTURE MODEL BASED SYSTEM IDENTIFICATION AND
CONTROL

By

Jing Lan

August 2006

C'I ny~: Josi4 C. Principe
Major Department: Electrical and Computer Engineering

In this dissertation, we present a methodology of combining an improved Gaussian

mixture models (GMM) with local linear models (LLM) for dynamical system identi-

fication and robust predictive model control. In order to understand the advantage of

the mixture model, its structure and training are discussed in detail. A growing self-

organizing nmap is utilized to improve the random initialization of mixture models,

which makes the GMM convergence more stable. To increase local modeling capa-

hility and decrease modeling error, local linear models are trained based on GMM as

one-step predictors. Following the local modeling approach, a series of controllers are

designed to realize a tracking application, among which the optimal robust control

shows better robustness over other controllers. Five application systems with differ-

ent dynamics are simulated in order to verify the modeling and control capability of

the improved Gaussian mixture model. Through experiments and comparison with

self-organizing maps, radial basis functions, and other methodologies, it is shown that

the improved GMM-LLM approach is a more flexible modeling approach with higher










computation efficiency than its competitors. The Gaussian model algorithm shares

with the self-organizing maps the ease of robust controllers design.















CHAPTER 1
INTRODUCTION

1.1 Mission of System Identification and Control

System theory, which has evolved into a powerful scientific discipline of wide

applicability, deals with the analysis and synthesis of dynamical systems [1]. In

particular, the analysis of linear identification and control for nonlinear dynamical

systems has been studied by many researchers leading to a better understanding of

various mechanisms which includes observability, controllability, stability, and robust-

ness [1, 2, 3]. The best developed aspect of the theory treats systems defined hv linear

operators using well established techniques based on linear algebra, and the theory of

linear differential equations. Nevertheless, academic research has been concentrating

around problems involving the identification, stabilization, and control of nonlinear

dynamical systems in the last several decades. These efforts have now matured into

a broad group theories on nonlinear dynamical systems applications. Initial efforts in

this area pursued parametric approached, inspired by the established linear system

theory, in which the dynamical equations of the system are assumed to be known from

physical principles, with some uncertainty in certain parameters. In this framework,

the system identification and system control problems are decoupled. And therefore

they can he solved sequentially. Recently, adaptive system identification and control

methodologies have been investigated too, which leads to a good understanding of

the adaptation in linear systems and a satisfactorily general insight to adaptation in

nonlinear control systems [4, 5, 6, 7].










Control theory deals with the system manipulation of the behavior of dynamical

systems to satisfy certain desired outputs constraints [8, 9]. A classical parametric

design procedure follows the system modeling (identification) and controller selection

stages. In the case of traditional identification based on model derived from physical

principles, the data are used to estimate the unknown parameters, with physically

motivated system, whereas modern approaches stemming from the advances in neural

networks modeling introduce black-box: function approximation schemes [4]. The

neural network modeling capabilities may be further enhanced using multiple sub-

models in the context of switching between multiple adaptive models to achieve close-

loop system that enhance transient behavior and cope with modeling uncertainties

better than single model. Following the system identification stage, depending on

the chosen modeling approach, the controller is designed typically using classical

techniques based on linear system theory as well as classical or neural-networks-based

nonlinear techniques [1, 10].

1\odeling techniques can he classified into global and local. Global modeling uses

a single parametric system and fits it all over the state space. The global modeling

methods estimate the dynamics of system in one framework. Using single kernel or

multiple kernels, the model is trained with the whole training data set. The most

distinguishing feature for global modeling is that it is trained with all the data points,

and in the system identification phase, all kernels will take effect on every step of

prediction. Good examples of global modeling are the multi-1 .,-;r perception (j!l Pl)

and the radial-basis-function (RBF), even though the last one uses local kernels.

When estimating the distribution in the input space, each kernel in the RBF's hidden

l o,-;r accepts all the data from the training data set. Recurrent neural networks










(RNN) are another kind of powerful computational structure for global modeling

[3]. But because of their complex inner structure, efficient training algorithms are

necessary to achieve successful learning for the given task. Another difficulty for

global modeling is when dynamical system characteristics considerably vary over the

operating regime, effectively bringing the issue of time varying nonlinear parameters

into the model design. Because of the complexity and possibly time-variability of the

nonlinear dynamical system, global modeling and designing corresponding controllers

is a difficult and time-consuming task.

Local modeling, which utilizes a divide-and-conquer approach to solve compli-

cated problems, breaks the fitting of the system into smaller and easier pieces which

can be managed by simpler topologies. Using multiple kernels, the local modeling is

trained in a way that a kernel only gets training information from those data which

are located within the region that kernel defined. In system training phase, every

input data is located first, and only the i.- -!ant, i" kernel will later take effect on iden-

tification. To better understand this point, let us take the self-organizing map (SOM)

[11] as an example with its winner-takes-all criterion. Therefore the local modeling

technique can be regarded as a piece-wise modeling approach, and the pieces can be

grouped together when it is necessary to approximate global dynamics. Specifically

when each model is linear, the resulting model is a piece-wise linear dynamical ap-

proximation to the globally nonlinear dynamical system. The advantages of such a

partitioning approach are the following:

1. system identification complexity can be reduced significantly due to the scaling
down of the estimation problem from one large task to multiple small and simple
task,










2. the local models can easily scale up to encompass more volume in the state space
by adding new models as new data portion is acquired, which is especially of
benefit when training data cannot cover the whole state space;

3. the design of a control system for the nonlinear system can he reduced to the
design of multiple simple local controllers among which cooperation is possible
to generate a single controller to the current plant.

The Gaussian mixture model with local linear models (GMM-LLM) is a local

modeling approach, yet GMM training basically follows the global modeling train-

ing criterion because of the distributing property of Gaussian density function. In

the GM13 approach, the model is generated by a multi-dimensional joint mixture

of Gaussian distributions [12]. In other words, the distribution of state vectors is

described by a soft combination of several Gaussian distributions. And those combi-

nation coefficients will be determined according to the Gaussian posterior probability

of current state vector giving certain Gaussian distribution. Following the Gaussian

posterior probability property, every Gaussian is trained by all the training data set.

Yet due to the nature of the nonlinear Gaussian distribution function, the weights

will be large when the data is close to the centers of some Gaussian, which make the

Gaussian kernel's training emphasize more on their neighbor data than the data far

away. In the prediction part, it becomes clear that Gaussian model is taking effect

locally. The benefit from this nature is that the GMM is globally well trained, and

local dynamics are addressed as well. The latter is also the reason why we call the

approach local modeling.

The problem of controller design can he further eased with the selection of local

modeling. In this case, the global nonlinear controller can he transformed into multi-

ple piece-wise linear controllers for nonlinear systems, for which many strong design

tools are available [8, 9, 13, 14].












0.0

0.4

0.2



-0.2
0 5 10 15 20 25 30



0.4
0.3 -Local Linear Models
02-Gaussian Models
0.1
0.

-0.1
0 5 10 15 20 25 30



Figure 1-1. 1-D function estimation by GMM-LLM.
Top: Nonlinear function and its linear approximation, Bottom: GMM-LLM pairs


1.2 Local Modeling with Improved Gaussian Mixture Model


The framework proposed herein is an improved GMM with local linear model,


which is shown in figure 1-1. The GMM uses the Gaussian family to reconstruct the


distribution of the state space vectors applying a soft-combination instead of a winner-


takes-all criterion. Each Gaussian component represents a certain characteristic into


a specific location of system dynamic space. Since the Gaussian distribution needs


only two parameters, mean and covariance, the computation efficiency can be largely

increased in GMM training. Because of this modeling property, GMMs have been


widely applied in speech recovery, spectral analysis, etc. Schoner applied GMM


for function approximation using supervised training [15]. Initializing the Gaussian


kernels' weights from growing self-organizing maps (GSOM), a more stable global


maximum convergence is achieved.










Local modeling is usually based on the Nearest-neighbors criterion. A clustering

algorithm divides the training data set into a group of smaller sets based on the

Euclidean distance or other similar criteria, and a model is trained corresponding to

each data set. As Farmer and Sidorowich [16] have already shown, local linear models

provide an effective and accurate approximation despite their simplicity. Local linear

models have shown very good performance in comparative studies on time series

prediction problems and in many cases have generated more accurate prediction than

global approaches [16, 17].

Building local mappings in reconstructed state space is a time and memory

consuming process even though the performance can be satisfactory. A simplified

approach is to quantize the state space and build local mappings only in positions

where prototype vectors are located. The improvement utilized in the Self-Organizing

Maps (SOM) has been applied by Principe et al. to model autonomous and non-

autonomous systems [18, 19]. Motter also utilized SOM based modeling to the design

of switching controllers [20].

One possible problem the SOM-based modeling cannot avoid is the quantization

error in the modeling phase, due to the Euclidean distance between the prototype

vector and the data closest to the vector. Th O-ae proc sue h

prototype vectors can represent the distribution of the whole state space and the

distance is negligible, yet that is true only when the amount of prototype vectors

is very large. Meanwhile, large amount of prototype vectors will lead to further

requirements on computational resource and training time.

The structure of GMM consists of a set of Gaussian models and a gatingf function.

The weighting coefficient for a certain data point x corresponding to the Gaussian










kernels is described by the gating function. As the distance between the state and

kernels determines the gating function value, it makes modeling error decrease and

produces smooth switching compared with the SOM method.

The parameters of the Gaussian mixture are initially unknown, and will be es-

timated based on incomplete data through the Growing-SOM method. This implies

that parameters are updated in a cooperative rather than competitive way. The pro-

cedure is an updated weighting of the contribution of all data points given a particu-

lar Gaussian kernel. The training of GMM is based on maximum likelihood criterion

through the expectation-maximization (EM) algorithm. Section 3.2 discusses the EM

training algorithm in depth. Following the Gaussian distribution density function,

one local linear model is generated from local data for each Gaussian kernel. Thanks

to the gating rule, the mixture of Gaussians framework is able to continuously vary

the infrastructure of each piece-wise linear local model.

When the GMM-LLM is compared with RBF, the advantage of GMM-LLM is the

modeling of local dynamics. Even though both the GMM-LLM and RBF use a linear

combination in the final stage, the mechanism of how they describe the local dynamics

is different. RBF modeling uses a single fixed weighted combination of a group of

nonlinear outputs from Gaussian kernels, the centers of the Gaussians are adapted,

but not the covariance. Although the GMM-LLM uses a linear model to represent the

local system dynamics, which makes the output changes in a linear manner instead

of a nonlinear way. Several such local models exist distributed throughout the state

space, and they are weighted by the gating function to increase performance.










1.3 Dissertation Outline

In this dissertation, we will setup a control framework for the problem of modeling

and control of nonlinear dynamical systems. We present a modified mixture Gaussian

modeling-based system identification and control framework. GMM, as a piece-wise

linear predictive model, is trained to estimate the distribution of state space using

the Expectation-Maximization algorithm in order to satisfy the maximum likelihood

criterion. Following the mixture Gaussian distribution, local linear models (LLM)

are generated based on Mean-Square-Error (jl!810) criterion to make accurate one-

step prediction, which is obtained by soft-combinational function evaluation. Under

the assumption that the state dynamics are smooth and sufficient data samples are

available, GMM can provide a good and efficient prediction performance compared

with other competitive methodologiees.

Based on the mixture Gaussian reference model, we develop a set of controllers

for non-autonomous and nonlinear plants. Starting from the EM training, the sys-

tem model can be represented by GMM-LLM and the modeling error is bounded.

Several controller such as dynamic inverse and pole-placement PID controllers, op-

timal robust controller, echo state neural controller are realized and emploi-x & using

the mixture model for several systems with different dynamics. We emphasize the

optimal controller and the echo state neural controller. We will show that the opti-

mal controller exhibits excellent robustness against model uncertainty and external

disturbance, and the ESN controller has better generalization capability.

The dissertation is divided into five parts. In C'!s Ilter 2, some general description

and definitions of nonlinear dynamic systems and local modeling, time-embedding de-

lay system reconstruction, and dynamic system identification are given. The criterion










to select embedding dimension will be discussed. Echo state network, a new kind of

recurrent neural networks, is present for the later use on controller design.

In ('!, Ilter 3, GMM training methodology analysis is presented, which also in-

cludes improved GMM training through GSOM initialization, the generation of local

linear models (LLM), and the comparison of GMM-hased approach to SOM-hased

approach in order to address the computational efficiency of Gaussian modeling.

In C'!s Ilter 4, the descriptions for a set of controller design approaches based

on GMM, the controllers' mathematical backgrounds, and their stability issue are

discussed in detail.

('!s Ilter 5 gives application examples on a set of real system model, and main

results of GMM-hased applications are presented with comparisons and discussed in

details. For different modeling methods, the modeling performance will be compared

based on signal to error ratio criterion. For controllers, the control performance will

be compared based on criteria of rising (falling) time, settling time, overshot, and the

covariance of control error.

('!s Ilter 6 gives conclusion, contribution and briefly summarizes future work.















CHAPTER 2
NONLINEAR DYNAMICAL SYSTEM MODELING

2.1 Nonlinear Dynamical Systems

General linear and nonlinear system theories provide tools which make it pos-

sible to fully estimate a nonlinear dynamic system from observation under the as-

sumption that noise is zero or negligible. The enabling theoretical insight is the time

7. AIrl embedding theorem, which will be briefly reviewed along with some important

system-characterization methodology. Reconstruction of the physical state space is

explicitly or implicitly fundamental to the prediction and characterization applica-

tions in chapter 5.

Dynamics describe how one system state develops into another state over the

course of time. Technically, a dynamical system is a smooth function of the reals or

the integers on another object (usually a manifold). "When the reals are .Il 1En the

system is called a continuous dynamical system, and when the integers are acting, the

system is called a discrete dynamical system" [21]. A dynamical system expressed as

a discrete time evolution rule can be described from the observation of its outputs.

The dynamic of the system in state space can be defined as


x(t + 1) = f (x(t)) (2-1)

y(t) = h(x(t))


where x(t + 1) is the next step state for discrete-time systems. x(t) are the system

states, f(-) is a nonlinear transfer function and is usually referred as the vector field,










and h(-) is the output function. If the vector field f(-) is a linear function of the

system states, the underlying system is linear; otherwise, the system is non-linear. It

is also assumed that the systems here are deterministic. Thus the transition from the

system's state r(tl) at time tl to the state r(t2) at time t2 1S governed by deterministic

rules.

2.2 Time Embedding State Space Reconstruction

Equation (2-1) describes a system that follows the internal governing equations

f (-) given a certain initial condition. And the system's dynamic is fully characterized

by its manifold encoded in the function f(-). Because of Takens Theorem, transform-

ing the time series into a state space form will result in a dynamical equivalent to the

original space [22]. Since the mathematical description of f(-) is unavailable for most

real-world systems, with the formulation of the time embedding theorem, Takens pre-

sented a methodology to recover an equivalent description of f from observation of

the system:

Theorem 1. Let M~ be a compact I,,r,.:;./..1. of dimension va. For pairs (4.y)

S: Af At
generic 1/**/'''l et that the mesp #4,,: At R2nz+1.1.#.1b


#4,,(r)= (y(:r), y( (:r)),..., y(,-' (:r))) (2-2)


is
There are several methods for estimating the optimal embedding dimension.

Buzug and Pfister designed two methods called fill factor and local deformation [23].

The first is a procedure that yields a global measure of phase-space utilization for









(quasi) periodic and strange attractors and leads to a maximum separation of trajec-

tories within the phase space. The second describes the local dynamical behavior of

points on the attractor and gives a measure of homogeneity of the local flow. Gao and

Zheng proposed the local exponential divergence plot method for a chaotic time se-

ries, based on which the embedding dimension, delay time, and the largest Lyapunov

exponent are estimated [24].

We determine the minimum time embedding dimension using Lipschitz quotient

[25, 26]. If the function is assumed to depend on n 1 inputs and it actually depends

on n inputs, the data set may contain several points that are very close in the space

spanned by the n 1 inputs but differ significantly in the nth input. Assuming

the system function is smooth, if the (n 1)-D inputs are close, it is expected that

the outputs from them are also close. If one or several relevant inputs are missing,

outputs corresponding to different input will take different values. In this case, it can

be concluded that (n 1)-D input is not long enough.

The Lipschitz quotients in 1-D case are defined as


la= |yi-~),i, jE RN,i/j (2-3)


where NV is the number of samples, is input. For the multidimensional case, equation

is extended as


1" i -y(j i, j E J", i / j (2-4)


where n is the number of input.

The Lipschitz index can be defined as the maximum occurring Lipschitz quotient


l" = max(l") (2-5)










As discussed before, if n is not hig enough, the Lipschitz index will be large. If all

relevant inputs are included, the Lipschitz index will stay near constant.

2.3 Global Dynamical System Modeling

System identification is a technique to build mathematical models of dynamics

based on input-output measurements. An important property for system identifica-

tion is its capability of making short term prediction of system dynamics. Normally,

the nonlinear dynamical behavior of real-world systems complicates the construction

of models that can accurately describe dynamics. 1\oreover, the knowledge about the

inner structure of systems is often unavailable. Under this situation, the modeling

training can only be driven through a finite collection of system's input and output

measurement. This kind of identification is referred as black box: approach.

Several approaches have been studied for global modeling. One representative

of global nonlinear modeling is the time delay neural network (TDNN) which was

first developed by Weibel and Lang in 1987 [27]. Using time dl 11i- segment of state

as input data, the TDNN is structurally close related to the multi-11w-;r perception

(j!L Pl). The TDNN has been shown to be a universal mapper in multi dimensional

functional spaces with finite memory (especially myopic functional space) [28]. An-

other method is the polynomial modeling. Polynomial modeling is applied as the

basis for the realization of nonlinear mapping. Fuhrmann uses it in geometric control

theory [29]. Recurrent neural networks (RNN) represents a kind of nonlinear model-

ing methodology. Researchers use RNN when regular mathematical equations cannot

generate satisfying results [:30, :31].

In the following sections, we will introduce polynomial modeling briefly as we

will use its first order expansion as a linear model in chapter :3. And we will discuss









the echo state network (ESN) with modified design method as a new global modeling

approach.

2.3.1 Polynomial Modeling

Taking the single-input-single-output (SISO) system as an example, the linear

input-output polynomial model is the autoregressive moving average with exogenous

input (ARX) model


y(u) = blu(n 1) +- + bDu(n D) aly(n 1) --- aDy(n D) (2-6)


The polynomial Modeling can be extended to a nonlinear ARMAX (NARMAX)

model by replacing the linear mapper in (2-6) with nonlinear function f(-)


y(u) = f (u(n 1), ..., u(n D), y(n 1), ..., y(n D)) (2-7)


A more straightforward way to utilize polynomials for nonlinear system identifi-

cation is to apply a polynomial for approximation of the one-step prediction function

(2-7) with low order expansion, which is called K Jnt.-y.-<. -c~~ Gabor polynomial mod-

els. And the benefit from polynomial model is that all the nonlinear coefficients

calculations are avoided. A simplified version of the K Jnt.-y.>l~rov-Gabor polynomial

model uses less high-order information, also called V/olterra Series model, describes

the nonlinearity only for the input


y(u) = f [u(n 1),..., u(n D)] aiy(n 1) - aoy(n D) (2-8)


Thus the second order second degree polynomial function can be expressed as


y(u) = Or + 02u8 3 )+8u(n 2) + 04y9 1 5y(n 2) (2-9)


+ 06 2( ) 87 2(n 2) + Osu(n 1)u(n 2)










With this simplification the number of regressors is reduced, thus the computation

of prediction is simplified. The price paid here is the necessary increase of order D

in order to describe the nonlinearity of the original system. For linear model, there

will be no high order expansion, and the embedding length will be increased when

necessary.

2.3.2 Echo State Network

Under the condition that the system model is treated as a black-box, time em-

hedding input-output data will incur some technical problems besides its advantage

of simpleness. One of them is the need for large memory to access the system's his-

tory, another is the d.l 1 i-o I system response. The Echo State Network (ESN) gives

us an alternative for system modeling, which does not need time embedding data

and can supply instant system response [:32, :33, :34]. Comparing with other nonlinear

neural networks (recurrent networks especially), the ESN has less amount of parame-

ters need to be trained. 1\oreover, the parameters are linear and they can he trained

through the Wiener-Hopf approach [:35].

Heterogeneous ordered networks or, more specifically, recurrent neural networks

(RNN) are convenient and flexible computational structures applicable to a broad

spectrum of problems in system modeling and control. There are ]rl .ny r- .--4 to

make this statement more precise. For instance, RNNs can he cast as representations

of the vector field of a differential systems, or he trained to embody some desired

attractor dynamics [:31]. In those applications, systems without input (autonomous)

are modeled. Since we wish to model input-driven systems, we adopt a standard

perspective of system theory and view a deterministic, discrete-time dynamical system









as a function, which yields the next system output as a prediction, given the input

and output history [36].

As a special case of RNN, the ESN was first proposed by Jaeger [36, 37]. The

ESNs have interconnected and recurrent topology of nonlinear PEs which constitute

a i. i-~ m r of rich dyr! 1ons. The interesting property of ESN is that only the

memoryless readout of ESN needs training, whereas the internal recurrent topology

has fixed connection. This key property dramatically reduces the complexity of RNN

training to simple linear regression while preserving some of the computational power

of RNN.

We consider the recurrent discrete-time neural network as given in figure 2-

1 with M~ input units, NV processing units, and L output units. The value of

the input unit at time n is X(u) = [Xl(u), X2(u), XM/(u)], that of inter-

nal units is s(u) = [sl(u), 82 8 sN(u)] and that of output units is y(u)=

[y1(n),2 ya8), yL(u)] The system transfer function of ESN can be expressed as


s(n + 1) =f (Wi,X(n + 1) + Ws(u) + Wbacky 8)
(2-10)
y(n + 1) = fo,,(Wout s(n + 1))

where, for most of the case, nonlinear output function font(-) is eliminated, and only

linear output is used.

The critical difference between ESN and RNN lies in the adaptation of the recur-

rent weights. ESN compensates for not training the recurrent connection weights by

increasing the dimensionality of the hidden 1.v. -r. In this way, ESN creates a diversity

of dynamics with a large number of PEs in its reservoir and the readout is simply

trained to pick up dynamical components required for a given desired output signal.

Hence, it is crucial for the operation of the ESN to include a reservoir with "enough















Win Wu Wout






Wbc


Fiur 21.Blckdigrm f heEN

rihes"o dnmcs n i os nineig pl caton ofE N .sse
idniiatote pia lna ape sobandt rog reusv algrtm
whc miiiemensur e ero (\3 rhge re ro oetm
In th S ,teeh ttsfombssfntostaaeannieryflee














all hats the linear s r a nd-u is doit ng istocomine the bcassfntions dfE ,erived from

theinpt fction cete apia givndesire sigalTer spbain of theog ESNcan bve defind asthe



set of e allteecosae fr a functions define by(-0) hre foo is linear, o alvluso Wout.ere,

veso fthe ciiaqusionpu sisnl how big the span idpe of the echo states is.Tiisdrclread








tothe selec tion ofthe w ie eight siga.Te of the ESN resrvir Curnltee arfied two th










to generate W. The first method selects W randomly from a sparse matrix with the

help of the echo state condition ||W|| < 1 [36, 37], where ||-|| is the spectral radius

(SR). For the second method, the W is selected such that the maximum entropy is

achieved [35]. The echo state condition is a stability condition that guarantees that

the states are only controlled by the input whereas the sparseness is required to avoid

much correlation between the states by minimizing the connections among them.

Assuming unknown dynamical system transfer function is g(-), the objective of

system identification is to generate an estimation function g(-) such that the error

prediction is small,


I'I +1 Uk+1 II = Uky, uk) 9 Uk, uk) II E (2-11)


Let the linear mapper (assume a single output) be denoted by the weight vec-

tor Wout. Considering the cost criterion is MSE, the readout vector can be easily

computed through Wiener-Hopf function as


Wout = R-l P (2-12)


where R = E[sT s], P = E[sT d], and s is ESN internal states trained from input

sequence, d is corresponding desire sequence from training data.

We now present a more general formulation of the training procedure for the

case with no input feed forward and output feedback.

Task. Given a training I/O time series(utrain(n), ytrain())n=,---..,nma, where the

inputs come from a compact set U. We want a RNN whose output y(u) approximates















0.0 *

0.6 ***

0.4 *,* *** *
*, ,. ...**
0.2C * ** **




-0.2 ~ *, */* *


-0.6 *.*

-0.8- *

-1
-1 -0.5 0 0.5 1



Figure 2-2. Distribution of ESN internal weights with maximum spectral radius being
0.9, with unit circle as a reference.


1. Procure an ESN. Construct a RNN that has echo states in an admissi-


ble state set ,4 with respect to input. (i). setup normalized random input weights


since it shows that echo state property is independent of input weights; (ii). setup


sigmoid network whose weight matrix W's spectral radius satisfies | Anzas. < 1. Ex-

perience shows that the weight matrix with maximum entropy has the best system

identification performance in the sense of 1\SE. The sample result is shown in 2-2.


2. Run network with training, dismiss initial transient. Start with an


arbitrary networks state s(0), and feed input to ESN through (2-10) to obtain internal


states ss. Then dismiss from this run a suitably long initial transient period.

3. Compute output weights which minimize the training error. Use


back-propagation or any favorite linear regression algorithm to train the readout


weight matrix. With these weights, the network is ready for use.









2.4 Local Dynamic Modeling

Dynamic modeling seeks to qualify the system that created the time series. Fol-

lowing the discussion in chapter 1, global modeling utilizes the whole training data

set to describe the whole system dynamics. Yet the problem is that the global mod-

eling needs large amount of memory, and every data point will take effect on all the

memory. Consequently, global models will show some difficulties to bring the time-

varying issue into the design when system characteristics change considerably over

the operating regime.

Local modeling approaches, on the other way, divide the dynamic space into

several parts. Each local model is trained fittingf one part of the data from the state

space. The local linear approximation can be considered a simplified alternative to

nonlinear modeling. The approach of locally linear ARX has been widely discussed

in the time series literature. It is a state-dependent sequential type of recursive algo-

rithm for identifying dynamic models using simple linear regression or interpolation

[2]. Under the assumption that f(-) is smooth in the vicinity of x(u), f(-) can be

approximated by the first order term of its Taylor series expansion,


f(x(u)) fn(x(u)) (2-13)




where a', is the Jacobian of f at x(u), and b, is a constant vector at x(u). The

implicit assumption of the method here is that the time-steps for state dynamics are

small. The parameters (n', 6) can be estimated for the local linear mapping in the least

square sense from the local data samples. Casdagli et al. stated that local modeling

approaches can provided approximations of higher accuracy than global methods,









especially for systems with complex behavior. Casdagli also stressed that the great

advantage of local modeling is the flexibility in building the model [39].

As the center mission of modeling, the predictive mapping f : RD R can be

expressed through time embedding as


X(n + 1) = f (x(u)) (2-14)


where f (-) is a deterministic ARMA model with time-embedded input vector x(u) =

[x(u), x(n 1),..., x(n D + 1)]. With the method of local modeling the state

space is partitioned into small regions and a local model is established to describe

the dynamics in different region. Thus local models vary from region to region.

The approach of locally ARMA models has been discussed in the time series lit-

erature by Priestley [40]. Improved approach was proposed by Farmer and Sidorowich

[16], who also extended ARMA models to locally polynomial approximations of higher

order. The first order expansion, which is a linear model, is an effective local modeling

method. And the approximation with high-order polynomials are not significantly

better than those obtained with first order. In our work, we will use the first order

expansion.

In many local model applications, the self-organizing map (SOM) has been uti-

lized to divide the operating region into small parts, where one local model is trained

corresponding to one part of the state [5, 11, 18, 19]. The SOM is appropriate

for switching nonlinear relationships of high-dimensional data into simple geomet-

ric relationships that preserve the topology in the state space [41]. Principe and

Wang modeled a chaotic system with SOM-based local modeling method [19]. Cho










et al. realized competitive local controller using SOM-hased local modeling approach

[18, 20].

The role of GMM in system identification is to estimate the distribution of state

space. According to the distribution density of each Gaussian kernel within GMM,

the current state of the system is described by one or several Gaussian models. To

finish the predictive modeling, a set of local models also trained based on GMM.

Following the GMAI's criterion, one local model is linked to one Gaussian kernel.

And these two make up the identification model pair. The detail of improved GMM

construction and training and corresponding LLM training will be discussed later in

chapter 3.

2.5 Summary

Treating dynamical system as a black boxr, polynomial modeling and time enthed-

ding data structure build the foundation for the application of GMM. The simplified

polynomial expression of system dynamic makes linear modeling feasible. However, it

also limits the application range of each single model especially when large modeling

error is presented. In chapter 3, GMM will be use to estimate the application range

of each local model, and a comparison will be made between GMM and SOM. In

chapter 5, GMM-LLM hased system identification and modeling will be applied to a

series of applications and construct the basis of modeling predictive control (jl PC).

Front the nature of ESN, we can see one advantage of ESN is its less-training

construction procedure even though it is not linear modeling any more. Since its

internal states are randomly generated, the only part which needs training is its

readout vector, and there is no training that is necessary for the states. And since

the readout is of linear form, the training for the readout vector can he easily done










through Wiener-filter similar methods. Thus through the functionality of !unt, I>..1

ct II. the ESN will require less training computation compare with other recurrent

network. Another advantage of ESN is its input structure over tinte-embedding data

structure. Since ESN has a 1. -- no in!~" of internal states and pre-defined feedback,

these internal states can he deemed an !lIs is sI (I. li- to input for a certain system,

ESN can record it yet tinte-embedding may need longer data structure. We will also

show that the ESN with nmaxiniun entropy can accomplish systenl identification work

with nxininiuni \SE.















CHAPTER 3
GAUSSIAN MIXTURE MODELS AND LOCAL LINEAR MODELS

3.1 Introduction

The Gaussian Mixture Model (GMIN), also called mixture of expert models,

is a framework for unsupervised learning. GMM, along with local linear models

(LLM), constitutes the infrastructure to solve one of the fundamental problems of

our research, the system identification (SI) step. Conceptually, however, we cannot

use GMM alone to solve the SI problem. SI, as the discipline of making mathematical

models of unknown dynamical systems, must extract the structure in the joint space

(input & desire). GMM, as an "information" clustering approach, estimates the

distribution of data in the input space. GMM is similar to the Partition Algorithm

(PA) (Hilborn & Lainiotis 1969; Lainiotis 1971), which includes a group of Gaussian

kernel systems and a gating function [42], however it is based on a well defined

statistical framework for density estimation. The GMM is also similar to the first lIn-cr

of the Radial Basis Function (RBF) network used in nonlinear system approximation.

In fact the kernels in the R BF are also Gaussian functions, which are spread over the

data space using a clustering algorithm [42, 43]. Normally the covariance of these

Gaussian kernels are not trained from the data. The RBF output 1n-,-;-r combines

these internal states linearly into a single projector that is trained with the desired

response to extract the information in the joint space. The GMM-LLM approach

combines these two hasic strategies, by creating a local partition of the input space

as in the PA algorithm, and employs multiple local linear models attached to each










partition, instead of nonlinear combining the intermediate nonlinear states as in the

RBF.

Figure 3-1 depicts the elements of the GMM-LLM architecture. The input data

is modeled by a mixture of Gaussian's model, and the influence of each Gaussian

is evaluated by a weight that represents the likelihood of the data being generated

by the corresponding Gaussian mode. During training the data from each Gaussian

cluster is used to train the LLMs. During' -1 4! the input is directly sent to all the

LLMs and their output weighted with the probability that the sample is generated

by each model before being combined by the output adder. Note that during testing

these weights change with the data providing a greater flexibility when compared

with other local modeling alternatives. This architecture is principled using well

established statistical methods, very flexible, and still very simple because it uses

only linear models.

The GMM-LLM approach is conceptually an alternative to the Self Organizing

Map (SOM)-LLM reviewed in chapter 2. The SOM clusters the system state space

into a group of small regions using a winner-take~s-rell operator [42]. Each small input

region, which is also called Voronoi tessellation, is associated to one linear model

that is trained with the data from the tessellation and that attempts to approximate

the desired response when instantiated. Globally the modeling equation could be

described as

1,.rE clu~sterks
k=) 0, otherwise (1


Using soft-combination instead of winner-take~s-rell criterion, GMM does the local

modeling in a cooperative manner. Even though it still follows the description of the

















Gaussian 1 ;LLM 1- utu



Initial
training
data / asia L




GSOM training Gaussian K LLMK K 0
EM training



Gating Network



LLM 1

o, 2 Output
Testing
data




aK

LLM K



Figure 3-1. Structure of GMM with LLM.
Top: Training schematic, Bottom: Testing schematic.


first part of (3-1), it doesn't limit the coefficients to be "1" or "O". Consequently, the

GMM-LLM approach can reach similar SI performance with less PE compared with

the SOM-LLM approach. Moreover, soft-combination can generate smooth-switching


among different models, and can alleviate the difficulty faced by winner-takes-all,

when it chooses the wrong winner, where the corresponding LLM will generate worse

SI result.










Another competitor to the GMM-LLM is the fuzzy logic approach. The most

popular fuzzy logic approach is the Takagi-Sugeno (T-S) fuzzy model [44] which is also

a local modeling method. Similar to the GMM-LLM, T-S fuzzy model uses multiple

linear models for SI and soft-combination of linear nientership functions to weight the

outputs front linear models. The T-S fuzzy model divides the state space into several

overlapped regions using a set of simple "if-then" rules, one rule corresponds to one

linear membership function. Besides the order of nientership function, the difference

between the T-S fuzzy model and the GMM-LLM lies in the training method. The

T-S fuzzy model needs pre-knowledge about the system for both nientership function

training and linear model training. Unlike the unsupervised training in GMM, the

T-S fuzzy model training is in a supervised manner which increases the complexity

of modeling.

Assuming the training data set covers the state space of the target system, the

state space could be described by the embedded (reconstructed) data in the appro-

priate dimensional space with an optimal delay. The goal is to use a set of Gaussian

kernels to estimate the structure of the eventually complex data distribution in re-

constructed space. Since the only necessary parameters for a Gaussian distribution

are its mean and covariance which can save time and computation on model training.

This simplicity is one reason why Gaussian density function is chosen as the kernel

function.

Setting p(.F) as the distribution density of the current state .2, p(.2) can he ex-

panded in a sunt over Gaussian clusters ck. All the Gaussian clusters cover the whole

state space defined by (pk, k). The probability density of each cluster reflects the










domain of influence of each cluster.












where g(-) indicates that we explicitly state the form of the conditional Gaussian

distribution densities


g(.F|Ck) 2 -P-lk)T i~--Pk) (
(27r)D/2 k~ 1/2

where .F is current state, p~k and Ek are nlean Vector and covariance matrix of cluster

Ok in the feature space. Also the sunination over all the clusters is normalized by the

cluster probabilities p(Ck) to satisfy





If the elements of input .F are independent of each other or the cross correlation is

negligibly small, the covariance matrix could be reduced to a diagonal matrix, which

in turn simplifies the computation of Gaussian density function.



d=1 2/ETd,k

where D is the input dimension.

Function (:3-3) and (:35) show that GMM uses the linear combination of non-

linear estimation models so as to build a powerful global system model. Meanwhile,










GMM describes the local data feature with relatively simple models which satisfy

description of (3-2).

3.2 Maximum Likelihood Estimation Training EM Algorithm

The goal of the GMM approach is to find an optimal representation for the

distribution density p(x) of each point while maintaining a low computational cost.

In order to simplify the problem, the optimization criterion is set as the maximum

log-likelihood function over the whole data set. Figueiredo et al. use non-parametric

EM training approach for Gaussian mixtures [45]. We describe the maximum log-

likelihood function as


L(X, p-1, E) = Incp(,)] (3-6)
n=1


n=1 k=1

where NV and K are the total amount of data and Gaussian clusters respectively.

Since the Gaussian model parameters are calculated from the observed data set

and describe the local distribution of data points, a recursive search will be necessary

to find the optimal Gaussian model parameters. During the search, the maximum

likelihood function is used as an optimization criterion, so there will be two estimators

combined in each joint update, one for the estimation of likelihood, and one for

distribution of each Gaussian kernel. The Expectation-Maximization (EM) algorithm

[46] is applied to search for the optimal parameters of the Gaussian kernels. The EM

algorithm has gained wide appreciation as an efficient algorithm for probabilistic

networks training. The EM assumes that the known states (the observed data) and

the unknown states (the Gaussian parameters) characterize the whole system model.









As an unsupervised approach, the EM training based GMM only needs the number

of Gaussian kernels K. All other parameters are obtained through training.

The recursive process of EM algorithm starts with the E-step (Expectation

calculation), during which the cluster parameters are assumed correct. Expectation

of the log-likelihood (3-6) of the complete conditional probability by the observed

samples is computed. According to B li- Im Theorem, the posterior probabilities

are proportional to the likelihood function.


p(Ck|2) oc p(X|Ck) -- L(X, p~, E)


So equivalently, the posterior probabilities, which relate each Gaussian cluster

to each data point in the conditional probability form p(ck | ), can be evaluated as

p(X|kPCk)p k





The sum in the dominator normalizes among the Gaussian clusters, and the

numerator determines the largest possible density function for each data point. Thus

the maximum likelihood of the whole data set will monotonically increase.

In the M-step (Maximization calculation), the distribution density function

of each data point, which describes the likelihood of data given certain cluster, is

temporarily fixed. The update improves the parameters of each Gaussian cluster:

unconditional cluster probability p(Ck), mean Vector and covariance matrix (pk, k~).

Thus the likelihood will be maximized recursively. In equation (3-8), considering

that the amount of data points are limited and the values are discrete in reality, the










integral is replaced by the sum over all training data.




~~q jp p(Ckn n)8

n= 1

The mean vector and covariance matrix of each Gaussian cluster are updated

according to the stationary points of the likelihood to the unknowns (pk, k~). The

derivative equation of likelihood to the mean is


Inp(,] (3-9)
n= 1

In[-l pIeg(,ci)] = 0
n= 1 i= 1


n= 1 i= 1

p(Ck 9 @nCk)2(,- k
pK (-1) = 0
,=, tiK-1p Ck 9 /Ci) 2Ek

The answer to update the mean will be obtained through maximum likelihood

criterion:


p(k ) = p k|2 -E (3-10)
n=1 n=1

pk N
L=l 1P Ck I X









The same result is achieved through the integration method as:



p(Ck dr


p(Ck)3
1v p(Ck
N p(ck)3

pN-1 V~p(Ckl ~

Similarly, the Gaussian cluster covariance is:


Ek,ij i =)'j kj k|d (3-12)


N -p(ck;) r


E-1 p(CkX

The basic model estimation procedure can be summarized into five phases:

1. Pick initial conditions;

2. Estimate the data distribution density p(7|. ,);

3. Estimate the posterior probability of each cluster for each data p(Ck 2);

4. Estimate the cluster probability p(Ck), Gaussian cluster parameters mean pk and
covariance Eki

5. Go back to step (2) until the increment of likelihood function (3-6) is within
certain range.

3.3 Estimation of Local Linear Model

Because GMM-LLM is a group of globally optimized local model based on distri-

bution of training data set, it preserves topological properties of target system in the

state space. Since the soft combination of Gaussian clusters can be used to weight the










estimation of the current system's dynamic, the dynamic estimation can be extended

to a one-step linear mapping y(n + 1) = f(x'(u)) = opT, x,, where f stands for

a certain reference model. And fopt is the optimized current local model based on

Gaussian gating function. LLM is the most simplified reference model of the real sys-

tem and it will simplify the design procedure of controller [16]. Fr-om equations (3-2),

(3-7), and (3-10)-(3-12), we thus can express one-step linear prediction function as


y,+l i: p(X,|Ck P Ck) LT 3
k=~~p~ 1 =19/Ci) p Ci)







where k~ is the kth local linear model (LLM) corresponding to Gaussian cluster k,

which has same dimension of pk and x'. y,+ is the estimation of y,+l. For simplicity,

we denote p~,n aS posterior probability of cluster k corresponding to data n.

Equation (3-13) can be switched to matrix form in order to simplify the calcu-

lation of local linear model LkS.


yn 1 = P, LT X (3-14)


where

P,T = [pt,n; p2,n; ; PK,n]






-KxD
For the training data, the desire signal y,+l will be available. Therefore the

optimal coefficients for each local model can be determined by deriving the mean










square error criterion (jl!810)


min(.I) = (U+- 0+12(3-15)





The extreme (minimum) happens when the first order derivative of .7 to L is "O"


1~~ ~ ~~ Un1-Sa1Pz I = 0
(:316)





j d~= Ii I= 0 (:317)



From (3-17), using the result from (:313), each row of dJcan he expanded as


=1~~ ~ ~~ Un =1L~t ) Pr,t *Z", = 0
(:318)
=1 n+1' r,tz *It 1 ~t =1 Pk~,tz'P~z'*t'*T

From (:318), we obtain the 1\SE-hased optimization function for local linear

model Lks. The difference between winner-take~s-rell solution and this solution is that

Pk~,, will not he ah--li- 0 or 1, and it determines where and how much cluster k will

take effect.










In order to obtain the values of all the local models, we can expand equation

(3-18) to matrix format to simplify the notations






L2




LK

=R, L


where L is a vector representation of L.

Repeat (3 -18) for all the Gaussian clusters, the local linear model can then

be described as


1~I R1 1

2 ~ 22
P =-R -L (3-20)


PK RK K

SL- = -1 -


Here the kth Gaussian cluster's local linear model is the vector L 's ((k-1) x D+1)

to (k x D) elements. With all the formulations above, the basic GMM based local

linear model estimation procedure can be summarized as:

1. Accomplish state modeling using the EM algorithm, obtain distribution para-
meters (pk, k) foT eVery Gaussian cluster, conditional probability p(,|Ck), and
the cluster probability p(Ck);

2. Compute the posterior probability p(Ck|) through the B li- Im function for
each cluster;










3. Compute the local linear models using distribution information from previous
two steps and least square criterion;

4. In the testing phase, re-calculate the p(,|Ck) and p(Ck 2) giVen neW data point,
then estimate the system output through local linear models.

3.4 Improvement on GMM

Even though the EM algorithm improves the likelihood at every iteration, the

random initial conditions usually slow down the converge speed, and many times,

make the model converges to local maximum [47, 48]. In order to optimize the GMM

initial conditions, shorten the EM training time, and stabilize the final GMM per-

formance, a Growing-Self-Organizing Maps (G-SOM) algorithm is applied. Another

benefit from G-SOM is that it supplies a flexible structure to change the number

of Gaussian kernels in the GMM. Vlassis and Likas set up a criterion to determine

optimal number of Gaussian kernels [49].

The initial topology of the G-SOM network is a two-dimensional structure with

3 PEs, which will be used to describe clusters of patterns in the D-dimensional data

vector sRD. For this reason, every PE c has a D-dimension weights vector w indi-

cating its virtual position in sRD. During the self-organizing process, one new PE

will be added to the map at the end of each training iteration, preserving a triangle

connection scheme at all time as shown in figure 3-2.

The adaptation in the G-SOM model can be formulated as:

1. For every input data, locate the winning PE through the Euclidean distance
s = min dis E' w' ;

2. Increase the matching performance for s and its direct neighbors Ns, with AG,
Eb 2~ 9s n I;: En 2' I;: )O (for C 6 Ms 1V)

3. Increase the winning frequency counter -r, of a with increment a-r, = 1;






















(a) (b)

Figure 3-2. Explanation of G-SOM algorithm.
Left: Training of G-SOM; Right: Growing of G-SOM


4. Decrease all signals' -r, counters by a small value so that the mean of winning
frequency doesn't change too much;

5. After feeding in all data, one new PE is added between the PE with the highest
-r, and its farthest neighbor until the pre-defined PE count is reached, and all
the winning frequencies are set to zero.

After training, the G-SOM PEs' weights HI are utilized as GMM's means' ini-

tial value p with balanced winning frequency. The GMM's covariance matrices are

initially set to a large value to make every Gaussian kernel cover the largest pos-

sible area, and the optimal value of covariance matrix will be trained through EM

algorithm. The improved algorithm was extensively tested on the LoFLYTE data

that will be fully presented in (I Ilpter 5. The LoFLYTE simulation uses 32-D time

embedding space, and the difference of performance between the conventional GMM

and the GMM with a G SOM initialization is shown in table 3-1. Every SER result

is an average of five independent experiments. Overall, the G-SOM initial weighted

GMM makes system identification performance 2.4dB better than random initialized

GMM with smaller variance and decreases the EM training iterations.









Table :31. Performance on :32-D LoFLYTE data

GMM GMM with G-SOM
40 dB SNR noise SER (dB) Variance SER (dB) Variance
p 2:3.6:37 0.4:338 2:3.405 0.2589
q 26.012 0.2849 24.396 0.:37:38
r 24.66:3 0.3859 22.55:3 0.175:3
u 21.016 0.3574 26.352 0.2877
v 22.682 0.272:3 28.544 0.2065
w 22.829 0.3404 25.668 0.1926
Average 23.473 0.3458 25.153 0.2491
Training loops 17 4
0 noise
p 25.514 0.1968 24.721 0.217:3
q 27.828 0.2189 25.8:38 0.30:36
r 22.05:3 0.:36:31 25.696 0.1:301
u 19.065 0.2817 27.12:3 0.2085
v 26.5:30 0.1977 29.444 0.1447
w 2:3.315 0.2661 25.85:3 0.1:350
Average 24.051 0.2523 26.446 0.1899
Training loops 14 2


3.5 Comparison of the GMM with the SOM

We will make a detailed comparison of the GMM with the SOM as SOM-LLM has

been such a popular approach for SI and these two are so close to each other. The Self-

Organizing Map (SOM) is another local modeling algorithm proposed by K~ohonen

and extensively utilized in the Computational NeuroEngineerign Laboratory, with

considerable success to system identification and control. Using the clustering idea

applied to the reconstructed data in state space, the SOM sets up representative

vectors locally, with a neighbor relationship determined by the niulti-dintensional

Euclidean distance.

Yet the 1! in r~ problems that SOM cannot overcome are: the proliferation of

prototype vectors in particular in high dimensional spaces, the sensitivity to noise,










the non-smooth switching between models because of the winner-take~s-rell mecha-

nism, and the local minima entrapment during SOM training [48, 50]. Nothing can

he done for the first problem except increasing the network size, which brings also

slow training, requirement of large amounts of data, and many local linear models.

Although the SOM is insensitive to noise below the quantization produced by the

voronoi tessellation, when the noise is higher the wrong model can he instantiated.

The SOM preserves neighborhoods so the difference is not drastic, but contributes to

modeling error. Because of the difference between the LLhis, when the models switch

there may be transients in the response that also add to the modeling error for more

than one time step.

The GM13 approach divides state space in coarser regions, which improves the

poor scaling up of the SOM with dimension, and also improves the noise robustness.

However, each region may not he as well modeled by a linear model as in the case of

the SOM. However, only experimental evidence can judge this trade-off. Instead of

using winner-trake~s-rell criterion, GMM-LLM combines multi models into the current

data. In the case the Euclidean distance between data and the closest Gaussian

kernel is large, maximum likelihood criterion makes sure more than one kernel takes

effect. This phenomenon is shown in figure :3-3. Also, figure :3-3 shows that compared

with other linear membership functions, the use of non linear Gaussian distribution

can guarantee smooth switching between models. An important characteristic of the

GMM-LLM is that the weighting is changing during testing, unlike the SOM-LLM.

This will help in the overall modeling accuracy.

The local minimum in the SOM training is difficult to control during the an-

nealing process. Also the SOM cannot control the winning frequency which may














02-





-8 -6 -4 -2 0 2 4 6 8 10






Figure 3-3. Soft combination region using GMM (between dash line)


also provide added difficulties in the clustering. The GMM is also sensitive to local

maxima. However, in order to make GMM converge to global optimal, in a more

stable manner, G-SOM makes the weights such that every cluster is generated with

similar amount of winning frequency [51, 48]. This means that for the whole training

data set, each Gaussian kernel occupies about the same portion as a winner with

large output weight. Fritzke, Chinrungr-ingr and Siiquin proved the weights growing

criterion and the optimal validity respectively [52, 53].

Since the SOM algorithm is derived from heuristic ideas and this leads to a

number of limitations, some of the advantages of GMM over SOM are listed below.

1. GMM defines an explicit probability density function in data space to estimate
the distribution of the state. In contrast, SOM uses winner-takes-all crite-
rion based on the Euclidean distance [18]. GMM's soft-combination mechanism
makes smooth switching possible.

2. In GMM, the neighborhood-preserving nature of the mapping is an automatic
consequence of the choice of Gaussian functions which improves GMM's noise
robustness. Smooth neighborhood-preservation is not guaranteed by the SOM
procedure.










3. Through the relative winning frequency criterion from G-SOM construction, the
GMM training avoids the local minima entrapment which SOM alone can not
overcome .

4.GMM improves the salability of SOM through a coarser clustering of the state
space. For the same reason, the training efficiency of GMM is increased.

3.6 Summary

With detailed explanation EM algorithm, GMM has been theoretically proven

feasible on modeling analysis, and its practical modeling capability could tested on

various system. Gaussian distribution makes smooth switch between multiple models

possible. And using the soft combination mechanism, as we made the comparison

previously, GMM can largely decrease computation burden on modeling problem com-

pared with SOM hased algorithm. Gaussian models part is trained in un-supervised

way as there is no desire distribution pre-assigned. The LLM part, as desire system

response is available, will be trained in a supervised way to make sure the convergence

of error square (jl!510). Since GMM hased modeling algorithm can ease the work of

controller design, the benefits will be explained in chapter 4 and the simulation results

and the detailed performance comparison will be shown in chapter 5.















CHAPTER 4
GAUSSIAN MIXTURE MODEL BASED CONTROL SYSTEM

Within the framework of predictive control, the proposed GMM based local linear

modeling approach simplifies the design of control systems for nonlinear plants. It is

difficult to design globally stable nonlinear controllers with satisfactory performance

at every point in the state space of the closed loop system, which is especially true

in the case of unknown plant dynamics. However, the GMM local linear modeling

approach presented above, coupled with strong controller design techniques [54] and

recent theoretical results on switching control systems [55, 56], achieves the control

goal. The topology of local controllers that naturally arise from the local linear model-

ing approach is illustrated in figure 4-1, where the local controllers are directly linked

to each local model. Within the framework of GMM-LLM, the system dynamics can

be described piece-wise-linearly as


Yn+1= GL' X,, (4-1)

L p,, -X,

Aoy, + Aiy,_i + + ADyu-D

+Bou, + Blu,_l + + BDun-D


where X, is the time embedding vector [- yi, -, ui, ], LOPT Stands for the

current optimal model coefficients [- Ai, Bi, ], yis are the system states,

uis are the control signals.































Figure 4-1. Structure of Control System hased on GMM-LLM


In section 4.1, a brief review on mixture model based controller design will be

given. In section 4.2, the GMM-hased dynamical inverse controller is explained in

detail, which is later demonstrated feasible on nonlinear system in chapter 5. A

pole-placement PID control approach will be discussed in section 4.3.

Considering modeling error, measurement noise, and model uncertainty, robust

approaches to control become very appropriate for real-world applications. Therefore

various approaches have been proposed to solve the robust control problem, including

the H, approach [57], Lyapunov approach [58], geometric approach [59], etc. Lin,

Brandt, and Sun proposed an optimal control based robust control scheme [60, 61],

and we will modify it and fit it into our GMM-LLM framework.

4.1 Introduction of Mixture Model Controller

The design of predictive controllers, especially for large-scale and complex appli-

cations, depends integrally on the model of the physical plant [62, 63, 64]. Indeed, the









model constitutes the vital link between the physical problem in which the optimal

control will be used, and the mathematical realm in which it needs to be designed.

The classical approach to the control design problem assumes a known model, even

if dramatically reduced. In the case that the model complexity is reduced unreal-

istically, the consequence will be the generation of ineffective controls. For global

nonlinear modeling control approach, which is more difficult compared with local

or linear modeling control, neural networks based control scheme have been studied

extensively [65, 54, 66].

Using the Partition Algorithm (PA), Lainiotis first introduced the multi-model

partition (11 l Pl) methodology for the design of adaptive/intelligent systems, where

the control problem is decomposed into a set of easier derived subproblems [64, 67].

Given a set of local models, MMP designs the adaptive control as





where Uk 8() is the optimal control corresponding to kth sub-model.

Fuzzy control is another popular design methodology which attracts wide interest

these d we~ [68]. Among them, Takagi-Sugeno (T-S) fuzzy logic model, which applies

fuzzy rules and membership function, is a widely studied and applied fuzzy modeling

and control methodology. In the T-S fuzzy model, the system is described by a set

of if-then rules which have local models in the consequent parts. The ith rule in the

model is of the form: If zi(t) is ii T ,, and ..., and zp(t) is if 1 then


ex(t) = Aix(t) + Biu(t)


























Figure 4-2. T-S fuzzy logic membership function


The effective regions of if-then rule are overlapped with each other, and the weights

of each rule is measured by linear membership function which is shown in figure 4-2.

The control signal for fuzzy logic controls take the form of (4-2), where cOk is coming

from membership function too. Tanaka, Iwasaki, and Wang use T-S logic switching

fuzzy logic model to control of an R/C hovercraft [69].

Based on the idea of a group of simple controllers instead of a single complex

controller, switching control has been applied in recent years. A switching control

consists of a high level discrete-event supervisor and a family of low level feedback

controllers [70, 7]. Following the local modeling theorem, multiple controllers are

applied to certain regions of data. Motter used a SOM based system identification

model to design a switching controller for high speed wind tunnel control [20]. Li

and Lee use a SOM structure to cluster the state space, and then apply fuzzy rules

to design the control signals [41]. Cho et al. uses a SOM and sliding mode controller

to realize robust control [71, 72]. Based on those approaches, our research will try
































Figure 4-3. Structure of Inverse Control System


to improve the performance of multiple controller near the intersection of different

clusters.

4.2 Inverse Error Dynamic Controller Using GMM

The mission of controller design for nonlinear dynamical system could be alle-

viated using local linear model generated from GMM. Once the optimal local linear

models have been chosen from the input-output data available for system identifi-

cation, inverse error dynamic controller design can he applied to meet goals based

on the desire signal, the state signal, and the local model. Compared with the ha-

sic structure of a mixture model controller, inverse control is an extended version of

single-model control since the soft-combination happens before the combination of

control weights and inverse is applied to the unique current model. The structure of

the controller is shown in figure 4-3.










The principle behind inverse error dynamic control (IEDC) is the .I-i-opll..l~e

convergence of system error. The control methodology can he described as fixing a

set of stable poles for the tracking error signal dynanxics. If at time n the desire plant

output is denoted as d,2, and the actual plant output is y,z, the instantaneous error

is defined as e, = d,2 y,2. And the control goal is to guarantee that the error signal

follows the 1-D error dynamic function:


e,w+l + Xle, +Xel 2 1- + X.' -. 1 = 0 (4-3)


the parameter vector =~ [AI, -, XI]' is selected such that the roots of the polynomial

1 + Xlr + + A,.rz = 0 are all within the unit circle and thus the error dynamics is

stable.

If the order of polynomial is zero, the error function is simplified as e,2+l = 0.

The error function corresponds to an exact tracking controller, which determines the

necessary control input for the next error value to he "O". This strategy uses only

current model information and current control error, so its response will be instanta-

neous. And obviously it is not robust to modeling errors or external disturbances as

there is no further model and dynamic information to smooth the disturbance front

model uncertainty. Through 2nd order error dynamical function, the error signal is









filtered first, and the local model based control signal can be obtained as


e, I + Are, + Xa26-1 = 0

j [dn+1 yn+1] 16X, 26n-1,_ = 0

j [d,+l yn+,] + Xle, + X26n-1 = 0
(4-4)



Sn+1nu n,u n le~n 26- =le 0 ~,l



where model [fax ,Ls]T = L urrent, = (E" 21) is .omingIF fr.oml (3-14) and (Are, +

Xa26-1) 1S from weighted error signals of last two steps.

For different choices order in (4-3), the control law will drive the tracking closed-

loop system in various r- 11-<, meanwhile the error signal will ahr-l- .- satisfy the linear

difference equation given in (4-3). As long as the location of poles are within the

unit circle, the controller will guarantee the stable convergence of the dynamics.

Considering that the modeling error from the GMM-LLM will affect the actual pole

locations, the poles in (4-4) cannot be chosen close to one in order to guarantee

stability.

Now we discuss the stability issue for the controller. From (4-4), Xis are co-

efficients to be designed such that two poles pis are within the unit circle where

p, + p2 -1 and pi x p2 2 X. Since the LLMs are trained based on MSE criterion,

the modeling errors at each step can be deemed a bounded i.i.d. random variable

with zero mean. It can also be assumed that E' : ,] < 00, which is reasonable if the

system identification is done properly, and the measurement noise levels are finite.










From the experimental results in the simulation section, we will show that the mod-

eling error power is finite. Under those conditions and assumptions about nonlinear

uncertainties on ess, the error dynamic function is changed to


e,+l + Are, + X26n-1 = CM/,n+1 16M/r,n 26Men,n-1) (4-5)


where eM/,iS are the modeling errors at each step.

We define the right side of equation (4-5) as a. Considering the worst case when

modeling errors reach their bound, a is set to its upper bound a, (|a| < |a,| and

a, is a finite constant). The expression of e, can be obtained through Z transform

and inverse-Z transform to yield


E~z) =-
1 z-l 1+ Alz-l A2 X-2
n a n+1


-a p"1 (4-6)
(1 p2 92a pi1

As n oo, the mean of the tracking error is obtained as


lim E[e,] lim E[ ]
ntoo ntoo ( t 1-p
.1+ z+ A2
= hm E[-eM/,n]
ntoo (1 pt) (1 p2)

= [-eM/,00 (7


As for the power of tracking error, the expression can be written as

a2
lim E[e~ ] lim E ["
ntoo ntoo (1 p )2( p922
.1+ +A X2
= hm E ]'
ntoo (1 p, )2 J Pa2 2

~1E ] (4-8)
(1 + At + Xa2 2










Under the assumption that the modeling error (and measurement noise) con-

tributions eM/,n are Wide sense stationary random variables, we can compute the

.I-i-mptotic (finite) tracking error in terms of the power spectral density of eM/,n and

the coefficients As. As discussed before, the mean of modeling error is zero or small

value. Consequently, the modeling error will take effect on tracking performance

and the pole-placement problem must be solved considering the trade off between

convergence speed (faster poles, wider pass-band) versus better disturbance rejection

(slower poles, narrower pass-band).

4.3 PID Controller

PID stands for Proportional, Integral, and Derivative. PID control became a

traditional control approach in the 20th century because of its simplicity and accu-

racy. Nowedl-ll- researchers are trying to find new methods to better fine-tune the

PID parameters. Considering our system identification framework, the design of the

piece-wise linear PID controller will be accomplished once the system identification

is finished. Based on the GMM-LLM framework, the pole-placement PID design

technique could be realized based on [73]. Lately, Skoczowski et al. proposed a new

method based on two-loop model following control (j \! PC) to improve robustness [74].

The mixture model based PID control scheme is shown in figure 4-4.

Simplifying the model function (4-1) to a second order time-embedded SISO

form

Un,+l= aly, + a29n-1+ blu,+ b~i _1

The model's current Z-domain transfer function can be written as


B(z) blz2 + b2X
F(z) =
A(z) z2 01Z a2




























Figure 4-4. pole-placement PID control system.


Defining the desire signal and controller transfer function as D(z) and G(z)=

D(z)/C(z) respectively. The close-loop control signal transfer function U(z)is given


D(z)
U(z) (D(z) Y(z)) (4-9)
C(z)

From the definition of pole-placement, the goal of this approach is to map the

coefficients of close-loop system function from open loop coefficients (al, a2, bl, b2) tO

redesigned controller coefficients (ci, d ), whose characteristic polynomial could be

described as

z2 1X 2 X (4-10)

The close-loop system transfer function is then [1z 1)()BzDz] and the system

characteristic polynomial becomes


A(z)C(z) + z- B(z)D(z)


(4-11)










Using the characteristic polynomial for the system model as (4-10), along with

a second order observer polynomial term z2, the design equation for pole-placement

PID is given as


A(z)C(z) + z-1B(z)D(z) = z2 ,,2 + 1X 2~)

(z2 1%r a2) Z l)( Z C) Z -1(b z2 + b~X(l2 d1 d2X d3

= zX2 ,, X1X 2 ) (4-12)


where a (z 1) factor is added to the denominator of controller function C(z) in

order to filter the low-frequency noise. This equation could be solved for cas and d s.

Consequently the control signal is obtained from (4-12) as


u, = (1 c)u,_l + c un-2 16di, d26n-1 3 6n-2 (1


Given the LLM on the current time step, the PID controller could be easily

obtained through (4-13). Its stability will be assured as we discussed in previous

section for the dynamic inverse controller. One possible problem caused under this

framework is that the PID coefficients need to be calculated on every step as the

system's local model and the gating weights from GMM are changed.

4.4 Optimal Robust Controller

In order to achieve better control performance, we are going to first translate

a robust control problem into an optimal control problem and then apply optimal

control approaches to solve the robust control problem. We will show later that, by

properly choosing a set of cost functions that reflects the modeling uncertainty, noise,

and control requirements, the solution to the optimal control problem will be also a

solution to the robust control problem [60, 61].









D. 60.1...0 4.1: Robust control problem

Find a feedback control law a = k(x) such that x = 0 of the closed-loop system


x = A(x) + B(x)NV(x) + B(x)k(x) (4-14)


is globally .I-- phl1'1 cally stable for all admissible perturbations NV(x). ]

D. 60.7..-0 4..: Op~timal control problem

Find a feedback control law a = k(x) that minimize the following cost function


(Niaz(x) + x~x+ Ju)dt
/OO

where NL~,(x) is the power of uncertainty compensation; XTX is the cost of states

change; ufu is the cost of control signals. x~x and ufu can be replaced by XTQx and

UTRU with Q and R being symmetric and positive semi-definite to adjust the relative

weights of states and controls.

Theorem: Assume that the solution to the optimal control problem exists. Then

the solution to the optimal control problem is also a solution to the robust control

problem.

Proof: Define


V(xo) = min /7(Niaz,(X) + XTx + uT)dt


to be the minimum cost of bringing the system x = A(x) + B(x)u from xo to the

equilibrium "O". The Hamilton-Jacobi-Bellman equation [75] gives


min[Niaz,(x) + XTx + U n + V, (x)(A(x) + B(x)U)] = 0


1 We assume the perturbation is admissible, so we use the same B(x) as coefficients.









where V,(x) = 8V(x)/8x. Thus, if a = k(x) is the solution to the optimal control

problem, then


Imm,,(X) + XTx + UTU + ,((x)(A(x) + B(x)k~(x)) = 0

2kT(x) + V,(x)B(x) = 0 (4-15)

Since V(x) satisfies


V(x) > 0, x /

V(x) = 0, x= 0


Also, V(x) = dV(x)/dt < 0 for x / 0, because


17 = (8V(x)/8x)"(dx/dt)

= (x) [A(x) + B(x)NV(x) + B(x)k(x)]

= (x) [A(x) + B(x)k(x)] + V,(x)B(x)NV(x)

-1Va,(x) xTx k~(x)Tk(x) + ~((x)B(x)NV(x)

-1Va,(x) xTx k~(x)Tk(x) 2k~(x)TN(x)

-1Va,(x) + NV(x)T1N(x) (NV(x) + k(x))T(NV(x) + k(x)) x~x






Thus the condition of the Lyapunov local stability theory are satisfied. The equi-

librium x = 0 of (4-14) is globally .I- a.i-'1 ;cally stable for all possible uncertainty

NV(x), i.e. u = k(x) is a solution to the robust control problem. Consequently, there

exists a neighborhood Ni~= {x : ||X|| < c} for some c > 0 such that if x(t) enters Ni,









then

lim x (t) = (4-16)
t->oo

But x(t) cannot remain forever outside Ni, otherwise ||x(t)|| > c for all t > 0. There-

fore,


V(x(t)) V(x(0)) =t V(x(-))dv
0t






Let t oo, we have

V(x(t)) < V(x(0)) c2 -0

which contradicts the fact that V(x(t)) > 0 for all x(t). Therefore (4-16) is true no

matter where the trajectory begins [60]. Consequently the stability is assured without

considering the initial condition.

4.4.1 LLM-based Optimal Set-Point Controller

Typically, the solution of the optimal controller problem for time-invariant time-

deb i-. I (TITD) system is obtained by solving the infinite-dimensional algebraic Ric-

cati equation which requires the solution of simultaneous partial differential equations.

For the mixture model control case, we have proven that, GMM-LLM modeling error

is bounded and not larger than single ARX modeling error. And the control error,

under the condition that the controller is stable, will .I-i-mptotically converge to the

modeling error bound. Moreover, as the Gaussian local model is changing at every

step, the Riccati equation will need to be calculated on every step.










The computational technique, which was originally proposed by Yashiki and

Matsumoto [76] on SISO single delay system and later expanded by K~lein and Ramirez

et al.[77] on MIMO multiple d. lIn-s system, needs to be transformed to our one-step

prediction model into state variable format for the purpose of controllability checking.

The general case of MIMO systems exhibiting multiple d. 1 .1-< is written as (4-1). And

the corresponding state space model for MIMO system can be expressed as


Y1,n+1


=AX, + BU, (4-17)
YD,n+1

Yu+1

0 0 Al y i,n B1


+ U,
0 AD YD,n BD

I I Ao I7 y Y Bo

where only the y, I is the actual system state. The remaining vector yl, yD is

called pseudo-state vectors, which represent the time-d. 1 li-. I portion of the system

states. The pseudo-state is defined as


(4-18)


Yi,n = Ai yi,,_i + Bi u,_i i = 1, D









A main advantage of this transformation is for the controllability test. Defining

A and B as the coefficient matrix and vector in (4-17)


At


,B
AD

Ao

is simplified


I I

the standard state controllability test

matrix S


B1



BD

Bo

to check the rank of the state


rank(S) = rank[ B AB -. A+Cdi-1g

In our experiments, all the LLM give non-zero results, and since the rank of S is

related to the dimension of LLM is relatively low (S = [B, AB] for 4-D LLM case),

the controllability is assured. Only in the case when LLM have a large part of close-

to-zero coefficients, which also means LLM are not well trained, the state matrix will

not be full rank.

To realize the design of robust control signal, we start the cost function J as the

basic design rule:


J= XQ,+U U)


(4-19)


Considering the modeling uncertainty and measurement noise, (4-19) can be

modified as


J = (N(X,) VN(X,) + XIQX, + UfRU,)


(4-20)










Here the modeling uncertainty and measurement noise are combined with the

state itself. In other words, the physical dynamical changes because of the model

uncertainty can he recognized as the increment of system dynamics which will in turn

cost more energy and larger control signal to overcome them. This is a reasonable

explanation as modeling error appears as a warping of system states. The solution

from this kind of function forms the traditional linear quadratic regulator (LQR)

problem and can he obtained through the solution from the discrete algebraic Riccati

equation (DAR E)


ATXA X ATX B(BTX B + R)- BTXA + Q = 0 (4-21)


There have been several methods to compute the unique stabilizing numerical

solution P = X of the discrete-time algebraic Riccati equation [77], from which the

gain vector and consequently the control signal are computed as


G = (BTPB + R)-1BTPA (4-22)

U,z = GX,z (4-23)


where X,z is the re-formulated current input with pseudo state elements, A and B

are from state space form of real system model or i4 and B from state space form of

LL1\. The schematic diagram of the optimal robust control system is shown in figure

4-5.

One possible disadvantage for the optimal robust controller is that the related

computation is larger than those of inverse controller, PID controller, etc. Especially

under the condition of mixture model, theoretically there will ahr--1-- he a change on

the local model as system dynamics are changing. Since DARE computation needs































Figure 4-5. Optimal robust control system schematic diagram.


to be finished right after the generation of Gaussian mixture model, consequently the

DARE (4-21) and the control gain matrix (4-22) need to be re-calculated at every

time step, which might he a challenge under some on-line training environment.

Necessary trade-offs need to be made when resources are limited.

4.4.2 LLM-based Optimal Tracking Controller

Designing front the decrenient of the cost function, the control signals) will

make the system dynamics converge to a zero equilibrium point. For many real

control problems, we want the system to be capable of taking some actions instead of

converging to zero. In order to make the system follow non-zero desired tracks, the

cost function needs to calculate the control gain matrix such that it can calculate the

convergence of not only modeling uncertainty, but dynamical error and the change of

control signal as well. Thus the new cost function (4-20) needs to be further modified









for state errors and change of control signals

.7 = :(N(X(l)V'',r7)l VNX) fQ,+ U.d, (4-24)


= :( E( Q E + d Uf Rd U,,)


where E, is the dynamical error of system states, dU, is the difference of current

control signals and their previous record, Q and R are symmetric and positive semi-

definite weighting matrix. Again, we assume the modeling uncertainty and noise

can be combined with state errors since errors can be described as distance between

states and non-zero equilibrium. Correspondingly, the transfer coefficients for error

signal and control signals need to be changed. Starting from the system state transfer

function in (3-14), the error transfer functions can be described as


U7n+l = alX + a2 n-1 + blun + b2un-1

en+1 dn+1 yn+1 dn+1 yn+1

= i+l (a x, + a2 n-1 + blu, + b2un-1)

=d,+l al(d, e,) a2 dn-1 en-1) blu, b2un-1

(dn+1 01du a2dn-1 b2un-1) + 16, + 26n-1 blu,

=ale, + a26n-1 bl(u, by(d,+i aid, aadn-1 b2un-1)

=ale, + a26n-1 bl(u, C,) (4-25)


Defining il, = (d I aid,, a~n-1l~ b2Un-1l) aS a pseudo control signal, the new

state space transfer function for the error dynamics and the control increment [E, dU]



E, 1= A- E,+B-dU,, A- I:,B I (4-26)
1 a -bi










Considering the modeling uncertainty, the none-zero set point control and track-

ing control problems can be realized through optimal control law (4-24). The new

discrete Riccati equation could be calculated in the same form with modified coef-

ficients (A, B) in (4-26). When the gain matrix from discrete Riccati equation is

calculated, the current step control signal can be expressed as


d U, = G E,

U, = Us + dU, (4-27)


In (4-24) and (4-27), we use the upper bound of modeling error to describe

the model uncertainty, and the real the model uncertainty and noise will be a ran-

dom values which on average are smaller than that. Consequently under small noise

and model uncertainty situations, the transient performance from optimal controllers,

compare with other control approaches, might not be the best. However, since opti-

mal controller is considering the long-term effect of disturbance, its performance on

robustness will outrun all the other competitors in large noise cases. In chapter 5,

we will simulate and discuss the performance of optimal controller on different real

systems with disturbance-free case and the case with noise and/or model uncertainty

presents. Neglecting the computational cost, optimal robust controller is the optimal

choice for model based prediction controller.

4.5 Summary

GMM based mixture model controllers are close to switching controller, yet they

consider the situation of combining weights, which dramatically decrease its compu-

tational cost comparing with SOM-similar vector quantization methods. For different

controllers, the Gaussian models provides a simple linear description of the current










system dynamics as reference model. Under the condition that the Gaussian model-

ing systems are stable, all controllers can give a stable result where optional robust

controller will theoretically give the best tracking performance. A plant model is

rarely a fully accurate description of the real plant. Neglected dynamics, approx-

iniately known parameters, and changes in operating conditions all contribute to

plant-niodeling errors that can jeopardize controller performance. In the following

chapter, we will answer the problems including the performance of the controllers in

real systems, which includes accuracy and robustness, performance curve vs. number

of PE, and performance comparison of GMM-LLM with other modeling approaches.

Among all the options, ESN controller is quite different compared with others,

which will be discussed in details in appendix. Alany papers mentioned that the

training of ADC and/or BPTT algorithms will sometime inevitably lead to unstable

control results because of its computational complexity. Besides the neural networks

(ESN here), BPTT length and the change of learning rate will also take effect on the

final performance. We will start froni some nonlinear systems with -!np!I-~" setup,

use it for set-point control.















CHAPTER 5
APPLICATIONS AND DISCUSSIONS

The GMM hased local linear modeling and control techniques which have been

presented in previous chapters need to be tested on various system in order to prove

their advantages including feasibility, simplicity, and robustness, etc. As an inde-

pendent candidate, ESN will be used for system identification and partly be used

for system control. In this chapter, different GMM-hased control approaches will be

tested on a set of system including chaotic time series prediction, synthetic SISO

linear system identification, SISO nonlinear system identification and control, NASA

LoFLYTE wave-rider aircraft identification and control. A complete set of simula-

tion and experimental results will be provided front the application of these principles

to the listed problems. Among all the systems, the Lorenz system, as a traditional

chaotic system with high order nonlinear dynamics, and the missile system, as a tra-

ditional SISO control system, will be discussed in details about system identification

performance and control performance respectively. For all the sample systems, we

will assume that the system transfer functions are unknown, and the reference models

will be constructed based on I/O measurements.

5.1 Chaotic Systems

The two chaotic systems we present here are the Lorenz system and the Duffing

system which demonstrate irregular, complex behavior. Even though they are deter-

nministic systems, their dynamics show unpredictable character. Recently considerable

effort has been in the attention of nonlinear dynamic literature since removing chaos







64






50.


N 20


0 200 400 600 800 1000 1200 1400 1600 1800 2000 2
10 1
60 0 1
y(t) -105
-20 -5 x(t)
0 20 40 60 80 10 120 10 160100 200 -30 \-15

(a) (b)

Figure 5-1. Dynamics of the Lorenz system
Left:x (top), y (middle), and z (bottom), Right: 3-D display.


can improve system predictability and avoid fatigue failure of the system. Several

methodologies have been applied for the identification and synchronization of a vari-


ety of chaotic systems. For example, a generalized synchronization of chaos through

linear transformation [78], adaptive control and synchronization of chaos using Lya-


punov theory [79], and an adaptive variable structure control system for the tracking

od periodic orbits [80]. The contribution of our work lies in the design of different

control systems based on GMM-LLM approach.

5.1.1 The Lorenz System

One widely studied chaotic system which is also autonomous system, the Lorenz


system equation (Lorenz, 1963), formulated to model the convection process in the

atmosphere [81]. The Lorenz system can be described as:



x = o--(y- x)


y7 = p.-x-y-x-z (5-1)


z = -f -z+x-y7










where, y, and z are the measurements of fluid velocity and horizontal and vertical

temperature variations. The Lorenz system can exhibit quite complex dynamics

depending on the parameters. In our experiment, parameters are set as o- = 10,

p = 28, and P = ( to obtain chaotic dynamics and the first state x is set as

the output. Using 4th order Runge-K~utta integration with time step 0.01, 9000

samples were generated for analysis. Using the Lipschitz index analysis the embedding

dimension is determined to be 3 and the embedding delay -r = 2. The GMM training

data is constructed as


Xn = [x(n 1), x(n 3), x(n 5)] (5-2)


The first 8000 samples from the time-series are used to create the reconstructed

state space through eight-kernel GMM algorithm, and local linear models are trained

for one-step prediction. Another 1000 samples are used for signal to error (SER)

performance test Figure 5.1.1 shows that the estimated output is quite close to the

original Lorenz signal. And figure 5-3 shows the optimal distribution of Gaussian ker-

nel among the state space. In order to further compare the identification performance

between GMM and other approaches, we plot the figure to verify the performance

increment of GMM-LLM as number of PE increase. From figure 5-5 and table 5-1,

we find that "soft-combination" enable GMM to use fewer PEs. The total amount of


1 Performance is evaluated by SER=C,(d )/ C,(e )





















0 200 400 600 800 1000 0 200 400 600 800 1000

(a) (b)

Figure 5-2. System identification performance on the "x" of the Lorenz system.
Left: using GMM: Original and identified signals (top) and corresponding error (bot-
tom), Right: using ESN model (top) and corresponding error (bottom).


parameters in SOM-LLM and GMM-LLM approaches can be expressed as


PSOM/-LLM/ = NVx Dx 2

PGMMn/-LLM/ = NVx Dx 3+ NV


where NV is the number of PE, D is the dimension of each PE. Considering the

comparison of GMM with SOM based modeling performance mentioned in chapter

3 and some other methodologies, it can be concluded that, as GMM uses far less

number of PE, GMM based modeling algorithm can achieve similar performance

with less burden of computation.

In order to make a complete comparison, along with traditional methods RBF

and TDNN as two references, two 300-PE ESNs are constructed. One ESN is designed

with even distribution of spectral radius [36, 37], the other comes with optimized de-

sign criterion where larger part of internal weights distributed along the maximum

spectral radius. Both use the same MSE training criterion. In table 5-1 we see that

ESN can make better one-step prediction compared with the other approaches. Since









Table 5-1. Performance on the Lorenz system

Approaches (# of PEs) # of parameters SER (dB)
RBF (50) 203 31.05
TDNN (50) 250 29.23
SOM (10 x 10) 600 33.67
GMM (8) 80 29.16
ESN1 (300, 1D input) 300 35.05
ESN1 (300, 3D input) 300 43.40
ESN2 (300, 1D input) 300 36.54
ESN2 (300, 3D input) 300 45.20


2_2 1 1 5 0 5 1 520

Figure 5-3. Distribution of the Lorenz system (dash line) and GMM weights (star)

the ESN training is focused on the linear readout vector, the training for larger size of

ESN can still be realized within reasonable amount of time. An interesting phenom-

enon in the result is that ESN rely less on the embedding dimension compare with

other approaches, as one dimensional input already generate comparable prediction

performance. And that complies with our theoretical explanation in chapter 2 and 4.

Once the GMM-LLM structure is optimized according to the proposed method-

ology, the controller design can be accomplished using standard linear feedback con-

troller design techniques. Since the Lorenz system is originally an autonomous system,













1 5


0 500 1000

0. 1

O 500 1000




0 500 1000
0.5



0 500 1000


0.5



0 500 1000

0.5

0 500 1000
1.


0 500 1000
1.


0 500 1000


Figure 5-4. Weights change of 8 GMM kernels,


the control could be realized for dynamic compensation. Given a desired signal do,


the controller needs to find a control signal u, such that the system output error


converges to zero. Under the condition that model prediction is close enough to the


real dynamics, the Lorenz system will follow the control signals) to the equilibrium


point.


lim |8, y,| = 0
ntOO


For the Lorenz system, the 3-dimensional control signal vector is


(5-3)


u, = x, I+ T (d x,)


fe XnI + T (d, x,)


(5-4)


where one control signal corresponds to one state variable, and the predicted state


vector fe,41 is obtained from GMM-LLM prediction. T is the error compensation


coefficient between [-1,1], which confirms the error converges to zero. Following the



































:



o uy -v --- v


o ,,1~------4

-t~--~-~-yl


)I
//
ii II i~/il
I I
I -ii


26
S24
22
20
18 -


10 20 30
# of PE


40 50 60


Figure 5-5. Performance comparison between SOM and GMM


Figure 5-6. Compensation performance
Left: stabilizing x (top), y (middle), and z (bottom), Right: synchronizing x, y, and
z (upper three). Control signals start from data point 1000 for both cases.


condition of desire signals as equilibrium points or a totally different dynamics, we

can realize a stabilizing and synchronizing control. Figure 5-6 shows stabilizing and


synchronizing performance where the controller takes effect at point 1000. We found

that for both cases, the controller can compensate the dynamic error very fast and

guarantees the convergence. One factor needs to be emphasized here is that the

Lorenz system, being a chaotic system, is very sensitive to appropriate modeling.


SOM
GMM










Otherwise the controller will not converge the dynamics to equilibrium in case the

system identification performance is not close enough to the real values.

5.1.2 The Duffing Oscillator System

We now consider another kind of chaotic oscillator system, the Duffing oscillator

system. The most general forced form of the Duffing equation is


x + 6x + (pZ3 0ZL'O) = 7COS(Lot + 4) (5-5)


Depending on the parameters chosen, the equation can take a number of special

forms. For example, with no damping and no forcing, 6 = y = 0 and taking the plus

sign, the equation becomes

x + Lo'x + PZ3 = 0


This equation can display chaotic behavior. For P > 0, the equation represents

a h! Id -pin g1 and for p < 0, it represents a "soft -pII' ".. If P < 0, the phase

portrait curves are closed [21].

Without losing generality, we simplify the duffing oscillator with control signal.

The system function can be re-written into state space format as





xi = x2

x2 = u 6 2 0 1l p: + rCOS(wt) (5-6)


with corresponding constant coefficients [6, wo, P, 7] = [0.4, -1.1, 1.0, 1.8] and w = 1.8

[82, 25, 21], and the control signal is set as |u| < 5.

Using 4th order Runge-K~utta integration with time step 0.2s, 8000 samples were

generated for analysis. The GMM-LLM is trained with the first 5000 points, and the









X1 direction





,'~ 0 5 100 150 200
*Y: ."'I C2 diredlon



2 -15 0 05 15 20 50 100 150 200



Figure 5-7. Dynamics of the Duffing oscillator as an autonomous system.
Left: State space description, Right: Time series.

Table 5-2. Performance on the Duffingf oscillator

MODEL (# of PEs) # of parameters SER (dB)
RBF (50) 254 27.94
SOM (64) 512 28.06;
ESN (300) 300 30.75
GMM (8) 104 26.46


remaining is used for modeling testing and control tracking testing. The embedding

dimension is determined as Xk = rth, y~lUk-, Gk G1]. From experiments, the size

of Gaussian models is set as eight PEs. For ESN part, the same amount of data

is used for both training and testing. The data embedding is set as "1" dimension

(Xk r tle Uk]T), and 300 internal weights are used for echo states.

The identification performance on duffing system is shown in figure 5-8, and sam-

ple of Gaussian distribution is shown in figure 5-9. Their SER performance long with

comparison with single model performance are listed in table 5-2. Considering the

structural difference, we will mainly compare GMM-LLM with RBF and SOM based

model. We find that GMM is using far less amount of models, which means a simpler

structure compare with other modeling paradigm, and the prediction performance is












1 ~2 P ree lon



30 20 0 0 0 0030 200 400 600 800 1000
Prediction error
02 01




-02 -01
0 200 400 600 800 1000 100 200 300 400 500 600 700 800 900 1000



Figure 5-8. System identification performance on the Duffing system
Left: GMM-LLM results; Right: ESN results.


still very good. Figure 5-9 shows that for a large part of the testing, soft-combination

is taking effect among the Gaussian models, which complies with our explanation in

chapter 3. ESN, on the other way, is using less input dimension, and the prediction

is one of the best. Modified ESN, from now on, will be our primary option for the

echo state setup as it is producing better results than its previous design.

Next, we try to realize two control missions based on Gaussian models. The

first sets system dynamics to the zero equilibrium point; the second makes the sys-

tem track a different dynamics. As the original dynamic is set as [G, of,7] =

[0.4, -1.1, 1.0, 1.8] and w = 1.8, the new one is set as [6, wo, P, 7] = [0.41, -1.1, 1.0, 2.0]

and w = 1.9.

For these control missions, we use the PID controller, sliding mode controller, and

optimal robust controller. Considering the significance of the problem, the robustness

comparison of different controller will be discussed in the nonlinear SISO system. For

PID controller, the poles are set as p1,2 = 0.6 to ensure stability and good converging

speed. The parameters for sliding mode controller are set as cl = 1, c2 = -2.5, eT =

0.0001, qT = 0.3 [83, 84, 85]. For the optimal controller, the gain matrix is designed
















05


800 850 900 950 101


0s


800 850 900 950 101


0,


800 850 900 950 101





800 850 900 950 101


05


800 850 900 950 1000


0


800 850 900 950 1000


05


800 850 900 950 1000


05


800 850 900 950 1000


Figulre 5 9. Sample of weights change of 8 GMM kernels,







Q1 -I R = 0.01



We compare the performance of controllers using 200s trajectory regarding falling


time, settling time, and steady state error. Because of the relatively low SER system


identification performance, none of the controller can make the Duffing system dy-


namic converge to desire track perfectly, as shown in figure 5-10 where Os ~ 10s of


dynamics is shown. Roughly -p.' II:;n the control performance of all three controllers


are close to each other. In the set point control mission, however, optimal robust con-


troller and sliding mode control clearly have shorter settling time and smaller steady


state error over PID controller. Regardless rising (falling) time and settling time in


the second figure, the PID controller shows large offset where system dynamics reach


highly nonlinear range, e.g. at the start and near the extrema. Since the modeling for








74





12
Desire


SSliding
08-

06-

04-

02-




-0 2

-040261




12
Desire


SSliding
08-

06-

04-

02-



-0-

-0 2

04024 6 810




Figure 5-10. Control performance of different controllers on the Duffing system

Top: Set point control, system dynamic starts from 1; bottom: Tracking control, new

track comes from a different system.



the Duffing system does not consider noise, optimal robust controller does not show


unique advantage here. Considering the robustness of optimal controller over distur-


hance, we can expect that it will outperform other competitors under the situation


where modeling uncertainty and noise are both present.










5.2 SISO System

5.2.1 Linear SISO Mass-Spring System

Now we start to study single-input-single-output systems. In order to keep the

consistency of our examples, we will start with a linear system to check the basic

capability of our approach and then follow with nonlinear systems. The linear SISO

system example is designed based on a single mass spring damper system, where the

dynamics of the system can be easily described by two states as


x z = 2 (5-7)


Z2 = -[kxt-bZ



where input xi's are states and xl describes the movement of the mass, y7 is the

output, and a is the external force [86]. In the experiment here u is limited within

+0.5, other parameters are set as m = 5, k = 100, and b = 0.1. The sample of system

dynamics is shown in figure 5-11.

The amount of Gaussian models is chosen as 6 from experiments. The 6-kernel

GMM was trained with 5000 samples of input-output pairs obtained from equation

(5-7) using 4th order Runge-K~utta integration with time step of T, = 0.05s, with a

being i.i.d. uniformly distributed random signal. The system identification data was

constructed with embedding dimension of 4, according to Lipschitz index, for input

and output both being 2,. Thus Ir(u7) =[xy (u,), xz(n7 1), U(u7), u(n2 1)]. For. ESN

mnodeling, the input dimension is again decreased to kIu) = [xi~(u), u(u1)].

The identified models were tested on original 2000 samples data, generated using

a new sequence of random input signal. The actual plant output, the model predicted











0 01



O 500 1000 1500 2000
01

M 0

-0 1
0 500 1000 1500 2000
05

-30

-0 5
0 500 1000 1500 2000



Figure 5-11. Dynamics of the mass system.
top: X1; middle: X2; bottom: control signal.

Table 5-3. Performance on the linear mass system

Model (# of PE) # of parameters SER (dB)
SOM-hased (8 x 8 = 64) 512 50.75
RBF (64) :324 47. 30
GMM-hased (6) 78 47. 24
ESN (200) 200 49.05


output and the estimation error for the GMM-hased LLM models are provided in

figure 5-12. Figure 5-13 shows that cooperative modeling is taking effect during

testing. The SER performance is 47.2:388 dB. The same data is applied to SOM and

RBF to make a comparison. Table 5-3 shows that GMM-LLM does not generate the

best result. Yet considering the amount of PE used in GMM-LLAL GMM-LLM is a

highly efficient modeling method.

The control mission for the plant can he accomplished through a series of con-

trollers. The zero order inverse controller will directly replace next step prediction

with desire signal. The PID controller design is carried out in the way of standard

pole-placentent technique. The corresponding PID coefficients are determined to











x 103 Single readout x 103
10 10
-Desire -Desire
Prediction Peito
5 5



50 200 400 600 800 1000 50 200 400 600 800 1000
x 10 Prediction error x 103




04 20 0 0 0 0050 200 400 600 800 1000



Figure 5-12. Sample of system identification performance on the Mass system.
Left: GMM-LLM results; Right: ESN results.


bring the close-loop response poles to 0.05 + i0.3 and 0.05 i0.3 in order to decrease

the transient overshot, which are all verified in the simulation of cross-validation. The

sliding mode controller, as a strong candidate for the robust control, is considered

as a reference with parameters cl = 1, c2 = -2.5, eT = 0.0002, qT = 0.5, which are

fine-tuned results from simulation. For the optimal robust controller, the gain matrix

are designed as


Q] -~ ,s1 R = 0.25



The set-point control mission is set as: system starts at a value of 0.5, and desire


signal is 0 for 3 second, then the desire signal is changed to 0.3. In order to identify the

robustness of different controllers, another set of experiment with 30 dB SNR noise

is added to the system dynamics as modeling uncertainty. The final performance

is shown in figure 5-14. We can see that in the noise free case, all controllers have

similar settling time and small steady state error. PID controller has big overshot,

short falling time. Sliding mode controller has longer falling time yet small overshot.


Optimal controller has both short falling time and small overshot. In 30dB noise case,












0~ 0

0 0 0 05 300 400 500 0 100 200 300 400 500



0 100 200 300 400 500~L~ 0 10 20 30 40 50


0 100 200 300 400 500 0 100 200 300 400 500


Fiue5-3 aml f egtscageo G Mkrnl orms sse

al hrecotoles onothae uc ife nc onflig(iig tmstln ie

and ovrht h piaotolrovosl upromohrtoo h rtro












dyaic can be3 describe bywihscag f6GM enl o asss




and oersh t. =h o 2 0.1costolerxbi)(5x y uprfr 4xfe +w x() -h 0.5 osti 5-8









whr nputhe nisO te rudder delctnione and in pactc n nisa limited withine +1whic

corresponds to he ra dngei of ruder aisl +90 degree. .Tesmlfidtos







79



Output & Desire signal
--Desire Desire
0 5 *-Optimal 0 5 Opma
-- Sliding Sliding
04 ~

03 0



01 01



-01 -01
0 05 1 15 2 25 0 1 2 3 4 5



Figure 5-14. Set-point control performance on the mass system.
Left: noise-free case; Right: 30dB SNR noise case.


60 10

0

-10
0 200 400 600 800 1000




0 200 400 600 800 1000




-_010 -5 0 5 10 0 200 400 600 800 1000



Figure 5-15. Missile system dynamics.
Left: state space description; Right: X1, X2, and control (top to bottom).



The GMM was trained with 6000 samples of input-output pairs obtained from


equation (5-8) using 4th order Runge-Kutta integration with time step of Ts = 0.05s,


which corresponds to 300 seconds of flight time. In order to excite a rich variety of


dynamical modes in the plant, the system identification data was generated using a


uniformly distributed random input signal within the specified limits [2]. GMM-based


LLM approach developed a eight-mode mixture models, resulting in eight cooperative


linear models. The embedding dimension, according to Lipschitz index, for input


and output were both selected to be 2, resulting in 4-coefficient local models. This











Desre -Desire
5 ~Prediction ) -Pred iction



-10 -10
0 200 400 600 800 1000 0 200 400 600 800 1000
05 05




-50 200 400 600 800 1000 -50 200 400 600 800 1000


Figure 5-16. Sample of system identification performance on the missile system.
Left: GMM-LLM results; Right: ESN results.

Table 5-4. Performance on the missile model

Model (# of PE) # of parameters SER (dB)
SOM-based (8 x 8 = 64) 512 31.70
RBF (100) 504 33.03
GMM-based (8) 104 31.00
ESN (300) 300 36.25


embedding delay dimension was also chosen in accordance with the dimensionality of

the state dynamics.

The identified models were also tested on original 50s-length data (1000 samples),

generated using a new sequence of random input signal. The actual plant output, the

model predicted output and the estimation error for the GMM-based LLM models

are provided in figure 5-16. The ESN is constructed with 300 PEs, and input is set to

state without time embedding. The SER performance, compared with 31.7dB from

64-PE SOM in table 5-4, is 31 dB [18 .

The control mission for the plant can be accomplished through a series of con-

trollers. The zero order inverse controller will directly replace the next step prediction












1 0.

00 0 I
O 100 200 300 400 500 0 100 200 300 400 500




0 100 200 300 400 500 0 100 200 300 400 500
0.5 -1



0 100 200 300 400 500 0 100 200 300 400 500

h1 l i l

0 100 200 300 400 500 0 100 200 300 400 500



Figure 5-17. Sample of weights change of 8 GMM kernels for missile system


with the desired signal. The PID controller design is carried out in the way of stan-

dard pole-placement technique. The corresponding PID coefficients are determined to

bring the close-loop response poles from the plant output to 0.5 + i0.1, and 0.5 i0.1i.

The sliding mode controller, as a strong candidate for the robust control, is considered

as a reference with parameters cl = 1, c2 = -1.85, eT = 0.0001, qT = 0.3 [8:3, 84, 85].

For the optimal robust controller, the gain matrix are designed as



Qi -: O3 R = 0.55



As we discussed in previous chapter, control performance will be determined by

modeling performance as well as controller performance. Under the condition that the

controller is stable, the control performance will eventually converge to the minimum

of modeling error. In order to deliver the results more clearly, we show the figures in











--MPID-GMM
-MPID-SOM
**** PID-MA
-TDNN


25 2 5 6 5 2 6 6 6
Time (seconds)


Figure 5-18. Performance of PID controller with different modeling approaches.


the following sequence: we show the different-model-same-controller results first to

verify the effect from modeling part, then the same-model-different-controller results.


Figure 5-18 di ptlli-, under noise-free condition, the set-point control perfor-

mance of PID controller with different modeling apps... llity! plus TDNN controller

as a reference. Recalling the system function (5-8), the region around -1.5 is where

nonlinear dynamics is taking the largest effect. And figure 5-18 clearly shows that,


1). the single model PID (PID-MA) has longer settling time and the largest vibra-

tion before the convergence because of its modeling performance; 2). TDNN, due to

its structural difference, has the shortest falling time yet the longest settling time;


3). the Gaussian mixture model PID controller (\!P'ID-GMM) has very close per-

formance to the SOM multiple model PID controller (\!P'ID-SOM); 4). MPID-SOM


gives the most stable performance and it needs more training. For now we can con-

clude that GMM-LLM, when compared with other modeling approaches, can save

much computational cost and the final control performance to a large extent.











Output & Desire signal
0 6 Desire Desire
PIDPID
-- -Optimal Optimal
0 4 Sliding04 Sliding

02 0




0-08

-0 84


55 6 65 7 75 8 85 9 95 556 65 7 75 8 85 9 95


(a) (b)
1 1
PID PID
08 -*-Optimal 0 Optimal
-Sliding -Sliding
06C 06










u012345678 u012345678


(c) (d)


Figulre 5-19. GMM-LLM based control performance on the missile system.
(a): Step response without noise (output and desire); (b): Step response with noise
(output and desire); (c): Tracking response without noise (output and desire); (d):
Tracking response with noise (output and desire).



Now we test the performance of controllers under the condition that modeling


uncertainty and measurement noise is present. The closed-loop nonlinear system is


tested in two cases: one with various step changes to the desire output and one


with a smoothly changing desired output, which is the sum of several sinusoids with


different amplitudes, phases and frequencies. To simulate the worst condition, the


disturbance is considered modeling uncertainty. After generating the training data,


we add 30dB SNR noise to the system model during testing. In that case, modeling










Table 5-5. Control performance comparison on missile system considering noise

Set-point Set-point Tracking Tracking
Error Mean Error Variance Error Mean Error Variance
MPID-GMM -0.0289 0.0272 0.0070 0.0108
Optimal -0.0222 0.0186 0.0048 0.0072
Sliding mode -0.0272 0.0202 -0.0024 0.0101












Time (Sec)

(a) (b)

Figure 5-20. ESN control performance on the missile system.
(a): Noise free set-point performance, (b): Set-point performance with 30dB noise.


effect is negligible and control performance is emphasized. From figure 5-19 it shows

that the step and tracking performance of all the designed nonlinear control schemes

in close-loop operation with actual plant model is very satisfactory. GMM-LLM

modeling error is insignificant in the final performance. Comparing with other two

approaches, PID needs the least calculation for coefficients. Considering the detail

of figure 5-19, as in table 5-5, the difference among those approaches are noticeable,

in which the optimal robust controller demonstrates the superior performance for

disturbance rejection compared with others.

Finally, we try the ESN controller for set-point control in order to verify the

feasibility of ESN controller. Following the structure in chapter 4, we construct a

300-PE ESN controller with random readout vector. The input to the controller is

the current system states without time embedding, and the model information is










derived from the Gaussian mixture reference model. BPTT length is set to 100 step,

which corresponds to 5 second in real system. The whole system is recursively trained

in 15 iterations. The final control performance is show in figure 5-20. In the figure,

we can see that the ESN controller can accomplish the regular control mission, yet its

performance is not comparable to the optimal robust controller. Since the advantage

of ESN is its generalization capacity, simplicity for training, and its independency of

time embedding input, further investigation needs to be made for ESN hased robust

controller.

5.3 Nonlinear MIMO System: LoFLYTE System

The last and the most complex application we have considered is a MIMO nonlin-

ear dynamic system, the LoFLYTE System from NASA. The LoFLYTE System has

6 degrees of freedom, 3-dimensional moving and 3-dimensional rolling, and 4-channel

inputs. Since the flight motion can he divided into longitudinal motion and lateral

motion, the modeling and control mission can he simplified into 2-input-2-output

mission. We will design MIMO system identification model first, and realize inverse

controller and optimal robust controller based on that.

5.3.1 System Introduction

The nonlinear MIMO dynamic system discussed here is the NASA LoFLYTE

tested aircraft, an unmanned aerial vehicle (UAV), which is designed and tested by

Accurate Automation Corporation (AAC). The LoFLYTE tested flies slower than

the speed of sound, Alach one, but they have the same i.-- i.; !II1 shape designed

for Mach five flight. The term i.-- i.; !II1 refers to the fact that aircraft of this

type ride on the shock waves that they create as they fly above the speed of sound.

Other supersonic and hypersonic aircraft suffer reduced performance because they



















Aileror1 ,2. I~
.. RUdder
Throttle




Elevatr 2


Figure 5-21. General description of an aircraft


fight against the shock waves. The waverider shape improves fuel consumption by

decreasing air resistance at speeds greater than Mach one. This full scale aircraft will

take off horizontally, then it will use air-breathing engines to accelerate to a cruising

speed of Mach five at a very high altitude. And it will end its flight by landing on a

conventional runway. The task here is to develop modeling and control strategies for

LoFLYTE based solely on input-output data.

According to classical aerodynamics, the flight dynamics of any rigid body are

determined by movements along and around three axes: roll, pitch (longitudinal

motion), and yaw (lateral motion). Besides the strong coupling between system

states, the main control effect can still be classified. The elevator be is the main effect

for controlling the longitudinal motion state variables (pitch angle, 8, and pitch rate,

q). The rudder 6, is primarily controls the lateral motion state variables (yaw angle,

pai, and yaw rate, r). The aileron be mainly controls the roll motion state variables

(roll angle, phi, and roll rate, p). Finally, the throttle be largely controls the aircraft's

longitudinal speed, and for some planes, deflectable thrust vectors might allow yaw


























Figure 5-22. Dynamics of the LoFLYTE system.
Left: p (top), r (bottom), Right: Control signal, aileron (top), rudder (bottom).


and roll contributions from the engine power in some case. Under certain symmetry

assumptions for the aircraft body, the state dynamics of the rigid-body aircraft can

be described as follows


u = (wq yr) g sin 0 + Fe/m

i = (ur wp) + g cos 8 sin + F,/m

wi = (vp + Uq) + g cos 8 cos + Fz/m

p = [ (ly, Izz) 97 + laz (qr pq) + L]/1zz

q (z-4-p+z@_2+ ]I

r = [ (law 199) pq + laz (pq gr) + N]/lzz,

S= p + q sin tan 8 + r cos tan 8


(5-9)


q cos r sin 8


q sin sec 0 + r cos sec 0


500~ ~ 100 150 2000 25030 30 005050




500 1000 1500 2000 2500 3000 3500 4000 4500 5000




(b)


0 500~ 100 150 200 250 300



0 500 1000 1500 2000 2500 3000










where u, v, and w are the speed components of the aircraft along its body axes .r, y,

and x respectively. p, q, and r are the angular speeds along those axes. And 4, 8, and

( are the Euler angles that define the rotation matrix between the body coordinate

frame and the inertial coordinate frame. The gravity y is along the down direction of

the inertial coordinate. The throttle, engine power, and aerodynamic effects generate

the forces F,, F,, and F, as well as the moments L, M~, and NV. The m, I,:,, I,,, and

I,, are the aircraft mass and moment of inertia respectively which are determined by

the aircraft's geometry.

The LoFLYTE program is an active flight test program at the Air Force Flight

Test Center at Edwards air force base. The LoFLYTE aircraft is simulated using

a software, C++ version or Matlab version, by ACC and is assumed to be close to

the true plant. Without losing the generality, the throttle is assumed constant and

the state variables p, q, r, u, v, w are available for external measurement. The goal

of the system identification and control problem is to determine local linear models

front four inputs (aileron, elevator, rudder, and throttle) to six state variables and to

control them in order to track a desired trajectory of flight.

5.3.2 System Identification and Control experiment

The experiment here is designed to use the Matlah version of the ACC Afliht

simulator. The control signal is generated by manually flying with 3-dintensional

joystick to imitate a test flight. The system transfer function is assumed un-known

and state variables, output data, are obtained front the Matlah output.

In order to simplify the problem, the longitudinal motion of the system, axis

.r ( for ward) and x (down) and pitch rate q, are considered decoupled with lateral

motions, consisting the axis y (right), roll rate p, and yaw rate r. Instead of dealing




















o


-Desire



0 01


-0

0 01


5


0


I:
I
111 11 I
II


0 500 1000 1500 2000 0 500 1000 1500 2000


(c) (d)

Figure 5-23. System Identification performance on the LoFLYTE system.
(a): desired p and GMM-LLM prediction (top), error (bottom); (b): desired r and
GMM-LLM prediction (top), error (bottom);(c): ESN on p (top), error (bottom);
(d): ESN on r (top), error (bottom).


with six variable at the same time, the whole system is broken up into two parts.

And the problem is simplified to modeling the LoFLYTE's p and r dynamics using


input-output data which includes aileron, rudder, p, and r. Figure 5-22 shows the

dynamics of the LoFLYTE using the corresponding control signals. And local linear

format of system transfer function can be expressed as


SA xn-i + B un-i
i=0 i=0

Xn+1


(5-10)


xn+1


Yn+1


II I I II1111111 1 I I:F 111111 111
I'Y II II II
,I I 11111 II' 111

I~ i
boo 3200 3400 3600 3800 4000 4200 4400 4600 4800 500





:llr ~ l~(r ~~e -~/1~