<%BANNER%>

Continuous Time Correlation Analysis Techniques For Spike Trains

xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101112_AAAAEG INGEST_TIME 2010-11-12T20:24:27Z PACKAGE UFE0020302_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 15233 DFID F20101112_AACOZS ORIGIN DEPOSITOR PATH park_i_Page_59.QC.jpg GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
d88d017423f78eb1137a9a38c4fc66f3
SHA-1
c268468ddc628e041b639b8605a0c64820f1c57a
2029 F20101112_AACOUV park_i_Page_03.pro
dc9f41e4ef2b1ce1676d185e08f65694
622cb8757249a77db7999254fc85d299f1f7b838
39219 F20101112_AACONA park_i_Page_67.jpg
ee6eadd3051d64d2240bf5baad672f8a
47c3b2346168db4e23c837fc1fda3b635b073638
12619 F20101112_AACOPY park_i_Page_65.pro
16582e95030b07df184aecd0e662cb26
75e99a6cf8f882e6426bb6dfe3a55468f46ec1de
2064 F20101112_AACOIC park_i_Page_57.txt
094accb447c51ba2413c24deb4ebc098
21bdc1fefc084ee4553792f9acfd26eaad28af0f
4261 F20101112_AACOZT park_i_Page_59thm.jpg
574420cad152d3426342a566bbe72465
4c56b4f1a72c986eb9e9e1e998667d76c8dcf1df
40410 F20101112_AACOUW park_i_Page_08.pro
5ae22e972e3f797e84a011ee284a6c29
8c77f9765dd2aa0abc2d411ccf2cb4860ce850ab
4120 F20101112_AACONB park_i_Page_09thm.jpg
23b837e167e9de1fc368c1c15a8f6039
500a18ff959560ab301c195ba3526dbcd04dd2fa
2854 F20101112_AACOPZ park_i_Page_63thm.jpg
ac15fd5ac53fba031386a5c623158b72
1a9577c769afd1e96a61a776b9a817d450017d2a
71840 F20101112_AACOID park_i_Page_57.jpg
c08af788353ca0d0d83424594b3ebe6b
682b6ee80bab4c529337c611267f29dbeb67d4af
4645 F20101112_AACOZU park_i_Page_60thm.jpg
ba89260e3e7e2b694613e32b6e46b212
3aa8e4efddba63bce4c86e618029c2174368fb1e
31862 F20101112_AACOUX park_i_Page_09.pro
9aadcedf6a3b80309e498fe43154b7ff
69436a4ebf34c12f009c5ab0e2c8324a726816b1
25271604 F20101112_AACOIE park_i_Page_57.tif
1a03fb3e5fda02a888681173709a936d
b0ce686b9168bc28b11b2d3e014da615a5b8f890
1051929 F20101112_AACONC park_i_Page_10.jp2
3c6ef893fa8c912aead7e638ab9dee30
08884ce85a5129ca876cb63d745a9a00bc89bfe7
14814 F20101112_AACOZV park_i_Page_62.QC.jpg
416f467cb24a4c4e7361b544ce827e9b
1013a28412409a9804b65a49cf1e41305e262c65
57397 F20101112_AACOUY park_i_Page_12.pro
910496e23dc72f7cf2f77d682456762e
998fcb3392bb1bdcfd0dd9b175e6efd1c03def71
5446 F20101112_AACOIF park_i_Page_54thm.jpg
e8469daa13bc7a3ad109e18d465ab9a7
1f81edfe14d87bd0f929b1f57bc2796f70ff1341
1051963 F20101112_AACOSA park_i_Page_05.jp2
7a2b383df4409e6b5f09186bba281c17
7ead869b588b12e026ae9329f05c0f7087ebacc5
3996 F20101112_AACOND park_i_Page_07.QC.jpg
f20e6ebae0e26aede26eabbe5acdb0cc
a73423b3f9ffc7bd5179ae03ecfbd8925f2f4415
4213 F20101112_AACOZW park_i_Page_62thm.jpg
6887e91e247788c4aecd13245fc2e19a
0b16d92d09572778f58a527c30fc7cd8ba5c9bff
42012 F20101112_AACOUZ park_i_Page_13.pro
fb88eed3a649c6166534350b6d51f97a
7da41bec2ecf03c0cf480c9c46f1959df861d346
1051949 F20101112_AACOIG park_i_Page_23.jp2
b2b296b0378b0f0b1ca3caf01dfc09df
bdc72be753161a5a82968ef7a5c8c8833495dc7d
1051954 F20101112_AACOSB park_i_Page_06.jp2
1905a0fe4ebe29172a058f53817c3277
203f0b3a1edba59cf0db28f309d35010a30cc7b3
F20101112_AACONE park_i_Page_30.tif
bb00711a895a92a30327ce7a58d09770
ff8f60bbe2ce7ec214267e3b862b2752a4481571
8576 F20101112_AACOZX park_i_Page_63.QC.jpg
1f79d525d07608ba8357c95986266875
7e4ea70d66a2b219c2077f1b21dac6b1ebc1cae4
1928 F20101112_AACOIH park_i_Page_18.txt
7c1726ba74e6b97e73a60105dedb392a
161e3e1b6706eae26ae9d3cbe9e83e7788095584
1051975 F20101112_AACOSC park_i_Page_08.jp2
27705a8bcf800864c2ba1ac2c2f5a06f
2b870f1ff570dbe33e2be2c26238cb412efde3fc
14035 F20101112_AACONF park_i_Page_70.QC.jpg
61f58b9b5b8b9145af504c0bb08761a0
7c3469efbcb67bd15e88a06cb33340002d28b73e
12819 F20101112_AACOZY park_i_Page_64.QC.jpg
32d133b620e661e7bf98cf0303dd36a7
0bab9b19df1013d663d34843811da184d747f663
679 F20101112_AACOXA park_i_Page_48.txt
103ca8e43b6f3c529e246186ea88b8b8
647808457cdcf7eb6427849f409ab1d444675fbb
31474 F20101112_AACOII park_i_Page_65.jp2
a99d2ad331d47ae37eb6e06ca597a7f0
55934a1b0557fcf43af0e9661f8e9db720f4f16f
1051981 F20101112_AACOSD park_i_Page_18.jp2
f26d26853f2f11a1fa4649686d8c24ee
28128d5c6cd1ada1d3d1fc71155120e033eced77
4032 F20101112_AACOZZ park_i_Page_64thm.jpg
14e64591a721dc9392520a9bc61af088
1f8d1a0cd1064921650feff9978182c0861a166e
2052 F20101112_AACOXB park_i_Page_49.txt
6c2004e67a00afd0bce2f9b448861363
632c282246b53519f124bedaf60138080e4eaafb
483877 F20101112_AACOIJ park_i_Page_67.jp2
706946cb727c3de05892fb36e09e1203
774e84bd6d4aee8bd78642451218a208de6470e1
1051970 F20101112_AACOSE park_i_Page_19.jp2
059269fdd18012cd7f2b93eaa288c367
d96ea792453d5015424caaaeb40a0d46caf23667
17204 F20101112_AACONG park_i_Page_80.QC.jpg
50bc092487bb88389fdc66372ecb00c6
820897100e125c19847664bfb215fd0791658c5b
1856 F20101112_AACOXC park_i_Page_51.txt
4b1e328d4394edd4080fe56d32f8cc99
135f237677154b69d5c7946c059fb244eb74d9f7
2258 F20101112_AACOIK park_i_Page_47.txt
7838ee29f1ec1c74a106f950c3d31c5b
a1f4f9b09773658663e563c2928d39e6fad4df31
895165 F20101112_AACOSF park_i_Page_22.jp2
dbde4b1ef020ac8a0571f7c33799cbba
34563b34cc30eee8008e2fe9ea16ae1f29b1d080
26690 F20101112_AACONH park_i_Page_12.QC.jpg
38c04c3e9636d2e12f61aa34348ec26a
e6110f7a370a1c8c1882a3b81967a6cece533da8
1762 F20101112_AACOXD park_i_Page_52.txt
475bb76f5484eb0ae39f16ca7c8ae579
193237fa59609915c63f03c8b3bcbbd5a01d76e5
2411 F20101112_AACOIL park_i_Page_77.txt
0a4c08f7bd4ce0750c34ba64bc203d95
61c65e7cebddbffaa39b9b99ade1c5e81b16bfbd
103031 F20101112_AACOSG park_i_Page_24.jp2
33ead42dfb209122290b4d12d12ae0a4
4413698a22de05e9f2999b8d7711b4725375b2cc
1051960 F20101112_AACONI park_i_Page_42.jp2
63ed99ed856c01b2b9ba4ad5a3a79444
cf0f294860fac86cd503ebfaad8328e0bfa392b2
1730 F20101112_AACOXE park_i_Page_53.txt
2f8774ffeab0de90fed18b570ea38ab1
601f6fb113b96a4ac0393d09f352bb0fceef3d35
F20101112_AACOIM park_i_Page_17.tif
6802ab51593210f63eb35940316c0d11
5affe68759f9a99858e667494674408d6620b139
641105 F20101112_AACOSH park_i_Page_29.jp2
37f11a5d04b8ca1e3f612e48f3d4a634
78e91890219d46d939821e3f9f49aafcb632ae81
F20101112_AACONJ park_i_Page_05.tif
657b1a3192f800168736bf36e22ebeef
4ad99a13fa4930b02f6fe18b3303d9bfa7e2ad20
1798 F20101112_AACOXF park_i_Page_54.txt
5d6d725b440df2b345f08ffc52e3ff60
d0ade316b9a2fc770195e4a92555f159c7a85b3c
39460 F20101112_AACOIN park_i_Page_22.pro
2b5392cdeb3fd1c7af4056f44eb6ec76
7888ada290856d4f5b8892b191c2e480b379c0ad
1031026 F20101112_AACOSI park_i_Page_34.jp2
b4efb6169e519f16d4b3ec3bae5d291d
871cfdb075451351eea74ed5d4f9a2dbf1fd36d9
1051923 F20101112_AACONK park_i_Page_32.jp2
dc47b2fed8a3dc75ce486adcc4b1ce77
5554224d2519d0fb4a2fcdbcbc33d02fac5550d0
1882 F20101112_AACOXG park_i_Page_55.txt
0ad03ce0a13395baf2fcf6ad81cedecf
261132eecc5bb08222bb482fa4f909462ff3f45e
1489 F20101112_AACOIO park_i_Page_58.txt
6c9508da579d208a23a4e4b9de65ab1f
c6bae5e913b81b35afb300cd3653956e9efdf785
911348 F20101112_AACOSJ park_i_Page_39.jp2
7feb0f8bfa84634a76b58591175fcc00
625f35ebec99487476f88ecad74bf59c33a77ec9
149 F20101112_AACONL park_i_Page_03.txt
a2c91ee14b1bec4ef61c24009b41cfb5
ccd583ff119be1cd10a6387b8067915e9960f814
1584 F20101112_AACOXH park_i_Page_60.txt
7784f248b1ea5c77c599146d2e200e7c
24b7d6aadc18a25f6e7688f553a6d4112f8609ee
74896 F20101112_AACOIP park_i_Page_17.jpg
68319780c4f3878014d88e0b443aec09
57c9dbc93c5876fe56968bf1035a2cb6f0022e92
80034 F20101112_AACOSK park_i_Page_41.jp2
7aba65235ae7738015818f03c22d726f
195cd5fe46a89267a4ad6fcf1db8e137ea075eb1
8423998 F20101112_AACONM park_i_Page_26.tif
4a1cd7efb920be433958c00520bcbdb2
1b847d3dabf3a7a73f1e0978f8afd98a0c079df2
559 F20101112_AACOXI park_i_Page_61.txt
0c2844b4df1e72e08e490400d299ebca
78b303b8a343f32f7328577bdb79f4971dac0fb9
1053954 F20101112_AACOIQ park_i_Page_78.tif
2fd223a637fafab9501d543b53721094
da754f03a5205ba1b4cb5354a914b15fc5d8ae3e
1965 F20101112_AACONN park_i_Page_61thm.jpg
6e06ac77d2aedd68611486b510fd74d9
97670f1855d7de9b1fc0a17dc356064960f25e7e
1441 F20101112_AACOXJ park_i_Page_66.txt
f366b8166c35d6178c362984c9cf7844
c8330ca87aac8be430f5912ffdf938646d301013
56603 F20101112_AACOIR park_i_Page_58.jpg
b81ef6f267d69e7853ea0d2748d3a7fd
fc463fe68f2fc4b0a513d63dff5248f1c026dbc4
1024891 F20101112_AACOSL park_i_Page_43.jp2
0d413304851d2b21f56e188827370db3
9528464c3b79fac16fb63358c5eee52fb17b1939
10045 F20101112_AACONO park_i_Page_48.QC.jpg
0faf8e138005184ca86175802f385b36
a6e0551014ff903faa98c71d46939f6e4334d412
914 F20101112_AACOXK park_i_Page_67.txt
0e77afd2b3f9e6186e68deff67e6ed3e
6ca1027aaf109990ad31656a7ab55059d98fedcc
F20101112_AACOIS park_i_Page_77.tif
d8b45fbf64bf0b6c7c320bb636eeb6d8
16e46246d2887610349111f377d57c3beaa1a873
41020 F20101112_AACOSM park_i_Page_44.jp2
1bb00d250db2788aa34cfeb387d81025
b2b0f4141fbad8dbfbfee31a8a1d8bde15af4d0c
1750 F20101112_AACONP park_i_Page_38.txt
3587a31f4e061cd576868ccda6f78290
94273e6bcfe69f7100de01adf62e4f86b874c6f4
178 F20101112_AACOXL park_i_Page_68.txt
f8704f18f951b90ad1b6492ffa1ad69e
8a2543a4d16b8391b44a7f232ca187e954e55020
2973 F20101112_AACOIT park_i_Page_70.txt
62398d0d4bea9111758e84987749b6ce
4573d8d8603da762909eee2750a6de558b60e7a5
76147 F20101112_AACOSN park_i_Page_46.jp2
995c722b65be45ed630210dac508cd89
a7a12aa4e8142f47cbef28be28b3d5d27b65d165
42477 F20101112_AACONQ park_i_Page_43.pro
d3364d2fe159bceb35f0121ecd269b4b
66178dd5db86ba774e53c2d7faf7fd6c180a933a
2371 F20101112_AACOXM park_i_Page_69.txt
4d5ed7b25cf5a21ebcbe684c225f38dd
0562d2642f364359f5be5d063194926e2dd46e79
26481 F20101112_AACOIU park_i_Page_42.QC.jpg
d05a1125335f55666a076c9ab9dab9c7
bfaa3cd1600f096489ae573a475b7cb79b0022e9
896706 F20101112_AACOSO park_i_Page_50.jp2
4abd19ab2e148501ce0954286c9782e2
a9e3179fffa73e788b550293d670febffa0af3c1
24787 F20101112_AACONR park_i_Page_23.QC.jpg
4dba4c23f2145b67f80daaa5597c0620
df3cf833359d6db9f4734a6d156c04ad9d7ebf6c
2966 F20101112_AACOXN park_i_Page_71.txt
80eaab34fbc7ea9dca96c28b8bb7ebd1
c21df540421f6f6c17324a3f3ce599ed9c0622be
F20101112_AACOIV park_i_Page_72.tif
3dd8a82c7ef437b68912c0eea9914305
2774c74fce8a870919160ecc6820b0dd8dce62e0
795574 F20101112_AACOSP park_i_Page_52.jp2
d3cb04721c9d7aace84db457415e441b
6da0c1d97c51113d04348f2052dbd7b17fa7ecb8
40502 F20101112_AACONS park_i_Page_56.pro
5028ca631d19061d4717faa846c9a21e
110dd927b7065f37e17de3d9e1ce135189cf2375
2445 F20101112_AACOXO park_i_Page_73.txt
3ac8d95a55ad3ad83aa75329096792c5
ce30878a5a642d5a3da18b080bfa53e85392601d
16155 F20101112_AACOIW park_i_Page_60.QC.jpg
044bff3a83ed98daace1fb4458c8cd77
55be2b2064562adc80ea2d3519e17b685d8d9679
687521 F20101112_AACOSQ park_i_Page_53.jp2
8347ede6c75e78511f6bc10c2481df70
e982b2edf10cc214de0be38a7a41a31cd827445d
1314 F20101112_AACONT park_i_Page_02thm.jpg
61274a324f20d840e526ae632e6bf49f
b8478382f5134413d6ed26713ed836e310d6dd22
2631 F20101112_AACOXP park_i_Page_76.txt
619b6f19f1b58915ce7c76b534afc4a4
5a4949fbcf8d3d01c68acc187651a0a8169f72b2
58914 F20101112_AACOIX park_i_Page_16.jpg
54ae4cecf4a8727ddf9df6b4e5ec0178
965f2596acbb5b707a07ee4d4083bf22a59483ba
874476 F20101112_AACOSR park_i_Page_55.jp2
eacae9948672e593b53a14ea52ad2e12
7470c533a307fe3f38247a62f5bc7b1322a0c698
793 F20101112_AACONU park_i_Page_04.txt
9608174d3a4637b8c7cc483d8af44613
7a75254c2e91b4a1fe92811e70619228e4963de0
F20101112_AACOIY park_i_Page_48.tif
dc085b1a8d97ad2273cdf58d452cb0fc
2040ce2db75e01299f106959458b386f28f65c62
902650 F20101112_AACOSS park_i_Page_56.jp2
ffefc5875bd363dd7042959c036537a7
6ae232115d0174bfc30a279120872c68dfc62c75
F20101112_AACONV park_i_Page_81.tif
4c8f4bb5af20e7fb38a2e09d01c0952b
90d1d1fd60109f8e2866183366c86857de38a320
1657 F20101112_AACOXQ park_i_Page_80.txt
8f21c33f35eba91aef0e8c995a45b1af
881271eba9f595e12463a4e6fde65d878fe97281
557618 F20101112_AACOIZ park_i_Page_33.jp2
687b98d7a6efbbc629293c961e5c85d2
1d31c160d0f501c1a154eacecf67e12e7885121b
988981 F20101112_AACOST park_i_Page_57.jp2
70dadcc316dcd2905b1a79259c59d801
1dda9ccba64efb14bbd99cb05279218986b4c149
5612 F20101112_AACONW park_i_Page_24thm.jpg
e2099bd1f433a9ab7f82020500e73a85
1c54683bbe266d331f0519815ad144cd54d1671f
20976 F20101112_AACOGA park_i_Page_56.QC.jpg
941f65426e5451fc808506ee989c0019
f26c420d2358780f75b141a36386916b713f41e3
571 F20101112_AACOXR park_i_Page_81.txt
923b5ae1171d553e398ae5a09f2e7166
677602aef930d69e404f61bfc250924e64e8e53c
75587 F20101112_AACOSU park_i_Page_59.jp2
7c73fc9c304ba22610c17b5be9d94655
800d8abbf1bfd0c26a958d2a4d63760e38690ba3
925134 F20101112_AACONX park_i_Page_13.jp2
a70f7d8207f2f4aafa0dfdf30d2ab137
dcc65ab65c3b96dbd32bf3d22d10cfac744d8caf
F20101112_AACOGB park_i_Page_41.tif
cb7528cc489c5b6e0d12f1b85248baa3
1989ee02af465044cb8755efe817e9306a5768e9
6701 F20101112_AACOXS park_i_Page_01.QC.jpg
a87f3b1194fa63bbb9551a8201eaf42b
68b04a6f23b4e1898e129e078173a4c6c5b7dc98
709645 F20101112_AACOSV park_i_Page_60.jp2
b70b1c5be3b1c587f1f0fd4a532b7064
38aadd4ba521bb9cf25328fc339be2106c55c9b0
F20101112_AACOLA park_i_Page_15.tif
19f7e26f317dad7be4f6df27a9026025
6378059b3d7503c9ee4082e797055092f4151e3a
5586 F20101112_AACONY park_i_Page_46thm.jpg
328304a4e4630c221109ec2671bc2f16
94cec458afab3ff2d5901ae1a0bdb52595e08663
1051941 F20101112_AACOGC park_i_Page_47.jp2
87de199b4b37b8df9df41122d5a653d0
f03aaba0726039cd42892d186a36a172e224e590
3056 F20101112_AACOXT park_i_Page_02.QC.jpg
12469a281d38ea9f25b48db8c36c2262
791db1492d40648837e241649dc7b5818774e8a4
17623 F20101112_AACOSW park_i_Page_61.jp2
ddf0ee5771667dbb17ca1bb3c8bceeae
13dd256148a58701ba6d6c8297df8c8c2e036431
65458 F20101112_AACOLB park_i_Page_24.jpg
1831d0449010505456e091515f1b36d3
e5ac877bdb2ae6b0c5bfdb4ffc21227de3474422
44480 F20101112_AACONZ park_i_Page_71.jpg
7b80871a3f08e8e05209bc557e5a8917
ef5c9234591d9b4d56adcad58a362358762aff12
31476 F20101112_AACOGD park_i_Page_16.pro
03512797f803bd80c39c7fe50ecf3c70
9c8cd721a8ae41ca88557296bb67b1cd6ab2719a
4493 F20101112_AACOXU park_i_Page_05thm.jpg
1e8de631a1978b1a946b4d35b99996d4
f85762ae76438fa60a6f30b60f64390f032e87af
64831 F20101112_AACOSX park_i_Page_66.jp2
4d8fd407a726f2973905771e5cdc88e3
fb9a254c585285874c2c316889adb852cb49dbce
F20101112_AACOLC park_i_Page_42.tif
06852301e670fb59a264a25fe4e68579
724b2c441a330f54d36aa1d7f0a92294a6186c11
4954 F20101112_AACOGE park_i_Page_16thm.jpg
36214eed874c59833f8c53fd6f716673
f28b965bbe336c2491b473945409d28d2b41101e
17431 F20101112_AACOXV park_i_Page_06.QC.jpg
8fc6f134cc0f32dce5e8442b56b3f540
35fe134afdd5459bfe57349ed21242de75b7fb40
71782 F20101112_AACOSY park_i_Page_71.jp2
4ef93d3a7c5120d423881c6a452ee91c
23ea9c01b0403ae73693d4dcfb14cbf4b80bf8fb
59886 F20101112_AACOLD park_i_Page_50.jpg
d84a599a78dac86387a5d4d63fa7e158
edd7db05893e95383d011226a73b69d4faae6cfb
34734 F20101112_AACOGF park_i_Page_58.pro
537a20492137d088bb9745ac22db904d
7889576fa20b42eebadddb3f07065e2681dc2ee2
F20101112_AACOQA park_i_Page_74.tif
a7c2b380ab842892f7ecefc3f7410f32
8f1c4f45955504319d68f111a9e49ce83b19cdf1
17891 F20101112_AACOXW park_i_Page_08.QC.jpg
3e5d502c364da560a1abfe19d159e79d
ad647f99152db3f29d5c5678d2113df54e307b21
64343 F20101112_AACOSZ park_i_Page_73.jp2
b470daf6f330068ff4341998ec0cc77d
d13a86dadbd8f06d01f7b144496c35e62b5e177a
6739 F20101112_AACOGG park_i_Page_10thm.jpg
63ba773e15e0edf3abb08df62afea7b0
fface043ce07de89bf1812bf2872c4b0e739d993
1834 F20101112_AACOQB park_i_Page_35.txt
700485b2816967c6671923cb89f0483d
da69f0147e585ea4fbb3aa86632ca3dab02d6485
14881 F20101112_AACOXX park_i_Page_09.QC.jpg
e3b267b1e76ad87729880ea93b309443
8799d4b0c820fb91999ac0a3c48f32795a1f80da
6596 F20101112_AACOLE park_i_Page_42thm.jpg
345a43bb29f2fa2062d0fa3ca950b45b
22629f2fe39df86d4c33c3311ad7207605b6fdc9
90951 F20101112_AACOGH park_i_Page_25.jpg
eeed8f7f9b7ea889c6cfd6705d9e2253
ef758c322622a3cf86e5191d3da3ffb8f9390513
18838 F20101112_AACOQC park_i_Page_54.QC.jpg
eca3edb10384a7f9a119753ffe3fa933
dca57afd378d4dd5bdfbfdfec0a2f422de71fd4b
6618 F20101112_AACOXY park_i_Page_11thm.jpg
9f195d39b8e8081dbbb74dff9b665812
2839ac349ca2acbeba13ae09b4273af34f771df4
40752 F20101112_AACOVA park_i_Page_14.pro
0ff24b0d02b6b79a6bf89202795fe111
c0a37ebf215a262877f097e1c8988bdb2776c954
2611 F20101112_AACOLF park_i_Page_79.txt
6cfeb7f8b4f20263b7325ead45f90233
5376730fd8064e888889c446a601de1175cd7842
59068 F20101112_AACOGI park_i_Page_40.pro
9273fc39c40bc08e067abf10972a96b0
ec18070b60aebcfced714449301e10fb09f67ac9
5595 F20101112_AACOQD park_i_Page_56thm.jpg
cd04bb8b0d9632369a81d19cee283f22
e7a4b7939a2eb878ad65edb5e691b5edab2cabdd
5583 F20101112_AACOXZ park_i_Page_14thm.jpg
e59be8053d5d8ee2292ca96029236da4
6c6ac7f3be5e36b32b29ae88220c4a305803021f
46523 F20101112_AACOVB park_i_Page_18.pro
ebd84f15682086a9ab3156bd49d24159
2f1ccf0b5b25d2ea91e336a74610e97fb58db6fe
3639 F20101112_AACOLG park_i_Page_29thm.jpg
d4550619fb07c0d8c598087d84b4ad14
574f12945b9436b2c87f66e7892608cb6456cff3
26919 F20101112_AACOGJ park_i_Page_27.QC.jpg
fd41232c01ecf050259cfbc6d1da434f
82f61d0da61d1d8bbfb7b30755f20a48015b1459
121353 F20101112_AACOQE UFE0020302_00001.xml FULL
3a6cf40609cd1150d1e8b0924ea38818
ce08900e391c50a52f9e24f8b9750b626ee289c9
52793 F20101112_AACOVC park_i_Page_23.pro
30e419f32ac8477f3c6632c71e08ab7a
bc337e6babe6f104cd7be121abdd40f2ace2e9f9
32718 F20101112_AACOLH park_i_Page_44.jpg
21bc6863112f4582e7a9081213fbbfcd
6f0a656c619d185501e4b68c6488b0f7a42b1902
6291 F20101112_AACOGK park_i_Page_18thm.jpg
e7666ae3b27aa615610b78ca9a17c100
2d1695e27111ab4d2122adb955e1fbedd4982c6c
47054 F20101112_AACOVD park_i_Page_24.pro
5c85fdb8e5ad53faadecda695263602d
d6d968afa590d9f4951d799614c5fea7dd0d2e72
3488 F20101112_AACOLI park_i_Page_71thm.jpg
e67fae33bc3ec3659b566e5e1bbdfcf8
7a98ec4a9a493ce2baae081c5bf8c06dbf0d8f5a
3463 F20101112_AACOGL park_i_Page_03.QC.jpg
8eea497955c9483a7b92821060818c74
b67b51010ab2765c484a121fff69cf5244c6e67d
56422 F20101112_AACOVE park_i_Page_27.pro
b7b4f0c8878a3f06f0dbdf8013b83527
96fdc0d81935f9ff6090d6875faed369ca9ac828
F20101112_AACOLJ park_i_Page_27.tif
4cb626f18ef529bcc2e388d24d1ef9cf
64c7a110dce5e0e99fd931df7f26181d0b3bab2d
23090 F20101112_AACOGM park_i_Page_17.QC.jpg
fde03ba506388574b17c529c2420b9ff
91923344937d25e1bd6b533433dab6e8e7aa75fb
21979 F20101112_AACOQH park_i_Page_01.jpg
0856c3430b9da894c68d25ab9b29e9a2
cab5e2beb81bab3f1432fa7a19b31788885ca82e
8224 F20101112_AACOVF park_i_Page_29.pro
adae994d5938f96717bab52adcb4e761
92e87e896b30017b7a031180e28142eeea67dc20
1051944 F20101112_AACOLK park_i_Page_25.jp2
6271fa99fcc949505567aa43dd2ad1fc
f27f3d84b03364f7313e3e73508d3f977a6d33be
1474 F20101112_AACOGN park_i_Page_03thm.jpg
6579efc184baec86d2e4627f1adde93a
fdcac91740ac3fb8124413c06b7ff3adbd457b13
31631 F20101112_AACOQI park_i_Page_04.jpg
ae135ceaae1ee6779ceb3a904fb80171
e9e6e85684d25cf20fed192cf799c389da21debb
57454 F20101112_AACOVG park_i_Page_30.pro
76f45e626a0ed2512059e93344af2a70
cf866d08d21ab9c7c5d417333965b0f12f70d0a9
1051979 F20101112_AACOLL park_i_Page_17.jp2
3d496bddd0a8f3a15e880d3bba8af509
730dd7ecfa8ee995366eef24933a93dfb808600e
2374 F20101112_AACOGO park_i_Page_24.txt
6d77043e822e66b02fa9891ba147b934
8b44aa6b1bbf7184be53e5a3b61a6aff69b0bb34
4146 F20101112_AACOVH park_i_Page_31.pro
d119a6a1f73245c2d455187fc82d4a48
ce1096dfedf67fa3e1ddbb94b806360155962da4
1380 F20101112_AACOLM park_i_Page_59.txt
c7b3c856d47fba4c728a3b33ad2d8f4c
0f3a84db6d0e78bdbabd89eb33d51e618d33b4d4
4526 F20101112_AACOGP park_i_Page_66thm.jpg
3673d72b67da896c53ed83b4cbf32d7f
9f28f38259a476153272f2b2278206ef82605e4e
65861 F20101112_AACOQJ park_i_Page_05.jpg
2030450fd1283d39229c2185d7504391
2b9b58e655d6a935b00da34558dd809f654e97a2
21822 F20101112_AACOVI park_i_Page_33.pro
7cc0cba330d5b47f02349534f54c40b7
54a679492878825075ff8194c21fc40a8991bde2
2307 F20101112_AACOLN park_i_Page_17.txt
a54fe28193c687e3f85cee8f9e263d73
fa68a712f8fd8e9c559a5890d389858187a0311e
2592 F20101112_AACOGQ park_i_Page_65thm.jpg
780d5557d960c1a307779c1d925d1185
d9ed020e205e5cc61356345735b50a1292400329
60921 F20101112_AACOQK park_i_Page_06.jpg
897787fef82db76d5a9646086bfff738
cdf0830e3851b431b271522e27a490084ca79902
47674 F20101112_AACOVJ park_i_Page_34.pro
d3c59a3aca43c343cc5dfb9347e94eb7
222e0c99bbdea6f0892c8f22f7672e778a8d074e
31168 F20101112_AACOLO park_i_Page_41.pro
251335bff96653dcb0cb61abfb2dd85f
0e5330f5c0339afb432955b4950797598329ead9
2114 F20101112_AACOGR park_i_Page_26.txt
4f490f05f7d7bd1853a09289b01e0695
fb7513bf8b02cd16945fdd8cfa716a1e09e7c1c6
12472 F20101112_AACOQL park_i_Page_07.jpg
4ba331bd6ebbef983f29596bb38f88a3
a54cf9ca4b8c4f6836902fa6e44f2c9df6aa370f
34781 F20101112_AACOVK park_i_Page_35.pro
5ca6decc4bfdb7364edd028077afffe8
2a64b2ffa3a12b0442be68f6269c29485b7513b7
114020 F20101112_AACOLP park_i_Page_75.jp2
d5b595c99bd28f2c62ebff26dd93b1b4
ef404a6d888b6e2e862b8a329b54b25d342eec0b
59903 F20101112_AACOGS park_i_Page_77.pro
aab79cc9f2196e499158236daa61b6a2
fdfccdba732bffdea713fa22e991a76e00e20d91
60840 F20101112_AACOQM park_i_Page_08.jpg
6b454a1b2bb59841414e95c424dfae43
105824b12a13ca3cdbaba5a4eac1990b70a5ce5c
34661 F20101112_AACOVL park_i_Page_36.pro
e256bb1667f50a9d569642f519f09f77
662c5551ab47e8aec1d6cb660d932ea36f0b1346
518672 F20101112_AACOLQ park_i_Page_64.jp2
d1b22cb7c4b8c067bf1f00a571bed074
b5b14d3734a35ffaa936ebd99ae19e2b09df6dd5
1975 F20101112_AACOGT park_i_Page_56.txt
10e85f69137eb198fc72287ffe8ec80e
151df7ae0b4a94fdca1f528692e372201dfe751a
46930 F20101112_AACOQN park_i_Page_09.jpg
f3e6051bbf4048384f7ac2c7edd95fbe
97a37da26e70a81566d5744e1d2096286549daf3
32561 F20101112_AACOVM park_i_Page_37.pro
8fd1e5f051134b56a284766eab529b71
00a074a87e430745a61785042cd52d7778c0351f
51830 F20101112_AACOLR park_i_Page_37.jpg
e7ca56a4d0d95b4e56b2c1d051e7bcca
e829850261823fcf2a3bde3e195ae7d815de2844
6484 F20101112_AACOGU park_i_Page_79thm.jpg
552bb4708c1acc19bcc1d21e7c35963a
2b599a750dd62e459f0cfd56ce892dd8c2afca62
84911 F20101112_AACOQO park_i_Page_10.jpg
381aa70f6dd7449e4f003987cedf98fe
d4d60057c5963634fce258eb8d0873fc2d253485
8426 F20101112_AACOVN park_i_Page_44.pro
094fcda9a6e238fa26dafe70c0b4bf49
99d020b5ad2a1d455438866e0c76fdb6aa79c0b2
76612 F20101112_AACOLS park_i_Page_77.jpg
b6ea85cc27b6f0871acf471cfc00df52
e31fc1ccda24edbd57d1d26519693de888b9459f
2436 F20101112_AACOGV park_i_Page_78.txt
4ab88f7dcda886547360472d5a0b5296
85c50f8076120c32bd9c97b239858956ffd9264d
86545 F20101112_AACOQP park_i_Page_11.jpg
f06377afcc4ca38143dd1cf3051f0128
e8cf8450a683b28f0f5947d33c0076e32654c6e5
2345 F20101112_AACOLT park_i_Page_32.txt
a2046e0ee16ed10a0a0f395200138c41
bea6955b1cda6f1e6f4d9c23ade2b9b3bfaed9ba
6024 F20101112_AACOGW park_i_Page_74.QC.jpg
96a70035c87bf683b53541d110c59841
764729e7738ccf3ae1a4b9e52ff8a70949e46a79
66808 F20101112_AACOQQ park_i_Page_13.jpg
88d2f889f7319ef4a23535301f58d068
47e2bcaf5efc93ab0b5290015eacb0d781f82a4f
51448 F20101112_AACOVO park_i_Page_45.pro
4baf64bba24173644d06c4987fb49325
09f76d55e08004ab04d66d593f0959fca8938a80
1491 F20101112_AACOLU park_i_Page_09.txt
9ed5fda17e401d91006b0422dde60287
ebd4d178dc94751a5b962dd51dcdbfb65335ed66
F20101112_AACOGX park_i_Page_55.tif
eb5e51544fcf37c37265429069e5ea0b
8ca273574d21ea1bb2f8f55e6b8b0299ae865c1e
78565 F20101112_AACOQR park_i_Page_19.jpg
5d10e8071786a7f1ad5d44d371a1ad0a
045e0ed92e57bca828d99c91d36759a095cdb8e7
18271 F20101112_AACOVP park_i_Page_46.pro
4719cca840d8e26d340dc4d381772ef5
b115d172bf9e2dee245084c084455c9ad51b4b25
890674 F20101112_AACOLV park_i_Page_54.jp2
44d41ff587ba571519c39953282f311e
da96387ed7237f21eac3c24a0d8b1608cd82c089
5673 F20101112_AACOGY park_i_Page_39thm.jpg
99d4de0d4d9ff54824b4f6cd26cbd5cb
3ce37dd63e2c30dc09dd9890313954598e34ea80
67748 F20101112_AACOQS park_i_Page_20.jpg
6d8d94ac5cdb17990a3e2efdcc6011ec
7ece9c5385cb6ea709c41fc5b1eecba84caf4865
48544 F20101112_AACOVQ park_i_Page_49.pro
63f8a74c2876f23f82038bbf79def024
004c20bb756c47741390a4cb1820efaf1aec7dc8
F20101112_AACOLW park_i_Page_79.tif
add0bab99230ba2a5281b3db92dc0474
e0c15365583c64bd41f3c5be78c59715d5f64094
933293 F20101112_AACOGZ park_i_Page_51.jp2
914d9b53b90159f7dabab52451c3d3d3
4ea1e99593c4f67d895d3bef65147f6c9bb5aacb
88886 F20101112_AACOQT park_i_Page_21.jpg
a0c62af95fce9d1244a8d5be94892654
917d7b5b92c731257fb0d590303a990143ca3a5f
39608 F20101112_AACOVR park_i_Page_54.pro
bf068e0cccfb0bc54972c6a3938d6bbd
507c6a868bc53ea03a119d68414262b97deb004e
1051982 F20101112_AACOLX park_i_Page_49.jp2
a495d621083054bfb5e541f64fc9ac18
1657bdf5029118a611785744cdb8bbc650dbed4e
63121 F20101112_AACOQU park_i_Page_22.jpg
a9da55a367989f848541c12907066018
b6b62781e31134498641d466c0150abad08b32f2
31904 F20101112_AACOVS park_i_Page_60.pro
ed6faf69f9d7d9a7d8e4e4ca7ed530dd
7faa0db6fb427e66856a62feb68d20bfb401f701
909592 F20101112_AACOLY park_i_Page_14.jp2
d3c050b9a467615c370049ce3272c934
dd1c9d22464737303f8d12a4c70afe4a1b099956
78017 F20101112_AACOQV park_i_Page_23.jpg
710dc1e1abd002c2c72d2bff6a7093ff
cbb627ceae86f907c8076098f702eb9fae63b025
6239 F20101112_AACOJA park_i_Page_76thm.jpg
df2b3f55412a755f1f6278785679c6a5
5b7dc1c1d18c9843bed29972edc7c794e9bb7a76
14358 F20101112_AACOVT park_i_Page_63.pro
23e5824fa7d924798a2fbd6c527acf76
3c8b937d2a744981ccfc926999974810153fd144
F20101112_AACOLZ park_i_Page_40.tif
8827bc543422fe38d64d930fdd6a5a66
37b84f8e1df32aa3cbcc820e340051fdde3d8eb9
58540 F20101112_AACOQW park_i_Page_26.jpg
2b6acdd148a0993cc0a91cb66a5069bf
cd2decae482cc10afe4d7aa058ae1dcc90f3b8cf
F20101112_AACOJB park_i_Page_25.tif
06e966cd276119e006f43bd9624287d7
bf90c01758a5d629ab61fcd69f0c12a1a91ccf68
25970 F20101112_AACOVU park_i_Page_64.pro
88bc6b37cd5823dfa2fd4f3ee1fadc0e
5de67443b8d8106202dda9623475b512e12752f6
87050 F20101112_AACOQX park_i_Page_27.jpg
3a653ff4081b2c66233b07b825ce5f24
d655bfed5d6978dc46d4379559dd832da5ca988f
4134 F20101112_AACOVV park_i_Page_68.pro
80f7ab45bd4e5688a2f68cd6e1645a19
4e41540133967b320f8b99f4a55cf0e2132c2f83
1663 F20101112_AACOOA park_i_Page_31thm.jpg
0fc2f9e4a5443e59501fa52b36ec056a
f10784ba8d4bacb416d94e17d8f6097db18c737d
36829 F20101112_AACOQY park_i_Page_29.jpg
076f24b39906426e418aada54b25ea97
92781069434e741c085857648ee9c324a2cd4a5a
30927 F20101112_AACOJC park_i_Page_66.pro
ba8adc8c0dd6e4b64179085ecb17250e
c8c6fef47247024ec3be952b179b7ba348b21df5
47283 F20101112_AACOVW park_i_Page_69.pro
4c5a6d4d38c5a83c1da6fc53339c5302
577e2f5756c653d8852940bbfb85ae9e0a877bbe
39853 F20101112_AACOOB park_i_Page_55.pro
9f5a3ccd8ec7e2878be2cdb14a76ef9c
59d9e9fb2cba94c28a162e5a4de78a17e0423728
86143 F20101112_AACOQZ park_i_Page_30.jpg
080d55415042fcaf7b1e6bdf014cb16e
b09f1d5be1def43fb9d6cc1232141d4dfc4bf45d
580 F20101112_AACOJD park_i_Page_65.txt
95f579bcf4c7e3fa0c74ff29e1ce4bb8
4ec71ddebf634accbdbe04c5ecf52b33c110d9e4
55948 F20101112_AACOVX park_i_Page_75.pro
947c6318785e904ac6414d5654eaa496
aab3b9cfcb5f7db4aef1e1b2283f9bd1e7e8159e
2264 F20101112_AACOOC park_i_Page_75.txt
538a2559a3db40ffd2afe0909b4576dd
d9025b55758679abca0a95644442c8c8ebad9e1a
1051977 F20101112_AACOJE park_i_Page_40.jp2
0bd3d8f5f4d6ce367a6402cecad8c222
ff33fb7086b1f32719057a9a25b05c17fe2c93ca
60579 F20101112_AACOVY park_i_Page_78.pro
cbdbc9793b1a1a759f8f425d15c70de8
2642999281674d638f4262e14ca456b84d8baf7b
6126 F20101112_AACOOD park_i_Page_77thm.jpg
1eca0f15517a0f4188b3fc1abc58fc82
1ab9f4516e424615858eeeaeb813bba5140b6641
18770 F20101112_AACOJF park_i_Page_58.QC.jpg
ddfd4565f7eabf5ca26170d3e626c004
fc54c307c3afa61e2264d3fabd4bf7356b7986cd
25950 F20101112_AACOTA park_i_Page_74.jp2
76ada445d5450d62aacb6a40fe813d9a
cb2cf99e10a5e0a4b02d8ff1c59ed3b00dfc2f5c
65345 F20101112_AACOVZ park_i_Page_79.pro
0b96cb052e3fda42fb409db9450f5147
8903c6fab770fe47e5aae51a2a871ff5e4f376f5
915730 F20101112_AACOOE park_i.pdf
c12d9de567933974f6c475586d6f6b66
ca53594808cac2dc78dbb9fa687c5e36ac7d3ace
16924 F20101112_AACOJG park_i_Page_48.pro
943695ea5af34e1f9713dba994f9b390
9d663c185d3664c090c5eb7f87de74be2935a10a
131705 F20101112_AACOTB park_i_Page_76.jp2
97062a8703c50b444a9c23bbb7712094
eff0f89982ef1249b95edd5f0c9c81e4e20cdcaa
20182 F20101112_AACOOF park_i_Page_39.QC.jpg
3bc921ee80965e71be87d12fa982c8df
b46ca3da20cfcf9a8d75206ab18b990020beed8c
55613 F20101112_AACOJH park_i_Page_42.pro
3613204aa532f4ac04b9fe09705b95b1
e011e605907ccb912ea933d39b9a4c191b1ffd76
125630 F20101112_AACOTC park_i_Page_78.jp2
c3998de83e179dda1ea0cb836ba6d882
86666cb5a4c94f2572cad648c9ac88f3fe649657
7785 F20101112_AACPAA park_i_Page_65.QC.jpg
1543bc18ae3dbcfa73b36c985091a4c8
5cb699b2cf79e8437e390c63f12a4a5f9ceba254
16561 F20101112_AACOYA park_i_Page_15.QC.jpg
76fca6b3b1e84af7d8fcdd3047fe08e2
f347461250fa176228968de11a20916a43507b04
71816 F20101112_AACOOG park_i_Page_43.jpg
e1519e54e6bb844ac4a319bfb15e9482
4618686938fad039e2462f8bf62867e24386fcd5
11008 F20101112_AACOJI park_i_Page_03.jpg
5789af8e74817df68367ae2a9049616d
cf31c98ffd718bb8e84e7198197a4250ce556501
32528 F20101112_AACOTD park_i_Page_81.jp2
120797ee9b845b0609b2a950d30fe83a
9557b63775ec495a4ea20a8b9e502630657fdbb5
12231 F20101112_AACPAB park_i_Page_67.QC.jpg
6bc9e894a03fc67fd793d0b83783a9b5
6379e9959fd2d51e72fe332156b19bf4b1868a2f
5092 F20101112_AACOYB park_i_Page_15thm.jpg
ee56d232d8cd1b1ccf5e149102813bbc
b99cb0a36ca20aa1e83352e0bd368f13fa395712
19096 F20101112_AACOJJ park_i_Page_22.QC.jpg
a2a59e01ea7ab6addd4677c554695b7d
ae8d17b9f8733a67ac21087919e0fc76a3fc2925
F20101112_AACOTE park_i_Page_02.tif
a9f580272bbc82d2683af0aa8825bdb1
426000d84fc915a695a021b4af1e976cc825002c
1680 F20101112_AACPAC park_i_Page_68thm.jpg
365ba0c3f1952a828ca17218c3d26c04
7bfd8a6235130962c96b8c25325ef36880cc12a6
18059 F20101112_AACOYC park_i_Page_16.QC.jpg
c2c831c834003c406aa646d6147e8889
1d80498461b674c9e77775daad6ef7942a090c3a
754493 F20101112_AACOOH park_i_Page_35.jp2
6c7bfe2b30e38a50f4ed503faf88e40c
977f3984e173ee752776d5c205a48c11476cc672
26696 F20101112_AACOJK park_i_Page_30.QC.jpg
cc7213788909e0e11c3402d3a4e024b4
0ce1d1ad3218b73d6d5e1fc572d925341f8b0bd4
F20101112_AACOTF park_i_Page_03.tif
c46e93e13da237ad4109a2ce62c8d889
dd21bc055155330add4a1c5a67e0fbfa25f5340e
12444 F20101112_AACPAD park_i_Page_69.QC.jpg
01b51a3755176c3c81983d10ab83f6ea
ab5b4b1770457eb4c94b08e9c5699b28523a162b
23308 F20101112_AACOYD park_i_Page_19.QC.jpg
92bb65a38944ece8c5fbe137537b1497
936c1e4c274e58d6805c7908dd3825ba72800c6d
20567 F20101112_AACOOI park_i_Page_13.QC.jpg
ceded23c34a1cbf127b5f42baa5cfcd5
20fbff98cafd3c2457e0f8d9eed2e6a3deda6b06
2325 F20101112_AACOJL park_i_Page_21.txt
c84db373aef928b100d490eda1b77335
fd030f715188a85c7ba32847bead5875a779ad70
F20101112_AACOTG park_i_Page_04.tif
f4b6700c1f609e918008c332dd34afeb
616fb71a58ff7f6ff343b2c36d8f4546d97e8bb1
3769 F20101112_AACPAE park_i_Page_70thm.jpg
a704d10dccc43ddbc49f84ee32406610
4fd72e8c5580b3ebe34459530b0a119871908ea5
6175 F20101112_AACOYE park_i_Page_19thm.jpg
ab3a83616e448d9c673974ea3616ab81
9330823d56793470491e8cd9ec6138f83a0315b4
84839 F20101112_AACOOJ park_i_Page_42.jpg
2ffee22c16b92022602b827865b1628d
b4fc5833ed3302fc6123405d5d516c19d915ce80
22199 F20101112_AACOJM park_i_Page_18.QC.jpg
6cfb61cd9c6f736cc6ac4428aec9ecf6
121a25f0ed05183dceb8f1d0c109bef72efdf890
F20101112_AACOTH park_i_Page_08.tif
01ffea5ec3d957314cb530c8c96f1983
9c2a752e00bd9826886174b8635310e191f2e75f
12923 F20101112_AACPAF park_i_Page_71.QC.jpg
ba06be3b477fb28e09e2675140534cbd
1dc6644fc2d557fbdb60da893cf1dcddc3fc6a8f
19132 F20101112_AACOYF park_i_Page_20.QC.jpg
c27b3c01ca278a90b499adbfdf4b039c
dd2a71b274962ea3eb77b50177edd177c286a9da
122721 F20101112_AACOOK park_i_Page_77.jp2
c44e1d2f3faed31558859e3f2a05f8da
75c5235b6573179965d1289a756ac0861395e8c5
12087 F20101112_AACOJN park_i_Page_73.QC.jpg
fabafcdd3ebfae9b72e98d3ca7e8578d
bfb765305e50831ccad85789d7dbbc16f53dbd82
F20101112_AACOTI park_i_Page_11.tif
991eea44e6ddad2758c342ef780af0ef
756e6dfb10f77f2f64b44abf75381c4e14c5f5cb
12168 F20101112_AACPAG park_i_Page_72.QC.jpg
60488a7ebe4d0d152b66693beaa599e8
527fbe4f15a8bb40516c1a469fe69381790e6ea9
27297 F20101112_AACOYG park_i_Page_21.QC.jpg
2331e62d52283913dd758124040c059e
7f624d21f27e0b4f41bcf18c0d3ea126ef282f43
F20101112_AACOOL park_i_Page_12.jp2
78e74fe6ecc0563b6ce60e1cc5b86660
69c487c4f26d9277c569b0b71bc1dabfe31bc854
49431 F20101112_AACOJO park_i_Page_72.pro
e4521e0444e11daa839ce541b75e1385
2ab14d613f570162edc2f00467c4d15dbfa06d8b
F20101112_AACOTJ park_i_Page_12.tif
6db11e897201d6d65bcfa7f7d799fac9
59deb2f78033ff03bbea4ae0f0bfdc3922a23f0e
3241 F20101112_AACPAH park_i_Page_72thm.jpg
b5486e97987165163d2e19e613f3f6be
3b1ea68345fcb042eec4a376a6696f7cff690daa
5609 F20101112_AACOYH park_i_Page_22thm.jpg
861311b171c16e1e73b77191ea54244d
3c8ed9c0762dc61cc2eaf4bee1202dce00887d5a
4160 F20101112_AACOOM park_i_Page_26thm.jpg
d217bfdd648d2a89989a50d1bd244f03
68c40f7c2d99116d1c0d08eebbf3f50bf9aaff46
66836 F20101112_AACOJP park_i_Page_14.jpg
eaf9c0c94c8a51561973eed1a531095a
940bdcf39d71b3dbee0e5f879569506fc13aa482
F20101112_AACOTK park_i_Page_13.tif
c13a3cb692b42d078cbf605e92ad39d8
9008a7f5e92a3394ec6e742d9c587d9bf2bae32b
14941 F20101112_AACOES park_i_Page_66.QC.jpg
d8d063af9d71016f4d3513cf9305a90c
6312cb007a4bd8945bb0cdbf60ae8d68d2a0a4d1
3478 F20101112_AACPAI park_i_Page_73thm.jpg
78cc36d35548e5e9ac46199065a6f7c5
a727ecc4874d545faadf8c2f6303fb1fd4fba6e8
19987 F20101112_AACOYI park_i_Page_24.QC.jpg
444161f37dee4801b19c8fb8309a1539
2ffba1eb736c5ee59e7b7825ae9558cd2ae284a6
35567 F20101112_AACOON park_i_Page_63.jp2
53cd9c577767bd17e70acdc34b465288
e3da5057c257bdd1244badcc33714fd8c373b4ef
755616 F20101112_AACOJQ park_i_Page_58.jp2
0d96259246c71c5a60c73bcae68bed0b
520d0d44af9d169ac6cbb55dc33d6d4e9bd43731
F20101112_AACOTL park_i_Page_14.tif
5562e6c81b1949c0f2119d1ff177a696
d95e10f6e7db417d9b8f7d99f0f8116e41b1e735
3770 F20101112_AACOET park_i_Page_67thm.jpg
fd85d0cdbb34fa27c907c312ed30343a
c4b3cb8070407bb38ca9088db3fb2c6cb97edd3c
21520 F20101112_AACPAJ park_i_Page_75.QC.jpg
30bd01da4588f4d478a8f1a059c48b6f
a05d7d4404283004af94f5c00cb4924160f50c8c
6978 F20101112_AACOYJ park_i_Page_25thm.jpg
903a5da08a9ae7afc24a5787e21125b7
9bb9a95b1ed8fe686260259053f4da2654a75dfb
72185 F20101112_AACOOO park_i_Page_28.jpg
0bfd801899a442b71d041fd967e98791
7e17b10d88d2f06e308e6c8050374202656dba8f
55488 F20101112_AACOJR park_i_Page_11.pro
2823f98a2f4545f0124ba55c3d97fe08
993729911c6d015e6f6a1016a4e97c81deeaafdf
604660 F20101112_AACOEU park_i_Page_62.jp2
6160869cff12eddbeef424836c9ed543
950f5fdc67ce9e41c4d2f4ba647a44986bae360f
5697 F20101112_AACPAK park_i_Page_75thm.jpg
3e68dde2defc83ebef687257f71aa0ba
490328a2348f536d0909ecceafef49543d1a6120
6628 F20101112_AACOYK park_i_Page_27thm.jpg
23fe701f8d154bf7d6dc1e8cdc809c75
bc30e35894c39bfe003d8d5af6d12f480ba6651f
707935 F20101112_AACOOP park_i_Page_15.jp2
f4a87504562d322c88ca12cb6e5c80ee
c60e70224f46a71c73c6767ac7ed2bee3254666e
10213 F20101112_AACOJS park_i_Page_04.QC.jpg
fe3335847b6801905d083945ec413e5a
3775597391e2016f0f615e62d2e7003139fe319d
F20101112_AACOTM park_i_Page_16.tif
e5380f469d09cda23bc447d63eeb6aa4
81e9ef82d89ca1144eb2717d8bed7da1c18e94c6
16604 F20101112_AACOEV park_i_Page_37.QC.jpg
2429dc9e2294ff7ec545babd8271a979
68ad6cb2434083454a2fc1efcb8bd45c8a7b42dd
22582 F20101112_AACPAL park_i_Page_77.QC.jpg
e653a7f19816936bc7ba948b9d829555
07f0640d09469431d1b326076b83dab180e6b6bf
11421 F20101112_AACOYL park_i_Page_29.QC.jpg
a9508288b6a3e83c52eb14a1e3017b78
bb1b5dc4e24953bd65252546d88afee6f67b8e05
18771 F20101112_AACOOQ park_i_Page_74.pro
26dc314af0f83aebd0ef678a675ae6fe
950a4a71249e53b25cfffb7c0e6e931c070a5f0e
5513 F20101112_AACOJT park_i_Page_20thm.jpg
00894465c4abc3add5d433633f67183c
235715a05788caa3d6a6be42621aca964b81cf8d
F20101112_AACOTN park_i_Page_18.tif
fdee312a0ce8fc8655e173cadeaa5600
620def9315c93e2197329615e4544924a11e830f
1852 F20101112_AACOEW park_i_Page_50.txt
73007b9482f4690d75da020ef55fd2fc
91cd6b8aa1bed44fea00484067010261a900ed94
23467 F20101112_AACPAM park_i_Page_78.QC.jpg
7526b4f07d9d9dc41e237aa3508ced54
0eed45e9d011115e02806a3cace53b72f55dcd2c
6741 F20101112_AACOYM park_i_Page_30thm.jpg
2b7eef753139060d1ab97e2eab91b295
ddc25357f2ae543d1eb48e590eb6c5af516e87ec
17617 F20101112_AACOOR park_i_Page_05.QC.jpg
63e22cd41a02264d8c70c06a0090424e
616cd2dad8ec1c5ef5742a95db8b326cda6d77f5
2843 F20101112_AACOJU park_i_Page_48thm.jpg
35809b98bd1b368ec9880a496e859fb1
31d63b6c4f706ab2ec396f34952421a4458e1389
F20101112_AACOTO park_i_Page_22.tif
6b3b1b816b93bd228ead066e48e60e25
9ba5d96e3bc64697945a8c870fe25276f5b7bc45
4489 F20101112_AACOEX park_i_Page_02.jp2
e088440792d9eb376bfa2103a5d04ade
d800545d7c4563355fea65f12a88c4c3c3e017d0
6102 F20101112_AACPAN park_i_Page_78thm.jpg
a96e7982d234070b6ecc0ab4d96f89b4
aafe3bfc165fefc702c4166117712e4d8484acb6
4292 F20101112_AACOYN park_i_Page_31.QC.jpg
ae71857c1017c4fde80425903929922c
7edc05f9206ad988700068129526f4b9097cd6e2
24652 F20101112_AACOOS park_i_Page_79.QC.jpg
505cb09b933c9255619cbfb355fe9571
165c687e371149c42ec8a8ef4b5375a7cbc4876e
65695 F20101112_AACOJV park_i_Page_76.pro
35ad8001388e1a6fff805cf44bfa4f3f
b065c5e17c53d2a5502758c0006cea8c0536747b
F20101112_AACOTP park_i_Page_24.tif
ba0372ad8db0e7e8ca57066048f28216
399dcba9a6f6b58a9d76fe4f6e27c5d4fb02e162
720 F20101112_AACOEY park_i_Page_63.txt
936712cfbb4ab95ed21887f18ae1123c
142eb4a62499f3715763bb635970bb8b6e75c5cc
4778 F20101112_AACPAO park_i_Page_80thm.jpg
f31af9780809c9737582263e4a8a8e1a
b051350140887bef10b401574ccf419a0bdf8737
26636 F20101112_AACOYO park_i_Page_32.QC.jpg
5ba072ecdbae780077ce51309653d4c5
1ac9392dbf620c09599d3d9d1980ed1e1ec56f72
699521 F20101112_AACOOT park_i_Page_37.jp2
2af2c2d27ba979834d0e328e9fe0716d
4e9b87f65168a2498edadc226e04a2864bd68f84
2405 F20101112_AACOJW park_i_Page_72.txt
87cda270aa8abd37de7a232296b80b0b
89a69586e4d1dc8cc426419b409aad5131fe18ef
F20101112_AACOTQ park_i_Page_29.tif
cc7de5bac22c6994102fff1dd810f13f
85980d6c118e96d55a8924e3e3b1bd495da25312
52038 F20101112_AACOEZ park_i_Page_60.jpg
f063871782e81b34bb6341db128555af
318aa0354988c300a9a2abb8459d4135ca87d326
8041 F20101112_AACPAP park_i_Page_81.QC.jpg
8bb112bf382ee8bf1716e326cc6357a0
38bab87581ea75d3e192c69401508f90a93b0fdd
14399 F20101112_AACOYP park_i_Page_33.QC.jpg
430de89534b1d59d120be9a31a1e41ed
6a9719dd9b1da4569d6c68a3abb6d9a6f27e71f6
454 F20101112_AACOOU park_i_Page_01.txt
bfea1dc1d2b2a590f75e569fc0e92e4e
f8e01c95e37987ad455ca0c5ef9c05fb4af128e4
15663 F20101112_AACOJX park_i_Page_26.QC.jpg
2379be81e2b5587dcca87cdcceb29a16
d74fefa4696c42ed5829831ec7c0ac14883772e1
F20101112_AACOTR park_i_Page_31.tif
5807e5a8eda51f63c496ed7d78e59bbd
6b35a829259697b5cc017f3d6b2562191a5e5bc9
2429 F20101112_AACPAQ park_i_Page_81thm.jpg
1b0ac2d72254870c7616190e309c78aa
ebea06baba77b707285fdc08e3755c8c29905f74
4215 F20101112_AACOYQ park_i_Page_33thm.jpg
0068510d31b967ca7939f35df343fe4c
4ed244e7f8d5feab8a7951e2852884b08a6f4dd7
F20101112_AACOOV park_i_Page_43.tif
41bcd9aa8f67130084eae715f26bef46
0044cbf0e5d9fb043e9941917917161e33a4ec94
57318 F20101112_AACOJY park_i_Page_10.pro
8b0a086423d699014a67db4a1f7f678a
04fc0bcc9466d9a49dd2bce9f22e0ce248b38926
F20101112_AACOTS park_i_Page_32.tif
cebd0d5bf646ff31ceebef15caa27e8a
f074a0d5ef0b8fef152f61119e015ad334ea9549
94019 F20101112_AACPAR UFE0020302_00001.mets
44ac85466ea7c812c421e392eb613f2c
9672ac4f32f3b47d28194f3131e99b39ec9f8558
1220 F20101112_AACOHA park_i_Page_62.txt
a69dd45b8dcb44dea4bd1558acb97d36
aa25b4f5cb1419e08e092b171e59564ce3fa5fb4
6743 F20101112_AACOJZ park_i_Page_32thm.jpg
f10bd20eb91ef3abcc21153c09edc7ec
bc8389847fd969d1a08ecbd09b5b184db6b1381d
F20101112_AACOTT park_i_Page_33.tif
b17b67b72b290651bb4a8f42bdb254fc
3fcb0b39f0645117b2d1ac78b340e5fa26212240
18987 F20101112_AACOOW park_i_Page_04.pro
8d8cbb219f2148dd1ff334d18980ebea
dc1b970b659242e3683db45f6fa7f6898e623cd9
22464 F20101112_AACOYR park_i_Page_34.QC.jpg
f11925c872aeb493056090f3c8ad59ae
ed6998183f83ea371775246f4fe50c5a75638daf
5327 F20101112_AACOHB park_i_Page_13thm.jpg
3617833b49fd85fbfaa5aaa0c3380acb
4edf30fac5f5eebd274b409de9f592cfbcb799f0
F20101112_AACOTU park_i_Page_34.tif
11c88dcb3eddaab2112252148bb47e8f
d71a1d06d462a251b3a28705677323d79a74d970
47683 F20101112_AACOOX park_i_Page_19.pro
4499a6184b88d75729825c9e551c6d04
fbbed6f5b83bdadb62ee7c19526a57f1122e0359
6176 F20101112_AACOYS park_i_Page_34thm.jpg
328ef28a464550aa34e13a5d32245d1a
38b2d7be7303f93159b1a0bd5dcce92c130c1e78
5065 F20101112_AACOHC park_i_Page_35thm.jpg
e9f41fc41ca167af1f70305a8bbe83c9
c2a0ecebbec7eddc066c5ba232ba1273c5064f8e
F20101112_AACOTV park_i_Page_36.tif
2fd47d9eeee8cd855eb7635c824f4efc
3ea93ff9ee6367af44a60c16dc0e077be9e894ec
5849 F20101112_AACOMA park_i_Page_49thm.jpg
61163de809cab7c463d01505a1a8a449
47c77e548d32b53403186c5ff2a3fcbfdf9c8091
3055 F20101112_AACOOY park_i_Page_07.pro
09a4743a00e724de986ebb0e14a2454e
0b8dfa4e27e3bd9f11ab72b5abaa1975487b625f
4784 F20101112_AACOYT park_i_Page_36thm.jpg
bcd609d6503a7fe9459ad831936f59dc
e25631a201aa37f261497f11bf38c894fbc39dfa
37013 F20101112_AACOHD park_i_Page_52.pro
efbb7de45b5efda778bbc271ce2d05fc
40dbd332171cfd4d855eaab1b4a8ee7665e1d03e
F20101112_AACOTW park_i_Page_38.tif
f8779611904da87418d18b6afab75c85
1a9bbc61d64a3255f13b0b8d8b7fa9912782c33e
F20101112_AACOMB park_i_Page_01.tif
e4a139469fd300db8c4cf9caccd15833
8750953fefd6d028c5c55ee00c902854a5047db9
18808 F20101112_AACOOZ park_i_Page_46.QC.jpg
7d2ce446dc9034968cd55eeb4f97fc6c
2e072647f4f62e2995a637457cf86c308a7dfb4f
5085 F20101112_AACOYU park_i_Page_37thm.jpg
98d1be5072da2b6e2dbe803e6bc1f722
ea51b67c27b60f51ec3a5b69627be90eaf2b43b8
1841 F20101112_AACOHE park_i_Page_13.txt
bc8f50a53521cb0926fb58a31e0f4261
b9d5f670005e1ad40ba0644def02076fdc8d57f6
F20101112_AACOTX park_i_Page_44.tif
0feedfcf2da6e7f5d16be2339293f0d4
71c0ccbbb6ee2e99c2a5cb7fd6892d18e330f5de
67831 F20101112_AACOMC park_i_Page_39.jpg
e427b5850162b749e37f071ba7af4cf7
88f0ae53254ba7c7699517a4f45cf83bcab03bfc
18522 F20101112_AACOYV park_i_Page_38.QC.jpg
7d67145a9a40c66eb56d84874ed84e0b
bcef4956e493351e76fa10b5b8b555590d6e8c79
3027 F20101112_AACOHF park_i_Page_44thm.jpg
0171ae77f7199c74ac3dd8e27230a9d2
aac7d70841a333c8de8bb168b980292e89f7003d
86047 F20101112_AACORA park_i_Page_32.jpg
b52f04f8169a08160d915e6a19252cce
36f969d34cfa70d4373c9218a90b720f5c58a0b5
F20101112_AACOTY park_i_Page_45.tif
6942de29ba927c4abbd409a42d1e7834
26386e9aa251f9c91446d0d1e337dd6f2204a017
35665 F20101112_AACOMD park_i_Page_28.pro
8d1862b2049faf34c031cff0be4f467a
aa63f8be80e42313c59310376561512d706b5185
5287 F20101112_AACOYW park_i_Page_38thm.jpg
ed4703d30a804141b230f8a4c925834a
c553416021617e3d4f1f6d26b0d0f8ac348137d5
15776 F20101112_AACOHG park_i_Page_36.QC.jpg
a9de9d86f085be29454bd2ff4039f037
3bcd645bcaabe458765bdf8b41201dc6da87055b
48089 F20101112_AACORB park_i_Page_33.jpg
81631d5cabf557e06f488105a4d65d18
fa5ac4fd676de53bc06f91d0ce21e38c057728d9
F20101112_AACOTZ park_i_Page_46.tif
b484bf759c1f51efe00709f93b0a401c
fabb1b196f08643e15363c4517c02872e2ed3b9f
F20101112_AACOME park_i_Page_10.tif
d60b6a40b4f2e2befdd67060eb94f11a
703852c3b99165883c70ab6344bdff675aa3a706
23953 F20101112_AACOYX park_i_Page_40.QC.jpg
51b7d6cd5a35d589744dd60175f7771f
efd030a45f3a4ff1e9df1fff5fa60cba3952f2cf
57449 F20101112_AACOHH park_i_Page_32.pro
04dbf25b1640f82b81f39114735a163a
df34e9af46a88fcda0d8f5a8649334775c5dd905
71347 F20101112_AACORC park_i_Page_34.jpg
6ee967c478dc7d58bae9c28b2058c776
201dd0e55f63acfb747498dac78d06852df72880
6269 F20101112_AACOYY park_i_Page_40thm.jpg
a4c860a37b73cfd398a871e303079f28
5e3efda32579243352040be5dc422a3a63b790c1
40915 F20101112_AACOWA park_i_Page_80.pro
70a7b5e3e9341910a2773cbddaf6a5ae
f3cad6af099c5b37a8eb73e92395b6a0a0137de2
5610 F20101112_AACOHI park_i_Page_51thm.jpg
1e9c49f4238771b552756c6ae2d38b9b
099ed485a18f7e13f5bda0d206cec4cd3c9bad69
57505 F20101112_AACORD park_i_Page_35.jpg
e37e0ecc0355c73ed6ec2bd1e94e5196
4049dc2358c49c19b6b88e8126435a79f16e5f83
F20101112_AACOMF park_i_Page_06.tif
3ced4442860ee09ca1c80bc024b6bd44
13170f4aea2e68c0bb15fd8f689c57957b853ef2
22947 F20101112_AACOYZ park_i_Page_43.QC.jpg
af7817691d716abd190495cdace89083
4bcf7b71dd606368faf98490ba9b2684353f72e8
13482 F20101112_AACOWB park_i_Page_81.pro
545fd5918a17c734270508128838b0dd
6d13a7d52d17165174757b94a78107111615e9f1
50405 F20101112_AACORE park_i_Page_36.jpg
ec3ee18fe8004b65ad9c625d1a0fb16c
e698a586165b99cd77b8ac19f81620a68f4bd5e7
761018 F20101112_AACOMG park_i_Page_16.jp2
f2d3956bdda8e995fbc1f596a4b720b4
30149a266e0b0dda59ddd17fefd3fb64a300aecc
134680 F20101112_AACOHJ park_i_Page_79.jp2
d43bc14aaafabf5dc752a300d923dd05
af47ee2708c40365d99410b394d3a8111fc3f058
1620 F20101112_AACOWC park_i_Page_05.txt
d112d04eb737aca47a86f4a42c7eeea6
d32fcf1ac1beb4bdf3255b77aab3f63d767b6bb5
79449 F20101112_AACORF park_i_Page_40.jpg
0e367a1f8e23ea94ff3d9b308c3fbb83
42a7a5960e78ba3c57add9388671040ccf12e0ff
76 F20101112_AACOMH park_i_Page_02.txt
5da5de3d2dfd5fff8dc782b4b696f8fb
096b1e5a502cec697e320cb1c7d38eeca5dfb45a
F20101112_AACOHK park_i_Page_35.tif
28e1fe8729d24ef3e722f53ec38559c7
fd5171de3238fdf8bc13b8a1e694c7576d9c9463
1945 F20101112_AACOWD park_i_Page_06.txt
3b05f701d1bb09379be8ef1a544ffa83
0aa18babb3f56176ece1a2400f4f6630b908781c
53266 F20101112_AACORG park_i_Page_41.jpg
db09ad67f531fa4aefebb7b7c303390e
1e6d5bbba6f3ffd0c06672ce79bf99ad0b36e3f4
5181 F20101112_AACOMI park_i_Page_61.QC.jpg
1fc313a94b40f119528e6253b0267d30
2648ff7a6c46eba54b8cdd2a7aefd8f7ea68f3a6
F20101112_AACOHL park_i_Page_75.tif
7ea2c5e440d8020ae2726ef7bb2e4d70
8f73eb64da4ba3e0b64372bf48ff0d6b4d9c0a41
131 F20101112_AACOWE park_i_Page_07.txt
85e4f176fecc4d82e4ece5b46e1ece26
a521327316cd72ff0aefe345a264da02bec54b3c
79187 F20101112_AACORH park_i_Page_45.jpg
5d516a6dd7b7f480e44e54b4c596f994
bd4b76084732e8e86925007069bc2fbdfb36a7c2
3427 F20101112_AACOMJ park_i_Page_69thm.jpg
afd77e41b74a60a64b2275bcc8610366
21805363f9e28d8c45277caf35f4de6831b58b96
4467 F20101112_AACOHM park_i_Page_06thm.jpg
36e5f5fa9b016e17592d69e44a57aade
ad6a274e36789397d6f449c1ed8e76ce7c45cf11
2185 F20101112_AACOWF park_i_Page_11.txt
b061e4c99644bd795581b39f8647e914
24463950682d8528bab21d7b1a6fe1f50581d7c6
58100 F20101112_AACORI park_i_Page_52.jpg
4c7080cc50ab71259ccd445f0e7fa022
e8b5e95ca064c8865a2377f313f791f40924df67
20380 F20101112_AACOMK park_i_Page_14.QC.jpg
3ed06d826708a0410b50e4b9faf953b5
e5bb1d308387b86a3e53ba5d19e2e4afdf035a33
66002 F20101112_AACOHN park_i_Page_51.jpg
2f7edfc9c08db1cd2e4b363387381aa5
9c9a6960b07b58ca180e6be36279375dab54247d
2273 F20101112_AACOWG park_i_Page_12.txt
ebbaa6b49f9fb5da68572d13ae03971e
44a5596eed06dfbe3a8f3bf9943359fbe82c2589
62075 F20101112_AACORJ park_i_Page_55.jpg
07ca7e528945f6182c24904a64216b7e
3bcf0e0e20df40beaef028c76c34c5dd64349c80
26226 F20101112_AACOML park_i_Page_10.QC.jpg
96d4f72b8c012cdb0bb736cafd955835
e24c727f77f117e620dd3eccff55a9b013a37dd4
F20101112_AACOHO park_i_Page_69.tif
038ee56fa1e00dab35694e8dfa1715e7
02b9426995969787b1af1cc3666f8b98b90d73eb
2006 F20101112_AACOWH park_i_Page_14.txt
7706a2f01f10d31db84eb8d0388a67c0
8b89d44c6b9154f49ff24cdc0ac3af3b99973b3b
99153 F20101112_AACOMM park_i_Page_68.jp2
7eec7ee54e43d9cc2429824b98b08f3c
4f8f71867d565ac02c60efb0e658389d8a893add
56322 F20101112_AACOHP park_i_Page_47.pro
3d602da39d6363a4eb318bc3e8b8d7d1
8073bdfe94b148eff6d99d9af7753073000ab36e
1675 F20101112_AACOWI park_i_Page_16.txt
23865b70cfb7849340ade97444c9fc1a
16a6dfb1863b548e371a46b53e1b340b4e73a0db
16152 F20101112_AACORK park_i_Page_61.jpg
e52d19f5dec38d0af815b8c95a92c9c1
964948f736957ac8d5b85aaa3d32eb05f429eb49
F20101112_AACOMN park_i_Page_39.tif
1c56f0558c8e121a452c0b57ed16f865
f04138227447559da347e851fdf7c7a45b058c02
57490 F20101112_AACOHQ park_i_Page_72.jp2
275081233bc59cfdd196afc1d868ce75
232ffe0a9ad28929fde1f7b4693a36fc99e2311e
1951 F20101112_AACOWJ park_i_Page_19.txt
0db419913f382b9212a83ff75f3348c4
d3abcf30628d62ed4cfa9e62ae747b5a024934c1
46492 F20101112_AACORL park_i_Page_62.jpg
1f1eb4b7daf6ff0a79103e43e651eb80
1e1d4c950c4fecb03c94edfc3fa0178798ed45ca
84102 F20101112_AACOMO park_i_Page_47.jpg
f90f68bd2383f0fc57be57013aca56b5
1f3432881523177348913da8cc505b5b63ff669d
1051932 F20101112_AACOHR park_i_Page_30.jp2
d3a6241a9bf609120edb6b0151587146
023fa75ac78c1d4d739ece74e54a07beb0490c03
2557 F20101112_AACOWK park_i_Page_20.txt
092c6eb001ce2ae1d670f592e92f5728
4ce23e780d9f41375785cb5cc031eb80ae4ccfcf
26682 F20101112_AACORM park_i_Page_63.jpg
f4823a8b00491e8e0699ca1682b246b5
380a71e8a5fc55b5b5586221aa51238edc868e55
13875 F20101112_AACOMP park_i_Page_68.jpg
937dff9b96ab5bcf6c4d43a6d12e8477
6971fa671440e8a33e507eda4b765538d9639f09
7783 F20101112_AACOHS park_i_Page_03.jp2
aa00f95f5a21535d7030cf2e8e6daab2
4cd2d9568fae356e686c143e1aa36fdb9829a9e0
1861 F20101112_AACOWL park_i_Page_22.txt
3d883e5b67080988d2684208338c3c60
0c1845598bd6e876ae9c45b77fbe258793786e00
24500 F20101112_AACORN park_i_Page_65.jpg
f767fc909d187df2ecd729f83b1c4dca
06dae4591c9ac9dae8a52e3469411e5332b1fc71
48970 F20101112_AACOMQ park_i_Page_53.jpg
284ab6bb837418509f93a07aaa3df15d
c4228d72eea4f7be1f70b55b95e6be335a622177
54577 F20101112_AACOHT park_i_Page_73.pro
834d4e7f8f886f7d40ef8505fdb2187c
06e7c4c5d5d4bf6357def1f6fe0c6068948d5bc1
2216 F20101112_AACOWM park_i_Page_23.txt
bb2c6d39a802c9d1912a45995469700c
35914458a5aa7f759cdea3f257b8f4e63ca14f1a
44549 F20101112_AACORO park_i_Page_66.jpg
c10d725e4448d35678e1ba7166ad1037
bf90f3159c93bcfdda4a358bd73d3ef570161019
F20101112_AACOMR park_i_Page_37.tif
e60111ae01fe5bc653aca1f12054f725
223e124a139d9e4cba0a9301560ab6d186e5fcd5
1596 F20101112_AACOHU park_i_Page_07thm.jpg
16ff385658b99cf82415b03a437e404c
b88baaca12542e14363691ffae6b70fe13882587
2354 F20101112_AACOWN park_i_Page_25.txt
18f4fda47fffc0086e3659da213ba3fd
cd6b727c659dfdcf30b41853945410d7a162a323
41131 F20101112_AACORP park_i_Page_69.jpg
b17f75757bbed49c8712d7e5b39bb12a
1b3cdf2657950c855831136716a4323ed57e1210
39541 F20101112_AACOMS park_i_Page_64.jpg
6feaefacab0064b8207905a8cddb83d7
6444340baa84c3091218dc85a6aab629b7110a2c
73020 F20101112_AACOHV park_i_Page_49.jpg
da43cb7d8942caea108379f17715e11b
6698921ead9e728941806f8ed7a03c495961ef47
2225 F20101112_AACOWO park_i_Page_27.txt
6d4fc7999ec5c9beb62cda4550e84851
899844803a515fbdcde9d02cdd5314511c10a6f5
49613 F20101112_AACORQ park_i_Page_70.jpg
da024cb88d3f57f1b71fcaf1b284b488
ee5fc4e6a3274cc2a2ce61580c8cba82e1345a6f
43746 F20101112_AACOMT park_i_Page_26.pro
512257ee93170cc45dcaf3358b18955d
31191b90417b4cc512eae5b146f11fb53226aa0c
1476 F20101112_AACOHW park_i_Page_41.txt
665fa9ba6de82f3abf1c1e0c9f05da73
024c8aef697ba53e63660abc4aa023852846cda1
21021 F20101112_AACORR park_i_Page_74.jpg
3573633c97b719d6ad4221c636be1954
27aea4076022e2ff8a14204e2387947e2914308c
6622 F20101112_AACOMU park_i_Page_12thm.jpg
0bbc253bed5d1a05944e3ea0df8a5fd6
3bbffa6ba711277d0ead5f22e4a41b8f690ed3c0
58769 F20101112_AACOHX park_i_Page_38.jpg
2701844a250d249b2003be53743eeb3b
5ca0c10b33008cadd7b1893b10d142cc70e213ff
1655 F20101112_AACOWP park_i_Page_28.txt
cbf373cd02f968fbf45529c750f00b03
1f99b236e4ded210f90f2cfc26d12b5cac4e687b
F20101112_AACOMV park_i_Page_19.tif
72d8b514ad4e099ecaac079e4ebf0b72
cf53e4db41eed973d6b579d3bdde40b11f4665f8
F20101112_AACOHY park_i_Page_20.tif
8a73695791233078b099c8166fe9e8cb
d747bc0e821dd047540a799085615be451714268
73220 F20101112_AACORS park_i_Page_75.jpg
d27c2fe5fefdde48d4acf6fc73255856
040d9f69c7866c2f7f4350272db2ee57bd9789e3
499 F20101112_AACOWQ park_i_Page_29.txt
3e7f4f8a3359ee6585172844ac0192a4
c46d125327f9fe8f8c7cdfbb6abb09a663577255
2204 F20101112_AACOMW park_i_Page_15.txt
59a46f836530a0ed80529b46f543413f
326f698aabc2a8cc2258236475621e5f4cddb833
F20101112_AACOFA park_i_Page_54.tif
3eb722403242d2144c3ce5caa1edf6a7
4a26f493adac39e731d674cf55df256ca8e23b83
922271 F20101112_AACOHZ park_i_Page_20.jp2
48e90fac44f00fb7662ef44744c28f64
6de843567027b5d697c6329524415a7cce9a520a
85377 F20101112_AACORT park_i_Page_76.jpg
841ae5b5ffc63d5b8a60e33c078c6d52
64812a1a27575a5e7cd91a4ea74aa369562d8f5d
2261 F20101112_AACOWR park_i_Page_30.txt
1c27d1b6871fd83d545c97ce1720eb8b
e440e39405e7ab300d2447f055e36c312b356402
21025 F20101112_AACOMX park_i_Page_67.pro
b982d52fd59f0e89f64f61f85d650dc5
75ad6fc205b40eaac3490ffcf9efb9ae91805263
783 F20101112_AACOFB park_i_Page_74.txt
b7e3b35bbd2ed03ceada90f997ffa3ff
e88602ad50bdc134aaf274f3fa0e51b242d08323
78669 F20101112_AACORU park_i_Page_78.jpg
e8472bbda92f964533d31374865d57f3
e017d01984a4a9a873672e872ee03d23416b9d1c
168 F20101112_AACOWS park_i_Page_31.txt
aac58715f0ee022f6102a511351307cb
1c242adcb2a8c4ebfb318e8083651d31c89fc494
F20101112_AACOKA park_i_Page_23.tif
c35538720c7db16c24b36912c2fa8560
de6308d56aa6108e2a039d6bc84ad2fa9f9c65e7
F20101112_AACOMY park_i_Page_80.jp2
b6b70018a0c61c5572886a09c3083042
b2d77840ba4551d464b8578898c3e98bb1ff852a
378970 F20101112_AACOFC park_i_Page_48.jp2
7b87e69d4f7a8f2af35e93fd71b4c970
fc665dcc4131d704abb7d5e1d1bfc9f8b71db140
83543 F20101112_AACORV park_i_Page_79.jpg
d8fd4afa8bb5f05671a6cefbd152a426
f5e1a595ff9ed69e152ec2b055037a0748d1b5ec
912 F20101112_AACOWT park_i_Page_33.txt
8cc689b142e3359910489e61d657a495
69831ecf166b7486c6db1d647bd59d5b4837be0f
18464 F20101112_AACOKB park_i_Page_28.QC.jpg
b44a47cacc88bc14475fe6e6c5758afe
59216660195e502ceed677d943c6fdb33d0de64a
2912 F20101112_AACOMZ park_i_Page_04thm.jpg
c42d2339caad2a245f7daf61fba3ee75
62c34b81d31a90f12361a6d371c008b6c1ff4c8f
11913 F20101112_AACOFD park_i_Page_31.jp2
c05c260c7736141d14b70e8b2b94c5b7
882212960b69d9e2152a43380656ab0cf9e22a7e
58182 F20101112_AACORW park_i_Page_80.jpg
ffc6d6dcd89f7abca5ee61cd0e1cd228
1c0c3dddac10f7d8154aa87559cb3217afac3d6b
1757 F20101112_AACOWU park_i_Page_36.txt
9d85abaa717a38921cd993db773480be
e1197cdb9c6ab929fec8423f2d5d502b7bc75a92
60533 F20101112_AACOKC park_i_Page_46.jpg
d3740da48cb95bc4f0840417448f46fa
e6960642b97d4dc6ba9b7a27d10fb8cb05181763
79444 F20101112_AACOFE park_i_Page_70.jp2
ddcd30869f2295f75583c3a1bb66fd60
6ab7e6a919a3986b8367b29d2e691068ac42cc3e
25136 F20101112_AACORX park_i_Page_81.jpg
d93622ab3748233ce3ea6bb255dd20d2
1a21bf60df6c32388fce33100ac9eed636c06e19
1696 F20101112_AACOWV park_i_Page_39.txt
f70f79957913fc2aabeb913431c9c3e9
07878c6c8e34e84c8330a5b29547e78fcce879bc
67638 F20101112_AACOFF park_i_Page_70.pro
76d1102e5a66886ee19755de12af5cbb
9865c251b43c272c964ef94438c6da760823afab
2365 F20101112_AACOPA park_i_Page_10.txt
49c103cbcbea119593e5a0dfc442c657
57959c57a26ed58b5b90cff302509c82050c6772
21902 F20101112_AACORY park_i_Page_01.jp2
11e0da7633afa5da248c062ffc4cde5e
2c81819857d4e13f6bc61a9874db1ac79cb0ec00
2685 F20101112_AACOWW park_i_Page_40.txt
0cb86d8436b93a6bc8b815086d977d51
51f740661dcafbb7ba3125fbd0261ede8bf6dc79
46287 F20101112_AACOKD park_i_Page_06.pro
61bfefbe30526d61e41b166b4ac765c5
7812245e0f6570fdf07d4ed3273c96011e34dce1
45013 F20101112_AACOFG park_i_Page_57.pro
292985ab6250102cc38038440923584d
8d5b853bf58fedc804e94db1110c8c0975d04447
2037 F20101112_AACOPB park_i_Page_74thm.jpg
b04eab1efc34cb1a7d4d4f5fdac64114
1dd341ffccfae9190293cceec3a1984b3795e369
44662 F20101112_AACORZ park_i_Page_04.jp2
4d40f662939fe5472ff9a65745c63975
08138ca99ee703a44a6bbe51185cac4b28affd98
2195 F20101112_AACOWX park_i_Page_42.txt
0ec38fafa08f7219551a6e0c7c5ebf97
7889b34f6577b1bcdfa3576d6c233b26ef20b51f
1051951 F20101112_AACOKE park_i_Page_27.jp2
61d15eaa6216267ae13198255a99951f
b42912fbf0573eecbc5078e38ba68b8bb3cb7410
1051933 F20101112_AACOFH park_i_Page_11.jp2
00bac6b948ecf5736e0fddd6ad6daa8b
8cd1f0be88bf3527ed5b8bb3a9efba7ae86c1954
51381 F20101112_AACOPC park_i_Page_15.jpg
6fbfb6bc2e30b1a0a7aa193db2957ad2
46e5925c10910bb36e46bd134377c5de08d588b3
457 F20101112_AACOWY park_i_Page_44.txt
c3192c395fbe38397d4655a75b91013d
79efa4f211be7eb13e8a13e7ede7a2dbcb7b32e2
F20101112_AACOUA park_i_Page_47.tif
9a53ac20006db386e30a76464215cf7e
41b14ecb1e0854c5190c842034be96837fc3d36f
25297 F20101112_AACOFI park_i_Page_62.pro
f5752c7f698db59dbac68ed1fe827b8c
62a2ae642732ddb2d8f5a5f0a163bc1182513c17
63519 F20101112_AACOPD park_i_Page_71.pro
4d76304173a286c7cf1650afa7bc5bc6
524ec442251685d57a685e0f1022442f7ba2ae5c
4577 F20101112_AACOKF park_i_Page_68.QC.jpg
d5d88fa494d25dc5b16b9511890a215d
691c645df8cfb69a9c526910d28aa36e0f7a5a7e
770 F20101112_AACOWZ park_i_Page_46.txt
a9347b1dd7c73cda40c097638544504f
96ef11cb58c5490c16ebef51744e1fb51c96d5f5
F20101112_AACOUB park_i_Page_49.tif
41798a8e2d3a64af9cb5ab9a0750e9b4
168b47c66ec1dfd8b9acd5b9c6335e32422391fe
45028 F20101112_AACOFJ park_i_Page_20.pro
8f81ea265a6bcb826e60e44a65b876d2
329f72a38c701dbb29d89cdb37a9e56a2aeb3dd9
42269 F20101112_AACOPE park_i_Page_73.jpg
5e41775defaa3d01acee70567c0fdea0
32a3afa036eb9d6bc3c21a6c611a2b9eb3540b41
810992 F20101112_AACOKG park_i_Page_38.jp2
1fc0b1b4a83231981c86bdbe42be51a6
55018a05f69a7d4176956154d4d041e2fe15d5d7
F20101112_AACOUC park_i_Page_50.tif
2d29b529b50bd3d6cf412d6d63ad1453
3509484a9441ef6a22613786c528dc85fde43290
59132 F20101112_AACOFK park_i_Page_25.pro
2b791031172c210f7deef24983023f00
183a06be7e5e91885be492b0e36f19382edce15a
18049 F20101112_AACOPF park_i_Page_35.QC.jpg
85dccfff1d6f86aa1c28cc2570cb3b2f
2b3d82202dcb6df12129116e1fca10d6eed6dab8
46733 F20101112_AACOKH park_i_Page_59.jpg
f49be384393c289c0e5946dc4c654089
9e411ce561eb66b2b9e88ecf84e10674637b7f2d
6117 F20101112_AACOZA park_i_Page_43thm.jpg
e79d41d108de9dd74323f5cb6594a52b
7610f7c02d2c526ccfa4eb280b8d7a944b147ea4
F20101112_AACOUD park_i_Page_52.tif
d13c569951beb48dadfa1c554262414f
a2170eb05fd9d4386921e5c8ca1368c679cb9bd1
31697 F20101112_AACOFL park_i_Page_59.pro
8839d1986af65de11d2d6172c88a60c9
77d4c9a8c1cc5801127afaf0f89efceb9f63e4f7
1665 F20101112_AACOPG park_i_Page_37.txt
8ebda218b2b1fa042467e7f4a5baa513
4896a0268a47aed71177b0cbeb9f68066da81fbd
68204 F20101112_AACOKI park_i_Page_56.jpg
70b0b51f006aa238fab6a848c70301c8
c9a9775e2553764dd18ebdaf7ddcaebf13a43227
9539 F20101112_AACOZB park_i_Page_44.QC.jpg
c2a2612f07c64ce163b78b17c119ebd1
df8d4715dcf048df3ed184dcaaf01e78bb00d603
F20101112_AACOUE park_i_Page_53.tif
7b226cff7b3d642e5a2aa850c4b2aa11
5c740cd7ae865fb37f777e5461ff90bc1dfb6b54
F20101112_AACOFM park_i_Page_21thm.jpg
f8ab0b3e92651ce44dbf956307d6b303
395cd91ffd6c079f1671e82d282a43908f8e1ed7
2080 F20101112_AACOPH park_i_Page_34.txt
3e70907905480d018038905af490b51f
ca78f9cc5b5f6915cd3b98b58163ca81dbd1d6fb
4144 F20101112_AACOKJ park_i_Page_41thm.jpg
657a61c6eedad7e4652e4fa2988d4476
2bddcae2daa4569f362e0244a9c9443e4a7539c2
23059 F20101112_AACOZC park_i_Page_45.QC.jpg
cd7354d49ee20ba5c1f30f4fb05d1922
2247859e0f05dcd8d47f620da90968c82a8ae94f
F20101112_AACOUF park_i_Page_56.tif
e2dfb447dfa5d8770a234eaed902bb53
f1382c93bc442becaec89a67ee847d45aec75133
32045 F20101112_AACOFN park_i_Page_53.pro
d6961f055f0f70c28b87a0cb26daf7b7
f587763464457f52bd8678d6c4c9d67e513247db
69987 F20101112_AACOKK park_i_Page_09.jp2
9eab22880396a713ef406cf665e0d691
c750c37376a00aac59395a6079311194d5e9a535
6115 F20101112_AACOZD park_i_Page_45thm.jpg
9119df4676691c0bf3b3d011a5a1ba5a
f3edbf32534096aac22d497293a5a6dc7d61c5f2
F20101112_AACOUG park_i_Page_58.tif
f0578f99cd093dee4c0cff9f81efb93d
53128b38f97ad72f92bf6c6c70db9d530482b50a
33353 F20101112_AACOFO park_i_Page_15.pro
a9d4de2a55af5c57422def0360486a7d
158200c080c81ea9b1293ab9c3d0ae71801f2a45
31358 F20101112_AACOPI park_i_Page_48.jpg
bd49b93b4ddf0ede0a9e61da92884bae
bb05b77948615edb87c98fb185b01093f845d5a4
6543 F20101112_AACOKL park_i_Page_23thm.jpg
7810013e1ceef9dd8dae7a9292059d8c
8e8ef0bca01af0234554438f4af2426beddae80b
27352 F20101112_AACOZE park_i_Page_47.QC.jpg
222c8c68957d32cbaed9eee084107725
aa51905b910077a5f0926cad7f970e48a49f0ebc
F20101112_AACOUH park_i_Page_59.tif
c681e229059f07168507c386dda2f42c
fae727d85947acc167fc55062ce7f8169d3a5061
F20101112_AACOFP park_i_Page_28.tif
4e647ebb18adefde46eca9789e0cb30e
8d5ecc2bd257d0c6667688970a9f47bcdf709401
1445 F20101112_AACOPJ park_i_Page_64.txt
c75f57da3846779d19f656bd3b2b0b1b
03996c73c925d89921fd4810d02dafc6e0302c6f
12912 F20101112_AACOKM park_i_Page_31.jpg
5973dda66e43cd8bcef58a2cd3df285c
6340cd5ac4a1a8e5625d719230858c17484bea4f
6886 F20101112_AACOZF park_i_Page_47thm.jpg
9efbcc4470763ee668d27980a227a77c
68773b5cea6f75b1bc1979713e43d37f8ea52105
F20101112_AACOUI park_i_Page_60.tif
8b229e7dc655ac44cf209e777f37cd4d
c976bf2da2ee4cccd303cb63aaa6686a811e326e
37029 F20101112_AACOFQ park_i_Page_38.pro
f23b543f73452db3651727bd84281cbc
d34019d34ffa004a266b9ee7dac23e38078c191c
24633 F20101112_AACOPK park_i_Page_76.QC.jpg
93d1cada614bc883c5883e0e9b767311
9f332d756ff9d32b6131f7eb0c6c97c9186cfe04
59065 F20101112_AACOKN park_i_Page_21.pro
c82c8b1af6d878c5bd963cc8ac4c0fcb
6c1d8bbcd71c54c32c4b698c0934d58cdce12c87
18721 F20101112_AACOZG park_i_Page_50.QC.jpg
c456114aa6221d1cd1dc5620f59ccda7
f0877909370f4f5264d3bcdb2ed1da2bbe347fe7
F20101112_AACOUJ park_i_Page_61.tif
1c6323c9e84bd1fffbaa6eef0bac56f4
3dfab090b84a69e2846dc7917a40c90ea8307a59
49034 F20101112_AACOFR park_i_Page_17.pro
50e8a39ac126290b3a265772026fc9b8
0fff4fb19ec85a66f177baf0a8b88187ce5d9e3c
60152 F20101112_AACOPL park_i_Page_69.jp2
b6511fc652c5cfd26df61e088163e0e4
dd9935666a69fb1521286ae42cfa2518e2670ed9
F20101112_AACOKO park_i_Page_67.tif
946a4f1e3f35cc8f079fbe685cec8f02
c3a6a2071381528b5944b95d72257cee6e071f27
5398 F20101112_AACOZH park_i_Page_50thm.jpg
4d90f531a9c028c9f2a4a4a0ec8ffe1f
a538590bb043de69269aa287cafda245bb4303c8
F20101112_AACOUK park_i_Page_62.tif
f6adc9cfd74e3bee9c4ddda44b3f09e3
0f1e952051292ab33f3e219d23f9ab60718b2f61
873719 F20101112_AACOFS park_i_Page_26.jp2
9a2efdb2570a6383cbae5a851f25eccc
81f7866805b26f4f65a5ab39c91aa6c6f96d8521
1051934 F20101112_AACOPM park_i_Page_45.jp2
c7bfca8fe9421946d6c062d1ddfe1a62
7da38e0dd3bf2e35731f545ad544605d8fb3e9db
40944 F20101112_AACOKP park_i_Page_39.pro
974305c5baa4f83c4833ef721bbe6465
a0184793ae83071f5bf9ccbfe8f49ce993b787b2
20809 F20101112_AACOZI park_i_Page_51.QC.jpg
4db8df662aeb0125e10c41d1e0f95a04
202f650bb6e488aaf60d45b4e7b057f69acca3fc
F20101112_AACOUL park_i_Page_63.tif
08b5cdade6285c14a30708577d7ac859
dd8feccb404a3c43081e61ebf002908fab99c005
1051917 F20101112_AACOFT park_i_Page_28.jp2
5f83f57ce0b1f00d8b32e1f37433ea79
d2a4b79d24aad9f815a7ec93ff11d284f72fbd99
22948 F20101112_AACOPN park_i_Page_49.QC.jpg
0a13bca457f41f64b0a4e291df29d88d
5fae4f117439dd7a75e026d96cf7c704bb170f30
F20101112_AACOKQ park_i_Page_09.tif
a3f60c53932d123f9855bda051a054ab
d5ab604994370cab4742a1f168355ec5bc547c8c
18583 F20101112_AACOZJ park_i_Page_52.QC.jpg
f9263134d9e034130c7e3cf7c14e8b85
ee6ac31c03f08bfe1bfa13d3e2b20a8b3f41d272
F20101112_AACOUM park_i_Page_65.tif
bf382e296ae8d1467fe138f152af21de
cc2b57e8a59e463cdb47574144eb881adabe8129
1705 F20101112_AACOFU park_i_Page_43.txt
34c9776d47120d1827588c4b8a39805f
2f494597e570dbf7a62bee4d32ca471ab8e5ebc1
1051931 F20101112_AACOPO park_i_Page_21.jp2
3f554a40d37cd39ca815474a92784b43
ea628b1d2464bb023168f1fd2fbd48b43fbffe8c
42147 F20101112_AACOKR park_i_Page_51.pro
877c98056b87e090204ebfc9c59b7c9a
f1af20c7f961670a06b7359abfd3bdb724a3c268
5074 F20101112_AACOZK park_i_Page_52thm.jpg
acf8512ed5d216b12c9dce9087473390
4206044d65c7aaa69e9397a6172ab426d777a768
40575 F20101112_AACOFV park_i_Page_50.pro
94c8309d47cabfdcdf26d42aeeeab77d
2d1625faee5747b381af020861aafd6c5c427dc9
84849 F20101112_AACOPP park_i_Page_12.jpg
8eab44f4e678acbe5b3e8d96925adacb
ffce2448f4ae4fc0375c006d873ee4e9f996ccfb
F20101112_AACOKS park_i_Page_21.tif
41f7b161107a271f3e3405c38a8ce250
3634483f8dcad61d99a18c21f76599a2099f4307
16279 F20101112_AACOZL park_i_Page_53.QC.jpg
889884a55393aff4722127a0769ee762
8194b523fc0d5c6612e1da715c11c33795985e00
F20101112_AACOUN park_i_Page_66.tif
b00356b9c181adb42eab6a186a320f9f
28896d11b8df4e5881bd3140d128a760c05f2931
9510 F20101112_AACOFW park_i_Page_02.jpg
90e0be3e32796eec2e94c6e8e9d2c5a2
88ab3624889762b7161de8cd3a3a04b346800e03
2095 F20101112_AACOPQ park_i_Page_01thm.jpg
c474c16f1f848679602441160f66245f
465499d6a7c4d07abb755ea44e9a5b9d9342919a
27088 F20101112_AACOKT park_i_Page_11.QC.jpg
3ee88f6c0b437413f1db29326017b835
5b687af6f8f060ff15a788a8a269847e998898be
4596 F20101112_AACOZM park_i_Page_53thm.jpg
436590f12f8295752373d39ac0aecb06
ad682a41971435073cf6ba55b34966ed583f4407
F20101112_AACOUO park_i_Page_68.tif
d38d78aebfb3464606f4fc31a7e524c5
48bf14e61fc534ec82183508003b56fc10c4a782
28225 F20101112_AACOFX park_i_Page_25.QC.jpg
32f5d576687fea203abee6962b165f01
9264056da38a8a9382d4bf3b83fca4d33610e72d
40950 F20101112_AACOPR park_i_Page_72.jpg
afdff6373b876f167aa24227e58d163f
a4672da6cb65c343d0408c66ef9e8adc2e2207f1
621 F20101112_AACOKU park_i_Page_02.pro
36a554f1d604f9143b2fc04b878cc27d
c50c63cc2548b273bfc3081d080135d02294f63b
20171 F20101112_AACOZN park_i_Page_55.QC.jpg
15dc6e9d666dc07f17fbec8b6a750998
dd2fab92a605c7987fb58bf68103c46f3a483da3
F20101112_AACOUP park_i_Page_70.tif
a8290d2a67c14f5bf57e3bc461ed4b3e
23354513357b4c5de003a264330d1729b3c3adb3
39824 F20101112_AACOFY park_i_Page_05.pro
e5b9b35792b33c8741fde811b6aaee4b
e75392e9f80fd15024e505be1459ebaa803a0233
114130 F20101112_AACOPS park_i_Page_07.jp2
a8256b7ac702f6dce3d5b0c4405bf7e9
5ecb7e883609912510de863666c6a57a17dd6886
F20101112_AACOKV park_i_Page_51.tif
828aae162a7bd8166085d31ea234c41e
109aed4748eb0e259fbcd6ba834048c00b281c90
5465 F20101112_AACOZO park_i_Page_55thm.jpg
fdb16cebb128b08407f424dee52e32ea
c962475374558121b7c85484e5b0d1d4cf48993a
F20101112_AACOUQ park_i_Page_71.tif
b6cd54672710848e4906c80421991e81
54f358a7522f88a2f551a7bc0317746ea6e3a757
4546 F20101112_AACOFZ park_i_Page_61.pro
04a48a020c22bf93871752b7fb5a56ba
14656636242a4ee0de0a1a12318ac51bd0e1c86b
2378 F20101112_AACOPT park_i_Page_45.txt
0cad6adff8ac6429c70ef86c62dbaab8
14022b2c54c6699670b6a56cc46bde0778765fe1
F20101112_AACOKW park_i_Page_64.tif
def60860a64d703cce5db2b4f75e6579
5e50cdca1e075f011784641ea25b061c90e9cad5
22101 F20101112_AACOZP park_i_Page_57.QC.jpg
c4d50a2ec3d3eb519b5e8641eaad715e
50a05007e96d1c445fd7a498c2b54538aca1953f
F20101112_AACOUR park_i_Page_73.tif
0b2d9b9a96a4e04512c45843def525f3
f1e8d75682ffb5434e2aed5906210ac5841b0446
6129 F20101112_AACOPU park_i_Page_17thm.jpg
431922142a38ee90d2a54738efa2530c
23b5630b128bb09dd3eb5ca722d30e0ba355fa6d
15433 F20101112_AACOKX park_i_Page_41.QC.jpg
79fb9b0c32195955db3ecbcbcdecdc9a
35bcf3c3c257dbf796eacb750e3c8fa596e60777
5995 F20101112_AACOZQ park_i_Page_57thm.jpg
9b638a30333af26493d591d9adfbbd46
e872700eacdde43e3da2124663c40c44572f386a
F20101112_AACOUS park_i_Page_76.tif
89ea647104421bcb6dee5a7e0f395b33
695030c012bec0c4a53d7a85836e7abb8b5afa35
4771 F20101112_AACOPV park_i_Page_28thm.jpg
700e30f395dd68d7c351df4c3287fb40
4275cf2d919bfabd01e185e11959684131867e59
F20101112_AACOIA park_i_Page_07.tif
e60864d7eddc9f2d167850950a72e96c
80ff2083f6a073a505bf8c51fa1695ca00d1ba7a
59715 F20101112_AACOKY park_i_Page_54.jpg
5f1b938d43d215765e11c228a245f0df
a395938225fe10d35dc296cd28def64f72149c84
4896 F20101112_AACOZR park_i_Page_58thm.jpg
2d1b8accc35ce07dc0f1b59aaa83d6eb
eafc231d628551ce62412f0cd9a113fbd6d62c37
F20101112_AACOUT park_i_Page_80.tif
5264f92890f972fbe6ac0fff5608d450
ffbbf8662936fd19ca1f53d344e316f539c05e6d
4719 F20101112_AACOPW park_i_Page_08thm.jpg
7f3779c9e04a06fb2966ae9ea507ce96
0d53952ce7451018f701a426adeea9e1aa2c74ad
73895 F20101112_AACOKZ park_i_Page_18.jpg
71b1d507d41cba8dc63765419e338cb6
fe5e64cbeccbe289aee9428f3bdbc47bf6fc9f71
7390 F20101112_AACOUU park_i_Page_01.pro
5abb0cbe4a6c514916d6ebfa5c1177f2
008b6b883a06c6370534d073e9827e25220ff77e
73683 F20101112_AACOPX park_i_Page_36.jp2
205a78b5355fb52092d10c26471b2fb9
b0a900ec9c61cdb8fd1f4cff6761782be2e185e9
1658 F20101112_AACOIB park_i_Page_08.txt
61add7adb2d73657436430caef568e6e
b559119a6c59bdf82fa480a3c49ce0c4c8523c90



PAGE 1

1

PAGE 2

2

PAGE 3

3

PAGE 4

IthankmyadviserDr.JoseC.Prncipeforallhisgreatguidance,mycommitteememberDr.JohnHarrisforinsightfulsuggestions,andDr.ThomasB.DeMarseforhisknowledgeandintuitiononexperiments.IthankmycollaboratorsAntonioR.C.PaivaandKarlDockendorfforallthejoyfuldiscussions.IalsothankDongmingXu(dynamics),Jian-WuXu(RKHS),VaibhavGarg,ManuRastogi,SavyasachiSingh(chess),AllenMartins(pdf),YiwenWangandAysegulGunduzofCNEL,JasonT.Winters,AlexJ.Cadotte,HanyElmariah(singing)andNickyGrimesoftheNeuralRoboticsandNeuralComputationLabfortheirsupportandhelp.Lastbutnotleast,Ithankmyfamilyandfriendsforbeingthere. 4

PAGE 5

page ACKNOWLEDGMENTS ................................. 4 LISTOFTABLES ..................................... 7 LISTOFFIGURES .................................... 8 ABSTRACT ........................................ 9 CHAPTER 1INTRODUCTION .................................. 10 1.1Motivation .................................... 10 1.1.1WhyDoWeAnalyzeSpikeTrains? .................. 11 1.1.2WhatAreSimilarSpikeTrains? .................... 12 1.2MinimalNotation ................................ 13 2CROSSINFORMATIONPOTENTIAL ...................... 14 2.1SmoothedSpikeTrainRepresentation ..................... 14 2.2L2Metric .................................... 14 2.3Cauchy-SchwarzDissimilarity ......................... 16 2.4InformationPotential .............................. 17 2.5Discussion .................................... 18 2.5.1ComparisonofDistances ........................ 18 2.5.2RobustnesstoJitterintheSpikeTimings ............... 20 3INSTANTANEOUSCROSSINFORMATIONPOTENTIAL ........... 22 3.1SynchronyDetectionProblem ......................... 22 3.2InstantaneousCIP ............................... 22 3.2.1DerivationfromCIP .......................... 22 3.2.2SpatialAveraging ............................ 23 3.2.3RescalingICIP .............................. 23 3.3Analysis ..................................... 24 3.3.1SensitivitytoNumberofNeurons ................... 24 3.4Results ...................................... 25 3.4.1High-orderSynchronizedSpikeTrains ................. 25 3.4.2Mirollo-StrogatzModel ......................... 27 5

PAGE 6

.................... 32 4.1DelayEstimationProblem ........................... 32 4.2ContinuousCorrelogram ............................ 34 4.3Algorithm .................................... 36 4.4Results ...................................... 39 4.4.1Analysis ................................. 42 4.4.2Examples ................................. 43 4.5Discussion .................................... 47 5CONCLUSION .................................... 49 5.1SummaryofContribution ........................... 49 5.2PotentialApplicationsandFutureWork ................... 49 APPENDIX ABACKGROUND ................................... 50 A.1PointProcess .................................. 50 A.1.1AnAlternativeRepresentationofPoissonProcess .......... 51 A.1.2FilteredPoissionProcess ........................ 52 A.2MeanSquareCalculus ............................. 53 A.3ProbabilityDensityEstimation ........................ 54 A.4InformationTheoreticLearning ........................ 56 A.5ReproducingKernelHilbertSpace ....................... 58 BSTATISTICALPROOFS .............................. 62 CNOTATION ...................................... 67 DSOURCECODE ................................... 69 D.1CIP ....................................... 69 D.2ICIP ....................................... 70 D.3CCC ....................................... 73 REFERENCES ....................................... 75 BIOGRAPHICALSKETCH ................................ 81 6

PAGE 7

Table page A-1Variousprobabilitydensityestimationkernels ................... 56 7

PAGE 8

Figure page 2-1L2distanceversusCSdivergence .......................... 18 2-2DistancedierenceofCSdivergenceforasynchronizedoruncorrelatedmissingspike .......................................... 19 2-3ChangeinCIPversusjitterstandarddeviationinthesynchronousspiketimings 20 3-1Spiketrainasarealizationofpointprocessandsmoothedspiketrain ...... 22 3-2VarianceinscaledCIPversusthenumberofspiketrainsusedforspatialaveraginginlogscale. ...................................... 24 3-3AnalysisofICIPasafunctionofsynchrony .................... 26 3-4Evolutionofsynchronyinthespikingneuralnetwork ............... 28 3-5Zero-lagcross-correlationforcomparison ...................... 29 4-1Exampleofcrosscorrelogramconstruction ..................... 33 4-2DecompositionandshiftofthemultisetA. ..................... 36 4-3Eectofthelengthofspiketrainandstrengthofconnectivityonprecisionofdelayestimation ................................... 40 4-4Eectofkernelsize(binsize)ofCCC(CCH)totheperformance ........ 41 4-5Schematicdiagramforthecongurationofneurons. ................ 43 4-6ComparisonbetweenCCCandCCHonsynthesizeddata. ............. 44 4-7EectoflengthofspiketrainsonCCCandCCH ................. 45 4-8Correlogramsforinvitrodata ............................ 46 8

PAGE 9

9

PAGE 10

1 ],whichisrelatedtoanon-EuclideanmetricproposedbyVictorandcoworkerswhichisanextensionoftheLevenshteindistance(alsoknownaseditdistanceincomputerscience)tocontinuoustime[ 2 ].ManyneuroscientistswerealreadyusingthevanRossumdistancebyintuitionintheformofcorrelation[ 3 { 6 ].WemappedthespiketrainstoarealizationofarandomprocessinL2,sothattraditionalsignalprocessingtechniquescanbereadilyapplied.Wewillanalyze 10

PAGE 11

7 ].Actionpotentialsaregenerationbythecomplexdynamicsofaneuron[ 8 9 ],andhasastereotypicalshapewhichcanbepropagatedthroughalongdistanceandcanresistnoisebecauseofitsall-or-nonetypeoftransmission.Therehavebeenevidencethatnotonlytheexistenceofanactionpotentialscarriesinformation,butthedurationoftheactionpotentialissystematicallymodulated[ 10 ],andrecentlyevensubthresholddendriticinputcanmodulatesynapticterminals[ 11 ].However,fromthecomputationalpointofview,itisbelievedthatthetemporalstructureoftheactionpotentialsismoreimportantthanindividualdetailsofanactionpotential.Experimentsmainlyinsensoryencodingdemonstratesprecisetiming(orprecisetimetorstspike)ofactionpotentials([ 12 { 14 ],see[ 15 ]forareview,and[ 16 ]forargumentsagainstit)whichsupportstheideaofencodinginformationonspiketimes.Theprecisionofspiketimingsislessthan100sinauditorysystem[ 17 ]andintheorderof1msinotherexperiments[ 14 ]. 11

PAGE 12

18 ],connectivityestimation[ 19 20 ],delayestimation[ 21 ],systemidentication[ 22 ],clusteringdierentspikepatterns[ 4 23 ],estimatingentropy[ 24 { 27 ],andneuraldecoding[ 28 29 ].Wewilltacklesomeoftheseproblemswiththeproposedtechniques. 1.1.1 ,thespiketimesproducedbyneuronsinresponsetorepeatedstimulusoftenshowsprecisetimingwithsomeerror.ThejittererrordistributiontswithaGaussiandistribution[ 13 ].Thepossiblenoisesourcesarethermalnoise,ionchannels,probabilisticsynapseactivation,spontaneousreleaseofvesicles.WhenthespiketrainismodeledbyaPoissonprocess,thejitternoiserestrictstheshapeoftheintensityfunction(instantaneousringrate)overtime.Inotherwords,thenoisewilllimitthenarrownessofapreciselytimedspike.Inaddition,thisimplicatesthatthespiketrainswithsmalltimingdierencesshouldbetreatedassimilartoeachother,thushavingasmalldistance(ordissimilarity 12

PAGE 13

30 ].Anothertypeofnoiseinspiketrainsisinsertionordeletionofspikes.Althoughspiketrainsofneuronsconservehighprecisionofspiketimingswhentheyoccur,thereisevidencethatneuronsoftenskipafewspikes[ 4 31 32 ].Whenaspikeisinsertedorremovedfromaspiketrain,thedistancediersbytheconstant1 2invanRossumdistance.Incontrast,acorrelationmeasuredoesnotdependonsignalpower(ornumberofactionpotentials),butonlyonthecoincidentalactionpotentialpairs.Inapplications,suchasclassicationofspiketrainswithtemplatematching,thecorrelationbaseddistancemeasure(Cauchy-Schwarzdivergence)canperformbetterthanvanRossum(L2)distance.Theconceptofcoincidentalspikesleadstosynchronybetweenspiketrains.Inaddition,therearestrongevidencesthatneuronsanddendritesworkasacoincidencedetectorandsensitivetoaerentsynchrony[ 26 33 { 36 ]. 13

PAGE 14

^i(t)=NiXm=1pdf(ttim):(2{1)Thisprocesscanalsobeviewedaslowpasslteringofthespiketrainstoestimatethepostsynapticpotentialofsynapses.Inthepointprocessliterature,thisisaspecialcaseoflteredpointprocess,andintheengineeringliteratureknownasshotnoise. 2{1 )convertsaspiketraintoacontinuoussignalthatcanbeinterpretedwiththesecondordertheorywithacontinuousmetric.Notethatthemappingisone-to-oneandonto:deconvolutionof^i(t)withpdfuniquelydeterminesaspiketrain. 3 ). 14

PAGE 15

2{1 ), (2{3b)andthecrossterm(innerproductinL2)becomes,Z1^i(t)^j(t)dt=NiXm=1NjXn=1(timtjn) (2{3c)where(t)=R1pdf(s)pdf(s+t)ds.isthekernelwhichcomputesthecorrelation.Ifanexponentialdistributionisused,i.e., u(t);(2{4)whereu(t)istheunitstepfunction,thentheL2distanceisproportionaltovanRossumdistancewithfactor1 2expjtimtjnj 15

PAGE 16

6 ):jhxjyijkxkkyk:Sinceeachquantityispositiveifxandyarenotzerovectors,andequalityholdswheneitherofthemarezero,wecandividebothsides,1kxkkyk jhxjyij:Bytakingthelogarithm,0logkxkkyk jhxjyij1:Itcanbeprovedthatthisquantityispositive,reexive,andsymmetric[ 37 ]ifweexclude0fromthespace.However,CSdivergencedoesnotholdthetriangularinequality,thusitisnotametric.ByexpandingthedenitionofinnerproductandnormofL2space, R1^2i(t)dtR1^2j(t)dt PNim=1PNin=1(timtin)PNjm=1PNjn=1(tjmtjn) 16

PAGE 17

A{8 ), 2{10c )canbeapproximatedasaconstant.However,dependingonthecorrelationofthespiketrains,thesecondtermwillvary.Sincethenegativelogarithmisamonotonicallydecreasingfunction,wetaketheargument,denoteasVij,anddeneascrossinformationpotentialforreasonsthatwouldbeexplainedinsection 2.4 A.4 forasummaryoftheinformationtheoreticlearningframework).InhomogeneousPoissonprocesscanberepresentedastwoseparaterandomvariables:oneforthenumberofspikesandtheotherforthetemporaldensity(seesection A.1.1 ).Thepdfforthetemporaldensityissimplyanormalizedformoftheintensityfunction(equation( A{2 )).Thispdfdoesnothavetheinformationofhowactivetheprocessis,thatis,theringrate.InformationpotentialofdensityfunctionestimatedusingParzenwindowwithpdfforthei-thspiketrainhasthefollowingform(compareequation( A{16 )), 2{3b ),normalizedbythe 17

PAGE 18

numberofspikes.Forapairofspiketrains,thecrossinformationpotentialcanbedenedasasimilarityindexbetweenthecorrespondingpairofpdfs.NotethatintermsofCSdivergence,thenormalizationwiththenumberofspikesinthespiketraincancelsaway. 2.5.1ComparisonofDistancesAsmentionedearlierinsection 1.1.2 ,althoughneuronsrewithhightemporalprecision,theyoftenmissspikes.Inthiscase,L2distancewoulddeviatebecauseofthemissingspike.CSdivergencewouldbelesssensitivebecauseitwillignoremissingspikes.Todemonstratethis,asimpleclassicationtaskwasperformed(seegure. 2-1 ).Twotemplatespiketrainswereprepared:template1with2spikesat3msand8ms,andtemplate2with1spikeat6ms.Then,wegeneratedinstancesoftemplate1byputtingGaussianjitterontiming(bluecircles)andremovingaspike(reddots).Forthenomissingspikecase,bothL2(94%)andCSdivergence(100%)correctlyclassiedtheinstanceastemplate1(theylieontheupperhalf).Butformissingspikes 18

PAGE 19

IncreaseordecreaseinCauchy-Schwarz(CS)divergence(dissimilarity)whenaspikeismissing.(Left)Whenacorrelated(perfectlysynchronizedinthiscase)spikeismissing,thedivergencedecreaseinverselyrelatedtothetotalnumberofspikes.(Right)Butifacorrelated(synchronized)spikeismissing,thedivergenceincreasesproportionaltothetotalnumberofsynchronizedspikes,andnotgreatlyinuencebythetotalnumberofspikes.Incontrast,L2distancetheincreaseanddecreaseareconstant(seetextfordetails). case,L2distance(51%)performedalotworsethanCSdivergence(93%).TheCSdivergenceshowslineswhenonespikeismissingbecausethedistance(quantiedasthedivergence)isalogofthekernelwhichisasingleLaplacian.Supposeindividualspikesareseparatedcomparedtothekernelsizeorexactlysynchronizedsothatwecanapproximatethenormandinnerproductbythenumberofspikes:normsquareofaspiketrainisthenumberofspikes,andinnerproductgivesthenumberofsynchronizedspikes.Thisisequivalenttomakingthekernelsizeinnitelysmall,sothatitconvergestoaDiracdeltafunction.LettherebetwospiketrainsAandT(fortemplate)withNAandNTnumberofspikesrespectively,andNATsynchronizedspikes.TheL2distancebetweenAandTisNA+NT2NAT,andtheCSdivergenceislogNANT 2invanRossumdistance)andforCS 19

PAGE 20

ChangeinCIPversusjitterstandarddeviationinthesynchronousspiketimings.Forthecasewithindependentspiketrains,theerrorbarsforonestandarddeviationarealsoshown.Thekernelsizeis2ms(left)and5ms(right). divergencethedecreaseis,logNANT (2{14)=logNT 2-2 foranillustrativeexample. 38 39 ]wherejitter,modeledas 20

PAGE 21

38 ].Adierentinterpretationfor"isthat,givenaspikeinaspiketrain,itquantiestheprobabilityofaspikeco-occurrenceinanotherspiketrain.Theeectwasthenstudiedintermsofthesynchronylevelandkernelsize.Figure 2-3 showstheaverageCIPfor10MonteCarlorunsoftwospiketrains,10secondslong,andwithconstantringrateof20spikes/s.Inthesimulation,thesynchronylevelwasvariedbetween0(independent)to0.5forakernelsizeof2msand5ms.Thejitterstandarddeviationvariedbetweentheidealcase(no-jitter)to15ms.Asmentionedearlier,CIPmeasuresthecoincidenceofthespiketimings.Asaconsequence,thepresenceofjitterinthespiketimingsdecreasestheexpectedvaluesofCIP(andtimeaveragedICIP).Nevertheless,theresultsinFig. 2-3 supportthestatementthatthemeasureisindeedrobusttolargelevelsofjittercomparedtothekernelsize,andiscapableofdetectingtheexistenceofsynchronyamongneurons.Ofcourse,increasingthekernelsizedecreasesthesensitivityofthemeasureforthesameamountofjitter.Furthermore,asinthepreviousexample,itisalsoshownthatsmalllevelsofsynchronycanbediscriminatedfromtheindependentcaseassuggestedbytheerrorbarsinFigure 2-3 .Finally,weremarkthatthedierenceinscalebetweentheguresisaconsequenceofthenormalizationofthekernelsothatitisavalidpdf.ThiscanbecompensatedexplicitlybyscalingtheCIPby.SimplynotethattheexpressionsprovidedinthepreviousexampleformeanICIP(andthereforeCIP)asafunctionofthesynchronylevelimplicitlycompensatefor. 21

PAGE 22

40 ],neuralcoding[ 31 41 ],neuralassemblies[ 3 ],bindingproblem[ 42 ],andtopulsecoupledoscillators[ 43 { 47 ].Analysisofsynchronyhasreliedonvariousmethods,suchasthecross-correlation[ 48 ],jointperi-stimulustimehistogram(JPSTH)[ 49 ],unitaryevents[ 50 ],andgravitytransform[ 3 ],amongmanyothers.SinceCIP(orCSdivergence)characterizesthesimilarity(ordissimilarity)ofspiketrainswithcorrelationofspiketimes,CIPcanalsobeusedasasynchronymeasure.However,CIPdoesnotprovideinformationaboutinstantaneoussynchrony.Aslidingwindowapproachcanbeusedwithsacriceofthetemporalresolution,asincrosscorrelationandgravitytransform. 3.2.1DerivationfromCIPLetusbreaktheintegralrangefromthedenitionofL2innerproduct(equation( 2{3c )). Figure3-1. Spiketrainasarealizationofpointprocessandsmoothedspiketrain.(a)Spiketrainofneuronirepresentedinthetimedomainasasequenceofimpulsesand(b)itslteredcounterpartusingacausaldecayingexponential. 22

PAGE 23

3.3.1 51 52 ]bystretchingthetimeaccordingtotheintensityfunction.Transformationofequation( 2{1 )intoaconstantringratetimescalefordierentspiketrainsdependsonindividualintensityfunction,andthereforethetransformedresultsarenotsynchronous.Thus,inordertoquantifysynchrony,the 23

PAGE 24

VarianceinscaledCIPversusthenumberofspiketrainsusedforspatialaveraginginlogscale.Theanalysiswasperformedfordierentlevelsofsynchronyandconstant=2ms(left),anddierentvaluesoftheexponentialdecayparameteronindependentspiketrains(right).InbothplotsthetheoreticalvalueofCIPforindependentspiketrainsisshown(dashedline). correlationoperationshouldbeperformedintheoriginaltimes,butwiththesmoothinginthetransformedspace.Therstorderapproximationofthiscanbeachievedbyredeningtheintensityestimatoras ^i(t)=1 3.3.1SensitivitytoNumberofNeuronsWenowanalyzetheeectofthenumberofspiketrainsusedforspatialaveraging.Thiseectwasstudiedwithrespecttotwomainfactors:thesynchronylevelofthespike 24

PAGE 25

3-2 forthescaledCIPspatiallyaveragedoverallpaircombinationsofneurons.Thesimulationwasrepeatedfor200MonteCarlorunsusing10secondlongspiketrainsobtainedashomogeneousPoissonprocesseswithringrateof20spikes/s.Asillustratedinthegure,thevarianceinCIPdecreasesdramaticallywiththeincreaseinthenumberofspiketrainsemployedintheanalysis.RecallthatthenumberofpaircombinationsoverwhichtheaveragingisperformedincreaseswithM(M1),whereMisthenumberofspiketrains.Asexpected,thisimprovementismostpronouncedinthecaseofindependentspikestrains.Inthissituation,thevariancedecreasesproportionallytothenumberofaveragedpairsofspiketrains.ThisisshownbythedashedlineintheplotsofFig. 3-2 .TheseresultssupporttheroleandimportanceofensembleaveragingasaprincipledmethodtoreducethevarianceoftheCIPestimator. 3.4.1High-orderSynchronizedSpikeTrainsFigure 3-3 showsICIPofdierentlevelsofsynchronyovertenspiketrains.ThesynchronywasgeneratedbyusingtheMIPmodel,andmodulatedovertimefor1secondsoftimedurations.Theringrateofthegeneratedspiketrainswasconstantandequalto20spikes/sforallspiketrains.ThegureshowstheICIPaveragedforeachtimeinstantoverallpaircombinationsofspiketrains.Becausethespiketrainshaveconstantringrate,thetimeconstantofthedecayingexponentialconvolvedwiththespiketrainswasconstantandchosentobe=2ms.Also,inthebottomplottheaveragevalueofthemeanICIPisshown.Thiswascomputedin25msstepswithacausal250mslongslidingwindow.Toestablisharelevanceofthevaluesmeasured,theexpectationandthisvalueplustwostandarddeviationsarealsoshown,assumingindependencebetweenspiketrains.Themeanandstandarddeviation,assumingindependence,are1andq 1 2+121,respectively(seeAppendixfordetails).TheexpectedvalueoftheICIPwhensynchrony 25

PAGE 26

AnalysisofICIPasafunctionofsynchrony.(Top)Levelonsynchronyspeciedinthesimulationofthespiketrains.(Uppermiddle)Rasterplotofrings.(Lowermiddle)AverageICIPacrossallneuronpaircombinations.(Bottom)TimeaverageofICIPintheupperplotcomputedinstepsof25mswithacausalrectangularwindow250mslong(darkgray).Forreference,itisalsodisplayedtheexpectedvalue(dashedline)andthisvalueplustwostandarddeviations(dottedline)forindependentneurons,togetherwiththeexpectedvalueduringmomentsofsynchronousactivity(thicklightgrayline),asobtainedanalyticallyfromthelevelofsynchronyusedinthegenerationofthedataset.Furthermore,themeanandstandarddeviationoftheensembleaveragedCIPscaledbyTmeasuredfromdatainonesecondintervalsisalsoshown(black). 26

PAGE 27

43 ] 3-4 ).Thesynchronizationisessentiallyduetoleakinessandtheweakglobalcouplingamongtheoscillatoryneurons.TherasterplotofthenetworkringpatternisshowninFig. 3-4 .Therearetwomainobservations:theprogressivesynchronizationoftheringsassociatedwiththeglobaloscillatorybehaviorofthenetwork,andthelocalgroupingthattendstopreservelocalsynchronizationsthateitherentrainthefullnetworkorwashoutovertime.Asexpectedfromtheoreticalstudiesofthenetworkbehavior[ 43 46 ]andwhichICIPdepictsprecisely,thesynchronizationismonotonicallyincreasing,withaperiodoffastincreaseintherstsecondfollowedbyaplateauandslowerincreaseastimeadvances.Moreover,itispossible 27

PAGE 28

Evolutionofsynchronyinthespikingneuralnetwork.(Top)Rasterplotoftheneuronrings.(Middle)ICIPovertime.Theinsethighlightsthemergingoftwosynchronousgroups.(Bottom)Informationpotentialofthemembranepotentials.Thisisamacroscopicvariabledescribingthesynchronyintheneurons'internalstate. 28

PAGE 29

Zero-lagcross-correlationcomputedovertimeusingaslidingwindow10binslong,andbinsize1ms(top)and1.1ms(bottom). 29

PAGE 30

A{18 )usingaGaussiankernelwithsize0.75mV. 3-5 wealsopresentthezero-lagcross-correlationovertime,averagedthroughallpairwisecombinationsofneurons.Thecross-correlationwascomputedwithaslidingwindow10binslong,sliding1binatatime.Inthegure,theresultisshownforabinsizeof1msand1.1ms.Itisnotablethatalthoughcross-correlationcapturesthegeneraltrendsofsynchrony,itmaskstheplateauandthenalsynchronyanditishighlysensitivetothebinsizeasshowninthegure,unlikeICIP(datanotshown).Inotherwords,theresultsforthewindowedcross-correlationshowtheimportanceofworkingin\continuous"timewhichiscrucialforrobustsynchronyestimationinthespikedomain.Othermethodsrelyingonbinningalsosuerfromsensitivitytobinsize,suchastheonesmentionedearlier.Forthisreason,thesemethodsarelimitedandunabletoachievethesamehightemporalresolutionasICIP.Inaddition,spiketrainsaregenerallynon-stationaryunlikesomemethodsassume.Theconventionalapproachistouseamovingwindowanalysissuchthatonlypiece-wise 30

PAGE 31

31

PAGE 32

15 53 ].Forexample,itiscrucialforcoincidencedetectionofauditorysignalprocessing[ 17 ].Oneoftheeectivemethodsforestimatingthedelayistouseacrosscorrelogram[ 54 ].Crosscorrelogramisabasictooltoanalyzethetemporalstructureofsignals.Itiswidelyappliedinneurosciencetoassessoscillation,propagationdelay,eectiveconnectionstrength,andspatiotemporalstructureofanetwork[ 28 ].However,estimatingthecrosscorrelationofspiketrainsisnon-trivialsincetheyarepointprocesses,thusthesignalsdonothaveamplitudebutonlytimeinstanceswhenthespikesoccur.Awellknownalgorithmforestimatingthecorrelogramfrompointprocessesinvolveshistogramconstructionwithtimeintervalbins[ 48 ].Thebinningprocessiseectivelytransformingtheuncertaintyintimetoamplitudevariability.Thisquantizationoftimeintroducesbinningerrorandleadstocoarsetimeresolution.Furthermore,thecorrelogramdoesnottakeadvantageofthehighertemporalresolutionofthespiketimesprovidedbycurrentrecordingmethods.Thiscanbeimprovedbyusingsmoothingkernelstoestimatethecrosscorrelationfunctionfromnitesamples.Theresultingcrosscorrelogramiscontinuousandprovideshightemporalresolutionintheregionwherethereisapeak(seeFig. 4-1 forcomparisonbetweenhistogrammethodandkernelmethod.)Inthispaper,weproposeanecientalgorithmforestimatingthecontinuouscorrelogramofspiketrainswithouttimebinning.Thecontinuoustimeresolutionisachievedbycomputingatnitetimelagswherethecontinuouscrosscorrelogramcanhavealocalmaximum.ThetimecomplexityoftheproposedalgorithmisO(TlogT)onaveragewhereTisthedurationofspiketrains.Theapplicationoftheproposedalgorithmisnotrestrictedtosimultaneouslyrecordedspiketrains,butalsotoPSTHandalsootherpointprocessesingeneral. 32

PAGE 33

Exampleofcrosscorrelogramconstruction.AandCaretwospiketrainseachwith4spikes.ExceptforthethirdspikeinA,eachspikeinAinvokesaspikeinCwithsomesmalldelayaround10ms.Brepresentsallthepositive(black)andnegative(gray)timedierencesbetweenthespiketrains.DshowsthepositionofdelaysobtainedinB.EisthehistogramofD,whichistheconventionalcrosscorrelogramwithbinsizeof100ms.FshowsthecontinuouscrosscorrelogramwithLaplaciankernel(solid)andGaussiankernel(dotted)withbandwidth40ms.NotethattheLaplaciankernelismoresensitivetotheexactdelay. 33

PAGE 34

55 ].Inaphysiologicalcontext,thereisaphysicalrestrictionofpropagationdelayforanactionpotentialtohaveacausalinuencetoinvokeanyotheractionpotential.Therefore,thisdelaywouldinuencethecrosscorrelogramasaformofincreasedamplitude.Thus,estimatingthedelayinvolvesndingthelagatwhichthereisamaximuminthecrosscorrelogram(inhibitoryinteractionwhichappearastroughsratherthanpeaksisnotconsideredinthisarticle).Smoothingapointprocessissuperiortothehistogrammethodfortheestimationoftheintensityfunction[ 30 ],andespeciallythemaxima[ 56 ].Similarly,thecrosscorrelationfunctioncanalsobeestimatedbetterwithsmoothingwhichisdoneincontinuoustimesowedonotlosetheexacttimeofspikeswhileenablinginteractionbetweenspikesatadistance.Insteadofsmoothingthehistogramoftimedierencesbetweentwospiketrains,werstsmooththespiketraintoobtainacontinuoussignal[ 57 ].Wewillshowthatthisisequivalenttosmoothingthetimedierenceswithadierentkernel.Acausalexponentialdecaywaschosenasthesmoothingkerneltoachievecomputationaleciency 34

PAGE 35

4.3 ).Smoothedspiketrainsarerepresentedas, u(ttim);(4{3)whereu(t)istheunitstepfunction.Thecrosscorrelationfunctionofthesmoothedspiketrainsis, 4{4 )canbeestimatedfromsamplesas, ^Qij(t)=1 ^Qij(t)=1 2TNiXm=1NjXn=1ejtimtjntj 58 59 ]fromtimedierencesusingaLaplaciandistributionkernel.ThemeanandvarianceoftheestimatorisanalyzedbyassumingthespiketrainsarerealizationsoftwoindependenthomogeneousPoissonprocesses.Eh^Qij(t)i'AB; 35

PAGE 36

DecompositionandshiftofthemultisetA. comparison:~Qij(t)=p 36

PAGE 37

4{10 )canberewrittenas 4{12 )canalsoberewrittenanddecomposed, +X2(At)e : : 4-2 ).Inotherwords,ift=n+1,forasmallchange2[0;n+1n),themultisetsdonotchangetheirmembership,i.e.((At))=(A(t)).Therefore,wecansimplifyanarbitraryshiftofQijwithsinglemultiplicationofanexponentialas, =Xt2((At))et =Xt2(At)et e =Qij(t)e : 37

PAGE 38

(4{17a)=Q+ij(t)e +Qij(t)e : +Qij(t)e 0;(4{19)Qij(t)isaconvexfunctionofwithintherange.Thus,themaximumofthefunctionvalueisalwaysoneithersideofitsvalidrange,onlylocalminimumcanbeinbetween.Inprinciple,weneedtocomputeequation( 4{10 )forallt2[T;T]toachievecontinuousresolution,whereTisthemaximumtimelagofinterest.However,ifweonlywantalllocalminimaandmaxima,wejustneedtoevaluateonallt2A,andcomputetheminimaandmaximausingequation( 4{17b )andequation( 4{18 ).Therefore,ifwecomputetheQij(n)foralln2A,wecancomputeforallintervals(n;n+1]ifalocalextremumexists.Thesecanbecomputedusingthefollowingrecursiveformulae. +1; 1: :(4{21) 38

PAGE 39

1 4{10 )isO(NiNj)foreachtimelagt.AssuminghomogeneousPoissonprocessforindividualspiketrains,theaveragetimecomplexitybecomesO(NlogN)whereN=ABT,Tisthelengthofspiketrain,andArepresentstheaverageringrateforthePoissonprocess.Notethattheconventionalcrosscorrelogramalgorithm[ 48 ]hasthetimecomplexityofO(N)onaverage. 4{9 ) 39

PAGE 40

(b)Eectofconnectionstrength Eectofthelengthofspiketrainandstrengthofconnectivityonprecisionofdelayestimation.Theprecisionisestimatedbythestandarddeviationin1000MonteCarlorunswithkernelsize=0:4ms(orbinsizeh=1:96ms).Thesmallerstandarddeviationindicateshighertemporalresolution. betweenthepossiblemaxima(butnottheminima).InordertocomparewithCCC,CCHisstandardizedinasimilarwaytoequation( 4{9 )accordingto[ 60 ].SinceCCHisessentiallyequivalenttousingauniformdistributionkernel(oraboxcarkernel)andsamplingatequallyspacedintervalsasopposedtotheLaplaciandistributionkernelusedinCCC,inordertomakeafaircomparison,wechoosethekernelsize(binsize)ofbothdistributionstohavethesamestandarddeviation.Tobespecic,ifthetimebinsizeofCCHish,thenwecomparetheresulttoCCCwithkernelsizeof=h 61 ].ThemethodisdesignedfortheestimationofringrateorPSTHfromameasurementassumingaPoissonprocess.However,sincethetimedierencebetweentwoPoissonprocessesofnitelengthcanbeconsideredasarealizationofaPoissonprocess,itispossibletodirectlyapplytotheCCH. 40

PAGE 41

(b)OptimalforCCC Eectofkernelsize(binsize)ofCCC(CCH)totheperformance.Theconnectionstrengthwas5%andthespiketrainsare10secondslong,i.e.5spikesarecorrelatedonaverage.(a)SensitivityofCCCandCCHonkernelsizefornoisestandarddeviation0.25ms.Thehorizontaldottedlineindicatestheperformancewhenoptimalbinsizeischosenforeachsetofsimulatedspiketimedierences.Themedianoftheoptimalbinsizechosen(right)andcorrespondingkernelsizeforCCC(left)areplottedasverticaldashedlines.NotethatCCCisrobustonkernelsizeselectionandperformsbetterthanCCH.(b)Fordierentstandarddeviationsofjitternoises,theprecisionisplottedversusthekernelsize.Notethattheoptimalkernelsizeincreasesasthejittervarianceincreases.Foreachpoint,3000MonteCarlorunsareused,andtheactualdelayisuniformlydistributedfrom3msto4mstoreducethebiasofCCH. 41

PAGE 42

62 ]eectthelocation,signicanceandwidthofthecrosscorrelogrampeak.Furthermore,iftheneuronsareinanetwork,multiplepaths,commoninputsources,recurrentfeedbackandlocaleldpotentialuctuationcaninuencethecrosscorrelogram.Inthissection,wemodelthetimingjitterwithaGaussiandistributionandanalyzethestatisticalpropertiesofCCCandCCHontimedelayestimation.ApairofPoissonspiketrainsofringrate10spikes/swerecorrelatedbycopyingaportionofthespikesfromonetoanotherandthenshiftingbythedelaywiththeGaussianjitternoise.Thefractionofspikescopiedrepresentstheeectivesynapticconnectivity.Thetotalnumberofcorrelatedspikesdependontwofactors:thelengthofspiketrain,andthesynapticconnectivity.Ingure 4-3 ,theprecisionofCCCandCCHarecomparedaccordingtothesefactors.Theprecisionisdenedtobethestandarddeviationoftheerrorinestimatingtheexactdelay.PrecisionofbothCCCandCCHimprovesasthenumbercorrelatedspikesincreasesinasimilartrend.CCCconvergestoaprecisionlowerthanhalfthejitternoisestandarddeviation(500s).Theoptimalkernelsize(orbinsize)whichgivesthebestprecisiondependsonthenoisejitterlevel.Ingure 4-4(a) ,CCCandCCHiscomparedacrossdierentkernelsizes.Ingeneral,CCCperformsbetterthanoptimalbinsizeandmostofthebinsizesCCH.Asmentionedabove,CCHissensitivetobinsize,butCCCisrobusttothekernelsizeforprecisionperformance.AlsonotethattheoptimalkernelsizeforCCCcorrespondstoequalmedianvalueofthevarianceoptimalbinsizeselected(verticaldashlines). 42

PAGE 43

I12.3ms13.7msB4.3ms9.3ms Figure4-5. Schematicdiagramforthecongurationofneurons. IncreasingthejitterlevelworsensthebestprecisionandincreasestheoptimalkernelsizeforCCCasshowninFig. 4-4(b) 4-5 .Individualsynapsesarestatic(noshort/long-termplasticity),withequalweightsandgenerateEPSP(excitatorypostsynapticpotential)withatimeconstantof1ms.EachneuronisinjectedwithpositivelybiasedGaussianwhitenoisecurrent,sothattheywouldrewithmeanringrateof35spikes/s.Thesimulationstepsizeis0.1ms.Asshowningure 4-6 ,bothCCHandCCCidentiesthedelaysimposedbytheconductiondelay,synapticdelay,andthedelayforthegenerationofactionpotentialbynoisyuctuationofmembranepotential.However,thetimelagidentiedbyCCCismoreaccuratethanthatofCCH,sincethetemporalprecisionprovidedbyCCHislimitedbythebinsizeandthejitternoiseondelay,butforCCC,itisonlylimitedbythejitter.Inotherwords,ifthereisnojitter,orasucientamountofspiketimingshastheexactdelay,thenCCCiscapableofquantifyingthedelaywithinniteresolution. 43

PAGE 44

ComparisonbetweenCCCandCCHonsynthesizeddata. 44

PAGE 45

Eectoflengthofspiketrains.Comparisonofcontinuouscrosscorrelogram(left)andcrosscorrelationhistogram(right)withdierentlengthofspiketrains(2.5,5,10seconds).Estimatedoptimalbinsizeis0.267ms. Ingure 4-7 ,weillustratethedierenceinperformanceofthemethodsaccordingtothelengthofthespiketrains.Whenthespiketrainsareonlyoflength2.5seconds,theCCChassignicantlylowertimeresolutionwherenospikeshadthattimedierence,yetmaintainingthehighresolutioninhighlycorrelatedpeaks.Incontrast,theCCHisuniformlysampledregardlessoftheamountofdata.Thenon-uniformsamplinggivessignicantadvantagetoCCCwhenonlyashortsegmentofdataisavailable.Totestthemethodfurther,spiketrainsrecordedinvitrowereused.WerecordedelectricalactivityfromdissociatedE-18ratcortexculturedona60channelmicroelectrodearrayfromMultiChannelSystems[ 63 ].Foraparticularpairofelectrodes,specicdelayswereobservedasshowninFig. 4-8 .Thosedelaysarerarelyobserved(3to5timesthrough5to10minutesofrecording),howevertheprecisionislessthan2mswhichmakesitsignicantinCCC.Thedelayspersistedatleast2days,andmanymoreinteractiondelayswereobservableastheculturematured.AsobservableintheCCHanalysis,itisalmostimpossibletodetectthedelaysandtheirconsistency. 45

PAGE 46

CCC(top)andCCH(bottom)of7DIV(daysinvitro)and9DIVcorticalculturerecordings.Spiketrainsfromtwoadjacentelectrodesareanalyzed.On7DIV,CCCshowstwosignicantpeaksandtheyarealsoobservableon9DIV,andsomenon-signicantspiketimedierencescorrespondstopeakson9DIV(markedwitharrows).Incontrast,CCHthisstructureisdiculttonote.Theoptimalbinsizeis3.8msfor7DIVand3.3msfor9DIVdata.Thetotalrecordingtimeis350secondsfor7DIVand625secondsfor9DIV. 46

PAGE 47

64 ].Onepossiblemechanismwouldbeararelyactivatedchainofsynapticpathwayfromacommonsourceneuronwithdierentdelays.Incontrasttoarecentstudyby[ 65 ]wherethedelaybetweentwochannelsisestimatedwithasingleapproximatedGaussiandistributionwithrelativelylargevariance,weobservemultipledelaysbetweenchannels. 30 ].Theonlyfreeparameteristhekernelsizewhichdeterminestheamountofsmoothing.Unliketheconventionallyusedhistogrammethod,theproposedmethodisrobustonkernelsize,however,theoptimalkernelsizedependsonthenoiselevelofthedelay.Inabiologicalneuronalnetwork,thenoiselevelmaydependonwhichpaththesignalwastransmitted.Thereforeeachpeakofthecorrelogrammayhavedierentamountofnoise.Wesuggestedtheusetheoptimalbinsizeforhistogramasaguidelineforthekernelsizeselection.Thecontinuouscrosscorrelogramcanbeviewedasageneralizationofthecrossinformationpotentialwherethecorrelationisinterpretedassimilarity(ordissimilarity)betweenspiketrainsaswediscussedinchapter 2 .Theproposedalgorithmcanbeused 47

PAGE 48

4{21 )).Thispreventsthealgorithmtobeusedasanonlineltertodetectcertainspiketrainpatterns,whileoineanalysiscanstillbedone.Theproposedalgorithmisnotlimitedtocross-correlations.Itcanbedirectlyappliedtosmoothanytypeofpointprocesseshistogram,suchasPSTH.However,onealwayshastobecautiouswhentheunderlyingprocessishighlynon-stationary.Variousnon-stationaritiescancausepeaksinthecorrelogram[ 66 ]. 48

PAGE 49

67 ]andpolychronousgroupofneurons[ 34 ]mayalsobepossiblebyusingCCC. 49

PAGE 50

51 ]). 51 ] 50

PAGE 51

1. 2. fort0s
PAGE 52

69 ]: 51 52 ]. 51 ]).

PAGE 53

51 ]page219-220forproof.Wecanchoosetheformof()tobe, 70 { 72 ],however,theclosedformishighlycomplicated.ThefollowingtheoremsupportsthatthecorrelationfunctionofthesmoothedspiketrainismeaningfulundertheassumptionofPoissonspiketrains. 51 ]). 51 ]. 53

PAGE 54

73 ].Inthisspace,theorderoflimitandexpectedvalueoperatorcanbeexchangeduptosecondorder. 74 ].Parametricmethodsassumesadistributionandtsthedatatothedistribution,whichisusableonlyiftheassumedmodelisatleastapproximatelycorrect.Ontheotherhand,nonparametricapproachmakesamilderassumption,usuallyintheformthatthepdfiscontinuous.Oneofthewidelyusednonparametricmethodisthehistogram.TheotherisParzenwindow,orotherwiseknownaskerneldensityestimation[ 59 ].Thesecanbemotivatedfromthe 54

PAGE 55

74 ]: ^F(x)=numberofsamplesin(xh;x+h) totalnumberofsamples:(A{11)Plugginginequation( A{11 )tothedenitionofpdf, 2h;(A{12)canbewritteninthefollowingform: ^f(x)=1 2;if1
PAGE 56

Epanechinikov 4(1x2) Uniform 2 2jxj 2 Variousprobabilitydensityestimationkernels.GaussianandLaplacian,hasinnitesupport,andtheotherkernelshave[1;1]asthesupport. Oneofthedrawbacksofkerneldensityestimationistheboundarybias.Whenthesupportofthepdfisnite,theinnitesupportkernelswillunderestimate,andevennitesupportkernelswillleaksomeofthedensitytooutsideofitssupport.Sincekerneldensityestimationprovidesrelativelyaccuratecontinuouspdfestimationwithanitesummation,asetofalgorithmsthatisbasedonpdfcanbewritteninecientmanner.Informationtheoreticlearning,aframeworkofsignalprocessingwithinformationtheoreticcostfunction,combinesReny'squadraticentropywithkerneldensityestimation,andnonparametricallyestimatesentropywithoutapproximations[ 37 76 ]. 77 ].ForarandomvariableXwithapdff(x),Renyi'squadraticentropyisdenedas,[ 78 ] 77 ].Asmentionedin A.3 ,Renyi'squadraticentropycanbeestimatedecientlywithkerneldensityestimation.Letfxi:i=1;:::;NgbeasetofNi.i.d.samplesofarandom 56

PAGE 57

^f(x)=1 A{16 ),yields, ^VX=1 Rf2i(t)dtRf2j(t)dt Rfi(t)fj(t)dt=logp 77 ].ItisimportanttoremarkthatICSisinfactapproximatingtheKullback-Leiblerdivergence[ 79 ]betweenthetwopdfs;however,asignicantadvantageistheeaseofcomputationofthismeasureusingtheinformationpotential.Noticealsothatintheargumentofthelogarithmthenumeratorcontainsthenormalizingterms.Inotherwords,thebehaviorofICSisdeterminedbythedenominatorterm,Vij,appropriatelycalledcrossinformationpotential(CIP).MuchliketheIP,theCIPexpressesapotentialduetointeractionsbetweenparticles,butfromdierentrandomvariables.BecausetheCIPnegativelyaectstheCSdivergence,itisineectmeasuringthesimilaritybetweenthetwodistributions. 57

PAGE 58

(I1) 5 (A{21)kxk=0()x=0by(I1)indenition 5 (A{22)

PAGE 60

80 ]). Proof. (A{26)Toensurethattheinnerproductiswell-dened,twodierentrepresentationoff;g2Bshouldleadtothesameinnerproduct,whichisobvious.LetuscompletethespacebyincludingalllimitsofCauchysequencesffnjn2N;fn2BganddenoteasHwhichisaHilbertspace.NotethatBisadensesubsetofH. 60

PAGE 61

(A{29)=Xl2LfflK(Sil;Si) (A{30)=f(Si) (A{31) 61

PAGE 62

60 ].TheanalysiscaneasilybeappliedtoCIPbymultiplyingthenormalizationfactor.Let(;T)betheclassofallpossiblehomogeneousPoissonspiketrainsofrateandlengthT.Theprobabilityofhavingarealizationi=fti1;ti2;:::;tiNigof(A;T)is,P[=ijA;T]=P[N(T)=Ni;ti1=ti1;ti2=ti2;:::;tiNi=tiNi] (B{1)=P[N(T)=Ni]P[ti1=ti1;ti2=ti2;:::;tiNi=tiNjN=Ni] (B{2)=(AT)Ni (B{3)=(AT)Ni 62

PAGE 63

2TNiXm=1NjXn=1ejtimtjntj 2Te(A+B)T1XNi=01XNj=0NiA 2Te(A+B)T1XNi=01XNj=0(AT)Ni 63

PAGE 64

dtimdtjn+ZTtZtimt0etimtjnt dtimdtjn+ZTtZTtimtetimtjnt dtimdtjn etimt dtim+ZTt1etimt dtimZTtetimTt 1dtim e0Tt ett +e0t +(Tt)2eTt +ett 2eTTt etTt +(Tt) (B{6d)=2(Tt)+2(eT+t +eTt 2et ) (B{6e)=2(Tt)+O(2): B{5d )gives,E[ij]^Qij(t)'AB2(Tt)+O(2) 2T; B{7 )canbeapproximatedbyABwhichisthedesiredvalue. 64

PAGE 65

42T2NiXp=1NjXq=1NiXr=1NjXs=1ejtiptjqtj 42T2TNiTNj1 65

PAGE 66

1 ))+O(3)+NiNj(Nj1) ))+O(3) (B{9c)Byassuming1T,wecanapproximateeT '0,O(3)'0.AndwefurtherapproximateA=Ni 42T2f(AB)2(2(Tt))2+(AB)((Tt))+(AB)(A+B)(42T)g=(AB)2Tt T2+AB T+(AB)(A+B)1 4T2:(B{11) 66

PAGE 67

59 ]Iindexsetspaceforspiketrains[p.??]Generalnotation 67

PAGE 68

58 ]kknormofavectorjjabsolutevaluex(t)y(t)convolution 68

PAGE 69

%ReturntheCrossInformationPotential. %Ifmorethantwoneuronsareprovidedaveragethroughallpaircombinations. % %X:Data,organizedasacellarray,witheachcellcontainingan %arrayofspiketimes(inseconds). %TAU:Kernelsize(inseconds). /*vim:setts=8sts=4sw=4:(modeline)*/ #include #include #include #include voidcip_func(intN,double*x[],intnSpikes[],doubletau,double*v); voidmexFunction(intnlhs,mxArray*plhs[],intnrhs,constmxArray*prhs[]){ mxArray*sts; mxArray*stp; intnSpikeTrain;/*numberofspiketrains*/ *spiketimes(sec)*/ *perspiketrain*/ *checkinputarguments */ mexErrMsgTxt("2inputsarerequired."); } elseif(nlhs>1){ mexErrMsgTxt("Toomanyoutputarguments"); } if(!mxIsDouble(prhs[1])) mexErrMsgTxt("TAUmustbeascalar"); *getinputarguments */ if(mxGetClassID(sts)!=mxCELL_CLASS){ mexErrMsgTxt("Xmustbeacellarray"); } nSpikeTrain=mxGetNumberOfElements(sts); if(nSpikeTrain<2){ mexErrMsgTxt("Atleasttwospiketrainsareneeded."); } nSpikes=(int*)mxMalloc(sizeof(int)*nSpikeTrain); x=(double**)mxMalloc(sizeof(double*)*nSpikeTrain); for(i=0;i
PAGE 70

* *@paramNnumberofspiketrains *@paramxarraytopointersforspiketrains *@paramnSpikesarraytolengthforspiketrains *@paramtaudecaytimeconstantfortheexponentialfunction *@paramvcomputedCIPwillbestoredhere,needtobepreallocated *@authorAntonioPaiva *@version$Id:cip.c522007-01-0316:55:26Zmemming$ { inti,j;/*countersforspiketrains*/ *paircombination*/ maxT=tau*100; *v=0; for(i=0;i<(N-1);i++) for(j=(i+1);j #include *ComputeICIPofasetofspiketrainswithchangingTAUmode. *@paramNnumberofspiketrains *@paramstsarraytopointersforspiketrains *@paramnstsarraytolengthforspiketrains *@paramvcomputedICIPwillbestoredhere,needtobepreallocated *@paramTtotaltime(sec) *@paramdttimebinsize(sec) *@paramBETAtheparameter$\beta$ofICIP(sec) *@paramFR_TAUthetimeconstantforfiringrateestimation

PAGE 71

*@version$Id:offline_icip.c512007-01-0219:53:01Zmemming$ { doublet;/*time*/ doubleEXP_FR; doubleONE_OVER_FR_TAU; doubleONE_OVER_BETA; double*q;/*charge*/ ONE_OVER_BETA=1/BETA; EXP_FR=exp(-dt/FR_TAU); idx=(int*)malloc(sizeof(int)*N); memset(idx,0,sizeof(int)*N); q=(double*)malloc(sizeof(double)*N); memset(q,0,sizeof(double)*N); f=(double*)malloc(sizeof(double)*N); memset(f,0,sizeof(double)*N); NPair=N*(N-1)/2; Nstep=(int)ceil(T/dt); for(t=dt,i=0;i
PAGE 72

Nstep=(int)ceil(T/dt); for(t=dt,i=0;i1){ mexErrMsgTxt("Toomanyoutputarguments"); } sts=(mxArray*)prhs[0]; if(mxGetClassID(sts)!=mxCELL_CLASS){ mexErrMsgTxt("Thefirstargumentshouldbeacellarray"); } nSpikeTrain=mxGetNumberOfElements(sts); if(nSpikeTrain<2){ mexErrMsgTxt("Atleasttwospiketrainsarerequired."); } nst=(int*)mxMalloc(sizeof(int)*nSpikeTrain); st=(double**)mxMalloc(sizeof(double*)*nSpikeTrain); for(i=0;i
PAGE 73

% %Input %st1,st2:spiketrainswithsortedspiketimings %tau:timeconstantforCIPkernel %maxT:correlogramrangewillbeeffectivein[-maxT,maxT] %T:lengthofspiketraininseconds %verbose:(optional/0)detailedinfo,usestic,toc % %Output %Q:cipogram %deltaT:timerange % %Seealso:cip_max_filter2,ncipogram % %Copyright2006AntonioandMemming,CNEL,allrightsreserved %$Id:cipogram.m532007-01-1423:24:21Zmemming$ verbose=0; end N1=length(st1); N2=length(st2); Nij=N1*N2; ifN1==0||N2==0 warning('cipogram:NODATA','Atleastonespikeisrequired!'); deltaT=[];Q=[]; return; end maxTTT=abs(maxT)+tau*10;%exp(-100)iseffectivelyzero %roughestimateof#oftimedifferencerequired(assumingindependence) %thisestimateisawefulifthespiketrainsarestronglycorrelated ifverbose;fprintf('Expectedtimedifferences[%d]/[%d]\n',eN,Nij);end deltaT=zeros(2*eN,1); k=1; forn=1:N1 form=lastStartIdx:N2 timeDiff=st2(m)-st1(n); iftimeDiff<-maxTTT lastStartIdx=lastStartIdx+1; continue; end iftimeDiff<=maxTTT deltaT(k)=timeDiff; k=k+1; else%thisistheendingpoint end end end deltaT=deltaT(1:(k-1)); N=length(deltaT); ifN<2 warning('cipogram:NODATA','Atleasttwointervalsarerequired'); deltaT=[];Q=[]; return; end ifverbose fprintf('Actualnumberoftimedifferences[%d]\nSorting...\n',N);tic; end deltaT=sort(deltaT,1);%Sortthetimedifferences Qplus=zeros(N,1); Qminus=zeros(N,1); Qminus(1)=1; Qplus(N)=0; EXP_DELTA=exp(-(diff(deltaT))/tau); fork=1:(N-1) Qminus(k+1)=1+Qminus(k)*EXP_DELTA(k); kk=N-k; Qplus(kk)=(Qplus(kk+1)+1)*EXP_DELTA(kk); end Q=Qminus+Qplus; Q=Q/2/tau/T;

PAGE 74

%Normalizedcipogramwith2ndorderstatistics. % %Input %st1,st2:spiketrainswithsortedspiketimings %tau:timeconstantforCIPkernel %maxT:correlogramrangewillbeeffectivein[-maxT,maxT] %T:lengthofspiketraininseconds %verbose:(optional/0) % %Output %Q:cipogram %deltaT:timerange % %Copyright2006AntonioandMemming,CNEL,allrightsreserved %$Id:ncipogram.m592007-01-2719:26:14Zmemming$ N1=length(st1); N2=length(st2); Nij=N1*N2; Q=(Q*T-Nij/T)*2*sqrt(tau*T)/sqrt(Nij);

PAGE 75

[1] M.C.W.vanRossum,\Anovelspikedistance,"NeuralComputation,vol.13,no.4,pp.751{764,2001. [2] J.D.Victor,\Spiketrainmetrics,"CurrentOpinioninNeurobiology,vol.15,no.5,pp.585{592,Sept.2005. [3] G.L.Gerstein,D.H.Perkel,andJ.E.Dayho,\Cooperativeringactivityinsimultaneouslyrecordedpopulationsofneurons:detectionandmeasurement,"JournalofNeuroscience,vol.5,no.4,pp.881{889,1985. [4] J.-M.Fellous,P.H.E.Tiesinga,P.J.Thomas,andT.J.Sejnowski,\Discoveringspikepatternsinneuronalresponses,"J.Neurosci.,vol.24,no.12,pp.2989{3001,Mar.2004. [5] S.Schreiber,J.M.Fellous,D.Whitmer,P.Tiesinga,andT.J.Sejnowski,\Anewcorrelation-basedmeasureofspiketimingreliability,"Neurocomputing,vol.52-54,pp.925{931,2003. [6] A.CarnellandD.Richardson,\Linearalgebrafortimeseriesofspikes,"inESANN,2005. [7] G.BuzsakiandA.Draguhn,\Neuronaloscillationsincorticalnetworks,"Science,vol.304,no.5679,pp.1926{1929,June2004. [8] Z.F.Mainen,J.Joerges,J.R.Huguenard,andT.J.Sejnowski,\Amodelofspikeinitiationinneocorticalpyramidalneurons,"Neuron,vol.15,no.6,pp.1427{1439,Dec.1995. [9] Y.Shu,A.Duque,Y.Yu,B.Haider,andD.A.McCormick,\Propertiesofactionpotentialinitiationinneocorticalpyramidalcells:evidencefromwholecellaxonrecordings(inpress),"J.Neurophysiol.,Aug.2006. [10] B.Hochner,M.Klein,S.Schacher,andE.R.Kandel,\Action-potentialdurationandthemodulationoftransmitterreleasefromthesensoryneuronsofAplysiainpresynapticfacilitationandbehavioralsensitization,"Proc.Natl.Aca.Sci.,vol.83,pp.8410{8414,1986. [11] H.AlleandJ.R.P.Geiger,\Combinedanalogandactionpotentialcodinginhippocampalmossybers,"Science,vol.311,no.5765,pp.1290{1293,Mar.2006. [12] A.-K.Warzecha,J.Kretzberg,andM.Egelhaaf,\Temporalprecisionoftheencodingofmotioninformationbyvisualinterneurons,"CurrentBiology,vol.8,no.7,pp.359{368,Mar.1998. 75

PAGE 76

[13] Z.N.Aldworth,J.P.Miller,T.Gedeon,G.I.Cummins,andA.G.Dimitrov,\Dejitteredspike-conditionedstimuluswaveformsyieldimprovedestimatesofneuronalfeatureselectivityandspike-timingprecisionofsensoryinterneurons,"J.Neurosci.,vol.25,no.22,pp.5323{5332,June2005. [14] Z.F.MainenandT.J.Sejnowski,\Reliabilityofspiketiminginneocorticalneurons,"Science,vol.268,no.5216,pp.1503{1506,1995. [15] R.VanRullen,R.Guyonneau,andS.J.Thorpe,\Spiketimesmakesense,"TrendsinNeurosciences,vol.28,no.1,pp.1{4,Jan2005. [16] M.N.ShadlenandW.T.Newsome,\Noise,neuralcodesandcorticalorganization,"Curr.Opin.Neurobiol.,vol.4,pp.569{579,1994. [17] H.Agmon-Snir,C.E.Carr,andJ.Rinzel,\Theroleofdendritesinauditorycoincidencedetection,"Nature,vol.393,no.6682,pp.268{72,May1998. [18] J.P.Donoghue,J.N.Sanes,N.G.Hatsopoulos,andG.Gaal,\Neuraldischargeandlocaleldpotentialoscillationsinprimatemotorcortexduringvoluntarymovements,"J.Neurophysiol.,vol.79,no.1,pp.159{173,1998. [19] D.R.Brillinger,J.HughL.Bryant,andJ.P.Segundo,\Identicationofsynapticinteractions,"BiologicalCybernetics,vol.22,pp.213{228,1976. [20] R.Dahlhaus,M.Eichler,andJ.Sandkuhler,\Identicationofsynapticconnectionsinneuralensemblesbygraphicalmodels,"JournalofNeuroscienceMethods,vol.77,pp.93{107,1997. [21] G.Schneider,M.N.Havenith,andD.Nikolic,\Spatiotemporalstructureinlargeneuronalnetworksdetectedfromcross-correlation,"NeuralComputation,vol.18,no.10,pp.2387{2413,2006. [22] T.Berger,M.Baudry,R.Brinton,J.-S.Liaw,V.Marmarelis,A.Y.Park,B.Sheu,andA.Tanguay,\Brain-implantablebiomimeticelectronicsasthenexterainneuralprosthetics,"ProceedingsoftheIEEE,vol.89,no.7,pp.993{1012,2001. [23] R.GutigandH.Sompolinsky,\Thetempotron:aneuronthatlearnsspiketiming-baseddecisions,"NatNeurosci,vol.9,no.3,pp.420{428,Mar.2006. [24] A.BorstandF.E.Theunissen,\Informationtheoryandneuralcoding,"NatNeurosci,vol.2,no.11,pp.947{957,Nov.1999. [25] L.Paninski,\Estimatingentropyonmbinsgivenfewerthanmsamples,"Informa-tionTheory,IEEETransactionson,vol.50,no.9,pp.2200{2203,2004. [26] W.BialekandA.Zee,\Codingandcomputationwithneuralspiketrains,"J.Stat.Phys.,vol.59,pp.103{115,1990.

PAGE 77

[27] J.D.Victor,\Binlessstrategiesforestimationofinformationfromneuraldata,"Phys.Rev.E,vol.66,no.5,pp.051903,Nov2002. [28] E.N.Brown,R.E.Kass,andP.P.Mitra,\Multipleneuralspiketraindataanalysis:state-of-the-artandfuturechallenges,"Natureneuroscience,vol.7,no.5,pp.456{61,may2004. [29] N.MasudaandK.Aihara,\Spatiotemporalspikeencodingofacontinuousexternalsignal,"NeuralComputation,vol.14,pp.15991628,2002. [30] M.Nawrot,A.Aertsen,andS.Rotter,\Single-trialestimationofneuronalringrates:Fromsingle-neuronspiketrainstopopulationactivity,"JournalofNeuro-scienceMethods,vol.94,pp.81{92,1999. [31] M.Bazhenov,M.Stopfer,M.Rabinovich,R.Huerta,H.D.Abarbanel,T.J.Sejnowski,andG.Laurent,\Modeloftransientoscillatorysynchronizationinthelocustantennallobe,"Neuron,vol.30,pp.553{567,2001. [32] P.ReinagelandR.C.Reid,\Preciseringeventsareconservedacrossneurons,"J.Neurosci.,vol.22,no.16,pp.6837{6841,Aug2002. [33] R.AzouzandC.M.Gray,\Dynamicspikethresholdrevealsamechanismforsynapticcoincidencedetectionincorticalneuronsinvivo,"PNAS,vol.97,no.14,pp.8110{8115,July2000. [34] E.M.Izhikevich,\Polychronization:Computationwithspikes,"NeuralComp.,vol.18,no.2,pp.245{282,Feb.2005. [35] J.delaRochaandN.Parga,\Short-termsynapticdepressioncausesanon-monotonicresponsetocorrelatedstimuli,"J.Neurosci.,vol.25,no.37,pp.8416{8431,Sept.2005. [36] P.Konig,A.K.Engel,andW.Singer,\Integratororcoincidencedetector?theroleofthecorticalneuronrevisited,"TrendsinNeurosciences,vol.19,no.4,pp.130{137,April1996. [37] D.Xu,Energy,entropyandinformationpotentialforneuralcomputation,Ph.D.dissertation,UniversityofFlorida,May1999. [38] A.Kuhn,A.Aertsen,andS.Rotter,\Higher-orderstatisticsofinputensemblesandtheresponseofsimplemodelneurons.,"NeuralComputation,vol.15,no.1,pp.67{101,2003. [39] A.Kuhn,S.Rotter,andA.Aertsen,\Correlatedinputspiketrainsandtheireectsontheresponseoftheleakyintegrate-and-reneuron,"Neurocomputing,vol.44{46,pp.121{126,June2002.

PAGE 78

[40] J.HopeldandA.Herz,\Rapidlocalsynchronizationofactionpotentials:Towardcomputationwithcoupledintegrate-and-reneurons,"PNAS,vol.92,no.15,pp.6655{6662,July1995. [41] J.J.HopeldandC.D.Brody,\Whatisamoment?transientsynchronyasacollectivemechanismforspatiotemporalintegration,"PNAS,vol.98,no.3,pp.1282{1287,2001. [42] A.K.EngelandW.Singer,\Temporalbindingandtheneuralcorrelatesofsensoryawareness,"TrendsinCognitiveSciences,vol.5,no.1,pp.16{25,2001. [43] R.E.MirolloandS.H.Strogatz,\Synchronizationofpulse-coupledbiologicaloscillators,"SIAMJournalonAppliedMathematics,vol.50,no.6,pp.1645{1662,Dec.1990. [44] D.HanselandH.Sompolinsky,\Synchronizationandcomputationinachaoticneuralnetwork,"Phys.Rev.Lett.,vol.68,no.5,pp.718{721,Feb1992. [45] A.Zumdieck,M.Timme,T.Geisel,andF.Wolf,\Longchaotictransientsincomplexnetworks,"Phys.Rev.Lett.,vol.93,no.24,pp.244103,2004. [46] E.M.Izhikevich,\Weaklypulse-coupledoscillators,FMinteractions,synchronization,andoscillatoryassociativememory,"IEEETransactionsonNeuralNetworks,vol.10,no.3,pp.508{526,1999. [47] L.F.AbbottandC.vanVreeswijk,\Asynchronousstatesinnetworksofpulse-coupledoscillators,"Phys.Rev.E,vol.48,no.2,pp.1483{1489,Aug1993. [48] P.DayanandL.F.Abbott,TheoreticalNeuroscience:ComputationalandMathemat-icalModelingofNeuralSystems,MITPress,Cambridge,MA,USA,2001. [49] A.M.Aertsen,G.L.Gerstein,M.K.Habib,andG.Palm,\Dynamicsofneuronalringcorrelation:modulationof\eectiveconnectivity","JournalofNeurophysiology,vol.61,no.5,pp.900{917,1989. [50] S.Grun,M.Diesmann,andA.Aertsen,\UnitaryEventsinmultiplesingle-neuronactivity.I.detectionandsignicance,"NeuralComputation,vol.14,no.1,pp.43{80,2002. [51] D.L.SnyderandM.I.Miller,RandomPointProcessesinTimeandSpace,Springer-Verlag,1991. [52] D.J.DaleyandD.Vere-Jones,AnIntroductiontotheTheoryofPointProcesses,Springer,1988. [53] Z.F.MainenandT.J.Sejnowski,\Reliabilityofspiketiminginneocorticalneurons,"Science,vol.268,no.5216,pp.1503{1506,1995.

PAGE 79

[54] D.H.Perkel,G.L.Gerstein,andG.P.Moore,\Neuronalspiketrainsandstochasticpointprocesses:II.simultaneousspiketrains,"BiophysJ,vol.7,no.4,pp.419{440,July1967. [55] C.K.Knox,\Cross-correlationfunctionsforaneuronalmodel,"BiophysicalJournal,vol.14,no.8,pp.567{582,1974. [56] R.E.Kass,V.Ventura,andC.Cai,\Statisticalsmoothingofneuronaldata,"Network:Comput.NeuralSyst.,vol.14,pp.5{15,2003. [57] W.Gerstner,\Codingpropertiesofspikingneurons:reverseandcross-correlations,"NeuralNetworks,vol.14,pp.599{610,2001. [58] P.DiggleandJ.S.Marron,\Equivalenceofsmoothingparameterselectorsindensityandintensityestimation,"JournaloftheAmericanStatisticalAssociation,vol.83,no.403,pp.793{800,Sept.1988. [59] E.Parzen,\Ontheestimationofaprobabilitydensityfunctionandthemode,"TheAnnalsofMathematicalStatistics,vol.33,no.2,pp.1065{1076,Sept.1962. [60] G.Palm,A.M.H.J.Aertsen,andG.L.Gerstein,\Onthesignicanceofcorrelationsamongneuronalspiketrains,"BiologicalCybernetics,vol.59,pp.1{11,1988. [61] H.ShimazakiandS.Shinomoto,\Arecipeforoptimizingatime-histogram,"inNeuralInformationProcessingSystems,2006. [62] B.L.SabatiniandW.G.Regehr,\Timingofsynaptictransmission,"AnnualReviewofPhysiology,vol.61,pp.521{542,1999. [63] S.M.PotterandT.B.DeMarse,\Anewapproachtoneuralcellcultureforlong-termstudies,"JournalofNeuroscienceMethods,vol.110,pp.17{24,2001. [64] H.KawaguchiandK.Fukunishi,\Dendriteclassicationinrathippocampalneuronsaccordingtosignalpropagationproperties,"ExperimentsinBrainResearch,vol.122,pp.378{392,1998. [65] J.leFeber,W.L.C.Rutten,J.Stegenga,P.S.Wolters,G.J.A.Ramakers,andJ.vanPelt,\Conditionalringprobabilitiesinculturedneuronalnetworks:astableunderlyingstructureinwidelyvaryingspontaneousactivitypatterns,"J.NeuralEng.,vol.4,pp.54{67,2007. [66] C.D.Brody,\Correlationswithoutsynchrony,"NeuralComputation,vol.11,pp.1537{1551,1999. [67] D.Nikolic,\Non-parametricdetectionoftemporalorderacrosspairwisemeasurementsoftimedelays,"J.Comput.Neurosci.,vol.22,pp.5{19,2007. [68] F.Rieke,D.Warland,R.deRuytervanSteveninck,andW.Bialek,Spikes:exploringtheneuralcode,MITPress,Cambridge,MA,USA,1999.

PAGE 80

[69] P.Diggle,\Akernelmethodforsmoothingpointprocessdata,"AppliedStatistics,vol.34,no.2,pp.138{147,1985. [70] E.N.GilbertandH.O.Pollak,\Amplitudedistributionofshotnoise,"BellSyst.Tech.J.,vol.39,pp.333350,1960. [71] S.YueandM.Hashino,\Thegeneralcumulantsforalteredpointprocess,"AppliedMathematicalModelling,vol.25,pp.193{201,2001. [72] J.Michel,\Apointprocessapproachtolteredprocesses,"Methodol.Comput.Appl.Probab.,vol.6,pp.423{440,2004. [73] E.Parzen,TimeSeriesAnalysisPapers,Holden-Day,1967. [74] J.S.Simono,SmoothingMethodsinStatistics,Springer,1996. [75] I.S.Abramson,\Onbandwidthvariationinkernelestimates-asquarerootlaw,"Ann.Stat.,vol.10,no.4,pp.1217{1223,Dec1982. [76] D.ErdogmusandJ.C.Prncipe,\Generalizedinformationpotentialcriterionforadaptivesystemtraining,"IEEETransactionsonNeuralNetworks,vol.13,no.5,pp.1035{1044,Sept.2002. [77] J.C.Prncipe,D.Xu,andJ.W.Fisher,\Informationtheoreticlearning,"inUnsupervisedAdaptiveFiltering,S.Haykin,Ed.,vol.2,pp.265{319.JohnWiley&Sons,2000. [78] A.Renyi,\Onmeasuresofentropyandinformation,"inSelectedpapersofAlfredRenyi,vol.2,pp.565{580.AkademiaiKiado,Budapest,Hungary,1976. [79] S.Kullback,InformationTheoryandStatistics,DoverPublications,NewYork,1968. [80] N.Aronszajn,\Theoryofreproducingkernels,"TransactionsoftheAmericanMathematicalSociety,vol.68,no.3,pp.337{404,May1950.

PAGE 81

IlParkwasbornonApril29,1979inGosla,Germany.HeattendedGyunggiScienceHighSchoolfor2years.HemajoredcomputerscienceatKAIST(KoreaAdvancedInstituteofScienceandTechnology).Hespent2001-2003inaninternetsecuritycompanyasadeveloper.HehasbeenworkingwithDr.JosePrncipeinComputationalNeuroEngineeringLaboratory(CNEL)since2006.HeisadmittedtotheBiomedicalEngineeringdepartmentforthePh.D.programinUniversityofFlorida. 81


Permanent Link: http://ufdc.ufl.edu/UFE0020302/00001

Material Information

Title: Continuous Time Correlation Analysis Techniques For Spike Trains
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0020302:00001

Permanent Link: http://ufdc.ufl.edu/UFE0020302/00001

Material Information

Title: Continuous Time Correlation Analysis Techniques For Spike Trains
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0020302:00001


This item has the following downloads:


Full Text





CONTINUOUS TIME CORRELATION ANALYSIS TECHNIQUES
FOR SPIKE TRAINS



















By

IL PARK



















A THESIS PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE


UNIVERSITY OF FLORIDA

2007
































S2007 II Park





































Memmings are Memmings,

computers are recursive,

brains are brains.









ACKNOWLEDGMENTS

I thank my adviser Dr. Jos6 C. Principe for all his great guidance, my committee

member Dr. John Harris for insightful -i-. -i. .i-, and Dr. Thomas B. DeMarse for his

knowledge and intuition on experiments. I thank my collaborators Ant6nio R. C. Paiva

and Karl Dockendorf for all the joyful discussions. I also thank Dongming Xu (dynamics),

Jian-Wu Xu (RKHS), Vaibhav Garg, Manu Rastogi, Savyasachi Singh (chess), Allen

Martins (pdf), Yiwen Wang and Aysegiil Gtindtiz of CNEL, Jason T. Winters, Alex J.

Cadotte, Hany Elmariah (singing) and Nicky Grimes of the Neural Robotics and Neural

Computation Lab for their support and help. Last but not least, I thank my family and

friends for being there.









TABLE OF CONTENTS


page


ACKNOW LEDGMENTS .................................

LIST O F TABLES . . . . . . . . . .

LIST OF FIGURES . . . . . . . . .

A B ST R A C T . . . . . . . . . .

CHAPTER

1 INTRODUCTION ..................................

1.1 M otivation . . . . . . . . .
1.1.1 Why Do We Analyze Spike Trains? ..................
1.1.2 What Are Similar Spike Trains? ....................
1.2 M inim al Notation . . . . . . . .

2 CROSS INFORMATION POTENTIAL ......................


Smoothed Spike Train Representation
L2 Metric ..............
Cauchy-Schwarz Dissimilarity ...
Information Potential .........
Discussion .............
2.5.1 Comparison of Distances .
2.5.2 Robustness to Jitter in the Spik


:e Timings


3 INSTANTANEOUS CROSS INFORMATION POTENTIAL ..........


3.1 Synchrony Detection Problem .. ......
3.2 Instantaneous CIP .. ...........
3.2.1 Derivation from CIP .. .......
3.2.2 Spatial Averaging .. .........
3.2.3 Rescaling ICIP .. ...........
3.3 A analysis . . . . .
3.3.1 Sensitivity to Number of Neurons .
3.4 R results . . . . . .
3.4.1 High-order Synchronized Spike Trains
3.4.2 Mirollo-Strogatz Model ........









4 CONTINUOUS CROSS CORRELOGRAM ........ ............ 32

4.1 Delay Estimation Problem ................... ..... 32
4.2 Continuous Correlogram .................. ......... .. 34
4.3 Algorithm .................. ................. .. 36
4.4 R results . .. . . . .. . . . .. 39
4.4.1 Analysis .................. .............. .. 42
4.4.2 Examples. .................. ............. 43
4.5 Discussion .................. ................. .. 47

5 CONCLUSION .................. ................. .. 49

5.1 Summary of Contribution .................. ........ .. 49
5.2 Potential Applications and Future Work .................. .. 49

APPENDIX

A BACKGROUND .................. ................ .. 50

A.1 Point Process .................. .............. .. 50
A.1.1 An Alternative Representation of Poisson Process . ... 51
A.1.2 Filtered Poission Process .................. ... .. 52
A.2 Mean Square Calculus .................. . ... 53
A.3 Probability Density Estimation .................. ... .. 54
A.4 Information Theoretic Learning .................. ... .. 56
A.5 Reproducing Kernel Hilbert Space ...... .......... .. 58

B STATISTICAL PROOFS ............... ......... ..62

C NOTATION ................... ..... .... ....... 67

D SOURCE CODE ............... .............. .. 69

D.1 CIP ................................ ...... 69
D.2 ICIP .............................. .. ..... 70
D.3 CCC ..... .............. .................. 73

REFERENCES .............................. ... ...... 75

BIOGRAPHICAL SKETCH .................. ............. .. 81









LIST OF TABLES


A-i Various probability density estimation kernels .. ...............


Table


page









LIST OF FIGURES


2-1 L2 distance versus CS divergence .. .....................

2-2 Distance difference of CS divergence for a synchronized or uncorrelated missing
sp ik e . . . . . . . . . . .

2-3 C'! is,, in CIP versus jitter standard deviation in the synchronous spike timings

3-1 Spike train as a realization of point process and smoothed spike train ......

3-2 Variance in scaled CIP versus the number of spike trains used for spatial averaging


in log scale.


3-3 Analysis of ICIP as a function of synchrony . . .....

3-4 Evolution of synchrony in the spiking neural network . . .

3-5 Zero-lag cross-correlation for comparison . . .....

4-1 Example of cross correlogram construction . . .....

4-2 Decomposition and shift of the multiset A.. . . .....

4-3 Effect of the length of spike train and strength of connectivity on precision of
delay estim ation .................

4-4 Effect of kernel size (bin size) of CCC (CCH) to the performance .. ......

4-5 Schematic diagram for the configuration of neurons .. ............

4-6 Comparison between CCC and CCH on synthesized data .. ..........

4-7 Effect of length of spike trains on CCC and CCH .. ..............

4-8 Correlograms for in vitro data .. .......................


. . . . .. . . 2 4


26

28

29

33

36


Figure


page









Abstract of Thesis Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Master of Science

CONTINUOUS TIME CORRELATION ANALYSIS TECHNIQUES
FOR SPIKE TRAINS

By

II Park

May 2007

C('! i: J6se Carlos Principe
Major: Electrical and Computer Engineering

Correlation is the most basic analysis tool for time series. To apply correlation

to train of action potentials generated by neurons, the conventional method is to

discretize the time. However, time inning is not optimal: time resolution is sacrificed,

and it introduces the notorious problem of bin size sensitivity. Since spike trains can

be considered as a realization of a point process, the signal has no amplitude and all

information is embedded in the times of occurrence. Instead of time inning, we propose a

set of methods based on kernel smoothing to analyze the correlations. Smoothing is done

in continuous time so we do not lose the exact time of spikes while enabling interaction

between spikes at a distance. We present three techniques derived from correlation: (1)

spike train similarity measure, (2) synchrony detection mechanism, and (3) continuous

cross correlogram.









CHAPTER 1
INTRODUCTION

1.1 Motivation

Signal processing tools such as adaptive filtering, least squares, detection theory,

Sll.-I, i .- and spectral analysis have brought engineers the power to analyze virtually

any signal. However, the application of such tools to the signal of the nervous system, the

spike train, has remained restricted. This is mainly because of the poor performance of

usual estimators for statistical variables such as mean and correlation function for point

process observations.

The foundation of signal processing tools is the L2, the metric space of random

processes with finite second order moment, which is a well defined Hilbert space. The

metric (distance measure) of the random process provides a continuous spectra of similar

signals, providing a friendly space analogous to Euclidean space. Also, the distance is

strongly related to correlation which is the inner product in L2. While point processes can

be theoretically treated in the same way, the main problem is to estimate the process from

the observation. In contrast to analog and digital signals, the distance estimator between

two point process observations in the traditional sense leads to natural numbers which is

not continuous but discrete, so the spectra of signals is lost. The discrete metric makes it

inappropriate to directly apply the signal processing tools to spike trains.

Neuroscience literature have been using several approaches to overcome this difficulty.

The most widely used approach is to use time bins to convert the times of occurrence to

a sequence of binary amplitude or discrete time series. Recently, van Rossum proposed a

metric for spike trains [1], which is related to a non-Euclidean metric proposed by Victor

and coworkers which is an extension of the Levenshtein distance (also known as edit

distance in computer science) to continuous time [2]. Many neuroscientists were already

using the van Rossum distance by intuition in the form of correlation [3-6].

We mapped the spike trains to a realization of a random process in L2, so that

traditional signal processing techniques can be readily applied. We will analyze









the properties of the mapping and the metric induced by the mapping. One of the

advantages we gain from this approach is that by choosing the appropriate mapping, the

computational cost can be minimized while the time resolution remains continuous. We

will derive correlation based measures from this space and recover the power of signal

processing tools for spike trains. Specifically, we propose three techniques, (1) the cross

information potential (CIP) as a similarity measure between spike trains based on

correlation, (2) the instantaneous cross information potential (ICIP) as a measure

of instantaneous synchrony among spikes trains, and (3) continuous cross correlogram

(CCC) as an extension of CIP to continuous time lags. All of the proposed has efficient

computation mechanism and will be accompanied by statistical analysis.

1.1.1 Why Do We Analyze Spike Trains?

Neurons communicate mainly through a series of action potentials, although

there are increasing evidence that field potentials are also essential in the brain [7].

Action potentials are generation by the complex dynamics of a neuron [8, 9], and has a

stereotypical shape which can be propagated through a long distance and can resist noise

because of its all-or-none type of transmission. There have been evidence that not only

the existence of an action potentials carries information, but the duration of the action

potential is systematically modulated [10], and recently even subthreshold dendritic input

can modulate synaptic terminals [11].

However, from the computational point of view, it is believed that the temporal

structure of the action potentials is more important than individual details of an action

potential. Experiments mainly in sensory encoding demonstrates precise timing (or

precise time to first spike) of action potentials ([12-14], see [15] for a review, and [16] for

arguments against it) which supports the idea of encoding information on spike times. The

precision of spike timings is less than 100 ps in auditory system [17] and in the order of

1 ms in other experiments [14].









The other reason that spike trains are widely studied is because it is relatively easy

to record with high accuracy and precision. Extracellular electrode arrays permits the

recording from massive number of neurons simultaneously in vivo and in vitro.

MI ii: methods have been developed to analyze spike trains for various problems

including correlation analysis [18], connectivity estimation [19, 20], delay estimation [21],

system identification [22], clustering different spike patterns [4, 23], estimating entropy

[24-27], and neural decoding [28, 29]. We will tackle some of these problems with the

proposed techniques.

1.1.2 What Are Similar Spike Trains?

As mentioned in section 1.1.1, the spike times produced by neurons in response to

repeated stimulus often shows precise timing with some error. The jitter error distribution

fits with a Gaussian distribution [13]. The possible noise sources are thermal noise, ion

channels, probabilistic synapse activation, spontaneous release of vesicles.

When the spike train is modeled by a Poisson process, the jitter noise restricts the

shape of the intensity function (instantaneous firing rate) over time. In other words, the

noise will limit the narrowness of a precisely timed spike. In addition, this implicates that

the spike trains with small timing differences should be treated as similar to each other,

thus having a small distance (or dissimilarity1 ).

We can exploit this and construct a probable intensity function from a spike train

by using the techniques of kernel density estimation. The kernel, which represents the

jitter timing distribution, will be placed where the spikes have actually occurred, and the

summation of all kernels will estimate the intensity function assuming a Poisson process.

Nawrot and coworkers have tried various kernels for single trial estimation of the intensity



1 Distance usually refers to a mathematical metric which satisfies positivity, reflexivity,
definiteness, symmetry and triangle inequality. However, we will also refer to dissimilarity
measures that lack the triangle inequality as a distance informally and interchangeably
with dissimilarity.









function from spike trains in a model, and concluded that the kernel size (bandwidth) is

more important that the shape of the kernel [30].

Another type of noise in spike trains is insertion or deletion of spikes. Although

spike trains of neurons conserve high precision of spike timings when they occur, there

is evidence that neurons often skip a few spikes [4, 31, 32]. When a spike is inserted or

removed from a spike train, the distance differs by the constant 2 in van Rossum distance.

In contrast, a correlation measure does not depend on signal power (or number of action

potentials), but only on the coincidental action potential pairs. In applications, such

as classification of spike trains with template matching, the correlation based distance

measure (Cauchy-Schwarz divergence) can perform better than van Rossum (L2) distance.

The concept of coincidental spikes leads to synchrony between spike trains. In

addition, there are strong evidences that neurons and dendrites work as a coincidence

detector and sensitive to afferent synchrony [26, 33-36].

1.2 Minimal Notation

We introduce the minimal mathematical notation. We assume that a number of spike

trains are observed, and indexed. Each spike train is a finite set of spike timings where

the action potentials are detected. For the spike train indexed by i, individual timings are

denoted as t where m is the index for spikes. The functional form of i-th spike train is

defined as,
Ni
sint) = 6pie tin (1-1)
m=l
where Ni is the number of spikes in i-th spike train, and 6(.) is the Dirac delta function.









CHAPTER 2
CROSS INFORMATION POTENTIAL

2.1 Smoothed Spike Train Representation

Given a spike train si(t), we assume inhomogeneous Poisson process and estimate the

intensity function by using a kernel. The kernel has to be non-negative valued and has

area of 1, that is, it has to be a proper probability density function. Denote this kernel as

Kpdf(t), then the estimated intensity function can be written as,
Ni
(t)= pdf(t t. (2-1)
nm=l

This process can also be viewed as low pass filtering of the spike trains to estimate

the post synaptic potential of synapses. In the point process literature, this is a special

case of filtered point process, and in the engineering literature known as shot noise. 1

The estimated intensity function is continuous if Kpdf is continuous. Assuming continuous

Kpdf, the mapping equation (2-1) converts a spike train to a continuous signal that can be

interpreted with the second order theory with a continuous metric. Note that the mapping

is one-to-one and onto: deconvolution of Ai(t) with Kpdf uniquely determines a spike train.

2.2 L2 Metric

The smoothed spike train, or estimated intensity function, can be considered as a

signal in L2. The distance in L2 of two smoothed spike trains is,


Af(t) A(t) J (Ad(t) Af(t))2dt (2-2a)
2 DC
/oO
i (Af)- 2A((t)j(t) Af)) dt. (2-2b)




1 When the underlying process is a homogeneous Poisson process, the filtered point
process is wide sense stationary (WSS) by Campbell's theorem (see appendix, theorem 3).










Using the definition of the estimator (2-1),


J (t)dt p df(t tn t Kpd dt (2(3a)
*O -'- 1 n- 1
Ni Ni
(t tY ) (2-3b)
rm1 n= 1

and the cross term (inner product in L2) becomes,

oo Ni N,
(t)A(t)dt Z- t(t tP) (2-3c)
m=1 n= 1

where t(t) = Kpdf() Kpdf(S + t)ds. K is the kernel which computes the correlation.

If an exponential distribution is used, i.e.,

1 t
Kpdf(t) -e eu(t), (2-4)
T

where u(t) is the unit step function, then the L2 distance is proportional to van Rossum

distance with factor 1. In addition, the combined kernel t(t) becomes a scaled Laplace

distribution kernel:

/oo 1 O- t-tNi Nj i
Aj(t)\A{(t)dt =- 2 y exp --m u(t ) exp t- u(t tj)dt
-oo m- l n=1

(2-5)

2 Ni Nj 2t d (26)
_- y exp xpm 2 t-t )u(t t)) (2-6)
2n 1 o



In l1 o 7 t)


Z exp (2-9)
1Ni N9


t exp (2-9)
m=ln n 1

Note that in terms of a linear filter, the causal exponential distribution corresponds to a

first-order infinite impulse response (IIR) filter with time constant T with gain of .
T








2.3 Cauchy-Schwarz Dissimilarity
An alternative dissimilarity measure that can be induced from inner product of L2 is
the Cauchy-Schwarz (CS) divergence. Recall the Cauchy-Schwarz inequality (see lemma
6):

I(xl < I X 1 \y11

Since each quantity is positive if x and y are not zero vectors, and equality holds when
either of them are zero, we can divide both sides,

I1< 1 I 1Y 1
1<
I(xll

By taking the logarithm,
0 < log () jl-< oc.

It can be proved that this quantity is positive, reflexive, and symmetric [37] if we exclude
0 from the space. However, CS divergence does not hold the triangular inequality, thus it
is not a metric. By expanding the definition of inner product and norm of L2 space,

lo V Z tdn t fr1t) (2 lb)
dcs(Ai(t), A,(t)) = log (2-(Oa)
A (t) (t) dt

( ,/ j VI I K (ti +i \ y : N y : N W 8 .
= log (2-10c)

( Ni Ni Nj Nj

i\m=n m=1n=




where dcs denotes the CS divergence.
If the spike trains are homogeneous Poisson with firing rate Ai and Aj respectively,
the expected value of the norm of estimated intensity function E [Af(t)] is the second order









moment of the shot noise, which can be obtained by equation (A-8),


E [A (t)]= A (t)dt. (2-11)
*OO

Therefore the first term in equation (2-10c) can be approximated as a constant. However,

depending on the correlation of the spike trains, the second term will vary. Since the

negative logarithm is a monotonically decreasing function, we take the argument, denote

as Vij, and define as cross information potential for reasons that would be explained in

section 2.4.
N, Nj
Vi E (Z t -t t) (2-12)
m=l n=1
This inner product term is essentially equivalent to correlation of smoothed spike trains.

CIP is inversely related to CS divergence, so it quantifies similarity between spike trains.

2.4 Information Potential

Given a probability distribution, entropy quantifies the peakiness and is related

to the higher order moments that the variance cannot capture. Renyf's entropy is a

generalization of the classic Shannon's entropy. Information theoretic learning (see section

A.4 for a summary of the information theoretic learning framework).

Inhomogeneous Poisson process can be represented as two separate random variables:

one for the number of spikes and the other for the temporal density (see section A.1.1).

The pdf for the temporal density is simply a normalized form of the intensity function

(equation (A-2)). This pdf does not have the information of how active the process is,

that is, the firing rate.

Information potential of density function estimated using Parzen window with Kpdf for

the i-th spike train has the following form (compare equation (A-16)),

N, N,
V- N Z -E t t) (2 13)
i m=l n=l

where Kc(t) f= j Kpdf(S)Kpdf(S + t)ds is defined as before. This coincides with the

definition of norm square of the smoothed spike train, equation (2-3b), normalized by the










CS distance


0.04
0.035
CM
S0.03
E 0.025
E 0.02
0
g 0.015
S0.01
0.005
n


Figure


L2 distance


o00


*<

c .


0 0.01 0.02 0.03 0.04 0 1 2 3 4 5
distance from template 1 distance from template 1

2-1. L2 distance versus CS divergence. Spike trains from template 1 is generated
and the distance (or divergence) from each template. Gaussian jitter with
0.7 ms standard deviation is added to the timings. Blue circles correspond
to spike trains with same number of spikes, and red dots correspond to spike
trains with missing spikes. The kernel K was Laplacian with time constant
r = 1 ms.


number of spikes. For a pair of spike trains, the cross information potential can be defined

as a similarity index between the corresponding pair of pdfs. Note that in terms of CS

divergence, the normalization with the number of spikes in the spike train cancels away.

2.5 Discussion

2.5.1 Comparison of Distances

As mentioned earlier in section 1.1.2, although neurons fire with high temporal

precision, they often miss spikes. In this case, L2 distance would deviate because of the

missing spike. CS divergence would be less sensitive because it will ignore missing spikes.

To demonstrate this, a simple classification task was performed (see figure .2-1). Two

template spike trains were prepared: template 1 with 2 spikes at 3 ms and 8 ms, and

template 2 with 1 spike at 6 ms. Then, we generated instances of template 1 by putting

Gaussian jitter on timing (blue circles) and removing a spike (red dots).

For the no missing spike case, both L2 (1 !' .) and CS divergence (1('1.) correctly

classified the instance as template 1 (they lie on the upper half). But for missing spikes


























Figure 2-2.


loosing one correlated spike
30 spikes total
60 spikes total


10 20 30 0 10 20 30
number of total spikes number of correlated spikes

Increase or decrease in Cauchy-Schwarz (CS) divergence (dissimilarity) when
a spike is missing. (Left) When a correlated (perfectly synchronized in this
case) spike is missing, the divergence decrease inversely related to the total
number of spikes. (Right) But if a correlated (synchronized) spike is rni--il:
the divergence increases proportional to the total number of synchronized
spikes, and not greatly influence by the total number of spikes. In contrast, L2
distance the increase and decrease are constant (see text for details).


case, L2 distance (51 .) performed a lot worse than CS divergence (9 :'). The CS

divergence shows lines when one spike is missing because the distance (quantified as the

divergence) is a log of the kernel which is a single Laplacian.

Suppose individual spikes are separated compared to the kernel size or exactly

synchronized so that we can approximate the norm and inner product by the number

of spikes: norm square of a spike train is the number of spikes, and inner product gives

the number of synchronized spikes. This is equivalent to making the kernel size infinitely

small, so that it converges to a Dirac delta function.

Let there be two spike trains A and T (for template) with NA and NT number of

spikes respectively, and NAT synchronized spikes. The L2 distance between A and T is

NA + NT 2NAT, and the CS divergence is log NA.

If we loose a spike that was not synchronous between A and T, the distance will

decrease by the constant 1 in L2 distance (I in van Rossum distance) and for CS


loosing one uncorrelated spike












-0.35
-0 0
0.6 0.1 0.1
-0.2 0.3 -0.2
0.3 0.3
0.5 -0.4 \0.4
S0.5 0.25 0.5
- 0.4
0.2
0.3
0.15


0.1 0.1
0 1 2 3 4 5 6 7 8 9 101112131415 0 1 2 3 4 5 6 7 8 9 101112131415
Jitter standard deviation (ms) Jitter standard deviation (ms)


Figure 2-3. C'!: i,_, in CIP versus jitter standard deviation in the synchronous spike
timings. For the case with independent spike trains, the error bars for one
standard deviation are also shown. The kernel size is 2ms (left) and 5ms
(right).


divergence the decrease is,

NANT NA(NT- 1) NANT NAT
log -log log (2-14)
NAT NAT NAT NA(NT- 1)
NT
Slog (2-15)
NT 1

Thus, if there are more spikes, the CS divergence decreases less for a missing non-synchronous

spike. (And if the last spike is lost, the CS divergence is not defined anymore.)

If a synchronized (correlated) spike is lost, NT and NAT are reduced by 1. L2 distance

increases by 3, and for CS divergence the increase is,

NA(NT- 1) 1NAN NTNT-1 NAT
log -- log NT NAT- (2-16)
NAT 1 lAT nT oAT 1

Therefore, if there are more synchronized spikes, the distance decreases more. See

figure 2-2 for an illustrative example.

2.5.2 Robustness to Jitter in the Spike Timings

CIP was analyzed when jitter is present in the spike timings. This was done with

a modified multiple interaction process (\I P) model [38, 39] where jitter, modeled as


[2ms]


[5ms]









i.i.d. Gaussian noise, was added to the individual spike timings. In the MIP model an

initial spike train is generated as a realization of a Poisson process. All spike trains are

derived from this one by copying spikes with a probability E. The operation is performed

independently for each spike and for each spike train. The resulting spike trains are also

Poisson processes. If 7 was the firing rate of the initial spike train then the derived spikes

trains will have firing rate Ey. Furthermore, it can be shown that E is also the count

correlation coefficient [38]. A different interpretation for E is that, given a spike in a spike

train, it quantifies the probability of a spike co-occurrence in another spike train.

The effect was then studied in terms of the synchrony level and kernel size. Figure 2-3

shows the average CIP for 10 Monte Carlo runs of two spike trains, 10 seconds long, and

with constant firing rate of 20 spikes/s. In the simulation, the synchrony level was varied

between 0 (independent) to 0.5 for a kernel size of 2ms and 5ms. The jitter standard

deviation varied between the ideal case (no-jitter) to 15ms.

As mentioned earlier, CIP measures the coincidence of the spike timings. As a

consequence, the presence of jitter in the spike timings decreases the expected values

of CIP (and time averaged ICIP). Nevertheless, the results in Fig. 2-3 support the

statement that the measure is indeed robust to large levels of jitter compared to the kernel

size, and is capable of detecting the existence of synchrony among neurons. Of course,

increasing the kernel size decreases the sensitivity of the measure for the same amount

of jitter. Furthermore, as in the previous example, it is also shown that small levels of

synchrony can be discriminated from the independent case as r-ii:-- -1 .1 by the error

bars in Figure 2-3. Finally, we remark that the difference in scale between the figures is

a consequence of the normalization of the kernel so that it is a valid pdf. This can be

compensated explicitly by scaling the CIP by r. Simply note that the expressions provided

in the previous example for mean ICIP (and therefore CIP) as a function of the synchrony

level implicitly compensate for r.









CHAPTER 3
INSTANTANEOUS CROSS INFORMATION POTENTIAL

3.1 Synchrony Detection Problem

Coincidental firing of different neurons has been a focus of interest-from synfire

chain [40], neural coding [31, 41], neural assemblies [3], binding problem [42], and to pulse

coupled oscillators [43-47]. Analysis of synchrony has relied on various methods, such

as the cross-correlation [48], joint peri-stimulus time histogram (JPSTH) [49], unitary

events [50], and gravity transform [3], among many others.

Since CIP (or CS divergence) characterizes the similarity (or dissimilarity) of spike

trains with correlation of spike times, CIP can also be used as a synchrony measure.

However, CIP does not provide information about instantaneous synchrony. A sliding

window approach can be used with sacrifice of the temporal resolution, as in cross

correlation and gravity transform.

3.2 Instantaneous CIP

3.2.1 Derivation from CIP

Let us break the integral range from the definition of L2 inner product (equation

(2-3c)).
/
Vi(t) ,(a),- (a-)da. (31)

Taking the derivative on time yields ICIP,


,(t)= At)Aj(t), (3 2)


(a) ttt t N


(b)
TO time T

Figure 3-1. Spike train as a realization of point process and smoothed spike train. (a)
Spike train of neuron i represented in the time domain as a sequence of
impulses and (b) its filtered counterpart using a causal decaying exponential.









by the fundamental theorem of calculus. Since the derivative provides the instantaneous

change of CIP at that time, ICIP quantifies instantaneous synchrony of the action

potential timing. If we use the exponential kernel for intensity estimation, ICIP can be

easily estimated by two IIRs and a multiplication, therefore requiring no memory, but just

two state variables.

3.2.2 Spatial Averaging

In the context of neural assembly, ensemble of neurons work together with synchronous

spikes. Current multielectrode recording technology has enabled the analysis of a number

of spike trains recorded simultaneously. It is possible to reduce the trial averaging by

combining the concept of neural assemblies and multiple spike trains recording. The

spatial averaging over the ensemble may provide high resolution of the events.

Consider a set of M spike trains. ICIP (and CIP) can be generalized to multiple spike

trains in a straightforward manner by averaging over all the pairwise combinations. That

is, the ensemble averaged ICIP is given by
M M
(t) M(M- 1) (t) (3-3)
i=1 j=i+l

Analysis of the spatial averaging is presented in section 3.3.1.

3.2.3 Rescaling ICIP

When precise timing is modulated with a fluctuation of the firing rate, the precision

of the timing may vary. In high firing rate regions, the experimenter would like to lp l

more attention to more precise synchronizations, since the spikes are dense. ('C! I.iI..i the

kernel size according to the general firing rate trend may help in these cases.

The time rescaling theorem states that an inhomogeneous Poisson process can be

transformed into a homogeneous Poisson process [51, 52] by stretching the time according

to the intensity function. Transformation of equation (2-1) into a constant firing rate

time scale for different spike trains depends on individual intensity function, and therefore

the transformed results are not synchronous. Thus, in order to quantify synchrony, the











-1 ms
S2 ms
10 1 -2 5 ms
100 10 ms
EL_ :-a_ \20 ms
810-2 10-3

0 10-



10-4 03 10 -5
-0.4
0.5
10 5 10-6
5 10 15 20 25 30 0 5 10 15 20 25 30
Number of spike trains Number of spike trains

Figure 3-2. Variance in scaled CIP versus the number of spike trains used for spatial
averaging in log scale. The analysis was performed for different levels of
synchrony and constant 7 = 2ms (left), and different values of the exponential
decay parameter r on independent spike trains (right). In both plots the
theoretical value of CIP for independent spike trains is shown (dashed line).


correlation operation should be performed in the original times, but with the smoothing in

the transformed space. The first order approximation of this can be achieved by redefining

the intensity estimator as


A (t) exp ( (t ti u(t t) (3-4)
m= 1

where fi(t) is also the estimation for the intensity function and 3 > 0 is a scaling constant

which specifies the value of '- when the firing rate is one. Therefore, at time t, the effective

time constant is approximately A It may seem like an oxymoron to estimate an

intensity function using an estimate of the intensity function, but f(t) is estimated with a

broader kernel for the firing rate trend, and A(t) has a small kernel size that corresponds

to the resolution of interest.

3.3 Analysis

3.3.1 Sensitivity to Number of Neurons

We now analyze the effect of the number of spike trains used for spatial averaging.

This effect was studied with respect to two main factors: the synchrony level of the spike









trains and the exponential decay parameter r. In the first case, a constant r = 2ms was

used, while the latter case considered only independent spike trains. The results are shown

in Fig. 3-2 for the scaled CIP spatially averaged over all pair combinations of neurons.

The simulation was repeated for 200 Monte Carlo runs using 10 second long spike trains

obtained as homogeneous Poisson processes with firing rate of 20 spikes/s.

As illustrated in the figure, the variance in CIP decreases dramatically with the

increase in the number of spike trains employ, -1 in the analysis. Recall that the number of

pair combinations over which the averaging is performed increases with M(M 1), where

M is the number of spike trains. As expected, this improvement is most pronounced in the

case of independent spikes trains. In this situation, the variance decreases proportionally

to the number of averaged pairs of spike trains. This is shown by the dashed line in the

plots of Fig. 3-2. These results support the role and importance of ensemble averaging as a

principled method to reduce the variance of the CIP estimator.

3.4 Results

3.4.1 High-order Synchronized Spike Trains

Figure 3-3 shows ICIP of different levels of synchrony over ten spike trains. The

synchrony was generated by using the MIP model, and modulated over time for 1 seconds

of time durations. The firing rate of the generated spike trains was constant and equal to

20 spikes/s for all spike trains. The figure shows the ICIP averaged for each time instant

over all pair combinations of spike trains. Because the spike trains have constant firing

rate, the time constant of the decaying exponential convolved with the spike trains was

constant and chosen to be r = 2 ms. Also, in the bottom plot the average value of the

mean ICIP is shown. This was computed in 25 ms steps with a causal 250 ms long sliding

window. To establish a relevance of the values measured, the expectation and this value

plus two standard deviations are also shown, assuming independence between spike trains.

The mean and standard deviation, assuming independence, are 1 and V/(r + 1)

respectively (see Appendix for details). The expected value of the ICIP when synchrony






















H IIIII IIII I IIII III II I HI I I IH I I I I r I HIIII NI I HIIII 1H I I IIII
S I III If I I I I I I I
1 I I IH I I I I 1 I 11H I [I 11III
I4I I H 1 11l I I II I N
I1 1 11 I II II III l I II
I I I I I I II I I I
1 III I I III IH I I I HH IHTI II T IH
HH I III I II I IV I I I Ill I II$ l I I fl l l I I I I llll
I l I I Il I l I l ll I ll ll I I I I J i 111l l ll l l ll II l ll l l l l I h ll l ll I I ll I I I II H


- m7 ifi n


I l I II II III I I I 1 1
I I I II
I I ll I


I I 1 1
III


6
5
4
-3
-2



10
0
0 1 2 3 4 5 6 7 8 9 10 11
Time (s)


Figure 3-3.


Analysis of ICIP as a function of synchrony. (Top) Level on synchrony
specified in the simulation of the spike trains. (Upper middle) Raster plot
of firings. (Lower middle) Average ICIP across all neuron pair combinations.
(Bottom) Time average of ICIP in the upper plot computed in steps of 25ms
with a causal rectangular window 250ms long (dark gray). For reference, it
is also di-pi' ,iv ,1 the expected value (dashed line) and this value plus two
standard deviations (dotted line) for independent neurons, together with the
expected value during moments of synchronous activity (thick light gray line),
as obtained analytically from the level of synchrony used in the generation of
the dataset. Furthermore, the mean and standard deviation of the ensemble
averaged CIP scaled by T measured from data in one second intervals is also
shown (black).


0

0


.5
-C










00
00
00
(3.












00
00
00
00









among spike trains exists is given by 1 + F/(2rA), with A the firing rate of the two spike

trains, and is also shown in the plot for reference.

In the figure, it is noticeable that estimated synchrony increases as measured by ICIP.

Moreover, the averaged ICIP is very close to the theoretical expected value and is typically

below the expected maximum under an independence assumption as given by the line

indicating the mean plus two standard deviations. The d, 1 i-, .1 increase in the averaged

ICIP is a consequence of the causal averaging of ICIP. It is equally remarkable to verify

that (scaled) CIP matches precisely the expected values from ICIP as given analytically.

3.4.2 Mirollo-Strogatz Model

In this example, we show that ICIP can quantify synchrony in a spiking neural

network of leaky-integrate-and-fire (LIF) neurons designed according to [43]1 and

compare the result with extended cross-correlation for multiple neurons. This is the

simplest pulse coupled network that was proven to be perfectly synchronized from almost

any initial condition (Fig. 3-4). The synchronization is essentially due to leakiness and the

weak global coupling among the oscillatory neurons.

The raster plot of the network firing pattern is shown in Fig. 3-4. There are two main

observations: the progressive synchronization of the firings associated with the global

oscillatory behavior of the network, and the local grouping that tends to preserve local

synchronizations that either entrain the full network or wash out over time. As expected

from theoretical studies of the network behavior [43, 46] and which ICIP depicts precisely,

the synchronization is monotonically ii,. i, -ii.- with a period of fast increase in the first

second followed by a plateau and slower increase as time advances. Moreover, it is possible



1 The parameters for the simulation are: 100 neurons, resting and reset membrane
potential -60 mV, threshold -45 mV, membrane capacitance 300 nF, membrane resistance
1 Mf, current injection 50 nA, synaptic weight 100 nV, synaptic time constant 0.1 ms and
the topology was all to all excitatory connection.


















,1 1,, ,, :',i,'i i l i i II 1 [ I I' 1 1 1
L' ', "' "' ["' "' "' "' "' "' 1 "' "' "' 1 "' "' I'
II I' 1 ^ ^l l '|''I ''''' ''; '
i : : ; :I I I I I ; I I I I I, I, I, I,

" 1" 1 1" 1" 1"1 1" 1" I I I i 1 iI I I I I I I I I I I





, i i, i, i, i, i, i, i i i i i i i i i i i i iI I
^ '^ y 'j y ',i ) '',i '',i ^ y i ''i''i''i' ', i ', i I','','','',i ,
I I I I I I I ''^Y ^ '^'^'^ ^


x 10-3


2.5

2

1.5


1.1 1.2 1.3 1.4 1.5
-1

-0


I I I


I I I


I I


I I


I I


I I


0 0.5 1 1.5 2 2.5
Time (sec)


3 3.5 4 4.5 5


Evolution of synchrony in the spiking neural network. (Top) Raster plot of
the neuron firings. (\l, !11 .) ICIP over time. The inset highlights the merging
of two synchronous groups. (Bottom) Information potential of the membrane
potentials. This is a macroscopic variable describing the synchrony in the
neurons' internal state.


Figure 3-4.


: lllllll lllllll N g lllllll










































0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Time (sec)


0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Time (sec)


Figure 3-5. Zero-lag cross-correlation computed over time using a sliding window 10 bins
long, and bin size ims (top) and 1.lms (bottom).









to observe in the first 1.5 s the formation of a second group of synchronized neurons which

slowly merges into the main group.

Since the model was simulated, we also have access to all the internal variables: the

membrane potential of individual neurons over time. Thus, we can compute the synchrony

of neurons in terms of membrane potential. Surprisingly, the information potential (IP)

of the membrane potentials reveals the same evolution as the envelope of ICIP, including

the plateau. The IP was computed according to (A-18) using a Gaussian kernel with

size 0.75mV.2 The IP measures synchrony of the neuron's internal state, which is only

available in simulated networks. Yet the results show that ICIP was able to successfully

and accurately extract such information from the observed spike trains.

For completeness, in Fig. 3-5 we also present the zero-lag cross-correlation over

time, averaged through all pairwise combinations of neurons. The cross-correlation was

computed with a sliding window 10 bins long, sliding 1 bin at a time. In the figure,

the result is shown for a bin size of 1 ms and 1.1 ms. It is notable that although

cross-correlation captures the general trends of synchrony, it masks the plateau and

the final synchrony and it is highly sensitive to the bin size as shown in the figure, unlike

ICIP (data not shown). In other words, the results for the windowed cross-correlation

show the importance of working in "continuous" time which is crucial for robust

synchrony estimation in the spike domain. Other methods relying on inning also

suffer from sensitivity to bin size, such as the ones mentioned earlier. For this reason,

these methods are limited and unable to achieve the same high temporal resolution as

ICIP. In addition, spike trains are generally non-stationary unlike some methods assume.

The conventional approach is to use a moving window analysis such that only piece-wise



2 The distance used in the Gaussian kernel was d(0i, Oj) = min (|10 Oj\, 15mV 10i Oj ),
where 0i is the membrane potential of the ith neuron. This wrap-around effect expresses
the phase proximity of the neurons before and after firing.









stationarity is necessary. The information theoretic framework of ICIP, and CIP, treats the

non-stationarity implicitly as a pdf estimation problem.









CHAPTER 4
CONTINUOUS CROSS CORRELOGRAM

4.1 Delay Estimation Problem

Precise time delay in transmission of a spike in the neural system is considered to

be one of the key features to allow efficient computation in cortex [15, 53]. For example,

it is crucial for coincidence detection of auditory signal processing [17]. One of the

effective methods for estimating the delay is to use a cross correlogram [54]. Cross

correlogram is a basic tool to analyze the temporal structure of signals. It is widely

applied in neuroscience to assess oscillation, propagation delay, effective connection

strength, and spatiotemporal structure of a network [28].

However, estimating the cross correlation of spike trains is non-trivial since they are

point processes, thus the signals do not have amplitude but only time instances when the

spikes occur. A well known algorithm for estimating the correlogram from point processes

involves histogram construction with time interval bins [48]. The inning process is

effectively transforming the uncertainty in time to amplitude variability. This quantization

of time introduces inning error and leads to coarse time resolution. Furthermore, the

correlogram does not take advantage of the higher temporal resolution of the spike times

provided by current recording methods.

This can be improved by using smoothing kernels to estimate the cross correlation

function from finite samples. The resulting cross correlogram is continuous and provides

high temporal resolution in the region where there is a peak (see Fig. 4-1 for comparison

between histogram method and kernel method.) In this paper, we propose an efficient

algorithm for estimating the continuous correlogram of spike trains without time inning.

The continuous time resolution is achieved by computing at finite time lags where the

continuous cross correlogram can have a local maximum. The time complexity of the

proposed algorithm is O(T log T) on average where T is the duration of spike trains. The

application of the proposed algorithm is not restricted to simultaneously recorded spike

trains, but also to PSTH and also other point processes in general.




















0 100 200
time (ms)


0


0 100 200 300


Figure 4-1.


-300 -200 -100 0 100 200 300


Example of cross correlogram construction. A and C are two spike trains
each with 4 spikes. Except for the third spike in A, each spike in A invokes a
spike in C with some small d,1 iv around 10 ms. B represents all the positive
(black) and negative (gray) time differences between the spike trains. D
shows the position of d, 1 ,l- obtained in B. E is the histogram of D, which
is the conventional cross correlogram with bin size of 100 ms. F shows the
continuous cross correlogram with Laplacian kernel (solid) and Gaussian
kernel (dotted) with bandwidth 40 ms. Note that the Laplacian kernel is more
sensitive to the exact delay.


300


-300 -200 -100 0 100 200 300


E


-200


200


BM









4.2 Continuous Correlogram

Two simultaneously recorded instances of point processes are represented as a sum of

Dirac delta functions at the time of firing event, si(t) and sj(t),

Ni
sit)= 6{- t-, (4-1)
nm=l

where Ni is the number of spikes and t' are the time instances of action potentials. The

cross correlation function is defined as,


Qt(At) Et [si(t)sj(t + At)], (4-2)

where Et [-] denotes expected value over time t. The cross correlation can be interpreted

as scaled conditional probability of j-th neuron firing given i-th neuron fired At seconds

before [55]. In a physiological context, there is a physical restriction of propagation delay

for an action potential to have a causal influence to invoke any other action potential.

Therefore, this d.l 1 would influence the cross correlogram as a form of increased

amplitude. Thus, estimating the delay involves finding the lag at which there is a

maximum in the cross correlogram (inhibitory interaction which appear as troughs

rather than peaks is not considered in this article).

Smoothing a point process is superior to the histogram method for the estimation of

the intensity function [30], and especially the maxima [56]. Similarly, the cross correlation

function can also be estimated better with smoothing which is done in continuous time

so we do not lose the exact time of spikes while enabling interaction between spikes at a

distance.

Instead of smoothing the histogram of time differences between two spike trains,

we first smooth the spike train to obtain a continuous signal [57]. We will show that

this is equivalent to smoothing the time differences with a different kernel. A causal

exponential decay was chosen as the smoothing kernel to achieve computational efficiency









(see section 4.3). Smoothed spike trains are represented as,


N 1 _i

mrl

where u(t) is the unit step function. The cross correlation function of the smoothed spike

trains is,

Q*(At)- Et [qi(t)qj( + At)]. (4-4)

Given a finite length of observation, the expectation in equation (4-4) can be

estimated from samples as,

1 Jo
Q*R(At) qi(t)qj(t + At)dt, (4 5)


where T is the length of the observation. After evaluation of the integral, the resulting

estimator becomes,
N.1 N a
m in 1
Q:ij(At) =CrT C-- (4-6)
m=l n=l
which is equivalent to the kernel intensity estimation [58, 59] from time differences using a

Laplacian distribution kernel.

The mean and variance of the estimator is analyzed by assuming the spike trains are

realizations of two independent homogeneous Poisson processes.


ij [L (At)] AXAAB, (4 7)

var(Q* (At)) 4 (4-8)


where AA and AB denote the firing rate of the Poisson process of which i-th and j-th spike

train, respectively, is a realization (see Appendix for derivation). Note that the variance

reduces linearly as the duration of the spike train is elongated. By removing the mean

and dividing by the standard deviation, we standardize the measure for inter-experiment









(AA)- At (AAt)
I I


... On-1 on 0n+1 0n+2 ...
I I
(AA- )- At (AAt-6)+

Figure 4-2. Decomposition and shift of the multiset A.


comparison:


Qi,(AtAt AAB) (4 9)


4.3 Algorithm

The algorithm divides the computation of the summation of continuous cross

correlogram into disjoint regions and combines the result. We show that there are only

finite possible local maxima, and by storing the intermediate computation results for

neighboring time lags, the cross correlation of each lag can be computed in constant time.

The essential quantity to be computed is the following double summation,

Ni Nj
Qij(At)e C (4-10)

The basic idea for efficient computing is that the summation of the exponential function

computed on a collection of points can be shifted with only one multiplication, i, exi+6

(i exi)e6. Since a Laplacian kernel is two exponentials stitched together, we need to
carefully take the regions into account.

Define the multiset of all time differences between two spike trains,


A { 0 t' -tt, m= 1,...,Ni, n= 1,...,N,}. (411)


Even though A is not strictly a set, since it may contain duplicates, we will abuse the set

notation for simplicity. Note that the cardinality of the multiset A is NiNj. Now equation









(4-10) can be rewritten as


Qij(At))= e t


Now let us define a series of operations for a multiset B C R and 6 E R,


{x x E B and x > 0},

{x x B and x < 0},

{xI y B and x y 6}.


(non-negative lag)

(negative lag)

(shift)


Since B can

be rewritten


be decomposed into two exclusive sets

and decomposed,


B+ and B-, equation (4-12) can also


Qij(At) e e 5 e- +
OEAAt 0E(AAt)+ 0E(AAt)
E + E
C -7+
OE(AAt)+ OE(AAt)

For convenience, we define the following summations


e o


(4-14a)


(4-14b)


Q,(At) = e (4-15)
OE(Ant)

Let us order the multiset A in ascending order and denote the elements as 01 < 02 <

.. < On < On+ < ... ON-Nj. Observe that within an interval At E (0 ,0,+1], the

multiset ((AA)')_At is ah-i- the same (see Fig. 4-2). In other words, if At = 0+i,

for a small change 6 E [0, 0~n+ On), the multisets do not change their membership, i.e.

((At))6 = (A(At_-))'. Therefore, we can simplify an arbitrary shift of Qf with single

multiplication of an exponential as,


QS(At- J)


E eT
tE(Aat-s)
tE (A~)
tE(AAt)


5E eT
tE((AAt))6s

C eTeI Q e(At)e'
tE(AA,)


(4-12)


(4-13a)

(4-13b)

(4-13c)


(4 16a)


(4 16b)








Thus, local changes of Qij can be computed by a constant number of operations no matter
how large the set A is, so that

Qi(At -6) QtAt 6) + Q (At 6) (4-17a)

= Qt(At)e, + Qj(At)e-. (4-17b)

If there is a local maximum or minimum of Qij(At 6), it would be where :- ( = 0,
which is,
6*-= (ln(Q (At)) ln(Q(At))) (4-18)

Also note that since the second derivative,

d2Q(A5) (Q(At) + Q (At)eo- > 0, (4-19)


Qij(At 6) is a convex function of 6 within the range. Thus, the maximum of the function
value is alv--, on either side of its valid range, only local minimum can be in between.
In principle, we need to compute equation (4-10) for all At E [-T*, T*] to achieve
continuous resolution, where T* is the maximum time lag of interest. However, if we only
want all local minima and maxima, we just need to evaluate on all At e A, and compute
the minima and maxima using equation (4-17b) and equation (4-18). Therefore, if we
compute the Q (On) for all 0, E A, we can compute 6* for all intervals (Os, n,+1] if a local
extremum exists. These can be computed using the following recursive formulae.

O0n+1 0n
Qj(O,+) = Q(0)e r + 1, (4-20a)

Q (Oni) = QS eOn 1. (4-20b)

In practice, due to accumulation of numerical error, the following form is preferable for

Q(+
Q+( ) = (Qt(O+l) + 1>) (4-21)









Initial conditions for the recursions are Q (01) = 1 a

pseudocode is listed as in Algorithm 1.

Algorithm 1 Calculate Qij
Require: T > 0, A /0, N= |A
Ensure: Qij(At) = teA e VAt E A
1: A sort(A) {O(NlogN)}
2: Q -(1) 1
3: Q+(N) = 0
4: fork = 1 toN 1 do
A(k+l)-A(k)
5: ed(k) = e
6: end for
7: fork 1 toN 1 do
8: Q- (k + 1) 1 + Q-(k) ed(k)
9: Q+(N k) (Q+(N k + 1) + ) ed(N k)
10: end for
11: for k = 1 to N do
12: Qij(A(k)) + Q+(k) + Q-(k)
13: end for


d Q j(ON)


0. The resulting


The bottleneck for time complexity is the sorting of the multiset A, thus the overall

time complexity is O(NiNj log(NiNj)).1 Note that the time complexity of straight

forward evaluation of equation (4-10) is O(NiNj) for each time lag At. Assuming

homogeneous Poisson process for individual spike trains, the average time complexity

becomes O(N* log N*) where N* = AAABT, T is the length of spike train, and AA

represents the average firing rate for the Poisson process. Note that the conventional cross

correlogram algorithm [48] has the time complexity of O(N*) on average.

4.4 Results

In this section, we analyze the statistical properties and demonstrate the usefulness

of the continuous cross correlogram (CCC) estimator compared to the cross correlation

histogram (CCH). The CCC is defined by the linear interpolation of equation (4-9)



1 It is possible to reduce the sorting to O(NiNj log(min(Ni, Nj))) using merge sorting
partially sorted lists. However, it is only a minor improvement in general.










14 Q 14 --
14 CCC, strength 0.05 14 CCC, length 1 s
12 \- CCH, strength 0.05 12 \ CCH, length 1 s
\ CCC, strength 0.1 CCC, length 10 s
10 a -CCH, strength 0.1 10 \ B -CCH, length 10 s


6 6-



--E e Ea 0-- O-Eal-f-f-f

Data length (sec) Connection strength
(a) Effect of spike train length (b) Effect of connection strength

Figure 4-3. Effect of the length of spike train and strength of connectivity on precision of
delay estimation. The precision is estimated by the standard deviation in 1000
Monte Carlo runs with kernel size 7 0.4 ms (or bin size h 1.96 ms). The
smaller standard deviation indicates higher temporal resolution.


between the possible maxima (but not the minima). In order to compare with CCC, CCH

is standardized in a similar way to equation (4 9) according to [60].

Since CCH is essentially equivalent to using a uniform distribution kernel (or a boxcar

kernel) and sampling at equally spaced intervals as opposed to the Laplacian distribution

kernel used in CCC, in order to make a fair comparison, we choose the kernel size (bin

size) of both distributions to have the same standard deviation. To be specific, if the time

bin size of CCH is h, then we compare the result to CCC with kernel size of 7 h

Since the histogram method is highly sensitive to bin size, we used the procedure of

optimal bin size selection of Poisson processes ----- 1 by [61]. The method is designed

for the estimation of firing rate or PSTH from a measurement assuming a Poisson process.

However, since the time difference between two Poisson processes of finite length can be

considered as a realization of a Poisson process, it is possible to directly apply to the

CCH.
kerne usdi Ci re omk arcmaiow hoetekre ie(i
size ofbt itiuin ohv h aesadaddvain ob pcfc ftetm
bin sieo C sh hnw opr h esl oCCwt enlsz fT=
Sic th hitga ehdi ihysniietobnszw sdtepoeueo
optma bi ieslcino oso rcse i-- I y[1.Temto sdsge
fo th siaino iigrt rPT rm esrmn suigaPisnpoes








CCH.al uswthkre ie- 04m o i ie .6m) h

























E)


0.5 1 1.5
Kernel size (ms
(a) CCC vs CCH


-CCC 8
CCH
CCH optimal 7
6
E
5
4


2
1-
0
2 2.5 3 0 0.5 1 1
) Kernel size (ms)
(b) Optimal T for CCC


Figure 4-4.


Effect of kernel size (bin size) of CCC (CCH) to the performance. The
connection strength was 5'. and the spike trains are 10 seconds long, i.e. 5
spikes are correlated on average. (a) Sensitivity of CCC and CCH on kernel
size for noise standard deviation 0.25 ms. The horizontal dotted line indicates
the performance when optimal bin size is chosen for each set of simulated
spike time differences. The median of the optimal bin size chosen (right) and
corresponding kernel size for CCC (left) are plotted as vertical dashed lines.
Note that CCC is robust on kernel size selection and performs better than
CCH. (b) For different standard deviations of jitter noises, the precision is
plotted versus the kernel size r. Note that the optimal kernel size increases as
the jitter variance increases. For each point, 3000 Monte Carlo runs are used,
and the actual delay is uniformly distributed from 3 ms to 4 ms to reduce the
bias of CCH.


10


.5 2









4.4.1 Analysis

For a pair of directly synapsing neurons, the delay from the generation of an action

potential of the presynaptic neuron to the generation of an action potential of the post

synaptic neuron is not ah--iv- precise. Various sources of noise such as variability in

axon conduction delay, presynaptic waveform, probability of presynaptic vesicle release,

and threshold mechanism [62] effect the location, significance and width of the cross

correlogram peak. Furthermore, if the neurons are in a network, multiple paths, common

input sources, recurrent feedback and local field potential fluctuation can influence the

cross correlogram.

In this section, we model the timing jitter with a Gaussian distribution and analyze

the statistical properties of CCC and CCH on time delay estimation. A pair of Poisson

spike trains of firing rate 10 spikes/s were correlated by c iing a portion of the spikes

from one to another and then shifting by the delay with the Gaussian jitter noise. The

fraction of spikes copied represents the effective synaptic connectivity.

The total number of correlated spikes depend on two factors: the length of spike

train, and the synaptic connectivity. In figure 4-3, the precision of CCC and CCH are

compared according to these factors. The precision is defined to be the standard deviation

of the error in estimating the exact delay. Precision of both CCC and CCH improves as

the number correlated spikes increases in a similar trend. CCC converges to a precision

lower than half the jitter noise standard deviation (500 ps).

The optimal kernel size (or bin size) which gives the best precision depends on the

noise jitter level. In figure 4-4(a), CCC and CCH is compared across different kernel sizes.

In general, CCC performs better than optimal bin size and most of the bin sizes CCH.

As mentioned above, CCH is sensitive to bin size, but CCC is robust to the kernel size

for precision performance. Also note that the optimal kernel size for CCC corresponds

to equal median value of the variance optimal bin size selected (vertical dash lines).





















Figure 4-5. Schematic diagram for the configuration of neurons.


Increasing the jitter level worsens the best precision and increases the optimal kernel size

for CCC as shown in Fig. 4-4(b).

4.4.2 Examples

In this section, we demonstrate the power of CCC using two examples: the first

example uses synthetic spike trains from a simple spiking neuronal network model, and for

the second we use recordings from a cortical culture on a microelectrode array (\! 'A).

Two standard leaky-integrate-and-fire neurons are configured with 4 synapses, two

from neuron A to neuron B, and two for the other direction as illustrated in figure 4-5.

Individual synapses are static (no short/long-term plasticity), with equal weights and

generate EPSP (excitatory pc-i-ii iptic potential) with a time constant of 1 ms. Each

neuron is injected with positively biased Gaussian white noise current, so that they would

fire with mean firing rate of 35 spikes/s. The simulation step size is 0.1 ms.

As shown in figure 4-6, both CCH and CCC identifies the d-i-.' imposed by the

conduction delay, synaptic delay, and the delay for the generation of action potential by

noisy fluctuation of membrane potential. However, the time lag identified by CCC is more

accurate than that of CCH, since the temporal precision provided by CCH is limited by

the bin size and the jitter noise on delay, but for CCC, it is only limited by the jitter.

In other words, if there is no jitter, or a sufficient amount of spike timings has the exact

delay, then CCC is capable of quantifying the delay with infinite resolution.























continuous cross-correlogram


30
25

. 20
S15-


5
0


-20 -15 -10


35

30

25
. 20

S15
0


-20 -15 -10 -5


-5 0 5
Time lag (ms)
cross-correlation histogram


0
Time lag (ms)


15 20


5 10 15


Figure 4-6. Comparison between CCC and CCH on synthesized data.










continuous cross-correlogram cross-correlation histogram
30- 30
2.5 s 2.5 s
/ 5.0 s -5.0 s
25 5Os 25
S10.0s .lOOs

20 20

15 15

10 10

5 5

0 0
-10 -8 -6 -4 -2 0 -10 -8 -6 -4 -2 0
Time lag (ms) Time lag (ms)

Figure 4-7. Effect of length of spike trains. Comparison of continuous cross correlogram
(left) and cross correlation histogram (right) with different length of spike
trains (2.5, 5, 10 seconds). Estimated optimal bin size is 0.267 ms.


In figure 4-7, we illustrate the difference in performance of the methods according

to the length of the spike trains. When the spike trains are only of length 2.5 seconds,

the CCC has significantly lower time resolution where no spikes had that time difference,

yet maintaining the high resolution in highly correlated peaks. In contrast, the CCH is

uniformly sampled regardless of the amount of data. The non-uniform sampling gives

significant advantage to CCC when only a short segment of data is available.

To test the method further, spike trains recorded in vitro were used. We recorded

electrical activity from dissociated E-18 rat cortex cultured on a 60 channel microelectrode

array from MultiC'l ,,ii, I Systems [63]. For a particular pair of electrodes, specific d,--.1-

were observed as shown in Fig. 4-8. Those d4-1 i- are rarely observed (3 to 5 times through

5 to 10 minutes of recording), however the precision is less than 2 ms which makes it

significant in CCC. The d- 1 persisted at least 2 d ,i-, and many more interaction d,--.1-

were observable as the culture matured. As observable in the CCH analysis, it is almost

impossible to detect the d, 1 ,i-, and their consistency.

















10


8


6-


4-


2-
-80


60 80


0
time lag (ms)


Figure 4-8.


CCC (top) and CCH (bottom) of 7 DIV (d-,v in vitro) and 9 DIV cortical
culture recordings. Spike trains from two .idi i:ent electrodes are analyzed. On
7 DIV, CCC shows two significant peaks and they are also observable on 9
DIV, and some non-significant spike time differences corresponds to peaks on 9
DIV (marked with arrows). In contrast, CCH this structure is difficult to note.
The optimal bin size is 3.8 ms for 7 DIV and 3.3 ms for 9 DIV data. The total
recording time is 350 seconds for 7 DIV and 625 seconds for 9 DIV.


-60 -40 -20 0 20 40
time lag (ms)









Note that the d.1-v 1l are much longer than the expected conduction time which

is estimated to be in the order of 2 ms for conduction speed of 100 pm/ms [64]. One

possible mechanism would be a rarely activated chain of synaptic pathway from a common

source neuron with different d-.1 i-. In contrast to a recent study by [65] where the delay

between two channels is estimated with a single approximated Gaussian distribution with

relatively large variance, we observe multiple d,-!v between channels.

4.5 Discussion

We proposed an estimator of cross correlogram from an observation of a point

process, and provide a efficient algorithm to compute it. The method utilizes the fact that

there are more samples where the correlation is stronger. Thus, computing the continuous

correlogram at the lags of samples provides non-uniform sampling advantageous for

estimating the precise delay. Unfortunately, this non-uniform sampling is disadvantageous

for inhibitory relations, therefore only positively related d-1-, 1 can be accurately

estimated. To achieve computational efficiency, the algorithm is limited to the use of

Laplacian distribution as the kernel. However, it has been shown that the bandwidth

(kernel size) is more important than the shape of the kernel for the performance of

intensity estimation [30].

The only free parameter is the kernel size which determines the amount of smoothing.

Unlike the conventionally used histogram method, the proposed method is robust on

kernel size, however, the optimal kernel size depends on the noise level of the delay. In

a biological neuronal network, the noise level may depend on which path the signal was

transmitted. Therefore each peak of the correlogram may have different amount of noise.

We -i.-.i- -1. I' the use the optimal bin size for histogram as a guideline for the kernel size

selection.

The continuous cross correlogram can be viewed as a generalization of the cross

information potential where the correlation is interpreted as similarity (or dissimilarity)

between spike trains as we discussed in chapter 2. The proposed algorithm can be used









to find the similarity between two spike trains over continuous time lags. However, due to

accumulation of numerical error, the algorithm has to be non-causal (see equation (4-21)).

This prevents the algorithm to be used as an online filter to detect certain spike train

patterns, while offline analysis can still be done.

The proposed algorithm is not limited to cross-correlations. It can be directly

applied to smooth any type of point processes histogram, such as PSTH. However, one

alv--i,- has to be cautious when the underlying process is highly non-stationary. Various

non-stationarities can cause peaks in the correlogram [66].









CHAPTER 5
CONCLUSION

5.1 Summary of Contribution

The techniques presented here are based on smoothing spike trains with a continu-

ous kernel which preserves the time resolution while obtaining a continuous signal. We

demonstrated the usefulness of Cauchy-Schwarz divergence as a metric for smoothed spike

trains when spikes can be missing. The CS divergence is related to the similarity measure

CIP which is the inner product of the smoothed spike trains in L2. ICIP, the derivative

of CIP, is proposed as an instantaneous synchrony measure and extended to ensemble

average. Finally, time lag is incorporated into CIP to obtain a cross correlation function

of spike trains. All three algorithms can be computed efficiently depending only on the

number of spikes, without approximation, and independent of the sampling rate.

5.2 Potential Applications and Future Work

Given a similarity (or dissimilarity/divergence) measure with efficiently computable

closed form, the possibilities are endless. Clustering, classification, system identification,

and adaptive filtering can be applied to spike trains. We have some preliminary results

on stimulus to response mapping and stimulus estimation from response in a dissociated

cortical tissue culture, and willing to apply the techniques to various experiments.

In neuroscience, connectivity estimation, delay estimation, and identification of

synchronous neural group would be the most obvious applications. Correlation of

synchrony and attention or behavior would also be interesting. In a more engineering

perspective, detection of seizure, building a liquid state machine from living tissue, and

study of synchrony dynamics in pulse coupled oscillators seem to be promising. Finding

valid delay subnetwork [67] and polychronous group of neurons [34] may also be possible

by using CCC.









APPENDIX A
BACKGROUND

A.1 Point Process

Point process is a statistical random process where events (points) are distributed

over a continuous space. Typically the magnitude of the event is ignored and only the

position (time) is described (otherwise it is called a marked point process). Distribution

of trees in a mountain, rain drops in space, earthquake instances over time, and action

potentials in a spike train are examples of point processes.

In this section, we will introduce some notation and definitions of point processes1

Point process is built up from counting random variables, which maps sample space to a

natural number that represents the number of events in a certain space.

Definition 1 (Counting Process [51]). Let 2 be the sample space consisting of realization

of points w {Xl, x2,...} E 2. We /. /;,.': the counting process N(A : w) as


N(A : a=)- IA(x,),


where IA(X) denotes the set characteristic-function of A,


IA(X) = A,
0, x A.

By taking the derivative of a realization of a counting process, a realization of a point

process can be obtained. In this case, the realization of point process will consist of delta

functions at on the locations of events. Spike trains will be treated as a realization of a

point process for the rest of the thesis.

The simplest type of point process is the Poisson process. In Poisson process,

each event is independent and the probability of firing at a location (time) is completely



1 Some materials of this section is replicated from Snyder [51]









determined by the functional parameter A(t). When A(t) is differentiable, we call the

derivative A(t) the intensity function.

Definition 2 (Temporal Poisson process). A temporal Poisson process for times t > to is

a co"i,,li:u, process {N(t) : t > to} with the following properties:

1. Pr[N(to) =0] = 1;

2. for to < s < t, the increment N(s, t) = N(t) N(s) is Poisson distributed with

parameter A(t) A(s),


Pr[N(s, t) = n]= (A(t)- A(s)) e-(A(t)-A(s))


where n is a nonnegative integer, and A(t) is a finite-valued, nonnegative, nonde-

creasing function of t;

3. {N(t) : t > to} has independent increments.

Proposition 1. Let [ti, u)i=1,2,...,k be disjoint intervals on [to, oc). If {N(t) : t > to} is a

temporal Poisson process, then the independent increments implies,
k
Pr[N(ti, ui) ni, N(t2, u2) 2, N(tk, Uk) ] Pr[N(ti i) ni].
i=i

We assume that the spike trains are realizations of Poisson process. The intensity

function A(t) corresponds to the underlying (instantaneous) firing rate. This assumption

is based on statistics observed from in vivo systems and frequently considered as good

approximation [68]. The simple formulation of Poisson process enables analytical analysis

for the tools (some of which are presented in the Appendix).

A.1.1 An Alternative Representation of Poisson Process

For any interval, there are only finite number of events in a Poisson process. The

statistics in the interval can also be described by a combination of two random variables.

The first random variable represents the distribution for the number of events in the








interval which follows the Poisson distribution f.


f(k, A) = k! (A-l)

where k is the number of events, A A(T) A() is the average intensity function where
T is the length of the interval, is the factorial operator. The second random variable X
represents the distribution of the finite events (points) over the interval. This distribution
is obtained by normalizing the intensity function A(t) over the interval to make it a
pdf [69]:

fx(x) T- )d (A-2)
0o A(t)dt
The equivalence can be shown by the joint distribution of the points and considering all
the possible order (order statistics) [51, 52].
A.1.2 Filtered Poission Process
Smoothed spike train is a form of shot noise, and if the underlying point process is
Poisson, we can get the moments analytically using the characteristic functionals.
Theorem 2 (Characteristic functional for a filtered Poisson process [51]). Let a Poisson
process with 0',/. ,iil function A(t) 1. I.' on t > to be filtered by a causal linear
filter with impulse response h(a, r; u), resulting in a continuous time '.:,'.r,1 y(t). The
characteristic functional of y(t) is 1, i ,.,, ,I as,

(jv) = E exp[j o y(a)dv(a)] (A-3)

has the evaluation (A-4)

-exp{j ArE exp[j h(a, 7; u)dv(a) l]d } (A-5)
ito
where v(.) is w,:,' function with

j J f(a, 3)dv(a)dv(3) < o









where


min(a,83)
f(a, /) A(r)E [h(a, 7; u)h(3, T; u)] dr
ato

+ ArE [h(a, 7; u)] dr ArE [h(3, 7; u)] dr.
to to

See [51] page 219-220 for proof.

We can choose the form of v(.) to be,


0, to < t
v(oJ) = (A-6)
a, t < a < T.

Then, the characteristic function for y(t) becomes,


1 ,,,-ja) exp{f A(T)E [ejah(t" 1] dr}. (A-7)
Jto

Therefore the n-th cumulant 7n for y(t) can be derived as



to
fA)E [h-(t, ,; u)] dT. (A-8)

There are v--i- to get the actual pdf [70-72], however, the closed form is highly

complicated.

The following theorem supports that the correlation function of the smoothed spike

train is meaningful under the assumption of Poisson spike trains.

Theorem 3 (Campbell's Theorem [51]). Shot noise of a homogeneous Poisson process is

wide sense stat:- ii

Furthermore, the power spectral density of the smoothed spike train is same as

exciting the system (filter h) with white Gaussian noise [51].

A.2 Mean Square Calculus

Statistical signal processing is based on second order 'i, .-, ;/ or mean square calculus,

of the random process. In this section, we give a brief introduction to the theory.









First, we introduce L2, the space of all random variables with a finite second order

moment.

L2 = {X I E [ X12] < 00} (A-9)

It can be shown that L2 is a Hilbert space [73]. In this space, the order of limit and

expected value operator can be exchanged up to second order.

Definition 3 (.\!, i,-square continuity). Let X(t) be a stochastic process 1. I.,., on the

real line. X(t) is continuous in mean square sense at t if

limE [ IX(t + h) X(t)2] 0

Definition 4 (i. ,i-square differentiability). The random process X(t) is mean-square

differentiable if the following limit exists

X(t + h) X(t)
lim
h-O h

Proposition 4. A random process with well /;.,, I correlation function belongs to L2.

Note that mean square error is equivalent to Euclidean distance.


(x(t) y(t))2dt ((t)2 + y(t2) dt 2 J x(t)y(t)dt (A-10)


A.3 Probability Density Estimation

Estimating a pI, ,,/l.::;i/ I, ,.;:// function (pdf) from a set of samples (observations)

has been one of the fundamental problems in statistics [74]. Parametric methods

assumes a distribution and fits the data to the distribution, which is usable only if the

assumed model is at least approximately correct. On the other hand, nonparametric

approach makes a milder assumption, usually in the form that the pdf is continuous. One

of the widely used nonparametric method is the histogram. The other is Parzen window,

or otherwise known as kernel density estimation [59]. These can be motivated from the









empirical cumulative distribution function F(x) [74]:


(x) number of samples in (x h,x + h) (A 11)
total number of samples

Plugging in equation (A-11) to the definition of pdf,

dF(x) lim F(x + h) F(x h)
f(x) d lim (A 12)

can be written in the following form:


f(x') A K i, (A-13)

where N is the total number of samples, h is the bandwidth, and K is defined as,


K, if 1 < < 1,
KIx) = (A-14)
0, otherwise.

This is the histogram method if x is evaluated for every non-overlapping interval of size h.

Note that equation (A-14) is a uniform distribution. By allowing any pdf as a pl './., :l.:i;l
I/, ,.:1,; estimation kernel, we can define the kernel density estimation. It had been shown

that under all nonnegative kernels with compact support, Epanechinikov kernel is optimal

for the .,i-mptotic mean integrated squared error (AMISE), however Gaussian kernel and

other kernels are widely used [74].

The free parameter h, the bandwidth, determines how smooth the estimate will

be, and in general depends on the number of samples in the region. When using fixed

bandwidth, AMISE provides optimal bandwidth which balances bias and variance of the

estimate [74]. There have been number of extension to the fixed bandwidth kernel density

estimation methods [75]. The general idea is to decrease the bandwidth in the region

where there are more samples, and use large bandwidth where there are less samples.









Kernel K(x)
Epanechinikov (1 a2)
Uniform 1
2
Triangle 2 X
Gaussian e 2
Laplacian e- lx
Table A-i. Various probability density estimation kernels. Gaussian and Laplacian, has
infinite support, and the other kernels have [-1, 1] as the support.


One of the drawbacks of kernel density estimation is the boundary bias. When the

support of the pdf is finite, the infinite support kernels will underestimate, and even finite

support kernels will leak some of the density to outside of its support.

Since kernel density estimation provides relatively accurate continuous pdf estimation

with a finite summation, a set of algorithms that is based on pdf can be written in efficient

manner. Information theoretic 1. iiiir.i- a framework of signal processing with information

theoretic cost function, combines Renyf's quadratic entropy with kernel density estimation,

and nonparametrically estimates entropy without approximations [37, 76].

A.4 Information Theoretic Learning

CIP is strongly related with information theoretic learning framework [77]. For a

random variable X with a pdf f(x), R6nyi's quadratic entropy is defined as, [78]


HR = -log f2(x)dx log E [f(x)]. (A-15)

The argument of the logarithm,


Vx f2(x)dx E[f(x)], (A-16)


is called the ,:f.', i,,r.:,n potential (IP) [77].

As mentioned in A.3, R6nyi's quadratic entropy can be estimated efficiently with

kernel density estimation. Let {xi : i = 1,..., N} be a set of N i.i.d. samples of a random









variable X. Then, the pdf of X can be approximated non-parametrically by,

SN
f)(E, N (A 17)
i=1

where Kt.,,t(, *) is the kernel. Substituting this estimator in the above definition of the

information potential, equation (A-16), yields,

1 N N NN
vx N = K /,I(Xit(X,XiJ,,t/x,Xj)dx 2Z Z{ (ix j). (A-18)
i=1 j=1 i=1 j 1

where K(xi, xj) =f K_ ,.,,(x, xJ)i,,,i(x, xj)dx. Note that we are estimating entropy of

a continuous random variable directly with sums of kernel evaluations and without any

approximation.

Let fi(x) and fj(x) be the pdfs of random variables Xi and Xj, defined on the same

probability space. A distance between the pdfs of the two random variables can be defined

in the space of the distributions with the Cauchy-Schwarz (CS) distance, Ics,


cs = log dt t ) log (A-19)
f fi(t)fj(t)dt ij

where Vi is the information potential of the ith random variable [77]. It is important to

remark that Ics is in fact approximating the Kullback-Leibler divergence [79] between

the two pdfs; however, a significant advantage is the ease of computation of this measure

using the information potential. Notice also that in the argument of the logarithm

the numerator contains the normalizing terms. In other words, the behavior of Ics

is determined by the denominator term, Vj, appropriately called cross information

potential (CIP). Much like the IP, the CIP expresses a potential due to interactions

between particles, but from different random variables. Because the CIP negatively affects

the CS divergence, it is in effect measuring the similarity between the two distributions.








A.5 Reproducing Kernel Hilbert Space
In kernel methods, the concept of reproducing kernel Hilbert space (RKHS) is often
mentioned. The smoothing of a spike train can be seen as applying a kernel method, and
projecting the spike trains to an RKHS. This can be seen from the fact that the Laplacian
distribution is a positive definite kernel. Indeed we are using a subspace of L2, which
consist of the smoothed spike trains and their linear combination, and also is an RKHS.
Being an RKHS provides the kernel trick, so that the algorithms can be efficient.
Definition 5 (Inner Product). The inner product of x,y E V where V is a vector space is
a I'n'l':',j (xy) : V x V -i K such that,
(11) Vx V, (xlx) > 0 and (xlx) = 0 x = 0
(12) (xly) = (6, )
(13) Vx, y, z V, Va, b e K, (ax + '",, :) = a(xlz) + b(ylz)
Definition 6 (Norm induced by inner product). For a Kfield-vector space V equipped
with an inner product, the norm of a vector is 1. i/.,/. as, I x| = .(xx)

Proof.

A, (AxlAx) A (xlx) A x (A 20)

IIx + yl < Ilx|| + Ilyll by lemma 5 (A-21)

I|x| = 0 = x x 0 by (I1) in definition 5 (A-22)



Lemma 5. IIx + Y12 = lx I12 + 2Re(xly) + I||Y|2

Proof. IIx + y112 =( i + I+ y)= Ilx\I2 + (1' > + (xly) + I/112 2

Lemma 6 (Cauchy-Schwarz inequality). I(xly)| < |x|| ||Ily| .









Proof. Suppose Ilyl / 0, for a A E K,


0 < IIx- Ayl = (x- ,I Ay)

(xlx) A(xly) A(II,) + IA12 (y>).

Let A = (xly)/(yy),

0 < (xlx) (i) (xl>)/(yy) (xly)(iI, )/(yy) + (xly)( ii, )/(yy),

which is equivalent to,

(xly(i> I 0) < (xlx)>(I'/).



Definition 7 (Cauchy Sequence). A sequence of elements x, indexed by n E N of a metric

space with metric d(-, -) is a C',,. I,;i sequence if for all E > 0, there exist a N E N such

that for all n, m > N, d(xn,, x) < E.
Definition 8 (Complete Metric Space). A metric space is complete if every C'.,,. I,

sequence converges to a point in the space.
Definition 9 (Hilbert Space). A vector space V complete under the norm induced by

inner product is a Hilbert space.

Definition 10 (Reproducing Kernel Hilbert Space (RKHS)). There exist a kernel

K : V x V K, such that, for all f E H, the reproducing property holds:


f(y)= (f(x) K(x, y)).

Remark 1 (Linear Operator View). RKHS is a sub-Hilbert space of L2. In particular,

RKHS is the +1 eigenvector space of the kernel. In other words, we are restricting the

general Hilbert space L2 to a smaller space where the reproducing j '. o holds. Also note

that L2 is not an RKHS.









Definition 11 (Positive Semi-definite Kernel). A positive semi-l/. f,'.:/I kernel is a

function on X x X with the following j'-* '" /I; For all natural number n, for all xl,... ,x,

in X, and for all ac,..., ac in a real or complex,
n n
SYaiajK(xiXj) > 0.
i= j 1

Theorem 7 (Moore-Aronzajn Theorem [80]). Given a -,iiiiiim ii.. positive semi-/, fi,..:I:

kernel K, There exist a unique RKHS H with K as the reproducing kernel.

Proof. Let A be the set of all functionals of the form )(-) = K(., i). Define the linear

combination of the functionals.


Vf, g -e Z R Va, bR Vx I (af + bg)(x) = a(f(x)) + b(g(x)) (A-23)

Let B be the vector space spanned by A. Now let us define the inner product (.].) and

norm I| fl f= of B.



Vf, e A (f g) (Yaf I(.) ag4(-.) (A-24)

iEZ jEl
5af ag (.)j 1K(.)) (A-25)

554YaK iKj( (A-26)
iEl jEl

To ensure that the inner product is well-defined, two different representation of f, g E B

should lead to the same inner product, which is obvious.

Let us complete the space by including all limits of Cauchy sequences {f, n E N, f, E

B} and denote as which is a Hilbert space. Note that B is a dense subset of '.









The reproducing property of 7HK is immediate.


(Ks(,f ),f) = (Ks af K ())
IELf

af(Ks, (.), Ks,,(.))
IELf

afK (Si, Si,)
IELf

1 aK(S,, S)
IELf

f(Si)









APPENDIX B
STATISTICAL PROOFS

To assess the significance of the correlation, it is necessary to know the probability

distribution of the estimator given the null hypothesis (independent Poisson spike

trains). However, instead of calculating the complicated closed form of the distribution

for Q* (At), we derive of mean and variance of the estimator Q-(At), and assume

Gaussianity. For the time inning with sufficiently small bin size case, Palm and coworkers

have derived statistics for the histogram of a Poisson process [60]. The analysis can easily

be applied to CIP by multiplying the normalization factor.

Let Q(A, T) be the class of all possible homogeneous Poisson spike trains of rate A and

length T. The probability of having a realization = {t',t ,... ,t'} of 2(AA,T) is,


P[= QI\AA,T] P[N(T)=

= P[N(T)

(AAT)N


(AAT)N
N=,e


(Bi-)

(B-2)

(B-3)


(B-4)


N t' ti t ti ....^ ti 1


- AT 1 A i ,N

T 1l 2 2 NiN


11 T Ni
m= I










The expected value of the estimator for all possible pairs of independent spike trains is,


S P[Qi, QjAA,AB, T1I'.(At)didQ j

S __ A -AATABB_ -ABT
NOO ONj0 0 N


Ni Nj t A
2T It m n
rT 1
m=In=i


1
27-T


(AA+AB)T 7 A AB
MNO=O N=0


T T- T 00100 e
m n-1 n- oo-oo


1
27T


It m t -Atl
I~ga


dt"dt'
rn n


/ 00


t d m jAt
Sdti dt3
Uin in


E [ij] Q (At)


(B-5a)


dtidt dt dt i dt .. dt3'
1..dtm 1t2dtj


(B-5b)


(AA+A )T V (AAT)Ni (ABT)N NiN
AN-! N! T2
N,=O N= 0


(B-5c)


(B-5d)









Let us evaluate the integral first, from the symmetry of tI and t' we can assume At > 0

without loss of generality.

jT T |ti --t At
j e dt dt' (B-6a)
J0o J0
A tt tn at -



+ t e dt dt
At 0
T T 4n -t -At
+ C \ e- d --1 (B-6b)
IAt Jt -At


t i -T-At t At A
-T2 e-- -- e d


+dt -r I t) dt' (B-6c)
'T f't \t
2 At-T-At 0 T At At-At 0-Af
_T Ie e C e C + C I


+ T(T At) T2 (-e t +

2 T TT At At At) T(T
- T2 -e + (T
2(C T_ t -T At
2-(T- At) +T 2(e +e

27(T At) + 0(T2).


- At)
At- 2e
2e-T)


(B-6d)

(B-6e)

(B-6f)


Approximating AA


, and substituting the integral to equation (B-5d) gives,

I* 2 2(T At) + O(2)
E [I ij] Q(At) AAAB
S27-T


(B-7)


where O(r2) are the terms with order of 72 or higher. Assuming 7
equation (B-7) can be approximated by AAAB which is the desired value.











Now let us evaluate the second-moment of the estimator.


E [ij] Q(At)2


J J[Qij\\ AA, AB,T (1' t 2didQaj


T A -AATA -ABT
NNNe N0eJ
Ni=0 N =0


1
4,2T2


N Nj N, N1

p=l q1=l r=l s=1


dti dt .. dt" dt'dt *** dt'
21i 122 Nj


o0000 NNi Nj
N\AT -XNBT
N T=ONN,=0


1 1
1 TN2T N, 1
4,2T2 T4


jT jT .T jT N, N N NC

0o Jo Jo r=1 s=

ddttdt dt3


t -t0 -At|


(B-8a)


p q tq Atf
p


tZ- t3 At


(B-8b)


tZ- t3 At
C


(B-8c)









Let us consider the integral part first.


T T T T Ni NJ Ni NJ At
Ip=14 q= =J1 e
p lq r ls 1p


-ti t'~ At
dt dtS W dtW
p q r


\ti -j T rt
D t'q At .t
I dt^dt 'q \


JO T
0


(B 9a)


i A
1^ t f


dt, dt'


-dt dt3


e C e 7 dtidt dt
|t -t -A | 4 ^ t| .


t- t -Ati Itr AI
p q))


At) + O(72))2


dtP dt3 dt'
q s


+ 2 (r(T At) + O(r2))

N, Nj (N, t) 2 + eT) + 0(731
+ P ( (2T(2 + e )) + O()

N+ NT (N -1) ()2T(2 + _)) + % 7)
+ 3 (2T(2


(B-9b)


(B-9c)


T
By assuming 7 < 1 < T, we can approximate e 0, 0(r3) 0. And we further

approximate AA = and N, 1 Ni. These approximations lead to,

S[j] Q(t) 4 {(A )(2(T1t))
E [1j] .(At)2 2 2 A B)2(27(T At))2
7,1 4,T2 T 2 AAB2T


+ (AAAB)(T(T At)) + (AAAB)(AA + A(42 T)

(A T At2 AAAB T -At
(AAB)~IT 4)7T T

+ (AAB)(AA + AB).
T


(B-10)


Finally, the variance of the estimator is given by


AAAB(T -At)
A) 4-T2


SNi Nj T T
p=l q=l r1p s1
N-p Nq1jTj 2 ti



p 1 q T rT rr

p=l q=l rlp 0


t N N T T T

p= q1= sq
N N,(N- 1) (N- 1)
P4 (27(T
T4


I i](Q, (At)


(B-t1)









APPENDIX C
NOTATION
Spaces

generic field

natural number

real field

d-dimensional Euclidean space

Hilbert space [p. 59]

index set space for spike trains [p. ??]


{t ::m= ,..., N}

si(t)

h(t;r)

qj(t)



A (t)

X,Y,Z

T

X(t), Y(t), Z(t)

N(t, s), N(t)

fx

K


Kr,,,


K


General notation

spike train as a set of spike timings [p. ??]

spike train as a function over time [p. ??]

impulse response of a linear filter

filtered (or smoothed) spike train

estimation of a general function g

intensity function of a Poisson process [p. ??]

random variables

random variable of time

random processes

counting process

probability density function of X

generic kernel

pdf estimation kernel

generic kernel with kernel size parameter r

CIP kernel, or reproducing kernel of a Hilbert space











Ex [g(x)]

(xly)
11)11
I1.
x(t) y(t)


Operators

expectation of g(x) over X
inner product [p. 58]
norm of a vector
absolute value
convolution















APPENDIX D

SOURCE CODE



D.1 CIP

function V = cip(x, tau)
%V = cp(X, TAU)
% Return the Cross Informatzon Potentzal.
% If more than two neurons are provided average through all pazr combznatzons.

% X: Data, organized as a cell array, wzth each cell contaznzng an
% array of spzke tzmes (zn seconds)
% TAU: Kernel szze (zn seconds).


/ vzm: set ts=8 sts=4 sw=4: (modelzne) *
#include
#include
#include
#include
#include

void cipfunc(int N, double *x[], int nSpikes[], double tau, double *v);

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
mxArray *sts;
mxArray *stp;
int nSpikeTrain; /* number of spzke trains *
double **x; /* array of vectors wzth the
spzke tzmes (sec) */
int *nSpikes; /* array wzth the number of spikes
per spzke trazn */
double tau; /* exponentzal decay parameter *
double *v; /* output argument, CIP */
int i;


check znput arguments


if (nrhs != 2) {
mexErrMsgTxt("2uinputsuareurequired.");

else if (nlhs > 1) {
mexErrMsgTxt("Tooumanyuoutputuarguments");

if (!mxIsDouble(prhs[1]))
mexErrMsgTxt("TAUumustubeuauscalar");


get znput arguments


sts = (mxArray *) prhs[0];
if (mxGetClassID(sts) != mxCELLCLASS) {
mexErrMsgTxt ("Xumustubeuaucelluarray");
}

nSpikeTrain = mxGetNumberOfElements(sts);

if (nSpikeTrain < 2) {
mexErrMsgTxt("Atuleastutwouspikeutrainsuareneeded.");


nSpikes = (int *) mxMalloc(sizeof(int) nSpikeTrain);
x = (double **) mxMalloc(sizeof(double*) nSpikeTrain);

for (i = 0; i < nSpikeTrain; i++) {
stp = mxGetCell(sts, i);
nSpikes[i] = mxGetNumberOfElements(stp);
x[i] = mxGetPr(stp);


tau = mxGetPr(prhs1])[0] ;

/*
allocate output

plhs[0] = mxCreateDoubleMatrix(1, 1, mxREAL);
v = mxGetPr(plhs[0]);
memset(v, 0, sizeof(double));

/*
compute CIP

cip_func(nSpikeTrain, x, nSpikes, tau, v);


/**
















* Compute CIP of a set of spzke trains.

* Oparam N number of spzke trazns
* Qparam x array to pointers for spzke trains
* Oparam nSpzkes array to length for spzke trazns
* Qparam tau decay tzme constant for the exponentzal function
* wparam v computed CIP wzll be stored here, need to be preallocated
* author Antonzo Pazva
* Overszon $Id: czp.c 52 2007-01-03 16:55:26Z memmzng $

void cip_func(int N, double *x[], int nSpikes[], double tau, double *v)
{
int i, j; /* counters for spzke trazns */
int m, n; /* counters for spzke tzmes */
int lastStartIdx; /* zndex to start computing the exponentzal /
double maxT; /* maximum range of exponentzal to have non-zero value */
double aux; /* auxzlzary variable: holds CIP for each
pazr combznatzon */
double tmp;

maxT = tau 100;

*v = 0;
for (i=0; i<(N-l); i++)
for (j=(i+1); j aux = 0;
lastStartIdx = 0;
for (m=O; m for (n=lastStartIdx; n tmp = x[j][n] x[i][m];
if (tmp < -maxT) {
lastStartIdx++;
continue;


if (tmp <= maxT) {
aux += exp(-((tmp
} else {
break;


< 0) ? (-tmp) : tmp) / tau);


(2*tau (nSpikes[i] nSpikes[j]));
aux;


v (N (N) 2)
*v /= (N (N-1) / 2);


D.2 ICIP


/* vzm: set ts=8 sts=4 sw=4: (modelzne) /
#include
#include
#include


* Compute ICIP of a set of spzke trazns wzth changing TAU mode.

* Oparam N number of spzke trains
* Qparam sts array to pointers for spzke trains
* Oparam nsts array to length for spzke trains
* Qparam v computed ICIP wzll be stored here, need to be preallocated
* Oparam T total tzme (sec)
* Qparam dt tzme bzn szze (sec)
* @param BETA the parameter $\beta$ of ICIP (sec)
* Qparam FR_TAU the tzme constant for fzrzng rate estzmatzon


}
aux /
*v +=


function [v] = offline_icip(st, T, DT, FRTAU, BETA)
% [v] = offlzne_zczp(st, T, DT, FR_TAU, BETA)
SOfflzne version of ICIP computation. For onlzne evaluation directly use
Sonlzne_zczp.
%
SInput
% st: cell array contaznzng spzke trazns (seconds)
S T: total length of spzke trains (seconds)
S DT: tzme step szze (seconds)
S FR_TAU: the tau for fzrzng rate estzmatzon (I/seconds)
% zf FR_TAU zs zero, zt zs constant tau mode
S BETA: kernel szze for ICIP at average fzrzng rate 1 Hz (I/seconds)
% (BETA should be smaller than FR_TAU^2 for accurate estzmatzon)
SOutput:
S v: ICIP over tzme

SYou may want to use tr = O:DT:(T-DT); whzch are the start tzme for each bzn

SCopyrzght 2006 Antonzo and Memmzng, CNEL, all rights reserved
S$Id: offlzne_zczp.m 32 2006-12-09 18:05:57Z memmzng $

% Actually zmplementatzon zs now zn offlzne_zczp.c
















* author Memmzng Park
* aversion $Id: offLzne_zczp.c 51 2007-01-02 19:53:01Z memmzng $
*/
void offline_icip(int N, double *sts[], int nsts[], double *v, double T, double dt, double BETA, double FR_TAU)

double t; /* time *
int i; /* tzme zndex */
int j, k; /* spzke trazn zndex /
int *idx; /* spzke zndex per spzke trazn */
int NPair;
double EXP_FR;
double ONE_OVER_FR_TAU;
double ONE_OVER_BETA;
double *q; /* charge */
double *f; /* fzrzng rate *
int Nstep; /* number of tzme steps (bzns) */

ONE_OVER_FR_TAU = 1 / FR_TAU;
ONE_OVER_BETA = 1 / BETA;
EXP_FR = exp(-dt / FR_TAU);

idx = (int *) malloc(sizeof(int) N);
memset(idx, 0, sizeof(int) N);
q = (double *) malloc(sizeof(double) N);
memset(q, 0, sizeof(double) N);
f = (double *) malloc(sizeof(double) N);
memset(f, 0, sizeof(double) N);

NPair = N (N 1) / 2;
Nstep = (int) ceil(T / dt);

for(t = dt, i = 0; i < Nstep; t += dt, i++) {
for(j = 0; j < N; j++) {
f[j] *= EXP_FR;
q[j] *= exp(-dt f[j] ONE_OVER_BETA);
while(idx[j] < nsts[j] && sts[j][idx[j]] <= t) {
idx [j++;
f[j] += ONE_OVER_FR_TAU;
q[j] += ONE_OVER_BETA;



for(j = 0; j < N; j++) {
for(k = (j + 1); k < N; k++) {
v[i] += q[j] q[k];


v[i] /= NPair;


free(idx);
free(q);
free(f);



Compute ICIP of a set of spzke trains wzth constant TAU mode.

Qparam N number of spzke trains
Oparam sts array to pointers for spzke trains
Qparam nsts array to length for spzke trains
Oparam v computed ICIP wzll be stored here, need to be preallocated
Qparam T total tzme (sec)
Oparam dt tzme bzn szze (sec)
Qparam TAU the tzme constant for the expnentzal (or Laplactan)
Author Memmzng Park
aversion $Id: off Ine_zczp.c 51 2007-01-02 19:53:01Z memmzng $

void offline_icip_const_tau(int N, double *sts[], int nsts[], double *v, double T, double dt, double TAU)

double t; /* time */
int i; /* time zndex /
int j, k; /* spzke trazn zndex */
int *idx; /* spzke zndex per spzke trazn /
int NPair;
double EXPTAU;
double *q; /* charge */
double *ONEOVER_TAU_F; /* (1 /(tau*fzrzng rate)); constant *
int Nstep; /* number of tzme steps (bzns) */

EXP_TAU = exp(-dt / TAU);

idx = (int *) malloc(sizeof(int) N);
memset(idx, 0, sizeof(int) N);
q = (double *) malloc(sizeof(double) N);
memset(q, 0, sizeof(double) N);
ONEOVERTAUF = (double *) malloc(sizeof(double) N);
memset(ONEOVERTAU_F, 0, sizeof(double) N);

for(k = 0; k < N; k++) {
ONE_OVER_TAU_F[k] = (1/TAU) (T / nsts[k]);
}

















NPair = N (N 1) / 2;
Nstep = (int) ceil(T / dt);

for(t = dt, i = 0; i < Nstep; t += dt, i++) {
for(j = 0; j < N; j++) {
q[j] *= EXP_TAU;
while(idx[j] < nsts[j] && sts[j][idx[j]] <= t) {
idx [j++;
q[j] += ONE_OVER_TAU_F[j];



for(j = 0; j < N; j++) {
for(k = (j + 1); k < N; k++) {
v[i] += q[j] q[k];


v[i] /= NPair;


free(idx);
free(q);
free(ONE_OVER_TAU_F);


void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
int nSpikeTrain;
mxArray *sts;
mxArray *stp;
double **st;
int i;
int *nst;
double T, DT, FRTAU, BETA;
double *v;

if (nrhs != 5) {
mexErrMsgTxt("5uinputsuareurequired.");
} else if (nlhs > 1) {
mexErrMsgTxt("Tooumanyuoutputuarguments");


sts = (mxArray *) prhs[0];
if (mxGetClassID(sts) != mxCELL_CLASS) {
mexErrMsgTxt("Theufirstuargumentushouldubeuaucelluarray");
}

nSpikeTrain = mxGetNumberOfElements(sts);

if (nSpikeTrain < 2) {
mexErrMsgTxt("Atuleastutwouspikeutrainsuareurequired.");
}

nst = (int *) mxMalloc(sizeof(int) nSpikeTrain);
st = (double **) mxMalloc(sizeof(double*) nSpikeTrain);

for (i = 0; i < nSpikeTrain; i++) {
stp = mxGetCell(sts, i);
nst[i] = mxGetNumberOfElements(stp);
st[i] = mxGetPr(stp);


if (!mxIsDouble(prhs[1]))
mexErrMsgTxt("Theuseconduargumentushouldubeuaudouble");
if (!mxIsDouble(prhs[2]))
mexErrMsgTxt("Theuthirduargumentushouldubeuaudouble");
if (!mxIsDouble(prhs[3]))
mexErrMsgTxt("Theufourthuargumentushouldubeuaudouble");
if (!mxIsDouble(prhs[4]))
mexErrMsgTxt("Theufifthuargumentushouldubeuaudouble");

T = mxGetPr(prhs[1])[0];
DT = mxGetPr(prhs[2])[0];
FR_TAU = mxGetPr(prhs[3])[0];
BETA = mxGetPr(prhs[4])[0];

plhs[0] = mxCreateDoubleMatrix((int) ceil(T / DT), 1, mxREAL);
v = mxGetPr(plhs[0]);
memset(v, 0, sizeof(double) (int) ceil(T / DT));


if (FRTAU == 0) { /* Constant tau mode */
offline_icip_const_tau(nSpikeTrain, st, nst, v, T, DT, BETA);
} else {
offline_icip(nSpikeTrain, st, nst, v, T, DT, BETA, FR_TAU);

















D.3 CCC

function [Q, deltaT] = cipogram(stl, st2, tau, maxT, T, verbose)
S[Q, tr] = czpogram(stl, st2, tau, maxT, T, verbose)

SInput
S sti, st2: spzke trains wzth sorted spzke tzmzngs
S tau: tzme constant for CIP kernel
S maxT: correlogram range wzll be effective zn [-maxT, maxT]
% T: length of spzke trazn zn seconds
S verbose: (optzonal/0) detailed znfo, uses tzc, toe

SOutput
Q% : cpogram
S deltaT: tzme range

SSee also: czp_max_ftzter2, nczpogram

SCopyrzght 2006 Antonto and Memmzng, CNEL, all rights reserved
% $Id: cpogram.m 53 2007-01-14 23:24:21Z memmzng $

if nargin < 5
verbose = 0;
end

N1 = length(stl);
N2 = length(st2);
Nij = N1 N2;

if N1 == 0 II N2 == 0
warning('cipogram:NODATA', 'Atuleastuoneuspikeuisurequired!');
deltaT = []; Q = [];
return;
end

maxTTT = abs(maxT) + tau 10; % exp(-100) ts effectively zero

% rough estimate of # of tzme difference required assumingg independence)
% thzs estimate ts aweful zf the spzke trains are strongly correlated
eN = ceil((max(N1, N2))^2 maxTTT 2 / min(stl(end), st2(end)));
if verbose; fprintf('Expectedutimeudifferencesu[%d]ul/[%d]\n', eN, Nij); end
deltaT = zeros(2 eN, 1);

% Compute all the tzme differences
lastStartIdx = 1;
k = 1;
for n = 1:N1
for m = lastStartIdx:N2
timeDiff = st2(m) stl(n);
if timeDiff < -maxTTT
lastStartIdx = lastStartIdx + 1;
continue;
end

if timeDiff <= maxTTT
deltaT(k) = timeDiff;
k = k +1;
else % thzs ts the ending poznt
break;
end
end
end

deltaT = deltaT(1:(k-1));
N = length(deltaT);
if N < 2
warning('cipogram:NODATA', 'Atuleastutwouintervalsuareurequired');
deltaT = []; Q = [];
return;
end
if verbose
fprintf('Actualunumberuofutimeudifferencesu[%dJ\nSorting ..\n', N); tic;
end

deltaT = sort(deltaT, 1); % Sort the tzme differences
if verbose; fprintf('Sortingufinished [%fusec]\r', toc); end

Qplus = zeros(N, i);
Qminus = zeros(N, 1);
Qminus(1) = 1;
Qplus(N) = 0;

EXPDELTA = exp(-(diff(deltaT))/tau);
for k = 1:(N-l)
Qminus(k + 1) = 1 + Qminus(k) EXPDELTA(k);
kk =N -k;
Qplus(kk) = (Qplus(kk+l) + 1) EXPDELTA(kk);
end

Q = Qminus + Qplus;
Q = Q / 2 / tau / T;
















function [Q, deltaT] = ncipogram(stl, st2, tau, maxT, T, verbose)
% [Q, tr] = nczpogram(stl, st2, tau, maxT, T, verbose)
% Normalized czpogram wzth 2nd order statzstcs.
%
SInput
S sti, st2: spzke trains wzth sorted spzke tzmzngs
S tau: tzme constant for CIP kernel
S maxT: correlogram range wzll be effective zn [-maxT, maxT]
% T: length of spzke trazn zn seconds
S verbose: (optzonal/0)

SOutput
% : czpogram
S deltaT: tzme range
%
SCopyrzght 2006 Antonzo and Memmzng, CNEL, all rights reserved
S$Id: nczpogram.m 59 2007-01-27 19:26:14Z memmzng $

[Q, deltaT] = cipogram(stl, st2, tau, maxT, T, verbose);

N1 = length(stl);
N2 = length(st2);
Nij = N1 N2;

Q = (Q T Nij / T) 2 sqrt(tau T) / sqrt(Nij);









REFERENCES


[1] M. C. W. van Rossum, "A novel spike distance," Neural Computation, vol. 13, no. 4,
pp. 751-764, 2001.

[2] J. D. Victor, "Spike train metrics," Current Opinion in N. ;.,...'i.,'i/; vol. 15, no. 5,
pp. 585-592, Sept. 2005.

[3] G. L. Gerstein, D. H. Perkel, and J. E. Dayhoff, "Cooperative firing activity in
simultaneously recorded populations of neurons: detection and measurement,"
Journal of Neuroscience, vol. 5, no. 4, pp. 881-889, 1985.

[4] J.-M. Fellous, P. H. E. Tiesinga, P. J. Thomas, and T. J. Sejnowski, "Discovering
spike patterns in neuronal responses," J. Neurosci., vol. 24, no. 12, pp. 2989-3001,
Mar. 2004.

[5] S. Schreiber, J. M. Fellous, D. Whitmer, P. Tiesinga, and T. J. Sejnowski, "A new
correlation-based measure of spike timing reliability," Neur .. ,in,,'.:,i vol. 52-54, pp.
925-931, 2003.

[6] A. Carnell and D. Richardson, "Linear algebra for time series of spikes," in ESANN,
2005.

[7] G. Buzsdki and A. Draguhn, \. uronal oscillations in cortical networks," Science,
vol. 304, no. 5679, pp. 1926-1929, June 2004.

[8] Z. F. Mainen, J. Joerges, J. R. Huguenard, and T. J. Sejnowski, "A model of spike
initiation in neocortical pyramidal neurons," Neuron, vol. 15, no. 6, pp. 1427-1439,
Dec. 1995.

[9] Y. Shu, A. Duque, Y. Yu, B. Haider, and D. A. McCormick, "Properties of action
potential initiation in neocortical pyramidal cells: evidence from whole cell axon
recordings (in press)," J. N. .*//;;,',.;..:.1 Aug. 2006.

[10] B. Hochner, M. Klein, S. Schacher, and E. R. Kandel, "Action-potential duration
and the modulation of transmitter release from the sensory neurons of Ayl;,'.:,'- in
presynaptic facilitation and behavioral sensitization," Proc. Natl. Aca. Sci., vol. 83,
pp. 8410-8414, 1986.

[11] H. Alle and J. R. P. Geiger, "Combined analog and action potential coding in
hippocampal mossy fibers," Science, vol. 311, no. 5765, pp. 1290-1293, Mar. 2006.

[12] A.-K. Warzecha, J. Kretzberg, and M. Egelhaaf, "Temporal precision of the encoding
of motion information by visual interneurons," Current B.:.'1. .,/; vol. 8, no. 7, pp.
359-368, Mar. 1998.









[13] Z. N. Aldworth, J. P. Miller, T. Gedeon, G. I. Cummins, and A. G. Dimitrov,
"Dejittered spike-conditioned stimulus waveforms yield improved estimates of
neuronal feature selectivity and spike-timing precision of sensory interneurons," J.
Neurosci., vol. 25, no. 22, pp. 5323-5332, June 2005.

[14] Z. F. Mainen and T. J. Sejnowski, "Reliability of spike timing in neocortical
neurons," Science, vol. 268, no. 5216, pp. 1503-1506, 1995.

[15] R. VanRullen, R. Guyonneau, and S. J. Thorpe, "Spike times make sense," Trends in
Neurosciences, vol. 28, no. 1, pp. 1-4, Jan 2005.

[16] M. N. Shadlen and W. T. Newsome, "Noise, neural codes and cortical organization,"
Curr. Opin. Neurobiol., vol. 4, pp. 569-579, 1994.

[17] H. Agmon-Snir, C. E. Carr, and J. Rinzel, "The role of dendrites in auditory
coincidence detection," Nature, vol. 393, no. 6682, pp. 268-72, May 1998.

[18] J. P. Donoghue, J. N. Sanes, N. G. Hatsopoulos, and G. Gaal, \. li I! discharge
and local field potential oscillations in primate motor cortex during voluntary
movements," J. N uir.-jphi; l vol. 79, no. 1, pp. 159-173, 1998.

[19] D. R. Brillinger, J. Hugh L. Bryant, and J. P. Segundo, "Identification of synaptic
interactions," B':.. ./',..,r C;i.,. i, I.:/ vol. 22, pp. 213-228, 1976.

[20] R. Dahlhaus, M. Eichler, and J. Sandkiihler, "Identification of synaptic connections in
neural ensembles by graphical models," Journal of Neuroscience Methods, vol. 77, pp.
93-107, 1997.

[21] G. Schneider, M. N. Havenith, and D. Nikoli6, "Spatiotemporal structure in large
neuronal networks detected from cross-correlation," Neural Computation, vol. 18, no.
10, pp. 2387-2413, 2006.

[22] T. Berger, M. Baudry, R. Brinton, J.-S. Liaw, V. Marmarelis, A. Y. Park, B. Sheu,
and A. Tanguay, "Brain-implantable biomimetic electronics as the next era in neural
prosthetics," Proceedings of the IEEE, vol. 89, no. 7, pp. 993-1012, 2001.

[23] R. Giitig and H. Sompolinsky, "The tempotron: a neuron that learns spike
timing-based decisions," Nat Neurosci, vol. 9, no. 3, pp. 420-428, Mar. 2006.

[24] A. Borst and F. E. Theunissen, "Information theory and neural coding," Nat
Neurosci, vol. 2, no. 11, pp. 947-957, Nov. 1999.

[25] L. Paninski, "Estimating entropy on m bins given fewer than m samples," In fr, I.,-
tion Th(..';, IEEE Transactions on, vol. 50, no. 9, pp. 2200 2203, 2004.

[26] W. Bialek and A. Zee, "Coding and computation with neural spike trains," J. Stat.
Phys., vol. 59, pp. 103-115, 1990.









[27] J. D. Victor, "Binless strategies for estimation of information from neural data,"
Phys. Rev. E, vol. 66, no. 5, pp. 051903, Nov 2002.

[28] E. N. Brown, R. E. Kass, and P. P. Mitra, '\dl l I.!. neural spike train data analysis:
state-of-the-art and future challenges," Nature neuroscience, vol. 7, no. 5, pp. 456-61,
may 2004.

[29] N. Masuda and K. Aihara, "Spatiotemporal spike encoding of a continuous external
signal," Neural Computation, vol. 14, pp. 15991628, 2002.

[30] M. N i.- it, A. Aertsen, and S. Rotter, "Single-trial estimation of neuronal firing
rates: From single-neuron spike trains to population activity," Journal of Neuro-
science Methods, vol. 94, pp. 81-92, 1999.

[31] M. Bazhenov, M. Stopfer, M. Rabinovich, R. Huerta, H. D. Abarbanel, T. J.
S, ii..'. -1: and G. Laurent, "Model of transient oscillatory synchronization in
the locust antenna lobe," Neuron, vol. 30, pp. 553-567, 2001.

[32] P. Reinagel and R. C. Reid, "Precise firing events are conserved across neurons," J.
Neurosci., vol. 22, no. 16, pp. 6837-6841, Aug 2002.

[33] R. Azouz and C. M. Gray, "Dynamic spike threshold reveals a mechanism for
synaptic coincidence detection in cortical neurons in vivo," PNAS, vol. 97, no. 14, pp.
8110-8115, July 2000.

[34] E. M. Izhikevich, "Polychronization: Computation with spikes," Neural Comp., vol.
18, no. 2, pp. 245-282, Feb. 2005.

[35] J. de la Rocha and N. Parga, "Short-term synaptic depression causes a
non-monotonic response to correlated stimuli," J. Neurosci., vol. 25, no. 37, pp.
8416-8431, Sept. 2005.

[36] P. Konig, A. K. Engel, and W. Singer, lii. 5,i Ii"r or coincidence detector? the role
of the cortical neuron revisited," Trends in Neurosciences, vol. 19, no. 4, pp. 130-137,
April 1996.

[37] D. Xu, Energy, entropy and information potential for neural computation, Ph.D.
dissertation, University of Florida, May 1999.

[38] A. Kuhn, A. Aertsen, and S. Rotter, "Higher-order statistics of input ensembles
and the response of simple model neurons.," Neural Computation, vol. 15, no. 1, pp.
67-101, 2003.

[39] A. Kuhn, S. Rotter, and A. Aertsen, "Correlated input spike trains and their effects
on the response of the leaky integrate-and-fire neuron," Neur ...injul.:, vol. 44-46,
pp. 121-126, June 2002.









[40] J. Hopfield and A. Herz, "Rapid local synchronization of action potentials: Toward
computation with coupled integrate-and-fire neurons," PNAS, vol. 92, no. 15, pp.
6655-6662, July 1995.

[41] J. J. Hopfield and C. D. Brody, "What is a moment? transient synchrony as a
collective mechanism for spatiotemporal integration," PNAS, vol. 98, no. 3, pp.
1282-1287, 2001.

[42] A. K. Engel and W. Singer, "Temporal binding and the neural correlates of sensory
awareness," Trends in Cognitive Sciences, vol. 5, no. 1, pp. 16-25, 2001.

[43] R. E. Mirollo and S. H. Strogatz, "Synchronization of pulse-coupled biological
oscillators," SIAM Journal on Applied Mathematics, vol. 50, no. 6, pp. 1645-1662,
Dec. 1990.

[44] D. Hansel and H. Sompolinsky, "Synchronization and computation in a chaotic neural
network," Phys. Rev. Lett., vol. 68, no. 5, pp. 718-721, Feb 1992.

[45] A. Zumdieck, M. Timme, T. Geisel, and F. Wolf, "Long chaotic transients in complex
networks," Phys. Rev. Lett., vol. 93, no. 24, pp. 244103, 2004.

[46] E. M. Izhikevich, \\. I.:!y pulse-coupled oscillators, FM interactions, synchronization,
and oscillatory associative memory," IEEE Transactions on Neural Networks, vol. 10,
no. 3, pp. 508-526, 1999.

[47] L. F. Abbott and C. van Vreeswijk, "Asynchronous states in networks of
pulse-coupled oscillators," Phys. Rev. E, vol. 48, no. 2, pp. 1483-1489, Aug 1993.

[48] P. D ,i- in and L. F. Abbott, Theoretical Neuroscience: Computational and Mathemat-
ical Modeling of Neural S1.-1/ i- MIT Press, Cambridge, MA, USA, 2001.

[49] A. M. Aertsen, G. L. Gerstein, M. K. Habib, and G. Palm, "Dynamics of neuronal
firing correlation: modulation of "effective con I. ii. ," Journal of N. ,;,',/ /i,;-,:./.i,~
vol. 61, no. 5, pp. 900-917, 1989.

[50] S. Griin, M. Diesmann, and A. Aertsen, "Unitary Events in multiple single-neuron
activity. I. detection and significance," Neural Computation, vol. 14, no. 1, pp. 43-80,
2002.

[51] D. L. Snyder and M. I. Miller, Random Point Processes in Time and Space,
Springer-Verlag, 1991.

[52] D. J. Daley and D. Vere-Jones, An Introduction to the Theory of Point Processes,
Springer, 1988.

[53] Z. F. Mainen and T. J. Sejnowski, "Reliability of spike timing in neocortical
neurons," Science, vol. 268, no. 5216, pp. 1503-1506, 1995.









[54] D. H. Perkel, G. L. Gerstein, and G. P. Moore, \. uronal spike trains and stochastic
point processes: II. simultaneous spike trains," B.: ./,I,;-1 J, vol. 7, no. 4, pp. 419-440,
July 1967.

[55] C. K. Knox, "Cross-correlation functions for a neuronal model," B.: /i,;...: ,rl Journal,
vol. 14, no. 8, pp. 567-582, 1974.

[56] R. E. Kass, V. Ventura, and C. Cai, "Statistical smoothing of neuronal data,"
Network: Comput. Neural Syst., vol. 14, pp. 5-15, 2003.

[57] W. Gerstner, "Coding properties of spiking neurons: reverse and cross-correlations,"
Neural Networks, vol. 14, pp. 599-610, 2001.

[58] P. Diggle and J. S. Marron, "Equivalence of smoothing parameter selectors in density
and intensity estimation," Journal of the American Statistical Association, vol. 83,
no. 403, pp. 793-800, Sept. 1988.

[59] E. Parzen, "On the estimation of a probability density function and the mode," The
Annals of Mathematical Statistics, vol. 33, no. 2, pp. 1065-1076, Sept. 1962.

[60] G. Palm, A. M. H. J. Aertsen, and G. L. Gerstein, "On the significance of correlations
among neuronal spike trains," B:.'. ..I.: l C;,I. i ,. 1./. vol. 59, pp. 1-11, 1988.

[61] H. Shimazaki and S. Shinomoto, "A recipe for optimizing a time-histogram," in
Neural Information Processing S;,-/ ii;- 2006.

[62] B. L. Sabatini and W. G. Regehr, "Timing of synaptic transmission," Annual Review
of Pi,;-,..: ..i,; vol. 61, pp. 521-542, 1999.

[63] S. M. Potter and T. B. DeMarse, "A new approach to neural cell culture for
long-term studies," Journal of Neuroscience Methods, vol. 110, pp. 17-24, 2001.

[64] H. Kawaguchi and K. Fukunishi, "Dendrite classification in rat hippocampal neurons
according to signal propagation properties," Experiments in Brain Research, vol. 122,
pp. 378 -392, 1998.

[65] J. le Feber, W. L. C. Rutten, J. Stegenga, P. S. Wolters, G. J. A. Ramakers, and
J. van Pelt, "Conditional firing probabilities in cultured neuronal networks: a stable
underlying structure in widely varying spontaneous activity patterns," J. Neural
Eng., vol. 4, pp. 54-67, 2007.

[66] C. D. Brody, "Correlations without synchrony," Neural Computation, vol. 11, pp.
1537-1551, 1999.

[67] D. Nikolid, Nun-parametric detection of temporal order across pairwise
measurements of time dJi1- ," J. Comput. Neurosci., vol. 22, pp. 5-19, 2007.

[68] F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek, Spikes: exploring
the neural code, MIT Press, Cambridge, MA, USA, 1999.









[69] P. Diggle, "A kernel method for smoothing point process data," Applied Statistics,
vol. 34, no. 2, pp. 138-147, 1985.

[70] E. N. Gilbert and H. O. Pollak, "Amplitude distribution of shot noise," Bell Syst.
Tech. J., vol. 39, pp. 333350, 1960.

[71] S. Yue and M. Hashino, "The general cumulants for a filtered point process," Applied
Mathematical Modelling, vol. 25, pp. 193-201, 2001.

[72] J. Michel, "A point process approach to filtered processes," Methodol. Comput. Appl.
Probab., vol. 6, pp. 423-440, 2004.

[73] E. Parzen, Time Series A,.l.i-. Papers, Holden-Day, 1967.

[74] J. S. Simonoff, Smoothing Methods in Statistics, Springer, 1996.

[75] I. S. Abramson, "On bandwidth variation in kernel estimates-a square root law,"
Ann. Stat., vol. 10, no. 4, pp. 1217-1223, Dec 1982.

[76] D. Erdogmus and J. C. Principe, "Generalized information potential criterion for
adaptive system training," IEEE Transactions on Neural Networks, vol. 13, no. 5, pp.
1035-1044, Sept. 2002.

[77] J. C. Principe, D. Xu, and J. W. Fisher, "Information theoretic 1. ,ri::. in
Unsupervised Adaptive Filtering, S. Haykin, Ed., vol. 2, pp. 265-319. John Wiley &
Sons, 2000.

[78] A. R6nyi, "On measures of entropy and information," in Selected papers of Alfred
R.fi;i' vol. 2, pp. 565-580. Akademiai Kiado, Budapest, Hungary, 1976.

[79] S. Kullback, Information Theory and Statistics, Dover Publications, New York, 1968.

[80] N. Aronszajn, "Theory of reproducing kernels," Transactions of the American
Mathematical S.. .. I,; vol. 68, no. 3, pp. 337-404, May 1950.









BIOGRAPHICAL SKETCH

II Park was born on April 29, 1979 in Gosla, G, i rii, ,v. He attended Gyunggi Science

High School for 2 years. He ii ii red computer science at KAIST (Korea Advanced

Institute of Science and Technology). He spent 2001-2003 in an internet security

company as a developer. He has been working with Dr. Jos6 Principe in Computational

NeuroEngineering Laboratory (CNEL) since 2006. He is admitted to the Biomedical

Engineering department for the Ph.D. program in University of Florida.