UFDC Home  myUFDC Home  Help 



Full Text  
xml version 1.0 encoding UTF8 REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchemainstance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd INGEST IEID E20101107_AAAABY INGEST_TIME 20101107T18:45:03Z PACKAGE UFE0022538_00001 AGREEMENT_INFO ACCOUNT UF PROJECT UFDC FILES FILE SIZE 41241 DFID F20101107_AABIHD ORIGIN DEPOSITOR PATH wei_y_Page_037.pro GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5 805d3241b236e97a3c920b306331b612 SHA1 84a4d7e67fcce1a991e9f2d4d44622008c790650 39273 F20101107_AABIGO wei_y_Page_022.pro 9bab7c8ff0bb183be652af6eed1f47ae cf697ad09d881b382e3daa9192bcb1fef4cfbfdf 35352 F20101107_AABIHE wei_y_Page_038.pro a1b41b44acdc29cc79bb6a092984e8c5 b3319c6e3922f51217b5a8c52be2cc0472d8b1ad 31528 F20101107_AABIGP wei_y_Page_023.pro 0474e61d2ae45aa9df12655a02a7cb78 5c6d9d8e5e2cf8b086b42ba94d946b3e2f68a765 49365 F20101107_AABIHF wei_y_Page_039.pro d05e2d89f239e69a0ba93d4104aa195e 4105b8d20478832f499b2d7177b0df38b29f43cc 13298 F20101107_AABIGQ wei_y_Page_024.pro 718d1a9aeeaaa3f492fc10e857e5f751 9b964e02ff2369ab11e0670bd8df7566bf1f3de1 56870 F20101107_AABIHG wei_y_Page_040.pro 479d2aea6f79770c9340664c398a887b 04c41a69020374c80872807e048a0b74df72caf2 43331 F20101107_AABIGR wei_y_Page_025.pro 41614239a4727d12ec3d9ebf97bf0dec f261287784a7312600940e4eff556ba7cd4ee866 13385 F20101107_AABIHH wei_y_Page_041.pro 0c47e14f64a40a3ef1428022eaffc4d5 5715b481a25793ca457e4c35fefc5736ddea0548 36042 F20101107_AABIGS wei_y_Page_026.pro f231de7967000d60117b4ea248f50eb8 a5227598f1656c7d73a6b725bc37d835add22ebd 49163 F20101107_AABIHI wei_y_Page_042.pro d358d8055f647a68961b713d71b27fb2 c52fea2e434e7fa30fac1447193de458c741d845 49847 F20101107_AABIGT wei_y_Page_027.pro be4e6001da4f7469c67e5307950ea415 dc635674fc1269189aa886ef3a93262433b29156 56218 F20101107_AABIHJ wei_y_Page_043.pro 1b2bb26ac92d2a11307c48a3909f85e7 8b531f43a532cb05c087b75bd995ffe984ff50a1 31773 F20101107_AABIGU wei_y_Page_028.pro 069dc3caf01d63f7980dc63fce754fdf 3ac32508e631dfde7570d299d46f9c2d0c19f2a1 55382 F20101107_AABIHK wei_y_Page_044.pro 6952f5b6cf38d03eab7922e64985e139 aa77ee9e52bcd5afc3079927de87b8e07b8de153 22414 F20101107_AABIGV wei_y_Page_029.pro 0764b3261745072ebe41e4dcdc8cb409 00fdf9e794cc9cf51e93982e75ace2a0f8a0d73c 53015 F20101107_AABIGW wei_y_Page_030.pro cd5dc24767cc55a8ed46cf6a796b74e0 f5918075386e059597b47269272a5f8a440f50d1 55004 F20101107_AABIHL wei_y_Page_045.pro cdeb1ae15ccdf1434b7feea8be9e1b00 451841b8d88f5426e6a1ba5e8ca9b38a042a2079 30243 F20101107_AABIGX wei_y_Page_031.pro b2d437660ac05297851c34e4a47f04a8 f93273c8fbd8e1c04562a85ffc8f6aa97ef80cdf 55264 F20101107_AABIIA wei_y_Page_060.pro 6dfbd1849413f0b6a9407c6a57f3ebd7 799ce1a82f375baca2b306d55f65cb9d307b9c52 56731 F20101107_AABIHM wei_y_Page_046.pro 292d6d84dab801bc33f3f28e01e4e4b6 0137a127ff9f6b20b0d4bd68dd35ce40e65cb85e 42905 F20101107_AABIGY wei_y_Page_032.pro fd8c6decb12d6e86ef46fd203aa201e8 b5d5748f85763aa21417700fb90832c48d58d94f 54332 F20101107_AABIIB wei_y_Page_061.pro 04d5e3f5f464e34b34b63d4a29dd0993 dd4ccab303b053191302cc0ff670969530c37021 53080 F20101107_AABIHN wei_y_Page_047.pro a4f4a0809364d5a1d4024a2eda06eddb 4b2268b6737d83aadb36bb53333365e1bcd76a00 42050 F20101107_AABIGZ wei_y_Page_033.pro 2fe99b3641bfdb876e83c4dcd64983b7 3c799e0102d6c0bde246d7e98e80da53c72dc3c1 53650 F20101107_AABIIC wei_y_Page_062.pro 788993ec74a429e4a9404ff546f2f9f7 2ee970467eee042b4061bd9aa8add7df773e76fe 50091 F20101107_AABIHO wei_y_Page_048.pro bc6c5986c2294899858a60694caaa8f7 b42d5a023b64be0b6e6b8c185d77a49cc58ddacd 52274 F20101107_AABIID wei_y_Page_063.pro d641aa23bcee9d70a11f36c8b67fc155 d87907a6249874add229133eb7b51ad97bf8a29d 22211 F20101107_AABIHP wei_y_Page_049.pro 239f9eacc08a66bb49307f1c30b1dfd2 3869e92971c4ae98da7190910c22a3195877c7c4 54637 F20101107_AABIIE wei_y_Page_064.pro d4e6e6ed0008ac0858f3cb8d41da4990 0010af81fa440ab5e0a8298e267d060fccbf6662 45733 F20101107_AABIHQ wei_y_Page_050.pro 271dad37a3e0a931d45de0f772d1ffa7 934f1468b29ecde29586ba0751247b53d6c7fd45 55927 F20101107_AABIIF wei_y_Page_065.pro 921fbe57a6d4d04b4e35d62458da8c3d 3ee33ef0ff22fddbc27f11592efd5306c5b6f3fa 53270 F20101107_AABIHR wei_y_Page_051.pro 6e2cb6fb6caa441edea27724f772070e b9aad57f25eec13b2f3fb2efe4c5abfa5830c8a3 52007 F20101107_AABIIG wei_y_Page_066.pro d513e8b1fa7d926201282768fe10652d fda6a40511f6e2371255dac5b9e492798225dcce 50421 F20101107_AABIHS wei_y_Page_052.pro c54133d58138c0d8afd0b9463e05bf99 3c62d9f37720c94e77196b7c57d907d369ab9446 55009 F20101107_AABIIH wei_y_Page_067.pro e47c64a779036d3878c269ce34bdbe4a e3a470011bd3b35f7962875ad7672729d3813849 4849 F20101107_AABIHT wei_y_Page_053.pro 02acc34c7a2701fb071d44988c77a614 b899ce15469fb493fc5d29474f7f5f1661c4da06 26397 F20101107_AABIII wei_y_Page_068.pro 336ce04773db950da5d3bd69ace33495 e733979e09bbd30f82ed6f12d882ac54f9758b4d 7350 F20101107_AABIHU wei_y_Page_054.pro 6ec7c171565590a73898ea744e2d8bf4 bfd00222903ab7024e2884e7d1fa7b91740a78f5 11267 F20101107_AABIIJ wei_y_Page_069.pro f57c97b3fe9cc68c601490f99e370b3d 3951fa204a1b254da0de773c0ce168bbd0527037 14117 F20101107_AABIHV wei_y_Page_055.pro ef4c2d2d2053d77a2a99871ace279384 f25be0287d410a3f64637ff47905fad54dff6b25 10929 F20101107_AABIIK wei_y_Page_070.pro 0715756efe315a99957df7a1156e5d68 5874c576756a5fb0db616d1f206d6643f1dda871 25440 F20101107_AABIHW wei_y_Page_056.pro 78345f823a7a120620e8ced64465dc7a f5fafee45a198177045beec4de75ead8109c5616 9914 F20101107_AABIIL wei_y_Page_071.pro 8b69a784bff67992f21438cdb0e141bc a349b7af3fdcfcd720b2c20ae7d8fb28a143f315 13935 F20101107_AABIHX wei_y_Page_057.pro 3d8149d5a80b95ef885b80032874731d 3e71f0cadeb07661d00aba867dfb9a24596951dd 13100 F20101107_AABIJA wei_y_Page_086.pro 9d3d7f31fabd75a7d25343dd3ab07ff1 1690fe534708019ac95cd0aab140c91935bf4c29 16951 F20101107_AABIJB wei_y_Page_087.pro 836c53967c778a21f25cf72c9692cd85 df5495bb136c928e4e65fe88ad62586fbda3f18e 12538 F20101107_AABIIM wei_y_Page_072.pro 7facf8e4a0cbd163ae3046d973b85e17 35c717adb61c481cd93ef95222133079608369f0 25418 F20101107_AABIHY wei_y_Page_058.pro 8486199adbee8379035718d437cc628c b12c95ab55803cd3dc4d5399971ff9d43c91e683 11992 F20101107_AABIJC wei_y_Page_088.pro 3750229e31f38f88f8fb33cf2e3f303c 078bc9edf1e4b45bb1b00ab8711cb8f49c78e8c1 13782 F20101107_AABIIN wei_y_Page_073.pro 5c6754a03c6375aff2b0a7529e7ba236 4507cd7958a15ce79bcd47af6ac080071e391128 51589 F20101107_AABIHZ wei_y_Page_059.pro e1a99105a1cb5cf15833699deb8c2b99 a4bc429f99d5307025b9608b430ff2acae58b07c 12588 F20101107_AABIJD wei_y_Page_089.pro bfd7d4ffe8f362fc50ce7e1d72292d35 fbae68c6681be9487707d081cefe8bd2d79c60b9 12451 F20101107_AABIIO wei_y_Page_074.pro cef8dddcc5c120228bb02bb4f18d67b1 bc9dc645c8c012092da8b6cd1d0fe4dbe744c84e 16682 F20101107_AABIJE wei_y_Page_090.pro 6608d8732c0890acad35fba2ffc719b4 a3951912dbc0a48c80e2e3b15035f36ffe918e4b 15448 F20101107_AABIIP wei_y_Page_075.pro ee8691c9180a39bc571eafb82f3f06da b049919037110fe98a89de8dbc4dc25a3b3a2a55 12970 F20101107_AABIJF wei_y_Page_091.pro 67416c017b0f7cf88f12dfb3802a9c20 0723d1219bc14af6d2e97a997d5a8dee0c3f7fac 14609 F20101107_AABIIQ wei_y_Page_076.pro e9d61abcda1c558e06453841386a6b8b 47f9b36143a84b25118347d41dab9d58075c5156 13362 F20101107_AABIJG wei_y_Page_092.pro 11a731a851856c61392c82c9645fba82 ff8271783307ec97218483937156951f8629a7de 13873 F20101107_AABIIR wei_y_Page_077.pro cf6d8d566dcb74f9d4e6b8ba1e80f536 59df833093a3f165df25dd73734727ef9be14230 18138 F20101107_AABIJH wei_y_Page_093.pro 03513e316522621a3b8b6bed18d30985 6458f6c0f1e379b4e10c6a9c178a748cd30717f7 16546 F20101107_AABIIS wei_y_Page_078.pro 5233e8aa0c8fea257fe148d7f7bb9ed2 4ab5b72e9ef35730b423dd589a04b8b2a4cd991b 20228 F20101107_AABIJI wei_y_Page_094.pro 07712eb6bb90d8ddaa10809cf07ade65 7b499c56fa5fa5dd9f731ba8bb439de0496743cf 11583 F20101107_AABIIT wei_y_Page_079.pro cfa18170ee9589f74aad1a5aef25e9b2 1dd27028e869c4be8e13f89f3f3c151308d7f205 12205 F20101107_AABIJJ wei_y_Page_095.pro 77a9c1d85516d1277c0f7b7185fa4540 8f6b0a2e8f3e63a6c7ff9141189d9d18adb6300b 12248 F20101107_AABIIU wei_y_Page_080.pro 90448a4528fb912479eb89a8200d2ce4 b96f5d464c0f54c5b5e7f0817bbde208ad30f9b1 17558 F20101107_AABIJK wei_y_Page_096.pro 2d4ca099ed0436cf7d78b101e95c6038 1b716a3645f147792c5c7af4312988b4952b404f 16532 F20101107_AABIIV wei_y_Page_081.pro 96d63f9161f8bdcaf01e61501b2bdcab 47df0691de79fc9e50e092aff999cc327ecda021 14320 F20101107_AABIJL wei_y_Page_097.pro 9ce7bcf82bbfd1041a8db1142e4daa0f 16e936f10cfe534fada848c1d7a1debf2991fc32 15995 F20101107_AABIIW wei_y_Page_082.pro ba9ec1e748eb54637458b1d911b53fec 53f8575ad5e98b2e387d00a77cf848cb83a5ab83 15468 F20101107_AABIJM wei_y_Page_098.pro 309c769a9c6655d5c011449a221cf15a 2eeab6511f07a423333abeca18f33026b15a1aac 13911 F20101107_AABIIX wei_y_Page_083.pro a0396a22751fe56a59dd759356925db2 ab3d5c30d1ca5bd1d4afe129613e4e34688491a3 15150 F20101107_AABIKA wei_y_Page_112.pro 70ad01a3d80c314deb2ffb0eba232a9e a653cac7f5dfecaafb2e9750bddc8d1b013d929b 17131 F20101107_AABIIY wei_y_Page_084.pro 4760dbc944b0989ab86a5a4543a9ad30 fce2d5a5dc8fc54a5fc8da0e560e5d1d54712720 19852 F20101107_AABIKB wei_y_Page_113.pro 68971dade229e5e3d863c6d4753cfcfe d829eb7273ca53a406468b61c8b66fddae4d5c88 18570 F20101107_AABIJN wei_y_Page_099.pro 75f794535f61dc357bde69a4f9aec759 89c196300f00cbe48898fe87fbe7da6289e5e25f 12252 F20101107_AABIIZ wei_y_Page_085.pro 029d02bff08e6e4bfa1f4a3eabac3d94 361e43825c2354429863a19e2fd67de9e242a94a 16193 F20101107_AABIKC wei_y_Page_114.pro dbf8937fc652c35a9273c66fc5d4f951 b9339454118af516c8ef5daace524616a148ddb6 20386 F20101107_AABIJO wei_y_Page_100.pro 88044482becd79267e160b17c467370f 8240caf94720187772861787f638151af5dd9ad5 18527 F20101107_AABIKD wei_y_Page_115.pro a598ec2ff5808313a8f46bb1138a2cab 60dde795e5051530615b318b2dee91f11c2f2454 18865 F20101107_AABIJP wei_y_Page_101.pro 322bf4e43821703fd2c8bc8ae6f55fde 44afd6b36365e7bba3bde38114e8cbf77235b910 16901 F20101107_AABIKE wei_y_Page_116.pro 7a9f289877e8c6b10a241c2d742da64b 8b286a87a510c0025f25867bd99b9396fc9a83c6 13844 F20101107_AABIJQ wei_y_Page_102.pro 4ff5eaf4f12d63aeada00623fbbebf14 304e5377c55e163b3d7cb0d8b6fd7b29cd4a8cbf 17387 F20101107_AABIKF wei_y_Page_117.pro abfba1e77e08cb626f1cfb6646b40eec 3d8a061217491c892d425731ba63f890c52df8fc 14970 F20101107_AABIJR wei_y_Page_103.pro ba0a5d17d41810a0adc215bd3a492316 0234a2c9f845e079b78b816c50385b6db2832868 14567 F20101107_AABIKG wei_y_Page_118.pro 348e1757d956f5483f89cc317683d4a8 0f5e8db08f6bb7bbfe3f3e17a9c7b203246556b7 11599 F20101107_AABIJS wei_y_Page_104.pro b50a72eac836398e021591dc1a73cecd 29f164fac6ee076d0ea2b13a5df35ddc78326f62 19179 F20101107_AABIKH wei_y_Page_119.pro 9d4e870b5a16a35b3c6abb881303c9fc 0b405454590d11251b9bdf60c3749ae9a6d1951f 13080 F20101107_AABIJT wei_y_Page_105.pro 83e94d9ad2576744f4c025742335407c e787c2f47be073a575968d268eb2b1f5d4e80e6b 11373 F20101107_AABIKI wei_y_Page_120.pro 2aebea74165be39f9a170ec555aeae74 733458d1f41af7df20c2119b61b417b981037f51 11657 F20101107_AABIJU wei_y_Page_106.pro 81248046d8374a4d52ad37236af5f44a d23d1888ec80ab1e35bfe4c4c25d09778bd584b1 15222 F20101107_AABIKJ wei_y_Page_121.pro 4432cf9129dbdfdde6cb276b1700a8d2 1f9ab98946dd6c8197ae560cef57ee718a453a07 16326 F20101107_AABIJV wei_y_Page_107.pro 7324423462630ee2d64a0d744f086a99 42f2d7f5baa6b0d59a352dbc709e847921901fde 51399 F20101107_AABIKK wei_y_Page_122.pro 0757e6f7a74b207aade43222b322e19f 3dda80bb577f14fc177ccfb52a92ae87c52814e5 12350 F20101107_AABIJW wei_y_Page_108.pro 215e5416664286ff2a50d7cf391a812b 1ecd5f1ab3eb0602ac65ab77bb6b60597e3054b3 56451 F20101107_AABIKL wei_y_Page_123.pro 1f2bc5585829b150c2a8f1eca259abd8 0a38b4a316a5ca03013d9aaa664169cbdb5eb817 16699 F20101107_AABIJX wei_y_Page_109.pro f05a257d1ebb8e3fe3a7f1a7c6514fff dd9acd0800085c2ae970ee89d0706b5347d353ba 19715 F20101107_AABILA wei_y_Page_138.pro c38d86ef14c1040532bd16b8cc1389ab 4e6bc286207bd35d1a75699243362b5a5301491e 58088 F20101107_AABIKM wei_y_Page_124.pro 37dc0fdefa4e41d2418a6962a9dd103c 9b997fac5db60b09f79f0e6242e0de9a8875b9aa 13825 F20101107_AABIJY wei_y_Page_110.pro 023a33f2c6fdb2deffbf12bb0cfd2e74 2c7542c182abf468ec8adc1fe208e0217e157bd8 18296 F20101107_AABILB wei_y_Page_139.pro 0b305c7dba735809c5e783dfe52d282f 84eef85d0d44ed09c5ccd7571bcf4f1266a6405d 54919 F20101107_AABIKN wei_y_Page_125.pro 554acd586b3b1c71a4983199823e8b5d 3ee5b2cfb081cdb0d51af337dddc2f08d0a68fae 15363 F20101107_AABIJZ wei_y_Page_111.pro 5425ffe16381c08c728d82af32f53758 59964ae87c7f659c0726a5daed9af9fe4e6e860b 15872 F20101107_AABILC wei_y_Page_140.pro 38ac0894e7e0a341f4ca76c57ab5dbe8 41a6479042088072650bc318ebeec47df09f9ee3 16930 F20101107_AABILD wei_y_Page_141.pro f46e94e864d7266833951f1fae464806 51f30381754e2a9d452b439e96da138b5a81b47b 54895 F20101107_AABIKO wei_y_Page_126.pro 2b6fb55f2d4a9dd74e00adad193b4d50 93da9261c769d08ef0d479a00b9cc96a4260c75e 16523 F20101107_AABILE wei_y_Page_142.pro be0ccc39ea2e5c40573695dea923fa3d d7fde8af00ac640879d47f35ea9664f338e34a49 56120 F20101107_AABIKP wei_y_Page_127.pro 8e91e0a9d5beaa285fee6bf88d8e3761 e0a30392d1fee2cff7a7020fe5705208701a3913 18109 F20101107_AABILF wei_y_Page_143.pro a133ff78c2918777cc1c19d21ee97cf0 3a9ff9c977e5fbc60aa4879ae299f7e797396a43 53903 F20101107_AABIKQ wei_y_Page_128.pro f5107641b42d63aab7e043c2b0efad98 d018f2937aa657ded00886063068091dba9a47bf 16612 F20101107_AABILG wei_y_Page_144.pro 72006155e719ff8b264143047836758b 45374ba995842673337c21ef3ccaba64069c154b 52478 F20101107_AABIKR wei_y_Page_129.pro 4f36cc608e0ed36e1b2317ea766618a1 e06fe751556c0d2395d6d3081e979bc7e4962ac1 13559 F20101107_AABILH wei_y_Page_145.pro 84f826ac0ed49922dca02d4cc121da47 1dc73e9a8d909aaed21cb31ef3617965d115451d 52027 F20101107_AABIKS wei_y_Page_130.pro fb0a76a93579b272b9ffdcebc6e3a3d3 c9298275b29f48f5ddda153dc9f50d85a2d4cd2e 16734 F20101107_AABILI wei_y_Page_146.pro 596015966cb64bd1a4dacf2fca221073 2c1fa0e7a2270ef4b4b0faf5d969325edf566fcb 55874 F20101107_AABIKT wei_y_Page_131.pro c70f85bdd22e7bd57ceacc5bfe9d7388 d31105209a3df220dcc3c38d2bd7c33f85be0911 17712 F20101107_AABILJ wei_y_Page_147.pro dfbaf570110b589ac49a3ea1f5115504 efc8c7b53cbb4a5f603df4662631cfc3ae08acad 10796 F20101107_AABIKU wei_y_Page_132.pro f5c9aa4b22ef495cba218260bd00d3e8 ca7e202a5d14e27ac5c3b6951695adf42bc8e554 15493 F20101107_AABILK wei_y_Page_148.pro ebe4005097d6d9bf19b14452f5d4c1e2 33870de6d9889a60dcf86cde9700d53cef7af68e 58852 F20101107_AABIKV wei_y_Page_133.pro 6c95c9481c36e62c8a5affb7c0d3ba23 5280c3e5372c6d332d8d39dd1b6b36b5bed1d563 15251 F20101107_AABILL wei_y_Page_149.pro bff134fd16de5ab4ad22a8aa0f35fa28 bf9797e127c71ef2f4e4add7fa8ca6e6df3a22e6 62244 F20101107_AABIKW wei_y_Page_134.pro 6d93738939437c37df4da1f348d84953 3fb3886d5c5a20d32a8336fc1d7257ee315b2ae5 13975 F20101107_AABIMA wei_y_Page_164.pro 7e19cd42bb8946fd5a84e48954124936 613cc088cf083aa969109592fdb013b6f5096b42 17100 F20101107_AABILM wei_y_Page_150.pro 32c7d8eece243fb1b1421832757670d4 05d4b2add903810327ed07a685a67584b4699b73 52996 F20101107_AABIKX wei_y_Page_135.pro 107477640f78df8d5cfcdca769ff1dbf 33f51382090f56a5c8ef9b3f82025be78b5e5dbd 13909 F20101107_AABIMB wei_y_Page_165.pro ac09f589b8e1fb35456427c55324c9bd 9f6c6d41eb94ca561e59ac642aaa34c4315f1353 20300 F20101107_AABILN wei_y_Page_151.pro 472ab68c078a62450d5e03bbf9da178e 2c22f5b4ffdf591ad75912695a2471ab82b1bfb5 55745 F20101107_AABIKY wei_y_Page_136.pro d56f24a407783c01bf757c7b4fc53acb 30bc38b1980e62a9397eaea4af468ed401924544 35240 F20101107_AABIMC wei_y_Page_166.pro 4400320ba739db013ad100edaf12a872 2341f5d64207bdda7734970ca7d617a3d2755542 18135 F20101107_AABILO wei_y_Page_152.pro a15acb3809ea6f1c3978d88aeee79c48 0154ffda790272fc79872e506ad1511cdb64cb56 7687 F20101107_AABIKZ wei_y_Page_137.pro 609ae331bf09fd29d285df70a3a08d2b 5edfba2c20707d12b0f5fbe5f040c2bb497059ae 15123 F20101107_AABIMD wei_y_Page_167.pro d56c29da08ee78707c40e6b1159ee6c4 f758ae72315932f99219812b68c3f453603e6e35 15883 F20101107_AABIME wei_y_Page_168.pro 3fa74393f28979cd3f8d5afff7c8e07a 68f9bc097cfabe10cce9b7113888a82fd93488e3 16785 F20101107_AABILP wei_y_Page_153.pro 0be64b159166b63f182a3ca4050004d3 7975b8dd79a11bcc534d67645a267c2bff65c059 13534 F20101107_AABIMF wei_y_Page_169.pro 956d9f6f9018f0f6ba39e89c4cfe6ef6 8dd1c3df1f3935c28685815a17f18748e7858eca 16535 F20101107_AABILQ wei_y_Page_154.pro 4051f23ea10d875c091a7e24ab17098c 632d030ac5cef52e1c18ed4662596b754a4e98ab 14459 F20101107_AABIMG wei_y_Page_170.pro d68e517e5f46498c6c82640851fd55e6 0d106a52d4e35eb2e91d8b9a242b10328ed8bef5 16242 F20101107_AABILR wei_y_Page_155.pro a5cdfb5b0d88ee3158abfd00bff91e52 1516b1d28dc756eef4f952847c6078b78ae005d5 15093 F20101107_AABIMH wei_y_Page_171.pro 8707961f7a43f93b6efba8f1bb1f2c35 802d5e414055ac12c48f0463e3c3131ef8b8ac00 16578 F20101107_AABILS wei_y_Page_156.pro 4b44db1d60a043142ea3e38db163810e e6624504d917d959ff17b343c6d36e7124db57d2 13672 F20101107_AABIMI wei_y_Page_172.pro be7cd00c0a1f2c45e44eb239ae20e7fd 72f4abba63d4bfd4a77eef0d3105cc3698b6ba3c 16555 F20101107_AABILT wei_y_Page_157.pro 9d7d3524c326eb33890ddb4352cc6ee5 be19d51a65e43daeafba55a8aa0e1302f9518b54 15415 F20101107_AABIMJ wei_y_Page_173.pro c093a31664e7d811d4d5317e68bbad5c f87e5aedf175820c7010004faef1f3b9dc3175ec 17022 F20101107_AABILU wei_y_Page_158.pro bee6e4ead22f8aac5049490f4c45c368 4402df3e24ea14bb9acd515024ca970be0c78ca3 17359 F20101107_AABIMK wei_y_Page_174.pro 33da854eb43fe3e56586f5cb6d93bcae 9b020d551919b4823c3bb399c2ee16af18446a51 17245 F20101107_AABILV wei_y_Page_159.pro 8ebf5e0cf85354527a40712aca322032 646439eab3c6680f15b649dfa4cb095b318eca73 24939 F20101107_AABHJJ wei_y_Page_132.jpg 75f8f648e5816ee59c5266fdfc6b1083 ab712498bff2eb32c0ec1a4c1f426ed65564acd1 13724 F20101107_AABIML wei_y_Page_175.pro 097ee48e3a6ba4f5a775b76c74c83fae 8d00e4e0cd04478fe1620997a3ae0ebbc34a9282 17418 F20101107_AABILW wei_y_Page_160.pro 6b679835952bd4d5c3c1f9570ff29d04 2c5adc2abed83a7bf06675a23760423391473372 219145 F20101107_AABHJK UFE0022538_00001.mets FULL 65a0aefefc6d3126af7f07db469cf5e2 1f0d0b6f3514ff9749662692a7d8896b777330fd 14086 F20101107_AABIMM wei_y_Page_176.pro f39f1c7920997311538e8f6b5c2982b2 b7c5e328907bc85baa87b72cc76270a7573f7405 18760 F20101107_AABILX wei_y_Page_161.pro ac3774c77165b1e0c4de9c402dda279b 266771775fb5376d3862274ae4b18950fd95bd90 59997 F20101107_AABINA wei_y_Page_190.pro d6abab18a4be0735115b0f09715eac1d d821c951a755d89a9ebe2ab8e4b082c8d8acfd14 15080 F20101107_AABIMN wei_y_Page_177.pro e3b54051dd1ca40f41c9a49cf94c5cb4 19d1137fd32bb32ef95e016657536bb9aea379a6 16250 F20101107_AABILY wei_y_Page_162.pro 9ce95d63509287a135aa045a5e5f2012 c66c7695d516b0ca7259567a191c1451cc0731fb 61596 F20101107_AABINB wei_y_Page_191.pro 292131c908bbfb7dfa6e0bc57151a138 fcc01e7d532a934d2d5abb334ad7e6c0fe307cef 13785 F20101107_AABIMO wei_y_Page_178.pro d36c8a45c2dcfdf8f93a526731962ed5 867c401a48d0e1c0d2446df9c7cfd19c73b435d1 15316 F20101107_AABILZ wei_y_Page_163.pro 3dce39ee5bf074b738413852558dbd3e dfa34fd216234fa551e121499493cc7df8ef2f3d 63912 F20101107_AABHKA wei_y_Page_014.jpg 613c0b5d389a233160106b2d4b567dc9 9cb9446c2de694744f65e60677b59441c87646d1 34328 F20101107_AABINC wei_y_Page_192.pro 5d5345d803d180fd0b51e3734f762400 47069305e8d662ed6a38ae6bbb088cdc22a92948 13777 F20101107_AABIMP wei_y_Page_179.pro dec1dcf36bae9d01ac26ab88dd9ee491 48990952e0980f6c6e7a687ec7bea9b8819bed33 54302 F20101107_AABHKB wei_y_Page_015.jpg 1ad02ca16c372b76012320bd5c970f19 eca2f9430fcb34d3c474fea451579552a0a61a58 18864 F20101107_AABIND wei_y_Page_193.pro e4ecb8f890640ca5a45c26ab58d39a1f 878716cc6ca948bc599501e852ec555fe303ede6 26649 F20101107_AABHJN wei_y_Page_001.jpg 77794096c1969daad1a946e48ceb9932 8f910edb27b1e02f65acb57465f69ce36a6fbc0d 82860 F20101107_AABHKC wei_y_Page_016.jpg 332fe35cf1f584a44aa85caf9ff5e167 4c2f97ab9706799279b3503d79aebef601546cc3 452 F20101107_AABINE wei_y_Page_001.txt b398d5027ac28a2d5cec5c7053caa5af 98d2df69b88750b643906e2ab4d2540f26a848eb 13839 F20101107_AABIMQ wei_y_Page_180.pro bf18bb8935281ed492062feb4ee25a06 905e833504a35efac2c7c64da3aced4ee16ef9ca 94579 F20101107_AABHKD wei_y_Page_017.jpg e9fef0e5ebd2ef70e93a8af602182943 74b32bf8409eb774b8d184621a15627daa85a307 86 F20101107_AABINF wei_y_Page_002.txt 0ea2353ceadd1a54cc3535649540fe8a a4d2c80791dafb619915c0d8978029834e2f763b 3539 F20101107_AABHJO wei_y_Page_002.jpg 5343707a14a0d1270fcb7f58323e81c4 05138aec02076ca6c51772b54d7fc2c1a7f955a1 15345 F20101107_AABIMR wei_y_Page_181.pro e8ee2c6b3030a6ff0fa7c18cce44d079 9722e6f7d306883f7f98089495545085e13798b8 86202 F20101107_AABHKE wei_y_Page_018.jpg 71bd2c9a4321b43a1804da54aa841122 fc0757757112c4bbfbf49d9361797c829b92f73b 578 F20101107_AABING wei_y_Page_003.txt 9e990e515e59ddc0d5529c964cea021b 70ec34b50a62e049ea21f86612f8c2af2202125e 30761 F20101107_AABHJP wei_y_Page_003.jpg 15d9b2b952ada319cd3aab4b62b31ce9 bdbbd88077e751d07e817a922ade64d4b0cd1586 14058 F20101107_AABIMS wei_y_Page_182.pro 9410d15450c025ebd6d3cf33454af75c b849e1aaf924bc2ff49a50b16933862e10a6cbee 93099 F20101107_AABHKF wei_y_Page_019.jpg dbe5127e36f225b30e2d598bcdbd1d3f 03e5686e29a1ac815db254426da67f301662ebdb 3798 F20101107_AABINH wei_y_Page_004.txt 789d8afc20fbffd662bbca97f7520f34 d23396c34e6addb1afd7445052804489cd9f21d7 117374 F20101107_AABHJQ wei_y_Page_004.jpg 9e44e49a5640b120b2ce966434e5ba29 f1189df84ef848defbcdd016b4101ed29813b162 15206 F20101107_AABIMT wei_y_Page_183.pro 3c08ea82ce8a9b77c30ff406f0e84ed4 db91f754e7ebd343c544571ff8bb13b1554a17c1 93401 F20101107_AABHKG wei_y_Page_020.jpg c5e99bb675a514626c3b9181102c093c 17d9abc2dbeca85a14a4fd6cf0b0da48048b6280 1955 F20101107_AABINI wei_y_Page_005.txt f95782df83d9cd5bd9b674a916bf02b5 da84d3baeaa9981e9be11216da3686b125c4c572 70229 F20101107_AABHJR wei_y_Page_005.jpg 99928dae5a35c7ab684778236ac42820 be3de4df11e0c66927aa190aad56a14a317c6491 18183 F20101107_AABIMU wei_y_Page_184.pro ca752f022f05c9918798892ab8b2ad71 fe40e7a82fafce4683fa787726e9576ae94ca57f 51257 F20101107_AABHKH wei_y_Page_021.jpg d25d5623f23635b71b225456962d6694 942037074938a49d4ef168a2842d9bc7d17859d6 684 F20101107_AABINJ wei_y_Page_006.txt 4a983c998836b12683f3c05c839f69aa b9bcce35cb0cd36343e64917d0c15bee83de11bf 30549 F20101107_AABHJS wei_y_Page_006.jpg 919f23fa5a4769e7edea3e5a7680ee07 ceec4c1dfb4986f0bfb4257df89aa60a091e80f1 78236 F20101107_AABHKI wei_y_Page_022.jpg 8828eb7eda9f7c4f1ed26ffa57efcf3b 4e5301e585a76e1546728da38446cedf151330f1 3053 F20101107_AABINK wei_y_Page_007.txt a489b94f55523050a6f400321b4a3918 d28a5a4b64e4c27da854addb130a27f664f04659 132100 F20101107_AABHJT wei_y_Page_007.jpg 80bc8a4fa1a692461b0d5c2544f2352f bf033dbade34da8d7d7285adf2513d69d90649dc 14549 F20101107_AABIMV wei_y_Page_185.pro c8dc0a5eb71648e2c871533dc50711c9 c18089de960e6446656f6b5631ae162dbfb8e4b3 71032 F20101107_AABHKJ wei_y_Page_023.jpg b563fcdc2e3561ef90dfa5452a925f48 3c5e089dffc7b1cf1faf1a2d275f9a81f9c507b9 111 F20101107_AABINL wei_y_Page_008.txt b8a166934800970ac1f97ef09e3d17d4 66e5db6c87b263709f6fc88aa239c58e99945ce0 10480 F20101107_AABHJU wei_y_Page_008.jpg c2b081fd073e12fb9e2c0ef7dcf534dc ec2666c04fa64beb1fd3a67b4f74f780d4b90d32 60248 F20101107_AABIMW wei_y_Page_186.pro f5f280623de31d40ad5271668d2ff9fc 573e9d1603738980656cc8fc0221cdeea9536b05 1530 F20101107_AABIOA wei_y_Page_023.txt ae1cd3146d6c9bc9b3e16661c721c97b 47469868fa4829e2eea2db9b4c9a04b600ca14ad 32068 F20101107_AABHKK wei_y_Page_024.jpg 09f0467e5d75e4a08c9ec44d4f7bc175 9d9edcb9f559f5eb028be2c4fd068f77bdbec198 1916 F20101107_AABINM wei_y_Page_009.txt 3f9c1848aa4a92943a35e170b9e02284 2560de470c6a0521b43f179131a14610628ca355 92442 F20101107_AABHJV wei_y_Page_009.jpg 94b5c4513badd3c29a4a780201bad38e 1772c32f15a256591088f3661d265115ff54de7b 65332 F20101107_AABIMX wei_y_Page_187.pro ca72cd8597958e2e1b585e94b21ea316 40b89406929b751c021dfcc6367c4bdb8237279e 1075 F20101107_AABIOB wei_y_Page_024.txt c6adf17a562ec35d77d7473cfe9893bd fd51863d34718581f399fbe8c42b7d83478a55b4 88061 F20101107_AABHKL wei_y_Page_025.jpg c1a64ef4425f147fba01dd4502d5c2d7 26b19eb8af95984371385d44c74f04017699562a 1391 F20101107_AABINN wei_y_Page_010.txt e87695abdd2ae772eb3c34b09f197cd4 a8620941792247239c3656eab36a69915143f5ce 72245 F20101107_AABHJW wei_y_Page_010.jpg b20e2ac70599e5ced1ed0287e761d2b0 6d23bc8717ac56cb1dc47faa1234a041498ebd93 61091 F20101107_AABIMY wei_y_Page_188.pro fc7d8093aeaf53bd33d2be2d63db6a95 649dd285c0826a93b9fbcdd0932732b01f96f805 2016 F20101107_AABIOC wei_y_Page_025.txt fd2719b63988b68a740c7fa071680d8c fe2073ff47462416e49003d3c59ef46ca61699af 116121 F20101107_AABHLA wei_y_Page_040.jpg af52dc97c885c6ba2709d8cc1161176a 5f8e1e2f548cf09da134ae75077d4af1e95b70d4 72956 F20101107_AABHKM wei_y_Page_026.jpg 6055c066ab6e4c875765eb8d95bf44cb 30da4206291e45d33daa83bf36933fc0a93c00a1 2165 F20101107_AABINO wei_y_Page_011.txt bbee1915655fec3edbedfc563f9b7f95 e46a95393fb187f533fb9c4c5dbb9057128271eb 105362 F20101107_AABHJX wei_y_Page_011.jpg 4ac3b7a23a3ff3ba121878244e0a430d 6de77a6182759094ee7669957b2002f7b6f15db5 61895 F20101107_AABIMZ wei_y_Page_189.pro 5d37e36b0aa7677c45642c6adb3bc048 16a9cba41b5cda2221c8c4290bb18e8086faa654 1700 F20101107_AABIOD wei_y_Page_026.txt 7f0459d5b1c7f009223e161970654a63 739147b60e83807667413e388143370a05c7edc0 2028 F20101107_AABINP wei_y_Page_012.txt 155c145535f4fd891cf21d8f8db39290 817a0792198fe77f62dcff8c083471b0117e8896 29502 F20101107_AABHLB wei_y_Page_041.jpg 9ff9fad6a30396ca3b51a0a46bab9657 4942c5a4e7b48e07e771fbf202f34fc155b8db40 101655 F20101107_AABHKN wei_y_Page_027.jpg 3eb5f979bb5a2d6524c1aeed89002148 6fb5be573e62b2d63ef0cb635b627b99b3ca34d6 104042 F20101107_AABHJY wei_y_Page_012.jpg b9c1a945731c382e642dfb9872cfa6b7 37264ae3ea09ba9ae671b47051d76f66e4e5aee0 1958 F20101107_AABIOE wei_y_Page_027.txt 11db8335ff9d47edceb115e62ab2b005 c9988917239504c9686e069836b9eee8468b5059 1533 F20101107_AABINQ wei_y_Page_013.txt 7c68598f6e9a03adde249b925138449e 1f47eca8b119f6b9f105c4e6f211538404400f1f 102660 F20101107_AABHLC wei_y_Page_042.jpg a327c196bfaea5bceacd6df14994cd09 15ddcf8cb149115b3a2ba6d01942dc3e8fc5e9af 65602 F20101107_AABHKO wei_y_Page_028.jpg b51157c95bd54d183a6afd917915e6d9 6f6b4eec2e4e86157354300b105b07c7d55ed852 65545 F20101107_AABHJZ wei_y_Page_013.jpg 207fc89961fee27d4f64b95b005b67de aa745b80e4d2d3cb45f6df809ab91302b70a69b0 1755 F20101107_AABIOF wei_y_Page_028.txt 934b65a261cb5ee976b27cdbd7425878 307b93bbc1c20fc75c4ceb51ba76b529f07185ef 112174 F20101107_AABHLD wei_y_Page_043.jpg ac45717d10ba0a462b5245c52f2253d3 3d42b40becef319e83875450366c44e0b364f119 1534 F20101107_AABIOG wei_y_Page_029.txt 585cf32505e996c00b9e31589117e0c8 425b6d91523eadd8af8c71d71cf588371f316acf 1505 F20101107_AABINR wei_y_Page_014.txt a9cb503c8f58c17630aaaf77d1b9c7ab a6a216ee7e52085b89c715734590ba15cc02597e 112694 F20101107_AABHLE wei_y_Page_044.jpg b8dcfa999f100275777df443587f221b 5cab87657fcee14d39ecf85414c5140860b69d89 45847 F20101107_AABHKP wei_y_Page_029.jpg e0a301e161cb3261194525f0d2e9a206 61073466f17e803471828574c11b427ec21c1b84 2084 F20101107_AABIOH wei_y_Page_030.txt 6e63a3b3f07526bae1824fba437c14c4 ad21865c9ef9da32478125f688f8bb7f4bace9c4 1929 F20101107_AABINS wei_y_Page_015.txt e04e823968d97713f78c4de599b9c1fe bbbec32a4ae6a6995c66bd47354a5881e4d19028 113832 F20101107_AABHLF wei_y_Page_045.jpg cb3aa9a5b1fea045f9cece80ffd21491 ada7e5fa69c15e67a830363bdcab2439d9d89f06 107552 F20101107_AABHKQ wei_y_Page_030.jpg 163c5d9e892cc52114574d714cbc8589 78f77048c7dce0d8c5fd3e88d94cbb32dafee7e1 1726 F20101107_AABIOI wei_y_Page_031.txt ebaccee6633b24777893d63c41f5d66b bc45fee108c702d186dda7b5ceedfe9e8a6f78f1 1774 F20101107_AABINT wei_y_Page_016.txt 02fe57ff22ed2a925eefdc3f2142f7e0 b0a6e54d7e3e751c6490f0e769884d416f0bea80 113922 F20101107_AABHLG wei_y_Page_046.jpg 65adccf8b5d0b6f3f33c89dff9a029f4 2e54520509f58a8042164581d918f22c5f4f4cb6 62488 F20101107_AABHKR wei_y_Page_031.jpg 76811fac45cfabfae8c9babeb7dad0cd d73605c4d79f75955067d3b30545dbcefaf8367d F20101107_AABIOJ wei_y_Page_032.txt e44faa77624ec267ac5701382696c24b 35270e1cf163d5884be09fdaae842ece3090d9d2 1842 F20101107_AABINU wei_y_Page_017.txt 09fd297c4cf5259ae4450ae092695e71 ebd60b5ded751aa8c79cd5e8182f0f173ede1d90 106461 F20101107_AABHLH wei_y_Page_047.jpg 97b2e00f9d4fda0e9b690442ed8ce6e4 fbe742c8e33527fa2ec11cef9d6e667adfe8c962 84987 F20101107_AABHKS wei_y_Page_032.jpg e1b2a7811e49858afb63cead5c58a735 4f05c55afa159440d9761abc3e0cd9d992065b14 1903 F20101107_AABIOK wei_y_Page_033.txt f9650936209ad1a81cc5298ed8851a0e 56c15f2d71ba82e7c32d2c49364254a088875119 1607 F20101107_AABINV wei_y_Page_018.txt cac46e847d7cca749b516eb42c61e1e8 0efb3b14ef2a07e6094469dfca4b9fdfd915dde3 101491 F20101107_AABHLI wei_y_Page_048.jpg 38d5a68181c59e1347893f1f96cf8265 a0d701fe227404dd54ed12b936afd627af0eb262 82491 F20101107_AABHKT wei_y_Page_033.jpg 5ccf6ecc8a978e227b9d3bcdcd3fd14c 009e5f7a79ff1a641b0124af824b248d6f491953 1632 F20101107_AABIOL wei_y_Page_034.txt 35d2dffaa8cf52a0f6dec81efb03cc09 3640386a7ea8e721723cd6e7caca9b2b5600ccc8 1741 F20101107_AABINW wei_y_Page_019.txt 36e1e270fa7c932ae3f9ea303d1f251d ad53802ea8a25dfcaa452aca8014f5d735283900 46165 F20101107_AABHLJ wei_y_Page_049.jpg edd704c8c0c777aa8d3a42273df5af54 25bccd1f4852f38957209805b6ba93d6ef907300 62404 F20101107_AABHKU wei_y_Page_034.jpg 854c5893a1d36b12d2e1d3f54ad450c2 a215316cf7125e82fc8b099112c78a4b61314700 1430 F20101107_AABIPA wei_y_Page_049.txt 2ce8b034d912b678ee5e156e3d19a607 2b6128f6a100f5a731f5e53b55168aabf24519a2 F20101107_AABIOM wei_y_Page_035.txt de0dba5e16dbd72bc426accbb4b0ad45 170b8cccd194f2a93a1288dc37d421d1390898fb 1770 F20101107_AABINX wei_y_Page_020.txt 8499377022b9e603111a5d555d5e1127 0bd683b44566ede4c659c3d28cadcc2646e3a08b 94201 F20101107_AABHLK wei_y_Page_050.jpg 5c775478e9fbf23c07634fa636c16c0c 9063cabe1d8993e6344e1adeeb88304d784a9502 70647 F20101107_AABHKV wei_y_Page_035.jpg 4bc6962982de42db1057395d4e28bf7f addf7987a91e2cc3a57544b2d342f1188e0043c9 1960 F20101107_AABIPB wei_y_Page_050.txt 5cea4f4f02a4074aaf1ea76a776fb9a0 219c0e0dcb00716c1c4070994c24a1ff0a2d0dc2 1866 F20101107_AABION wei_y_Page_036.txt 8061f8eda33b51fcf7f6fd706c32f95c c93f8159f0d210aafb973029c9a8efb3f4a59cfa 1620 F20101107_AABINY wei_y_Page_021.txt 7bef6581e2107376717067c17915a00e 3bc9b54dabdc04a56e9b9a720c50798c18b983a5 109799 F20101107_AABHLL wei_y_Page_051.jpg c061a5d2bc1d206a810afd9f56648f3e 75e563a8e7f3bcec15c13ec937eb060d6e88daa6 82548 F20101107_AABHKW wei_y_Page_036.jpg 53d33b9003b81dea16bc8ed837a41063 260fab70d4aa4df8d22a781188c9be2e41ea051b 2098 F20101107_AABIPC wei_y_Page_051.txt a06f35f8c0241dbd765a184fd6d0b701 2abd060387970932d8f83c26b8e5bfc63a74e393 F20101107_AABIOO wei_y_Page_037.txt a7025384e45fe11944dd2f09c5de9920 abc7e6ca71265fbaffd138d7a7d1879da4e2f37f 2211 F20101107_AABINZ wei_y_Page_022.txt 9421788f9856f20eac5ec9ba72e7cd03 89d6369e415a05ec0d0551d845c096011664271c 106934 F20101107_AABHLM wei_y_Page_052.jpg b608a2821d3eb9b51fd6bc3a4bbd47b7 f84b3a1cbe6b0fd294bb5bb8ac19ae7b80f6f560 86636 F20101107_AABHKX wei_y_Page_037.jpg 9167c632745ec8e25f3e8a6386e0460d 2475ac85012d4b31a87f51f1a541efd59f1be0aa 105905 F20101107_AABHMA wei_y_Page_066.jpg e54c544f2af3a8e5820a7a16d3a7971a 1826928f7947a5c76b1db3c769a23320aad483f3 1994 F20101107_AABIPD wei_y_Page_052.txt eb4de7844f567474aca33befbb9e0abb ec058d44e61ef5d4aad6e2713484af7b6176ca6b 1595 F20101107_AABIOP wei_y_Page_038.txt 7040945917d6eb331ec23d29265d57e7 c86f75b23ce5a7feefbab6ffa89898dc811809c0 12677 F20101107_AABHLN wei_y_Page_053.jpg b37bf9eff60694f5302087048bc7fdcc 629953265ba6ff851c6ca7e2fcece55ec0717455 74547 F20101107_AABHKY wei_y_Page_038.jpg 7ced44ef5ecec1d996c31d068f56afcc e3f44790bc13e11d9d7beb4b5486463ea1d338c6 108082 F20101107_AABHMB wei_y_Page_067.jpg 042ecb565f77ee0600212d9d3ac3eaa6 bc0d1a99e89f0e977812bfbae4dfe50ab79e61b5 195 F20101107_AABIPE wei_y_Page_053.txt 90f629fc9f94931ba6e6a2418877824c cb7bcb62e080509bd65ed5721d7635b7bdf59947 2029 F20101107_AABIOQ wei_y_Page_039.txt 892450d86f9ef8aa0301399be43a4bde bb1d8384583944a87bd939844e6df4b7c6d98f92 21210 F20101107_AABHLO wei_y_Page_054.jpg ebdb5a00a3a6e04eadf6d9a08dd626ea 223c3652dd8a820755244da1343ca1d1496a8646 102547 F20101107_AABHKZ wei_y_Page_039.jpg 2d284c2512534bc31a6b13555c6ad7bd efa504f177b89200ad6b9caaedfde2fc57967630 53801 F20101107_AABHMC wei_y_Page_068.jpg 43fec9559e6b41c26e00a71152319be1 cfdf401b7f66572ffc826fe4ee7e77ff38b4fc5b 364 F20101107_AABIPF wei_y_Page_054.txt 39e4e951fc8f0f7c3abf3a8206f517b0 439a3c41e1eea25e188749b9b09ae1a363de5373 2256 F20101107_AABIOR wei_y_Page_040.txt 7217eb91b0fcdedbd1f6bf8fa7c31ea7 be36af83aec6d264f8154356bdba6744cfb26db8 44440 F20101107_AABHLP wei_y_Page_055.jpg 976879137cb64140237f71c072897767 6949db89471e144ee069eec82c8d5667ef3c55f5 48787 F20101107_AABHMD wei_y_Page_069.jpg 2c565adcf361680dd328331844575d8e ec5da180c25c659d71ff2b271e767c48b1b9221e 767 F20101107_AABIPG wei_y_Page_055.txt 7abf97af9998bdfaa2e4cb4764e71124 39f0b704f3fdd73ec29d57b35d14940cd3bf3adb 46470 F20101107_AABHME wei_y_Page_070.jpg b715b71e78117034c679853823272b24 73805f4949355efb020b9c83bfbae9ad420cbf8c 1369 F20101107_AABIPH wei_y_Page_056.txt 479c08585759d1c8afebec5bf35cd665 42d4f60febeb4dfb2ca985d3e37b0c9fc86edbb3 533 F20101107_AABIOS wei_y_Page_041.txt e1c0b3df0ec1bc6757a968478a4f4e52 b3dca327ff8fea1b08341dcbd5d9c67abaf2be51 77073 F20101107_AABHLQ wei_y_Page_056.jpg b0c0e280f4fe6da13bc0f97a40413acc d0be6c4e55b3f2255f97320395ae804b7dfb9051 46701 F20101107_AABHMF wei_y_Page_071.jpg 05950d62e14bf745458c9cca1e828191 6c600d76f6b4e351a7f69d73e40f450e38b98c61 756 F20101107_AABIPI wei_y_Page_057.txt bb00c3a1b4e803bcccd66663cd6f6a04 302d70b89ef9e3c2a14ab90d7a2ee0802541d96c 2049 F20101107_AABIOT wei_y_Page_042.txt 00353682ab509cb5259d56afcd5004c0 e13bd45382957cf17d1bf048e161a45a51d2e191 44265 F20101107_AABHLR wei_y_Page_057.jpg e178b38a2f69bfffa8bea85caea7455d be7bff8b97cdc0a800485ee44e7037d1b0f4a0bb 57536 F20101107_AABHMG wei_y_Page_072.jpg 29f817d544f15d3fd32957001198939c 5d32d52fc6652f085765ae75b795c86f560f2f19 1358 F20101107_AABIPJ wei_y_Page_058.txt d3c3aa7c2165d3931fc1670b551da59d 16075a92a620c4933a840774e8372e6504539530 2200 F20101107_AABIOU wei_y_Page_043.txt 515cab80fc4cb5a549f58861f781ad21 4e600762c975e26821f71a5604aac8e414d3ab80 76782 F20101107_AABHLS wei_y_Page_058.jpg 967de1dafb31a0c62aca12cfe263fb4e adcc50c31603c4dab5be7656e26f606e57a9a354 54561 F20101107_AABHMH wei_y_Page_073.jpg 49399d5611df4103a3972a4ff0060248 b9e6aa2f62cd165ec61ea10f2cbba2e3db55cb19 2106 F20101107_AABIPK wei_y_Page_059.txt 1aef788945ea95057d632f70b4551864 87b9d5ec8900fd70e509708e6e9baef55a3b0931 2169 F20101107_AABIOV wei_y_Page_044.txt 10570f6dd92948d6c2ad964da37d6dae dc31d4761cfa7fabca48e11cecdb5cf7113318c6 103829 F20101107_AABHLT wei_y_Page_059.jpg 463eda8e263c21f1b3b60b7fbf43b300 277d05cbcfc1f5e1ec024c5a912cdc9cfb6009d1 54372 F20101107_AABHMI wei_y_Page_074.jpg 979bfe1bdc25d7ed03284e8cd5120812 692ed9c007ef9045697d459f3fe0e65cb61284a4 2174 F20101107_AABIPL wei_y_Page_060.txt 8acf93377fb0e7cf976cc2fdcb53d735 1037b322950affc7071ee159e04e68a486eed707 2155 F20101107_AABIOW wei_y_Page_045.txt 10495f8e2f0185df251f4a8d0292d571 3ad665f7f8558b0099f1dc666365e7e6cafd6690 109197 F20101107_AABHLU wei_y_Page_060.jpg 98ddbf2841df0ff46dc4070384f26571 f532f182b7a2e871d95513394e7314c6958bf7bf 55571 F20101107_AABHMJ wei_y_Page_075.jpg e882d97bc5c02e38b24050ace57c9521 9243c76b03957e46f7de3e9e019884faadd44899 773 F20101107_AABIQA wei_y_Page_075.txt aa658b0f65893c7c3b5ad19b234e54bc 7bec0398a4075ba35aa2740304a77032791ee4b9 2156 F20101107_AABIPM wei_y_Page_061.txt 46051df14516d63c6b1cac79559ffbce f626777034c9cb6f91c12e6a6be66b7a4a284a7a 2221 F20101107_AABIOX wei_y_Page_046.txt 8ee70516b594d47895a258757d42bc90 a5f335ec0c0147a6ea54d643dbae5c1a18ed70bc 109289 F20101107_AABHLV wei_y_Page_061.jpg 191ebbe96ebfc5f31d6ed99d7f2b20b6 4fe45848ca0b6fc5250bf498fca49e5bf19ab34e 52773 F20101107_AABHMK wei_y_Page_076.jpg 6331f06cc7515a9089231b4b443c64de 31e0b784329ece1104436cbfdfabfae95009b2e6 746 F20101107_AABIQB wei_y_Page_076.txt e1f3dc79973206606f46d991d14d4e14 8e1ac66bf3cd29b606f4c37e9c29b863903fc7b4 2126 F20101107_AABIPN wei_y_Page_062.txt 3673611f6e1b9fe491f534a8c86370d6 ea557b2ce885af25f2b4928a3709a4b09b4d8859 2088 F20101107_AABIOY wei_y_Page_047.txt 8b6658325d38d704c77572b9bb6ef017 ab8743265968a91b0ddab5d6bce165dd9a4de4e2 111244 F20101107_AABHLW wei_y_Page_062.jpg c00bf48fc2a30fcf60ff25b8c6075422 bb428b353a6df9d1c52fdd09ac35940d4007c29d 52182 F20101107_AABHML wei_y_Page_077.jpg 31c1abf191cb6afc025b3e6507121595 f199f9a7c00adbc2087b11c2b940089753b84a1e 764 F20101107_AABIQC wei_y_Page_077.txt e99d4ee6e9dfef55359e6b049d253feb 916e39a707f20415f923ae162fadd6de393e3005 2074 F20101107_AABIPO wei_y_Page_063.txt 3e1aa9786d0a4acd0d9fb794bbf24638 87368540b09818def312e515b846570a8f6fe22b 1975 F20101107_AABIOZ wei_y_Page_048.txt 47f11f20fd47120ec98ce7536c141d4c a2ca525ea886267ef79d7de3142269af1c414670 106131 F20101107_AABHLX wei_y_Page_063.jpg d7b789426fe9f6573195810217d7fe8d af37246169c8a78234cdc390d4da55201aa44be6 52159 F20101107_AABHNA wei_y_Page_092.jpg 1cec54fe089178e8dbcbc30032191e5d 01a3b9779a12992c25344bd1611298bafddf2f55 57728 F20101107_AABHMM wei_y_Page_078.jpg 0ea76ecd55a914e3082a37b3fa279837 375c44633f229d5a05b51aac8037bc6f032307a6 871 F20101107_AABIQD wei_y_Page_078.txt 9404f016c922d00c71774999e5fc0f22 e65a97a692f579329fbddf927f427f42e7175bb3 F20101107_AABIPP wei_y_Page_064.txt 1e998006696f822f1fd54292335e1e0c b92283ddb63d28ba7861ae4666271b66a624ff86 107775 F20101107_AABHLY wei_y_Page_064.jpg d1771269e84994b31fd8c8e7457c30e1 923e30f0b3204fceae0103d54988f95e3d3c8143 68539 F20101107_AABHNB wei_y_Page_093.jpg 45ead9cee61d50e8d336054bfbda20ad 453699848770d41ad50d826110555bb66726c0f5 53383 F20101107_AABHMN wei_y_Page_079.jpg 96ebc6325bd2ce4eb9e2e074a8d24b4a 3404bd9bb35d6cab66d83fca3d93a21b73c12c4c 656 F20101107_AABIQE wei_y_Page_079.txt 702d9df271f6d6aa8c4361da7446c3b1 652c6baeb80e81b72239f4f1f780f106df428d50 2198 F20101107_AABIPQ wei_y_Page_065.txt 1bc0fd1976d6323c67fa9047cd3bc277 afc398d785e1f01d05fecd4eee5a671605a48d60 112889 F20101107_AABHLZ wei_y_Page_065.jpg 6f24dee827d28705c17ad6ce3c4306c0 caeb6ed85ff8e5ff8df07c313da5116783de12fb 66008 F20101107_AABHNC wei_y_Page_094.jpg eee700b662e049b1a702900cd3faa47f de9c9b09679fe02dff75725dae395d0f6257d218 52753 F20101107_AABHMO wei_y_Page_080.jpg 88d6dbf254660c230670204476922bb5 fb3c157683959dc88b443e966b8b453e24dedfdc 651 F20101107_AABIQF wei_y_Page_080.txt 289a39488767131837be263ce665585f 6cf1c981dfa7df0ce9f91e43676c06a795b43966 2045 F20101107_AABIPR wei_y_Page_066.txt fc6fb2442cdf8265eb4e3f378dc0d9d6 f90448dccb216e526cf345c122507200a0fd381d 66266 F20101107_AABHND wei_y_Page_095.jpg 660f744a29c68f2e8a8e2cfcc6b41ec2 554888671fe6ee480edc252bd9ee8c89873c474d 60571 F20101107_AABHMP wei_y_Page_081.jpg b91e4a12e5c18c87e0d12a752150d99f f92edf67e8ceb0a1334d361a62aacf45b1c4783c 851 F20101107_AABIQG wei_y_Page_081.txt 295091e22dded60aee48fa06ec638814 4fdb0165954834c53f43174f61438b71067b915c 2167 F20101107_AABIPS wei_y_Page_067.txt b20ddeeb48470cf15b2e1fc5413ba747 92b5ba2ee4efeba518660d9f4a9e33612480e420 68164 F20101107_AABHNE wei_y_Page_096.jpg 3069d9a3764672d2def8f23f2a59c1a0 d6cf27031bf19b74452f2f3e07426431f08c80c2 55853 F20101107_AABHMQ wei_y_Page_082.jpg 077178a9a240b1b34515d696da9dbf41 9cbfca4f37d0cecc0263a322bc0dd84f05f3d98f 900 F20101107_AABIQH wei_y_Page_082.txt 3a076beba73a68ac8cbff3828b6523c4 3d7c95f6fd77c7e5ef5fb030e030f6852535eb06 64217 F20101107_AABHNF wei_y_Page_097.jpg 542b417b47443de50c1a23fdd5a51aa6 9f4474bbf8e4385dec9506c5ab46a3051a59b2cf 698 F20101107_AABIQI wei_y_Page_083.txt fc67db7838c0a08abd5d81546e4c4692 06d0a9717b28c0c02252cef057a2a202592c5804 1055 F20101107_AABIPT wei_y_Page_068.txt 2dd81c9a074fe8bc035cc6562a0eea15 a3c6627d8bccba401bcad633bb47053eb4c1ae63 63525 F20101107_AABHNG wei_y_Page_098.jpg 7fee8e4a2c4748682bbed9c364df09f9 2ebbdc3804b9e02cee353d588b62c633b42e329f 55533 F20101107_AABHMR wei_y_Page_083.jpg 2d800056b8ea7a680099a2d63caf8259 5ee2551b368efd80f4d1a6b56f51709af1530a24 888 F20101107_AABIQJ wei_y_Page_084.txt ac976a85be406f9800b5b9c63b126e3a 831756f589a71a0e767027f9fab7799621aa5607 607 F20101107_AABIPU wei_y_Page_069.txt 4d7988a0d47db3f6fb3eb884cf63a719 20828f2be587c20fd407b157999803c58caa5a9c 70052 F20101107_AABHNH wei_y_Page_099.jpg d2291be45afb03a9dd1dd5b21c6be7d9 e3fbdfb6af57198d71023ce94db1d190b11bc2b8 58851 F20101107_AABHMS wei_y_Page_084.jpg 668cf5019ac6939e2deed7251ed13ad0 a17364aba2a644423d063ee6e1bfe23509219f61 660 F20101107_AABIQK wei_y_Page_085.txt 3a79568e28108c3d5f79f1101bad1a45 acd22afbde5045a2a7e223aed7def300bbb0ac19 704 F20101107_AABIPV wei_y_Page_070.txt 850f2b366a71a7bf04848748630c2ba6 ff35ef381c9493da7965c983391cad641a4310aa 66452 F20101107_AABHNI wei_y_Page_100.jpg 9e24dde00fecd643c9a130d73720aea3 9bcb10ec1427df135b1c160416004fd8c3487e34 54707 F20101107_AABHMT wei_y_Page_085.jpg d30c0f7ab56ef409b94faeb17c478b4c 05d608a7449f37a28ade6ef55b739ff7d116cce5 697 F20101107_AABIQL wei_y_Page_086.txt 2da1d8b0c4b55d1ddffa38f9053f83b7 12ab92ad72e7de1645a1a21a8fcb278571fbeb83 666 F20101107_AABIPW wei_y_Page_071.txt 29e1791f12764d6f6df59fc08abcce9e e38cc80d143f1733617a938ebde1e3e0c49a989d 66336 F20101107_AABHNJ wei_y_Page_101.jpg 3398d769b729490be7f0a1e5cc1ee004 36f1020c3943ed32ac41280f4b39e96d0e41ae34 54046 F20101107_AABHMU wei_y_Page_086.jpg fe52bc829da2f6cc9226d70b4db6957c 90df5c026cb5da78ddddbfec43fb14f17472ac64 969 F20101107_AABIRA wei_y_Page_101.txt 4d55204f734d906f3e96eba1e2a58d0e 17743ce1682d949169cf3e6f305f0046d0362f5a 883 F20101107_AABIQM wei_y_Page_087.txt 6d6ffc57b71866a8cb3b6ffcce5fbd16 a60b41827cdb6934e377f0f51b594b90b2c8faf9 600 F20101107_AABIPX wei_y_Page_072.txt fbd6d1f8ec0ec1d8ab371e8667ffa219 4153b418dd6527dbf059f7676b16f25dcb9ec5f5 77122 F20101107_AABHNK wei_y_Page_102.jpg 636bba303accd736b7f0992e3aba8391 6c11ef501b33619a648a255e72232c0c99a3f71c 58940 F20101107_AABHMV wei_y_Page_087.jpg 857da1aaf6da4cd590f63468bff9912e 37a6af0f60a2edc712b38ccb0530ebe333511222 954 F20101107_AABIRB wei_y_Page_102.txt 34a31beabd7045279d4b8bdcc74b9be8 4c078c681e3f3132f3a304e81918e7ebadfdd638 672 F20101107_AABIQN wei_y_Page_088.txt 03c94319862f954fcc857f754bab5e8c 728f0ad1d34368d6a3169af7c750bddde3f7b012 742 F20101107_AABIPY wei_y_Page_073.txt f98b0b8e7e4629c899aa7a26674704d9 713450f6a5beef0421dc90fcb074ea00cd5bc039 72960 F20101107_AABHNL wei_y_Page_103.jpg aa1a8e4e8986e367b9c43fdefd8922bb 40b3344ea9a74a243db11f073af7eecccd78c690 53867 F20101107_AABHMW wei_y_Page_088.jpg e3bbbbff5f6d26f84f586a6292e82382 e7b533589d837e568dc1cb26ff99694607a139e3 779 F20101107_AABIRC wei_y_Page_103.txt 7b49b6cd5f651ba28ddcd0081dc982ef 531c78afa6a92072d4f5f7c1d69925719f6745ce 653 F20101107_AABIQO wei_y_Page_089.txt 7237e3422995480b68a08907ba29ce30 30d27f648a49707d51eb4e3fbb32b71180e0deee 652 F20101107_AABIPZ wei_y_Page_074.txt d82434fba6750d1ee0daff80348006fe 4ee2c6808bd4547094940a4b24d7fd0af8573198 40393 F20101107_AABHOA wei_y_Page_118.jpg e22cece7d4f2cf1cd0a9f208e61bd2bb 02b1b7fcb8d835fc0cfc94164b24f2925cb73e0e 72512 F20101107_AABHNM wei_y_Page_104.jpg cb707a65a7d7950c81771d534f5f7de2 5db83124095d7dfbd94a8e70a7a5a977da72b5a9 53018 F20101107_AABHMX wei_y_Page_089.jpg 366a8c1de6091a044747d3b858022625 688ff3549e607e5a681811058d3a936d91945c68 794 F20101107_AABIRD wei_y_Page_104.txt a0c4e8feadf3ec5c9ab7990535c3a568 fdb346ce1053adf8dd7309502fdd35c427907602 905 F20101107_AABIQP wei_y_Page_090.txt bff024339a8814b7dc4917670e7ac3ea 333cf3713073e4fd679d7515fd67c49134bf058e 46050 F20101107_AABHOB wei_y_Page_119.jpg 0cebf5a1f60dfcfdf6642b185835aa77 e75a37de4d0cedead2e737744a97f1eba98706d6 80375 F20101107_AABHNN wei_y_Page_105.jpg 91eebd79de54ffa11fc0dec160334a88 a47c6da63b5a6f59c02fcf80aecd253e72c4c888 58371 F20101107_AABHMY wei_y_Page_090.jpg 2357d59e1b44331b4987791f845241d9 1b1424cc8cf877b8ab1af2f5719c4fd30591094d 801 F20101107_AABIRE wei_y_Page_105.txt 65c6e49fb7edeb94245b1a0ce568a0a0 9ad4950af45a1e3f093a8924a29c2e97939f2e72 712 F20101107_AABIQQ wei_y_Page_091.txt b2abb9163878b8125cb6ce1d957bc2b3 d2bc614ebd0824be47a2c92b98e8aa3428c0c13c 36074 F20101107_AABHOC wei_y_Page_120.jpg 3156fd6e01dce00d23c2a1ccca334f36 ae751789ed756e36a2ae1c4b306dba102e4dc508 75025 F20101107_AABHNO wei_y_Page_106.jpg c283a2e0fcaf3e78136d310cb802b03b c0d2a2442df0cc7a30e9a1a464b10463595ebc37 53057 F20101107_AABHMZ wei_y_Page_091.jpg a4d1de4ef7eb3bff9ed7159c62dba57b d930ce39831b56d1751eba8f6c26c85807e37f7b 946 F20101107_AABIRF wei_y_Page_106.txt d9f120b36e51d31cd5f719212179916a ca9084f4fd7af516d3947c7d1519159dd9fe770f 678 F20101107_AABIQR wei_y_Page_092.txt 3626b80db8d1f72bc30c270f2a8745d3 477b9e31f017e2bf0961fb65c2fc551d65becd42 43276 F20101107_AABHOD wei_y_Page_121.jpg 4daa229eb3097250dd51f982fbadaeaa 7a1695afa35a58f1c40d06b70b4874f063de90ca 76954 F20101107_AABHNP wei_y_Page_107.jpg 75ed3dde6e937cd26a971c094b49d266 472a21015eaf0a51f6f676f9f56bcdc144ac80ba 1249 F20101107_AABIRG wei_y_Page_107.txt c06e19cf6348f1ffe4680b28d1134027 1f95d76fa0f051ff16d05974c6c9cb0958a3667c 1319 F20101107_AABIQS wei_y_Page_093.txt 4a70e14135db08fdbd99a9175ab4b503 655cf71524abefe3ed7ae909295820837e03856c 104659 F20101107_AABHOE wei_y_Page_122.jpg fb4f435e14e4fd1ec4de26e1466b6710 753f1d7cb3a84c18a147f4489caeb2997d14ad27 78233 F20101107_AABHNQ wei_y_Page_108.jpg bde371a8472d4ee0361602115af11bfa 487dff48423f3cc3c11713027190c2aef218bf59 784 F20101107_AABIRH wei_y_Page_108.txt 659672e686f6fd56f2641d68eae152a7 a857fcc9f37f99dfc508febf9fbad9ea8ce3e180 981 F20101107_AABIQT wei_y_Page_094.txt 7064b6147802d1c10fec46aada36940f 690d27da4ec165a29d4153d8bb67aadacb9c7bbd 112638 F20101107_AABHOF wei_y_Page_123.jpg 9c729842637815246d9ffd18278cc2c7 84605d37ffe4bd9b90e0031e74809748e686a08e 74810 F20101107_AABHNR wei_y_Page_109.jpg 346c0c68191a4215752fb838acdc14f9 d022867da8e16a07452c39c81e0a04203caa40d9 922 F20101107_AABIRI wei_y_Page_109.txt ad45f7de726253bd69924f304cdd1e0f c79dbcacbde760a5eb05a637f1ce907ede45c1cc 116983 F20101107_AABHOG wei_y_Page_124.jpg 2edc0838a80c66b1cbb1baa6e9b0bca7 9f3bf21058b1177efdc5b135716cdde5feadf2c1 913 F20101107_AABIRJ wei_y_Page_110.txt abad9815a76b21338618a9eef8217b28 fd66bfa1a346f249060f628bf6a8c1834dbaabd9 613 F20101107_AABIQU wei_y_Page_095.txt 1558c29efc4b67f7e0d35dbd9d62a21a 5d3a3f8e1173ec3fd122b37a1abec3598dea52d7 111007 F20101107_AABHOH wei_y_Page_125.jpg f2b9d7149d2e16d34ee8585b11529176 cef306ddfbffdaebfffece0bf13fe559ff40a0dc 74415 F20101107_AABHNS wei_y_Page_110.jpg 07c79529109687d1e9945b95a4091f46 bc34fe4727ce4b86e8a596f5a6cc068d5eead829 780 F20101107_AABIRK wei_y_Page_111.txt 123bb6abc04bc66e04c906bb340f63cc f62febac624cc68cd851208c2270654a614448a2 902 F20101107_AABIQV wei_y_Page_096.txt 8c9ec2d0f814805814cfa51a50ebf52e 3caca096e05c465a9639be939a1a1a68ca579215 86846 F20101107_AABHNT wei_y_Page_111.jpg 9e029b80261f6eb95a70cfc624a0439b 832177b36e20ddd575b353de87491b3006c7d228 112825 F20101107_AABHOI wei_y_Page_126.jpg d60f81fafc36f3cbecd7c2058100120a 118b19a391a088e7353fab5aaab63f9447376703 804 F20101107_AABIRL wei_y_Page_112.txt e5cecdca295824e0782c0eb625f2e29a 61301c0fde937d17891fd40bbde8fce2ae132b14 716 F20101107_AABIQW wei_y_Page_097.txt 11898de1548f061a8368b05ef5dfd948 91b80ccc011d66b4150cf549b9f8588f80178cc3 86530 F20101107_AABHNU wei_y_Page_112.jpg cc6d97e7b68b829246294f82bb3b1f0c 723853e46acec166d6c42187bcd7416a85033cef 111972 F20101107_AABHOJ wei_y_Page_127.jpg e2204fff1ed64e0d001f81ad936c6841 9e134fb15eecccb742ebb0dac555a31afc1dd65a 2226 F20101107_AABISA wei_y_Page_127.txt 8fe0d14134246fc18c9b54cecc568536 066cb94117e265fb0e5d1df0e7e5953966a73eca 1187 F20101107_AABIRM wei_y_Page_113.txt da79d26066724c54e9bad0816deed982 63c71d26f08c216b3d70d899438150061d2fd97e 863 F20101107_AABIQX wei_y_Page_098.txt 429218dfc44328d5c22cc263ab2ca448 bc009e03b90f0be9381f4d811d7c09156f267cc2 77214 F20101107_AABHNV wei_y_Page_113.jpg b2fd5cccebcdf73367b4f75ad9bcf16c 78b7bb177ea014bd7bf6bc8335c46232820565ff 110072 F20101107_AABHOK wei_y_Page_128.jpg 66d0709e91012adde5674e4a7277a58f 7e064e836d0fbca0e4c30095a515d05a9b268774 2137 F20101107_AABISB wei_y_Page_128.txt b171661533e02b95627a533287775869 5f17e979081e5347cf82c9eaef7f9679f0f0f229 869 F20101107_AABIRN wei_y_Page_114.txt 384adabbb22ad36e10bb6c5eba82b04d e89bf816f1e5716488eef5d9ba37f8a120cfecdf 975 F20101107_AABIQY wei_y_Page_099.txt a51e130faec586a9d3e3afa2694d88a5 d97ce1a6c72c63a004bf6ce9adab5e52c7ab4ed2 73476 F20101107_AABHNW wei_y_Page_114.jpg b74a7149f85ff55e981cf429a602ebc6 1154f847b6947d92c937145b3f6007e80856f44c 105371 F20101107_AABHOL wei_y_Page_129.jpg 1a8692ae82a2e6ebf32f85317f656a5a 3eadc41ed5cb3b15d32a2ed513d731ed1675ac40 2069 F20101107_AABISC wei_y_Page_129.txt b5d6f9381ec2f580de346797760ad065 77ae15edf6e77b0cfb309912617a49aa544e0153 F20101107_AABIRO wei_y_Page_115.txt d425614fe175a5cb2911e7e922f8f8c7 93c4d4db152f1e8eb1066fa4d8d47d5e0140b44e 994 F20101107_AABIQZ wei_y_Page_100.txt 03635da03bc58240d5af4149e9cf7d92 39081a9b7d7ea6f262a2abad6cd06ef63ee490eb 72161 F20101107_AABHNX wei_y_Page_115.jpg a8ea627640b6b11a93375e7a4689019b bfb00e2b5fb667a764e0289e00e8c8d2eeae0c7e 84884 F20101107_AABHPA wei_y_Page_145.jpg 7f45b2f6cfd314206974dc30f3f35d4c 7dce48cdc5dce2ec2ebcb706964534626261c5ba 104641 F20101107_AABHOM wei_y_Page_130.jpg b09d2cc81dce61fa83c63fb31be62fb5 4f613252f6613c752e367103253cfc2891598efb 2076 F20101107_AABISD wei_y_Page_130.txt 321c0da0da80db42ad0fead955719e78 2e964910a19d5491222a4f4353babfbcd4a8850d 1439 F20101107_AABIRP wei_y_Page_116.txt 5e6b8fb18330fe55192ce7a5e8e84261 10c2349a1df4ed8e828aee9d427315c5d55b48a5 40288 F20101107_AABHNY wei_y_Page_116.jpg 09590136a7add1feba00d98153f04202 89458bc96f1176da671b583beec382c94d956ec7 83739 F20101107_AABHPB wei_y_Page_146.jpg 864bb061a02409f4c1f929a8d9daf39d 3aea44aa0aca6fb8682bce19b295c2efadd6a592 111561 F20101107_AABHON wei_y_Page_131.jpg 8896bfbb1a2e3d290f1ad80d751924b6 036161998f07f0c21a41f9eeab59e67220645c43 2218 F20101107_AABISE wei_y_Page_131.txt f555b2182cc78c336a77921903027d7d c705c721a7d64eaffdb08c52f0510f4aa2561a22 1255 F20101107_AABIRQ wei_y_Page_117.txt 79d64b84a9903c605f84cc32e6dbc613 49328aed49c1b6f3b44d7ecdca9f5069f6cf6623 45967 F20101107_AABHNZ wei_y_Page_117.jpg 0e87b5abe5c85981bf6287c12f2ad2e2 04ecb2947a02ac15c686bf4c54ccc22e2547e297 83781 F20101107_AABHPC wei_y_Page_147.jpg d21e8ed262b5631d43da5a6e433cff10 9b441a3412637ac6133b3967e42f6965650fb1de 114260 F20101107_AABHOO wei_y_Page_133.jpg 49f8e90d29ed31e346b196cd04d48072 ac22fbbb183fe82382ed745edde061e3a63f7159 437 F20101107_AABISF wei_y_Page_132.txt a70ebb2ea0d0251e680e344e120c3e13 8d62dc1604d4cd95fa5799d9c574295e9d3bc49d 1184 F20101107_AABIRR wei_y_Page_118.txt 10286317ccfd1ae249071289ef4e46d8 e00c5788b9425b73661c4f2d2b8bf8c7d91dd107 84116 F20101107_AABHPD wei_y_Page_148.jpg 544f0a313ef5c3ca72dfa0fc6f5febfc bab47e65ff4815aa20d983e073a92d09dd25df7c 119279 F20101107_AABHOP wei_y_Page_134.jpg 9d949043c18fff9231e9659bfb8c4ac3 8c01f253c6015efc94084b8fba44c3c2d63b4b90 2480 F20101107_AABISG wei_y_Page_133.txt b655fcd97251603d51d7a004d804f4ea 10dbc82472667a47f8177bf6673047db1159d748 1320 F20101107_AABIRS wei_y_Page_119.txt 4f9045dbf715b7af5fd946ca1ba73857 a24433c9f9f1ecbbdabc138d10cbfd833a95bb58 84080 F20101107_AABHPE wei_y_Page_149.jpg 696403c2fdfd84ce7e5e6de9ef2d4243 02c04d1b0be2e8ee00a723f61e002424629d9bd2 107291 F20101107_AABHOQ wei_y_Page_135.jpg d1cd6be40988a81f05046dc588cf8a88 ceb573c4bdd7afb72276d6382fb7e7b4c7e97cb6 2544 F20101107_AABISH wei_y_Page_134.txt 7d0c8869bb65f258c8e3a9482b7dc199 2fed9217ae865bc4ff52d5d4781ecb1e50674836 F20101107_AABIRT wei_y_Page_120.txt 8f51b4ce996a6f6b5c4fc27b1ca456b6 ba187906748f7fd0eff1c89bcdf06fed461e3a9e 84888 F20101107_AABHPF wei_y_Page_150.jpg 2150b5f0dd61dd6a72f1904b2e9537cd 8de3d6a7c58a09be927d59236bba095ce369cda1 112450 F20101107_AABHOR wei_y_Page_136.jpg e693dc2f82287d0f3f3d6ef6306f810b 3611f572020fea419b670d0e2a497006288b5f7c 2092 F20101107_AABISI wei_y_Page_135.txt 435f06e58e28e9092e420299a94d5eb6 c693dc83f26bb327dccd70ab379ece60fe11dd02 1568 F20101107_AABIRU wei_y_Page_121.txt 603e21eccfac3324de6521f3aa811af8 01e2681f73c8f1efa6ec6043933195a5aedd2b5d 86849 F20101107_AABHPG wei_y_Page_151.jpg 5cc9fa8bbc77809a5f5d6d9b97758ece 8b9b8ed1673f26056f8fd4327541f5c7790474c2 18424 F20101107_AABHOS wei_y_Page_137.jpg 42277810eea190634c87d25764de8d83 d879842706661652f501f1e0c6fdcf44beb35ca9 2189 F20101107_AABISJ wei_y_Page_136.txt 82ac07756d99a5a12bf2e9e412d0ec46 7d45f49d15315fb7b9c46c06b25c53063ffbe090 85044 F20101107_AABHPH wei_y_Page_152.jpg 513f0bda26ceb5c275fd720b4e90d81a a495ca2d9f1e98b3a74733841fad226435869fbd 309 F20101107_AABISK wei_y_Page_137.txt 0333faa129241a8e8eb68009e8561ee0 681d3f89fd6113b170179d12aa142973a6bfecbe F20101107_AABIRV wei_y_Page_122.txt 093d2823f1e8202e5337ffd7f161cb30 fc7b6bf0f3b6c89792824bf8b5ad290fe71bc587 85545 F20101107_AABHPI wei_y_Page_153.jpg 98b65478478fc6bf4770fad1aedd7011 6a753f3991d1305bda80f53e027e4fe7ba9d53c2 88915 F20101107_AABHOT wei_y_Page_138.jpg 2dffbf7841404cda3c24f133a6d80280 7ceed38c8701efffa0cd174d083e8ce9902f222b 1256 F20101107_AABISL wei_y_Page_138.txt 5cc00712d9aa4f330db7c8f776bb22e2 fdcfb79d81168d5dc112128c65733daf294071c1 2216 F20101107_AABIRW wei_y_Page_123.txt 411d0e5bda9cd512a8c24021860eea81 c627f78ffbe1e8a5595a7f5ef496784024fe8373 84001 F20101107_AABHPJ wei_y_Page_154.jpg 28b01ad413c1cffaad70cb35117307c8 656d9a81e51e69457e1e1b4b39d2584002469a91 85906 F20101107_AABHOU wei_y_Page_139.jpg be4efa37adc41afe49be0cf0e7a0ddc8 5d06a29b87eaee4261e89a2d71b0cdec99e2854a 925 F20101107_AABITA wei_y_Page_153.txt a0ac30c741f1a9123e76fa58f771f995 7772b1f2e39925affdd9e823ef10fe3be7386cee 865 F20101107_AABISM wei_y_Page_139.txt e5932a13ee79622243d56df18786ffec 39b79fd0a4b2207c3666c78fb8a485c903144a8e 2302 F20101107_AABIRX wei_y_Page_124.txt f522c5ab93d5179b8e15eecd8367a973 8dea30b2f47ec02686447a8ca4be9b87a9d27d1e 84825 F20101107_AABHPK wei_y_Page_155.jpg 2ea4c35af25001d09b79dde40bd6d92b 236e25f28245a566ad5883e5103096432c855974 86839 F20101107_AABHOV wei_y_Page_140.jpg 62ad83da3ba229577a7987cd418dfc50 f0cf99585a5b7bec22b451d4032e722d969d4509 836 F20101107_AABITB wei_y_Page_154.txt 642890cabcaf4dc9aba2652364eda080 a5450651cf1ea0eda1670512181c07c68121f93d 829 F20101107_AABISN wei_y_Page_140.txt e501e0dca71c3ca795af2694f1ea41bd 02d3f3adcbee8fd1fceb01d1f3a863f1bcd5c519 2159 F20101107_AABIRY wei_y_Page_125.txt d1a3d1c41b1491328f6fc04b5f326b78 9a8da46911fc596800870101f7ff659ce9ec5f2a 83848 F20101107_AABHPL wei_y_Page_156.jpg 0adc3116ca473edea7138cb23d140a3a 161699d456ba2e2d224cd562e3f21a48ffae8408 86604 F20101107_AABHOW wei_y_Page_141.jpg dac146669244afccea685b68a90df2d0 43a36de2e0f38f4f0ddde5d1e60b459d738dcba9 862 F20101107_AABITC wei_y_Page_155.txt 9e3e20bd91bd8bd9191a3e4953fb084b bdeec14106f33566d60e9deaafefd909515018ce 828 F20101107_AABISO wei_y_Page_141.txt f35db62b5593c3ef84dbdaa48cc5c1ec 6530ab8baa8e883bf5ccff463cddb887ae794a5c 2190 F20101107_AABIRZ wei_y_Page_126.txt dd0f78a7f7665d3bcf4715d53a3b1844 8ace42369bb1740c93aad8b1e52f7b9ee269f0ff 85453 F20101107_AABHQA wei_y_Page_171.jpg 420a860199f0405ade03fc8c479a6634 35d4e5f378ff9388616cb881abae16c3ce678c94 84186 F20101107_AABHPM wei_y_Page_157.jpg 3ab648435d47e8db33c6183b3c08aaaf 80ec837f7378344e345f3a76cb6d412853590203 84449 F20101107_AABHOX wei_y_Page_142.jpg 16505491b73c1c56c1b25842cbb1bd54 0b772d552664ee9ff3a7aa51856da51d63bc2553 903 F20101107_AABITD wei_y_Page_156.txt ce414d78efe49018fba49356c7a4e9be 4b2ad0ec50c1b2a8bc9ccdb20865afe6c5b46faf 856 F20101107_AABISP wei_y_Page_142.txt 8c2fe0d7a5c12c9f4e90aad77fc73fe6 e2c96480a2fb6711da7d47c4b9aa46afa0857996 85209 F20101107_AABHQB wei_y_Page_172.jpg 66f10b0d9adaea831f320a0b9adc02e3 f6e8fc9db3d58fd46f235c62158068aa1a0d30bb 83200 F20101107_AABHPN wei_y_Page_158.jpg 312bd4d36abbcb1901b4ddfe0349d7a8 24eb7a315f7e78da05e7fe1981f60938a3a08fce 83719 F20101107_AABHOY wei_y_Page_143.jpg 5cbcde946dd88838af97d146199b27dc 93d6c8862de466a7947d7be35d3ba57f955e61a2 835 F20101107_AABITE wei_y_Page_157.txt 41e886f0a2ecc96e31c66f4a64645bf8 01bd60180328454db8ef0095b7de36844ce5c959 1134 F20101107_AABISQ wei_y_Page_143.txt dde764437b2d84ad837c6ea800aac62a 405729edfbdc24550be7afcd4bfb9407337925a7 85509 F20101107_AABHQC wei_y_Page_173.jpg b92b7a991c410772096de2b97d18e622 fd0244cd5b5f2c5a4dda24fbdbd4538635b69532 83511 F20101107_AABHPO wei_y_Page_159.jpg b89f4954cc44830761a1243d7260f4b1 1d305ff19c70443d8b7b35eb82ab3a5296c0eb41 85194 F20101107_AABHOZ wei_y_Page_144.jpg be3b52b035f886054fc7ec681867495a 066571c4bdcb438c586845319f1170964ee2a978 876 F20101107_AABITF wei_y_Page_158.txt ccb8c6ea0632943d5939d9813248f07b 44933bed6e362d6861e14fb79593c12a01e3dfbb 1028 F20101107_AABISR wei_y_Page_144.txt 88a970958c6807584b4dd3977b33431f 715e7087dd735729e557b721fe6757ca3c5e32f1 87596 F20101107_AABHQD wei_y_Page_174.jpg 7b962255856b8175bd67418fadeeabd8 427c1e74e34fcfa2e7a7e85cb3badc05c102f578 83347 F20101107_AABHPP wei_y_Page_160.jpg 96541863d6cb21cfbff35af7f6878435 cd4a033fbba37877ebcf675b5938ca32da562f15 955 F20101107_AABITG wei_y_Page_159.txt ffc41b6a0feca7584448638d360f4f53 542d9f2d487b19fefd7fd2f278442420db4da604 751 F20101107_AABISS wei_y_Page_145.txt 04dcd74f0dd22c7a7f91cc28e403cea7 c42570dc83e03f576a1e40b29c8d270ec36aaec2 87294 F20101107_AABHQE wei_y_Page_175.jpg 909f875c77a2539276cb700886f42d6b eaee619c56def26798cd9a0b8993eb22a47a823c 83573 F20101107_AABHPQ wei_y_Page_161.jpg f36b5d113b967fa76c9375fbdd2cca3e 23f2ffbebcf1f7c0a553953d6758dc90b1ff662d 870 F20101107_AABITH wei_y_Page_160.txt 09c8ad7c9049770464840a2f8976653b 1fff1608f600afa79d28116dd25ca8f8a4a27ec7 1030 F20101107_AABIST wei_y_Page_146.txt 96f56a3ecd5039e0798c76a5c7df7e7c f9f59e822c09b9c0204b4743fe032000126ae10d 87097 F20101107_AABHQF wei_y_Page_176.jpg 055220816a970a771bca5c2d3a6509c9 30558635eec302f563ceb5dc47903fc39ec76f34 87682 F20101107_AABHPR wei_y_Page_162.jpg a7cb8874fa64b705019546406113c6b5 0d1af334cfb1c70b759b5caa0877f1221cb05314 1032 F20101107_AABITI wei_y_Page_161.txt 48009d9efa19ff6f284cbd4b0bf673ed 56be387fbfa0191ff759ef4dcd2db9fc1235c053 858 F20101107_AABISU wei_y_Page_147.txt 77d9eb5a35d0a000dcd58b709cb21efd 703fdd882cd2b53b49fdbf5220c5ee4eedb5c2ea 87106 F20101107_AABHQG wei_y_Page_177.jpg 8421312606cb0136035050dd308befc7 ed31758c4a528af824134ea8276aed3228bbf343 87888 F20101107_AABHPS wei_y_Page_163.jpg 8f61af8727322b8ddf6ac2e020c64494 bb47d509c059643169cda5d9586c62d2d0a1f468 827 F20101107_AABITJ wei_y_Page_162.txt d0275a12717d8ff0b965e22bea79613d eaa2e682a55ddc0be3d1787aa89f5621440b21c2 745 F20101107_AABISV wei_y_Page_148.txt 8530924ce26f94ecb71b732de9b5ce37 4e4773fdd8073b9fcd85f9b16caa2eca3b78ddbe 85794 F20101107_AABHQH wei_y_Page_178.jpg 959d2e0de963339bb81aa6030fd0843d 93ae281fbd19651edb118093dda2fc3f9ad6c475 87405 F20101107_AABHPT wei_y_Page_164.jpg 2c10c7ce241f66181b692fd5fab4cb39 9b8dae900f7990e90df3302a761006bd40f8905e F20101107_AABITK wei_y_Page_163.txt 652da84b6e16fc042e8ddf480112c837 50c75a2cbbe4197df72dcacf7818ff1ffd585bad 85954 F20101107_AABHQI wei_y_Page_179.jpg 3e7be00bb42c8d118b853ee40b9942be 94fd4f286a311d12799e4eef12e1e6293d5e6120 677 F20101107_AABITL wei_y_Page_164.txt 1df546e619ebaf7b3f26fd19b3c00e0a 079321db4c2a6099c1c34bb9d97c69840a5d05cd F20101107_AABISW wei_y_Page_149.txt 3044aeda59350aa4be8c6b274920bf43 a73603525a753a37ecf19be3f10de6d92c3351be 85327 F20101107_AABHQJ wei_y_Page_180.jpg 9d7f4c6cc4b11f05d806750c1d9bf4c2 ece16ca43e1c261afdf37681b45beff33933b801 87750 F20101107_AABHPU wei_y_Page_165.jpg 58d4a26e81b98e4cdbee9fc01c7cb94a b6769a68bad3a85c601fce26f23a4209e764de7c F20101107_AABIUA wei_y_Page_179.txt d0cb0a8fa2369c7e2228268846684a63 3ff46898c5f376dd3930c2b1126a26dcee2d76b2 F20101107_AABITM wei_y_Page_165.txt 3bef15807df7be39e958877bbcca6d27 14fb0df07b0c83af208b893fcc3844c68fa4b4f0 899 F20101107_AABISX wei_y_Page_150.txt 3b46900c31672dcb20c68af540d5012e aa941a6fa8b1f8dfa508ad8632229abf6c5af33a 85013 F20101107_AABHQK wei_y_Page_181.jpg a4c4b6bf51e993cf4fc984c98eb5f49c e12c5d05f23c14eaab27a7b3a97c8011abfa09d1 85851 F20101107_AABHPV wei_y_Page_166.jpg 5956cdf1f903a60f247425fa3769b1d3 88feb8846e2b6e993c2c1671bdb430970abcb4a2 686 F20101107_AABIUB wei_y_Page_180.txt 71d228701d43eac624d55e9f05814672 2b60e5e75c35e7c0b9a73a25d4addb690afb7ba8 1698 F20101107_AABITN wei_y_Page_166.txt ff3fc455f4d7ff3591506cc5c06b732c 449d32b254d3651a6971941818910d06e5ff320f 1041 F20101107_AABISY wei_y_Page_151.txt 7779fd768962d581047ce3442b05ec86 08f76005b6f2c62c9e2f2a5f3ef6e121cba046a3 84774 F20101107_AABHQL wei_y_Page_182.jpg b6157f5850a6be616835f9670761931b db595716e7d095108f34a37251d1ac4c56e8af4d 86407 F20101107_AABHPW wei_y_Page_167.jpg 8ec3f9eea8bc01a6e78cbeed89d49c46 c41bde0b7a749221290e16fc932c13f104c8ba55 807 F20101107_AABIUC wei_y_Page_181.txt 919e005d403eef16c8b9836e6fec4354 0eaf44d7b6265a2c0acec551e070ad19b5775429 826 F20101107_AABITO wei_y_Page_167.txt 6f55e909ab270386e5281391d19cd5a8 1d59b7bae8067cd09de5614a43abc6ec6b145dd5 889 F20101107_AABISZ wei_y_Page_152.txt 90c1a9bd4456d4810e063c06a59d6769 49f6dcce1de342a0a43c2de8ca81fd6f87f99948 84879 F20101107_AABHQM wei_y_Page_183.jpg 1d32bf1ece3c898a2e04834ec34abf25 b4d12c598baf55184947795a54708e203f3a9328 86517 F20101107_AABHPX wei_y_Page_168.jpg 5c255763735e4463ce00617ac64c7b07 8e1ea665bf1d5292b022fdb261c0b1d763bd7cff 1051983 F20101107_AABHRA wei_y_Page_004.jp2 f9225a3c7128bb0fe1630d3aed5fac60 30d28f3695202822bfc92f80438cb5cc5f82da57 703 F20101107_AABIUD wei_y_Page_182.txt 0e1d9c3457542e2776adb1f9d024e393 3fd4969a933343888379c07cc4c36d1a593c7ec9 832 F20101107_AABITP wei_y_Page_168.txt d1c8c1e6d1b97463c109d8a22d8491c1 16cb7020230cafd40fe6572b53101b6de24eba71 79085 F20101107_AABHQN wei_y_Page_184.jpg dac9e3219ceebcc6e91dbc314bde8309 1d8e89a3f492e212d1bd987e0a1b0d5029ffa550 86314 F20101107_AABHPY wei_y_Page_169.jpg 08bc3bb4c4b633ae21515670dda123f0 b66f8cd9ba9da4ef17ed6451215cf0215c8e6bac 1051978 F20101107_AABHRB wei_y_Page_005.jp2 1f00a5571ef920ed678203fe1b84343e 965e2a892fc02f8240245d7feb391fd1c2d1d787 741 F20101107_AABIUE wei_y_Page_183.txt b6b6669ceb055e69c6c1732808308e23 589aca27a9fd9c0af4b361fe75cd636be17981d8 673 F20101107_AABITQ wei_y_Page_169.txt 817c38dc2f40e07f4d84dc9f206e72c4 f9c6c76af19ac6316f68572bf74ef75218d87750 85367 F20101107_AABHQO wei_y_Page_185.jpg 42938378eb89f76908be885b536eab57 64d7265c0384237cebb37d9c7f08f15a94dc09bd 85333 F20101107_AABHPZ wei_y_Page_170.jpg d6e41747a59dd1c8e1ec1e7bbccca388 4439dcfc604dee16182e1f636382a47a11407b0f 519907 F20101107_AABHRC wei_y_Page_006.jp2 859c0aa5442e52109afa199fa57b33bc b7ea501df902ce3aded402b33c6e81e5500f7b9b 9119 F20101107_AABJAA wei_y_Page_045thm.jpg 9ced25251ba645e995490ee0696de47d e08a5fbac8bcdc1e126da830a8c679d278fa93d8 852 F20101107_AABIUF wei_y_Page_184.txt 414e573d2c43f9ac559c6f5d9b3530e0 8a3ae57c94c1fbb99f85fa0b628f1b2b11abe9bf F20101107_AABITR wei_y_Page_170.txt be09590c1a0763a979e3337f4d4cc0b1 6e064f0f9742d01dcc5e1b865472cfdac24dab5c 122570 F20101107_AABHQP wei_y_Page_186.jpg ba62dba549b40ba9c59f8d3b2c390603 6e16aeffdd75e74b2cca6029e2f5058233726923 1051980 F20101107_AABHRD wei_y_Page_007.jp2 2f6c9a32c9da38b6381cfb4e6eaf4951 f9c14bdcf9d27aa2a741d2d98afabf1b448c63db 36893 F20101107_AABJAB wei_y_Page_046.QC.jpg a13ca8d5262fa5a5bbe4321fd1732f87 4f99ca0344fbd17f4859e1d3afeb5ad1a8f378bc 713 F20101107_AABIUG wei_y_Page_185.txt 8347e432c21f440f08d4212ef9921025 e1be4f6253ae612d0d50015400bceb2c728d473e 682 F20101107_AABITS wei_y_Page_171.txt 84af2faa22e26a1c3902800df25f3dc7 ac2cad8ce2e6ef14c84984f33b5240b107638da6 131461 F20101107_AABHQQ wei_y_Page_187.jpg 6fea5f6794d51817afb5cb6e9eb70a70 0c296a8c6a87e96a046180fbd76aa31324448a46 137722 F20101107_AABHRE wei_y_Page_008.jp2 b13b26ec5436f08e4d52ca0de024784c 802bf129bd184f29d58198ff5cbb1b0fadd6078e 9124 F20101107_AABJAC wei_y_Page_046thm.jpg 75d8677c0704747ad972f2512aa8de31 cfb2f6953997ce154875c949f6cc1dbb7f22ca48 2453 F20101107_AABIUH wei_y_Page_186.txt 911e4157712f3effeb816a9359044bd9 3df420a8de7df1f8050297b1eb82afc372713c9d 646 F20101107_AABITT wei_y_Page_172.txt b96a10eb559e565670b0f419ec58bebd dade81ce43821276f7ef106236a0ddaf58d1f42f 124900 F20101107_AABHQR wei_y_Page_188.jpg 40a3cd49f06ac0cf627d0321700e325c b74c31c532f9224aeec4cccb0bacc56e77b99a70 983332 F20101107_AABHRF wei_y_Page_009.jp2 b4f02cea683bbcc1a204687db4aab07f 8aa301ed04c5fd378ad3bac80a75e434dfa4537c 2625 F20101107_AABIUI wei_y_Page_187.txt e25e4694143efb1d7778f2675ca51a3e 8625d417ff2f0077869d0093bea2d7ddd62c4986 760 F20101107_AABITU wei_y_Page_173.txt 79031d00b8657f9c73b8b773f7c449e5 d0f1da1025cf7df556f002126357c063025bc02f 123067 F20101107_AABHQS wei_y_Page_189.jpg 9c8740f27f741aa86da507cfb64868f7 845945d0d8656ddb7883229e0d151fc06c1125cd 785677 F20101107_AABHRG wei_y_Page_010.jp2 0203923baf2467a14a3b1df63ca9bf6d 5b4829f25fd6f64eda118b33dbd661228020a0df 34445 F20101107_AABJAD wei_y_Page_047.QC.jpg 0bff13689fcc884ef986984fc37bea3f 914ecf0f8a76021b74fc4172fa1e8963d874f4ae 2465 F20101107_AABIUJ wei_y_Page_188.txt 45d5b322c4bb8536898980a8f2da9156 f829f58a854d517dda91bfd9df83e903ba0437a0 926 F20101107_AABITV wei_y_Page_174.txt ff5ac9d71c4d472698d51729051b8a98 ab3a05917d934eb4e12f0a6ec7a468d25caab5fa 116770 F20101107_AABHQT wei_y_Page_190.jpg 2fc995a763170282447a508da8830e24 a4f5a468da1232d56485f2580d111f8bb3c6e0c1 F20101107_AABHRH wei_y_Page_011.jp2 c6c5155c22985fdc203f2d149cb55401 e57b4a319e76dd96955cdfa2e9dd9f782d0ac867 8622 F20101107_AABJAE wei_y_Page_047thm.jpg d0751c1c8a102d6e241e109a78f6132e 246ea06c5a952bdffb5e1d5010ffd4f0b6eda73f 2493 F20101107_AABIUK wei_y_Page_189.txt 74d08687d880819bf1bd3e07e319e78a 9211b1c928d37ccc8323cc1611be2ad36cda4bff 729 F20101107_AABITW wei_y_Page_175.txt 75e5f2844e97a83ee5f647110e79c39c cd3a5ff71eae51788f692fcd0cdde3118e5d5ca1 129725 F20101107_AABHQU wei_y_Page_191.jpg 37f2b5ef8f1f9ba50f09f797a705d183 2bc1b963fbf025fba81c26281b13bd132c846d85 1051939 F20101107_AABHRI wei_y_Page_012.jp2 3be923df10ca748bd863a9ec9232bdc3 5fcbf923b4b1fa6c3985531084e1aa0318e20233 33600 F20101107_AABJAF wei_y_Page_048.QC.jpg 756f26ccd5dc5820e044dafc28075dfc b18912549b54f5edf0b186a1fa3c0e4d9e7c8c64 2422 F20101107_AABIUL wei_y_Page_190.txt 163e82f29ac1b68b76f34cecc17999d1 a1baa4f03d51873fcc9a0171edc88be06861ef5d 694844 F20101107_AABHRJ wei_y_Page_013.jp2 8adc76d95d49730efae68f02e8f9f0c8 f3467ac491d871ec9d319f1046e36a38972df201 8075 F20101107_AABJAG wei_y_Page_048thm.jpg a7cd5840e2340bf2085d46b7da91edf9 41ff1695eba457470f32d4d49aa57ed6cb641aff 9739 F20101107_AABIVA wei_y_Page_142thm.jpg c30cc1a1bfd20058a5dcf9b471c09b44 68d8d2aeb0fc09d588b2f81abb83539eab48b42a 2490 F20101107_AABIUM wei_y_Page_191.txt 9b7161014abb101e192e95559610ee49 795f71408298d710b62bfd7d26ceeb54df828519 F20101107_AABITX wei_y_Page_176.txt d4c275de32cdd92c002ee5f3d86a1221 cb52168704434c591cccc963ef9b46b9b5a8e72d 75931 F20101107_AABHQV wei_y_Page_192.jpg e10cbf723d914f96cf05e0ce3f18edc6 a110314b2bef8669f1fc8d2443fc4488a22a1668 666516 F20101107_AABHRK wei_y_Page_014.jp2 e855eecb3ae9da7612b9f048cf4d82e3 416a9686fffe914b0520c1adcbc7932d2f1e8f55 15322 F20101107_AABJAH wei_y_Page_049.QC.jpg 2db2b231b6e41085ed56511c33998655 ce607cf3517bd3e14d262bfb751c56df15fdfa47 1795 F20101107_AABIVB wei_y_Page_137thm.jpg c30809b6e7b8e785aeaf4382ccf012f4 3fc3cf4fb0056def364389f4b4d796c5ded871f6 1402 F20101107_AABIUN wei_y_Page_192.txt 1f39d0e6eb120d247492fe6659b5fd93 27710ed5c74549710c73eaff6d2faab74985671d F20101107_AABITY wei_y_Page_177.txt f2741a252fa17fd13dac698c27a27a75 7b3ded23751f30e64f6d7cbcd99325626d50ef77 43989 F20101107_AABHQW wei_y_Page_193.jpg 769855eca5c4d279e23b52d07bd0d4e6 8233ae3dd070e9ad1d2d18a366f054d72aaebd61 560383 F20101107_AABHRL wei_y_Page_015.jp2 cb5ecb77a962a1b1b6d8e6c3dce058d9 22a1bba229e25952c510911215c8f5b3eb579f4f 4681 F20101107_AABJAI wei_y_Page_049thm.jpg eff070b699151a7771af436909c66dcc 4f99fcf09f3afe7f674a0fdb4165f6b02f7002cb 9715 F20101107_AABIVC wei_y_Page_185thm.jpg 09fbcb3a6ec0fbcff3ec2e61a95d30ac 491c757fe013f1a2b6a31b3ee82193074d9c470e 788 F20101107_AABIUO wei_y_Page_193.txt aff475d95e2ec5333f46b9712588ab8d 584cdf88b40e70b7176cade0191e214a73e689c4 F20101107_AABITZ wei_y_Page_178.txt dab0455a9c819a5c55461ba434e64bc7 e613a82c8c6b40fc806fdb595c8c07c5a03c55e1 248761 F20101107_AABHQX wei_y_Page_001.jp2 ca957400f32c430dc5bc96870cb3274f 396a9a2a1ddee7d3c9b0c7c870b087d5909f9407 1051967 F20101107_AABHSA wei_y_Page_030.jp2 7c262fe6568b4c10afe6f1db09d1189f c621c9d6f6a737b613e7b30f625e0ffda931f7ee 909873 F20101107_AABHRM wei_y_Page_016.jp2 1047b403a456e3682fd89995f40d2c50 037f48d137f812524c9a764556fc4db7a4d8c84f 31623 F20101107_AABJAJ wei_y_Page_050.QC.jpg 09ae25fb16db299d50d66c7720148454 160fb4e5731cc0a891b866fb167ae55988a5bdb3 9688 F20101107_AABIVD wei_y_Page_157thm.jpg 84c40706798d6aa9bc57170965e861de a7d2c96cca1d70d93c9f06c9c6500345370cde63 1985 F20101107_AABIUP wei_y_Page_001thm.jpg e7eafa3c42fe2ea3eee27d7071eb1040 221b336534d7890bbd4d2469b38b33e240586d03 22843 F20101107_AABHQY wei_y_Page_002.jp2 8794ca15e60c6d573997439686653c3f eea1a3c35edac5a55b1ce02c9f8b240d453f50e1 652833 F20101107_AABHSB wei_y_Page_031.jp2 bb29b7e5e84c628027eb25ac82e83051 73bc9309ac577c0f1319be0ed0bd0f49650343c2 1018880 F20101107_AABHRN wei_y_Page_017.jp2 3f2287080d8cb7d4d41759870af789b7 c99138b655421a181fab55632adeed9c3d990c55 7835 F20101107_AABJAK wei_y_Page_050thm.jpg 098735bed40e771d9986934aab830cd3 72e6152481c69ea80c93686eb8933a5ecf514308 9627 F20101107_AABIVE wei_y_Page_108thm.jpg baf1b9aeb8e1a0ad572651f2f1d47653 9005e385c975b2a67c384b9442748035ab03f742 4411890 F20101107_AABIUQ wei_y.pdf bd68cac58c4726daee6288977bcfc556 4c9fb4c464e3e47d379daf78d2c4869e0abc2068 308460 F20101107_AABHQZ wei_y_Page_003.jp2 7d2ece8f88efe50eac3bc7a018e6e50c bb4cef96a982e88beb39c1f685cc4bb6171f25cb 933001 F20101107_AABHSC wei_y_Page_032.jp2 46a019aa8cc2f4ebb4f84d512b890c36 42913e3a29dd141f86b1f318357ed0b68586b475 910056 F20101107_AABHRO wei_y_Page_018.jp2 bb9e15e307e7219f6b667c90445f215b 3ea712b95ac9045fa6bacdc76f79cf559f893bf1 31266 F20101107_AABJBA wei_y_Page_059.QC.jpg ba55a78c5c06c848b67272ce70451ee6 78ff555541746f26c7b024af4de26575eebd77a3 35994 F20101107_AABJAL wei_y_Page_051.QC.jpg 9b5760dd75f3505baea8326e8107398b 56cf71adc45216da04259c4e3dee91193939cedc 5856 F20101107_AABIVF wei_y_Page_028thm.jpg 3f73497adb28cf7fcff56b81e70b8bce ae934b4d36d8a6e6d2e30e920cb98c5f7d04e525 9935 F20101107_AABIUR wei_y_Page_151thm.jpg 57c98066ebc8e3cf37e1ba4c463b4a01 2b5130c6b7cb9ac62548f655a45200b2e993daf3 914365 F20101107_AABHSD wei_y_Page_033.jp2 821c3488bd160531e3a03212ee0b4093 fff43d25e714ca0cb0df2b352afd257d4628ab16 1024672 F20101107_AABHRP wei_y_Page_019.jp2 499e0c71f111d5c3cd339f6d4172ff8a 6205fffd82e47adf40df43090668a73b2af59de6 36228 F20101107_AABJBB wei_y_Page_060.QC.jpg 900c552edb15868c82a21076e2ab2c5e 974a7ba525fcf60f09dacbb996e266cd2640dc36 8450 F20101107_AABJAM wei_y_Page_051thm.jpg e43998e7927099b54bc61518c8f2b63e a95fa9c602d8a3895eb225055b8598b52e1d2fec 20390 F20101107_AABIVG wei_y_Page_076.QC.jpg 1ed2f0593316940a459bc21d6f90e3ff 7503ca439d2597a1c55958bb1337e506041f67e3 21054 F20101107_AABIUS wei_y_Page_088.QC.jpg f0bc7908365737711fcc43c65ed3658f 00b87bd975daa39a904ead483570c9dcba063341 627839 F20101107_AABHSE wei_y_Page_034.jp2 04fdac0da0db8f1686b9d03cf0e8a2bf bcb30ca8091d2cda03e72f381b9a2ca458424be3 1019919 F20101107_AABHRQ wei_y_Page_020.jp2 c1d893ac54660dcb4fdd696f68ef6695 c4591ed2d50d9659198eebc8d0d1a1ed3a098dcd 8992 F20101107_AABJBC wei_y_Page_060thm.jpg d45ff0ef839dfccb8ff411c1b2a7cc1f c7285faf65004d02b462740c83fef6c37b759d38 35948 F20101107_AABJAN wei_y_Page_052.QC.jpg 2c27aa5afdc51b320ed1dc6b025d80b3 c0711e85cf0b569560ffe9ecdc6056b89393d2f5 9247 F20101107_AABIVH wei_y_Page_184thm.jpg 40f9d9bfed1314895c9eca1e0e3358f5 cce905b5922ea3bd1034ca3a01a4ad91ff972503 36070 F20101107_AABIUT wei_y_Page_191.QC.jpg 34f33cb8a332f4ff3eaa1bc37f98e814 1ac3be242aac708d5431fd123c5604cf868ec625 764629 F20101107_AABHSF wei_y_Page_035.jp2 a818dd149641d46cb3ce9b14a9819e4b c5da63bda49469851cd6ac74ba4b0656394f9433 528099 F20101107_AABHRR wei_y_Page_021.jp2 67f29c6b6dd78bd5a73afb5630d886ca 781aa6fed174000ed5c3885424621f1240ba1b4a 36208 F20101107_AABJBD wei_y_Page_061.QC.jpg c59919938085bb1fee5c6671f7000c57 efcff4046149bd5152b5496dcbabc7ea9dbd5287 8270 F20101107_AABJAO wei_y_Page_052thm.jpg ab18d79cabea8adad2f23b163c3d5806 cf5a227e1dee57a02ab4527ed6dc4eeff5a12daf 9116 F20101107_AABIVI wei_y_Page_110thm.jpg 561742a417899d8238dbde76aa5b8bf5 8eaa960bd3bf85221c3326b95637de7d0f076400 9886 F20101107_AABIUU wei_y_Page_180thm.jpg 1c05844e7069b0410b82cf53cbfafa68 0cd0677e41b97038659f3cceb7d1d1b064203979 862290 F20101107_AABHSG wei_y_Page_036.jp2 fb3cbbacd7140e729bc819b53c15a345 ed7c37ade38973061efdbdc76aaf6916edc4ed09 832474 F20101107_AABHRS wei_y_Page_022.jp2 1d84144529c05ac2726e92688aba188a 209e46b4193a5e24650f54476aefb21e79c2f3fc 4804 F20101107_AABJAP wei_y_Page_053.QC.jpg 9464f1a4911dc073c8028fcf5d39f820 b068a8143955102b849d96ab7b912a20068a60b1 1115 F20101107_AABIVJ wei_y_Page_002.QC.jpg 198a0d6f6fa9e83ee1b3d58cfc8b538f ee30a664da43d0815475f494b806724e0410b0b0 10093 F20101107_AABIUV wei_y_Page_139thm.jpg dabad021365fbfff1bcf1727518f614f a346da00fdae35d83c5de79d835516396ce458cf 927416 F20101107_AABHSH wei_y_Page_037.jp2 37b367b462600161598de730e56e6f04 1a1d69a118d5e75a8cb26d112ab013a62e905527 737868 F20101107_AABHRT wei_y_Page_023.jp2 0586accec8789c2b2c7662dd6a42f35f 4ef7a773937e509fc1a2682a8996fcedd877a81c 9117 F20101107_AABJBE wei_y_Page_063thm.jpg 0163693da8210473091216a44a7d1777 54f2fc59543c8c178f5068890d599c56fa396be4 1210 F20101107_AABJAQ wei_y_Page_053thm.jpg eaac16a9ae0ef0f216ed0f0e9d946d45 2cedc6e533666387a34432eaa65d07a22839a09b 6666 F20101107_AABIVK wei_y_Page_080thm.jpg 7e56be260765f7d5c97eda910945b621 7e0a57b87205304003d40d41297753262dc9071b 35604 F20101107_AABIUW wei_y_Page_189.QC.jpg 3b6fe5a3d058ca0e311ba1b3271e93cd 48f3b60b919df6db99ef33d918e041adbae28892 792004 F20101107_AABHSI wei_y_Page_038.jp2 6d842e37f3a44453a24cae57d4e4d866 e8a7a7340b8d4f2815e7b4b1248ce3185bf29c59 310390 F20101107_AABHRU wei_y_Page_024.jp2 c019e8f874a2fd2e23c271aa4eb7eec3 487282e8a239860deed171d363f8c24aca10eb5e 8757 F20101107_AABJBF wei_y_Page_064thm.jpg 675c213a1d9dd94851ce0aa3f0fa388b 3eae4152506ba348b4fec28794ea55f315578d89 7696 F20101107_AABJAR wei_y_Page_054.QC.jpg 59fcde4b09e575ae20d33f03075b278b 8830a78055fd6db5be4df0af17bff8c04101b771 8543 F20101107_AABIVL wei_y_Page_059thm.jpg f59e1e0dcaa8659ac1de67cd9c110285 db94bd68d440d6f0986633ff770d4604ff650024 9658 F20101107_AABIUX wei_y_Page_156thm.jpg 71cf333df9e75e31f55f5275a2c2569b 5d8d0ea9b08c8d0bfa44a5e054e2c499bbd0146c 1051976 F20101107_AABHSJ wei_y_Page_039.jp2 972e80722b4f6d05853a5fdc76afe872 47c1ea11b87c4998e7f90ac00be8d0cad6ee7b24 956047 F20101107_AABHRV wei_y_Page_025.jp2 684a4392a2aeccadc41958cdbe53f0ec 174fe9068ac94284f7270ebfddd3bb3785b6a6b0 36584 F20101107_AABJBG wei_y_Page_065.QC.jpg 4b5026ef9882aaceaf0004ac28320244 61ee5f7a209c4adc570b06ba21845d74ba812537 2195 F20101107_AABJAS wei_y_Page_054thm.jpg e45a8f6da8f72224edd3d9d042fb422c f98cd0b2f64c0b84743647a3d93f79dbd8aca47f 6994 F20101107_AABIWA wei_y_Page_088thm.jpg 69f428c80f976cf616bf526ff78ca314 5458cc453550113c755cf03797dc2ad9ef95f235 6193 F20101107_AABIVM wei_y_Page_058thm.jpg a7453fc166f7924196973aec7f0a89c5 2e131094aacb283a2ab687c2bd9cd0db39d02e42 1051942 F20101107_AABHSK wei_y_Page_040.jp2 fa2a6cc9618384aa8d23a8c2c0f6518f d072180b04c04deb42c0aed824a3dcf86e45b650 9007 F20101107_AABJBH wei_y_Page_065thm.jpg b89a4a5e1b8f5e5fcaff153c50c8e2e3 c4243e4450f2ba1568e1383315c91047def340e0 13147 F20101107_AABJAT wei_y_Page_055.QC.jpg 53c594a18b1dabab2f7f765637ce7b9b 840c6a8b1a9f83af2df72bd90a902c1bfa2e4791 35034 F20101107_AABIWB wei_y_Page_030.QC.jpg e0473863c4be1078ad871396ce313fd9 aba873342ae83aae8b683287d2a8c0b746f23a90 9925 F20101107_AABIVN wei_y_Page_178thm.jpg 1ae5b88ec4e3c284ec7050f59c335f9b b9e4e1264c172f63bdcd709411d01a6f45c358f2 9059 F20101107_AABIUY wei_y_Page_061thm.jpg def9c9853ff1802a3db34e70fa08e162 4934fe90a826f46deb8d7d3636d4c2fa2a2e54fa 303847 F20101107_AABHSL wei_y_Page_041.jp2 980def7b65d5e8f813abb32b6e0e1ff8 818369a810222a8e158e42bd92880d46eb5241fd 779926 F20101107_AABHRW wei_y_Page_026.jp2 2b08368f2014e20b831dcbc2f1a77b22 82e41f72282156888a5ba09ed95262c2eb41ea1d 34781 F20101107_AABJBI wei_y_Page_066.QC.jpg c7bdd6c6c780a41ca4636650d7376f73 70321309bcfd4ba093d909fc970c8e368e5f0367 3685 F20101107_AABJAU wei_y_Page_055thm.jpg cb839f287e21b6d2c191dc8ab4e1d355 701f1669d3ae2f9f42578af5ead6eebe768a11da 9907 F20101107_AABIWC wei_y_Page_177thm.jpg ada8f6ef80f2d16ee299131e792b7723 3db35d5dab78af90bcf05fc7125eebf887565993 26615 F20101107_AABIVO wei_y_Page_096.QC.jpg 23422151e02ab30abda95c526f4699bd 0695e7494bb61346c16a719a0c45031cb6574cca 10082 F20101107_AABIUZ wei_y_Page_174thm.jpg 1dd6f02ac7913d99eb13adf4d5cf8d55 bda52d3f7a16d35803d2b88792199637bc048739 701282 F20101107_AABHTA wei_y_Page_056.jp2 de68ed79898396cd40fa328d0ab7aa05 bb22a197759f95c797ecee1543eb1548187134fb 1051986 F20101107_AABHSM wei_y_Page_042.jp2 8b18a5762d4edbdb2c2a764a641d5ba5 d22d6cc2f9b8efba4d1895ab9b1fdfef5dbf2590 1051985 F20101107_AABHRX wei_y_Page_027.jp2 23f49346e860599553320417014ecbf1 dcfd93781c455f6e122a5d0ec0414f3e8ec18203 8561 F20101107_AABJBJ wei_y_Page_066thm.jpg ab9788bdeab7c70e88e611a625b25216 ed0b71a74a8c0e9336d0b50e33749233124e7fb4 22344 F20101107_AABJAV wei_y_Page_056.QC.jpg 2ecab62ccbc4083adc22dcb2d0d5b3a3 eb01772778ac3b337ded4b5ee8c228c596aae184 F20101107_AABIWD wei_y_Page_062thm.jpg 6bb62c186780c1243ed61e23b036c95e 70f80057cbba8f15386f40f01099b8cd57700cd2 9608 F20101107_AABIVP wei_y_Page_146thm.jpg 788b0066cf818559ee088a839464b90b 8361c77d6fe9a82274ab618901ab96f1cf6cc33f 390940 F20101107_AABHTB wei_y_Page_057.jp2 786974fe7be59324b7523d36ee8220fe 6f3f0b7001ac37a7508229b7bbce71e470e990b5 1051879 F20101107_AABHSN wei_y_Page_043.jp2 cb229b3db106c521a707484db5d3edbb 47a0d7315bb9580aaa5e59a06f7c7f9751bf5f93 688306 F20101107_AABHRY wei_y_Page_028.jp2 8b64af526a2c832169212f96a2b097e9 92467eec60d3e9bc6bfcafd985c05f221cb6206c 37035 F20101107_AABJBK wei_y_Page_067.QC.jpg f1969624a36477f78971473440f13210 78a938a9a1fd32d7706ac3729fb48ae9d822892d 6206 F20101107_AABJAW wei_y_Page_056thm.jpg 3d7d8f39b120b785fc1d7d2a0dadd04a ad67854e92a67e78bc7629dafe0f1c06693bf2ec 7333 F20101107_AABIWE wei_y_Page_072thm.jpg 3fedb42d9893527be64d903e92bf695f 01e59f0e2c21329b9c978f389e50904d3fdf5463 9894 F20101107_AABIVQ wei_y_Page_173thm.jpg 58bb094f033e3207237ab15b8118bc4e 30bc25795169df2fa841c2a255f489b9efe25aed 703617 F20101107_AABHTC wei_y_Page_058.jp2 3d682e406a7cacfa8b9d70d3461a7694 13a8fce6b3dd0f993c3cb216ebea66744b326b8e 1051945 F20101107_AABHSO wei_y_Page_044.jp2 c2f1b04e9e81b446562a76eea7f0c53f 72809cc0ee143f87000ed164d0071e2d7784312e 475592 F20101107_AABHRZ wei_y_Page_029.jp2 8a71afd1ec4303e648986713a02a5eaa 3d6d91339aed37bb38c2f5517094a46e9fd4ffb5 6873 F20101107_AABJCA wei_y_Page_075thm.jpg ec10c89ae8517679827018b9617baaf9 6ac0361b7a3a7ce6436e57add3ad3244a677a25c 9271 F20101107_AABJBL wei_y_Page_067thm.jpg c0a7501c285f3611d4bc453c824a3844 61a1a3b7255461576916bedc2972a4bc13288e2a 13098 F20101107_AABJAX wei_y_Page_057.QC.jpg 7e031d61cf5450fbc658a12c8b0b962c 97ab6bd452fb5de033701ce9a821f67db519b5dd 6720 F20101107_AABIWF wei_y_Page_092thm.jpg 4138c465610e2d437258d772ab3167eb fbda4f7c7735105c57c0b820bd68720915b67be9 16860 F20101107_AABIVR wei_y_Page_119.QC.jpg cc47233f5b5b038863df007b4f41886e 88bd7044e6f223e10b54d49fa8ffb47dac8afa7e 1051958 F20101107_AABHTD wei_y_Page_059.jp2 913e659f95aaaede8812231bed1426cc 49803f8d73181e0fe9a7e47f498be8df52e8a226 1051964 F20101107_AABHSP wei_y_Page_045.jp2 6d9f2aae8990bd05d588e558b89ca106 70990bf06ad371835d47373e87a7781c238cacf5 6560 F20101107_AABJCB wei_y_Page_076thm.jpg 292a8c299ce1a9efea8ea842a5b5d91a 7b22290ae23ae0d69779abedfca31aa15bef9c47 18759 F20101107_AABJBM wei_y_Page_068.QC.jpg 5aa8ff8752e9b58ade1f6f963fcbca1d c0d82d6201de6b2cbab9ac93c47aa14d93a4a683 3698 F20101107_AABJAY wei_y_Page_057thm.jpg e332ea2f8ca49e43ba1dba0f86fc2107 102446b4814ab95bf678dae62c4e10a761d2c823 8678 F20101107_AABIWG wei_y_Page_191thm.jpg 250a6b5d511ac168c5218ce0545d6450 dc76a42105277e4edf68b02e8ef0033fe7af950a 9980 F20101107_AABIVS wei_y_Page_175thm.jpg b3543226e145373ae1ac277ff13b64db ea84b193aefa845839bb98ec61be4ad1c988b6e0 1051981 F20101107_AABHTE wei_y_Page_060.jp2 41a7394c4f21dd5d2107c0e28509865a f2cadfb23ba562e085a2340c28ddbfa3c40603e0 1051936 F20101107_AABHSQ wei_y_Page_046.jp2 14ed4bc8371a3fbb8e0c2673df8462c2 d467602e9c3aa7c5be8469b495093a0146e4c9bc 19997 F20101107_AABJCC wei_y_Page_077.QC.jpg 25cb4b4ded66861ba43a723b65e34a5d 48d0acae26a969a4eb17e4da69aab72a62e52222 4989 F20101107_AABJBN wei_y_Page_068thm.jpg 3e19d1a4468875ae2fbdde54b0f00693 bcd39a6dcac0620d591c94db562fffefa50fdbf5 22274 F20101107_AABJAZ wei_y_Page_058.QC.jpg 126cc5092429ae17811d2f9d6ceda61d 87d516a5981bbb813ee8a2fc9e9f0c402627ab55 36758 F20101107_AABIWH wei_y_Page_062.QC.jpg 37850796695a0b394f82fc3efe49c9f1 5234898e4991207848913362d227644ee7d92992 10097 F20101107_AABIVT wei_y_Page_141thm.jpg e407c3a7012e5777220da02f37d92b68 e269c0bbfb3d2061360e42dd98e0db5e7a697d6a 1051970 F20101107_AABHSR wei_y_Page_047.jp2 5027711c6ca07779a41f9e9b5192b393 2d497a65a4fbc8e612631c3aaa288e0f62210a39 1051965 F20101107_AABHTF wei_y_Page_061.jp2 394c1f69dfe0ffb4a34e32cbb9778232 d784cc652f35d6caf0d9695c22444f8cae466968 6375 F20101107_AABJCD wei_y_Page_077thm.jpg 95692c62569dd3c3b007183d12140390 9744ac1825db4789a9a02fc45a0d1ff26a490192 19742 F20101107_AABJBO wei_y_Page_069.QC.jpg c0fb2bc26580cbe6867574f78eca8872 035c6aa77a55afea346d18d58498ef81d723c151 35600 F20101107_AABIWI wei_y_Page_064.QC.jpg 9aaa598c05a78b596f2025975885fe10 777d627f14d9a14cccef41c27040f59e08838067 34462 F20101107_AABIVU wei_y_Page_063.QC.jpg 0094f6fc9fae9a9fadb53926feafac0d ddbed0db3cb9f2d915163d3c9b7493ea35c97162 1051957 F20101107_AABHSS wei_y_Page_048.jp2 dd091e03ceb42644cca2014040aaa087 f24df94ef7fb10d8744c9bd131647affacf28c4e 1051975 F20101107_AABHTG wei_y_Page_062.jp2 3515b82f261224b5395684f00ed5ebee 15101749f7275691dd4af6fa31474f8f951011f6 22314 F20101107_AABJCE wei_y_Page_078.QC.jpg aa9a6904cd9291c2446d0340c18e2c58 1ed3192e84d23042d050bb2415f8198ec48a5411 7042 F20101107_AABJBP wei_y_Page_069thm.jpg a8f91839c588673d11f582ea41fea22d 6676c800db5882953a5c4bc542d89e3274e2536f 9990 F20101107_AABIWJ wei_y_Page_167thm.jpg 8bf8ba83b3f29e0e6cd863806269b74c df892c11cbd2839fb591929b40f57915946a75f9 21140 F20101107_AABIVV wei_y_Page_086.QC.jpg 3325a84d252ab56d6c18d1528239e099 630496925b72718261de14e25d3ba8464efb31b2 488841 F20101107_AABHST wei_y_Page_049.jp2 ab403f921863f19a574314ca5b38feec 8695874309dd3036f506ada62217b5e758b4e001 1051950 F20101107_AABHTH wei_y_Page_063.jp2 f51057c342d2e5f27531648ab4357b61 410ea97153d7f4f48cc3bada69130931772c5ec1 18747 F20101107_AABJBQ wei_y_Page_070.QC.jpg a17c400ec275afaa8059e6606878c37a 6489c423433c409c7278410b97be65931fc274d3 30635 F20101107_AABIWK wei_y_Page_157.QC.jpg bb13559dd26e9912f8320b4bbab94cc9 d2be141181592cb008e17952b299a2bc2b368292 9175 F20101107_AABIVW wei_y_Page_102thm.jpg fc853c306fe4005a87af4522145d02a7 1697c7e99ee99b2500f1e70fd5808e265047be3a 1024689 F20101107_AABHSU wei_y_Page_050.jp2 c59f111db850a0734f8bd109796a0040 579ad45a4ed818f601494252cdfd4cb927c12619 1051972 F20101107_AABHTI wei_y_Page_064.jp2 0626e21b0790d76f7485b7af69ad9b81 db3504978237ce104f5afdd16f17e3156cee0c95 7190 F20101107_AABJCF wei_y_Page_078thm.jpg 69dd529961ed589d6d94477617bb3675 205bf59f6fb08e3d58d02ba802842c3a7aa0f7ee 6885 F20101107_AABJBR wei_y_Page_070thm.jpg ce5edc51cdcf307fb5ee08d836ac3345 4d60c94a2ded1369f052f5f18ffabb608f09784e 20438 F20101107_AABIWL wei_y_Page_089.QC.jpg 5d5d99e90dc02ccf252179f584acd295 4de96b225375c017c8ddab5e81ee6b5e17de50c2 9711 F20101107_AABIVX wei_y_Page_155thm.jpg a35b6143aec469bb3ed2367e728324e2 6f017bb4122c48b6f1f91d4745e717759704db76 1051961 F20101107_AABHSV wei_y_Page_051.jp2 3a3bdcaebc161cd95a6e22fd96e9c6f7 e012672851ed2de46dfaa2c377fd556396574ecc 1051947 F20101107_AABHTJ wei_y_Page_065.jp2 7ab4d7739b9d757b3fe34f018974d6b8 23ee42a1c72f0c38e69ab626a8a0d76b19633436 21019 F20101107_AABJCG wei_y_Page_079.QC.jpg a9c888992ae9155c42f4b2b25c8e51f3 b192598fec287da13c7ecf22decb4f74f2266d1e 19050 F20101107_AABJBS wei_y_Page_071.QC.jpg e5340512c78ece8615f3299144bab0ab 40e2245646783faa570b40bfdab9f8b55cf2dec8 25557 F20101107_AABIXA wei_y_Page_004.QC.jpg 9221461b71adfe5ce75c90b724b94815 8d8f06c4384224f29f196c0ddae4092aa5800ebb 24934 F20101107_AABIWM wei_y_Page_098.QC.jpg 49b261068349af7aa38cb46723b9f720 39ce45c94f2279dd01699cbef6e533c47632ebf1 7472 F20101107_AABIVY wei_y_Page_025thm.jpg 273b811423cb258a04d00854d441daf7 97041ae6310e65e0612ca07376b681bc21f7e685 F20101107_AABHSW wei_y_Page_052.jp2 2de9595aab3dc8d2c5f3ce4a79ebc852 77357f286c4f5e26f58fae2b0979905a58a3d176 1051930 F20101107_AABHTK wei_y_Page_066.jp2 09179db9c4804329efa4b8e277e4b636 e9fc3058abfaef01dedc374258ce610bbf3f3dbc 6654 F20101107_AABJCH wei_y_Page_079thm.jpg 7f724cc6cb5504a1c89d1752ed5a6c29 3e0df543055e45e52642675e78018a5a7d362ade 6863 F20101107_AABJBT wei_y_Page_071thm.jpg 03662d61193db284b9a664f6f26e403b 1ed8389ba520333faac510c908d725cee79b3254 6279 F20101107_AABIXB wei_y_Page_004thm.jpg 1525f6aaf4a6118453ad6d996b9b9341 288d48b86d35b887334ba6d6be2dd8c2954c1ca7 7326 F20101107_AABIWN wei_y_Page_019thm.jpg 7c92a41540b928e9a0e20fb7b2007c31 e7fc7a546b1fdf8646bac9330a7403ebf0b7695f F20101107_AABHTL wei_y_Page_067.jp2 a01e2e5667b158dd01dd2b1c4a8509da 13834c8aee9213076558e054f576b2aa99a04642 20544 F20101107_AABJCI wei_y_Page_080.QC.jpg cf704f7034564528fa700b1cd907559a 070567734f7b15361a43c22a68df836211226946 22109 F20101107_AABJBU wei_y_Page_072.QC.jpg 8f94eb7640acc9e4dc9289a5f9f95809 6541fea5d55e16f91e1d8ee48981bff88a9981ac 16743 F20101107_AABIXC wei_y_Page_005.QC.jpg 956608ab6e1803e3ba52d7c70d95fc8d 2f2e5aede20dcad5aae570ba1f4696dc63cece76 10104 F20101107_AABIWO wei_y_Page_163thm.jpg 0f1e33186fa34fce5efc6012167742aa 1f7b67f6708344eb6d61aefe10c10304ff639345 5369 F20101107_AABIVZ wei_y_Page_021thm.jpg aec009436bdb15f53f8056ee45053e73 0eddc83640f144186af18e2a0ff1ae82c6e8bb12 110305 F20101107_AABHSX wei_y_Page_053.jp2 3f5e93e2e4a7315f77a88959ec7e42c5 cb5a1ba94505e1fbf7f95089fb8766080b9c2804 431096 F20101107_AABHUA wei_y_Page_082.jp2 1e892c61098e0c9005a621c670ab0efc 4349a798b1d417e04368da5042cbae599eff5542 587310 F20101107_AABHTM wei_y_Page_068.jp2 78125e2120a2075004533bdc38c6abf5 3ed656fa5866ca30fe61786065f83f020c418471 23488 F20101107_AABJCJ wei_y_Page_081.QC.jpg b0a1aed359fd5d46d6277ebf7be1df3f 1f32a943678f09797dabc4f09c260a0867aefa37 21170 F20101107_AABJBV wei_y_Page_073.QC.jpg 48363664f953d6db42ac5ae1b58609f7 b7daefb810486405bb5a12483aa8c6e73c321a25 4143 F20101107_AABIXD wei_y_Page_005thm.jpg fdc8aaf149ea82e064b616dd58bd065a f20fd3488e845281fd7746d479497a1c63952941 8518 F20101107_AABIWP wei_y_Page_096thm.jpg 5c507fa25bb7196c623d9cb79a75dbc8 0e20655090790e36112fff015949524f70b812eb 178992 F20101107_AABHSY wei_y_Page_054.jp2 805a0dd314e5cbe7e28c2cb24ad85d90 47a68be0e1d29f59b456aaabcfd96c4a633bc7bf 427376 F20101107_AABHUB wei_y_Page_083.jp2 07851a31d0a4e0e13b1b1b5114cf3f87 1e5db8cfea1da5e7b91fec3c9fc5a10a6506a5c8 383866 F20101107_AABHTN wei_y_Page_069.jp2 21f7211edbc466b526982bc21ea52b2b 506081e8e0dafaec8d2c91fcbd8ea2fd03ae8665 7521 F20101107_AABJCK wei_y_Page_081thm.jpg c48a4ed7d74d91b158207d6402df7654 90115db7eb41af0e9d257678789cc6a4b412c7e6 F20101107_AABJBW wei_y_Page_073thm.jpg 10ed55dab377de2a5a7e10520d7f1499 e260b430e3283d32cf484442a96b4b74d4891ad8 9236 F20101107_AABIXE wei_y_Page_006.QC.jpg 69343bc0f49960e43a7097d44bd24bbc 26ec9ef765eb947c9789be7355030e46e0c08cee 9055 F20101107_AABIWQ wei_y_Page_099thm.jpg 4eebdccedf46207829d89f57a0e9d359 4d9e1b4810cacb50e5660d854de2cbcbbdf36cc9 390163 F20101107_AABHSZ wei_y_Page_055.jp2 9851c45c5d19081aceafe08418e2d3ba 849d0bc25d3dd2d1b6c40934a6d669b29b6a2c6a 470743 F20101107_AABHUC wei_y_Page_084.jp2 fd24f36e9121f2142ec33dd851ffe6f6 51dd7af15c9fe43a3698055258a7b1d8a639566e 358070 F20101107_AABHTO wei_y_Page_070.jp2 4f78f87ce752d14b66e41602ee3e8c1e 321dde431c22ff053ff50b05325a2d2d6410521d 27252 F20101107_AABJDA wei_y_Page_093.QC.jpg e454d3256a51de1a8b4563b08f703106 6ea9242efbb74ff70d3ad4f042dd51b9b6a14142 7053 F20101107_AABJCL wei_y_Page_082thm.jpg f86fee667a16d154acdd04ba82a0213e 4daa20b547313a6e611dc51185eefe001162d0c2 21102 F20101107_AABJBX wei_y_Page_074.QC.jpg d05455163dc081dd18a81007cf506486 bdcacf1ace73518a5c484576ab77b4201bda9b67 2593 F20101107_AABIXF wei_y_Page_006thm.jpg c060b9a57fd5850db2b4e6192297bcce 53015b90404c85877be6b1ab871633b230c2e3ed 7110 F20101107_AABIWR wei_y_Page_086thm.jpg 7712dfcda6f621981e3824840e8a13bd 0f3c367072a9edcfe84dc915066e0da63e8dc36c 421548 F20101107_AABHUD wei_y_Page_085.jp2 724f968bd2a58fea55cd0a8589d3e60f bb9047fb8157466373a3f2a67ae2837fdb146462 357736 F20101107_AABHTP wei_y_Page_071.jp2 c136e76cf569ee1daded84512215c8e6 c0d2b0bfe92d1cb6f6be0d7e6fe89cd64bdbc304 8834 F20101107_AABJDB wei_y_Page_093thm.jpg 1fe82bfb01008a9172227ec1b58a71de bd6dad3a07585d9dd099b4b3eeae18e4979a31ad 21144 F20101107_AABJCM wei_y_Page_083.QC.jpg 42d3715bd3c3cef0a4c9647a6f2c9ccd c576e65347793caccf137fc9824ef81b68fd188d 6963 F20101107_AABJBY wei_y_Page_074thm.jpg 80bc4fb800651700f66ba24faa43217c b09fb04ce47c6c8186be4d2a273ad321812555e6 37278 F20101107_AABIXG wei_y_Page_007.QC.jpg 85e05dd7ee3f46ee5c6c959be2ecf9d3 1b9ff3b9a393fd76de20428ad44c3672d3e1ccab 27472 F20101107_AABIWS wei_y_Page_099.QC.jpg eecdc668066cde14fb74d6d691fb3cfe 2a995734f96526ada28f0f134ad0a9b16bf07930 420971 F20101107_AABHUE wei_y_Page_086.jp2 7130318a88828474c4f6de8ad3588b9d ce969775e0b4a4d5d718238e17e83632dd4922a6 454884 F20101107_AABHTQ wei_y_Page_072.jp2 4e7d70e5813cdbf08806ee0c97662ca7 18e0fb061186f389cb6406ca0a4f5620c132f215 26328 F20101107_AABJDC wei_y_Page_094.QC.jpg 838db6c0fbd2322b94e3358948c487a6 dc056b63f29b22980c83d5559d81f14432fce8ec 6956 F20101107_AABJCN wei_y_Page_083thm.jpg 8ff7cddac95a7b252f9eb10c172faab8 f7e69338c175749e72b05c218def56e7693ec2ea 21063 F20101107_AABJBZ wei_y_Page_075.QC.jpg 18e6ac604b99d99c1447d4b4fa49d559 dfa453565a84cb066a245f40dcbb6d23b42aa907 9314 F20101107_AABIXH wei_y_Page_007thm.jpg ccc7d07ee94e6247c0bb2b30c2978887 f3f87a818db191a8531587619e62c287178ff1f4 21563 F20101107_AABIWT wei_y_Page_082.QC.jpg 13b578934b804ee9f847f62c9cb45c62 15d4ceb0842b90cc759b89200b9eb5a1f66f27b2 25271604 F20101107_AABIAA wei_y_Page_045.tif b8a6a83710de5b95250dbee4fcd66208 264ddd61ddd5f0a2743b52f666c5dc3108a3cd90 480287 F20101107_AABHUF wei_y_Page_087.jp2 50170aa81c2dc1aa91e30e1162efeedc 2fbdb375fc8919a6864e978a7e987471934fe1af 426008 F20101107_AABHTR wei_y_Page_073.jp2 7fa4455473469d2b2c79474a9fbd7912 12a74fedee653ccf430ed7fff6b5bfe01abbb192 8491 F20101107_AABJDD wei_y_Page_094thm.jpg 659b852f34065def94217789e9d01121 e7e627f736d00d6ef72039bbe076345f797e5eb7 22857 F20101107_AABJCO wei_y_Page_084.QC.jpg d178d9100efd095ffb15e8a7042ccb02 067be8c81fd1f24221a277619f26ddc47a5e6bd0 3654 F20101107_AABIXI wei_y_Page_008.QC.jpg a60915dbfca7d9dc27da730a1c61725a 0327e40595338065cf3f8c4078dd6ec5740a4d28 9069 F20101107_AABIWU wei_y_Page_126thm.jpg 6c40a62b4135893d724aed0305a6577d 3b24b7917fc8639e833a21ec6cd336c0fbb2ec7a F20101107_AABIAB wei_y_Page_046.tif 9b45dbe5ba4b289ae94a8ddc791586db fce3186a65fb60bccf89b76db33c451ecfa3a74d 417704 F20101107_AABHUG wei_y_Page_088.jp2 0aa24a27e19c7bf09fa75721279d5603 4c87169426885bdbac77346f09ccc656be90251a 425604 F20101107_AABHTS wei_y_Page_074.jp2 5b18029efa6d88de9e02ca40f91375df 201c179b7b565a4605198d36bd7b6ce729b769e9 26342 F20101107_AABJDE wei_y_Page_095.QC.jpg 6b9bd5de6ddd552081cbd1fe48c7ff74 762105a4ba21cab368d9c23c473accf3b5019bef 7731 F20101107_AABJCP wei_y_Page_084thm.jpg 67cfa86d7543381c66edf3cf30c9fd47 264a9f1cdf43f358019e19468536fe4642566695 1160 F20101107_AABIXJ wei_y_Page_008thm.jpg d430e7ca401dc2ba0e9bb9e4403ead50 21ae202ad4d825b5b70139e235a12a7c819c5e95 283566 F20101107_AABIWV UFE0022538_00001.xml 2a1e9de692dadae9adb38ebe2a0aee7c d58f9c5de0f11f705f8f95b861d1b226b97eac65 F20101107_AABIAC wei_y_Page_047.tif 5a8019e59476deacfd9c900f64649f5b 828338d78075dde8e5b2c5290f21f5c2ca4f277a 413264 F20101107_AABHUH wei_y_Page_089.jp2 44046b5b7b74a826e75d38bbaad2ac35 247e1f89f959d11f959fb01a0bfa1dd8f3247853 448568 F20101107_AABHTT wei_y_Page_075.jp2 ee7d7b81991bb802d220b47f3e0c91b6 b2aa7091412629d0900b6d1c04fd5a882f67122c 8464 F20101107_AABJDF wei_y_Page_095thm.jpg ee0ee09342627742b4209f39e4f46603 8bf0d1d2fbcb6e7c07f9e5b6803682404e112b1e 21113 F20101107_AABJCQ wei_y_Page_085.QC.jpg 2b88b74161db3317ed627773303bc036 3b3f6d0baaf2f9aea2544ea90ec94791f79aaf00 28020 F20101107_AABIXK wei_y_Page_009.QC.jpg 1ae4d3788cd141c3735fee077d8125fe 4e289ee7a3524caa252f401a0da937d1408c60d4 8205 F20101107_AABIWW wei_y_Page_001.QC.jpg dc64f581decaf7e0b87903f445659535 550ff19e85870caa467a80abbad596eab6254e30 468364 F20101107_AABHUI wei_y_Page_090.jp2 ad4256d29cd2e21596817efe87a45319 4369d0d062e29dbaa3789ecc3790d5f91b4b6441 414088 F20101107_AABHTU wei_y_Page_076.jp2 5507ee3245afea885269b866287f53c2 6385555f26ac4d7b87e10431ff8f48eb8ad28331 F20101107_AABIAD wei_y_Page_048.tif 1780acaec5a2ca159f72c0e7e244fee0 034e0d58145f1d90bc77dc5652a366a161ba2c6f 7120 F20101107_AABJCR wei_y_Page_085thm.jpg e23412c1424dceb65185f5cc51ed120c 51e77257d5fcb8a6f60aabcd45f3eb859ab9a063 6935 F20101107_AABIXL wei_y_Page_009thm.jpg 6b1c7d938bb5ece7d7265c8617e79603 d8c00d0c702f4ebf95c4828f9ebd37c923124288 497 F20101107_AABIWX wei_y_Page_002thm.jpg 80a7c91092f383d242262b698a95c5c8 1e49437b2ad6c808d0928f68f233dc87be17c3d8 413901 F20101107_AABHUJ wei_y_Page_091.jp2 321fbb25f99ad1d1c1bbe11810608fee d55eea733ea13d93fc6b16b7a63596741da09852 408208 F20101107_AABHTV wei_y_Page_077.jp2 39dcc4a1ac97e7b74d34166463658009 89714130361c30fb4f514483e53fdaf1c86792b3 25480 F20101107_AABJDG wei_y_Page_097.QC.jpg 2f75bb0c7e59c899568356bd0ac2e2a4 147e43840b9329acda4a6660eab36addaa7c5ca0 22636 F20101107_AABJCS wei_y_Page_087.QC.jpg 977bbb9f095e07f3dbac256a8424c4cb 91be14a5a96b5501a52994074aea19c46b111d1c 31033 F20101107_AABIYA wei_y_Page_017.QC.jpg b24fe98b290e5b65d3edfe717d116a44 57d8819cd4d2a6349b84e9f06f696a960985a1ad 23543 F20101107_AABIXM wei_y_Page_010.QC.jpg 85c0c6310beda25cabdbfefe876afa8b de1c1e118c40e811e3120b26e224fa8df6cc49e5 9960 F20101107_AABIWY wei_y_Page_003.QC.jpg 7aa5ce712a227d814befbf82f456986c 2897b60f8c8477493a09fffb3cb5445b5a4a1657 409661 F20101107_AABHUK wei_y_Page_092.jp2 2b6ffcbe5e8b8cfe6d12b0ca8cc9e795 4c7042ea3dbefe845a55e053fbeb2d10e5a4f9c9 461399 F20101107_AABHTW wei_y_Page_078.jp2 439b66a68f89131fc75fa45d29b18b02 ef8850312250036c6aae2866314e2f0ec2918584 F20101107_AABIAE wei_y_Page_049.tif 5732d41adda74994e56a6acb181b53dd 39df374ebcaf4bed82ff0d02f9b0b74d1506cab8 7996 F20101107_AABJDH wei_y_Page_097thm.jpg ddcb14476380e12ce9d85669737cd687 f4c83bdc07c6ad64d0c46cf7a63806381a147457 7658 F20101107_AABJCT wei_y_Page_087thm.jpg 6d1ef9a8c1cbd41346a26a5fc7faa6fa 10249d869d830f4d19912c3970bf87b05b575844 7793 F20101107_AABIYB wei_y_Page_017thm.jpg fcc9c3d348a60044720f809713ca074d afc5d77f9597729c9bc1e01eef511d18c82fbc38 5819 F20101107_AABIXN wei_y_Page_010thm.jpg dfc1c7afe00737f5f5f72fe8fc74c660 fdc7cc991ac86ff88cd9a608f7150b848a16ebb0 2588 F20101107_AABIWZ wei_y_Page_003thm.jpg 88e81044e60fee92c8396d78f7c58e26 3edcf9e2aa0a89be146ad9746f500fca15e3afa6 526184 F20101107_AABHUL wei_y_Page_093.jp2 4df0da68e8364758b562b3b21909a75f 13b7b881ad1333f9294173cc3407429dad3c91d1 413544 F20101107_AABHTX wei_y_Page_079.jp2 d5fb8e8eb1405150a7c77c0a8c01fb95 6275c8dfddc7e87fb5b97beee7b4cbe79f676cef F20101107_AABIAF wei_y_Page_050.tif 998b5de969d632e073a03121d07461ef c0b9ff386e8f9bc34b631da13830a69a0ae2a862 7961 F20101107_AABJDI wei_y_Page_098thm.jpg b0e2c6a50fda60d74ec7bf430a4d2e1f 83a7c30a3e6f4adb600b1e16c91376146613eee6 6850 F20101107_AABJCU wei_y_Page_089thm.jpg f2ec77d185d8cba68c115f0e50b09e00 47586af9710a0e00f94bc21cc707eda63075ecbf 28130 F20101107_AABIYC wei_y_Page_018.QC.jpg a4f01a82fcb9b0fab9ec8e9d77468681 d4e4264b11d031abe4d18d44754aa0bd9601f238 33632 F20101107_AABIXO wei_y_Page_011.QC.jpg 3304aef0e5ceca0df4cc41296eb4cc99 c656de04c62670dd618e27ca3acb2c62c25791ff 597326 F20101107_AABHVA wei_y_Page_108.jp2 0c79d164a7f6fc9583f46af77f59fca1 95914a5687b9df69a931619ec2279cf87ee474ea 498382 F20101107_AABHUM wei_y_Page_094.jp2 e66e5523d97d6126ad01c7b370113cd0 afb9a4d058c8d2f41d727a8a08443c85c46a64f3 F20101107_AABIAG wei_y_Page_051.tif 5344c549bd13046743ac9cdadb5225a3 c8c7ef523769ebc71a92ecd5ec7aced6a932b30b 26100 F20101107_AABJDJ wei_y_Page_100.QC.jpg 31d74f555f301f34a2f8259fbfba9ba3 cd172e37befe0fe18e0f3c4ffaa39d4825ad28ce 21797 F20101107_AABJCV wei_y_Page_090.QC.jpg 2e06a4bbba89aaef7787eb7981b0983c 5b22d70fa0640904981de1ba814f45ab4c3954af 7691 F20101107_AABIYD wei_y_Page_018thm.jpg 92052508c77f1936e3d15601f5e3fbc2 ac3606427766c8d1de1bdb1f2f2b3586e02a2da1 8258 F20101107_AABIXP wei_y_Page_011thm.jpg bc306b6cba44a43c6534e65c55236a37 dc3d837222809e9c88dfec97d675045788d33b2d 549768 F20101107_AABHVB wei_y_Page_109.jp2 8ce0b2c5b0e0c383eb6c0a9416457bd6 e9fb3e93e12e1a113a95e2497e18f4cb8da701ef 497401 F20101107_AABHUN wei_y_Page_095.jp2 6ab62785eb1a0fe044fd0a07353faf44 cc718a9985e262ff842efc92a72060d08d6d6e1a 409828 F20101107_AABHTY wei_y_Page_080.jp2 2f0c5a850ea3c5c9c7ee9cf232ea63e1 ac4fed553a8941191f3360ce77eb7a7682a1531e F20101107_AABIAH wei_y_Page_052.tif b4fe29af317cfd00c7554491cd3c0171 beac25323474fe7e0f00c9a440f10984c7149239 F20101107_AABJDK wei_y_Page_100thm.jpg c13eef6b6ee7c4d1c20895c0340c2600 0bbbd64728419c837aa829e1199f501f6fdd2008 7336 F20101107_AABJCW wei_y_Page_090thm.jpg 58121c47a0ff95b37e2f7a2961df8ee5 a4c375cbf3ececd18d984c78c12fa3d114441b55 30731 F20101107_AABIYE wei_y_Page_019.QC.jpg e3bfa557e2ca0a700e31dd5d0c01680a 105bc9dc4c5bf5415b9bbf5cc77f2c02925a9738 34035 F20101107_AABIXQ wei_y_Page_012.QC.jpg 48bc46e262641897efa2aa6c04e38699 208e9e5e906480d82dc2557a9eb808c77e44c2e8 549220 F20101107_AABHVC wei_y_Page_110.jp2 b7b6e4bf7c5e1c30e32bea1344a8d0e0 a4691a0a1ae4868b4456f9e187bdd4488c0de7f2 530352 F20101107_AABHUO wei_y_Page_096.jp2 08d6b5ff95b3a9a11ca97e370f2ab86b c92836e0c4bd0caa6f0e007c95bbd4026896960d 479044 F20101107_AABHTZ wei_y_Page_081.jp2 beb958a04d31e9ab7480acf964c32088 1291fb92ee1779502f0ecfc31f108c8f5a13227d F20101107_AABIAI wei_y_Page_053.tif 2decd3e1e4b5bce0bca03a436f12e5c0 60e360d0f15c5a3c9bfedbc8c7edc4282c449dae 9054 F20101107_AABJEA wei_y_Page_109thm.jpg 27f21fcb2ade9d70818576b8077c19c8 ddcb9dbd9a6f469e2e7395f02f35a574e6a623b0 25904 F20101107_AABJDL wei_y_Page_101.QC.jpg ff55fd9e42dd18d4cd337b962d9d857a 3494f4c1e51fddf73eea0405f5a48650165179bb 20549 F20101107_AABJCX wei_y_Page_091.QC.jpg c6e60f1b0f1c0efe6c1aab37a0e73bea 078afeeb67c15465440ee17aff27b9b9a9192556 29930 F20101107_AABIYF wei_y_Page_020.QC.jpg 28160a3eda8d39d2a1232ab20b9aa919 de7bbea29ba8640b60753c171a0e25edeb7772ba 8919 F20101107_AABIXR wei_y_Page_012thm.jpg e77d41f85633b1824dcae025f2c2f464 622a108ec1ce3f8437cd47cccc20baf30b24fc86 720846 F20101107_AABHVD wei_y_Page_111.jp2 cbb165a381ca40187b68f7fb007d6556 493a1779e55e2f2daaa2c7f1cb32e41dbd1a6d28 481642 F20101107_AABHUP wei_y_Page_097.jp2 8d4a7de819d66f63ef20e189a1e21eb4 80a7850295f83fbde31635b0d1fd7dd54136b026 F20101107_AABIAJ wei_y_Page_054.tif 2f3467bd78e3dc07b2f7192271e13c0b 0fd9832711e0e378e0b990bd83901e3bf0051893 28119 F20101107_AABJEB wei_y_Page_110.QC.jpg ba6c226842ccf144f28a38c8235dc04a 8f8305dd77fa61167a18b2ae9f2751f81d9ec0ad 8487 F20101107_AABJDM wei_y_Page_101thm.jpg 173873228b0836bf60455c4c962c6e95 3d6d55f78075c368625105b33484dbf8063ee01b 6814 F20101107_AABJCY wei_y_Page_091thm.jpg 3aa3a59c9efee7af23e056b47ca9e783 3c439d8e209deb8db24074ddddd1553770a8705b 7517 F20101107_AABIYG wei_y_Page_020thm.jpg 4b3883a4cbf662ba9e1671c32d7c91c3 96e2a3fd3230e6229d578e4d752876f529bb3cf0 22142 F20101107_AABIXS wei_y_Page_013.QC.jpg 10eac8a5ddde731952fb6432a783fec9 c6b3665aebba3ec8b40713e885153ff427b72a5e 720090 F20101107_AABHVE wei_y_Page_112.jp2 5c87470c93668b92dcec7fb4192cfae4 e533e52dbbabb9c21f6d8caadb1dae3952829ded 477595 F20101107_AABHUQ wei_y_Page_098.jp2 6417cfb077877f94586b6a501836d15b 70fddb058e84678db7337ad94b7a86a9083b88d4 F20101107_AABIAK wei_y_Page_055.tif afb2a149ee52e66430fec2c39b0b371d f6db2422de21f67a156fd144e6fae3a9aed116b2 31700 F20101107_AABJEC wei_y_Page_111.QC.jpg 46ef922b242ebe79b6a40e48f5dff84e 7e03902422164cace04bd3ea636f329c1f042450 29167 F20101107_AABJDN wei_y_Page_102.QC.jpg 35e8ab26245074cfe3ae6a6160e95d8b e2b47ae5c00dbc574e2856024583673db5ef5dc3 20305 F20101107_AABJCZ wei_y_Page_092.QC.jpg bbd4056d8baa377dbc93c20b6d555eba fb49fbac7f6c58be0ff2f6c9a9d61180fe5000e0 17073 F20101107_AABIYH wei_y_Page_021.QC.jpg a9f9f1b04183fcc736a78304419f9e75 69bd9eaab318c023ccddea1c59ca852ca0664a5a 6194 F20101107_AABIXT wei_y_Page_013thm.jpg ffbe6a192b05b008b27f66f956c27003 30f7a46128219453f788242461745791ac983f6f 588072 F20101107_AABHVF wei_y_Page_113.jp2 c82ea5a3b3b7685005da4a578826fe15 4550c8b142bfeeddf3224fa6a2b4e20efca8eab2 545241 F20101107_AABHUR wei_y_Page_099.jp2 447dc533bfad0136e92edccb18a072a8 d6f9ad81371e8191b0b49ef3ee737d3b6e44dd34 F20101107_AABIBA wei_y_Page_071.tif 32527a3db2ac239b89349db5610d76e3 67058ec22c3fdc564fc4502081d8ab754431ef4f F20101107_AABIAL wei_y_Page_056.tif 03e7b4311ec838bc066e0d1d5edf28b1 566d1f9a495bd77e52d099bc0f2751bb9e415821 9946 F20101107_AABJED wei_y_Page_111thm.jpg 30eb109ddcf6fdf58cb716f5d8936268 53e4864f565b3f7053bfdd5ab39cf32a6f818034 28150 F20101107_AABJDO wei_y_Page_103.QC.jpg 98ef2b9e0eb77f94167836b0ff4c452c db5ab097282496dd2af14042bf85f232ff02e678 23218 F20101107_AABIYI wei_y_Page_022.QC.jpg 54286f107f7bf28b1fa431844a589d35 adaa044c40413c2e13d3cd44c13602a5b945458f 20589 F20101107_AABIXU wei_y_Page_014.QC.jpg 3f1bd55c331c3c18bdf4b32516af1441 08c021f5f8ee5e93f0ca2845c40219385b141750 541029 F20101107_AABHVG wei_y_Page_114.jp2 188222c1afae5b09a7370d0387911a7c a1f687d2932580e9b14b4a80da9db40a971e32a4 499567 F20101107_AABHUS wei_y_Page_100.jp2 171b002527bc217e6028aab00d2023fa b50a4c40f3a981849f9acda76767da7b2ce2e270 F20101107_AABIBB wei_y_Page_072.tif 2cd27ae75796cf9650e036618af2b988 564bcc7a17c97e665f796df6e06070dd18e866c0 F20101107_AABIAM wei_y_Page_057.tif 69279410f3134077915001a7495deb52 b918a28641218c140acbe6ad7e1037a2801ea7bc 31416 F20101107_AABJEE wei_y_Page_112.QC.jpg 2a0b84d2c6e7b81141257923c28549df 48c934eeacbcf0192152c8dd6c85efb88967a881 8651 F20101107_AABJDP wei_y_Page_103thm.jpg 5726c070a54eeb1d4d73b48946111ca1 c070fbdf0872643a265740dc3482700aef7e08a4 6968 F20101107_AABIYJ wei_y_Page_022thm.jpg 1147fd35a87517505e78d73f5af423a7 892010eb8e0a861d90916780484e995fc63cc58c 6380 F20101107_AABIXV wei_y_Page_014thm.jpg 201b4473316cf0bcc18a61a2af480228 30e5881f397b9010a7233ecd86c39fec94317560 532739 F20101107_AABHVH wei_y_Page_115.jp2 fbd5cfcda76d0b5651cac9abc9a84e66 661f8ac08488bab76c48a28e4a354e8c3ea29213 498498 F20101107_AABHUT wei_y_Page_101.jp2 cbc12424097283c68c8a44554b584314 2b7ba1487a618414d6e3c84ca09f17c3cadf85d9 F20101107_AABIBC wei_y_Page_073.tif 11771b3d84fdb73a719383b8813da4e5 8542758f6edd4f4c39a7cd9034ade1dc973da96f F20101107_AABIAN wei_y_Page_058.tif 9d2c36f99743fd65423614ce46924b1b a7dc2ddac6edf51c354ed2d7ecac27f05a3cf7a6 9970 F20101107_AABJEF wei_y_Page_112thm.jpg 9954f5f4701f3275757c9f5e6f2dcb7b 5685d9729f2d85a68c4112149d7957802efb7dbd 28065 F20101107_AABJDQ wei_y_Page_104.QC.jpg 440c38aa22c8124496d801a01e9c57ce b97bd7af9711f4e4ba28fc3ece11285e4bf5a408 26057 F20101107_AABIYK wei_y_Page_023.QC.jpg 457f54e5adfd321f5beedd96f5765bd6 273ebed1fd5ee5023d88d9277b0a97d44bd4c3b8 19257 F20101107_AABIXW wei_y_Page_015.QC.jpg 2fb147a0640f197d96452be40ac603fb eedfaf291b1d2bed655859fe07fa8edf75b89eb4 390661 F20101107_AABHVI wei_y_Page_116.jp2 c4fb17a12d6051d305d1719979611305 d0f22abe68a01d06c20b032e1b7a22ba8c29ed3a 584571 F20101107_AABHUU wei_y_Page_102.jp2 7904086319cbb4050c0c722e8d116bf5 3a856a065e3bfd694f03d3c7833d4ee91b50b192 F20101107_AABIBD wei_y_Page_074.tif d3660206df2e34eb612a66183030dbad 230e7a3bf48c732edddc8532f84997d7174a88a4 F20101107_AABIAO wei_y_Page_059.tif 9231bc96b033e04a6dd41b45135c8ee9 6a0b3a59b23adc13e111ad8cf42cde72efff65ae 28917 F20101107_AABJEG wei_y_Page_113.QC.jpg 9166cdfbec55e3829655db6a1edf000c 8da9a50fb6b6ca7f5b8643131631786cfed8669e 8742 F20101107_AABJDR wei_y_Page_104thm.jpg c2262fd8f6b9b2749631d00887d4a0fe bdea8e1a9552c1bc5bd5eac84e881848883f501d 6969 F20101107_AABIYL wei_y_Page_023thm.jpg 95074861779b739de00372e87fae6574 123dc0172ffe62a6b11730091dec680eb855c476 5666 F20101107_AABIXX wei_y_Page_015thm.jpg 7b8ec3639f0a4060c417503358cbd7cc 6ed991faf7e67184b52b0c367d687cde22c701ad 416819 F20101107_AABHVJ wei_y_Page_117.jp2 3693be3e1a8540b064e59fb25fbc7a54 b0cdfa66a9ec1f76aa1f69bc57cbb4b711498d9a 537605 F20101107_AABHUV wei_y_Page_103.jp2 d78bb7d3a697550c1d67ace06b73e6a9 e017682e66dc14fd008c004d19b3f55ce5c1b4f3 F20101107_AABIBE wei_y_Page_075.tif 01486052d806a2e15e233edb04f9417c 766543c4aa277b62c00a56751da1e2fd4b6416a1 F20101107_AABIAP wei_y_Page_060.tif a21d34b4279aa16cf208bb182b8efcb0 c8733cc5de494fad5679b25bf5b0f144c5b5b1ca 30421 F20101107_AABJDS wei_y_Page_105.QC.jpg 0c73762670bf5f592a3ca15c4215f72c 40de0c6215943af3db657fc4cf26ce5c58cafbcb 7670 F20101107_AABIZA wei_y_Page_032thm.jpg 588d40fb070739aa01745261be5307fa 8f15ce2115b862d8f06e130322313fd7ef7b5d9b 11356 F20101107_AABIYM wei_y_Page_024.QC.jpg 4342457d58aa21e6ef161c2c8bd82ae7 182129dc642ff41e066c3cea51e379343a63de68 28426 F20101107_AABIXY wei_y_Page_016.QC.jpg bc1f87619663661ebcadd1d0ab53259b ece83af37b5c19f33d6fb754351fcbf0ef713304 369787 F20101107_AABHVK wei_y_Page_118.jp2 10073ef1118780557ab6d0a62402f2ad b91cd8dbb61802b6a5fb9aeae829f2228175ca00 536381 F20101107_AABHUW wei_y_Page_104.jp2 d517ac2e1670ce3fde533e30f3fbfbf0 78c218d40075455d815080432a8d0b463b540a88 F20101107_AABIAQ wei_y_Page_061.tif 9ad9669cd6859d92dbd1ee34a3e2c670 0b676a13ef3848e41cd7b7b30c5048a73adb9d77 9507 F20101107_AABJEH wei_y_Page_113thm.jpg 94e453e1f4c5c36bea477e86a69c96e3 c4dcdca37f03b15e4d83ffc2b2c046ef53ea3a45 F20101107_AABJDT wei_y_Page_105thm.jpg def14f54ba493411a7ba65ac88d0a88b 4560f0839c7a0a3d883c07ff5f09d79eb548669d 28269 F20101107_AABIZB wei_y_Page_033.QC.jpg 67dd349946756380aec91978633815a6 2708288dc1611c79ccb804938f08d290ec2c20d7 F20101107_AABIYN wei_y_Page_024thm.jpg 8ebad1f0ea7840c0048cc7d5f7a31913 3fcad3bff691cb18e4b395cbd6f27784555a6d9a 7169 F20101107_AABIXZ wei_y_Page_016thm.jpg a28015f4873342cf32b35f68fd704a75 6ed13af9087881e1fcca068a013acccc59e67920 464910 F20101107_AABHVL wei_y_Page_119.jp2 fb609eb36b07f5335d01ebf2b5c5c410 6831748c5806fcdd88b4584dbceb82b8726685f5 608470 F20101107_AABHUX wei_y_Page_105.jp2 36b09d0f54cc2bdbe6e71bdc06faecc2 e215da9eb8fb861427db54b168d3db49fc7dad5e F20101107_AABIBF wei_y_Page_076.tif a240eff20404a3095a42ee890bc04a83 e771eb8dbb3132cfab9126283e59a2c3ddfd8c7c F20101107_AABIAR wei_y_Page_062.tif 7092b500d4effdae092efacc6b1b778d 2b952fe7204a5903cf4e426ad0c9ed24069e9172 27555 F20101107_AABJEI wei_y_Page_114.QC.jpg 20e205445ddcea2f6f75887189414d3d 3b437fd0f46ab7a034e51ff0a2401b8e44e7796d 29065 F20101107_AABJDU wei_y_Page_106.QC.jpg 8f19f7d2137fd106851613bfd9f7b5bd 4af3c37fc6c25e4b1095714aa807e1e2ed459c44 7137 F20101107_AABIZC wei_y_Page_033thm.jpg be035131d7f8b3d48c3d383c83387b2e 0d0768d4e43e6a94dfa1d56987c298b5256adb36 28289 F20101107_AABIYO wei_y_Page_025.QC.jpg c452dd371719c23ff832bd6acf15a177 2b71ac63bb7c7c026dc6d71378f9d7abd850a42e 387467 F20101107_AABHVM wei_y_Page_120.jp2 e6ffbeb47f1ae01c99bdc0ba2b3289b7 45ca54478ee5cfd8df640cfd6d9e42ffb4898529 565934 F20101107_AABHUY wei_y_Page_106.jp2 6514adbcbcbf61a13782baa1275218d5 b3972ab9a0a7059a368d45956972c6a6d67bb501 F20101107_AABIBG wei_y_Page_077.tif 60930f3b5de65abf8dc285f59cf6f06c 957ca57b6022c01f0d8ba7fb9171d76470714bfb 1051954 F20101107_AABHWA wei_y_Page_134.jp2 6a1ec2af5d3dcc3144d24004a32d32a4 822c2efb285f570574883d52918d9859d5269e80 F20101107_AABIAS wei_y_Page_063.tif 2416a4193ef0199eb440a61b1cec5fed 8b937167879fd7c50c48638c4c23663316d9d22a 8807 F20101107_AABJEJ wei_y_Page_114thm.jpg f312afc7eb60f8dd633bd76a60399a5c afb51cee18d717101e9c982c8e82509ef55a96d6 8955 F20101107_AABJDV wei_y_Page_106thm.jpg 90b10544ae5a0a77fad3035efa284f44 c815407d7e1929c84d3beb8b8e6dc5ae6c95095b 20560 F20101107_AABIZD wei_y_Page_034.QC.jpg 07ff9a27fa64946900e9c01fc325383d 1b8d7bb09adcfca8f27ed18e9abc39305c1de427 22972 F20101107_AABIYP wei_y_Page_026.QC.jpg 7882489395dc528a24b3689dd2b18c31 69765f9f0ab6e0ad76ceee31b0359991d79237b8 471611 F20101107_AABHVN wei_y_Page_121.jp2 8cf718fd062d51c239a31f5ae5e01f73 9b0b335549478992405a420510e676899f45f325 F20101107_AABIBH wei_y_Page_078.tif 8775a86f8492edfef497fd4f86be7ac2 8589360368b4825474534f88ea06d5e03e6f9755 1051977 F20101107_AABHWB wei_y_Page_135.jp2 2e6771e6ed43830cf55b280de97ee02c 587a451595ada8a1df89ee64b821cdfc13bf8208 F20101107_AABIAT wei_y_Page_064.tif e30553438716e1a34bbb1918615b5a3e 83f4accc548e94ccaf8ac509754fd4171d7d60d1 27147 F20101107_AABJEK wei_y_Page_115.QC.jpg 97773113223aaa77c4da0ab96225d0c2 c25214e23e19964da35708734fce621c1a2f79ae 29121 F20101107_AABJDW wei_y_Page_107.QC.jpg 1010bdf485a6ebf65b5a673d615b7c7e 029f1d48946b691f0d61362d27f4a355fc25d40b 5783 F20101107_AABIZE wei_y_Page_034thm.jpg cbfe59a82478aa3bc80034188fc3b8d3 3d9831cecfbb962a34246ae437ea1da49003d22f 6414 F20101107_AABIYQ wei_y_Page_026thm.jpg b39b63143d008ea6868130b74fe98039 6939c56cfbc1dcc7d3910c1b1cb8beb36b92a4fc F20101107_AABHVO wei_y_Page_122.jp2 fc54cb4dbdb8a119746b204299dc8bea 6874b5fe0fb2214c1aac1e641614534b4a9ea564 564280 F20101107_AABHUZ wei_y_Page_107.jp2 b56af7d7e4a9407a29e9fef6d989e155 8d9a2a2443afc91949838a00050fd75abbb45416 F20101107_AABIBI wei_y_Page_079.tif 4cc5a43abb7416561b4c7d55cd4b2023 ff7234587475929a0b0dcbe0014d2b76a767e6c8 1051973 F20101107_AABHWC wei_y_Page_136.jp2 39cc1b95884b8c76ec523a33be53ba6f d02def4d87170bc2cf62eebbcaf369f8b6132b58 F20101107_AABIAU wei_y_Page_065.tif d724add5a5d2e31266aace3dd4133ec5 9ea3298be7fd2c68fd3063a957f89a0409097fc5 8945 F20101107_AABJFA wei_y_Page_123thm.jpg 3404b36b1cb6cc1fbe271b424ebebaf5 59fe04a455356bff0e6eb2768e1fe2e3bc8a2b96 8882 F20101107_AABJEL wei_y_Page_115thm.jpg c02ce4b6b4e82852eb929d00c8563090 f43e1ad4967c178fb24ec652f03e8ad885612736 9248 F20101107_AABJDX wei_y_Page_107thm.jpg fe4d3188676ca5fa8af99f8c7f4ff947 9a8b9fd1749163572b2f7f86086015580fe30a86 22883 F20101107_AABIZF wei_y_Page_035.QC.jpg 847a764d8db13d45a174cb5ec009c05f 8052f7ee072103dd434fb6f09629b706d34f37c2 32664 F20101107_AABIYR wei_y_Page_027.QC.jpg 3272b8f6346f1436ee4e932ab441a5b4 75a8524eefa34bb4467cdc15edaebda5358222d7 F20101107_AABHVP wei_y_Page_123.jp2 87e0af650d3fe7a4f719fd1e28e14a74 6336798a90680d965a2a7f66c8a68b9973a2c116 F20101107_AABIBJ wei_y_Page_080.tif b2ed692a6022b8b8d01f763f269988e8 012c2963f09f16c51257cced317e54f6efbcd0b9 179112 F20101107_AABHWD wei_y_Page_137.jp2 c070b48d8534da1abc6129dcac377624 4c0a811a382284d67ad2510e1aa9a64e63512c36 F20101107_AABIAV wei_y_Page_066.tif 392f69593a28c4a01d96390a2af1cbb6 25f7f1e040f3a88cf8d966e77f53fca411ff6bd5 38134 F20101107_AABJFB wei_y_Page_124.QC.jpg 140aa34eb87fd985d588ad93862a70dd 29adb06e7984684f9e68a07dc759b1fb199cd1b0 15294 F20101107_AABJEM wei_y_Page_116.QC.jpg 963c2283b7282458804de975748c6c64 6de4c392cc97b5e397ec31b28385a51d399d2efc 29513 F20101107_AABJDY wei_y_Page_108.QC.jpg f066fb9370166474654cbf96b095f8a0 03cc0bfbbcd0c0e79a93f21f8807529e787891ad 5979 F20101107_AABIZG wei_y_Page_035thm.jpg 8c0eac8a36b0052c1fcaebc2ae009e5c 33d9bf6a7275f30ef04e896d85359eecdf2a9630 8725 F20101107_AABIYS wei_y_Page_027thm.jpg 6ddc4767863d1ca61b47845da9c43679 a16185f40cb61a609fd34fdd2cd6b67c8e19ea51 1051982 F20101107_AABHVQ wei_y_Page_124.jp2 9f7e7d781170b5bb0b68322b7764ddd3 f70731f9d9a89f6db8538717ec6131403ce31cff F20101107_AABIBK wei_y_Page_081.tif fa9e50c58ece6be1a9dd5c2ebfefe361 d6028c1481b9147918b56640eb7f2534d6605889 710817 F20101107_AABHWE wei_y_Page_138.jp2 b05e46de8ef223f8f70c4e2ee8047f6a 1ce7a46d7a66fae06bc9fe3afa4ca09f017e1cc6 F20101107_AABIAW wei_y_Page_067.tif 5b5b2641972a7ff93ad9715be750ceef 80fe60f153f89f82860dac36d51dcf4c89208eeb F20101107_AABJFC wei_y_Page_124thm.jpg c8aa81278e62072a56b603edd058c545 8964f4bc06cf41db2025a3ef1882ac7a05449a6d 5173 F20101107_AABJEN wei_y_Page_116thm.jpg 6504f2befdefb4374f02014d9dc505c8 1125591e72e6ca069df47a5e2efef0839736e2cf 28164 F20101107_AABJDZ wei_y_Page_109.QC.jpg dcae8e16c883d193b2d6837c2048486a c651d916615aea4edf2625d2cb5d55acc3e5ff60 25428 F20101107_AABIZH wei_y_Page_036.QC.jpg 4300a677b97e5b28a7cd2bb330781218 6a2e6343921f61c9dc24f704786b5fac8213b4c1 22197 F20101107_AABIYT wei_y_Page_028.QC.jpg 47c6c29b82016d1ebc6aa7f85eba5265 fe9edef2fa6f911f33830363f7bb84dcdef226de 1051946 F20101107_AABHVR wei_y_Page_125.jp2 d4d2f505144acef44aea32d11a893225 2c8cd7b5eabbdd4d9cef15a897f6ad7cce391af3 F20101107_AABICA wei_y_Page_097.tif c1c34a9da367f011ee578fb4224a960a 1ef2911720924a3ff73397e16cca4924f68a0184 F20101107_AABIBL wei_y_Page_082.tif 4b8d8745d000abd4e930b6f590663567 46e1d21b1416afbbcc7dd55e1303833f4d224350 721224 F20101107_AABHWF wei_y_Page_139.jp2 ee33b0902bcf9e42a8c68d10d065e865 2a412d4d5d94f67e3a9e7072f9a7d7413ff5a200 F20101107_AABIAX wei_y_Page_068.tif d2961023ec0a1d86adddc871660ec8ce cdfdc833e14d39ddbbca712ead11fffbc5171cad 35694 F20101107_AABJFD wei_y_Page_125.QC.jpg 43bbb7c003cfea0cea5514f317b6c093 723a6185324a0ef5efad93a93d180c9c13122c6e 15847 F20101107_AABJEO wei_y_Page_117.QC.jpg 37c1cd9bdeeb1289eaa1fe4947034043 5392343011b5a08761d362afd2536b7d0e51c0a5 7108 F20101107_AABIZI wei_y_Page_036thm.jpg 299457f704fe9a316b568023b6a13ed9 4a2b5cb16af46f44080ca01a14180966a96eaf41 15177 F20101107_AABIYU wei_y_Page_029.QC.jpg 3b0cdc2663d9529910dccbfd97e6b9ab d94c92fad0a289ec269ecf89c9c19ad911613e0f 1051979 F20101107_AABHVS wei_y_Page_126.jp2 40bf35fa64d65855aaaa00f6752a5860 2eb57023fd2c07c9e45ed3a0bde9842524c830b2 F20101107_AABICB wei_y_Page_098.tif c6ee6666d0940f2f958b6f4ed16bf5c3 0e870a442feb8c582f69695e867aab984ee78976 F20101107_AABIBM wei_y_Page_083.tif 8ee7cf90da6173ca4fc4ab988096553f e863667fff0830b1a18b432f96aa7d8b20fc4420 722711 F20101107_AABHWG wei_y_Page_140.jp2 3a4b7df7f232bae2c8db5c516483446c 4654a027e09b7d476d5aa61ce1788c0788ab34bc F20101107_AABIAY wei_y_Page_069.tif bb9709b0a4cde4afbcabc0f5c41078af 660f62e64369672e4add1612ab3a753f30664f1a 9096 F20101107_AABJFE wei_y_Page_125thm.jpg cf19b22cb39e0765fb26be9adca4908c c95ea76478c1b9732d06e7b46d14af08f0ff308f 5240 F20101107_AABJEP wei_y_Page_117thm.jpg 0472b0dc0588dda9c186397edf7bf6a6 402e54adc9fc3472ed6d0e4181c3c8161eeab8e9 28023 F20101107_AABIZJ wei_y_Page_037.QC.jpg 278519faa582c5252b0cbd396c8fe9ce 255c5f3ba367311fb648ab516462f43102ac3c31 4453 F20101107_AABIYV wei_y_Page_029thm.jpg 7439a6fea15c6163fac5851ddb496df1 b9980b014e83e4eb058d52eefc1c2e49dae0d7bd F20101107_AABHVT wei_y_Page_127.jp2 9c9e9030d1a9baecc8e58bb36c36911a fe02dae5a70d5864ef8d9014c933442705f5ef1e F20101107_AABICC wei_y_Page_099.tif 93104688e79c09d973d387cb03dc20fb eaf034c6f1b883bc930ec4e22863f842f21a8285 F20101107_AABIBN wei_y_Page_084.tif d8d34b5e5e7b0b5f3021cce0769f2970 697cfd90b8fa076a49d2e04f636b0b0c4d8495af 722487 F20101107_AABHWH wei_y_Page_141.jp2 2c10adb786a7ba285e18a62c4ff6f534 b95e5c7c52038271e26f9da16e366ee22ca27d84 F20101107_AABIAZ wei_y_Page_070.tif d43326caa5a5c99843d7e72d2d6847b0 dcb491b622cce9ced10b00a78b5d0c3c6ec6a65f 36954 F20101107_AABJFF wei_y_Page_126.QC.jpg 4af8a0aa3b6b975dce54a4a6c57ba702 9fa441a7327202fa69b2743ab724fb9dad82ef14 14608 F20101107_AABJEQ wei_y_Page_118.QC.jpg d71c6e6f2f44e793d31d94c7495c87d9 c4ad1d696c659f60d75f012c345fc9ba70dd3673 F20101107_AABIZK wei_y_Page_037thm.jpg 8c11524d5bd8f4a3add90a6752038d09 dc89ca1d77d919fbfb1d2fa71c89b4c2fa843a54 8436 F20101107_AABIYW wei_y_Page_030thm.jpg 52fb4fc603a93577d5d3007053e2f8b4 d00341c4d868ad933c2f1e2bc07abfac9319d476 1051969 F20101107_AABHVU wei_y_Page_128.jp2 01f6ec23dc151400c15e8f3c405d1097 3dd20d70bc1dc50c21730a19af334eec5b638056 F20101107_AABICD wei_y_Page_100.tif 3509b2e43b60bedb015bcfb7274c1d92 f63bcc2b98bf75b37189e6b9e7a815912416ce45 F20101107_AABIBO wei_y_Page_085.tif fa8adae45f883ff8ff15136340d49cf0 33f39c528cc19f5a16ef46aaf82fdf7b49acbd8d 708198 F20101107_AABHWI wei_y_Page_142.jp2 6b0835ddaf877452df129b1af575039c 27603097e87496f427e12a5c11af713316d6f7c1 36993 F20101107_AABJFG wei_y_Page_127.QC.jpg 0ba71424fd54ea31daa9bf450f131090 bb8b409f739202a57a9506ee88891577666b341a 5106 F20101107_AABJER wei_y_Page_118thm.jpg d774ff554e28ea63508e8c099b76c2f1 1d868f18d22396016a8d36c523abeefc339754a1 23764 F20101107_AABIZL wei_y_Page_038.QC.jpg ad6fd3578c5664cae5fcc6ce6d52c154 18e716f04d5c9f76a55c0273c791fbbe368c3639 20550 F20101107_AABIYX wei_y_Page_031.QC.jpg e15c62ae50b2b0bfe8e38cdcce80c3c9 1eb57064baafbabd74c2a75eb8fde8221fcf6cb7 F20101107_AABHVV wei_y_Page_129.jp2 aee56e4053945d984a806bba940aa034 01113fbf355f1c5b7b6e0cbe1c5cabbc1b804847 F20101107_AABICE wei_y_Page_101.tif ad528199c3dd9cba94ec87233cfbe54e 24a4e6f9d1d5b42b55b77f1080e281483703be7a F20101107_AABIBP wei_y_Page_086.tif d8d5e0eafc44d9a87df3856c4c7303b8 4c4ec9ccabdb523c100a9fd1c288fb1d4ca24b0c 686928 F20101107_AABHWJ wei_y_Page_143.jp2 b1efc6e94bbc9b0aca2b51cc757fd1bf 24f90f4aa7736925e3cb3319d95920e3ba80bd39 8886 F20101107_AABJFH wei_y_Page_127thm.jpg 39e6cdc137427930a2c4e382928a72c8 5e5237938ea47c11de0bb98b1d04ad37db71dd8b 5487 F20101107_AABJES wei_y_Page_119thm.jpg f121f6b8f5d077c8fdfd8bdbf7955a71 b501c1eb88bd0b5c8d0c779dc3575e190b6c8727 6717 F20101107_AABIZM wei_y_Page_038thm.jpg cc401602362ea9841ca66192126c91da 414d27608c3c255bdbfaf56e2c807940e8442a7b 5894 F20101107_AABIYY wei_y_Page_031thm.jpg c245d295cb594f329be8e1167dbe1e9d 764ab8d53394403826e6dbc698b1486c6c5880db 1051919 F20101107_AABHVW wei_y_Page_130.jp2 0acd7bea6205eb55151729aeaf343fc9 2418d1b289a65da9462dfce12dd76a70b07262b8 F20101107_AABICF wei_y_Page_102.tif fd786c85aeaffa9c2427c9181cdfcf13 1e2a9ce9cfae35666198c4506cce3b242c003ab1 F20101107_AABIBQ wei_y_Page_087.tif ba4ce25ae3a83a232af25ab6ba04886f 3fc475ac14ed7d0bc31291449186386e3f9909b4 710531 F20101107_AABHWK wei_y_Page_144.jp2 370e06447f4e693f200b99eea48ce179 16bfc6cc2a8347ee45cf5f5f23c66269bf3743ee 14491 F20101107_AABJET wei_y_Page_120.QC.jpg e642a7e418863fd2f68fa34a9f3b8d88 01afae5f541778c443fb1b0eee626ea40e850bd9 33829 F20101107_AABIZN wei_y_Page_039.QC.jpg 183d93ecb71b54d940dc4e5c671ba006 a590ce85a3432c0935c8135f0e3878051e163b40 28250 F20101107_AABIYZ wei_y_Page_032.QC.jpg e073dc0633d97c7fcc9551c9189ca09b 6f325bafe991879a4a776e8cd2b1eaf265bcfb0c F20101107_AABHVX wei_y_Page_131.jp2 5df2d5bdf9f7cfe2544c8cc03b4cf1d6 2c01ff6d4578b79955adbd3974d86eca6f750f40 F20101107_AABIBR wei_y_Page_088.tif 86204c02941fd162cb8f03bd00ae0499 e7c1b053b950b2fea06f043f62ac6175b90e31b2 713726 F20101107_AABHWL wei_y_Page_145.jp2 f4ee1bb2ff85cd184421e1537690d765 de519e7155f89487ed06167945d32a78df455a1d 36282 F20101107_AABJFI wei_y_Page_128.QC.jpg c432bc82364ed52c1fea9104fd5dddbf 24c33ec174f1085908bc33879ee209cc9099b0e8 5177 F20101107_AABJEU wei_y_Page_120thm.jpg 1fa8d4dd197f8032976b805c698be0a8 ee8bcc97d887ed09c0db673722bcd663da77018e 8476 F20101107_AABIZO wei_y_Page_039thm.jpg 2989f199ca30368a9fead790c13f6fc8 0a8a545fc373c559f5f139bce680d239d3141fb9 249719 F20101107_AABHVY wei_y_Page_132.jp2 3399fcc1c8cd66d05904251f984fa6fb 6a7dd1ab762c50baa3d07b572cf3be5b058d7ef7 F20101107_AABICG wei_y_Page_103.tif c8ca09537e6d0d4c5297a509e5970497 6d71a9d35463030dfb743bef0bf26dbc3aea5da9 702380 F20101107_AABHXA wei_y_Page_160.jp2 503161bb97d5cf911d3a3853316fe8ef d0414f9eb1d137a3ee4a45859c90a96c2013d10b F20101107_AABIBS wei_y_Page_089.tif 507aa5bb8109802df511424dc876ca07 66fa1abe85fb61fbb4f239d86f1ae5c02e064c8d 700766 F20101107_AABHWM wei_y_Page_146.jp2 4c66f30b75a42ace00c8bc81e5c7ea22 d74c0744e12720bd9d8ea4900af22d622e77fb9e 8925 F20101107_AABJFJ wei_y_Page_128thm.jpg b555b62f9caf53dcf9c18e96ed90b94c d3e51d1dbc0f7e7956c4bc808e401fe07aadcbb2 17115 F20101107_AABJEV wei_y_Page_121.QC.jpg 942d270bce179ec9c17c849e78bda574 90107312fd88a1159229e5bbd8ea8ba656297eae 37806 F20101107_AABIZP wei_y_Page_040.QC.jpg 4bc8ec329565467a7e47d29d86002157 7f4013a87a2c1c4f0d5cce17ce200f51f64418a9 F20101107_AABHVZ wei_y_Page_133.jp2 f89ff0dd15bba3574ca4fbbdab3fb9a1 7849e2caff697a7778e53a70e924105b397d6dff F20101107_AABICH wei_y_Page_104.tif 1fb0296ddaa5f513dbe48f6dd1878542 59f1a27fd70d40b37882c70af1cbfa268b620338 698130 F20101107_AABHXB wei_y_Page_161.jp2 a52235eae16bd8aabb700aed97352abf 7b5d3fd74a77930d4f2b392de0c03301565f607f F20101107_AABIBT wei_y_Page_090.tif 3a0ae67406d7becb5cc4d64c5c9b24ae 52ada4382b42b076bd89a371adf35488f1d7f9c9 702886 F20101107_AABHWN wei_y_Page_147.jp2 7716174f837e1c35636208567676556f 056ca6309a430f34262fcd542e22ceed69fc8575 34672 F20101107_AABJFK wei_y_Page_129.QC.jpg 6ab0880380dce3111304bd7b5e8dc267 e2fd8fc4724cc39c69aed6bb664e978b14e33061 5607 F20101107_AABJEW wei_y_Page_121thm.jpg 535dce019fa8de53239aa06060ef235f 3ebc97ddb314f9372cc2eaaea9820d2f41d65729 8980 F20101107_AABIZQ wei_y_Page_040thm.jpg 3cd24ef94ba153e19ee1006f135a5b6d 9828f53476d4e895e31ea093dfcc5c8a44e60871 F20101107_AABICI wei_y_Page_105.tif e6d7254f76f2432bedc90265f9677dbd 3d5f9087ddab3b6feb3ba9965821ae4978bf4c55 728394 F20101107_AABHXC wei_y_Page_162.jp2 a9c43d8eebdf8dd7b0df3af24a9d5e50 b13c10567ce7b8d45a6e30d553a65e7ed8f64936 F20101107_AABIBU wei_y_Page_091.tif 448d44f9b73abb27c68243aeec9818a3 8cf3f9e0d50721e3257ff317c6337e65440c4a90 705254 F20101107_AABHWO wei_y_Page_148.jp2 aea8ee7438dad1d5b98fb4e15d30b65b 8ecbcda147ad87e74da73fd0a88b04f528a823dd 6631 F20101107_AABJGA wei_y_Page_137.QC.jpg 9500da0dad9f91e910b6982c9fe2a0d4 d2dab99ab8da55825450be4a447ff7cabc54f787 8795 F20101107_AABJFL wei_y_Page_129thm.jpg e8bec1345966f4a36f28a34780b79f1c 2a3db9aac1fc4027cd00e349accb01b8190ad464 33787 F20101107_AABJEX wei_y_Page_122.QC.jpg 94f7739d85cfd5c5ee3cfaf833c054bb e72fa4d88f86c89b07fed965ebd1f812c529f22b 9810 F20101107_AABIZR wei_y_Page_041.QC.jpg 0c639234bfad00e4b0d538c3e390b1ca 3923e432b70127092efd0fd3d1e1045572df8ea2 F20101107_AABICJ wei_y_Page_106.tif 3054d6b9a198bb1964050dfa330b2a52 e14be029f183cd8b03858b90b0062fc7dee9075f 729776 F20101107_AABHXD wei_y_Page_163.jp2 9e52c5de85af361340dc2030d23e38e9 fb37d78104497e36569b37f6075908c0ea0f2d84 F20101107_AABIBV wei_y_Page_092.tif 4b33b694c1c51a57ac5eae00842ad48e 93cea8db69ad92a5c8d8b4a3c285d3c87c81934e 705940 F20101107_AABHWP wei_y_Page_149.jp2 a46d211a16616198ab62148aefc5795a b772ea00aaed27cbbfb7ae5c949079329bc1a151 30438 F20101107_AABJGB wei_y_Page_138.QC.jpg 49fe303217566467a2e993be0c303591 32d2aaf1b885d97d2fa87d47dd983230a079232d 33960 F20101107_AABJFM wei_y_Page_130.QC.jpg dee794649f136baf3b2c470b35c58a45 a6ce5754da6327d9cca35eeb15735c07f81dade0 8522 F20101107_AABJEY wei_y_Page_122thm.jpg f70ec2a9bf4d8fdf0e9c27fee372220d 643349f565f56b3c7a32dc4e3769c3c0ffe11e3a 2463 F20101107_AABIZS wei_y_Page_041thm.jpg 36fcc5e6bbbd38f165f0fcaa9a28ffea b7049e63b0416e7a0cf1668577783f7a451fb9da F20101107_AABICK wei_y_Page_107.tif 30032cd911b9f584a1574868e5b17fae 95905a19d086b88bd7dcbb5de158d3c71f3c64c0 729203 F20101107_AABHXE wei_y_Page_164.jp2 5e0cb3df4b4d5269b0616b792bd89415 7ec153ef7f101c1e46d3e03110c965067a08be84 F20101107_AABIBW wei_y_Page_093.tif 3bf59c0b76daa162d954afce9278f53e 2af65b0b993cb9ea6658413032bc8bcfb298c0fb 706063 F20101107_AABHWQ wei_y_Page_150.jp2 ea16c8dade21345d9362ad419758507b 55bf8b61300ef195a438798c24d3e5924f24cb4c 9486 F20101107_AABJGC wei_y_Page_138thm.jpg 38518549b7ca2c0898dab1592ea74343 9833bab3e39139ed2ff41c4a93fe46036de22eb2 8511 F20101107_AABJFN wei_y_Page_130thm.jpg d8838512de378901c24f8d947751ffc9 41338125aa8cf8665d296e608979bb1a0179140c 36616 F20101107_AABJEZ wei_y_Page_123.QC.jpg 8266e10ad9e70dbd031ad65cc3ede664 2889e6f0124769dad1967e055fff66b091c6974e 31548 F20101107_AABIZT wei_y_Page_042.QC.jpg 83814adcf4b1fc355c07872e931a341f 9897bb74c3920d6b6ecfb2ebbefcb5878ba5566a F20101107_AABICL wei_y_Page_108.tif f93a58d44195a4c29008eb6edc4eed56 f6f6896fb17f13c9753a9fb5ee51a0ca4035d5f4 731963 F20101107_AABHXF wei_y_Page_165.jp2 f2a9b6170b76b119ab536b90e852c8e1 6bf64125314c244be0e0645d2ecfbab6854031ce F20101107_AABIBX wei_y_Page_094.tif 170f5aa8f2f799456e725e9027830b11 69a017a58fb720861fea94961806b099a5a8da26 714390 F20101107_AABHWR wei_y_Page_151.jp2 7b8c2ab964b87987d2d7c6e77c8782d2 8f3f0e8f229c2531d0ba52ede9d910bff2af4a6a F20101107_AABIDA wei_y_Page_123.tif cc589d99d6612639ad2bf8611698935e 2b0881a1afb21640e83b0c74aab6158b693b934c 31738 F20101107_AABJGD wei_y_Page_139.QC.jpg c06e31e2721af68c37f32c298b60cac8 bc83218f4fffa759851848ad1a14056215b6acde 36385 F20101107_AABJFO wei_y_Page_131.QC.jpg c032c909f7ce14c466d7484cfaade32e a2e5e45113cdaf9a5714fbb792c231b11f24447e 8321 F20101107_AABIZU wei_y_Page_042thm.jpg b19944702aa11d5b42d82124fc931fb6 4d625ab0dc63598ba81fe49db87ecfdfc38ecc2e F20101107_AABICM wei_y_Page_109.tif e58407540cb6d5bc8cffbf33340b419d cc1b39ecf81e4d2573e62cea67294cf490ba937c 719216 F20101107_AABHXG wei_y_Page_166.jp2 59ef94cec74cedf7206ead798f24f5b9 bf8cd92c1959b98d3aed41dbc89a413e728f99e4 F20101107_AABIBY wei_y_Page_095.tif 27be5fc3b0f0236f36f94e4f9fbdda80 7c7741c3616193ab6becc41afc796e12330c40ce 708273 F20101107_AABHWS wei_y_Page_152.jp2 571d9b9f1fb51c87d56c0e60d4dd4be6 f1f0147c7a57a232faac6f89e71e1e3361197e88 F20101107_AABIDB wei_y_Page_124.tif fb4fcb44afb15c62f02506a3daaadb06 b21f9985b53a779f907434df223a6845bec37e26 32011 F20101107_AABJGE wei_y_Page_140.QC.jpg a0e81d5c7bef5b11732bbaab500b38b9 0d395d3b997706e6aa6ac6775c9621705e40f5fd 8588 F20101107_AABJFP wei_y_Page_131thm.jpg 94bf2f7e2def9f1c0a6b207682e3ae7b 6f41b25d7f2fab4d3f35508e3eb6ca8f3724c9bc 35985 F20101107_AABIZV wei_y_Page_043.QC.jpg 00fdf77f869d78160c047b999de71db6 bbaf4735a5e104cac2a627e4a16286ee8ec3cadd F20101107_AABICN wei_y_Page_110.tif abcaa9c92e216e599de1476accfb47b3 6555d5b1403e74a16bf41d0fc07bc8b695d2efa5 722864 F20101107_AABHXH wei_y_Page_167.jp2 b565ef0eb6ca9f0d8c6dbc0984dd5aef a01f13c8ca288f9e23f8d4be31f6aaac3ec04dcf F20101107_AABIBZ wei_y_Page_096.tif 6d055cfebc16a0cb5a3123429d79d169 5327d53ca9547270a0347247f3916dca285c625c 710612 F20101107_AABHWT wei_y_Page_153.jp2 0c2c80b6878d7f698ca55d178049e26d beff1c0103be5c178fcc064eab404849ee7ce779 F20101107_AABIDC wei_y_Page_125.tif 2cb81974a1cf2e6bced5b14abd361090 1c6ff6881b413b383e57dfd130963c12bc0c5dbe 10134 F20101107_AABJGF wei_y_Page_140thm.jpg 62baf35042c7a9e9d3bf00c846f037d0 9427f70e5297277dabb35b2452c9e96a0f5a6d81 8150 F20101107_AABJFQ wei_y_Page_132.QC.jpg 65a234c94fa53ea44c2844f8f4afb4d9 63f8385dfe0da1cef801a6a2370a5426a91d1e68 8963 F20101107_AABIZW wei_y_Page_043thm.jpg b779fca26bcf2723a788a696ef1de569 b2ca698082db6412fd1e5ac430433bdf7cc55c52 F20101107_AABICO wei_y_Page_111.tif b0e976bc4c9f0ae90d8fdfe53df4f98d 19d04ba8d7ebaed3788aad06de228802f7b34ce1 724204 F20101107_AABHXI wei_y_Page_168.jp2 919286e42e2823d29a237ed3d95ffaf7 17448fefd968c936848f4f7a6b6bb069f03f48f2 703931 F20101107_AABHWU wei_y_Page_154.jp2 c2cadbc9d4cd53445aa47bc6d6387d27 78ec221eb229ef67d5b47ce4fe1f005e9af7e4b8 F20101107_AABIDD wei_y_Page_126.tif 0a1a8182ad25374701aa96dc50a2326f f4cc3b35e9826a01e0b0cd6549890f0ebee97dc4 32015 F20101107_AABJGG wei_y_Page_141.QC.jpg b09f68acd459f7416d687fc44ee0890c e1fcf634a3b329738aa17bb0895b76a256838332 2191 F20101107_AABJFR wei_y_Page_132thm.jpg 9c1e49fe6d15106241885b945b520735 20378470c57bb58dcdc6ad10d6896c6d6e2318bd 35987 F20101107_AABIZX wei_y_Page_044.QC.jpg 0d1ac4b3ff3fed82aea39bf1b2423526 d248aea2ce988a8beea8884dbae685926e5c27e7 F20101107_AABICP wei_y_Page_112.tif b128eeb2123b6680dee800d0f42bb623 0f9dcc5564b0faa433f6ed38ee67ca5038c124a6 723965 F20101107_AABHXJ wei_y_Page_169.jp2 a8fe4a29fb92ba92ef5f434d95824d44 732d8af866d50e04c5077a17832f377d9aa96fe1 706572 F20101107_AABHWV wei_y_Page_155.jp2 f8c1ad7cf0f35be772af6961a17b860d 891f47ec7e677e4dece43737c9945385d79d4f0f F20101107_AABIDE wei_y_Page_127.tif 914149fc7fd7ba67c0866a19c5aaea4c 7a7159f36ae74a7c653d1e37b5b688330b512f90 30792 F20101107_AABJGH wei_y_Page_142.QC.jpg d234a6f696926629ac270ca41956ff95 1aee9852cf6b25cba1f555db3ab8feb6e3af8474 34097 F20101107_AABJFS wei_y_Page_133.QC.jpg 28cc18165d26736a2a287e02a651f9fa d9fed5085aa3e79a68eba9bb220c18c9347de71b 9107 F20101107_AABIZY wei_y_Page_044thm.jpg 75cf80441c382a194ae87e6bdbc78c5d bcd27f70f4cc5e8b5b078f21ba79bc194c738918 F20101107_AABICQ wei_y_Page_113.tif 61fec49fc73c49126a8d001d2755dcfb e6558ed00c51dca521d714992c36ba9235346616 716300 F20101107_AABHXK wei_y_Page_170.jp2 b1c873edb3a499bf061dc2bbd171fdf2 241bec1558b49698aa898df03b333e6218685079 702792 F20101107_AABHWW wei_y_Page_156.jp2 32266d5a6a4275a284daa68ea092b861 34e258ce09fe5bc817bc84f11c5748846002f965 F20101107_AABIDF wei_y_Page_128.tif 9e16c7e56f1011586677c9f345f10f6e 20e3ef41475f8f4cc0f994c731e334091815c8bc 31265 F20101107_AABJGI wei_y_Page_143.QC.jpg 022d2043ae83cd6e5385f2365f5dbdc2 4d174fd5d3ba38f6b31c5ee465d7899c26ac56db 8439 F20101107_AABJFT wei_y_Page_133thm.jpg 1fb3bcf20fe74383adc7af43e5851bac 605f6f14f765340cb1806014696996ef05684a8a 36898 F20101107_AABIZZ wei_y_Page_045.QC.jpg ef7f827ddcfdce34a3d4c126d7986e7d 49a48b60eada2323067b424855b27fc6cbd34ea8 F20101107_AABICR wei_y_Page_114.tif 97bbf3ced9d81b49c7e22302e965dfe1 a4248587e580a82041ba5dc5a3f2720f5692f58f 717706 F20101107_AABHXL wei_y_Page_171.jp2 82df9ab50e52b9a76e0b726720366492 c12893363424412f140ebb102098d563e9211660 704970 F20101107_AABHWX wei_y_Page_157.jp2 977392a93cee5901804ab75e9ad36ce0 21ce3879c35ce62bcfd6ef9a11e4abca897d2455 F20101107_AABIDG wei_y_Page_129.tif 06ed03fefdf5aa08f2389d66d3ece93d c0e2fdb37a2e30a26e3bdc2aff41bcea61d3f602 36180 F20101107_AABJFU wei_y_Page_134.QC.jpg b85911dff5a92b509f7589724a3f475b b620db04ccf835548d2f6415c2c7aa066a776efd F20101107_AABHYA wei_y_Page_186.jp2 864ff85b30f0dfc54d2e76ad08db291b 3dddce570c4f65314d0665920e49917db646282b F20101107_AABICS wei_y_Page_115.tif 072c6f68dd8dc3782c90201badbc0eda 57849eadac66774e698aec34eb86462e422120c0 716123 F20101107_AABHXM wei_y_Page_172.jp2 6365b205e6c28f4c2fa59aa2836ee6c4 bd94448c5f942302abe309e9cc2595b0bcbaa9a0 699160 F20101107_AABHWY wei_y_Page_158.jp2 d3994221ba08ee198b629cdc9f336447 4eb9b02dc60a46b3c401b8daaf0fc98e8d674c0a 9900 F20101107_AABJGJ wei_y_Page_143thm.jpg d075ca098cef58816207f9b598b2acad a7ceabf69a2b988467d37ce80f4b3f889024f334 9079 F20101107_AABJFV wei_y_Page_134thm.jpg fd05d248ceffef07c611fdccabe4f19a b5b8f739888ed9c5f72cb85478ab38087fc44518 1051959 F20101107_AABHYB wei_y_Page_187.jp2 b6fa292cd24a81c7bbe08438ca95d94a 220553a25c9bd7f1be26f093bf240130872346da F20101107_AABICT wei_y_Page_116.tif 25ce2f24aa7062b7108db9caa6f1de71 1feb8d937bc45b5e955addd4c556922f1674a327 719914 F20101107_AABHXN wei_y_Page_173.jp2 d8c2d9f995fad36089d23473053f2b8d 5af9c68e3916d84e7a0b8944694b38b8175bf50c 702132 F20101107_AABHWZ wei_y_Page_159.jp2 8fad21e597787593ced26b6986e74106 affb6f00a7b6d2a292528cde6688065233674ad0 F20101107_AABIDH wei_y_Page_130.tif 28c671b32c5e122123f0e2f388b847be 45a209d73d1ab3d019f4da6b02306958662857cb 31013 F20101107_AABJGK wei_y_Page_144.QC.jpg 493b7666710935bfc3a23785b65a0b62 c9f018dea2efa4f0e3771a7110055613f84a78e1 34516 F20101107_AABJFW wei_y_Page_135.QC.jpg effc9a1a382c7fe9afb2ffa6a2351bd5 b1e06119420c531d2e5d95ae6d1a15c0f286f005 F20101107_AABICU wei_y_Page_117.tif b6599f0b514da82e7655de75c830bac3 32122612c549cd6cfe9a6cc780ecaebe04b684fa 727309 F20101107_AABHXO wei_y_Page_174.jp2 3659d280893dac15fcf2d09876bb0e86 0aa46f1916d63fa455b89c27a125dc1b78c36729 F20101107_AABIDI wei_y_Page_131.tif 900ed8e08c57242c17c1785184cfc405 d6b268c280a4f9c25af4b5fe3e7dc3d20b0f3b02 F20101107_AABHYC wei_y_Page_188.jp2 f3a058b08af81757a293c73dfb180ce9 df25b2edf0603a202b5931056b1e1761c0d0ed62 30875 F20101107_AABJHA wei_y_Page_153.QC.jpg b48fb05e66189826b80f50a6bfa39fc8 2f00f503844b11decc3f38317771e244e89e7e23 9691 F20101107_AABJGL wei_y_Page_144thm.jpg 7b416cc42cff27a0143b5e7a8791d2f5 905c237d18b4af7a2ed291ce2b0a9b9a148d054c 8721 F20101107_AABJFX wei_y_Page_135thm.jpg b863eaf25c2bb903c2f68a6ba275f8b3 2068304d0a33e31ac9804034267ddc97531f12d9 F20101107_AABICV wei_y_Page_118.tif 05660cd95c902dc23d513d6a41568b5b 860aaec1e00d6f05dd188eeccf80ce11e6fbb447 725452 F20101107_AABHXP wei_y_Page_175.jp2 908c98321bef29d5c027263025ed29e3 dd425e933d663a633c7b8e62fdf34128e173eced F20101107_AABIDJ wei_y_Page_132.tif cb8f81f483bfd93924b94f61cc8b2f15 233a100325d1b96d610e9217592d63108ce373ef F20101107_AABHYD wei_y_Page_189.jp2 3f75c8bbc2527b5fd6ff38aa18322e87 9df07806d431a6f7c2f3c39d6d5560fecb2e4d9f 9717 F20101107_AABJHB wei_y_Page_153thm.jpg 0668043c4e89f7ec285e3c309b04646a 5087b4d4ba4bd04d0792d114f3e4f3edf0da5c8c 31174 F20101107_AABJGM wei_y_Page_145.QC.jpg af8e5e6496d5e52e7c5a16b5c7b4a4db 843b1fe87279de9e00da2928620859c667dde4d0 36435 F20101107_AABJFY wei_y_Page_136.QC.jpg 237557e12bf80f58b29746db3a553705 863fc9758c0214c2d85939b09e74b6e82b43aa7d F20101107_AABICW wei_y_Page_119.tif c9bbf7e38b8d8afbbd5cb587d7d5e98e c89664eefb17da79565b0e658f03e01052144542 728744 F20101107_AABHXQ wei_y_Page_176.jp2 955b9c36fead9223746ff5e9a7b94e08 64cbdd710fa77577ffa2b1ebc33e5e486c905e27 F20101107_AABIDK wei_y_Page_133.tif fc18efe7fe425dbb352ac11ad07f0d0e 36d4ae0c02390b195c0767ab246f5112ba7db402 F20101107_AABHYE wei_y_Page_190.jp2 313545e843199a602bc46f5ffb81684a d46343d776199376f8df688a8d2c305a64ca91cd 30441 F20101107_AABJHC wei_y_Page_154.QC.jpg db7fa15e23f6df563983e4705c74c2ec c6b74a7a4d558e66dc8f8ebdefcfb2fddce4147f 9750 F20101107_AABJGN wei_y_Page_145thm.jpg c5900f8b9a6b3f5ff16a0530758c652e 834b75725958b6b9c94a77563f35f650b982002e F20101107_AABJFZ wei_y_Page_136thm.jpg dfbe2564a52b9230a0f78071ff64e71b 3d1905f150e7696ba846b5f00322b669c3a4430c F20101107_AABICX wei_y_Page_120.tif 141c630ea055edf5d243a6017e9c005d cced2b614662300eef8183282c63916163017645 722346 F20101107_AABHXR wei_y_Page_177.jp2 a483bcc16d7a6d675f506861a400e2b2 55144d65f208f56cc51a2f627d26637e373b8d38 F20101107_AABIEA wei_y_Page_149.tif f8ae5d872a6e5709d6ffdefaf0cd254d cea5808857688238959608de75ed061a81009902 F20101107_AABIDL wei_y_Page_134.tif 72ae300736d90b4ba3cfde688fac1f8a 37ba800d88e74f98ddc828e461a552470688e827 F20101107_AABHYF wei_y_Page_191.jp2 8a52262845a35de9508720f53a068a81 4a257947e78f7ac410c51e296c37973514801c0a 9653 F20101107_AABJHD wei_y_Page_154thm.jpg 22c64cac2b586cc013288d69db78b7aa efb6f4edf78cab7ba624c2e7800c4f7bac162a71 30418 F20101107_AABJGO wei_y_Page_146.QC.jpg d5e524c3ad17cb694250ce2e4373a044 b3a59aa663ca08e5223ac2cef3196fec999aa937 F20101107_AABICY wei_y_Page_121.tif 38587ea47178b63bf2519511c8e8a6ea 6541394deca91faaa5e7dc9ad61f15a4a94cdb86 719580 F20101107_AABHXS wei_y_Page_178.jp2 a458bc0bd03739756a85c810ba0ece5c 61b2cd985546fd73d294e86a2aad2c45ed256c6d F20101107_AABIEB wei_y_Page_150.tif f0853b22a71a47fb3ec341fafccd8115 a43a9b6140d7178066167c009c3c73958c9b893e F20101107_AABIDM wei_y_Page_135.tif 46455ca00c4bcc550ae79bc512bcd135 71b7fdabd730baefa0813730ed8ef9e708df21c8 802897 F20101107_AABHYG wei_y_Page_192.jp2 9d4b7026331eb00f5552fc98236b0947 9fffd87bba67dfbd897d11fc6e23b9cde3b0bc50 30672 F20101107_AABJHE wei_y_Page_155.QC.jpg 2a031242939f8d66eb6e46b9c3f11555 6618950a4dc5c33d477f55a8b175f7b2d3e26f05 30563 F20101107_AABJGP wei_y_Page_147.QC.jpg e6d197673eedb5672cbb36532326b13f 3e868f501e0e6a5ef846ed60dfbaa3b4dcaddc03 F20101107_AABICZ wei_y_Page_122.tif 23fc6a56d0d928f74cf40ffff09bd0fd dad832c56d35addc4ffa8bcef394b059ed324eb7 719028 F20101107_AABHXT wei_y_Page_179.jp2 01f72c773b80975c0bf3105eee94657f 49f50e58e207928ecd85f8c09717e720d7f7d4d1 F20101107_AABIEC wei_y_Page_151.tif 3cf58cfd986048f9a020776289157ba9 7478adf491709520268ca867bf88c2e62a9c9792 F20101107_AABIDN wei_y_Page_136.tif c3b421e58bf2ca14ff917de78866ac6e 79ce01e9458f790b43344c81e3252cccaa62b035 448385 F20101107_AABHYH wei_y_Page_193.jp2 bc40227a8e37863ae53c5f7ca576f030 d934ceedaafe94a0ec46db908a25e5557332f037 30630 F20101107_AABJHF wei_y_Page_156.QC.jpg 2ed49ba9109cc9a2e3c0241838f3fb32 5dcd8005c07a9fc07fd18b125b22e0223c9c9f19 9621 F20101107_AABJGQ wei_y_Page_147thm.jpg 79316525463cb2409cd53f8f5f959cb0 b24aaaf83acfe324b5334313e1905c384113e1b6 715686 F20101107_AABHXU wei_y_Page_180.jp2 db2bd01eb36fa56b684cc4bc59e18467 267fee197207779237b585dfe4a16a0109b4ec41 F20101107_AABIED wei_y_Page_152.tif 80caad96f7ab6ff16f7791e48ad0f3de c6033dfb628fcd288e898e6119861d1b8c41f7d6 F20101107_AABIDO wei_y_Page_137.tif 7b679168edf063f810e2d5cf112e4af9 ec8175874c09eb1501708db86f70442fb99abe07 F20101107_AABHYI wei_y_Page_001.tif ba83cb86c4049c7b5c79607664b83d41 991afef56a7cf17a7f5b75be793e15359ae2a61c 30347 F20101107_AABJHG wei_y_Page_158.QC.jpg 38d8d04feadab768502a7ca87ac75bd4 a8aa51aaa3924d39f9d94542dc579fbf1529a62d 30710 F20101107_AABJGR wei_y_Page_148.QC.jpg d4787f5763cfb1e6aa020f74d0206c2a 775bde66efa6f7fed2bdcbd92c56b5455883cdbd 713996 F20101107_AABHXV wei_y_Page_181.jp2 bd92315f4d41a124ba1b925aefc866db 1f89f79d58092e8859023a192e211115ac1ba0e0 F20101107_AABIEE wei_y_Page_153.tif d5f0e97abfb0f381a65f5b00802764dc 14907e50dfd31d78bca9720664ec8b4c433a9f66 F20101107_AABIDP wei_y_Page_138.tif c0dc00182b7eaec95c99684846c00c7b 369767fffdb11afab7769e5e4ca72bb3dc8fe526 F20101107_AABHYJ wei_y_Page_002.tif d4fb526729bb095520ab4247172f6724 723b1ef3df98bbd4b561b4beaeab6a27d0579d9c 9601 F20101107_AABJHH wei_y_Page_158thm.jpg 46b46ab6dadefff4003a6cc09b226e17 7d05482110b2ebc01a1e8ebef2268c5f27ebc7bf 9648 F20101107_AABJGS wei_y_Page_148thm.jpg 6df1ceba5803579bf71c8005841b5b63 e941cf3282d7c89ede0ffc359d846eea1a1f66f8 711416 F20101107_AABHXW wei_y_Page_182.jp2 25b73e06b89361b40e56e0e507d96bb4 3ef51d747096fdac9ecbd11bcd06a60b7835722e F20101107_AABIEF wei_y_Page_154.tif 93da63aeabeb69942a685fc02cdb989e 780f8c1e3f270b8292429618eda3a33bf028b787 F20101107_AABIDQ wei_y_Page_139.tif 53394f839dae999e8b154a856e2e2f97 46ad4f5c8d879f9e94c20e1aa79c0206edd2693b F20101107_AABHYK wei_y_Page_003.tif 53e223e9a2322ce3daf59835fac330f5 f3a65b715df8a1e9911570970556bba3a4b97dd3 30531 F20101107_AABJHI wei_y_Page_159.QC.jpg 8d7c9777a4673f7ba8fbd1ee64889c5a e6f33c9e56914f3941b8626cdf94f98729c86599 30770 F20101107_AABJGT wei_y_Page_149.QC.jpg c485c3c72252aac4f6d77d20f15e757a 3f0401ec40960f13e7713cf3843df7b954c9668b 715561 F20101107_AABHXX wei_y_Page_183.jp2 a042733405399c9afed20275876804ba c2912581b02c4d53cb70786a90f53d496b7e146e F20101107_AABIEG wei_y_Page_155.tif ffc0a7d3b32d03548125f2199083b9c4 0a8d5ef8af31bdad281350bc6f7b32fc72ddf29e F20101107_AABIDR wei_y_Page_140.tif 5ea4d5c58b4357f8fa655f5f2f5509c3 cbe327003664dbb03d711272900a0da2506313c3 F20101107_AABHYL wei_y_Page_004.tif 234c5001d6658b1dfcdca9c039045b6f ce0859532b2a0540eaf4a477e35aa15b136dc9bf 9662 F20101107_AABJHJ wei_y_Page_159thm.jpg 505a66262588f204ee4fc40311d86d5e 1d3772c1c5a989ea03b91e1131e37ffa15e2b7b5 F20101107_AABJGU wei_y_Page_149thm.jpg ddcce38ca21360b44810a81313940874 074a89d12d383b4949cdbcd3b85ba3b9e2ac1306 680650 F20101107_AABHXY wei_y_Page_184.jp2 5840dde53dbbd4e9af2a983b1322dc74 04c45291f4de87f38107824b63b1465897af0e56 F20101107_AABIEH wei_y_Page_156.tif 3442c82d7be4741fed6c9c8d61012253 a29febafea3f9162c3443c9c343cb6e7863640f6 F20101107_AABHZA wei_y_Page_019.tif c470d20f8ef115e3f0c08e021a9cf5ac 9a038677a9de7a0a1a529aa19112c189a22ea32e F20101107_AABIDS wei_y_Page_141.tif 8bf1d123c4110a0b8c1b38fcdfef6be6 9329da16fbcb885106d99389fdf3a44b9d394034 F20101107_AABHYM wei_y_Page_005.tif c540e781f94f4734980d36879a5b5f1b 7228ebd17ae5f42e401cc508d88180bd051de580 30669 F20101107_AABJGV wei_y_Page_150.QC.jpg 496bcfc40ee692a21d2a27d16df6a0cd b0194cfe03be907e735976666eeee02bacdeb87b 715355 F20101107_AABHXZ wei_y_Page_185.jp2 55ad394bd2c051f56728143d44518dfe 819486164a43bccffe5bddd7b2f9b62c800387c4 F20101107_AABHZB wei_y_Page_020.tif 7b9bb94d1f9216472e2708314154e19a d012d7149c4b8a10ca7df90904f47f857678c0f2 F20101107_AABIDT wei_y_Page_142.tif 897e4c22f48b2cef60f829516fe7e3e5 e14da3fe39bf899eb6429ff94c00fad7d2cf8a0c F20101107_AABHYN wei_y_Page_006.tif b653c77c2670f46fd54691c8adbf42e1 7a2a2cc07cf40afc10f9c69293de34457b819c8f 30402 F20101107_AABJHK wei_y_Page_160.QC.jpg 2c2df5f1233ee5f39c22e3af5d7e6733 5bc2a82c0c937d0997a695335f83ec7b67b6687f 9631 F20101107_AABJGW wei_y_Page_150thm.jpg 2f6993aec2fdb01a8740919d6c183068 83ce0f56b052270484cbe6f8b342e1adb341b640 F20101107_AABIEI wei_y_Page_157.tif 9aad48f07e035612facf1863aeee55c6 a986d458c5dadbf38f3a29ed4659d1f775788d16 F20101107_AABHZC wei_y_Page_021.tif a46d1d3487e0aeb945f73688199092e6 360a468c0f82963a85f78700c6464747176bc6e4 F20101107_AABIDU wei_y_Page_143.tif ec992bf7eb00ba1a9c0786c8128bfc5a baa20cef10716355df2b64765e9264266a4fc8cd F20101107_AABHYO wei_y_Page_007.tif bcf13c29fb79474adba66eedc8ef5bc3 bca7f66249929ef4e8792181452321abdd816585 31702 F20101107_AABJIA wei_y_Page_169.QC.jpg 19cc8c1bc132e177f35a93516f730ac4 f30725c55158c5fa6a0b20073d844f05c771287e 9577 F20101107_AABJHL wei_y_Page_160thm.jpg b00930ee6af4b3eb7994259e81b5a960 ef147909d1b3fe02c73ba81f002ced7929b87e9c 30889 F20101107_AABJGX wei_y_Page_151.QC.jpg c8b33f0ca99989ccb012f6b27336ea2d b5945634774a63db78cb7a92f58444b21964843e F20101107_AABIEJ wei_y_Page_158.tif bdad0d06758aa7fe4ee4c12caf329d0c 1765a302b1cddf41deba07b35ec46c48b20bba14 F20101107_AABHZD wei_y_Page_022.tif d2fcfb85a6b1edeccae543076cde004c 54fd502d04a236167bc148f852b6c1c1af35953d F20101107_AABIDV wei_y_Page_144.tif d64f291aa1531342e1fedef59c8bd386 d34dd2e342d6fa90a5ae91bbc03b3ee33db76c23 F20101107_AABHYP wei_y_Page_008.tif cbebfb00e103e7f1181f8dc3216a26de 036e00791e0e8162ee375f3a8302122059d94c0d 10022 F20101107_AABJIB wei_y_Page_169thm.jpg 4e438d2e9f5dd2cb34cef918ba53c007 46688aa8ed450478749f69808b8957c4ef56b75a 30282 F20101107_AABJHM wei_y_Page_161.QC.jpg 7f1956ff234d750987a8babf33eb539a 41323a7041003e0edfe219ef1b1445d362ff5cb5 30790 F20101107_AABJGY wei_y_Page_152.QC.jpg 77d2afb01afe0b018964c7937e7e8ab2 1ad2871e0cdd42de4ae16b7e60222185a3074248 F20101107_AABIEK wei_y_Page_159.tif 7193ad942f6e2405123aaf13bc706815 c7687532b2b0006d969de933c26d9e6be55ecf84 F20101107_AABHZE wei_y_Page_023.tif 00edd5686b7099a36eee4064d1abd73b 0bf30480d441e0258bb81ded9ac22a1730f97b5f F20101107_AABIDW wei_y_Page_145.tif 832e234e7cc408c5161d8b9ab30fa59e d022156a2227dd850683471fbbf0847c0da0114c F20101107_AABHYQ wei_y_Page_009.tif 6b1c8d453b3165a28ff915c7bb50f4f7 f4f84d3c51bcff61bf1549ac5e65106acc4ed861 31141 F20101107_AABJIC wei_y_Page_170.QC.jpg a63bb94e1f7dceb65790d4da834d2d0a 64f63b9f90433477fdde0e37bda1ec59b707c028 9602 F20101107_AABJHN wei_y_Page_161thm.jpg c223b2aaa4045c6fb78696bbf3ec600c c6bb6d0d7faeca1ab80e4619aabcceb72166b638 9681 F20101107_AABJGZ wei_y_Page_152thm.jpg 11d83a6e6e46461c93a325e8fc4514cf 7e38bc69a10a8175615f392b00b1680f2320123f F20101107_AABIEL wei_y_Page_160.tif f1025350a641835f8d82e29eb05e241d acdb635c1808941300f5c9413329fe8558ab1f7b F20101107_AABHZF wei_y_Page_024.tif 75636aec281217dd361cf4dac2b5ef6a f74a88a6a59ddc158f5b11f720fbe117f41b91b0 F20101107_AABIDX wei_y_Page_146.tif 0a4001572a92129d71ad4dbfa6c74336 d0126e0d84d85d9fed83f6ac279f8e7035c147fb F20101107_AABHYR wei_y_Page_010.tif 3be02b5911b7989a3808a3de468d92f4 847a31404c928995b1c818796c52f19cd8aaae23 F20101107_AABIFA wei_y_Page_175.tif c2e12f11afdf6334c9bc7f165e627192 cae77a429248e0b28162ed9c436645093654cf74 9881 F20101107_AABJID wei_y_Page_170thm.jpg da316b049bcacd261d566cd47984b8cb 722e79870db2814e456e0d733fa8f20a4b165ffe 32226 F20101107_AABJHO wei_y_Page_162.QC.jpg bad2cab0aa009442e35a8fd7f1d13fe9 ef3c83c88bbcd8a055ebc81b0a44fb2348a67e9b F20101107_AABIEM wei_y_Page_161.tif 5090a972b48bb69d183451aafa48bbea ca96b67d9a6a00cb41a2fbb965375e41a4272778 F20101107_AABHZG wei_y_Page_025.tif 24ce5af71a9a4ab95999e378b22823e0 68e28c69f3f6f815f0eadb1b86c49edb39d73e3f F20101107_AABIDY wei_y_Page_147.tif 0912c0a81801d280cd8c1a16c2200757 5fa12d8f293b8cf66cb1abef8d5aa7f6463e146a F20101107_AABHYS wei_y_Page_011.tif 72a323f49b9d66404802d2177ca4b269 4292ce2cd17f8fedddc057ced2b7f5c3cbe85af2 F20101107_AABIFB wei_y_Page_176.tif 0f377028267eb87adfcc5b2876f15c4d 50e496eadca6890e48104eda564bc431cbe5cb3a 31312 F20101107_AABJIE wei_y_Page_171.QC.jpg 46792c92f0efcb1cbcb7753d3c83f232 a3ccc97f18d35459564ca1852f13d8cc07e3d0e1 10188 F20101107_AABJHP wei_y_Page_162thm.jpg 35647c9bd60733eeb67a5544adacab7a dacaf5f4feb5bcb249a534ab63838a69bcacacd4 F20101107_AABIEN wei_y_Page_162.tif 7e25087b7d37c3cca0c14c8857ddc087 e5604e619de955b9e9142f3c543e73c6b3cb7a9e F20101107_AABHZH wei_y_Page_026.tif 662acfa706eb608d7004aad198f30a7b 1e510296feac98adb4bdad2751fe4ca4fa84ba0c F20101107_AABIDZ wei_y_Page_148.tif 66e13b42ab8e0b8d93124d7aee15c1df 6b81b2a8e359bd31872f15e2e7cc012edf83e953 F20101107_AABHYT wei_y_Page_012.tif ed6e566357788c7d1427c595de0d9605 c50641dbe82930bb49cafb21b3623a5a196b5719 F20101107_AABIFC wei_y_Page_177.tif a8d9cf760aa63496dcd203222671d073 2401a38e6d8271408efa668543fb1d2cce1f6f59 9924 F20101107_AABJIF wei_y_Page_171thm.jpg ba88c156b7e4c74ca8226a26301cf534 3a8cce32ba28ec664e1c5c0efcccff4a23cd5c26 32218 F20101107_AABJHQ wei_y_Page_163.QC.jpg d789eb4684fc9170176439de256941c6 0c74d2dd56bd1dd78f66e28482b3b53173988072 F20101107_AABIEO wei_y_Page_163.tif 32a69f9db1e463347287348c2ecf4b9d 8e43c32f1fac882e63916dfa091a1ad00d3b410d F20101107_AABHZI wei_y_Page_027.tif 0f959db3d06797b6c63f47c28bc9705a 0adbdd7a8b91623c07b954cf077d8657bbbf4831 F20101107_AABHYU wei_y_Page_013.tif 696a7df1efa25261d7d9f03be31cac32 c44663ee601a7d3f0300093bd2c0d5240a246fda F20101107_AABIFD wei_y_Page_178.tif d81c4f084c5678dcbb0d23b0e52270f1 6e5a5bd42f1552f20ebe4c7cf53dbfabbac67113 F20101107_AABJIG wei_y_Page_172.QC.jpg 8c24db9b3eff553898f64cd01ac14d77 9ebde612103e789c1b4d91fd7597725afd711258 32375 F20101107_AABJHR wei_y_Page_164.QC.jpg d58fd95b9b69c3edc56b29c1924ed1ba 21e672b41e0fbf2e42f2ee7818bdbdba093ff353 F20101107_AABIEP wei_y_Page_164.tif 47cdd8a00f1ea8eaea6de4cd84161b6c 1e2fa90ad12de63671631c6dfbd2cea8c6314086 F20101107_AABHZJ wei_y_Page_028.tif 80c26ef270b57dfc8f76313b499dad21 24ee6fd3cc3b1589fd9f54de566961fa5008e3a9 F20101107_AABHYV wei_y_Page_014.tif bcddcdf36a618de403d591e212c6f138 99153c4522b7b53a245e0856c4919e3f82340513 F20101107_AABIFE wei_y_Page_179.tif 6c918f0520d373112864e242fce5b21b c81f2a7e256b38284e8c085befa85600eeae5d69 9855 F20101107_AABJIH wei_y_Page_172thm.jpg 4366270bba0fc3e378abd1c4764f9fcc 24f281768e7fe6cc6616f48cfaf122862efe8689 10186 F20101107_AABJHS wei_y_Page_164thm.jpg 8a034c967bc8d60d7b055763a7796ae7 171dc4e67872445db278b4408fc0b953d265e346 F20101107_AABIEQ wei_y_Page_165.tif 5a7062289e7e834a8e83642a47057b82 4da5791c249704c99d4cbadddf8802685d162e76 F20101107_AABHZK wei_y_Page_029.tif 51063553ab4a6f5d78dc88a744c932b7 2a9c5325a0f5f78055c1d6d86d7107cc62e5ed08 F20101107_AABHYW wei_y_Page_015.tif 3f8fd0f8e5b364b2d408f3f386875577 27c25056c8ed5720ea8b15e5ac6f727a50b90fa6 F20101107_AABIFF wei_y_Page_180.tif 83c967fda61776aa37bbca495782eef4 52f29a77393f14633e662bcadfde56b2ede54125 31264 F20101107_AABJII wei_y_Page_173.QC.jpg bed9b96a4d8031310b3615616f3279ae 5525abcdd085fb6b6c7a0532d16f1b5832c8f3d4 32144 F20101107_AABJHT wei_y_Page_165.QC.jpg 86acaa515499dd8dee28a0f3c4eae8c1 4cdcbe4aee71c2bbbd29eac7021b6c801a0f0f5b F20101107_AABIER wei_y_Page_166.tif cdcc0fce905303c98f1a34919c8cdb5b 181250e30c4231e0b0a1d1a47f083ab90d396dec F20101107_AABHZL wei_y_Page_030.tif 9f2b6621a2b83e5f67dc232488e2f1f5 f67d532541f2e7bbdf9c740051502b0a0d15da8b F20101107_AABHYX wei_y_Page_016.tif ac81752d22b27eaaff84f01c59aed390 9d8212610203c12add7a39733d24b3cef02cb05c F20101107_AABIFG wei_y_Page_181.tif 23bf846aa88c4dca64072fbaeebe11c5 50196e5f5b4d2de7d6f762bcdb5d6a5ce637bdef 32082 F20101107_AABJIJ wei_y_Page_174.QC.jpg 1dd4266f5c39a8b6c0b9dbd75ecd1385 86347c5ae23538564e68ebfebe93d133e653ed4d 10108 F20101107_AABJHU wei_y_Page_165thm.jpg 6ec0a3e62469351df49c23f94c008a92 8000bb89c6136fe9c4397cfd1b93bbe297be9d1d F20101107_AABIES wei_y_Page_167.tif bb9edc031cc90a674dc313b744939d8b c2a7b52ab8b1c17891505322452a1fc5c60ef4de F20101107_AABHZM wei_y_Page_031.tif f7bac70f05095275f7b78791c069725a 05d584fb9288e96a2a29bcc4fda703b5ef133f28 F20101107_AABHYY wei_y_Page_017.tif a29d33ce1be37d6deb9ee1ab9a2c94b7 5c73beddcca8cd52a1c0e4c0e42f41b437a2fe02 F20101107_AABIFH wei_y_Page_182.tif 8adb492d132f0a9a6ca791f093a72e7d f56a2d6107fad06789d77a908e38e572ef60d96b 32281 F20101107_AABJIK wei_y_Page_175.QC.jpg 65b6c0753ac455c4f76151feb758ed23 369a94fd89884d726c9abed292452e7b49cf28b7 31443 F20101107_AABJHV wei_y_Page_166.QC.jpg 54f13e032ec305b15a8b9cd4607a6de5 f886e25f9c5bbb991ee0d3129559a6d05278a0b4 F20101107_AABIET wei_y_Page_168.tif 3a2bbf77f060547a1a449d2817bf83e4 69b24b49f44702a3f1cdcb613acb7dd6617c5bfe F20101107_AABHZN wei_y_Page_032.tif f5c33c39ed2dc401707fa946e106e90f 67af8c503279b72afb7d973547216ba0e29b3389 F20101107_AABHYZ wei_y_Page_018.tif d0946cf2f8723a1bc7db4ea2e37faa89 96b0ed9deb1fe74da5d33c428ad2e0e840fe782c F20101107_AABIFI wei_y_Page_183.tif 13b5e833b2859a59280727218ac7a13b 774c2088c3f0fe15e79c4edf3bc022c3fe554fc3 10027 F20101107_AABJHW wei_y_Page_166thm.jpg 052dc3b9b9bff2ab47fc3618667aab58 d629658e317d9fa3cbc9d7be09f7750eff0fa221 F20101107_AABIEU wei_y_Page_169.tif 337e1f442d9aa53e5bb8bb19246c8a2b d695f88ed390ac0fd9ba1bc3598704274906d3b8 F20101107_AABHZO wei_y_Page_033.tif ecc295f177cb1f47d9126f36071e0fa6 e2de9fbdb7d9d32f0a5737dbfb1f8acddde0df4a 35611 F20101107_AABJJA wei_y_Page_186.QC.jpg cffdbd5628f380f1a21d47fbd732d875 66d7589eacc7911a92e5be39e022244eb1f820c8 31962 F20101107_AABJIL wei_y_Page_176.QC.jpg 56525698613c8e9adb6ac5217f921f0b 260262d72c1a8eb7b28ad6699a9d7ed3d9e33c89 31560 F20101107_AABJHX wei_y_Page_167.QC.jpg 13ecfcdbd50b403693591ffb66369eb5 6431e2f21ad2247cea9ae477fdad4c5e9921e79e F20101107_AABIEV wei_y_Page_170.tif b7699b314a16f66e3ed58083d8613b3a 9357ee065e5f10c1ab28d30d61d70970546aad35 F20101107_AABHZP wei_y_Page_034.tif d0635d2f32de75d88a31c5e58aa5117a 89c5d1ea9c2ba54f97420f0001f8f09422723520 F20101107_AABIFJ wei_y_Page_184.tif 622f16a0ebd7168382de653c2481dace 08e40169874debbb579a3944ee8f14c18cf21d30 8883 F20101107_AABJJB wei_y_Page_186thm.jpg f74f5fd33f8e2a95ee74d3d1bd227518 7ddf57746a5cf7675c0812ccc283f60a31194bed 10065 F20101107_AABJIM wei_y_Page_176thm.jpg 815f15fd34fa32b2098e87300e67246a f863fdc686c441a53248bda0dd4b8b2a2abe36cc 31675 F20101107_AABJHY wei_y_Page_168.QC.jpg 8b0e22b9254f6c0f9a8c7eeaa46dda30 82b4eb0ec9bd4ae8487aab36af0fb1a1f2490b1e F20101107_AABIEW wei_y_Page_171.tif d077e8db3613574dd2c6c6d632afd130 d44d236b5f07b7d4434127801cfa64c04a8ada72 F20101107_AABHZQ wei_y_Page_035.tif 27b18919cf3ee06beb391ba26fdb33d7 63b21f87c34196f3f4a1b3d067c5539adfeeb5c7 F20101107_AABIFK wei_y_Page_185.tif d6f9f217ee525f54e3ef2401237010f5 9d4c2f9d48689e6c23f840e4476b983a9f75588a 37523 F20101107_AABJJC wei_y_Page_187.QC.jpg 397f6e03c5fc8919e49dd8b6de44b505 6a2cc1cc21169d696341fbb53915912c62d92287 F20101107_AABJIN wei_y_Page_177.QC.jpg f578bb8a19a07e64c3c76adacf98a1bf 5dd684f3ee760c44f702cae005239a29799a61d6 9989 F20101107_AABJHZ wei_y_Page_168thm.jpg 90df07ddcbe75f2b48f0bf75734733b1 5c981f3d419595def719a141da43c63d998f3816 F20101107_AABIEX wei_y_Page_172.tif 8d7ac910378d95320c77feea66166d97 22700ba31b6bd7f452e7cf6d8d1116ec13f5721a F20101107_AABHZR wei_y_Page_036.tif 3b00ebfbed945a6f1463c7722960aef2 e91d0f771566d5a763daaed54de52c76c93b01d2 2690 F20101107_AABIGA wei_y_Page_008.pro a73dee609c9f60ab64bd440b61f0f011 50f9c5685ed948dbda685e6e397b2338b51f142f F20101107_AABIFL wei_y_Page_186.tif aa78910368de91ae81b6188ee1c6082d 36f8b59026f6ff5355328516d7c15001a771a28c 9026 F20101107_AABJJD wei_y_Page_187thm.jpg 526fef2f95bb8db836caef3027da511f c248af19a83cf7abc1c2ef710aa6b30d1bf2ace2 31409 F20101107_AABJIO wei_y_Page_178.QC.jpg 55ec8fef93c4ab30b1a2cac83271a0b7 78a42066a7b2b5f9553778ede5811e2af79098f9 F20101107_AABIEY wei_y_Page_173.tif b00236e901002be380ee0ec78d0b3594 2298e6be615655e5d02fae3f436ae1b2d4f59736 F20101107_AABHZS wei_y_Page_037.tif 75a98eed3fbf797a853e603e418981eb e5b187b9d3e72efedd04f5cc4be302ffeafd54e6 43257 F20101107_AABIGB wei_y_Page_009.pro 4dfb8423c0b487d762f5b3ce2c38dd53 b16f102342d67f8f8d9976e1a9a2b353fb599a93 F20101107_AABIFM wei_y_Page_187.tif 71845b36a26fb45bbb78e87e3e0ad955 3355af75bb9d507adf71998e1a5862c6c2af5fc1 35699 F20101107_AABJJE wei_y_Page_188.QC.jpg 7eb73130ff2067cfeb3646d0b40b88b0 e63474902beff905609968a2b54059b17f51b220 31530 F20101107_AABJIP wei_y_Page_179.QC.jpg f8f4c8a330187ddd1ed7c5c62a65bc02 7293e79deff15cf46a4d12e2df03d522d4c23299 F20101107_AABIEZ wei_y_Page_174.tif fad1c3364f33095df397d043ee4a531d 3c507eb5c6d54c5c898bcd830ed36ad6c9830843 F20101107_AABHZT wei_y_Page_038.tif efa4faf07d8c3c6a97ee78815a7bb9dc 803d067f0a3ec7dbf629642c7fe997b08015e912 F20101107_AABIGC wei_y_Page_010.pro 25edbfea16ccf347934f81b58e11281d 963ae7c3b36c11bfb29f59e3bb2d1dbbf17fe4ed F20101107_AABIFN wei_y_Page_188.tif fab8d4836cd3e5cea09014868c075431 b3b0684a8ca8ce4aa7ba50919d0935edfddad45e 8893 F20101107_AABJJF wei_y_Page_188thm.jpg 5624fcf1a88f82a7865eaad39f3b6496 81f4dd319d7fb934ab77b8ce1d6720992a721f58 10272 F20101107_AABJIQ wei_y_Page_179thm.jpg 473a07daf58d5d8712762f95e0539361 b620160cbae945c527cebf30036dc2abae0a3b47 F20101107_AABHZU wei_y_Page_039.tif 3fcd96aa13e9df39a9fed22edfcea6e2 595b994538e85a3c15660d505567bc3faab95305 53042 F20101107_AABIGD wei_y_Page_011.pro 249df77759883dee466f1df6b14337f2 1d16fc54e84ce92c0334d94594d8e9fbe9f97dd0 F20101107_AABIFO wei_y_Page_189.tif e93229d687d92556276caaf956cd98e7 22821d1c7a3dd893757d3661d5dcfe888aba9fe1 9134 F20101107_AABJJG wei_y_Page_189thm.jpg 11a975a7b2d9922de5fb3249e2594ed5 21d7f620b4953734b3a5e2189d82b2fb274e629a 31244 F20101107_AABJIR wei_y_Page_180.QC.jpg ae300879ed6bcfac1dd307087f6bc840 4f62d7c50f80ea2d5be6d258e3ee356e8f2e8db7 F20101107_AABHZV wei_y_Page_040.tif 2267b2c2ccd98fe302cc91b7ec99179b f09352b5ab732f9e6730e786fa60510e5f5758a5 51343 F20101107_AABIGE wei_y_Page_012.pro 8ffd36b4610d46574889473f27039a05 eb8a7e7a324acce4826700e99c9f22799d9da469 F20101107_AABIFP wei_y_Page_190.tif e15db0fe27e4e9d6807f451b38a80c6a 864c8c085f336a38adbd874138a5eb8e909bc0ef 34785 F20101107_AABJJH wei_y_Page_190.QC.jpg 6317a7c2466a8aad4bbb6eecb9cbbd1b 334da345d1d11827e92e0e2ff4979f05cade04cb 30833 F20101107_AABJIS wei_y_Page_181.QC.jpg 18e3c23a0b21f9029199d8d86e6f7227 785c0bfde3bc991e287b49f00b22eeb8266dbd41 F20101107_AABHZW wei_y_Page_041.tif d09f1310131bef915e10935d3b717986 c95a365aa7780790307974bed1733b28b189e66e 30827 F20101107_AABIGF wei_y_Page_013.pro 577500bf690fab48c345ed77e8e6fe81 1b30d02378a1459b7f559df36fb788618ee1eb8a F20101107_AABIFQ wei_y_Page_191.tif e13763e6ffcdc3d332c4c5336b7834f8 b33eeaaf201a7d5b1f46d3325fbba911ae376aa3 8705 F20101107_AABJJI wei_y_Page_190thm.jpg b02ab1b22db13705cdee630a993d3b94 bcad47d92db617195db6f019fdf94cfbc96508ce 9817 F20101107_AABJIT wei_y_Page_181thm.jpg 77552c3b93cf894a40b9729115287a9f c4272dcbaf58f24ae0697e646a2c9b1a2ebbf1a3 F20101107_AABHZX wei_y_Page_042.tif fc602b8040cc7d6dc64704955bedc599 b6f59dc8f76977691864399fe5442eabecc9b0fc 30317 F20101107_AABIGG wei_y_Page_014.pro c5b5b7205ae8f7523e513377872bc19e b565aef9e9fc01920e64bf7fe9a5723fc0cb4047 F20101107_AABIFR wei_y_Page_192.tif ac6f2a298c05a429667ef9fb90ac9f62 909aabf0824d6ef13c246a761cb7a527065eb01c 21183 F20101107_AABJJJ wei_y_Page_192.QC.jpg dd8f4ffca8ece0c0cc556b7aa18dec47 1d5216a7f5e69290ccfff61ee39f3f7ae83c85d8 30872 F20101107_AABJIU wei_y_Page_182.QC.jpg 49e97c36bb9a0f29d02db8e95f330112 f243e0c65eb541a1010562d8ef12f003cf3421e6 F20101107_AABHZY wei_y_Page_043.tif 5d56a26f89448bf9ed58e7cef68afcc8 bb7e0326fe2ec1d7535ca25e1441a00f6a71e5a8 29791 F20101107_AABIGH wei_y_Page_015.pro e3e90b3b263e1228625c8c2455b8d3f2 3b1d26045404697135f953e50409fa8503216933 F20101107_AABIFS wei_y_Page_193.tif 26a5456cf4d1501bd4d30007bf4b14f2 4b58f1cbe153deb371f090a6562dd21ef3982cef 5076 F20101107_AABJJK wei_y_Page_192thm.jpg 9a2cd4a92025e1fcccad427ae3acc0df 91f20fee3d981610aa5744cce5c8170ecf09bf13 9816 F20101107_AABJIV wei_y_Page_182thm.jpg 998a58ec16f7d7e7580194baae9a854e 4a9efdc0c321a0a896a60fa9ba9799f42174b933 F20101107_AABHZZ wei_y_Page_044.tif b33be9dd96257a2dac3abe7efc426c73 86cb76dc8e788e1bb371b9a6f9bf22c7c6fa1d0b 39475 F20101107_AABIGI wei_y_Page_016.pro 5b832f517f372c07779cd386e33879f0 6074885cebc5955a0707bdafde99eb0558e1f027 8344 F20101107_AABIFT wei_y_Page_001.pro c53372603dcf20097f6e31fb8d5267b9 ba6eb54d8405df9686cd60180dd4bb745336a567 13985 F20101107_AABJJL wei_y_Page_193.QC.jpg f5a88a569d506938070c0058ab90b5f0 1461a1849acb7fc89855a245bc470d3465ec3ce7 31043 F20101107_AABJIW wei_y_Page_183.QC.jpg ebdd15f2160417afdf5cec2fac9ab280 c7dd6cfdd469159cdd63d36f4a12009d49370b7b 44144 F20101107_AABIGJ wei_y_Page_017.pro b7cff22d376eac410d0bca224043ecaa 9ac0f504f34604bba50d835ce7cf0630a6c0c82a 728 F20101107_AABIFU wei_y_Page_002.pro 708311055e426923adf46266d935d3cd 94d939fc3eb1743200b2e6e1facce7bc081c950b 9932 F20101107_AABJIX wei_y_Page_183thm.jpg 14bc1f5bd71b7780e3b9274e1462508a 289072cc8343ec6e6342ea9a652e1bc4cf5873bf 13322 F20101107_AABIFV wei_y_Page_003.pro b32616f037ea6b7a253586edfde8e643 f1b2931974cff4add6bcdc2cf936b0805f61f97e 3663 F20101107_AABJJM wei_y_Page_193thm.jpg 1eff32dae778753e7f92cc9071a522bf c967e43555f97c91afeb4baa68785cff8523e4c3 29702 F20101107_AABJIY wei_y_Page_184.QC.jpg 99324a2543ccd507bd167907bfef82b8 3060e2d56bf5b05e373579e788de1410630bcf46 38286 F20101107_AABIGK wei_y_Page_018.pro 069dc853a82f57d1a2edbe3f338fc833 fb8969c6f74b3b3bbe3603b630ee595a89b2fbf7 91322 F20101107_AABIFW wei_y_Page_004.pro 367bb101bbcef9e251c50f642a91b764 9d6f4ba3b43e1862d2d367520a69aa45cf5dd4e3 31067 F20101107_AABJIZ wei_y_Page_185.QC.jpg de9fe8421cc72b6cf4bde1e72b1565d0 cf138f8e9dd318f9fec95151438ac982d2baca81 28724 F20101107_AABIHA wei_y_Page_034.pro 5667d362bd78d983209519879baed720 9c0c767ea1e0f2ee48c4685ca8b8d057a300071a 43647 F20101107_AABIGL wei_y_Page_019.pro 502a7ec300aa035fe9b70a62a97ecf3b 1ba4f9b2e0870cd56c5df926816e03b1ce5d9555 48834 F20101107_AABIFX wei_y_Page_005.pro 8bc1387199ad7f4b676c4c9923ef7f9f 8769691246255527b863969580a7d99f1e5c07a2 34036 F20101107_AABIHB wei_y_Page_035.pro 8c07853fca439d02b3224a23582ced39 788fe9d0f1d24c906938cbdbda3193dc2651556d 44285 F20101107_AABIGM wei_y_Page_020.pro 182c148ede19f7b695059ce9bacb5be6 99e7fac47478bf08d094af8806c80c81d1c4bc12 16315 F20101107_AABIFY wei_y_Page_006.pro 64aa4242ea1f95eda468a2bcf2d82042 b6add4897c5ae2d87690f0cef947b6cf28f2fbf1 38252 F20101107_AABIHC wei_y_Page_036.pro 53319618ff1057aac83ee29150bcdd8c 392d1a0beb58ec473d973c9055c68af3bda69053 24591 F20101107_AABIGN wei_y_Page_021.pro e3b64ecbda2f174bca4a483097c04e5f 3bd98f1341345d111795b2a9003c4171d731f20f 75711 F20101107_AABIFZ wei_y_Page_007.pro 017ae02ca1ad06d66d7e1ab17c528f94 8da162122e8a5dd936f72abf03d4c3d112a01d0e A SIMULATION STUDY ON THE PERFORMANCE OF FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS By YOUHUA WEI A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2008 2008 Youhua Wei ACKNOWLEDGMENTS I would like to express my sincere appreciation to Dr. James J. Algina, my supervisory committee chair, for providing valuable guidance and support. I would also like to thank other committee members, Dr. M. David Miller, Dr. Walter L. Leite, and Dr. Zhihui Fang, for their time and effort on this project. I thank my parents and my brothers and sisters for their continuous and unconditional support and encouragement. Finally, I thank my wife, Yan Zhang, for her love and support. TABLE OF CONTENTS page A C K N O W L E D G M E N T S ...............................................................................................................3 L IS T O F T A B L E S ................................................................................. 6 LIST OF FIGURES .................................. .. ..... ..... ................. .7 ABSTRAC T .......................................................................................... CHAPTER 1 INTRODUCTION ............... ................. ........... ......................... .... 11 U nidim ensional IR T M models ......................................................................... ................... 13 L ogistic M odel ................. ......... .........................................13 N orm al O give M odel ........... .. ....................... ........ ...... ........ .... 14 U nidim ensional IRT Scale Linking ......... .. .................. ........... ................................... 14 Scale Transformation........ .......... ..... ... ... .... ....... ...... ....... 14 Scale Linking .............. ........................................... 16 M ultidim ensional IR T M models ....................................................................... ..................20 Logistic M odel ................. ......... .........................................20 N orm al O give M odel ........... .... ............................. ........ .. ........ .... 23 Multidimensional IRT Scale Linking .................................................. 25 H irsch's M ethod ........................................... ........................... 25 L i's M eth o d ........................................................................... 3 0 M in's M ethod ......... ..... ..................................................................... ..3........ ........ 33 Oshim a and Colleagues' M ethod ............................................................................. 35 Purpose of the Study ........... ............................... .......... ................... 40 2 METHODOLOGY ............................. ...................... ........42 Design .................... .... ........... ...................... 42 Independent Variables or Experimental Conditions............................................ 42 Dependent Variables or Evaluation Criteria......................................... ............... 47 P ro c ed u re ......................................................... ................................... 4 9 D ata G en eration .................................................... ................ 4 9 P aram eter E stim action ............. .. ...... .......................................................... 51 Result Analysis ................................................52 3 R E SU L T S .............. ... ................................................................59 General Performance of the Different Linking Methods................................................61 Performance of Linking Methods for Different Test Structures .............. ............ ......62 Performance of Linking Methods for Different Test Lengths.................. ..... .............63 Performance of Linking Methods for Different Sample Sizes .............................................64 Performance of Linking Methods for Groups with Different Ability Distributions .............65 Performance of Linking Methods for Test Items with Different Parameter Values .............67 4 D ISC U S SIO N ............................................... .......................................... 122 R results from Previous Studies ....................................................... ............ ..................122 E effects of D different Test Structures......................................................................... ...... 124 Effects of Different Test Lengths ................................................. ........................ 126 Effects of D different Sam ple Sizes......................................... ........................ ............... 127 Effects of D different Ability D istributions..................................... ......................... ......... 128 Effects of Different Item Param eter Values .................................... .................................. 130 Perform ance of D different Linking M ethods .................................. .................................... 131 5 C O N C L U SIO N S ................. ......................................... .......... ........ .. ............... .. 133 C onclu sions.......... .............................. ...............................................133 F u tu re R research .......................................................................... 134 APPENDIX: ACCURACY AND STABILITY FOR DIFFERENT LINKING METHOD S..... 138 L IST O F R E F E R E N C E S .................................................................................. ..................... 186 B IO G R A PH IC A L SK E T C H ......................................................................... ... ..................... 193 LIST OF TABLES Table page 21 Ability distributions for exam inee groups .............................................. ............... 54 22 Item parameters for 20 items with approximate simple structure............ ...............55 23 Item parameters for 40 items with approximate simple structure............ ...............56 24 Item parameters for 20 items with complex structure ................................................. 57 25 Item parameters for 40 items with complex structure ................................................. 58 LIST OF FIGURES Figure p e 31 Accuracy and stability for different linking methods ................................................69 32 Accuracy and stability by linking method and test structure...........................................72 33 Accuracy and stability by linking method and test structure: N = 2000..........................75 34 Accuracy and stability by linking method and test length for approximate simple stru ctu re te sts ........................................................................... 7 8 35 Accuracy and stability by linking method and test length for complex structure tests: N = 5 0 0 ......................................................................................... 8 1 36 Accuracy and stability by linking method and test length for complex structure tests N = 10 0 0 : ....................................................................................... 8 4 37 Accuracy and stability by linking method and test length for complex structure tests w hen G2 w as excluded: N =1000 ....................................................................... 87 38 Accuracy and stability by linking method and test length for complex structure tests: N = 2 0 0 0 .................................................................................9 0 39 Accuracy and stability by linking method and sample size.............................................93 310 Accuracy and stability by linking method and sample size for approximate simple stru ctu re te sts ........................................................................... 9 6 311 Accuracy and stability by linking method and sample size for complex structure tests ...99 312 Accuracy and stability by linking method and group for approximate simple stru ctu re tests ............................................................................102 313 Accuracy and stability by linking method and group for complex structure tests: N = 5 0 0 .............. ...................... ........................................ ......... ...... 10 5 314 Accuracy and stability by linking method and group for complex structure tests: N = 10 0 0 .............. ..................... ....................................... ......... ..... 10 8 315 Accuracy and stability for different linking methods: COM, n=40, N=1000, G2...........111 316 Accuracy and stability for different linking methods: COM, n=20, N=1000, G2...........112 317 Accuracy and stability by linking method and group for complex structure tests: N = 2000.......... ... ....................... .............................................. ...... 113 318 Linking accuracy and stability and item parameter values: COM, n=20, N=1000, G3 ..116 319 Linking accuracy and stability and item parameter values: APP, n=40, N=2000, G4 ....119 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy A SIMULATION STUDY ON THE PERFORMANCE OF FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS By Youhua Wei August 2008 Chair: James J. Algina Major: Research and Evaluation Methodology Scale linking is the process of developing the connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. It is the prerequisite for many applications of IRT, such as test equating and differential item functioning analysis. Unidimensional scale linking methods have been studied and applied frequently over the past two decades. The development of multidimensional linking methods is at the infancy stage and more research is needed to obtain definitive results. As an extension of previous research, the purpose of this study was to use simulated data to evaluate the performance of four multidimensional IRT scale linking methods, the direct method, equated function method, test characteristic function method, and item characteristic function method, under various testing conditions, which include different test structures, test lengths, sample sizes, and ability distributions. There were one hundred and ninetytwo experimental conditions in this study and five hundred replications were conducted for each of the conditions. The linking performance evaluation was based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for the equated group across the test items. The mean and standard deviation of the differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Our results indicate that for approximate simple test structure, each of the four linking methods worked approximately equally well under all testing conditions. The results also suggest that for complex test structure: (a) The equated function method did not work well under any testing conditions, (b) the performance of other three linking methods depended on other testing conditions including sample size, test length, and ability distribution difference between groups, and (c) the direct method was the best linking procedure for most testing conditions. In addition, the study shows that the item parameter values influenced the linking performance. Under most of the testing conditions, the linking results for the discrimination parameter tended to be less accurate and less stable when the item parameter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depended on the item parameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values. CHAPTER 1 INTRODUCTION Suppose a set of test items is administered to nonequivalent groups of examinees and item response theory (IRT) is used to estimate the item parameters for each of the groups. The parameter estimates will be on different scales because the metric defined by each separate calibration is different (Stocking & Lord, 1983). Specifically, IRT parameter estimation procedures often scale the ability for each group with mean of 0 and standard deviation of 1, although the actual ability distributions of the two groups may be different (Kolen & Brennan, 2004). Therefore, to compare the parameter estimates from different IRT calibrations, they should be transformed on the same scale. Scale linking is the process of developing the connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. The objective is to establish a common metric for all sets of parameter estimates. Scale linking is an important issue in psychometrics, and many applications of IRT require that item parameter estimates from independent calibrations be expressed on the common metric, including test equating and differential item functioning (DIF) (Stocking & Lord, 1983). Based on Kolen and Brennan (2004), equating is "a statistical process that is used to adjust scores on test forms so that scores on the forms can be used interchangeably." (p. 2), and linking refers to relating scores on tests which are not built to the same content or statistical specifications. Different terminologies have been used to describe the process of establishing relationship between scores on two or more tests (for a complete review, see Kolen, 2004a, 2004b). Scale linking is used in this study to refer to the process of linking different scales rather than the process of linking test scores. However, scale linking is the prerequisite for establishing the connection between different test scores. Therefore, scale linking is an important step in test equating (Cook & Eignor, 1991; Kolen & Brennan, 2004) and satisfactory equating results require successful scale linking. If different groups of examinees have different probabilities of success on an item after they have been matched on the ability of interest, the item has differential functioning. In IRT, DIF is defined as the differences in the model parameters for the comparison groups (Clauser & Mazor, 1998). The item parameters for different groups should be compared only after they are placed on a common metric. Therefore, DIF identification depends heavily on the quality of scale linking. Some procedures have been developed to detect DIF by improving scale linking (Candell & Drasgow, 1988; Lautenschlager & Park, 1988; Lautenschlager, Flaherty, & Park, 1994; Park & Lautenschlager, 1990). In addition to psychometrics, scale linking is also very important to educational and psychological studies. Multigroup confirmatory factor analysis or mean and covariance structure analysis has been increasingly used to compare constructs across different groups (for a comprehensive review, see Vandenberg, 2000) and some unresolved issues are closely related to the difficulty of linking scales across groups (Millsap, 2005). Therefore, successful scale linking has the potential to produce satisfactory comparison studies on psychological constructs across different groups. In sum, scale linking is very important for educational measurement to be fair and objective for different groups of examinees. Unidimensional scale linking methods have been studied and applied frequently over the past two decades (for more information, see Kolen & Brennan, 2004; Yen & Fitzpatrick, 2006). The development of multidimensional linking methods (Davey, Oshima, & Lee, 1996; Hirsch, 1988, 1989; Li, 1997; Li & Lissitz, 2000; Min, 2003; Oshima, Davey, & Lee, 2000) is just at the infancy stage and more research is needed to obtain definitive results (Yen & Fitzpatrick, 2006). In this chapter, unidimensional and multidimensional models and linking methods are reviewed and the purpose of the current study is presented. Unidimensional IRT Models Logistic Model The threeparameter logistic (3PL) model (see Hambleton & Swaminathan, 1985; Lord, 1980) assumes that the probability of a correct answer to a dichotomously scored item j by an examinee with ability ,O is P,; (X 1 = l10,; aj,bj,cj a, (0, b) l \ e  1+e c+ +1c +e e_[ bj)], where x, is the item response (0 or 1) for person i on test item j, a, is the item discrimination parameter, bi is the item difficulty parameter, and c, is the guessing parameter or the pseudochance score level, representing the probability of correct response when the ability assessed by the item is very low. Sometimes the 3PL model is expressed as ^(x, =1,;a,,b,,c)= +1 + lc 1, (12) with D=1.701, so that a normal ogive model item characteristic curve (ICC) and a logistic model ICC with the same item parameters are almost identical. If ci is 0, the 3PL model becomes twoparameter logistic (2PL) model: x = 1;a,b) = +e 1 j]. (13) For 2PL model, if ai is 1, it becomes oneparameter logistic (1PL) model or Rasch model: pX = 11;b )= 1e,b (14) Normal Ogive Model There are also three normal ogive models or cumulative normal distribution models in IRT: one parameter model: 0(8,b,} 1 1 t(2 P ;b)=7 e 2 dt; (15) two parameter model: S1e 2 dt; (16) and three parameter model: (x,, =1 ,,c )= c, c (8,b,) 1 d (17) Pz(X I 11Ob c) (i b e 2 dt (17) Many IRT models have been developed for test items that are polytomously scored using ordered categories, including graded response model (Samejima,1969), partial credit model (Masters, 1982), generalized partial credit model (Muraki, 1992), rating scale model (Andrich, 1978), and nominal response model (Bock, 1972). Unidimensional IRT Scale Linking Scale Transformation The IRT parameter estimates produced from independent calibrations using data obtained from different groups of examinees are often on different metrics. Lord (1980) demonstrated that the relationship between the metrics of any two independent item calibrations is linear. Therefore, a linear equation can be used to transform the IRT parameters on scale E (representing the linked scale or equated scale) to scale F (representing the base scale). For person i and item j, F = AE +B, (18) aE (19) aF = A bF = AbE + B, (110) CF = CE, (111) where 6 aF b* and c* represent the transformed values from the linked scale to the base scale. A is the slope and B is the intercept. The constants A and B can be expressed as A = (112) aF B= AOE = b AbE (113) A and B can also be expressed for any two individuals i and i* or two items j and j* OF, bF bF A = (114) E, OE bE bE E B=bFb AbE= 0F AOE, (115) or expressed for groups of items or examinees (see Kolen & Brennan, 2004): c(b) (0) (aE)(116) A= = (116) a(bE) c(O,) (aF) B p=(bF) A(bE,)= (0) Ap(0). (117) The E U (O, ) value for the original parameters on scale E will be the same as the Pj (oF ) value for the transformed parameters on scale F as demonstrated by C + (1 1 1+e ' =C 1 CD (C ) D OE +BAbE Bb 1+e Therefore, the logistic function is invariant under a linear transformation of item and ability parameters. Most of the unidimensional IRT scale linking methods are based on this important feature. Scale Linking In practice, both test item parameters and examinees' ability parameters need to be estimated and the ability estimates are often scaled to have means of 0 and standard deviations of 1. Parameter estimates obtained from different groups of examinees are often on different scales due to nonequivalence of the groups even though all ability estimates are scaled with means of 0 and standard deviations of 1. Therefore, some data collection procedures are required to establish the connection between different scales by using the linear transformations mentioned above. In test equating, three data collection designs are often used, including random groups design, single group design, and commonitem nonequivalent groups design. The IRT parameter estimates for the first and second designs are assumed to be on the same scale because of the randomly equivalent groups of examinees and single group of examinees (Kolen & Brennan, 2004) if random sampling errors are ignored. For the third design, the parameter estimates are assumed to be on different scales due to the nonequivalent groups. The third design is the most often used equating design (Kolen & Brennan, 2004) and it is very similar to the design used for exploring DIF. Two approaches have been used to establish a common scale for parameter estimates for this design. One is to estimate parameters for all items on both test forms together. This method is often called concurrent calibration (Wingersky & Lord, 1984). Both BILOGMG (Zimowski, Muraki, Mislevy, & Bock, 1996) and MULTILOG (Thissen, 1991) have the function of simultaneously obtaining parameter estimates for two test forms and two groups on the same scale. The second approach is to link the two scales by using the parameter estimates for the common items. This study will focus on the second approach. The following IRT linking methods have been developed to establish a common metric for parameter estimates. Mean/sigma method. This method (Marco, 1977) uses the means and standard deviations of the b parameter estimates for the common items to calculate the constants A and B in the linear transformation equation: A= B= (bF A (bE(118) Mean/mean method. This method (Loyd & Hoover, 1980) uses the means of a parameter estimates for the common items to calculate A and the means of b parameter estimates for the common items to calculate B in the transformation equation: A=/a B= (bF)A/(bE). (119) Item response function method. In this procedure (Haebara, 1980), the constants A and B are estimated by minimizing the sum of the squared difference between the item characteristic curves for the common items over examinees: Hd =y zF F, FbF F aF,," AbE, +,C, j (120) Test response function method. The constants A and B are estimated by minimizing the sum of the squared difference between the test characteristic curves for the common items for examinees (Stocking & Lord, 1983): SLd z Zr (F,a J ,bF,) Z O ;E ,AbE, +BEJ (121) Item response function method and test response function method are often referred as the characteristic curve methods (Stocking & Lord, 1983). Specifically, the former is called item characteristic method and the latter test characteristic curve method. Minimum ,2 method. This method (Divgi, 1985) combines information of each item's parameter estimates and the variancecovariance matrix of sampling errors for each item from the item parameter estimation procedure. The constants A and B are estimated by minimizing the following quadratic function: X2 = a, ,FA, A B)] [Bi1F +iF F ,J (AbE +B (122) where Fj is the estimated variancecovariance matrix of the sampling errors for the item parameter estimates for itemj on the F scale and is the estimated variancecovariance matrix of the sampling errors for the item parameter estimates for itemj which are transformed from the E scale to the F scale. Comparison studies have been conducted for these methods with dichotomous IRT model. Based on a comprehensive literature review (Kolen & Brennan, 2004): (a) The characteristic curve methods produced more stable and accurate results than the mean/mean and mean/sigma methods, (b) the mean/mean method was more stable than the mean/sigma method, (c) the concurrent calibration method yielded more accurate results than the test characteristic curve method for a small number of common items and both procedures had the similar accuracy for a larger number of common items, and (d) the concurrent calibration method might be less robust to violations of the IRT assumptions than characteristic curve methods. These methods have been extended to link scales with polytomous IRT models. For example, Cohen and Kim (1998) extended mean/mean and mean/sigma methods to the graded response model and Kolen and Brennan (2004) suggested using mean/mean and mean/sigma methods for the generalized partial credit model. Baker (1992) generalized the test response function method to the graded response model and Baker (1993) used the item response function for the nominal response model. Kim and Cohen (1995) tried the minimum 2 method for the graded response model. There are also some comparison studies for these methods with polytomous IRT models. A simulation study (Cohen & Kim, 1998) comparing the mean/mean method, mean/sigma method, weighted mean/sigma method, test response function method, and minimum 2 method for the graded response model found that all methods produced similar results. Another simulation study (Kim & Cohen, 2002) comparing the test response function method and the concurrent calibration method for the graded response model found that the concurrent calibration was relatively more accurate. Multidimensional IRT Models Logistic Model Unidimensional IRT models appear to be adequate for scaling achievement test items in most practical situations (Yen & Fitzpatrick, 2006). However, it is reasonable to believe that the performance of examinees on some test items depends on more than one trait or ability and some consequences of applying unidimensional models to multidimensional data have been identified (see Yen & Fitzpatrick, 2006). The number of dimensions necessary to model the test item responses depends not only on the number of ability dimensions and the level on those dimensions exhibited by the examinees but also on the number of skills to which the test items are sensitive (Reckase, 1997a). Therefore, multidimensionality can occur in different ways depending on the interaction between a specific group of examinees and certain set of test items. There are two types of multidimensional IRT (MIRT) models for dichotomously scored item response data: the compensatory model and the noncompensatory model. In the compensatory model, a low 0 value on one dimension can be compensated for by a high 0 value on another dimension (McKinley & Reckase, 1983; Reckase, 1997a). In the noncompensatory model, an increase in the 0 value on one dimension cannot compensate for a lower value on another dimension (Simpson, 1978). Since estimation programs and linking methods have not been well developed for noncompensatory model, the most often used compensatory model is discussed and used in this study. The compensatory multidimensional threeparameter logistic (M3PL) model is a direct generalization of the unidimensional 3PL model (Reckase, 1997a): P(xJ = 10,;a,,djcj) (a,0,+d, = C +(1 CJ, e (a, +d,) (123) 1+e whereP(x, = 10,;a, dj, cj) is the probability of a correct response (x, =1) for person i on test item j, x, is the item response (0 or 1) for person i on test item j, a, is the vector of item discrimination parameters, d, is the scalar parameter related to the difficulty of the item, c, is the lower asymptote or guessing parameter, and 0, is the vector of ability parameters for person i. This model can be expressed in the following scalar form: P(x, = 10,; a,d ,c,) j ark k+d, j = c + (1 c) e k(124) Z "jkOk+dj 1+e = c + 1 c 1 a "lOkk+dj' l+e where m is the number of dimensions. When c is 0, the compensatory M3PL model becomes the compensatory multidimensional twoparameter logistic (M2PL) model (McKinley & Reckase, 1983): e(a,,+ d,) 1 p(xj =10,;aj,d)= e(a,+d,) 1a,,+d,) (125) 1+e(a+j) +e(JO+d This model can also be expressed as the following scalar form: a > O,kk + d, e = 1 P(x, = 1O,; a,d)= e 1 (126) Z "jakk+dj a"jkk+d, k=l kl= 1+e 1+e Compared with unidimensional IRT models, multidimensional discrimination and ability parameters are described in the form of vectors instead of scalars. If the 0, dimensions are orthogonal, the observed correlations among the item scores will be accounted for by the a, parameters. Otherwise, the item correlations will reflect both the aj parameters and correlated 0 dimensions. In MIRT, the probability of a correct response to an item depends on multidimensional ability and is defined as an item characteristic surface (ICS). Assuming orthogonal axes of dimensions in the surface, an itemj can be described by the following three characteristics (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991): multidimensional discrimination (MDISCj): MDISC, = a., (127) which is the discrimination power of the item for the most discriminating combination of dimensions; multidimensional difficulty (MDIFFj): d MDIFF = (128) SMDISC which, similar to the difficulty parameter in unidimensional model, is the distance from the origin of the 0 space to the point of steepest slope in a direction from the origin; and direction (a jk) of the greatest slope from the origin: ajk = arccos a (129) MDISC 1 which is the angle that the line from the origin of the space to the point of steepest slope makes with the kth axis for the item. Normal Ogive Model By adapting Thurstone's multiple factor model (1947) to dichotomous item response data, Bock and Aitkin (1981) proposed a multidimensional normal ogive model by firstly assuming that an unobserved continuous response variable, y,, for person i and item j is a linear combination ofm latent variables, 0, weighted by the factor loadings, a: yj = ajlO,1 + a 2,O2 +O+ a ,,,, +dj (130) where 0 N(0, I), y ~ N(0,1), and 6 N(O,G,) (Note that the as in Equation 130 are not the as in Equation 129). It is assumed that there is an underlying process which generates a correct observed response, x = 1, when y, equals or exceeds a threshold, y,, and produces an incorrect observed response, xj =0, otherwise. Then the probability of obtaining a correct item score is Pj(x, =110,; j,a)I 1 1exp k (131) Yj ZcjkOk, m wh=r (D  where 1 a k This is a compensatory model because greater ability on one dimension k=1 can make up for lesser ability on other dimensions. This model can be reparameterized to produce similar parameters in multidimensional logistic model (Bock, Gibbons, & Muraki 1988; Muraki & Engelhard, 1985) by y 2 J0 ') (t)dt (132) where m Z(o,)= ZaJkO + d k=l aO + dj, ajk ctjk C U J d j (71 It can also be shown that ajk a]k k q, di r' 7 ', qj with m q = 1+ ak . k=l When Os are correlated with covariance matrix O, it can be shown that ajk 1 _ (DCFc (133) (134) (135) (136) (137) (138) (139) d = (140) ak k (141) k 1 + aa 7, = d/ (142) 1+a'#a where a is vector of factor loading for itemj and a is the vector of discrimination for itemj. Multidimensional IRT models for polytomously scored test items have also been developed, including multidimensional logistic models and multidimensional normal ogive models (Kelderman, 1997; Muraki, 1999; Muraki & Carlson, 1995). Multidimensional IRT Scale Linking The multidimensional scale linking is more complicated than the unidimensional scale linking because it involves the transformations of scale locations, variances, and covariances of several ability dimensions obtained from different calibrations and more technical problems need to be resolved. Just as MIRT can be considered either as a special case of factor analysis or an extension of unidimensional IRT, the multidimensional scale linking can be realized either by borrowing methods from factor analysis (Hirsch, 1988; Li, 1997; Min, 2003) or by extending the unidimensional IRT linking methods to the multidimensional situations (Oshima et al., 2000). Hirsch's Method Hirsch (1988) is possibly the first author to explore the feasibility and effectiveness of multidimensional linking and equating by using the commonexaminee design. Hirsch presented three technical issues in multidimensional linking and provided three possible resolutions. The first issue is to establish scale transformations to keep the M2PL function invariant. The following transformation equations can be used for a twodimension (dimension 1 and 2) M2PL model: O, 1t A 02 _82 (143) a = aOoal, a2 = U20a2, (144) d* = d +a iu + +a 2u20, (145) where parameters with superscript "* are transformed parameters on a new scale. The M2PL function is invariant by this transformation: P(x, = 1 ;aJ, ) 7 t0i Pl +o20 '2 a 2 2 0 +(dd+al,1Oa 01j220) 1+e [a j(ll1 o)+ . 2)+(d+al1lO+j2e20) 1+ e [al, j +a2 ~z2+ d] =P(x = 10,;aj,dj) This scale transformation method can be extended to M2PL models with more than two dimensions. Hirsch's multidimensional scale linking method was based on the invariance of multidimensional function under the above transformations of item and ability parameters. The second technical issue is that the correlation between dimensions obtained from the first calibration may be somewhat different from the correlation estimated from the second calibration due to some nonparallel items for commonexaminee design. If this occurs, the parameter estimates from two calibrations are composites or linear combinations of different basis vectors. Therefore, it is necessary to transform the basis vectors from one calibration to those of the second calibration. This can be realized by transforming the two sets of ability parameter estimates of the common examinees from two calibrations so that they are as similar as possible. The third technical issue is the joint rotational indeterminacy of the item discrimination and ability parameters. That is, the dimensions can be rotated and produce many possible sets of 0, and aj parameter estimates without affecting the M2PL item characteristic function. As suggested by Wang (1985), the procrustean rotation in factor analysis (Schonemann ,1966) can be used to transform the parameter estimates from one calibration to those from the other calibration. Hirsch's linking method for the commonexaminee design includes four steps. In the first step, two sets of item and ability parameters (OF, F,) and (E, aE, ) for the common examinees but on different metrics are estimated from two independent calibrations. In (0F, F ), 0~ is a Nxm matrix, where N is the number of examines, and aFi is a nxm matrix, where n is the number of items. In the second step, three transformations are used to obtain common basis vectors for the two sets of parameter estimates. The first transformation by T1 refers the discrimination parameter estimates from the first calibration (aF, ) to a set of orthogonal basis vectors instead of the basis vectors defined by the ability estimates (0, ). The second transformation by T21 refers the discrimination parameter estimates from the second calibration (aE, ) to a set of orthogonal basis vectors instead of the basis vectors defined by the ability estimates (E, ). The third transformation by T3= Ti* T21 refers the discrimination parameter estimates from the first calibration (F, ) to a set of common basis vectors for both calibrations. In the third step, orthogonal procrustean transformation is used to rotate the ability estimates from the first calibration (F ) to those from the second calibration (0E,). This fourth transformation matrix T4 can be found by minimizing the sum of squared difference between each element of the two sets of ability parameter estimates (OF,) and (E, ). The method was called orthogonal procrustean transformation developed by Schonemann (1966). Specifically, suppose S = F'0E, SS' = PDP', and S'S = QDQ', then T4 =PQ Given the above four transformations, the means and standard deviations of the ability parameters for the common examinees from the two calibrations are estimated in the fourth step. For the commonexaminee design, the linking parameters can be estimated by equating the means and standard deviations of the ability estimates from the first calibration (F, ) and those transformed from the second calibrations (1 ). The linking parameter estimates are then used to transform the parameter estimates which have already been transformed by the procedure described in the second step. For example, suppose one uses the common examinee design and the M2PL model with two dimensions, the following relations exist: OF,, UF,, OEl  UlEl Fl1o C 10 '2, _'U'20 Op2, 'Up'2 F2 F20 E2 E (146) OF20 OE2 So oe1 l El/ \6 71 0 E, L FlO/ F0l o 0E2 E22 20 F20 60 = (147) OF( 2 0'S26 M 10\ = 1o I M20 =/iE2 /F2 (148) 10 10 F2 0F,, S2 = E2 (149) CF20 Then the transformed parameters from E scale to F scale are 0 M S 21 (150) SF2 2 aF1p = SioaE, aF =S2 (151) dF* =d + aE MlO + a M2, (152) where the parameters with "*" as superscript and "F" as subscript on the left side of equations are the final transformed parameters on F scale, and the parameters with as superscript and "E" as subscript on the right side of equations are the transformed parameters on E scale by the first three transformations. The function of this fourstep scale linking procedure for M2PL model was evaluated by test equating results performed on both simulated and real data sets using the commonexaminee design (Hirsch, 1989). The equating results were examined by comparing the mean differences and the mean absolute differences of the true scores and ability estimates between the base tests and equated tests. Satisfactory equating was found for true scores but not for ability estimates. Hirsch's linking method was originally developed for the commonexaminee design. However, it can easily be modified to conduct scale linking for commonitem nonequivalent groups design which is most usually used in test equating and DIF study. As Hirsch (1988) suggested, the basis vector transformation would be the same. The procrustean transformation would use the common item discrimination parameters instead of the ability parameters. The item difficulty parameter for each item would need to be regressed onto each of the ability dimension parameters and therefore produce one unique difficulty parameter for each of the dimensions (Reckse, 1985). Then the mean and sigma method would be used for the common item difficulty parameters for the final transformation. However, more study is needed to verify the adequacy of this modified linking procedure. Li's Method Compared with Hirsch's procedure, Li's (1997) multidimensional linking methods are more straightforward and consistent with MIRT computer estimation programs. Most MIRT programs solve the identification problem by requiring multidimensional abilities be distributed as multivariate normal MVN (0, 1). Therefore, the metric of the item parameter estimates is typically referred to orthogonal reference axes with unit length. Given this condition, one reference system can be transformed onto the other reference system by a composite transformation: an orthogonal procrustean transformation for rerotating the reference system, a translation transformation for shifting the point of origin, and a single dilation for rescaling unit length. Specifically the following equations are used in the reference system transformation: aF =kT aE, (153) d = d +(aT (154) OF, = (1/k)(T E m). (155) It can be shown that the M2PL function is invariant to these transformations: P(X,, =19*;a*,^ P(x, = 10;aFF ,dF) 1+ [kTaE ][(1/k I ',. m)][d +(a T)m 1 +e kaEjTl(1/k)(T 10Ez m)[dE ajTn] 1 1+e aJE 0 aEJ Tm][dE +aEJTm] 1 1+e ;EJo +dE =P(x, = 10E;aE,dE). The question is how to find T, m, and k. Li (1997) proposed several methods to estimate the scale linking parameters. The rotation matrix T can be estimated by orthogonal procrustean transformation procedure as mentioned in Hirsch's method above. Let S = a aEj SS' = PDP and S'S = QDQ', then T = PQ' The origin shift coefficient m and unit change coefficient k can be estimated simultaneously by minimizing the sum of squared difference between test characteristic functions for the common items obtained from the two calibrations, which was originally developed by Stocking and Lord (1983) for the unidimensional linking: f(m,k)= i PF (0;;F ,dF ) PF (;aF J (156) e 1N t=1 j=1 where N is the number of grid points of values. The origin shift and unit change coefficients can also be estimated separately by different procedures. For example, the origin shift coefficient can be estimated by minimizing the sum of squared difference between the two difficulty parameter estimates obtained from two calibrations: f (m) = (F d F), (157) J1 where n is the number of common items. This was called least squares procedure (Li, 1997). The unit change coefficient can be estimated as the ratio of the square root of the maximum eigenvalues of the matrices aFp aF and aE aE, obtained from the two calibrations: Maximum sig( apF, Fj k = (158) Maximum sigi HE H where sig() represents the singular value or the nonnegative square roots of the eigenvalue. This was called ratio of eigenvalues procedure (Li, 1997). Similar to the least squares procedure for the estimation of origin shift coefficient, the unit change coefficient can also be estimated by minimizing the sum of squared difference between the two sets of discrimination parameters estimated from two calibrations. This is also referred as least squares procedure (Li, 1997): f(k)= FC, F,. (159) J=1 The rotation matrix T and unit change coefficient k can also be estimated simultaneously by a least squares method developed for fitting one matrix to another through a rotation matrix, a translation vector, and a central dilation vector (Schonemann & Carroll, 1970). In this case, the rotation matrix and dilation scalar were estimated by minimizing the sum of squared errors of the following residual matrix: E=(kaFi T) a, (160) It can be shown that trace (T a )acE k = (161) trace (aC aF where aF = iF aFaCE, = iE aE, (162) with aiF as the mean of iF and a, as the mean of aE This was called ratio of trace procedure (Li, 1997). The translation vector was not estimated by this method because item discrimination can not provide information about origin shift. Comparing the effect of different combinations of reference, translation, and dilation transformation procedures on the multidimensional linking parameters estimation, Li (1997) found that the most appropriate MIRT linking method is the combination of procrustean rotation approach (for dimensional transformation), the ratio of trace procedure (for dilation), and the least square procedure (for translation). This linking method could produce accurate estimation of item parameters, approximately equivalent estimation of ability parameters, but unsatisfactory true score estimation. Min's Method Min (2003) challenged Li's (1997) two reasons for using a single dilation parameter, that is, mathematical tractability and the assumption of constant variance across dimensions, and argued that one single dilation is insufficient for describing the scale unit changes for multiple dimensions. Two independent calibrations may change the scales of the multidimensional dimensions to different degrees. To address this problem, Min (2003) modified Li's (1997) method by replacing the single dilation parameter with a diagonal dilation matrix to model different unit changes on different dimensions. The reference system transformations are performed as follows: aF = K'Ta (163) dF =dE + (aET)n, (164) 6 = K I(T 10 m), (165) where K is a diagonal dilation matrix. It can be shown that the M2PL function is invariant to these transformations: P(x, = 10;aF,d) 1 l+e [KTaE [(K 1XT 10E m)[dE d(aET'jT 1 1 +e r'EjTKIK 1T 1E m)dEj (aEjT) 1 1 +e rEaEj ETKm +.aEjTm] 1 1+e Eo, +dEj I =P(x, = 10,;a, dE). For twodimensional model, K becomes where k1 is the dilation parameter for the first dimension, and k2 for the second dimension. The least square method (Schonemann & Carroll, 1970) of estimating a rotation matrix, a translation vector, and a central dilation vector for fitting two matrixes can be followed to find T, K, and m in the transformation equations (Min, 2003). Mathematically Li's (1997) method and Min's (2003) method produce the same solution for T and m and the only difference of linking results comes from the different dilation parameters. Reckase and Martineau (2004) identified an important weakness in Li's (1997) and Min's (2003) method for MIRT models with high dimensionality and provided a solution to the problem by employing a nonorthogonal procrustean transformation. However, this approach needs to be examined by further empirical studies. Oshima and Colleagues' Method All multidimensional linking methods mentioned above borrowed an important procedure, procrustean rotation, from factor analysis to transform the dimensional axes. Oshima et al. (2000) extended four scale linking methods within IRT from unidimensional to multidimensional models. According to their methods, the following equations were used to transform the IRT parameters on one scale E to another scale F (to distinguish IRT linking methods from the factor analysis methods described above, different indices for linking parameters are used). For person i and item j, a =(A 1a, (166) dF =dE a'EA P, (167) O, AOE, +, (168) where the rotation matrix Am m adjusts the variances and covariances of the ability dimensions (scale), and the translation vector P,,, changes the means of the ability dimensions (location) on the two scales. The model indeterminacy can be shown as the following: P(x, = OF ;a ,dF) 1 (+e [( )'a [AOE +P[E E aEAip] 1+e 1+e faEA 1AoEz 'E A ip]} 1 1+e IaEz+ EA dE aEA 1]} 1 l+e E'+d i =P(x = 1O,; aE, d,) As in the unidimensional IRT, suppose that two nonequivalent groups of examinees take common test items and independent calibrations produce two sets of parameter estimates (siF ,d, ) and (El ,d, ). These two sets of parameter estimates are on different scales F and E, and scale linking needs to be conducted to place the two sets of parameter estimates on a common scale. Using the above equations, (. dE ) on E scale can be transformed to the F scale (aF ,d ). The values of the two sets of item parameter estimates (a ,dF ) and (a. ,d* ) should be similar due to the invariance of common item characteristic in IRT. Unidimensional IRT linking methods can be extended to multidimensional IRT model to minimize some functions of the difference between the two sets of item parameters. Again, the question is how to find the values of A and P so that the connection between the two scales can be established. The direct method. This method was a multivariate extension of the minimum chi square linking method for unidimensional IRT model (Divgi, 1985). The values of A and P are estimated by minimizing the sum of squared difference between the two sets of item parameter estimates over all items. However, the direct method is different from the original method in that it does not consider the variancecovariance matrix of sampling errors for item parameter estimates in the function: f(A, p)= j(A, 0) ] + dF (169) n(!mn + 1) j= k=\ j1 where n is the number of items, m is the number of ability dimensions, and (a k d, ) are transformed parameter estimates from E scale to F scale. The equated function method. This method is the multidimensional extension of the mean and sigma methods for the unidimensional IRT model (Loyd & Hoover, 1980; Marco, 1977). A more general system of scale linking equations is used to specify that some functions of the common item parameters from the first calibration (a ,dF ) are equal to the same functions of the transformed common item parameters from the second calibration (a di ). The transformed item parameter estimates can be obtained by using the above scale transformation equations with the linking parameters A and P. The values of A and P are estimated by minimizing the sum of squared difference between the same functions of the two sets of selected elements of the estimated (aF, ,dF ) and (aF dF ). The number of functions needed (p) depends on the number of dimensions (m) or elements in A and p, with p = m2 + m. For example, in the two dimensional case (m = 2), four parameters in A and two parameters in P need to be estimated. Therefore, six functions are required to estimate the six linking parameters and they could be the means of a1,, aj2, and c for the first and second halves of the common items (or other block of items). The scale linking functions are flexible in terms of which item parameter estimates to use and what function to use. Different systems of scale linking functions may produce different values of the linking parameters A and P. The quality or appropriateness of linking functions can be evaluated by their stability across random examinee samples, the character of the common item sets, and the true values of the linking parameters (Davey, et al., 1996). For example, if the mean is the chosen linking function, the function to be minimized is f(A, p) 1 pF ) 2, (170) P u=1 where HiF, I pF, ... are the estimated means ofp separate sets of elements of the estimated (aF, dF ), and Fu, ,, up are the estimated means ofp separate sets of elements of the estimated (aF d* ). The test characteristic function method. This method is an extension of the test response function method developed by Stocking and Lord (1983) for the unidimensional IRT model: f(A,p)=I W ;aF, P ;aFI (171) qo O FwF JF where q is the number of matching 0 vectors, W. is the weight taken at different 0 values. The W. is used to emphasize that some 0 values are more important than others to estimate the linking parameters. The weight can also be considered equal along the ability scale. The item characteristic function method. This method is the multidimensional generalization of the item response function method for unidimensional IRT model (Haebara, 1980): f(A,p)= 1 W [P(O;a,dFP(O;aFdF)1. (172) nxq 1 Based on a simulation study comparing the four IRT linking methods under different ability distributions (Oshima et al., 2000), all of the four methods were acceptable under almost any of the minimization criteria and offered dramatic improvement over not linking at all. It was also found that the test characteristic function method and item characteristic function method were more stable and recovered the true linking parameters better than the direct method and equated function method. The multidimensional linking methods developed by Hirsch (1988), Li (1997), Oshima et al. (2000), and Min (2003) can all be directly or indirectly performed for the commonitem nonequivalent groups design, which have been a widely used in test equating (Kolen & Brennan, 2004). Accordingly those methods have the potential for establishing calibrated item pool and exploring DIF. Another multidimensional linking method proposed by Thompson, Nering, and Davey (1997) can be used for test equating in a design without common items or examinees. With the assumption of the same origin, axes, and correlation between axes for the two randomly equivalent groups of examinees, this method solve the rotational indeterminacy by identifying similar item content clusters on different tests and then rotating them in the same multidimensionalreference system. Further studies need to be conducted to evaluate the performance of this method. Multidimensional scale linking is a new research area. There have been very few studies conducted for each of the proposed methods (Hirsch, 1988; Li, 1997; Min, 2003; Oshima et al., 2000) and even fewer studies for comparing different methods in the literature. Therefore, it is currently difficult to evaluate the function of different methods. The only comparison study by Min (2003) compared Li's method, Min's method, and Oshima and colleagues' test characteristic function method in terms of accuracy and stability of scale transformations under different conditions varying in sample size, structure of dimensions, and ability distribution. The results indicate that both Oshima and colleagues' and Min's methods were better in transforming discrimination parameters than Li's method, and Min's and Li's methods performed better than Oshima and colleagues' method in transforming the difficulty related parameters. In addition, Oshima and colleagues' method performed better than Min's and Li's methods in transforming test true scores, and Li's and Min's methods were better than Oshima and colleagues' method in maintaining the structure of dimensions through orthogonal rotation. Purpose of the Study Based on the literature review of the multidimensional linking methods, Li's methods have been evaluated under various circumstances such as different linking procedures, sample sizes, equating situations, number of anchor items, linking situations, and ability distributions (Li, 1997). Min's method has also been examined with comparison with other methods under different conditions including different sample sizes, dimensional structures, and ability distributions. The performance of Oshima and colleagues' four IRT linking methods has been examined under fewer testing conditions, that is, for different ability distributions, using simulation study with only 20 replications (Oshima et al., 2000). A comparison study (Min, 2003) indicates that one of the four IRT linking methods, that is, the test characteristic function method, outperformed other methods in transforming item discrimination parameter estimates and equating true score estimates. This suggests that the IRT procedures are promising methods for multidimensional linking and equating. Further studies are needed to examine the performance of these four methods under more testing conditions. As an extension of previous research (Oshima et al., 2000), the purpose of this study was to evaluate the performance of the four multidimensional IRT scale linking methods, the direct method, equated function method, test characteristic function method, and item characteristic function method, under various testing conditions, which include different test structures, test lengths, sample sizes, and examinees' ability distributions. CHAPTER 2 METHODOLOGY A comprehensive review of the unidimensional scale linking and test equating (Cook & Petersen, 1987) provides us a framework for exploring the performance of multidimensional scale linking methods. According to Cook and Petersen's discussion, the results of linking and equating depend on linking or equating methods, sample characteristics, and properties of the common items. In addition, the multidimensional structure underlying the test item responses makes scale linking more complicated and should be considered as one important testing condition. In this simulation study, the performance of the four MIRT scale linking methods (Oshima et al., 2000) for the commonitem nonequivalent groups design was evaluated with the compensator compensatory M2PL model under different testing conditions, including different test structures, test lengths, sample sizes, and examinees' ability distributions. The M2PL model had two dimensions. Design Independent Variables or Experimental Conditions IRT linking method. This study was to evaluate the performance of the four multidimensional IRT scale linking methods proposed by Oshima et al. (2000): the direct method, equated function method, test characteristic function method, and item characteristic function method (see the section Multidimensional IRT Scale Linking in Chapter 1 for detailed description). The equated function, test characteristic function, and item characteristic function methods were implemented in a manner consistent with the implementation in Oshima and colleagues' study (2000). For the equated function method, the means of ajl, Cj2, and dc for the first and second halves of the items were used as the equated function. For the test and item characteristic function methods, seven equally spaced 01 points from 4 to 4 and seven equally spaced 02 points from 4 to 4, making 7 x 7 = 49 grid points, were used with equal unit weight along the ability scale. The four IRT linking methods have been compared under different ability distributions (Oshima et al., 2000). It is unknown how they perform under other circumstances. Therefore, this study can be considered as an extension of Oshima and colleagues' study (2000) from one testing condition (ability distribution) to various testing conditions (see the following for the detail). Test structure. In IRT, the test dimensionality for a particular population is the minimum number of latent abilities required to produce a monotone and locally independent model (McDonald, 1981, 1997; Stout, 1990). In the geometrical representation of a test structure, the coordinate axes of a multidimensional space is defined by a complete set of latent abilities examined by the test, and each item is described by a vector in the space with its orientation representing the ability composite that is best measured by the item (Ackerman, 1994, 1996; Reckase, 1985, 1991). According to the literature review by Tate (2003), based on the number and nature of the abilities required for the response to each item in the test, there are three types of test structure: simple structure, approximate simple structure, and complex structure. In the simple structure, all item vectors are exactly aligned with one of the axes in the multidimensional space after an appropriate rotation, so all the items under each dimension measure the same ability. If all item vectors are approximately aligned with one of the multiple axes and therefore the contribution of one ability is dominant over the contribution of all other abilities, the test has approximate simple structure. In complex test structure, the response to one item depends on more than one ability. The first type of structure has been considered as an ideal one and the second and third types as more realistic item structures (Kim, 1994; Roussos, Stout, & Marden, 1998). Following the method in previous studies (Batley & Boss, 1993; Min, 2003; Mroch & Bolt, 2006; Oshima et al., 1997; Oshima & Miller, 1992; Tate, 2003), two types of two dimension test structure were created by using the three MIRT item characteristics: MDISC, MDIFF, and direction (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991). In the approximate simple structure, there were two sets of items: The responses to the first half items depended on one composite ability with the first dimension as the dominant dimension and the second dimension as the minor dimension; The responses to the second half items depended on another composite ability with the second dimension as the dominant dimension and the first dimension as the minor dimension. In the complex test structure, there were four sets of items with equal number of items in each of the set. Two sets of items loaded heavily on one of the two dimensions and lightly on the other dimension, and the remaining two sets loaded heavily on both dimensions. Test length. The test items are used to establish the common metric for the two sets of parameter estimates obtained in separate calibrations. Therefore, the feature of items is very important for sale linking. The estimation of linking parameters depends not only on the number of items, but also on the characteristics of the item parameters. Based on some literature reviews (Brennan 1987; Cook & Petersen 1987; Kolen & Brennan, 2004), 1530 common items are necessary for unidimensional IRT linking, although the required number also depends on other conditions, such as the linking methods, examinees' ability distributions, and characteristics of the items. Different numbers of items have been used in multidimensional linking studies. Li (1997) used 15 and 25 items in his study and found that the number of items had a significant influence on the stability of transformation parameter estimates for multidimensional linking. Oshima et al., (2000) created 40 item parameters to examine the performance of their four IRT multidimensional linking procedures. Twenty items were used in Min's study (2003) to compare three multidimensional linking methods. In this study, 20 and 40 items were used to evaluate the four MIRT linking methods under different numbers of items with the consideration that more items may be needed for MIRT than unidimensional IRT linking. Sample size. Theoretically the performance of linking methods depends on the accuracy of parameter estimates and parameter estimation is affected by the sample size (Li, 1997). So the linking function depends in some extent on different sample sizes. Compared with unidimensional IRT models, a large number of examinees are required for MIRT calibration because more parameters need to be estimated. Based on some MIRT researchers' (Ackerman, 1994; Carlson, 1987) recommendations, 2000 examinees is a reasonable sample size to obtain satisfactory item parameter estimates for compensatory multidimensional model. Reckase (1997a) reported that NORHARM (Fraser & McDonald, 1988) and TESTFACT (Wilson, Wood, & Gibbons, 1987) generally produced stable parameter estimates for long tests and sample sizes exceeding 1000 cases. A comprehensive study (Tate, 2003) using both simulated and real data found that most of the oftenused multidimensional computer programs performed well for the sample size of 2000 examinees. To acquire stable item parameter estimates, Hirsch (1988) used 2000 examinees to evaluate the proposed multidimensional equating. In his first study, Li (1997) used three different sample sizes, 1000, 2000, and 4000, to examine the performance of three multidimensional linking methods and found that the sample size had a prominent role in estimating transformation parameters. Li used 2000 examinees to evaluate the best linking method in his second study (Li, 1997). Min (2003) also found the significant effect of sample sizes, 500, 1000, and 2000 on the accuracy and stability of different multidimensional linking methods and suggested that the sample of 500 examinees showed unreliable results and the sample of 1000 showed somewhat acceptable outcomes (note that approximate simple structure and complex structure were used in the study). It is not unusual in testing practice that the sample size is less than 1000 especially in nonachievement area and the performance of the four IRT linking methods need to be evaluated under this condition. In this study, three different sample sizes, 500, 1000, 2000, were used to examine the robustness of the four multidimensional IRT scale linking methods against parameter estimation errors. As defined in other studies (Li, 1997; Min, 2003), the sample size of 2000 is the base for comparing the effect of different parameter estimation errors. The sample size of 500 can be used to examine the robustness of IRT scale linking for small sample size. The sample size of 1000 was used to examine the effect of sample size between 500 and 2000 on scale linking and it was also consistent with a study using multidimensional linking for identifying differential item functioning (Oshima et al., 1997). Examinees' ability distribution. Based on the review by Kolen and Brennan (2004), the performance of scale linking also depends on the similarity between the two groups of examinees. The more similar the groups are, the more adequate the linking will be. Large difference between groups may produce significant problems in estimating scale linking parameters. Groups of examinees may differ in many characteristics, such as cultural background, attitude, motivation, and personality. A comprehensive review on population invariance in equating and linking (Kolen, 2004) found that equating is population dependent except under highly restrictive conditions, such as two test forms with similar content, difficulty, and reliability. This suggests that scale linking parameters that are used to obtain the equivalent scores should also be dependent on the populations used in the estimation. The ability distribution is an important characteristic of the examinees and has a significant influence on test equating and scale linking under both unidimensional (Cook et al., 1985) and multidimensional circumstances (Li, 1997; Min, 2003). As summarized by Cook and Petersen (1987), the similarity of ability distribution between groups also affects other conditions required for test equating, such as the number of common items. Groups of examinees may differ from each other in terms of mean, variance, and covariance of the dimensions. Oshima et al. (2000) examined the four multidimensional IRT scale linking methods under six conditions of the ability distributions across two groups: no difference at all; differences in 0 variances; differences in 0 correlations; differences in0 means; differences in 0 means and variances; differences in 0 means and correlations. Min (2003) used four conditions similar to those investigated by Oshima et al, such as differences in 0 correlations; differences in8 correlations and means; and differences in 0 correlations, means, and variances. However, in all conditions the ability dimensions were uncorrelated in the base group. In education and psychology, most constructs and dimensions within a construct are correlated. Two groups should have similar structure of construct before scale linking and equating are conducted. Given these two considerations and to keep the scope of the study manageable, correlations between dimensions were set at the same level, but not zero, across all groups and the two groups varied only in ability level and variance. One purpose of this study was to explore twodimensional linking methods under the following four ability distributions: no difference at all, differences in 0 means, differences in 0 variances, and differences in 0 means and variances (See Table 21 for the detail). Dependent Variables or Evaluation Criteria Different statistics have been used to evaluate multidimensional linking methods. Bias and root mean square error (RMSE) are often used to evaluate the accuracy and stability of results across replications of experiment in IRT simulation studies. For example, using a common examinee design, Hirsch (1988) evaluated the effectiveness of multidimensional linking and equating by examining the means and standard deviations of the differences and absolute differences between the true scores, ability estimates, test characteristic response surfaces, and contour plots of the common examinees on the base and equated tests. Li (1997) used bias and RMSE to evaluate three multidimensional linking methods, but he used both the bias and RMSE of linking parameters and item and ability parameters over replications in his study. Oshima et al. (2000) compared the means, standard deviations, bias, and RMSE of linking parameters for different methods. Another criterion for common item scale linking in IRT framework is to evaluate how small the differences are between the item parameter estimates for base group and the transformed item parameter estimates for equated group across the common items (Min, 2003; Min & Kim, 2003). This criterion was used in this study. Specifically, the common item nonequivalent groups design was used and simulation was performed to create the data for both base and equated groups. The parameters for the two groups were estimated and then transformed onto a common scale. Specifically, the parameter estimates for equated groups were transformed onto the scale for the base groups by using the transformation equations described before. The linking coefficients in the transformation equations, A and P, were estimated through the four IRT multidimensional linking methods. After the common item parameters estimated from base and equated groups were placed on the same scale, the performance of the four linking methods were evaluated by examining the differences between the two sets of item parameter estimates. The mean difference and difference variation across replications (r) for each item were used to evaluate the accuracy and stability of the four linking methods, as described by the following statistics: SD (aJ ) = di (22) r where diff = a F, (23) rdiff idff (dj) r= (24) SD )= dJ df (25) r where diff = d dF. (26) Procedure Data Generation The following compensatory, twoparameter, twodimension IRT model was used to create the item responses with different testing conditions described above: P(x, = l0,;aj,d) d D aD(all+a22+d (27) l+e First, five sets of ability parameters for each of the three sample sizes (500, 1000, and 2000) with multivariate normal distributions with various means, variances, and covariances were generated. One set of ability parameters was used for the base group and the other for the four equated groups (see Table 21 for the five group ability distributions). Second, two sets of item parameters (one with 20 items and another with 40 items) for each of the two test structures (approximate simple structure and complex structure) were created using the three MIRT item characteristics: MDISC, MDIFF, and direction (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991). Based on the pooled results from past empirical studies (Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991), the estimated MIDSC has a lognormal distribution with mean of 1.37 and standard deviation of 0.54 and the estimated MDIFF has a normal distribution with mean of 0.28 and standard deviation of 0.69. The item parameters of MDISC and MDIFF in this study were selected randomly from lognormal and normal distributions with the same value of means and standard deviations. The test structure was created by manipulating the angle of each item with the first dimension. For the items that loaded on one dominant dimension, the angle between the item and its dominant dimension was selected from a lognormal distribution with mean of 100 and standard deviation of 20. For the items loaded heavily on both dimensions, the angle between each item and two dimensions were selected from a normal distribution with mean of 450 and standard deviation of 100. Next, the discrimination parameters, al, a2, and the difficulty parameter, d, were computed by the following formula ((Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991): a, = MDISC* cos a,, (28) a2 = MDISC cos a2, (29) d = MDISC MDIFF (210) (See Table 22, 23, 24, and 25 for specific parameter values for different test structures with different test lengths) Next, dichotomous item responses were created using the twoparameter and two dimension IRT model described by Equation 27. To produce more precise and stable results, replications were conducted for each of the combinations of testing conditions. In IRT simulation studies, the number of replications depends on the purpose of the study, the desire of minimizing the sampling variance of the estimated parameters, and the need for statistical tests of results (Harwell, Stone, Hsu, & Kirisci, 1996). The previous studies on multidimensional linking or equating methods used 0 (Hirsch, 1988), 20 (Oshima et al., 2000), 50 (Min, 2003), 100, and 200 (Li, 1997) replications to evaluate the accuracy and stability of linking or equating results. Based on Harwell and colleagues' (1996) recommendation of using a minimum of 25 replications for IRT simulation studies and given the level of complexity of this study, 500 replications were used for each of the combinations of testing conditions to evaluate the accuracy and stability of the four multidimensional IRT linking methods. Parameter Estimation The parameters of MIRT models can be estimated using different methods and computer programs. The often used estimation methods include unweighted least squares (ULS) factor analysis of tetrochoric correlations, weighted least squares (WLS) analysis of the matrix of polychoric correlations, and robust WLS analysis methods performed by MPLUS (Muthen & Muthen, 1998), least squares estimation method based on the matrix of raw product moments of item scores by NOHARM (Fraser & McDonald, 1988), marginal maximum likelihood estimation method by TESTFACT (Bock, Gibbons, Schilling, Muraki, Wilson, & Wood, 1999). The study focusing on model parameters recovery by Knol and Berger (1991) suggests that "for multidimensional data a common factor analysis on the matrix of tetrachoric correlations performs at least as well as the theoretically appropriate multidimensional item response models" (p. 457). A study comparing TESTFACT and NOHARM (Gosz & Walker, 2002) found that NOHARM provided better solutions for predicting item performance. The comprehensive comparison study by Tate (2003) found that MPLUS, NOHARM, and TESTFACT performed reasonably well over a relatively wide range of conditions in assessing the test structure and estimating parameters. This result was confirmed by another recent study (Stone & Yeh, 2006). Based on these studies, all these methods can provide satisfactory estimation for model parameters. NOHARM was used in this study due to its consistently good performance in previous studies. After the MIRT item parameters were estimated by NOHARM, the linking parameters estimated by the four multidimensional IRT linking methods (direct method, equated function method, test characteristic function method, and item characteristic function method) were obtained by the computer program IPLINK, which was developed by Lee and Oshima (1996). Result Analysis Some previous multidimensional linking studies used descriptive analysis (Hirsch, 1988; Oshima et al., 2000) and some studies used both descriptive and inferential analysis (Li, 2000; Min, 2003). In this study, the means and standard deviations of differences between the item parameter estimates for base group (aF ,dF ) and the transformed item parameter estimates for equated group (aF ,dF ) across 500 replications were compared under different testing conditions. Specifically, the accuracy and stability of the four multidimensional IRT linking methods were evaluated by examining the mean differences and difference variations of al, a2, and d for all items in the test under different testing conditions. Based on the experimental conditions described above, there are 5 factors in this study: multidimensional linking method (4), test structure (2), test length (2), sample size (3), and ability distribution (4). Therefore, the total number of experimental conditions is 4 x 2 x 2 x 3 x 4 192. Five hundred replications were conducted for each of the conditions. Table 21. Ability distributions for examinee groups Base Group Group 1 Group2 Group3 Group4 0] 1 .5 0 1 .5 .5 1 .5 0 .8 .4] .5] L .8 .4 0 .5 1 0 .5 1 .5 .5 1 0 .4 .8 .5 .4 .8 Note: All the correlations between dimensions are .5. Table 22. Item parameters for 20 items with approximate simple structure Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 d 0.70 0.23 2.19 0.75 1.18 0.77 1.12 1.26 1.86 1.46 0.72 0.17 0.51 0.07 1.38 2.66 0.48 0.68 0.77 0.73 MDISC 1.13 2.29 1.41 1.03 1.71 0.99 1.26 0.95 1.68 2.06 1.33 1.10 1.89 0.63 1.00 1.10 1.17 0.85 2.40 1.06 MDIFF 0.62 0.10 1.55 0.73 0.69 0.78 0.89 1.33 1.11 0.71 0.54 0.15 0.27 0.11 1.38 2.42 0.41 0.80 0.32 0.69 Table 23. Item parameters for 40 items with approximate simple structure Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 d 1.28 0.21 1.45 0.28 1.59 0.45 0.79 1.07 0.80 0.94 0.27 0.40 0.14 0.32 1.66 0.65 1.50 0.46 0.07 0.74 0.66 0.94 0.20 0.86 1.09 0.12 1.97 0.35 0.90 0.23 0.18 0.37 0.18 0.60 0.10 1.75 0.62 0.25 0.63 0.44 MDISC 2.33 1.12 1.45 0.58 0.93 0.98 1.20 1.13 1.00 1.92 0.57 1.38 1.18 1.69 1.41 0.80 1.10 1.19 0.83 0.71 1.34 1.22 1.65 1.21 1.00 0.78 1.54 0.98 1.47 1.10 0.88 1.61 0.86 1.50 2.62 1.42 1.42 1.06 0.90 1.45 MDIFF 0.55 0.19 1.00 0.48 1.71 0.46 0.66 0.95 0.80 0.49 0.48 0.29 0.12 0.19 1.18 0.81 1.36 0.39 0.09 1.04 0.49 0.77 0.12 0.71 1.09 0.16 1.28 0.36 0.61 0.21 0.20 0.23 0.21 0.40 0.04 1.23 0.44 0.24 0.70 0.30 Table 24. Item parameters for 20 items with complex structure Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 d 0.70 0.23 2.19 0.75 1.18 0.77 1.12 1.26 1.86 1.46 0.72 0.17 0.51 0.07 1.38 2.66 0.48 0.68 0.77 0.73 MDISC 1.13 2.29 1.41 1.03 1.71 0.99 1.26 0.95 1.68 2.06 1.33 1.10 1.89 0.63 1.00 1.10 1.17 0.85 2.40 1.06 MDIFF 0.62 0.10 1.55 0.73 0.69 0.78 0.89 1.33 1.11 0.71 0.54 0.15 0.27 0.11 1.38 2.42 0.41 0.80 0.32 0.69 Table 25. Item parameters for 40 items with complex structure Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 d 1.28 0.21 1.45 0.28 1.59 0.45 0.79 1.07 0.80 0.94 0.27 0.40 0.14 0.32 1.66 0.65 1.50 0.46 0.07 0.74 0.66 0.94 0.20 0.86 1.09 0.12 1.97 0.35 0.90 0.23 0.18 0.37 0.18 0.60 0.10 1.75 0.62 0.25 0.63 0.44 MDISC 2.33 1.12 1.45 0.58 0.93 0.98 1.20 1.13 1.00 1.92 0.57 1.38 1.18 1.69 1.41 0.80 1.10 1.19 0.83 0.71 1.34 1.22 1.65 1.21 1.00 0.78 1.54 0.98 1.47 1.10 0.88 1.61 0.86 1.50 2.62 1.42 1.42 1.06 0.90 1.45 MDIFF 0.55 0.19 1.00 0.48 1.71 0.46 0.66 0.95 0.80 0.49 0.48 0.29 0.12 0.19 1.18 0.81 1.36 0.39 0.09 1.04 0.49 0.77 0.12 0.71 1.09 0.16 1.28 0.36 0.61 0.21 0.20 0.23 0.21 0.40 0.04 1.23 0.44 0.24 0.70 0.30 CHAPTER 3 RESULTS As described in Chapter 2, the criterion used in this study to evaluate the four multidimensional IRT linking methods was based on the differences between the item parameter estimates for the base group and the transformed item parameter estimates for the equated group across 500 replications. Specifically, after the item parameter estimates from the two groups were transformed to a common scale, the mean and standard deviation of their differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. For each of the 192 experimental conditions, there were three parameter estimates a,, a2, and d; therefore, the mean and standard deviation of the differences were computed for a,, a2, and d across 500 replications for each item of the test. Then the distributions of the means and standard deviations of the differences of a,, a2, and d for all items in the test were obtained. Based on the characteristic of item parameter invariance in IRT, the item parameter estimates from the base and equated groups should theoretically be equal after they are transformed to a common scale. So their differences, and accordingly the means and standard deviations of their differences across 500 replications, should be 0. Therefore, the performance of the four multidimensional IRT linking methods can be evaluated by examining how close the means and standard deviations of the differences are to 0. There is currently no generally accepted criterion about how close the item parameter estimates for the two groups should be in order for the linking to be considered accurate and stable. To describe the distribution of difference, histograms of means and standard deviations of the differences of a,, a2, and d for the 192 experimental conditions were prepared. The appendix contains the histograms for all 192 conditions. In this chapter, histograms selected to illustrate the trends in the results will be presented. The following midpoints were used to construct the histograms for means: 0, 0.2, 0.4, 0.6. All values smaller than 0.5 and larger than +0.5 were included in the categories with midpoints of +0.6. For the histograms of the standard deviations, 0.1, 0.3, 0.5, 0.7, 0.9, 1.1, and 1.3 were used as the midpoints. All values beyond 1.2 were classified into the category with midpoints of 1.3. If all or most of the items in the test had means and standard deviations close to 0, the linking method was considered accurate and stable. Otherwise, the linking method was inaccurate and unstable. The performance of the four multidimensional IRT linking methods was evaluated in this way under different testing conditions. On the histograms, the direct method, equated function method, test characteristic function method, and item characteristic function method are labeled Linkl, Link2, Link3, and Link4. For test structure, the approximate simple structure is abbreviated as APP and the complex structure as COM. For test length, the number of items in the test is indicated by n = 20 or n = 40. For sample size, the number of examinees is indicated by N = 500, N = 1000, or N = 2000. For ability distribution differences between the base and equated groups, the condition is abbreviated as Gl if the mean vectors and covariance matrices were equal for the two groups, G2 if only the mean vectors were different, G3 if only the covariance matrices were different, and G4 if the mean vectors and covariance matrices were not equal for the two groups. As will be shown subsequently, inspection of the results indicated that the effects of linking methods depended on the test structure. Therefore, the decision was made to focus primarily on the effects of linking methods within each of the test structures. Inspection of the results for APP suggested that the interactions of all other factors were small in size, therefore the focus was on the main effects of the factors. Inspection of the COM results suggested that there were twoway, threeway, or fourway interactions of other factors, so the performance of the linking methods were described taking into account these interactions. This chapter consists of six sections. The first section compares the general performance of the four linking methods. The second section compares the four linking methods for different test structures. The third section compares linking methods for tests with different lengths. The fourth section compares linking methods for different sample sizes. The fifth section compares linking methods for groups with different ability distributions. The last section shows the relationship between scale linking performance and item parameter values. General Performance of the Different Linking Methods The performance of the four multidimensional IRT linking methods was first compared across all testing conditions by collapsing the means and standard deviations of a,, a2, and d for all items under different testing conditions. The histograms in Figure 31 show the distributions of means and standard deviations for al, a2, and d across all items and both test structures. Based on the percentage of items with the means and standard deviations of differences close to 0, Linki (direct method) produced more accurate and stable linking results than Link4 (item characteristic function method), and Link4 yielded more accurate and stable results than Link3 (test characteristic function method). Link2 (equated function method) did not provide accurate and stable results for a high percentage of items. The performance of the four linking methods was also examined separately for different test structures. Figure 32 shows the distributions of means and standard deviations for a,, a,, and d for APP and COM conditions. Comparing the histograms for the four linking methods on the left side of the figures, one can see that there was no apparent difference among the four linking methods for APP conditions. The histograms on the right side of the figures show that there was obvious difference among the four linking methods for COM conditions. Specifically, based on the accuracy and stability of linking function, (a) Linki (direct method) worked well, (b) Link2 (equated function method) worked poorly, and (c) the performance of Link3 (test characteristic function method) and Link4 (item characteristic function method) was between that of Linki and Link2, with Link4 being slightly better than Link3. In sum, Linki (direct method) was consistently the best method and Link2 (equated function method) the worst method under most COM conditions; the four linking methods worked equally and consistently well under most APP conditions. Performance of Linking Methods for Different Test Structures In this section, the performance of the four linking methods is compared between APP and COM conditions. Figure 32 shows different linking results for the two test structures. Based on the histograms for APP and COM conditions in the figure, all the four linking methods produced more accurate and more stable results for APP tests than for COM tests, but the difference in quality of linking varied across the linking methods. Specifically, LinkI (direct method) results were slightly better for APP tests than for COM tests, especially for parameters a, and a2; Link3 (test characteristic function method) and Link4 (item characteristic function method) results were much better for APP tests than for COM tests; Link2 (equated function method) yielded very poor results for COM tests, but good results for APP tests. However, for the large sample size (N = 2000), LinkI (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) worked almost equally well for APP tests and COM tests; the linking performance difference between APP and COM conditions still remained for Link2 (equated function method) due to its poor function for COM conditions. Figure 33 shows the results of four linking methods for APP and COM tests when the sample size is 2000. With smaller sample sizes (N = 500 and N = 1000), the linking performance difference between APP and COM conditions increased for Link3, Link4, and Linki (see specific histograms in the appendix). Therefore, test structure had its smallest effects on Linki (direct method), larger effects on Link3 (test characteristic function method) and Link4 (item characteristic function method), and the largest effect on Link2 (equated function method). Link2 worked well for all APP tests, but poorly for all COM tests in this study. Due to the strong influence of test structure on the function of the four linking methods, most of the results in the following sections are presented separately for the APP and COM conditions. Performance of Linking Methods for Different Test Lengths Given the different performance of linking methods for APP and COM conditions, the influence of test lengths on the linking function was explored separately for APP and COM tests. The distributions of means and standard deviations of differences for al, a2, and dfor short and long tests under the APP conditions are presented in Figure 34. Based on the histograms in the figure, one can see that for APP tests, although the linking performance was not strongly influenced by test length, all four linking methods produced slightly more accurate and stable results with long tests. Inspection of the results indicated that under COM conditions, the performance of the linking methods depended on the sample size and test length. Therefore, the influence of test lengths was next explored separately for different sample sizes for COM tests. Figure 35 shows the linking results for short and long tests with sample size of 500. Although none of the four linking methods worked well, the histograms still show that the results for Linki (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) for short tests were better than those for long tests and that Linki to some extent performed similarly for different test lengths. Figure 36 illustrates the linking results for short and long tests with sample size of 1000. From the figure, it was difficult to state at which test length linking performance was better. Subsequently reported results will show that the performance depended to some extent on the ability distribution difference between the base and equated groups. When the linking results for ability condition 2 (G2: unequal mean vectors) were excluded, the linking function for long test was obviously better than that for short test except for Link2 (equated function method), as presented in Figure 37. Therefore, in Figure 36, the performance under G2 masks the positive effect of test length on the linking accuracy and stability. Shown in Figure 38 are linking results for different test lengths with sample size of 2000. It is very obvious that the linking results for long tests were better than those for short tests except for Link2 (equated function method). In sum, the linking results for long tests were better than those for short tests except in some COM conditions when the sample size was small. Performance of Linking Methods for Different Sample Sizes Inspection of results indicated that sample size had stable and consistent influence on the linking performance, but with different degrees of influence for different test structures. Therefore, the effect of sample size is first shown across all other testing conditions then presented separately for APP and COM conditions. Figure 39 contains the linking results for different sample sizes. Comparing horizontally the histograms for different sample sizes, one can find that both the linking accuracy and stability increased for Linki (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) with the sample sizes changing from 500 and 1000 to 2000. However, the linking performance increased at different degrees for different test structures. Figure 310 shows the linking results for different sample sizes for APP tests. Figure 311 shows the results for different sample sizes for COM tests., From Figure 310, it can be found that the accuracy of the four linking methods was fairly good at all sample sizes and the stability of the four linking methods increased when the sample size became large for APP tests. Figure 311 suggests that although the accuracy and stability of Linkl, Link3, and Link4 increased when the sample size became large for COM tests, the linking performance was poor for sample sizes of 500 and 1000 especially for Link3 and Link4. In addition, for COM tests, the accuracy and stability for Link2 (equated function method) were very poor for all sample sizes and relatively unaffected by sample size. Based on these findings, (a) Linki (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) for APP tests were less affected by different sample sizes than were COM tests; (b) Linki (direct method) was less affected by sample sizes than were the other linking methods for COM tests. Performance of Linking Methods for Groups with Different Ability Distributions Inspection of the results suggested that the linking results for different ability distributions depended on other testing conditions. Therefore, the influence of ability distribution was first explored separately for APP and COM tests. Figure 312 shows the linking results for groups with different ability distributions under the APP conditions. Comparing horizontally the histograms across G1 (equal mean vectors, equal covariance matrices), G2 (unequal mean vectors, equal covariance matrices), G3 (equal mean vectors, unequal covariance matrices), and G4 (unequal mean vectors, unequal covariance matrices) indicates that: (a) for a, and a2, the linking results for G1 were slightly better than those for other ability conditions; (b) for d, the results for G2 were somewhat worse than those for other ability conditions; (c) Link2 (equated function method) was least affected by ability distributions. The results imply that a difference between groups in the mean vectors was more influential than a difference between the groups in the covariance matrices. Inspection of results for COM tests indicated that the influence of ability distribution was moderated by sample size; Therefore, the effect of ability distribution was explored separately for N=500, N=1000, and N=2000 for COM tests with the concentration on Linki (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method). Figure 313 shows the linking results for groups with different ability distributions with sample size of 500. One can see from the figure that although none of the linking methods worked well for the small sample size, the linking results for G2 (unequal mean vectors, equal covariance matrices) and G4 (unequal mean vectors, unequal covariance matrices) were worse than for G1 (equal mean vectors, equal covariance matrices) and G3 (equal mean vectors, unequal covariance matrices). Even though Linki (direct method) was relatively unaffected by betweengroup difference in ability distributions in groups, it still did not work well in linking d for G2, which indicates the strong influence of the mean difference between groups. The linking results for different groups with sample size of 1000 presented in Figure 314 shows that linking methods did not work well under G2, especially for d. However, there was some interaction between group differences and test length. For long test (n = 40), the linking methods did not work well for G2 (see Figure 315); for short test (n = 20), the linking methods worked relatively well for G2 (see Figure 316). The linking results for different groups with sample size of 2000 shown in Figure 317 suggest that the linking methods worked approximately equally well for the groups with different ability distributions. In sum, the influence of ability distributions on linking results depended on other testing conditions: (a) betweengroup differences in ability distributions did not have a strong influence on the performance of the four linking methods for APP conditions or for COM conditions with a large sample size; (b) mean difference between groups had negative influence on the linking results especially for conditions with small sample size. Performance of Linking Methods for Test Items with Different Parameter Values Two types of scatterplots were used to examine the relationship between linking performance and item parameter values under each of the 48 testing conditions. The first type of scatterplot was used to evaluate the effect of item parameter values on the accuracy of different linking methods, with y axis as the mean of the differences and x axis as the true parameter values which were used to generate the item response data. The second type of scatterplots was used to evaluate the effect of item parameter values on the stability of different linking methods, with y axis as the standard deviation of the differences and x axis as the true parameter values. These two scatterplots were constructed for each of the three parameter estimates (e.g., a,, a2, and d) under each of the 48 testing conditions. However, the results of Link2 (equated function method) were not included in the scatterplots under the COM conditions due to its consistently poor performance. Given the limitation of space, the main outcomes are illustrated by some representative examples. The results suggest that: (a) Under most of the testing conditions, the linking results tended to be less accurate for a, and a2 when the two parameters had extreme values, and (b) under most of the testing conditions, the linking results became less stable for a, and a2 as the parameters values increased. The results also indicate that: (a) The accuracy of linking results for d was not closely related to their true parameter values under most of the testing conditions, and (b) the stability of linking results for dwas also not closely related to their true parameter values when the sample size was not large. The scatterplots for one testing condition (COM, n = 20, N = 1000, G3) illustrate these relationships between linking performance and item parameter values (see Figure 318). However, for large sample size (N = 2000) the stability of linking results for d was closely related to their absolute true parameter values. Specifically when the absolute parameter values of d were closer to 0, the linking results were more stable; when the absolute parameter values of d were farther away from 0, the linking results were less stable. The scatterplots for another testing condition (APP, n = 40, N = 2000, G4) show that: (a) the linking results tended to be less accurate and less stable for a, and a2 when the two parameters had extreme values; (b) the linking results tended to be less stable for d as the absolute parameters values increased (see Figure 319). 100 80 60 40 20 0 100 80 BO 40 20 0 100 80 60 40 20 0 100 so 40 20 0 II 0.6 OA 02 0 0.2 OA 0.6 Mean of Diferences for al Ee '18 100 so 80* do 40 20 0 100 80 jo 40 20 0 100 80 *6o 40 20 0 100 80 40 20 0 Figure 31. Accuracy and stability for different linking methods 1! 0.1 03 0.5 0.7 0.9 1.1 1.3 Standard Deviation of Diferences for al eII 100 80 i60 40 20 0 100 80 so 40 20 0 100 80 s60 40 20 0 100 80 is0 40 20 0 I I 0.6 OA 0.2 0 0.2 OA 0.6 Mean of Differences for a2 1 II Ee '18 100 so 80 do 40 20 0 100 so jo 40 20 60 40 20 80 100 80 so 40 20 100 80 20 0 0.1 03 0.5 0.7 0.9 1.1 1.3 Standard Deviation of Diferences for a2 Figure 31. Continued a ~1~~ eII 100 I 80 BC 40 20 0 100 I I 80 40 20 0 I I 100 40 20 C 0.6 OA 0.2 0 0.2 OA 0.8 Mean of Diferences for d 1 II Ee '18 100 60 40 20 0 100 80 sol 40 20 0 100 80 60 40 20 0 100 80 40 20 0 Figure 31. Continued 0.1 03 0.5 0.7 0.9 1.1 1.3 Standard Deviation of Diflerences for d III IAPP I 100 Va o leo 1 o S40 20 0 100 E 80 40 20 0 100 P 40 20 0 100 S80 S40 20 0 OB OA 02 0 0.2 0.4 0.6 0. OA 0.2 0 Mean of DIferences fbr al 0.2 0.4 0.6 IAPP I COM 0.1 0.3 05 0.7 0.9 1.1 1.3 0.1 03 0.5 0.7 0.9 1.1 13 Standard Devialon of Differences for al Figure 32. Accuracy and stability by linking method and test structure COM I I I I I IAPP I 100 V 0 140 20 0 100 E 0 40 20 0 100 P 40 20 0 100 s 80 S80 S40 20 0 OB OA 02 0 IAPP I 0.2 0.4 0.6 0. OA 0.2 0 Mean of Dllerences fbr a2 0.2 0.4 0.6 cOM 0.1 0.3 05 0.7 0. 1.1 1.8 0.1 03 0.5 0.7 0. Standard Deviation of Differences r a2 Figure 32. Continued 100 I 80 S60 140 20 0 100 S80 so 140 20 0 100 S40 20 0 100 so 60 S40 20 0 1.1 13 I COM I I I I I IAPP I 100 1a 0 eeo S40 0 100 BO 60 a40 20 0 100 80 S80 I 40 20 0 100 1so S40 20 0 0.6 OA 02 0 0.2 0.4 0.6 0. OA 0.2 0 0.2 0.4 0.6 Mean of Derences for d IAPP I COM IF 0.1 0.3 05 0.7 0.9 1.1 1.3 0.1 03 0.5 Standard Deviation of Dllerences lor d 0.7 0.9 1.1 Figure 32. Continued 100 V 8o slo 1 640 20 0 100 a40 20 0 100 S40 20 0 100 80 S40 20 0 13 COM . IAPP I 100 I 80 Sleo 40 20 0 80 S40 20 0 100 s co S40 20 0 100 so 60 S40 20 0 08 04 0.2 0 IAPP I 0.2 0.4 0.6 0.6 0.4 0.2 0 Mean of Differences for al 02 0.4 0.6 cOM 0.1 0.3 0 0.1 0.3 05 I I I I 0.7 0. 1.1 1. Standard Deviation I I 0.1 0. of DIferences 0.5 0.7 0. 1.1 13 br al Figure 33. Accuracy and stability by linking method and test structure: N = 2000 100 I 80 e 60 140 20 0 100 80 S40 20 0 100 80 P 40 20 0 100 so 60 S40 20 0 I COM IAPP I 100 I 80 Sleo 40 20 0 80 100 seo S40 20 0 100 S80 S40 20 0 100 so 60 S40 20 0 08 OA 0.2 0 IAPP I 1 0 0.1 0.8 05 0.7 0.2 0.4 0.6 0.6 0.4 0.2 0 Mean of Dfferences for a2 0.2 0.4 0. cOM 0. 1.1 1. 0.1 03 0.5 Standard Deviation of Dfferences r a2 0I I I I1 0.7 0.9 1.1 1i Figure 33. Continued 100 I 80 e 60 140 20 0 100 eso S40 20 0 100 S40 20 0 100 so 60 S40 20 0 I I ii COM IAPP I 100 I 80 e eo 40 20 0 80 100 seo 140 20 0 100 S80 140 20 0 100 60 S40 20 0 0. A0 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dlfferencs for d IAPP I I  I 0.1 0.3 05 II LI 0.7 0.9 1.1 1.3 0.1 0o 0.5 Standard Deviation of Dflerences for d I 0 I 0.7 0.9 1.1 1i Figure 33. Continued cOM I 100 a 60 140 20 0 100 40 20 0 100 S40 20 0 100 60 S40 20 0 ii COM 20 1 100 I 80 Sleo 140 20 0 100 eso J 80 S40 20 0 100 s 6o S40 20 0 100 s 80 60 S40 20 0 0. OA 0.2 0 0.2 0.4 0.6 0.0.4 0.2 0 0.2 0.4 0.6 Mean of Differences for al 140  0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7 Standard Devialon of Dfferences br al 0. 1.1 13 Figure 34. Accuracy and stability by linking method and test length for approximate simple structure tests 100 I 80 e 60 40 20 0 80 S40 20 0 100 140 20 0 100 so 60 S40 20 0 140   120 1 100 Va o 1 40 20 0 100 160 a40 20 0 100 P 40 20 0 100 s 80 S80 S40 20 0 OB OA 02 0 0.2 0.4 0.6 0.6 OA 0.2 0 0.2 0.4 0.6 Mean of Dllerences fr a2 140 100 I 80 S60 40 20 0 100 S80 40 20 0 80 40 20 0 100 so 60 S40 20 0 1.1 13 0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7 0. Standard Deviation of Dfferences br a2 Figure 34. Continued 140  120 1 100 I 80 60eo 140 20 0 100 60 140 20 0 100 s ao P 40 20 0 100 so 60 S40 20 0 0.6 OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dflerences for d 140 100 I 80 o60 140 20 0 100 Eso S40 20 0 100 S40 20 0 100 so 60 S40 20 0 0.1 0.3 05 0.7 0. 1.1 1. 0.o 003 0.5 0.7 0. Standard Deviatn of Diearences for d Figure 34. Continued 1.1 1l 140  100 I 80 e eo 40 20 0 0 100 S40 20 0 100 140 20 0 100 so 60 S40 20 0 08 OA 0.2 0 U nn 0.2 0.4 0.6 0.6 0.4 0.2 0 Mean of Differences for al 02 0.4 0.6 140  100 I 80 e 60 40 20 0 100 60 140 20 0 100 S40 20 0 100 s 80 S60 S40 20 0 l 0.3 05 0.7 0. 1.1 1. 0.1 03 Standard Devialon of Differences 0.5 br al 0.7 0. 1.1 Figure 35. Accuracy and stability by linking method and test length for complex structure tests: N= 500 l 0.1 I I I I I I I I I I I 40  a 100 I 80 o60 40 20 0 100 J 80 seo 40 20 0 100 S40 20 0 100 so 60 S40 20 0 140 l  I I  0.1 0.3 05 0.7 0. 1.1 1.8 0.1 03 0.5 0.7 0. 1.1 13 Standard Deviation of Differences r a2 Figure 35. Continued I A 0B OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dfferences for a2 100 I 80 o60 40 20 0 100 80 seo 860 140 20 0 100 60 140 20 0 100 so 60 S40 20 0 l II L~L m m m m 20 1 a 100 I 80 e eo 40 20 0 100 Seso 140 20 0 100 S40 20 0 100 so 60 S40 20 0 0.1 0.3 05 0.7 a a 0.2 0.4 0.6 0.6 0.4 0.2 0 Mean of Dflerences for d 0. 1.1 1. 0.1 03 0.5 Standard Deviatn of Diffrences for d 0.2 0.4 0.6 0.7 0. 1.1 Figure 35. Continued 08 OA 0.2 0 100 V 80 l 60 140 20 0 100 60 140 20 0 100 S40 20 0 100 s 80 S60 S40 20 0 140 L  I I JL I 114l 20 1 100 I 80 Seo 140 20 0 80 100 S40 20 0 100 s co S40 20 0 100 so 60 S40 20 0 ItI 0. OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Differences for al 140  100 I 80 o60 40 20 0 100 60 140 20 0 100 S40 20 0 100 so 60 S40 20 0 0.1 0.3 05 0.7 0. 1 1. 1 0.1 03 Standard Devialon of Differences 0.5 0.7 0. 1.1 1 br al Figure 36. Accuracy and stability by linking method and test length for complex structure tests: N= 1000 140  I I I I I  100 I 80 leo 140 20 0 80 100 seo 140 20 0 100 S80 240 20 0 100 so 60 S40 20 0 120 1 1r Il 0.6 A0 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dfferences for a2 40 100 V 80 slo 140 2 40 0 100 160 a40 20 0 100 80 0 1 40 20 0 0.1 0.3 0.5 0.7 0.9 1.1 1.3 0.1 03 0.5 0.7 0.9 Standard Devlation of Differences for a2 1.1 1s Figure 36. Continued 140  120 1     100 I 80 Seo 140 20 so 0 100 140 20 0 100 S40 20 0 100 s 80 60 1 40 20 0 140 0.1 0.3 05 0.7 09 1 1.3 1 0.1 0 5 Standard Deviatln of Dellrences for d 0.7 0. 1.1 Figure 36. Continued 0B OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dfferences for d 100 I 80 o60 40 20 0 100 80 so 140 20 0 100 S40 20 0 100 so 60 S40 20 0 25 140 L C  m 20 1 100 I 80 Seo 140 20 so 0 100 140 20 0 100 S40 20 0 100 s 80 60 1 40 20 0 08 OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 Mean of Differences for al 0.2 0.4 0. 140  0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7 Standard Devialon of Dferences br al 0. 1.1 12 Figure 37. Accuracy and stability by linking method and test length for complex structure tests when G2 was excluded: N=1000 100 I 80 S60 140 20 0 100 80 so 140 20 0 100 S40 20 0 100 so 60 S40 20 0 140   120 1 1r L~Z 100 I 80 leo 140 20 0 100 60 S40 20 0 100 S80 240 20 0 100 so 60 S40 20 0 140 0.1 0.3 05 0.7 0. 1.1 1.8 0.1 03 05 0.7 0. Standard Deviation of Differences r a2 1.1 12 Figure 37. Continued 0.6 OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dfferences for a2 100 I 80 o60 40 20 0 100 ,e4O 60 140 20 0 100 180 P 40 20 0 0 140  120 1 z a 100 I 80 Seo 140 20 0 100 S80 so 140 20 0 100 s co S40 20 0 100 60 S40 20 0 140 0.1 0.3 0.5 0.7 0.9 1.1 1.8 0.1 03 0.5 Standard Deviation of Dilerences for d I7 I I I 0.7 0.9 1.1 1i Figure 37. Continued 0. A0 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dfferences for d 100 0so slo 140 2 40 0 100 160 a40 20 0 100 80 0 1 40 20 0 140  20 1 100 I 80 Seo 140 20 0 80 100 S40 20 0 100 s co S40 20 0 100 so 60 S40 20 0 0. OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Differences for al 140  100 I 80 o60 40 20 0 100 ,e o 60 140 20 0 100 S40 20 0 100 so 60 S40 20 0 I  0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7 0. Standard Deviatlon of Dfferences br al 1.1 12 Figure 38. Accuracy and stability by linking method and test length for complex structure tests: N = 2000 140   120 1        100 V 80 S60eo S40 20 0 100 J 80 S60 140 20 0 100 s 6o S40 20 0 100 s 80 60 S40 20 0 140 0.1 0.3 05 0 0.1 0.8 05 0.7 0. 1.1 15 0.1 03 0.5 0.7 0.9 Standard Deviation of Differences r a2 Figure 38. Continued 0. OA 0.2 0 0.2 0.4 0.6 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of fferences for a2 100 I 80 o60 140 20 0 100 eso S40 20 0 100 140 20 0 100 so 60 S40 20 0 1.1 13 140 I  I II 120 1  I  i 1_ 100 V 80 Sleo 140 20 0 100 80 140 20 0 100 s co S40 20 0 100 so 60 S 40 20 0 0.2 0.4 0.6 0.6 0.4 0.2 0 Mean of Dlfferences for d 0.2 0.4 0. 140 Li0.1 0.3 05 1 I 0.1 0.3 05 0.7 0. 1.1 15 0.1 05 05 0.7 0. Standard Deviatin of Diffrences lor d Figure 38. Continued 0B OA 0.2 0 100 I 80 e 60 140 20 0 100 80 140 20 0 100 S40 20 0 100 so 60 S40 20 0 1.1 12 140  1000 I so Oh 0.2 100 t 80 100 so 40 20 0 100 S80 s0 40 20 0 100 80 s0 S40 20 0 100 80 60 I 40 20 0 I I O. 0.6 I 0.8 0.2 0.2 0.6 Mean of DIerences for al Jl000 LI ]0. 0.6 1500 I I I I I I I I I I I I I I I 0.1 0.5 i 11000oo II II II 0.8 13 0.1 0.5 0.9 1.3 Standard Devialtn of Dlferences for al LI LI 0.1 0.5 0.9 1.3 Figure 39. Accuracy and stability by linking method and sample size I I 02 0.2 100 t 80 s0 40 20 0 100 80 80 40 20 0 100 80 o0 S40 20 0 100 80 60 P 40 20 0 I I a 11100011200 I so0 100 40 20 0 100  80  40 20 100 0 0 20 0100 02 I5o 100  t 80  40 20 100 80  100 40 0 100 80 80 _ 100 80  60 S40 20 0.1 05 ~I ] 02 0.6 0.6 02 0.2 0.6 Mean of Differences for a2 111000 I I I I I I I I I I 1I [ [ ii I I I I I I 0.6 02 0.2 0.6 11I2o I I I I I I I I I I I uL I I I I I I I I I 1 0.1 0.5 0.9 1.3 0.1 0.5 0.9 1.3 Standard Deviatlon of Differences for a2 Figure 39. Continued i tI I "II .  & 1 mi Ito i i 111000 112000O i ,L 100 t 80  0  40 20 Oh 0.2 ILI I JLw 0.2 0.6 0.6 0.2 0.2 0.6 Mean d DIIerences for d ]LI 0.6 I 2 0.2 0.2 0.2 Ism lI 110 I I I I I I _ I I 0.1 0.5 0.9 13 0.1 0.5 0.9 1.3 Standard Deviation d DIerences for d = I 11111 zzz 0.5 0.0 1.3 Figure 39. Continued 100 t 80 20 40 20 0.6 I 500 100 80  0  40 20 0 100 80  0 40 20 0 100 80 0 100 80 60 40 20 0  0 [ I[ Il 0I I 02 0.6 1000 I I 0.6 02 0.2 0.6 Mean of DIerences for al JL Iloon I I I , I __]I 0.1 0.5 0.9 13 0.1 0.5 0.9 1.3 Standard Deviatin of DIIerences for al 2m I LI LI I I I 0.1 0.5 0.9 1.3 Figure 310. Accuracy and stability by linking method and sample size for approximate simple structure tests 0.6 0.6 0.2 I ,02 02 0_2 0.2 S1500 100 80  40 20 0 100 80 40 20 100 80 40 0 100 80 20 0 I I 1500 so so 40 20 100  80  so0  40 20 0 100 80  so60  40 20 0 100  so 40 20 0 0 0 [ I[ I[ 0I I 02 0.6 1000 I zII 0.6 02 0.2 0.6 Mean of Differences for a2 I 0.8 J L2oon 100  BO I 80 100  40 20 I 100 80 40 0.1 0.5 0. 1 St II MOI II I I I IB I 0.1 0.5 0.9 1.3 andard Deviatlon of Differences fr a2 0.1 0.5 0.9 1.3 Figure 310. Continued I 02 0.2 0.6 02 I. 0.6 I12ooo i limo 1200 I I I I I I I 02 0.6 0.6 02 0.2 0.6 Mean d Dlerences for d 111000 100  loo t 80  So0  40 20 0 0.8 02 0.2 0.6 11oo I 100  t80 40 20 0 100 E 80  40 0 100 j40 20 100 80~ 20 0 L^^^ itL I I I I I I I I I I L 0.1 0.5 0. 13 0.1 0.5 0.9 1.3 Standard Devialton d Dlerences for d Figure 310. Continued 0.1 0.5 0.9 1.3 Il1000 O. 02 I 0o I I I L I I  I mm I B ~I 11I 112000 IL 1500 1100120   a 0.6 02 0.2 0.6 Mean of DIerences for al 111000 1I I I I I I I 0.8 02 0.2 0.6 112o0 t80I 20 I I 80 60[ 100  o 0.1 0.5 0.9 13 0.1 0.5 0.9 1.3 0.1 0.5 0.9 1.3 Standard Devialti of DIlrences fr al Figure 311. Accuracy and stability by linking method and sample size for complex structure tests 100  t 80  80  40 20 0a 100 E 80  0  40 on  I I I I I I 2 Oh 0.2 I I 0.2 0.6 I 500o Ij I I I I M I I I L I 50so 11 1000 112000o I I I I i_ _I_ ""I' 100 t 80 0  20  100 0 80  40 ! 100 80 O0 S40 20 0 100 80 60 S40 20 0 iii O. 02 0.2 0.6 aao n 0.6 0.2 0.2 0.6 Mean of Differences fr a2 11000 I I I I I I I 1I C G r I I I I I I 0.8 0.2 0.2 0.6 112000 100 80  0  40 20 0 100 80 0  40 20 0 100 80  60  40 20  0 80 I I I I I I I _I IIIJLII 'Es I I I I I I I I I 13 0.1 0.5 0.9 1.3 0.1 0.5 0.9 1.3 Standard Deviallon of Differences for a2 Figure 311. Continued I' as I I 0 111000 1120oo00 1 I ii 100  t 80  So0  40 20 100 So 20  Ill"" 100 I so 60 I SI40 o0 02 0.2 0.6 112000 S M E I 0.8 I I 0.2 0.2 Mean d Dlerences for d Ioo 111000 I I I I I I 0.8 0.2 0.2 0.6 112000 I I I I I I I It. I I 0.9 13 0.1 0.5 0.9 1.3 Standard Deviatlon d Dlerences for d 0.1 0.5 0.9 1.3 Figure 311. Continued I1. 0.1 0.5 I I I I II ,, r ~I I I I I I .I Im 1 ! I   . .  IG1 G l2 O G4 100 0 40 20 40 20 0 gQ I I __  100I I I I 00 I  S0 I I 40 100 I I I I 20 100 I I I I 0.1 05 09 1. 0.1 0 0. 0.1 0.35 .0 13 0. 1 1 0.5 0. 1. Standard Deviaton of DIerences for al Figure 312. Accuracy and stability by linking method and group for approximate simple structure tests iG G2 ll iG4 ii L A iII 0.6 0.2 02 OB 0.6 0.2 0.2 0.6 0.6 02 0.2 0.6 0.6 02 02 0S Mean of Dilerences for a2 100 5I I I I 40 100I I I I0 I I I 20 00  0 100 0 I I I so w:3www 0.1 05 09 1.3 0.1 0.5 0.9 1.3 0.1 0.5 0.2 13 0.1 05 09 1.3 S'andeard Devialon of Dierences far a2 Figure 312. Continued 20 IG1 11 2 11 l I Mean dl Dllfeences for d 100 40 a 100 80I I 20 0 100  0.1 05 02 1.8 0.1 0.52 0.20.8 0.8 02 0.2 0.8 O 02 02 Os Standard Deviation D Ierences far d 0Figure 312. Continued so 20 100 0 0.1 0. 0.9 1.3 0. 0.1 1. 6 0.1 0. 0. 9 1. 0.i Oh 0.1 105 Standard Deviation 1 DIlrences fo r d Figure 312. Continued IG1 11 2 11 l I lo 100 I I0I 1 l 40 100 20 0 100 0. 02 02 OS 1 0. 0.2 0.2 O. 0. 0.2 0.2 0.8 0. 02 0.2 O. I I I I 20 I I I I 0 00 500 20  0.1 0. 0.2 18 0.6 0.2 0.2 0.8 0.1 02 0.2 0. 1 OA 0. 0.2 0. Mman a f DM Irences tfr al M I I I 20 N = 500 II 11 11 Il I lo 20 100 I i 80 I I I 0 I 40 20 40 100 I 0 100 0. 0 02 O1.3 0.6 0.2 0.2 0.6 0. 2 0.2 0.6 0. 02 0. 1.3 MeStan dard Deviaon of Differences r e 31. C I I 20 100 I i I I 0 100I I I I 20 60 i Figure 313. Continued IG1 11G 2 lG4 I 100 i Sg I 40 100 I 80  40 0.6 02 02 OB 0.6 0.2 0.2 0.6 0.6 02 0.2 0.6 06 02 02 Oh Mean d Dilbrences far d 20 0 100 I I I I I I 100 I 20 I I I I 20 100I I I I l0 t I 80 60 0.1 05 09 1.3 0.1 0.5 0.1 1.8 01 0.5 0. 1 0. 5 O0. 1.3 Standard Devialton d DIlrences for d Figure 313. Continued G1 G2 O G4 100 i i wz 80w 20 0 100 0. 02 0.2 OS 0. 0.2 0.2 O. 0.8 0.2 0.2 0.8 Oh 02 0.2 O.h Mean of D ibrences for al 20 I 0 100 s0 100 I 0. 05 02 1. 0.1 0.5 0.2 0.6 0.6 02 0.2 0. 0. 092 0 1. MSt an ofat Dll fences for al el G2 S A I 20 0 N 1000 Bo I I 20 N = 1000 G1 G lG2 IG4 I 100  10  40 100  SI 80 20 I 0 100 i i iww I I I I I I 60 0.6 0.2 0.2 0 0.6 0.2 0.2 0.6 0. 0.2 0.2 0.8 0. 02 0.2 0. Mean of Differences for a2 Igr Iw GS 11Gnue SOI I I I 20 0! so OI I I I 20 I i I I so 20 IL 100 I I I I 0o 80 0.1 0.5 0 1.3 0.1 0.5 OA 1.3 0.1 05 0. 1 0.5 09 1.3 Standard Deviallon of DlfAerences for a2 Figure 314. Continued G1 G 12 G4 I o 40 1 I I I I I I I S100 1 a 20 40  i0 iW I ] W 20 O I I I I I II I I I I I 40 Mean d Dierences for d F00 14I I I IC n 80I I 40 20 40 20 100 0.1 0. 0 1.6 0.1 0.5 0. 1.3 0.1 0.5 0.9 10 0.i 0. 0 1.3 Standard Dvialn d D~llenances for d Figure 314. Continued Figure 314. Continued 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Dllfrences for al 0.6 0.4 0.2 0 0.2 A 0.6 Mean of Dferences for a2 I , EI i 0.6 0.4 0.2 0 0.2 .A 0. Mean of Difrences for d so 0 80 40 0 80 40 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Diferences for al 40 80: 40 II 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dferences or a2 0 0 0 0 ,0 80 40 0.1 0.3 0.5 0.7 0.9 1.1 1.3 Standard Devlaian of DIlfrences fr d Figure 315. Accuracy and stability for different linking methods: COM, n=40, N=1000, G2 0.6 0.4 0.2 0 0.2 0A 0. Mean of Dfferences for al 0.6 0.4 0.2 0 0.2 0.4 B Mean of Dfferences for a2 40 .. .... 40 0.6 0.4 0.2 0 02 A0.4 O Mean of DIffrences for d 80 40 0 I 0 a0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences fr al 40 0 80 0 I80 0.1 0.8 05 0.7 0.9 1.1 1.3 Standard Deviation of Dfferences tr a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Figure 316. Accuracy and stability for different linking methods: COM, n=20, N=1000, G2 IG1 11 2 11 l I 40 100 i I I 84E I I I 20 S40 10 S0 I I I I I 0.6 02 02 OB 0.6 0.2 0.2 0.6 0.8 02 0.2 0.8 06 02 02 06 Mean of Dllerences for al Ie 11G 12 G CA I 100 I I I I 40 20 0 I1 1 1 1 100 I I 2100  20 i so 0.1 05 09 1.3 0.1 0.5 0.a 1.3 0.1 0.5 o0. 13 0.1 05 09 1.3 Standard Deviaoln of Dllferences for al Figure 317. Accuracy and stability by linking method and group for complex structure tests: N = 2000 100 W lW W 40 100 I I I Mean of Dlirerences for a2 8EI I I I 40 S0 100 20 0. 02 0.2 0 0.8 0.2 0.2 0. 0.8 02 0.2 0.8 Oh 02 02 Oh Mean of D differences fbr a2 IFg11. 117 Cnl I I I I I 0 100I I I I 60 20 0 100 I 0.1 05 0.9 1.3 0.1 0.5 0.9 1.3 0.1 0.5 0.9 12 0. 05 0.9 1.3 Standard Deviallon of Diferences fo r a2 Figure 317. Continued IG1 11 2 11 l I 100 40 a 100 I I I I I I I I 2 0 o SIIso I I 0 0 I I I I I II II II I 10 100I 0 I I I 60 1I I I I 20 0 1 I ,I I I 1 100 I I I I I : I : I I : I I I I I I I I 20 0.1 05 09 1.3 0.1 0.5 0.g 1.3 0.1 0.5 0.9 15 0.1 05 09 1.3 Standard Devialton d Dllrences for d Figure 317. Continued Mean of Differences 0.17 0.12 0.07 0.07 A 0.02 ** . ..    0.03 8A 0 13 0.08 A 0.138 D 0 0.18 0.23 028 0.28 ,0 *** Unki AA.A Unk3 0 D 0 Unk4 0 1 2 3 Parameter Value: al SDof Differences Unki A A A UnkS O ULnk4 0.8 0.7 * 0.6 0.5 A * OA 0.3 A A D 0.2 0.1 0 1 2 3 Parameter Value: al Figure 318. Linking accuracy and stability and item parameter values: COM, n=20, N=1000, G3 116 Mean of Differences 0.12 0.07 0.02 0.03 0.08 0.13 0.18 0.23 n 0.28 0.28 0.8 m 0.1 0.2 0.8 04 0.5 0.6 0.7 OB 09 1.0 1.1 1.2 1. 1.4 15 1.6 1.7 1.8 19 2.0 2.1 Parameler Value: a2 SOD Differences 0.8 0.7 0.6 OA 0.2 0.2 0.1 0.1 0.2 0 OA0 '1' '1' I ' ''' I .'.' I''1 I''' 1 I '''.1' '1 '' '1 'I''1 1 1 1 ' 0.5 0.6 0.7 02 09 1.0 1.1 1.2 1. 1.4 15 1.6 1.7 1.8 19 2.0 2.1 Parameter Value: a2 a B 6 El A AA E S D SUnk nknk * Unki AA"A UnkS DO Link4 A S I Figure 318. Continued Mean of Differences 0.14 0.13 0.12 0.11 0.10 0.09 0.08 A 0.07 0.086 0.05 0 0.04 0.03 *A 0.02  0.01 a 0.01 0.02 O ** Unki AA Unk8 Do UInk4 0.03 3 2 1 0 1 2 Parameter Value: d SD f Differences 07 Unk AA Unlk D O L nk 0.6 0.5 0.4 6 0.8 OAO 0 0.2 0.1 * 0.0 3 2 1 0 1 2 Parameter Value: d Figure 318. Continued 118 Mean of Difference 0.10 0.09 0.08 0.07 0.08 0.05 0.04 0.03 0.02 0.01 .h. . 0.00     0.01 0.02 8 a 0.043 0.05 0.06 0.07 0.08 *0 UnkI *** I.nlk2 AA A InS 0 3 Unk4 0.09 0 1 2 3 Parameter Value: al SOof Differences 0 .e UnkI ** hIk A A Unlk U Linknk4 0.32 0.30 * 0.28 0.28 0.24 0.22A 0.20 * 0.18 0.16 0.14 0.121 Ns 0.10 f 0.08 0.068 0 1 2 3 Parameter Value: 81 Figure 319. Linking accuracy and stability and item parameter values: APP, n=40, N=2000, G4 119 Mean of Differences 0.03 0.02 0.01 0.00 0.01 0.02 0.03 0.04 0.05 0.06 Parameter Value: a2 SDo Dillerences 0.6 0.5 O  a 0.4 0.9 0.2   0.1 0.0 Parameter Value: a2 Figure 319. Continued a A A A   A    a A * * ** * Uink *** i Unk2 AAA UinkS Unk4 * nkl *** Link2 A A Link3 D D Unk4 I a~~ rjs a Mean of DIIerences 0.05 0.04 0.03 :* 0.021 0.01 A 0.02 Ii ** Lnlk2 Ln nk4 0.08 fl  . I .. I .. Il* ^ 2 1 0 1 2 0 Aink Lnk A inkS . Pamme:r Value: d 0.142 0.02 S*** Unl^ ***n Unk2 AAA ELln l Urnk4 0.103  S oo 0.12 D 0.12 [ ;A 0.11 2 1 1 2 Parameter Value: d 0.10 D U 0.0 A I0.07' I A E Paramet:r Value: d Figure 319. Continued 121 CHAPTER 4 DISCUSSION By using simulated data, the performance of the four multidimensional IRT scale linking methods was evaluated under different testing conditions, which include different test structures, test lengths, sample sizes, and ability distributions. The results illustrated in Chapter 3 suggest that test structure had a strong influence on the performance of the four linking methods. For approximate simple test structure, each of the four linking methods worked approximately equally well under all testing conditions. For complex test structure, the equated function method did not work well under any testing conditions; the performance of other three linking methods depended on different testing conditions; the direct method was the best linking procedure for most testing conditions. In addition, the item parameter values influenced the linking performance. The results are discussed in this chapter by seven sections. Results from Previous Studies Theoretically, there are at least two main components in linking errors: error caused by parameter estimation and error produced by scale transformation (Li, 1997). A simulation study (Kaskowitz & Ayala, 2001) found that linking was more accurate when there was less error in the item parameter estimates. Therefore, it is important to review previous studies about IRT parameter estimation and linking accuracy under different testing conditions, although it is difficult, if not impossible, to decompose the parameter estimation error from the linking error in testing practice. Based on Li's review (1997), the following factors can cause error in parameter estimation in IRT: (a) Examinees' ability distribution. Item difficulty for easy and hard items will not be well estimated when the examinees are normally distributed around their mean; Examinees with ability levels above or below the item difficulty are more informative for estimating item discrimination parameter; (b) Item parameter value. Item difficulty parameters that are small or large and discrimination parameters that are small or large produce larger estimation error; (c) Sample size. Larger sample sizes reduce estimation error. However, the standard error of parameter estimates depends on the combined effect of these factors (Thissen & Wainer, 1982). Using the bias and RMSE between the transformed linking parameter estimates and the true linking parameters across replications as the criterion, Li (1997) found that the linking accuracy of his three methods improved as sample size or test length increases. In the second study, Li (1997) used the bias and RMSE between the transformed item parameter estimates and the true parameter values across replications as the criterion and found that one of his linking methods (e.g., the combination of procrustean rotation approach for dimensional transformation, the ratio of trace procedure for dilation, and the least square procedure for translation) produced accurate linking of items. In addition, the positively skewed distribution of the second dimension in equated group did not negatively influence the linking accuracy and stability. To evaluate the performance of the four multidimensional IRT scale linking methods, Oshima et al. (2000) used different criteria, including mean and standard deviation of the linking parameter estimates over 20 replications, bias and mean square error (MSE) between the estimated and true linking parameters, correlation and mean absolute difference of linking parameter estimates across different methods, and minimized function values by different methods. The results indicate that: (a) The direct method and equated function method tended to yield similar linking results and the test characteristic function method and item characteristic function method tended to produce similar results, (b) the test and item characteristic function methods were more accurate and stable than the other two methods, and (c) the accuracy and stability decreased as ability differences between the groups increased. Min (2003) used the bias and RMSE between transformed item parameter estimates and the initial item parameters across the common items as the criterion to compare Li's (1997) composite procedure, Oshima and colleagues' test characteristic function method, and Min's extended composite procedure. Based on the repeated measures analysis of variance for bias and log transformed RMSE, The author found that: (a) The ability distribution, test structure, and linking method accounted for large portion of the variation in bias for discrimination parameter estimates but only linking method was an important factor for the variation in bias of difficulty parameter estimates, (b) the sample size, ability distribution, and linking method were important for linking stability of discrimination parameter estimates and sample size and linking method were critical for linking stability of difficulty parameter estimates, (c) as the sample size became larger and the two groups were more similar, the linking results became more accurate and stable, and (d) linking methods had significant interaction effects with testing conditions. In sum, the linking methods and the three testing conditions, e.g., the ability distribution, test structure, and sample size, significantly affected the linking accuracy and stability. Effects of Different Test Structures In the present study, the performance of all four linking methods worked much better for APP than for COM tests. This is consistent with the fact that the test structure and item parameters are typically more easily and accurately estimated for APP test than for COM test. As Tate (2003) found and discussed in a study comparing different estimation methods, including NOHARM, for assessing the test structure of item responses, the default rotation methods in exploratory analysis are usually developed to transform the initial solution to simple structure, therefore the procedures may not always successfully describe nonsimple test structure. A study (Gosz & Walker, 2002) comparing the performance of TESTFACT and NOHARM found that the item parameter estimation of NOHARM depends heavily on the number of bidimensional items in the test with its better performance for fewer bidimensional items and worse performance for more bidimensional items. In addition, NOHARM is good at estimating items with very low values on one discrimination parameter and high values on the other discrimination parameter. In this study, for APP tests, all items had higher values on one discrimination parameter and lower values on the other discrimination parameter; for COM tests, half of the items had approximately similar values on both discrimination parameter values. An investigation of item parameter estimation for the simulated data used in the present study indicated that the NOHARM program provided better item parameter estimation for APP tests than for COM tests. The superior estimation for the APP tests is likely the source of the superior linking results for the APP tests. However, in a study (Min & Kim, 2003) comparing Li's composite procedure and Oshima and colleagues' test characteristic function method under different testing conditions, no apparent linking difference was found between APP and COM tests (see Figure 27, Min & Kim, 2003). One possible reason is that the item's loadings on the two dimensions in COM tests in this study were more similar than those in Min and Kim's study. Specifically the heavily cross loaded items in their study had the direction of 50 65 and 25 40, and the direction of heavily crossloaded items in this study was selected from a normal distribution with mean of 450 and standard deviation of 100. According to the finding by Gosz and Walker (2002), the item parameter estimates for COM tests in this study were less accurate, so that the linking results were more different between APP and COM tests. Another possible reason is related to the different criteria used to describe the linking performance. This study used the percentage of items with different means and standard deviations between the item parameter estimates for the base group and the transformed item parameter estimates for the equated group over 500 replications to evaluate linking results. Min and Kim's study (2003) used the bias and RMSE between true parameter values and transformed parameter estimates over both 20 items and 50 replications, which may have difficulty in identifying the differential influence of APP and COM tests on the linking results. As shown in the Results chapter, the linking results (except for equated function method) were very similar for APP and COM tests when the sample size became large (N = 2000). This may be related to the possible improved item parameter estimation for larger sample size for both APP and COM tests. However, the attribution of different linking performance for the two types of tests to estimation error needs to be investigated by more controlled studies in the future. Effects of Different Test Lengths It was illustrated in the last chapter that the linking results for long tests were typically better than those for short tests, which is consistent with Li's finding (1997) that the linking accuracy of his three methods improved as test length increases. This result was not unexpected since more items can provide more information to set up the linkage between the scales for the base and equated groups. The positive effect of large number of items on linking and equating performance has already been found in various unidimensional equating conditions (Budescu, 1985; Fitzpatrick & Yen, 2001; Kaskowitz & Ayala, 2001; Kim & Cohen, 2002; Peterson, Cook, & Stocking, 1983; Swaminathan & Gifford, 1983; Wingersky, Cook, & Eignor, 1987). Therefore, this effect of the number of items can be extended from unidimensional to multidimensional linking and equating situations. However, there was an exception that the linking results for short COM tests were better than those for long COM tests when the sample size was small (N = 500). Li could not find this exceptional result because he used sample sizes of 1000, 2000, and 4000 in his study. One possible reason for this exceptional result is that small sample size was not large enough to produce accurate item parameter estimates for long test because more item parameters needed to be estimated, which accordingly affected the linking performance for long COM tests. Therefore, the strength of large number of items in scale linking and equating depends on the quality of the item parameter estimation, which in turn requires enough sample size. Lord (1980) stated that it is test length in combination with sample size that affects the quality of parameter estimates. Compared with unidimensional IRT models, a larger number of examinees are required for MIRT calibration because more parameters need to be estimated. In addition, this study found that the effects of test length on scale linking performance also depended on the ability distributions for the two groups. As described in Results chapter, the long test (n = 40) did not improve linking performance when the means of ability distributions were different for base and equated groups for COM test when the sample size was 1000. This phenomenon confounded the general positive effect of large number of items on linking results. Klein and Kolen's study (1985) suggests that test length has little effect on the equating quality when groups are similar in ability, but becomes very important when two groups differ in ability level. They found that a larger number of common items did improve equating when groups were dissimilar. However, the exceptional result from this study mentioned above did not confirm their finding. Further studies are needed to examine the conflicting findings by controlling more conditions. Effects of Different Sample Sizes Based on the results from this study, the effects of sample size were very obvious and straightforward. Generally speaking, the linking accuracy and stability improved with the sample size increasing. This is consistent with the fact that large sample size can improve item parameter estimates. The same pattern was also found in the other two multidimensional scale linking studies (Li, 1997; Min & Kim, 2003). In addition, the positive effect of large sample size has also been found in unidimensional linking and equating studies (Fitzpatrick & Yen, 2001; Hanson & Beguin, 2002; Kim & Cohen, 2002; Peterson, Kolen, & Hoover, 1989; Ree & Jensen, 1983). However, the linking performance improved at different degrees for APP tests and COM tests. The performance of direct method, test characteristic function method, and item characteristic function method increased much more rapidly for COM tests than for APP tests when the sample sizes became larger. In fact, the linking results for APP tests were consistently good for different sample sizes. However, the linking results for COM tests were very different for different sample sizes, although the linking function improved with the sample sizes increasing. This result was not found in Min and Kim's study (2003). They showed similar effect of sample size on linking accuracy and stability for APP and COM tests (see Figure 27, Min & Kim, 2003). As we discussed for the effect of test structures on linking performance, this may be related to the different manipulations of COM test items and different evaluation criteria used in these two studies. Effects of Different Ability Distributions Based on this study, for all APP conditions and the COM conditions with a large sample size, betweengroup differences in ability distributions did not have a large influence on the performance of the four linking methods. For COM conditions with small and medium sample size (N=500, N=1000) betweengroup differences in mean ability had a negative influence on the linking results. It seems that mean difference was more important than variance difference. These results were consistent with what Oshima et al. found in their study using very similar ability conditions (see Table 5 and Figure 1, Oshima et al., 2000), although they did not divide tests into APP and COM tests. However, we need to be very cautious about the possible differential effect of mean and variance differences on scale linking in both studies because they were controlled at different degrees, with mean difference at 0.5 and the variance difference at 0.2. Based on the study by Min and Kim (see Figure 27, Min & Kim, 2003), it seems that the influence of ability distributions on scale linking by the test characteristic function method was approximately similar for APP and COM conditions (see the above explanation for possible reasons for this conflicting findings between their study and this study). However, they did find that the influence of betweengroup differences in ability distribution on scale linking depended on sample size, with less influence for large sample size (N = 2000) and more influence for small sample size (N = 500). This is consistent with the results from this study. Li (1997) used a different manipulation of the betweengroup difference in ability distribution than was used in the present study: for the base group both ability distributions were normal; for the equated group one ability distribution was normal and the other was positively skewed. No negative effect was found on the linking performance by using his three methods. The reason may be that although the second ability had positively skewed distribution, the mean and standard deviation were still controlled at 0 and 1, which were the same as for the based group for the second dimension. It seems that mean and standard deviation were more important than the normality of the distribution. However, this conclusion needs to be confirmed for the MIRT linking methods. Based on the research on unidimensional scale linking and test equating (see the review by Kolen and Brennan, 2004), the similarity between two groups of examinees affects linking and equating performance: the more similar the groups are, the more adequate the linking and equating will be; large differences between groups may produce significant problems. Based on results from multidimensional scale linking, this conclusion can be extended to the multidimensional cases, but with cautious consideration of the interaction between ability distribution, test structure, and sample size. Effects of Different Item Parameter Values As mentioned in the first section, estimation of the item difficulty parameter is less accurate when the parameters are small or large, estimation of discrimination parameter is less accurate when discrimination parameters are small or large, and error in item parameter estimation affects scale linking performance. Therefore, linking quality is likely to be influenced by the item parameter values, especially by the extreme parameter values. This conceptual inference and conclusion were confirmed in this study: under most of the testing conditions, the linking results tended to be less accurate when the absolute item parameters had extreme values and less stable when the absolute item parameter values became large. This pattern of results was more apparent when (a) the test had approximate simple structure, (b) the sample size was larger, and (c) the linking performance for discrimination was evaluated. The only other multidimensional scale linking study evaluating the effects of different item parameter values was conducted by Li (1997). Based on that study (see Figure IV116, Li, 1997), the linking results for difficulty were more accurate and stable when the absolute item parameter values became larger; the linking results for discrimination did not change consistently with the item parameter values. Therefore, the effects of different item parameter values on scale linking were more apparent for discrimination in this study and more obvious for difficulty in Li's study. This is reasonable given Min and Kim's (2003) conclusion that Li's method worked better than Oshima and colleagues' test characteristic function method (2000) for difficulty parameters and Oshima and colleagues' method worked better for the two discrimination parameters. Performance of Different Linking Methods The effects of test structure, test length, sample size, ability distribution, and item parameter values on scale linking performance were separately discussed above. However, these factors interacted with each other and had both main and combined effect on the performance of the four linking methods. As summarized at the beginning of this chapter, generally speaking, all four linking methods worked approximately equally well under all testing conditions for approximate simple tests. For complex tests, the direct method was the best linking procedure; the item characteristic function method and the test characteristic function method were the second and third; the equated function method did not work well for complex tests. These results were based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group for the common items. It is not entirely surprising that the direct method, which minimizes the sum of squared differences between the two sets of item parameter estimates over items, was the best one across different testing conditions because the evaluation criterion was consistent with the method. However, the equated function method estimates the linking parameters by minimizing the sum of squared difference between the means of the two sets of selected item parameter estimates in the test. It uses the accumulative information of some items. Therefore, it is possible that even though the mean parameter estimates were similar for the two groups, individual parameter estimates were not. In the same way, item characteristic function method uses the combined information of discrimination, difficulty, and ability item by item. The test characteristic function method uses the accumulative information of discrimination, difficulty, and ability over all items in the test. Therefore, item characteristic function method was better than test characteristic function method using the criterion based on difference between item parameter estimates. Why do the four linking methods worked equally well for approximate simple tests but differentially poor for complex test? One of possible reason is that there is complicated interaction between item parameter estimation error and the characteristics of the four linking methods. More simulation studies need to be conducted to differentiate the two types of effect on the performance of scale linking. CHAPTER 5 CONCLUSIONS The purpose of this study was to use simulated data to examine the performance of four multidimensional linking methods under different testing conditions. There were one hundred and ninetytwo experimental conditions in this study: four linking methods (direct method, equated function method, test characteristic function method, and item characteristic function method) by two test structures (approximate simple test structure and complex test structure) by two test lengths (20 items and 40 items) by three sample sizes (500, 1000, and 2000), and by four different ability distributions between two groups (no difference, only mean difference, only variance difference, and both mean and variance difference). Five hundred replications were conducted for each of the experimental conditions. The linking performance evaluation was based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group for the common items. The mean and standard deviation of the differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Conclusions Conclusion 1: The performance of the four linking methods. Generally speaking, the direct method was the best linking method; the item characteristic function method and test characteristic function method were the second and third best method; the equated function method was the last method. However, their linking performance depended on the following testing conditions. Conclusion 2: The effects of test structure. For approximate simple test structure, each of the four linking methods worked approximately equally well for all testing conditions; For complex test structure, the equated function method worked poorly under all testing conditions; the performance of the other three linking methods depended on other testing conditions; the direction method was the best method for most testing conditions. Conclusion 3: The effects of test length. The linking performance for long tests was typically better than that for short tests except for complex tests when the sample size was small. Conclusion 4: The effects of sample size. The linking performance improved when the sample size became larger, especially for complex tests. Conclusion 5: The effects of ability distribution. Quality of linking performance declined when there was difference in ability distribution between the two groups, especially for complex tests; however, it seems that a betweengroup difference in the means was more important than a difference in the variance. Conclusion 6: The effects of item parameter values. Under most of the testing conditions, the linking results for the discrimination parameter tended to be less accurate and less stable when the item parameter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depended on the item parameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values. Future Research In this study, there are a number of limitations, which should be considered for making the conclusions described above. For example: (a) Although the item parameters for short and long approximate simple tests and complex tests were randomly created in the same way and from the same distributions, they did not have the same exact values. This should be considered when comparing the linking results for the four types of tests; (b) The test structure was not constructed by randomly arranging the items in the test. For the approximate simple test, the first half of the items had higher discrimination values for the first ability and lower values for the second ability, and the second half of the items had lower discrimination values for the first ability and higher values for the second ability. The equated function method in this study used the means of the first half of items (all with lower or higher values), second half of items (all with lower or higher values) as the function to estimate the linking parameters. This may affect the linking performance of the equated function method; (c) This study used the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group as the criterion, which is consistent with the minimized function of the direct method and accordingly may favor this method. The linking performance should be evaluated using other criteria which are consistent with the other methods to examine the possible dependence of the results on the criteria used. These criteria include the differences between the means of the selected item parameter estimates obtained from the two groups, the differences between the test characteristic functions for a given range of ability, and the differences between item characteristic functions for a given range of ability. As mentioned in the first chapter, the development of multidimensional linking methods is just at the infancy stage and more research is needed to obtain definitive results. Therefore, a substantial research needs to be conducted to explore and evaluate different procedures for multidimensional scale linking. Here are some future research topics on multidimensional IRT scale linking. First of all, different specific procedures within each of the four linking methods need to be explored, compared, and evaluated so that the best method can be chosen for some specific purpose. For example: (a) For the test characteristic function and item characteristic function methods, how should the theta region or levels be chosen? Should we use the equally spaced grid theta method or empirical theta method? If we choose empirical theta method, which examinee group, base group, equated group, or combined group, should be used? Which method is better? Should we give different weights to different theta regions and how to choose different weights? (b) For equated function method, which item parameter estimates should be used? What characteristics should be considered to choose the appropriate sets of items? What function should be used to produce good linking performance? Second, what kind of criteria should be used to evaluate the performance of different linking methods? Within the multidimensional IRT linking and equating studies, different criteria have been used. Even within one study, different criteria have been used. For example, Li (1997) used bias and RMSE between the transformed linking parameter estimates and the true linking parameters across replications in his first study and then used bias and RMSE between true item parameter values and the transformed item parameter estimates and ability recovery in his second study. Oshima et al. (2000) used mean and standard deviation of the linking parameter estimates over 20 replications, bias and MSE between the estimated and true linking parameters, correlation and mean absolute difference of linking parameter estimates across different methods, and minimized function values by different methods. Min (2003) used bias and RMSE between transformed item parameter estimates and the initial item parameters across common items for the simulated data, and used the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group across the common items for the real data. Given these criteria, which one should we use for which purpose for scale linking? This is a critical issue in evaluating different methods. Third, as we discussed in last chapter, there are at least two main components in linking errors: error caused by parameter estimation and error produced by scale transformation. The problem is how to differentiate the estimation error from the linking error when scale linking is conducted? To answer this question, many studies need to be conducted to evaluate the performance of different estimation programs for multidimensional IRT. In addition, some methods need to be developed to differentiate the estimation error from linking error and evaluate the effects of estimation error on the performance of scale linking. Finally, the two approaches, multidimensional IRT approach and factor analysis approach, have different strengths and weaknesses in linking different scales. As Min and Kim (2003) found in their study that Li's method worked better for difficulty parameters and Oshima and colleagues' method (e.g., test characteristic function method) worked better for the two discrimination parameters. Therefore, how to use the strengths of the two approaches to develop a combined method for multidimensional scale linking is an important topic in the future research. APPENDIX ACCURACY AND STABILITY FOR DIFFERENT LINKING METHODS I I I 80 S 40 i so 0 a 40 si a I a SI I 4 i 4 0 0 a 0.6 0.4 0.2 0 0.2 0. 0.6 Mean of Dlferences for a2 4 a 40 0.6 0.4 0.2 0 0.2 0.4 0. Mean of Dilhrences for d 80 40 0 so 40 0 880 40 80 o 040 0 80 40 0 0 I 0.1 03 05 0.7 0.9 1.1 1. Standard Deviation of DIlarences for al 0.1 03 05 0.7 0.9 1.1 13 Standard Deviation d Dilerences for a2 0.1 03 05 0.7 0.9 1.1 13 Standard Deviation of Dferences for d Standard Dvaio tDilrncsfr d Figure A1. Accuracy and stability for different linking methods (APP, n=20, N=500, Gl) 138 0.6 0.4 0.2 0 0.2 0.4 0. Mean of DIlerences for al 80 40 a 0 / I40 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences for al so 40 40 40 s o 40 0.6 0.4 0.2 0 0.2 0.4A 0.6 Mean of Dfferences fbr a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences tr al 480 40 0 80 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Dferences tor a2 ao 40 0 0 0o 0 0: 0.6 0.4 0.2 0 0.2 0.4 B Mean of DIferences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatan of Dlherences for d Figure A2. Accuracy and stability for different linking methods (APP, n=20, N=500, G2) ] 0 40 a 80 40 40 40 0a  I J   ,I OPEL. 80 40 iu 40 0.6 0.4 0.2 0 0.2 0A OB Mean of Dlferences for al 40 0 0.6 0.4 0.2 0 0.2 0.4A 0. Mean of Differences br a2 0 40 80 0 so 0.8 0.4 0.2 0 0.2 0A OB Mean of DIffrences for d 80 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences fr al 40 40 O' 80 40 0.1 0.8 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences br a2 8o 40 0 80 S.40:M^^^ 0^ 0I 80 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Figure A3. Accuracy and stability for different linking methods (APP, n=20, N=500, G3) 80 40 a 0 40 40 0.6 0.4 0.2 0 0.2 0A OB Mean of Differences Mr al 0 40 a 8I 40 0.6 0.4 02 0 02 0.4 0.6 Mean of Dfferences for a2 8K 40 A 0 4 I ll II0 0.6 0.4 0.2 0 02 A0.4 O Mean of Dfferences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences fr al 0 140W 0 40 0.^^^^^ 80 0.1 0S 05 0.7 0.9 1.1 1. Standard Deviation of DIlerences fr a2 80 40 0 S40 0*^^^ I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatian of Dllerences or d Figure A4. Accuracy and stability for different linking methods (APP, n=20, N=500, G4) 80 40 a1 8: 4 . 8 0 40 0.8 0.4 0.2 0 0.2 0.4A OB Mean of Differences or al 0 4: 80 40 0.6 0.4 0.2 0 0.2 0. B Mean of Dflferences for a2 80 40 0 , 40 40 s o 0.6 0.4 0.2 0 0.2 0.4A 0.6 Mean of DllIrences for da Mean of Dlffrences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaion of DIfferences or al 0.1 0.8 05 0.7 0.9 1.1 1.8 Standard Devialon of DIfferences for a2 0.1 0. 05 0.7 0.9 1.1 1.8 Standard Deviation of Dllerences br d Figure A5. Accuracy and stability for different linking methods (APP, n=20, N=1000, Gl) so i40 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dierences for al 0 so 40 0 P40 0 .H  0.6 0.4 02 0 0.2 0.4 0.8 Mean of Differences for a2 4 0 P4 I 1:4 0. 0.4 0.2 0 0.2 0.4 OB Mean of Dlffrences for d 80 40j 0 a sao 440 0 80 40 a 0.1 0. 05 0.7 0.9 1.1 1. Standard Deviation of Diferences for al 80 40 40 0 40 sio 40 0JI 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dferences for a2 80 40 i: I 0 0 0.1 0.3 0.5 0.7 0.9 1.1 1.8 Standard Devlaton of Dlfferences for d Figure A6. Accuracy and stability for different linking methods (APP, n=20, N=1000, G2) 143 80 40 0.8 0.4 0.2 0 0.2 0.4 0.B Mean of Differences or al1 a 40 0 Ma o 40 40 0.6 0.4 0.2 0 0.2 0.4 0. Mean of Dfferences tr a2 s i 40 40 0 0.6 0.4 0.2 0 0.2 0A OB Mean of Dfferences for da2 Mean of DNlerencas for d 40 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences tr al O' o' 0I 40 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dfferences for a2 iII 0 O' 40 0.1 0.8 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Figure A7. Accuracy and stability for different linking methods (APP, n=20, N=1000, G3) 80 40 a 80U I 80 40 40 0.6 0.4 0.2 0 0.2 4A OB Mean of Dllerences for al 80 40 80 0.6 0.4 0.2 0 02 CA 0. Mean of Dtlleence for a2 0.6 0.4 0.2 0 02 CA OS Mean of Dlflrena for d 0 0 4' 0 0.1 0.3 05 0.7 0. 1.1 1.8 Standard Deviation o DIferences for al Standard DeviatA n of DIferences for a2 0.1 0. 05 0.7 05 1.1 1.3 Standard Deviation of Dlerences for d Figure A8. Accuracy and stability for different linking methods (APP, n=20, N=1000, G4) 80 40 0.8 0.4 0.2 0 0.2 0.4 0.B Mean of Differences or al a 0 ! 40 O t 0 a 40 0.6 .4 0.2 0 0 0.4 0. Mean of Dfferences or a2 40 40 o.  0 , 4 0 , 80 40 0 , 0. 0.4 0.2 0 02 0.4 OB Mean of Differences for d Mean of DNlerencas for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIferences hbr d I I Figure A9. Accuracy and stability for different linking methods (APP, n=20, N=2000, Gl) 80 40 ao 0 U 40 . . 80 40__ 8O 40 0.6 0.4 0.2 0 0.2 0. OB Mean of Differences for al 0 , s: 40 s: 40 0.6 0.4 0.2 0 0.2 0.4A 0.6 Mean of Differences for a2 : i 4 : i 8 s o 0.6 0.4 0.2 0 0.2 0.4A 0.6 Mean of DIferences for d &ILI_ Ei 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences or a2 0.1 0. 05 0.7 0. 1.1 1. Standard Deviation of Differences r d Figure A10. Accuracy and stability for different linking methods (APP, n=20, N=2000, G2) 80 40 a 8 40 40 0.6 0.4 0.2 0 0.2 0. OB Mean of Dfferences for al 40 0 , 40 40 0.6 0.4 0.2 0 02 0A .B Mean of Differences for a2 40 s: 40 s: 40 0.6 0.4 0.2 0 02 OA OB Mean of DIferences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences for al I i 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIfferences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaton of DIerences for d Figure A11. Accuracy and stability for different linking methods (APP, n=20, N=2000, G3) a80 40 80: _ 0 0.6 0.4 0.2 0 0.2 0.4 .OB Mean of Differences for al 80 80 40 0.6 0.4 0.2 0 02 0A .B Mean of Differences for a2 40 s: 40 s: 40 0.6 0.4 0.2 0 02 OA OB Mean of Differences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences br al I i 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIfferences tr a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaton of Dlherences or d Figure A12. Accuracy and stability for different linking methods (APP, n=20, N=2000, G4) 80 40 0 40 0 0.6 .4 0.2 0 02 .A OB Mean of Differences or al 40 0 40 40 0.6 0.4 0.2 0 02 OA .B Mean of Differences for a2 40 40 0 , 40 0.6 0.4 0.2 0 02 OA OB Mean of Dfferences for d 80 40 0 . 40 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaton of Differences ar al oi 40 40 0. 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIfferences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatin of Dllerences Ifr d Figure A13. Accuracy and stability for different linking methods (APP, n=40, N=500, Gl) _ I  I I I I I I O80 40 0 S8o0 j40 0 80 40 0 I80 0 0.6 0.4 0.2 0 0.2 0.4 0. Mean of Differences for al 0.6 0.4 0.2 0 0.2 0.4 0. Mean of Differencs for a2 0.6 0.4 0.2 0 0.2 A0.4 0 Mean of Diflrences for d 80 j40 0 0 j40 0 80 j40 0 0 40 0 80 0 80 j40 a I80 0 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation df Diferencs for a2 I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Difrences for d Figure A14. Accuracy and stability for different linking methods (APP, n=40, N=500, G2) I I I I I I I I I I I I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al 80 80 40 SI 0 0.6 0.4 0.2 0 02 0A OB Mean of Differences or al 80 40 80 80 40 40 0.6 0.4 0.2 0 02 OA .B Mean of Differences or a2 0 , s: 40 :o 0 , 40 0.6 0.4 0.2 0 02 OA OB Mean of Dfferences for d 80 40 80 400 0 80 40 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences fr al 80 40 0 80 40 40 80 0.4 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIfferences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatin of Dlherences or d Figure A15. Accuracy and stability for different linking methods (APP, n=40, N=500, G3) _ I I I I I I I I 80 40 8 0 40 80 40 0 0.6 0.4 0.2 0 0.2 CA 0. Mean of Differences hor al 40 40 0.6 0.4 0.2 0 02 0A 0. Mean of Differences for da 40 40 0.6 0.4 0.2 0 0.2 A 0.6 Mean od Dlbranoes Ibr d 80 40 80 0 80 a^B 80 40 0.1 0. 05 0.7 0. 1.1 1.3 Standard Devlatlon of Dlerences or al 40 0' 40 0 80 40. 0^^ 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIferences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatlon of Dlerences for d Figure A16. Accuracy and stability for different linking methods (APP, n=40, N=500, G4) I I I I I I  80 40 40 0 40 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences for al I 0 :... . 80 40 80 0.6 .4 0.2 0 0 .2 A 0.B Mean of D flerences or a2 0 , 40 s: 40 0.6 0.4 0.2 0 02 0A OB Mean of Dlberences for d I mm i . Ei 0.1 0.8 05 0.7 0.0 1.1 1.8 Standard Devilion of DIferences hr al I K. K. LI Ei 0.1 0.8 05 0.7 0.0 1.1 1.8 Standard Devllon of DIerences hr a2 0.1 0. 05 0.7 0. 1.1 1. Standard Devlation of DIbrences for d Figure A17. Accuracy and stability for different linking methods (APP, n=40, N=1000, Gl) 80 40 80 40 m . 40 40 0.6 0.4 0.2 0 0.2 0.4 A Mean of Differences for al 40 o i : i 0.6 0.4 0.2 0 0.2 A 0. Mean of Differences for a2 0 , 40 40 0.6 0.4 0.2 0 02 0.4 0.8 Mean of Dlflerences for d EL I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaion of DIfferences tr al U 0.1 0.8 05 0.7 0. 1.1 1.8 Standard Deviallon of DIMerences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaton of Dllerences for d Figure A18. Accuracy and stability for different linking methods (APP, n=40, N=1000, G2) 80 40 80 40 m . 0.8 0.4 0.2 0 02 0.4 .OB Mean of DIfferences or a2 40 0 s: 40 s: 40 0.6 0.4 0.2 0 02 0.4 Mean of Dlferences for a2 ; iI i 0.6 0.4 0.2 0 0.2 OA B Mean of DIferences for a2 Mean of Dlflerences for d I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences tor al I 0.1 0.38 05 0.7 0.9 1.1 1.38 Standard Deviation of DIfferences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Figure A19. Accuracy and stability for different linking methods (APP, n=40, N=1000, G3) 80 40 40 8 0 0... .... 40 40 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences for al 80 40 0 40 80 40 80 40 0.6 .4 0.2 0 02 OA OB Mean of Dflerences or a2 0 , 4: 0 , s: 40 8s 40 0.6 0.4 0.2 0 02 0A OB Mean of Dllerences for d HaL 0.1 0.8 05 0.7 0.0 1.1 1.8 Standard Deviatilon of DIferences hr al I U 0.1 0. 05 0.7 0. 1.1 1. Standard Deviation of Dlerences tor a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatlan of Dllferences bfr d Figure A20. Accuracy and stability for different linking methods (APP, n=40, N=1000, G4) 80 I  40 0 40 0  40 80 40 0.6 0.4 0.2 0 0.2 0. OB Mean of Dlfferences for al 80 40 0 , 0 , 0 , 40 80 40 0.6 0.4 0.2 0 02 0.4A OB Mean of Dlferences for a2 40 0 , 0 a s: 0 a 80 40 0 0.6 0.4 0.2 0 0.2 0.4A 0.6 Mean of DIAerences bfor d I I uI Standard Deviation of Differences or a I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIIerences or a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlherences or d Figure A21. Accuracy and stability for different linking methods (APP, n=40, N=2000, Gl) 80 40 a1 40 80 .A . 40 0 0.6 .4 0.2 0 02 A 0.B Mean of Dfferences or al 0 8 0 0 , 40 40 0.6 0.4 0.2 0 0.2 0A OB Mean of Dferences for a2 40 0 , 80 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of DIllerences Ibr d I I E i 0.1 0. 05 0.7 0.9 1.1 1. Standard Deviallon of DIferences f a1 I HI I i 0.1 0. 05 0.7 0. 1.1 15 Standard Deviatlon of DIferences for a2 I  StnardDm l atbn of lernce ar 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatlin of Dllferences br d Figure A22. Accuracy and stability for different linking methods (APP, n=40, N=2000, G2) 80 40 a0 o80 40 0 o80 40 80 40 0  0.6 0.4 0.2 0 0.2 0.4 0. Mean of Differences for al 80 40 80 40 0 0.6 0.4 0.2 0 0.2 0. O Mean of Dfferences bor a2 80 0 , s: 0 , 0 , 40 0.6 0.4 0.2 0 0.2 0.A O Mean of DIffrences for d I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dllerences Ir d Figure A23. Accuracy and stability for different linking methods (APP, n=40, N=2000, G3) uI uI I IK 0.1 0.8 05 0.7 0.9 1.1 1.8 Standard Deviation of DIerences for al I l *,_ 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences tr a2 so 40 i 4s 40 80 40 0 0 0.6 .4 0.2 0 0.2 0.4 0 Mean of Differences or al 0 , 40 0 , 40 80 40 0.8 0.4 0.2 0 02 0.A4 B Mean of Dlferences hor a2 so 0 , 40 : i 0.6 0.4 02 0 0.2 4A 0. Mean of DIferences for d uI ILI I I I I I i 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences or al Standard Deviagion of DIfferences tor a2 I I I I I I 0.1 0.8 5 0.7 0.9 1.1 1.8 Standard Devlatilon of DIlerences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dllerences for d Figure A24. Accuracy and stability for different linking methods (APP, n=40, N=2000, G4) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences or al 80 40 a 80 40 a 80 40 a 80 40 a 1 * I I I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al I 0.1 0.3 0.5 0.7 09 1.1 1.3 Standard Devlation of DIIffrences for a2 ~Emli _p Ii a 0.6 0.4 0.2 0 0.2 0.4 O Mean of Differences for a2 0.6 0.4 0.2 0 02 0.4 0.6 Mean of Dilerences for d III 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Derences for d Figure A25. Accuracy and stability for different linking methods (COM, n=20, N=500, Gl) p 1.rrn~nrrn. 02 0.4 O Mean of Dlferences for al 0.6 0A 0.2 0 0.2 0.4 0.6 Mean of Differences for a2 0.6 0.4 0.2 0 0.2 0A 06 Mean of Dfferences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devation of Differences for al I H 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for a2 II 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Stadad Dvll~n o Dl~nce frI Figure A26. Accuracy and stability for different linking methods (COM, n=20, N=500, G2) 0.6 0.4 0.2 0 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Dfferences for a2 0.6 0.4 0.2 0 0.2 0.4 B Mean of Dfferences for d S! 0.1 0. 05 0.7 0.9 1.1 1. Standard Deviatdon of Diferences for al 0.1 0.8 05 0.7 0.9 1.1 1.3 Standard Deviation of DIferences for a2 i i_ 0.1 0. 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for d Figure A27. Accuracy and stability for different linking methods (COM, n=20, N=500, G3) 0.6 0.4 0.2 0 02 0.4 OB Mean of Dllerences for al 0.6 0.4 0.2 0 0.2 0.4 .B Mean of Dfferences for a2 0.1 0. 05 0.7 0.9 1.1 1. Standard Devlallon of Dferences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Dfferences tor a2 ^^3 l__I 0.1 0.3 Standard 05 0.7 0.9 1.1 1.3 Deviation of DIereances for d Figure A28. Accuracy and stability for different linking methods (COM, n=20, N=500, G4) 0.6 0.4 0.2 0 0.2 0A 0. Mean of Dfferences for d BnI = 0 0 404 0. 0.4 0.2 0 02 CA 0.8 0.1 0.3 05 0.7 0.9 1.1 1.3 Mean of Differences hor al Standard Deviation of Differences hor al S 80 80 4 40 i 40 S a80 ___,0m 0 = 0 0 0 0.6 0.4 0.2 0 02 0A .B 0.1 0.3 05 0.7 0.9 1.1 1.3 Mean of Differences or a2 Standard Deviation of Differences tbr al . o __ _o 0 i I o : = 0 = 40 0 0 S  o so S 40 40 = 0 = 0 0.6 .4 0.2 0 0.2 A04 0. 0.1 0.3 05 0.7 0.9 1.1 1.3 Mean of Dllerences for d Standard Deviation of Diferences for d Figure A29 Accuracy and stability for different linking methods (COM, n20, N 1000, a 0 I 0.6 OA 0.2 0 0.2 0.4 0. 0.1 0.8 05 0.7 09 1.1 1.8 Mean of Dillerances for d Standard Devialan of DIerences for d Figure A29. Accuracy and stability for different linking methods (COM, n=20, N=1000, GI) 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences or al 0.6 0.4 0.2 0 0.2 0.4 .B Mean of Differences for a2 "40 0 , 40 40 0.6 0.4 0.2 0 02 0.4 OB Mean of Dillerences for d 80 40 a 40 I a 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al 0 40 80 40 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences tr a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviaton of Dlerences for d Figure A30. Accuracy and stability for different linking methods (COM, n=20, N=1000, G2) 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences or al 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences for a2 0 oi Io 40 0 AII 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences br al 80 0 80: 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences br a2 40 401 0.6 0.4 0.2 0 0.2 0A 0. 0.1 0. 05 0.7 0.9 1.1 1. Mean of Dfferences for d Standard Deviatln of Diferences for d Figure A31. Accuracy and stability for different linking methods (COM, n=20, N=1000, G3) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 0.2 4A 0. Mean of Differences for a2 0.6 0.4 0.2 0 0.2 A 0.6 Mean of DIfferences bfr d 80 40 aI 80 80 Standard Deviallon of Dflerences for al 40 0.1 a 05 0.7 09 1.1 1.3 Standard Deviaton of DIferences for a2 so 6I 0.1 0. 05 0.7 0. 1.1 1.3 Standard Deviatin of Dlferences for d Figure A32. Accuracy and stability for different linking methods (COM, n=20, N=1000, G4) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Dlferences for al 0.6 0.4 0.2 0 0.2 0.4 O Mean of Differences for a2 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Dfferences for d ,* 0.1 0.3 0.5 0.7 0.9 1.1 1.3 Standard Devlaton of Differences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation do DIfferences for a2 *1 I I I 0.1 0.S 5 0.7 0.9 1.1 1.3 Standard Deviation of Diflerences for d Figure A33. Accuracy and stability for different linking methods (COM, n=20, N=2000, Gl) 0.6 0.4 0.2 0 0.2 0. OB Mean of Diferences or al 0.6 0.4 0.2 0 0.2 0. OB Mean of Dierences for a2 0.1 0.3 05 0.7 09 1.1 1.3 Standard Deaaion of Differences for al i 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation f DIfferences or a2 0 40 440 0.6 0.4 0.2 0 0.2 0A 0. 0.1 OA 05 0.7 0. 1.1 1. Mean of Dfferences for d Standard Devation of Differences for d Figure A34. Accuracy and stability for different linking methods (COM, n=20, N=2000, G2) 0.6 0.4 0.2 0 0.2 OA 0. Mean of Differences for al 0.6 0.4 0.2 0 0.2 0A 0. Mean of Differences for a2 0.6 0.4 0.2 0 0.2 OA 0. Mean of Dfferences for d 0.1 0S 05 0.7 0.9 1.1 15 Standard Devlatdon of Diferences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Diferences for a2 ! 0.1 0.8 05 0.7 0.9 1.1 1.8 Standard Deviation of Dlhrences for d Figure A35. Accuracy and stability for different linking methods (COM, n=20, N=2000, G3) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences or al 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences or a2 0.6 0.4 0.2 0 0.2 A0.4 O Mean of Dferences for d & * I Ui 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devation of Differences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIfferences or a2 ! 0.1 aS 05 0.7 0. 1.1 15 Standard Deviation of Dilerences for d 0.1 08 05 0.7 .0 11 1. Stadad Dvitio &D~henes r! Figure A36. Accuracy and stability for different linking methods (COM, n=20, N=2000, G4) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences or al 0.6 0.4 0.2 0 0.2 A 0.6 Mean of Differences for a2 40 0 o 40 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences tr al 0 40 o 0.1 0.83 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences br a2  I ii01 ' S, I . 0.6 0.4 0.2 0 0.2 0.4 0. 0.1 0.3 05 0.7 0.9 1.1 1.3 Mean of Differences for d Standard Deviatan of DIerences or d Figure A37. Accuracy and stability for different linking methods (COM, n=40, N=500, Gl) 0.8 0.4 0.2 0 0.2 0A B0 Mean of Differences for al 0.6 0.4 0.2 0 0.2 0.4 0. Mean of Dfferences for a2 I  0.6 0.4 0.2 0 0.2 .A 0. Mean of Dffrences for d i____I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIfferences hor al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIIerences or a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIerences bfr d Figure A38. Accuracy and stability for different linking methods (COM, n=40, N=500, G2) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 0.2 A 0.6 Mean of Differences for a2 0.6 0.4 0.2 0 0.2 A0.4 0 Mean of Dfferences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Diferences for al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of Differences for a2 I I I I I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of DIerences for d Figure A39. Accuracy and stability for different linking methods (COM, n=40, N=500, G3) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences or al 0.6 0.4 0.2 0 0.2 0.4 0 Mean of Dfferences fr a2 E 0.6 0.4 02 0 0.2 0A 06 Mean of DIffrences for d Iu. 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al 1 5 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dfferences br a2 I Standard Devlation of DIbrence~s br a2 I* I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Figure A40. Accuracy and stability for different linking methods (COM, n=40, N=500, G4) 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 0.2 0.4 O Mean of Differences for a2 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Dfferences for d 0.1 0S 05 0.7 0.9 1.1 15 Standard Devoaton of Difterences for al 0.1 0. 0.5 0.7 0.9 1.1 1.3 Standard Devilatlon of Dlerences for a2 SI 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devialon of DIllreances for d Figure A41. Accuracy and stability for different linking methods (COM, n=40, N=1000, Gl) 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 0.2 0.4 06 Mean of Differences for a2 I 80 0.6 0.4 0.2 0 0.2 0.4 O Mean of Dfferences for d 0.1 0 05 0.7 0. 1.1 1 Standard Devialon of DIHerences or al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation f Differences for a2 I___ 0.1 0.3 05 0.7 0. 1.1 1.3 Standard Deviation of Dlerences for d Figure A42. Accuracy and stability for different linking methods (COM, n=40, N=1000, G2) 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 02 0.4 0.6 Mean of Differences for a2 0.6 0.4 0.2 0 0.2 4A 0. Mean of Dfferences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al 0.1 0s 05 0.7 A 1.1 1.3 Standard Devialation DIMerences for a2 I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviatlan of DMerences for d Figure A43. Accuracy and stability for different linking methods (COM, n=40, N=1000, G3) 0.6 0.4 0.2 0 02 0.4 OB Mean of Dlferences for al 0.6 0.4 0.2 0 0.2 0.4 O Mean of Differences for a2 0.6 0.4 0.2 0 0.2 0.4 O Mean of Dfferences for d 40 0 40 0 80 40 0 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Differences for al 80 40 0 0 0.1 0.3 05 0.7 0.9 1.1 1.3 40 0.1 0.8 05 0.7 0.9 1.1 1.8 Standard DevIation of DIlFerences for da2 a 0.1 0.8 05 0.7 0.9 1.1 1.8 Standard Devialan of DIlerances Ior d Figure A44. Accuracy and stability for different linking methods (COM, n=40, N=1000, G4) 0.6 0.4 0.2 0 02 0.4 OB Mean of Dlferences for al 0.6 0.4 0.2 0 0.2 0.4 B Mean of Dllerences for a2 80 40 80 4 0 40 so 40 0.6 0.4 0.2 0 02 A0.4 O Mean of DIlbrences for d 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIfferences fr al 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Devlation of DIferences fr a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dlerences for d Figure A45. Accuracy and stability for different linking methods (COM, n=40, N=2000, Gl) 0.6 0.4 0.2 0 02 0.4 OB Mean of Differences for al 0.6 0.4 0.2 0 0.2 0.4 O Mean of Differences for a2 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Diferences for d 5;I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Diferences for al .1 0.1 0S 05 0.7 0.9 1. 5 Standard Deviaton of DIMerences for a2 5! 0.1 0a 05 0.7 O0 1.1 15 Standard Deviaton of Dilaerences for d MI III 0.1 0. 5070.!. . Stadar Dvialo ofDIerece Ib a HI Figure A46. Accuracy and stability for different linking methods (COM, n=40, N=2000, G2) 80 40 0.8 0.4 02 0 0.2 0.4 0.6 Mean of Diferences for al I 0 80 40 80 40 0 i  , i  i 0.8 0.4 0.2 0 0.2 0.4 0.8 Mean of Differences for a2 80 4 n 0 I 80 40 0 0.6 0.4 0.2 0 0.2 0.4 0.6 Mean of Diferences for d i I I I I 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation f DiWerences for a2 a... 0.1 0.3 05 0.7 0.9 1.1 1. Standard Deviation of Differences for d Figure A47. Accuracy and stability for different linking methods (COM, n=40, N=2000, G3) I UI Ei , II 0.1 0.3 05 0.7 0.o 1.1 1. Standard Deviation of Diferences for al . . n I I I I I 0.6 0.4 0.2 0 0.2 A0.4 0 Mean of Differences or al 0.6 0.4 0.2 0 0.2 0.4 OB Mean of Dfferences for a2 0.6 0.4 0.2 0 0.2 0.4 O Mean of Dfferences for d *E i 0.1 0. 05 0.7 0.9 1.1 1. Standard Deviallon of Derences for al 0.1 0.3 05 0.7 09 1.1 1.3 Standard Deviation Dirkrences for a2 0.1 0.3 05 0.7 0.9 1.1 1.3 Standard Deviation of Dferences for d ! Standard Devilian of DIHrences for d Figure A48. Accuracy and stability for different linking methods (COM, n=40, N=2000, G4) LIST OF REFERENCES Ackerman, T. A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255278. Ackerman, T. A. (1996). Graphic representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311329. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561573. Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological Measurement, 16, 8796. Baker, F. B. (1993). Equating tests under the nominal response model. Applied Psychological Measurement, 17, 239251. Bateley, R. M., & Boss, M. W. (1993). The effects on parameter estimation of correlateddimensions and a distributionrestricted trait in a multidimensional item response model. Applied Psychological Measurement, 17, 131141. Bedescu, D. (1985). Efficiency of linear equating as a function of the length of the anchor test. Journal of Educational Measurement, 22, 1320. Bock, R. D. (1972). Estimating item parameters and latent ability when the responses are scored in two or more nominal categories. Psychometrika, 37, 2951. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443459. Bock, R. D., Gibbons, R., & Muraki, E. (1988). Fullinformation item factor analysis. Applied Psychological Measurement, 12, 261280. Bock, R. D., Gibbons, R., Schilling, S. G., Muraki, E., Wilson, D. T., & Wood, R. (1999). TESTFACT 3: Test scoring, items statistics, and fullinformation item factor analysis. Chicago: Scientific Software International. Carlson, J. E. (1987). Multidimensional item response theory estimation: A computer program. Unpublished manuscript. Cattell, R. B. (1978). The scientific use offactor analysis. New York: Plenum. Cohen, A. S., & Kim, S. H. (1998). An investigation of linking methods under the graded response model. Applied Psychological Measurement, 22, 116130. Comery, A. L., & Lee, H. B. (1992). A First course in factor analysis. Hillsdale, NJ: Erlbaum.Cook, L. L., Eignor, D. R. (1991). IRT equating methods. Educational Measurement: Issues and Practice, 10, 3745. Cook, L. L., Eignor, D. R., & Taft, H. (1981, April). A comparative study of curriculum effects on the stability of RT and conventional item parameter estimates (RR8538). Princeton, NJ: Educational Testing Service. Cook, L. L., & Petersen, N. S. (1987). Problems related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225244. Cureton, E. E., & D'Agostino, R. B. (1983). Factor analysis: An applied approach. Hillsdale, NJ: Erlbaum. Davey, T. C., Oshima, T. C., & Lee, K. (1996). Linking multidimensional item calibrations. Applied Psychological Measurement, 20, 405416. Divgi, D. R. (1985). A minimum chisquare methods for developing a common metric in item response theory. Applied Psychological Measurement, 9, 413415. Fitzpatrick, A. R., & Yen, W. M. (2001). The effects of test length and sample size on the reliability and equating of tests composed of constructedresponse items. Applied Measurement in Education, 14, 3157. Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor analysis.Multivariate Behavioral Research, 23, 267269. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum. Gosz, J. K. & Walker, C. M. (2002, April). An empirical comparison of simple versus complex multidimensional item response data. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Gosz, J. K., Walker, C. M. (2002, April). An empirical comparison of multidimensional item response data using TESTFACT andNOHARM. Paper presented at the Annual Meeting of the National Council for Measurement in Education (NCME), New Orleans, Louisiana. Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw Hill. Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144149. Hason, B. A., & Beguin, A. A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the commonitem equating design. Applied Psychological Measurement, 26, 324. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff Publishing. Harwell, M., Stone, C. A., Hsu, T. C., & Kirisci, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20, 101125. Hirsch, T. M. (1989). Multidimensional equating. Journal of Educational Measurement, 26, 337349. Hirsch, T. M. (1988). Multidimensional equating. Unpublished doctoral dissertation, Florida State University, Tallahassee, FL. Kaskowitz, G. S., & De Ayala, R. J. (2001). The effect of error in item parameter estimates on the test response function method of linking. Applied Psychological Measurement, 25, 39 52. Kelderman, H. (1997). Loglinear multidimensional item response models for polytomously scored items. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. Kim, H. (1994). New techniques for the dimensionality assessment of standardized test data. Unpublished doctoral dissertation. University of Illinois at UrbanaChampaign. Urbana Champaign, IL. Kim, S. H., & Cohen, A. S. (1995). A minimum chisquare method for equating tests under the graded response model. Applied Psychological Measurement, 19, 167176. Kim, S. H., & Cohen, A. S. (2002). A comparison of linking and concurrent calibration under the graded response model. Applied Psychological Measurement, 26, 2541. Klein, L. W., & Kolen, M. J. (1985, April). Effect of number of common items in commonitem equating i i/th nonrandom groups. Paper presented at the annual meeting of American Educational Research Association, Chicago. Knol, D. L., & Berger, M. P. F. (1991). Empirical comparison between factor analysis and multidimensional item response models. Multivariate Behavioral Research, 26, 457477. Kolen, M. J. (2004a). Population invariance in equating and linking: Concept and history. Journal of Educational Measurement, 41, 314. Kolen, M. J. (2004b). Linking assessment: Concept and history. AppliedPsychological Measurement, 28, 219226. Kolen, M. J., & Brennan, R. L. (2004). Test equating,. scaling, and linking: Methods and Practices (2nd ed.). New York, NY: Springer. Lautenschlager, G. J., Flaherty, V. L., & Park, D. G. (1994). IRT differential item functioning: An examination of ability scale purifications. Educational and Psychological Measurement, 54, 2131. Lee, K., & Oshima, T. C. (1996). IPLINK: Multidimensional and unidimensional item parameter linking in item response theory. Applied Psychological Measurement, 20, 230. Li, Y. H. (1997). An evaluation of multidimensional IRT equating methods by assessing the accuracy of transforming parameters onto a target test metric (Doctoral dissertation, University of Marryland, 1997). Dissertation Abstract International, UMI Number 9816494. Li, Y. H., & Lissitz, R. W. (2000). An evaluation of the accuracy of multidimensional IRT linking. Applied Psychological Measurement, 24, 115138. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum. Loyd, B. H., & Hoover, H. D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179193. Marco, G. L. (1977). Item characteristic curve solutions to three intractable testing problems. Journal of Educational Measurement, 14, 139160. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149174. McDonald, R. P. (1981). The dimensionality of tests and items. British Journal ofMathematical and Statistical Psychology, 34, 100117. McDonald, R. P. (1997). Normalogive multidimensional model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah, NJ: Erlbaum. McDonald, R. P. (2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99114. McKinley, R. L., & Reckase, M. D. (1983). An extension of the twoparameter logistic model to the multidimensional latent space (Research Report, ONR 832). Iowa City, IA: American College Testing Program. Millsap, R. E. (2005). Four unresolved problems in studies of factorial invariance. In A. MaydeuOlivares & J. J. McArdle (Eds.), Contemporary Psychometrics. Mahwah, NJ: Lawrence Erlbaum Associates. Min, K. S. (2003). The impact of scale dilation on the quality of the linking of multidimensional item response theory calibrations. Unpublished Dissertation, Michigan State University, East Lansing, MI. Mroch, A. A., & Bolt, D. M. (2006). A simulation comparison of parametric and nonparametric dimensionality detection procedures. Applied Measurement in Education, 19, 6791. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159176. Muraki, E. (1999). POLYFACT version 2 [Computer program]. Princeton, NJ: Educational Testing Service. Muraki, E, & Carlson, J. E. (1995). Fullinformation factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 7390. Muraki E., & Engelhard, G. (1985). Fullinformation item factor analysis: Applications ofEAP scores. Applied Psychological Measurement, 9, 417430. Muthen, L. K., & Muthen, B. (1998). MPLUS: The comprehensive modeling program for applied researcher: User's guide. Los Angeles: Muthen & Muthen. Oshima, T. C., Miller, M. D. (1992). Multidimensionality and item bias in item response theory. Applied Psychological Measurement, 16, 237248. Oshima, T. C., Davey, T. C., & Lee, K. (2000). Multidimensional linking: Four practical approaches. Journal ofEducational Measurement, 37, 357373. Park, D. G., & Lautenschlager, G. J. (1990). Improving IRT item bias detection with iterative linking and ability scale purification. Applied Psychological Measurement, 14, 163173. Peterson, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137 156. Peterson, N. S., Kolen, M. J. & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L. Linn (Ed.), Educational measurement (pp. 221262). New York: American Council on Education and Macmillan. Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401412. Reckase, M. D. (1991). The discriminating power of items that measure more than onedimension. Applied Psychological Measurement, 15, 361373. Reckase, M. D. (1997a). A linear logistic multidimensional model for dichotomous item response data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. Reckase, M. D. (1997b). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 2536. Rechase, M. D., & Martineau, J. (2004, October). The vertical scaling of Science Achievement Tests. Paper commissioned by the Committee on Test Design for K12 Science Achievement, Center for Education, National Research Council. Ree, M. J., & Jensen, H. E. (1983). Effects of sample size on liner equating of item characteristic curve parameters. In D. J. Weiss (Ed.), New horizons in testing (pp. 135146). New York: Academic. Roussos, L. A., Stout, W. F., & Marden, J. I. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 130. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17. Schonemann, P. H. (1966). A generalized solution of the orthogonal procrustes problem. Psychometrika, 31, 110. Schonemann, P. H., & Carroll, R. M. (1970). Fitting one matrix to another under choice of a central dilation and a rigid motion. Psychometrika, 35, 245255. Simpson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 8298). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201210. Stone, C. A., & Yeh, C. C. (2006). Assessing the dimensionality and factor structure of multiple choice exams. Educational and Psychological Measurement, 66, 193214. Swaminathan, J., & Gifford, J. A. (1983). Estimation of parameters in the threeparameter latent trait model. In D. J. Weiss (Ed.), New horizon in testing (pp. 1330). New York: Academic. Tate, R. (2003). A comparison of selected empirical methods for assessing the structure of responses to test items. Applied Psychological Measurement, 27, 159203. Thissen, D. (1991). MULTILOG user's guide: multiple, categorical item analysis and testscoring using item response theory [Computer program]. Chicago, IL: Scientific Software International. Thissen, D., & Wainer, H. (1982). Some standard errors in item response theory. Psychometrika, 47, 392412. Thompson, T. D., Nering, M., & Davey, T. (1997, June). Multidimensional IRTscale linking 1 ithi,,t common items or common examinees. Paper presented at the annual meeting of the Psychometric Society, Gatlinburg, TN. Thurstone, L. L. (1947). Multiple factor analysis. Chicago: University of Chicago Press. Vandenberg, R. J.,& Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practice, and recommendations for organizational research. Organizational Research Methods, 3, 470. Wang, M. (1985). Fitting a unidimensional model to multidimensional item response data: The effects of latent space misspecification on the application of RT. Unpublished manuscript. Wilson, D., Wood, R., & Gibbons, R. D. (1987). TESTFACT. Test scoring, item statistics, and item factor analysis. Mooresville, IN: Scientific Software. Wingersky, M. S., Cook, L. L., & Eignor, D. R. (1987). Specifying the characteristics of linking items usedfor item response theory item calibration (Research Report 8724). Princeton, NJ: Educational Testing Service. Wingersky, M. S., & Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347364. Yen, W. M., & Fitzpatrick, A. R. (2006). Item response theory. In R. L. Brennan (Eds.), Educational Measurement (4 ed.). West Port, CT: Praeger. Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). BILOGMG: Multiple group IRT analysis and test maintenance for binary items [Computer program]. Chicago, IL: Scientific Software International. BIOGRAPHICAL SKETCH Youhua Wei was born in China. He received his B.Ed. in school education from Nanjing Normal University in 1992 and his M.Ed. in psychology from East China Normal University in 1995. From 1995 to 1997, he worked as a psychological counselor at Southeast University in Nanjing. From 1997 to 2001, he worked as a research associate at Shanghai Academy of Educational Sciences. In 2004, he earned his M.S. in research, measurement, and statistics from Texas A&M University in College Station. He began his doctoral study in research and evaluation methodology in the Department of Educational Psychology at the University of Florida in fall 2004. He was awarded the Ph.D. degree in August 2008. PAGE 1 1 A SIMULATION STUDY ON THE PERFORMANCE OF FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS By YOUHUA WEI A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2008 PAGE 2 2 2008 Youhua Wei PAGE 3 3 ACKNOWLEDGMENTS I would lik e to express my sincere appreciati on to Dr. James J. Algina, my supervisory committee chair, for providing valuable guidance and support. I would also like to thank other committee members, Dr. M. David Miller, Dr. Walter L. Leite, and Dr. Zhihui Fang, for their time and effort on this project. I thank my parents and my brothers and sisters for their continuous and unconditional support and encouragement. Finally, I thank my wife, Yan Zhang, for her love and support. PAGE 4 4 TABLE OF CONTENTS page ACKNOWLEDGMENTS...............................................................................................................3 LIST OF TABLES................................................................................................................. ..........6 LIST OF FIGURES.........................................................................................................................7 ABSTRACT.....................................................................................................................................9 CHAP TER 1 INTRODUCTION..................................................................................................................11 Unidimensional IRT Models.................................................................................................. 13 Logistic Model.................................................................................................................13 Normal Ogive Model.......................................................................................................14 Unidimensional IRT Scale Linking........................................................................................ 14 Scale Transformation....................................................................................................... 14 Scale Linking...................................................................................................................16 Multidimensional IRT Models............................................................................................... 20 Logistic Model.................................................................................................................20 Normal Ogive Model.......................................................................................................23 Multidimensional IRT Scale Link ing..................................................................................... 25 Hirschs Method.............................................................................................................. 25 Lis Method.....................................................................................................................30 Mins Method..................................................................................................................33 Oshima and Colleagues Method....................................................................................35 Purpose of the Study........................................................................................................... ....40 2 METHODOLOGY................................................................................................................. 42 Design.....................................................................................................................................42 Independent Variables or Experimental Conditions........................................................42 Dependent Variables or Evaluation Criteria .................................................................... 47 Procedure................................................................................................................................49 Data Generation...............................................................................................................49 Parameter Es tim ation....................................................................................................... 51 Result Analysis................................................................................................................ 52 3 RESULTS...............................................................................................................................59 General Performance of the Different Linking Methods........................................................ 61 Performance of Linking Methods fo r Different Test Structures ............................................ 62 Performance of Linking Methods for Different Test Lengths ................................................ 63 Performance of Linking Methods for Different Sam ple Sizes...............................................64 PAGE 5 5 Performance of Linking Methods for Groups with Different Ability Distributions .............. 65 Performance of Linking Methods for Test Item s with Different Parameter Values.............. 67 4 DISCUSSION.......................................................................................................................122 Results from Previous Studies.............................................................................................. 122 Effects of Different Test Structures...................................................................................... 124 Effects of Different Test Lengths.........................................................................................126 Effects of Different Sample Sizes.........................................................................................127 Effects of Different Ability Distributions.............................................................................128 Effects of Different Item Parameter Values......................................................................... 130 Performance of Different Linking Methods.........................................................................131 5 CONCLUSIONS.................................................................................................................. 133 Conclusions...........................................................................................................................133 Future Research....................................................................................................................134 APPENDIX: ACCURACY AND STABILITY FOR DIFFERENT LINKING METHODS .....138 LIST OF REFERENCES.............................................................................................................186 BIOGRAPHICAL SKETCH.......................................................................................................193 PAGE 6 6 LIST OF TABLES Table page 21 Ability distributions for examinee groups......................................................................... 54 22 Item parameters for 20 items w ith approxim ate simple structure...................................... 55 23 Item param eters for 40 items with ap proximate simple structure...................................... 56 24 Item parameters for 20 items with complex structure....................................................... 57 25 Item parameters for 40 items with complex structure....................................................... 58 PAGE 7 7 LIST OF FIGURES Figure page 31 Accuracy and stability for different linking m ethods........................................................69 32 Accuracy and stability by li nking m ethod and test structure............................................. 72 33 Accuracy and stability by linking m ethod and test structure: N = 2000............................75 34 Accuracy and stability by linking method and test length for approxim ate simple structure tests................................................................................................................ .....78 35 Accuracy and stability by linking method and test length for com p lex structure tests: N = 500..............................................................................................................................81 36 Accuracy and stability by linking method and test length for com plex structure tests N = 1000:...........................................................................................................................84 37 Accuracy and stability by linking method and test length for com plex structure tests when G2 was excluded: N=1000....................................................................................... 87 38 Accuracy and stability by linking method and test length for com p lex structure tests: N = 2000............................................................................................................................90 39 Accuracy and stability by linking m ethod and sample size............................................... 93 310 Accuracy and stability by linking method and sample size for approxim ate simple structure tests................................................................................................................ .....96 311 Accuracy and stability by linking method and sample size for com plex structure tests... 99 312 Accuracy and stability by linking me thod and group for approxim ate simple structure tests................................................................................................................ ...102 313 Accuracy and stability by linking method and group for com plex structure tests: N = 500....................................................................................................................................105 314 Accuracy and stability by linking method and group for com plex structure tests: N = 1000..................................................................................................................................108 315 Accuracy and stability for differe nt linking m ethods: COM, n=40, N=1000, G2...........111 316 Accuracy and stability for differe nt linking m ethods: COM, n=20, N=1000, G2...........112 317 Accuracy and stability by linking method and group for com plex structure tests: N = 2000..................................................................................................................................113 318 Linking accuracy and stability and item para meter values: COM, n=20, N=1000, G3.. 116 PAGE 8 8 319 Linking accuracy and stability and item para meter values: APP, n=40, N=2000, G4.... 119 PAGE 9 9 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy A SIMULATION STUDY ON THE PERFORMANCE OF FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS By Youhua Wei August 2008 Chair: James J. Algina Major: Research and Evaluation Methodology Scale linking is the process of developing th e connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. It is the prerequisite for many applications of IRT, such as test equa ting and differential item functioning analysis. Unidimensional scale linking methods have been studied and applied frequently over the past two decades. The development of multidimensional linking methods is at the infancy stage and more research is needed to obtain definitive results. As an extension of previous research, the purpos e of this study was to use simulated data to evaluate the performance of f our multidimensional IRT scale linking methods, the direct method, equated function method, test ch aracteristic function method, and item characteristic function method, under various testing cond itions, which include different test structures, test lengths, sample sizes, and ability distri butions. There were one hundred and ninetytwo experimental conditions in this study and five hundred replicat ions were conducted for each of the conditions. The linking performance evaluation was based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for the equated group PAGE 10 10 across the test items. The mean and standard deviation of the diffe rences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Our results indicate that for approximate simp le test structure, each of the four linking methods worked approximately equally well under all testing conditions. The results also suggest that for complex test structure: (a) The equated function method did not work well under any testing conditions, (b) th e performance of other three linki ng methods depended on other testing conditions including sample size, test length, and abil ity distribution difference between groups, and (c) the direct method was the best linking pr ocedure for most testing conditions. In addition, the study shows that the item para meter values influenced the li nking performance. Under most of the testing conditions, the li nking results for the discriminati on parameter tended to be less accurate and less stable when the item parame ter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depended on the item pa rameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values. PAGE 11 11 CHAPTER 1 INTRODUCTION Suppose a set of test items is administered to nonequivalent groups of examinees and item response theory (IRT) is used to estimate the item parameters for each of the groups. The parameter estimates will be on different scales because the metric defined by each separate calibration is different (Stocking & Lord, 1983). Specifically, IRT parameter estimation procedures often scale the ability for each group with mean of 0 and st andard deviation of 1, although the actual ability distributions of the two groups may be different (Kolen & Brennan, 2004). Therefore, to compare the parameter esti mates from different IRT calibrations, they should be transformed on the same scale. Sc ale linking is the pro cess of developing the connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. The objective is to establish a comm on metric for all sets of parameter estimates. Scale linking is an important issue in psychometrics, and many applications of IRT require that item parameter estimates from i ndependent calibrations be expressed on the common metric, including test equating and differentia l item functioning (DIF) (Stocking & Lord, 1983). Based on Kolen and Brennan (2004), equating is a statistic al process that is used to adjust scores on test forms so that scores on the forms can be used interchangeably. (p. 2), and linking refers to relating scores on tests which are not built to the same content or statistical specifications. Different terminologi es have been used to descri be the process of establishing relationship between scores on two or more te sts (for a complete re view, see Kolen, 2004a, 2004b). Scale linking is used in this study to refer to the process of linking different scales rather than the process of linking test scores. However, scale linking is the prer equisite for establishing the connection between different test scores. Therefore, scale linking is an important step in test PAGE 12 12 equating (Cook & Eignor, 1991; Kolen & Brennan, 2004) and satisfactory equating results require successful scale linking. If different groups of examinees have diffe rent probabilities of success on an item after they have been matched on the ability of intere st, the item has differential functioning. In IRT, DIF is defined as the differences in the model parameters for the comparison groups (Clauser & Mazor, 1998). The item parameters for different groups should be compared only after they are placed on a common metric. Therefore, DIF iden tification depends heavily on the quality of scale linking. Some procedures have been developed to det ect DIF by improving scale linking (Candell & Drasgow, 1988; Lautenschlager & Pa rk, 1988; Lautenschlager Flaherty, & Park, 1994; Park & Lautenschlager, 1990). In addition to psychometrics, scale linking is also very important to educational and psychological studies. Multigroup confirmatory factor analysis or mean and covariance structure analysis has been increas ingly used to compare constructs across different groups (for a comprehensive review, see Vandenberg, 2000) and so me unresolved issues are closely related to the difficulty of linking scales across groups (Millsap, 2005). Therefore, successful scale linking has the potential to produce sa tisfactory comparison studies on ps ychological constructs across different groups. In sum, scale linking is very important for educational measurement to be fair and objective for different groups of examinees. Unid imensional scale linking methods have been studied and applied frequently over the past tw o decades (for more information, see Kolen & Brennan, 2004; Yen & Fitzpatrick, 2006). Th e development of multidimensional linking methods (Davey, Oshima, & Lee, 1996; Hirs ch, 1988, 1989; Li, 1997; Li & Lissitz, 2000; Min, 2003; Oshima, Davey, & Lee, 2000) is just at the infancy stage and more research is needed to PAGE 13 13 obtain definitive results (Yen & Fitzpatrick, 2006). In this chapter, unidimensional and multidimensional models and linking methods are reviewed and the purpose of the current study is presented. Unidimensional IRT Models Logistic Model The threeparameter logistic (3PL) model (see Hambleton & Swaminathan, 1985; Lord, 1980) assumes that the probability of a correct answer to a dichotomously scored item j by an examinee with ability i is 1 1 1 1 1 ,,;1][jij jij jijba j j ba ba j j jjjiijije cc e e cc cbaxP (11) where ijx is the item response (0 or 1) for person i on test item j jais the item discrimination parameter, jbis the item difficulty parameter, and jcis the guessing parameter or the pseudochance score level, representing the probability of correct res ponse when the ability assessed by the item is very low. Sometimes the 3PL model is expressed as ] [1 1 1 ,,;1jijbDa j jjjjiijije cccbaxP (12) with D=1.701, so that a normal ogive model item ch aracteristic curve (ICC) and a logistic model ICC with the same item parameters are almost identical. If jc is 0, the 3PL model becomes twoparameter logistic (2PL) model: PAGE 14 14 ][1 1 ,;1jijba jjiijije baxP (13) For 2PL model, if ja is 1, it becomes oneparameter logistic (1PL) model or Rasch model: jib jiijije bxP 1 1 ;1. (14) Normal Ogive Model There are also three normal ogive models or cumulative normal distribution models in IRT: one parameter model: jib t jiijdte bP 22 12 1 ; ; (15) two parameter model: jjiijijbaxP ,;1 = jijba tdte22 12 1 ; (16) and three parameter model: jjjiijijcbaxP ,,;1 = jijba t j jdte cc22 12 1 1 (17) Many IRT models have been developed for te st items that are polyt omously scored using ordered categories, including graded response model (Samejima,1969), partial credit model (Masters, 1982), generalized partial credit mo del (Muraki, 1992), rating scale model (Andrich, 1978), and nominal response model (Bock, 1972). Unidimensional IRT Scale Linking Scale Transformation The IRT parameter estimates produced from in dependent calibrations using data obtained from different groups of examinees are often on di fferent metrics. Lord (1980) demonstrated that PAGE 15 15 the relationship between the metrics of any two independent item calibrations is linear. Therefore, a linear equation can be used to transform the IRT parameters on scale E (representing the linked scale or equated scale) to scale F (representing the base scale). For person i and item j BAi iE F*, (18) A a aj jE F*, (19) BAbbj jE F*, (110) j jEFcc *, (111) where *iF, *jFa, *jFb, and *jFc represent the transformed values from the linked scale to the base scale. A is the slope and B is the intercept. The constants A and B can be expressed as *j jF Ea a A (112) j j i iE FE FAbbAB *. (113) A and B can also be expressed for any two individuals i and *ior two items j and *j: * j j j j i i i iEE FF EE FFbb bb A (114) i i j jE FE FA AbbB (115) or expressed for groups of items or examinees (see Kolen & Brennan, 2004): F E E F E Fa a b b A (116) E F E FA bAbB (117) PAGE 16 16 The iEijP value for the original parameters on scale E will be the same as the iF ijP* value for the transformed parameters on scale F as demonstrated by 1 1 1 1 1 1 1 1 1**** *i j E i E j E j j j E i E j E j j j F i F j F j j iEij b Da E E BAbBA A a D E E b Da F F F ijP e cc e cc e cc P Therefore, the logistic functi on is invariant under a linear transformation of item and ability parameters. Most of the unidimensional IRT scal e linking methods are based on this important feature. Scale Linking In practice, both test item parameters and examinees ability parameters need to be estimated and the ability estimates are often scaled to have means of 0 and standard deviations of 1. Parameter estimates obtained from different groups of examinees are often on different scales due to nonequivalence of the groups even though a ll ability estimates are s caled with means of 0 and standard deviations of 1. Therefore, some da ta collection procedures are required to establish the connection between different scales by using the linear transformations mentioned above. In test equating, three data collection designs ar e often used, including random groups design, single group design, and commonitem nonequi valent groups design. The IRT parameter estimates for the first and second designs are assu med to be on the same scale because of the randomly equivalent groups of examinees and single group of examinees (Kolen & Brennan, 2004) if random sampling errors are ignored. For the third design, the parameter estimates are PAGE 17 17 assumed to be on different scales due to the nonequivalent groups. The third design is the most often used equating design (Kolen & Brennan, 2004) an d it is very similar to the design used for exploring DIF. Two approaches have been used to establish a comm on scale for parameter estimates for this design. One is to estimate parameters for all items on both test forms together. This method is often called concurrent calibration (Wingersky & Lord, 1984). Both BILOGMG (Zimowski, Muraki, Mislevy, & Bock, 1996) an d MULTILOG (Thissen, 199 1) have the function of simultaneously obtaining parameter estimates for two test forms and two groups on the same scale. The second approach is to link the two scales by using the parameter estimates for the common items. This study will focus on the second approach. The following IRT linking methods have been developed to establish a common metric for parameter estimates. Mean/sigma method. This method (Marco, 1977) uses the means and standard deviations of the b parameter estimates for the common items to calculate the constants A and B in the linear transformation equation: E Fb b A E FbAbB (118) Mean/mean method. This method (Loyd & Hoover, 1980) uses the means of a parameter estimates for the common items to calculate A and the means of b parameter estimates for the common items to calculate B in the transformation equation: F Ea a A E FbAbB (119) Item response function method. In this procedure (Haebara, 1980), the constants A and B are estimated by minimizing the sum of the s quared difference between the item characteristic curves for the common items over examinees: PAGE 18 18 ij E E E FijFFFFij diffj j j i jjjicBbA A a PcbaP H2 ; ; (120) Test response function method. The constants A and B are estimated by minimizing the sum of the squared difference between the test characteristic curves for the common items for examinees (Stocking & Lord, 1983): ij E E E Fij FFFF j ij diffj j j i jjjicBbA A a PcbaP SL2 ; ; (121) Item response function method and test resp onse function method are often referred as the characteristic curve methods (Stocking & Lord 1983). Specifically, the former is called item characteristic method and the latter test characteristic curve method. Minimum 2method. This method (Divgi, 1985) combines information of each items parameter estimates and the variancecovariance matrix of sampling errors for each item from the item parameter estimation procedure. The c onstants A and B are estimated by minimizing the following quadratic function: 1 2 j E F E F FF E F E FBbAb A a a BbAb A a aj j j j j j j j j j, (122) where jF is the estimated variancecovariance matrix of the sampling errors for the item parameter estimates for item j on the F scale and *jFis the estimated variancecovariance matrix of the sampling errors for the item parameter estimates for item j which are transformed from the E scale to the F scale. Comparison studies have been conducte d for these methods with dichotomous IRT model. Based on a comprehensive literature review (Kolen & Brennan, 2004): (a) The PAGE 19 19 characteristic curve methods produced more stable and accurate results than the mean/mean and mean/sigma methods, (b) the mean/mean method was more stable than the mean/sigma method, (c) the concurrent calibration method yielded more accurate results than the test characteristic curve method for a small number of common items and both procedures had the similar accuracy for a larger number of common items, and (d) the concurrent ca libration method might be less robust to violations of the IRT assumptio ns than characteristic curve methods. These methods have been extended to li nk scales with polytom ous IRT models. For example, Cohen and Kim (1998) extended mean/mean and mean/sigma methods to the graded response model and Kolen and Brennan (2004) suggested using mean/m ean and mean/sigma methods for the generalized partial credit mode l. Baker (1992) generalized the test response function method to the graded response model a nd Baker (1993) used the item response function for the nominal response model. Kim and Cohen (1995) tried the minimum 2method for the graded response model. There are also some comparison studies for these methods with polytomous IRT models. A simulation study (Cohen & Kim, 1998) comparing the mean/mean method, mean/sigma method, weighted mean/sigma method, te st response function method, and minimum 2 method for the graded response model found that all methods produced similar results. Another simulation study (Kim & Cohen, 2002) compari ng the test response function method and the concurrent calibration method for the graded response model found that the concurrent calibration was relatively more accurate. PAGE 20 20Multidimensional IRT Models Logistic Model Unidimensional IRT models appear to be adequate for scaling achievement test items in most practical situations (Yen & Fitzpatrick, 2006). However, it is reasonable to believe that the performance of examinees on some test items depends on more than one trait or ability and some consequences of applying unidimensional models to multidimensional data have been identified (see Yen & Fitzpatrick, 2006). Th e number of dimensions necessa ry to model the test item responses depends not only on the number of ability dimensions and the level on those dimensions exhibited by the examinees but also on the number of skills to which the test items are sensitive (Reckase, 1997a). Therefore, mu ltidimensionality can occur in different ways depending on the interaction betw een a specific group of examinees and certain set of test items. There are two types of multidimensional IRT (MIRT) models for dichotomously scored item response data: the compensatory mode l and the noncompensatory model. In the compensatory model, a low value on one dimension can be compensated for by a high value on another dimension (McKinley & Reckase, 1983; Reckase, 1997a). In the noncompensatory model, an increase in the value on one dimension cannot compensate for a lower value on another dimension (Simpson, 1978). Since estimation programs and linking methods have not been well developed for non compensatory model, the most often used compensatory model is discussed and used in this study. The compensatory multidimensional threepa rameter logistic (M3PL) model is a direct generalization of the unidimensi onal 3PL model (Reckase, 1997a): PAGE 21 21 1 1 1 1 1 ),1(' 'jij jij jijd j j d d j j jjjiije cc e e cc c,d xP a a aa; (123) where ),,1(jjjiijcd xPa; is the probability of a correct response (1 ijx) for person i on test item j ijx is the item response (0 or 1) for person i on test item j ja is the vector of item discrimination parameters, jd is the scalar parameter related to the difficulty of the item, jc is the lower asymptote or guessing parameter, and i is the vector of ability parameters for person i. This model can be expressed in the following scalar form: 1 1 1 1 1 ),,;1(1 1 1 j d m k ikjk a j d m k ikjk a j m k ikjke cc e e cc cdaxPj j da j j jjjiij (124) where m is the number of dimensions. When jc is 0, the compensatory M3PL model becomes the compensatory multidimensional twoparame ter logistic (M2PL) model (McKinley & Reckase, 1983): jij jij jijd d d jjiije e e d xP a a aa; '1 1 1 ),1 ( (125) PAGE 22 22 This model can also be expressed as the following scalar form: j d m k ikjk a j d m k ikjk a j m k ikjke e e daxPda jjiij1 1 11 1 1 ),;1( (126) Compared with unidimensional IRT models, multidimensional discrimination and ability parameters are described in the form of vectors instead of scalars. If the i dimensions are orthogonal, the observed correlations among th e item scores will be accounted for by theja parameters. Otherwise, the item co rrelations will reflect both the japarameters and correlated dimensions. In MIRT, the probability of a correct response to an item depends on multidimensional ability and is defined as an item characteristic surface (ICS). Assuming orthogonal axes of dimensions in the surface, an item j can be described by the following three characteristics (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991): multidimensional discrimination (MDISCj): m k jk ja MDISC1 2, (127) which is the discrimination power of the item for the most discriminating combination of dimensions; multidimensional difficulty (MDIFFj): j j jMDISC d MDIFF (128) which, similar to the difficulty parameter in unidimensional model, is the distance from the origin of the space to the point of steepest slope in a direction from the origin; and direction (jk ) of the greatest slope from the origin: j jk jkMDISC a arccos (129) PAGE 23 23 which is the angle that the line from the origin of the space to the point of steepest slope makes with the k th axis for the item. Normal Ogive Model By adapting Thurstones multiple factor model (1947) to dichotomous item response data, Bock and Aitkin (1981) proposed a multidimensional normal ogive model by firstly assuming that an unobserved continuous response variable, ijy for person i and item j is a linear combination of m latent variables, weighted by the factor loadings, : ijmijm ijijijy 2211, (130) where ~, 0IN, 10y,~N, and 2,~jN 0 (Note that the s in Equation 130 are not the s in Equation 129). It is assumed that there is an underlying process which generates a correct observed response,1ijx when ijy equals or exceeds a threshold, j and produces an incorrect observed response, 0 ijx otherwise. Then the probability of obtaining a correct item score is 2 1 exp 2 1 ,;12 ij j m k kijk j ij j m k kijk ij j jjiijijdy y xPj (131) where m k jk j 1 2 21 This is a compensatory model because greater ability on one dimension can make up for lesser ability on other dimens ions. This model can be reparameterized to PAGE 24 24 produce similar parameters in multidimensional logistic model (Bock, Gibbons, & Muraki 1988; Muraki & Engelhard, 1985) by 2 exp 2 1 ,;12 ij z Z jjiijiji idtt dt t d xP a (132) where jjj m k ikjk iddaZ a' 1, (133) j jk jka (134) j j jd (135) It can also be shown that i jk jkq a, (136) j j jq d (137) with m k jk ja q1 2 21. (138) When s are correlated with covariance matrix it can be shown that ` 1jk jka (139) PAGE 25 25 ` 1j jd (140) aa`1 jk jka, (141) aa`1 j jd, (142) where is vector of factor loading for item j and a is the vector of discrimination for item j Multidimensional IRT models for polytomously scored test items have also been developed, including multidimensional logistic models and multidimensional normal ogive models (Kelderman, 1997; Muraki 1999; Muraki & Carlson, 1995). Multidimensional IRT Scale Linking The multidimensional scale linking is more complicated than the unidimensional scale linking because it involves the tran sformations of scale locations, variances, and covariances of several ability dimensions obtained from differen t calibrations and more technical problems need to be resolved. Just as MIRT can be considered ei ther as a special case of factor analysis or an extension of unidimensional IRT, the multidimensional scale linking can be realized either by borrowing methods from factor an alysis (Hirsch, 1988; Li, 1997; Min, 2003) or by extending the unidimensional IRT linking methods to the multi dimensional situations (Oshima et al., 2000). Hirschs Method Hirsch (1988) is possibly the first author to explore the feasibility and effectiveness of multidimensional linking and equating by using the commonexaminee design. Hirsch presented three technical issues in multidimensional linking and provided three possible resolutions. The first issue is to establish scale transformati ons to keep the M2PL function invariant. The following transformation equations can be used for a twodimension (dimension 1 and 2) M2PL model: PAGE 26 26 ,1 11 1 i i 2 22 2 i i, (143) ,11 1 j jaa 22 2 j jaa, (144) 2211 j jjjaadd (145) where parameters with superscript are transformed parameters on a new scale. The M2PL function is invariant by this transformation: .,;1 1 1 1 1 1 1 ),1(2211 2211 222111 2211 2 22 22 1 11 11) ()()( ) ( *** jjiij daa aad a a aad a a jjiijd xP e e e d xPjijij j jj ij ij j jj i j i ja a; This scale transformation method can be exte nded to M2PL models with more than two dimensions. Hirschs multidimensional scale li nking method was based on the invariance of multidimensional function under the above transf ormations of item and ability parameters. The second technical issue is that the corre lation between dimensions obtained from the first calibration may be somewhat different fr om the correlation estim ated from the second calibration due to some nonparallel items for commonexaminee design. If this occurs, the parameter estimates from two calibrations are co mposites or linear combinations of different basis vectors. Therefore, it is necessary to tr ansform the basis vectors from one calibration to those of the second calibration. This can be realized by transf orming the two sets of ability parameter estimates of the common examinees from two calibrations so that they are as similar as possible. PAGE 27 27 The third technical issue is the joint rota tional indeterminacy of the item discrimination and ability parameters. That is, the dimensions can be rotated and produce many possible sets of i and ja parameter estimates without affecting th e M2PL item characteristic function. As suggested by Wang (1985), the procrustean rotation in factor analysis (Schonemann ,1966) can be used to transform the parameter estimates from one calibration to those from the other calibration. Hirschs linking method for the commonexamin ee design includes four steps. In the first step, two sets of item and ability parameters ) (jiFFa and jiEEa for the common examinees but on different metrics are estimated from two independent calibrations. In (,), aijFF iF is a Nm matrix, where N is the number of examines, and ajF is a nm matrix, where n is the number of items. In the second step, three tr ansformations are used to obtain common basis vectors for the two sets of paramete r estimates. The first transformation by T1 refers the discrimination parameter estimates from the first calibration (jFa) to a set of orthogonal basis vectors instead of the basis vector s defined by the ability estimates (iF ). The second transformation by T2 1 refers the discrimination parameter estimates from the second calibration (jEa) to a set of orthogonal basis ve ctors instead of the basis ve ctors defined by the ability estimates (iE ). The third transformation by T3= T1* T2 1 refers the discrimination parameter estimates from the first calibration (jFa) to a set of common basis vectors for both calibrations. In the third step, orthogonal proc rustean transformation is used to rotate the ability estimates from the first calibration ( iF) to those from the second calibration (iE ). This fourth transformation matrix T4 can be found by minimizing the sum of squared difference between PAGE 28 28 each element of the two sets of ability parameter estimates (iF ) and (iE ). The method was called orthogonal procrustean transformation developed by Sc honemann (1966). Specifically, suppose S = iiEF ', 'PDPSS and 'QDQSS, then 4PQ T Given the above four transformations, the means and standard deviatio ns of the ability parameters for the common examinees from the two calibrations are estimated in the fourth step. For the commonexaminee design, the linking parameters can be estimated by equating the means and standard deviations of the ability estimates from the first calibration (iF ) and those transformed from the second calibrations (*iF). The linking parameter estimates are th en used to transform the parameter estimates which have already been transformed by the procedure described in the second step. For example, suppose one uses the common examinee design a nd the M2PL model with two dimensions, the following relations exist: 1 1 1 1 1 1E EE F FFi i 2 2 2 2 2 2E EE F FFi i (146) So 1 1 1 1 1 1 1 1F E F F E E E Fi i 2 2 2 2 2 2 2 2F E F F E E E Fi i (147) PAGE 29 29 Let 1 1 1 11F F E EM 2 2 2 22F F E EM (148) 1 11 F ES 2 22 F ES (149) Then the transformed parameters from E scale to F scale are 1 1 *1 1S Mi iE F 2 2 *2 2S Mi iE F (150) 1 *1 1 j jE FaSa, 2 *2 2 j jE FaSa, (151) 2 1 ***2 1MaMaddj j j jE EEF, (152) where the parameters with as superscript and F as subscr ipt on the left side of equations are the final transformed parameters on F scale, and the parameters with as superscript and E as subscript on the right side of equations are the transformed parameters on E scale by the first three transformations. The function of this fourstep scale linki ng procedure for M2PL model was evaluated by test equating results performed on both simulate d and real data sets us ing the commonexaminee PAGE 30 30 design (Hirsch, 1989). The equati ng results were examined by comparing the mean differences and the mean absolute differences of the true sc ores and ability estimates between the base tests and equated tests. Satisfactory equating was found for true scores but not for ability estimates. Hirschs linking method was originally developed for the commonexaminee design. However, it can easily be modified to c onduct scale linking for commonitem nonequivalent groups design which is most usually used in test equating and DIF st udy. As Hirsch (1988) suggested, the basis vector tran sformation would be the same. The procrustean transformation would use the common item discrimination parameters instead of the ability parameters. The item difficulty parameter for each item would need to be regressed onto each of the ability dimension parameters and therefore produce one unique difficulty parameter for each of the dimensions (Reckse, 1985). Then the mean a nd sigma method would be used for the common item difficulty parameters for the final transforma tion. However, more study is needed to verify the adequacy of this modified linking procedure. Lis Method Compared with Hirschs procedure, Lis (1997) multidimensional linking methods are more straightforward and consis tent with MIRT computer es timation programs. Most MIRT programs solve the identification problem by requi ring multidimensional abilities be distributed as multivariate normal MVN (0, 1). Therefore, the metric of the item parameter estimates is typically referred to orthogonal reference axes with unit lengt h. Given this condition, one reference system can be transformed onto the other reference system by a composite transformation: an orthogonal pr ocrustean transformation for rerotating the reference system, a translation transformation for shifting the point of origin, and a single di lation for rescaling unit length. Specifically the following equations are used in the reference system transformation: PAGE 31 31j jE FkaTa' *, (153) mTa' *j j jEEFdd, (154) m T i iE Fk1 */1. (155) It can be shown that the M2PL function is invariant to these transformations: ).,;1( 1 1 1 1 1 1 1 1 ),;1(' ' 1 1 '/1 /1 ***jji j E i E j E j E j E j E i E j E j E j E i E j E j E j E i E j E jjiEEEij d d d k k d k k FFFijd xP e e e e d xP a a a Tma Tma a mTa m TTa mTa m T aT The question is how to find T, m, and k. Li (1997) proposed seve ral methods to estimate the scale linking parameters. The rotation matrix T can be estimated by orthogonal procrustean transformation procedure as mentioned in Hirschs method above. Let S = jjEFaa', 'PDPSS and 'QDQSS then 'PQ T The origin shift coefficient m and unit change coefficient k can be estimated simultaneously by minimizing the sum of squa red difference between test characteristic functions for the common items obtained from the two calibrations, which was originally developed by Stocking and Lord (19 83) for the unidimensional linking: N i n j n j FFF FFFjj j jj jdPdP N kf1 2 11 *** ; ; 1 a a m, (156) where N is the number of grid points of values. PAGE 32 32 The origin shift and unit change coefficients can also be estimated separately by different procedures. For example, the origin shift coeffi cient can be estimated by minimizing the sum of squared difference between the two difficulty parameter estimates obtained from two calibrations: n j FFj jddf1 2 m, (157) where n is the number of common items. This was called least squares procedure (Li, 1997). The unit change coefficient can be estimated as th e ratio of the square root of the maximum eigenvalues of the matrices jjFFaa' and jjEEaa'obtained from the two calibrations: jj jjEE FFsig Maximum sig Maximum kaa aa ', (158) where sig represents the singular va lue or the nonnegative square r oots of the eigenvalue. This was called ratio of eigenvalues procedure (Li, 19 97). Similar to the leas t squares procedure for the estimation of origin shift coefficient, the unit change coefficient can also be estimated by minimizing the sum of squared difference betwee n the two sets of discrimination parameters estimated from two calibrations. This is also referred as least squares procedure (Li, 1997): n j FFj jkf1 2 aa. (159) The rotation matrix T and unit change coefficient k can also be estimated simultaneously by a least squares method developed for fitting one matrix to another through a rotation matrix, a translation vector, and a central dilation vector (Schonemann & Ca rroll, 1970). In this case, the rotation matrix and dilation scalar were estimate d by minimizing the sum of squared errors of the following residual matrix: PAGE 33 33 'j jE FkaTaE (160) It can be shown that '' 'Taa aajj jjCFCE CFCFtrace k trace, (161) where j j j j j jEE CEFF CFaaaaaa (162) with jFaas the mean of jFa and jEaas the mean of jEa This was called ratio of trace procedure (Li, 1997). The translation vector was not estimat ed by this method because item discriminations can not provide information about origin shift. Comparing the effect of different combinati ons of reference, tran slation, and dilation transformation procedures on the multidimensiona l linking parameters estimation, Li (1997) found that the most appropriate MIRT linking met hod is the combination of procrustean rotation approach (for dimensional transf ormation), the ratio of trace pr ocedure (for dilation), and the least square procedure (for tr anslation). This linking method could produce accurate estimation of item parameters, approximately equivalent estimation of ability parameters, but unsatisfactory true score estimation. Mins Method Min (2003) challenged Li s (1997) two reasons for using a single dilation parameter, that is, mathematical tractability and the assumpti on of constant variance across dimensions, and argued that one single dilation is insufficient fo r describing the scale unit changes for multiple dimensions. Two independent calibrations may change the scales of the multidimensional dimensions to different degrees. To address this problem, Min (2003) modified Lis (1997) method by replacing the single d ilation parameter with a diagona l dilation matrix to model PAGE 34 34 different unit changes on different dimensions The reference system transformations are performed as follows: j jE FaTKa''*, (163) mTa' *j j jEEFdd, (164) m TK i iE F 11 *, (165) where K is a diagonal dilation matrix. It can be shown that the M2PL function is invariant to these transformations: ).,;1( 1 1 1 1 1 1 1 1 ),;1(' ' 11 11 ''***jji j E i E j E j E j E j E i E j E j E j E i E j E j E j E i E j E jjiEEEij d d d d FFFijd xP e e e e d xPa a a Tma TKma a mTa m TKTKa mTa m TKaTK For twodimensional model, K becomes 2 10 0k kwhere 1k is the dilation parameter for the first dimension, and 2k for the second dimension. The least square method (Schonemann & Carroll, 1970) of estimating a rotation matrix, a translation vector, and a central dilation vector for fitting two matrixes can be followed to find T, K, and m in the transformation equations (Min, 2003). Mathematically Lis (1997) method and Min s (2003) method produce the same solution for T and m and the only difference of linking results co mes from the different dilation parameters. PAGE 35 35 Reckase and Martineau (2004) identified an important weakness in Lis (1997) and Mins (2003) method for MIRT models with high dime nsionality and provided a solution to the problem by employing a nonorthogonal procrustean transformation. However, this approach needs to be examined by further empirical studies. Oshima and Colleagues Method All multidimensional linking methods mentioned above borrowed an important procedure, procrustean rotation, from factor analysis to transf orm the dimensional axes. Oshima et al. (2000) extended four s cale linking methods within IRT from unidimensional to multidimensional models. According to their methods, the following equations were used to transform the IRT parameters on one scale E to another scale F (to distinguish IRT linking methods from the factor analysis methods de scribed above, different indices for linking parameters are used). For person i and item j j jE FaAa' 1*, (166) Aa1'j j jEEFdd*, (167) A *i iE F, (168) where the rotation matrix mm Aadjusts the variances and covarian ces of the ability dimensions (scale), and the translation vector 1 m changes the means of the ability dimensions (location) on the two scales. The model indetermin acy can be shown as the following: PAGE 36 36 ).,;1( 1 1 1 1 1 1 1 1 ),;1(' 1' 1' 1' 1' 1' 1***jji j E i E j E j E j E j E i E j E j E j E i E j E j E j E i E j E jjiEEEij d d d d FFFijd xP e e e e d xPa a a Aa Aa a Aa A Aa Aa A aA As in the unidimensional IRT, suppose that two nonequivalent groups of examinees take common test items and independent calibrations produce two sets of parameter estimates (jjFFd a) and (jjEEd a). These two sets of parameter estimates are on different scales F and E, and scale linking needs to be conducted to pla ce the two sets of parameter estimates on a common scale. Using the above equations, (jjEEd a) on E scale can be transformed to the F scale (** jjFFda ). The values of the two sets of item parameter estimates (jjFFd a) and (** jjFFda) should be similar due to the i nvariance of common item character istic in IRT. Unidimensional IRT linking methods can be extended to multidimensional IRT model to minimize some functions of the difference between the two sets of item parameters. Again, the question is how to find the values of A and so that the connection between th e two scales can be established. The direct method. This method was a multivariate extension of the minimum chisquare linking method for unidimensional IRT model (Divgi, 1985). The values of A and are estimated by minimizing the sum of squared diffe rence between the two sets of item parameter estimates over all items. However, the direct method is different fr om the original method in that PAGE 37 37 it does not consider the variancecovariance matr ix of sampling errors for item parameter estimates in the function: n j FF n j m k FFj j jk jkdd aa mn f1 2* 11 2*] [] [ 1 1 A, (169) where n is the number of items, m is the number of ability dimensions, and (** jjkFFda ) are transformed parameter estimates from E scale to F scale. The equated function method. This method is the multidimensional extension of the mean and sigma methods for the unidimensi onal IRT model (Loyd & Hoover, 1980; Marco, 1977). A more general system of s cale linking equations is used to specify that some functions of the common item parameters from the first calibration (jjFFd a) are equal to the same functions of the transformed common item paramete rs from the second calibration (** jjFFda). The transformed item parameter estimates can be obt ained by using the above scale transformation equations with the linking parameters A and The values of A and are estimated by minimizing the sum of squared difference between th e same functions of the two sets of selected elements of the estimated (jjFFd, a) and (**,jjFFda). The number of functions needed ( p ) depends on the number of dimensions ( m ) or elements in A and with p = m2 + m For example, in the two dimensional case (m = 2), four parameters in A and two parameters in need to be estimated. Therefore, six functions are required to estimate the six linking parameters and they could be the means of 1ja 2ja and jd for the first and second halves of the common items (or other block of items). PAGE 38 38 The scale linking functions are flexible in terms of which item parameter estimates to use and what function to use. Different systems of scale linking functions may produce different values of the linking parameters A and The quality or appropriateness of linking functions can be evaluated by their stability across random examinee samples, the character of the common item sets, and the true values of the linking parame ters (Davey, et al., 1996). For example, if the mean is the chosen linking function, the function to be minimized is p j FFp pp f1 2 1 A, (170) where pF FF 21 are the estimated means of p separate sets of elements of the estimated (jjFFd, a), and *** 21pFFFare the estimated means of p separate sets of elements of the estimated (**,jjFFda). The test characteristic function method. This method is an extension of the test response function method developed by Stocking and Lord (1983) for the unidimensional IRT model: 2 11 *** ; ; 1 n j n j FFE FFFjj j jj jdPdPW q f a a A (171) where q is the number of matching vectors, W is the weight taken at different values. The W is used to emphasize that some values are more important than others to estimate the linking parameters. The weight can also be considered equal along the ability scale. PAGE 39 39 The item characteristic function method. This method is the multidimensional generalization of the item response function me thod for unidimensional IRT model (Haebara, 1980): n j FFEFFFjj j jj jdPdPW qn f1 2 *** ; ; 1 a a A. (172) Based on a simulation study comparing the four IRT linking methods under different ability distributions (Oshima et al., 2000), all of the four methods were acceptable under almost any of the minimization criteria and offered dramatic improvement over not linking at all. It was also found that the test charac teristic function method and item characteristic function method were more stable and recovered the true linki ng parameters better than the direct method and equated function method. The multidimensional linking methods develope d by Hirsch (1988), Li (1997), Oshima et al. (2000), and Min (2003) can a ll be directly or indirectly performed for the commonitem nonequivalent groups design, which ha ve been a widely used in te st equating (Kolen & Brennan, 2004). Accordingly those methods have the potential for estab lishing calibrated item pool and exploring DIF. Another multidimensional linking method proposed by Thompson, Nering, and Davey (1997) can be used for test equating in a design without common items or examinees. With the assumption of the same origin, axes, and correlation between axes for the two randomly equivalent groups of examinees, this method so lve the rotational inde terminacy by identifying similar item content clusters on different tests and then rotating them in the same multidimensionalreference system. Further stud ies need to be conducted to evaluate the performance of this method. Multidimensional scale linking is a new research area. There have been very few studies conducted for each of the proposed methods (Hirsch, 1988; Li, 1997; Min, 2003; Oshima et al., PAGE 40 40 2000) and even fewer studies for comparing different methods in the litera ture. Therefore, it is currently difficult to evaluate the function of different me thods. The only comparison study by Min (2003) compared Lis me thod, Mins method, and Oshima and colleagues test characteristic function method in terms of accur acy and stability of scale transformations under different conditions varying in sample size, structure of dimensions, and ability distribution. The results indicate that both Oshima and colleagues and Mins methods were better in transforming discrimination parameters than Lis method, an d Mins and Lis methods performed better than Oshima and colleagues method in transforming the difficulty related parameters. In addition, Oshima and colleagues method performed better than Mins and Lis methods in transforming test true scores, and Lis and Mins methods were better than Os hima and colleagues method in maintaining the structure of dime nsions through orthogonal rotation. Purpose of the Study Based on the literature review of the multidim ensional linking methods, Lis methods have been evaluated under various circumstances such as different linking procedures, sample sizes, equating situations, number of anchor items, li nking situations, and ability distributions (Li, 1997). Mins method has also been examined with comparison with other methods under different conditions including di fferent sample sizes, dimensi onal structures, and ability distributions. The performance of Oshima and colleagues four IRT linking methods has been examined under fewer testing conditions, that is for different ability distributions, using simulation study with only 20 replications (Oshima et al ., 2000). A comparison study (Min, 2003) indicates that one of the f our IRT linking methods, that is, th e test characteristic function method, outperformed other methods in transforming item discrimination parameter estimates and equating true score estimates. This suggests that the IRT procedures are promising methods for multidimensional linking and equating. Further studies are needed to examine the PAGE 41 41 performance of these four methods under more tes ting conditions. As an extension of previous research (Oshima et al., 2000), the purpose of this study was to evaluate the performance of the four multidimensional IRT scale linking methods, the direct method, equated function method, test characteristic function method, and item characteristic function method, under various testing conditions, which include different test structures, te st lengths, sample sizes, and examinees ability distributions. PAGE 42 42 CHAPTER 2 METHODOLOGY A com prehensive review of the unidimensi onal scale linking and test equating (Cook & Petersen, 1987) provides us a framework for exploring the performance of multidimensional scale linking methods. According to Cook and Petersens discussi on, the results of linking and equating depend on linking or equating methods, sa mple characteristics, and properties of the common items. In addition, the multidimensional structure underlying the test item responses makes scale linking more complicated and shoul d be considered as one important testing condition. In this simulation study, the perfor mance of the four MIRT scale linking methods (Oshima et al., 2000) for the commonitem nonequiv alent groups design was evaluated with the compensator compensatory M2PL model under different testing conditions, including different test structures, test lengths, sample sizes, and examinees ability distributions. The M2PL model had two dimensions. Design Independent Variables or Experimental Conditions IRT linking method. This study was to evaluate the performance of the four multidimensional IRT scale linking methods propos ed by Oshima et al. (2000): the direct method, equated function method, test characteristic f unction method, and item characteristic function method (see the section Multidimensional IRT Scale Linking in Chapter 1 for detailed description). The equated function, test characte ristic function, and item characteristic function methods were implemented in a manner consiste nt with the implementation in Oshima and colleagues study (2000). For the equa ted function method, the means of 1ja, 2ja, and jd for the first and second halves of the items were used as the equated function. For the test and item characteristic function me thods, seven equally spaced 1 points from 4 to 4 and seven equally PAGE 43 43 spaced 2 points from 4 to 4, making 7 x 7 = 49 grid points, were used w ith equal unit weight along the ability scale. The four IRT linking methods have been compared under different ability distributions (Oshima et al., 2000). It is unknow n how they perform under other circumstances. Therefore, this study can be considered as an extension of Oshima and colleagues study (2000) from one testing condition (ability distribution) to various testing condi tions (see the following for the detail). Test structure. In IRT, the test dimensionality for a particular population is the minimum number of latent abilities required to produce a monotone and locally independent model (McDonald, 1981, 1997; Stout, 1990). In the geom etrical representation of a test structure, the coordinate axes of a multidimensional space is defined by a complete set of latent abilities examined by the test, and each item is described by a vector in the spac e with its orientation representing the ability composite that is best measured by the item (Ackerman, 1994, 1996; Reckase, 1985, 1991). According to the literature review by Tate (2003), based on the number and nature of the abilities required for the respons e to each item in the test there are three types of test structure: simple structure, approximate simple structure, and complex structure. In the simple structure, all item vectors are exactly aligned with one of the axes in the multidimensional space after an appropriate rota tion, so all the items under each dimension measure the same ability. If all item vectors are approximately aligned with one of the multiple axes and therefore the contribution of one ability is dominant over the contribut ion of all other abilities, the test has approximate simple structure. In complex test structure, the respons e to one item depends on more than one ability. The first type of structur e has been considered as an ideal one and the second and third types as more realistic item structures (Kim, 1994; R oussos, Stout, & Marden, 1998). Following the method in previous studies (Batley & Boss, 1993; Min, 2003; Mroch & PAGE 44 44 Bolt, 2006; Oshima et al., 1997; Oshima & Miller, 1992; Tate, 2003), two types of twodimension test structure were created by using the three MIRT item characteristics: MDISC, MDIFF, and direction (Ackerma n, 1994; Reckase, 1985; Reckas e, 1997a; Reckase & McKinley, 1991). In the approximate simple st ructure, there were two sets of items: The responses to the first half items depended on one composite ability with the firs t dimension as the dominant dimension and the second dimension as the minor dimension; The responses to the second half items depended on another composite ability with the second dimension as the dominant dimension and the first dimension as the minor di mension. In the complex test structure, there were four sets of items with equal number of ite ms in each of the set. Two sets of items loaded heavily on one of the two dimensions and lightly on the other dimension, and the remaining two sets loaded heavily on both dimensions. Test length. The test items are used to establish the common metric for the two sets of parameter estimates obtained in separate calibrati ons. Therefore, the feature of items is very important for sale linking. The estimation of linking parameters depends not only on the number of items, but also on the characteristics of the item parameters. Based on some literature reviews (Brennan 1987; Cook & Peters en 1987; Kolen & Brennan, 2004), 1530 common items are necessary for unidimensional IRT linking, althou gh the required number also depends on other conditions, such as the linking methods, examinees ability distributions, and characteristics of the items. Different numbers of items have been used in multidimensional linking studies. Li (1997) used 15 and 25 items in his study and found that the number of items had a significant influence on the stability of transformation parameter estimates for multidimensional linking. Oshima et al., (2000) created 40 item parameters to examine the performa nce of their four IRT multidimensional linking procedures. Twenty items were used in Mins study (2003) to compare PAGE 45 45 three multidimensional linking methods. In this study, 20 and 40 items were used to evaluate the four MIRT linking methods under different numbers of items with the cons ideration that more items may be needed for MIRT than unidimensional IRT linking. Sample size. Theoretically the performance of linking methods depends on the accuracy of parameter estimates and parameter estimation is affected by the sample size (Li, 1997). So the linking function depends in some extent on different sample sizes. Compared with unidimensional IRT models, a large number of examinees are required for MIRT calibration because more parameters need to be estimated. Based on some MIRT researchers (Ackerman, 1994; Carlson, 1987) recommendations, 2000 examinees is a reasonable sample size to obtain satisfactory item parameter estimates for compensatory multidimensional model. Reckase (1997a) reported that NORHARM (Fraser & Mc Donald, 1988) and TESTFACT (Wilson, Wood, & Gibbons, 1987) generally produced stable paramete r estimates for long te sts and sample sizes exceeding 1000 cases. A comprehensive study (Tate, 2003) using both simulated and real data found that most of the oftenused multidimensi onal computer programs performed well for the sample size of 2000 examinees. To acquire stable item parameter estimates, Hirsch (1988) used 2000 examinees to evaluate the proposed multidimen sional equating. In his first study, Li (1997) used three different sample sizes, 1000, 2000, and 4000, to examine the performance of three multidimensional linking methods and found that the sample size had a prominent role in estimating transformation parameters. Li used 2000 examinees to evaluate the best linking method in his second study (Li, 1997). Min (2003) also found the significan t effect of sample sizes, 500, 1000, and 2000 on the accuracy and stability of different multidimensional linking methods and suggested that the sample of 500 examinees showed unrel iable results and the sample of 1000 showed somewhat acceptable outcom es (note that approximate simple structure PAGE 46 46 and complex structure were used in the study). It is not unusual in testing practice that the sample size is less than 1000 especially in nonachievement area and the performance of the four IRT linking methods need to be evaluated under this c ondition. In this study, three different sample sizes, 500, 1000, 2000, were used to examine th e robustness of the f our multidimensional IRT scale linking methods against parameter estimation errors. As defined in other studies (Li, 1997; Min, 2003), the sample size of 2000 is the base fo r comparing the effect of different parameter estimation errors. The sample size of 500 can be used to examine the robustness of IRT scale linking for small sample size. The sample size of 1000 was used to examine the effect of sample size between 500 and 2000 on scale linking and it was also cons istent with a study using multidimensional linking for identifying differe ntial item functioning (Oshima et al., 1997). Examinees ability distribution. Based on the review by Ko len and Brennan (2004), the performance of scale linking also depends on the similarity between the two groups of examinees. The more similar the groups are, the more adequate the linking will be. Large difference between groups may produce sign ificant problems in estimating scale linking parameters. Groups of examinees may differ in many characteristics, such as cultural background, attitude, motivation, and persona lity. A comprehensive review on population invariance in equating and linking (Kolen, 2004) found that equa ting is population dependent except under highly restrictiv e conditions, such as tw o test forms with similar content, difficulty, and reliability. This suggests that scale linking para meters that are used to obtain the equivalent scores should also be dependent on the populat ions used in the estimation. The ability distribution is an important char acteristic of the examinees and ha s a significant influence on test equating and scale linking under both unidimensional (Cook et al., 1985) and multidimensional circumstances (Li, 1997; Min, 2003). As summarized by Cook and Petersen (1987), the PAGE 47 47 similarity of ability distributi on between groups also affects ot her conditions required for test equating, such as the nu mber of common items. Groups of examinees may differ from each other in terms of mean, variance, and covariance of the dimensions. Oshima et al. (2000) examined the four multidimensional IRT scale linking methods under six conditions of th e ability distributions across two groups: no difference at all; differences in variances; differences in correlations; differences in means; differences in means and variances; differences in means and correlations. Min (2003) used four conditions similar to those investigated by Oshima et al, such as differences in correlations; differences in correlations and means; and differences in correlations, means, and variances. However, in all conditions the ability dimensions were uncorrelated in the base group. In education and psyc hology, most constructs and dimensions within a construct are correlated. Two groups should have similar stru cture of construct before scale linking and equating are conducted. Given th ese two considerations and to keep the scope of the study manageable, correlations between dimensions were set at the same level, but not zero, across all groups and the two groups varied only in ability le vel and variance. One purpose of this study was to explore twodimensional linking methods under the followi ng four ability distributions: no difference at all, differences in means, differences in variances, and differences in means and variances (See Table 21 for the detail). Dependent Variables or Evaluation Criteria Different statistics have been used to evaluate multidimensional linking methods. Bias and root mean square error (RMS E) are often used to evaluate the accuracy and stability of results across replications of experiment in IRT simulation studies. For example, using a common examinee design, Hirsch (1988) evaluated the effectiveness of multidimensional linking PAGE 48 48 and equating by examining the means and standard deviations of the differences and absolute differences between the true scor es, ability estimates, test char acteristic response surfaces, and contour plots of the common examinees on the base and equated tests. Li (1997) used bias and RMSE to evaluate three multidimensional linking methods, but he used both the bias and RMSE of linking parameters and item and ability paramete rs over replications in his study. Oshima et al. (2000) compared the means, standard deviat ions, bias, and RMSE of linking parameters for different methods. Another criterion for common ite m scale linking in IRT fram ework is to evaluate how small the differences are between the item pa rameter estimates for base group and the transformed item parameter estimates for equa ted group across the common items (Min, 2003; Min & Kim, 2003). This criterion was used in this study. Specifically, the common item nonequivalent groups design was used and simulati on was performed to create the data for both base and equated groups. The parameters fo r the two groups were estimated and then transformed onto a common scale. Specifically, th e parameter estimates for equated groups were transformed onto the scale for the base groups by using the transformation equations described before. The linking coefficients in the transformation equations, A and were estimated through the four IRT multidimensional lin king methods. After the common item parameters estimated from base and equated groups were placed on the sa me scale, the performance of the four linking methods were evaluated by examining the differe nces between the two se ts of item parameter estimates. The mean difference and diffe rence variation across replications (r) for each item were used to evaluate the accuracy a nd stability of the four linking methods, as described by the following statistics: PAGE 49 49 r diff Mr r jdiff1a, (21) r diffdiff SDjdiff 2 a, (22) where j jFFdiffaa*, (23) r diff dMr r jdiff1, (24) r diffdiff dSDj 2 (25) where j jFFdddiff*. (26) Procedure Data Generation The following compensatory, twoparameter, tw odimension IRT model was used to create the item responses with different testing conditions described above: j d ij a ij aDe daxPjjiij 22111 1 ),;1(. (27) First, five sets of ability parameters fo r each of the three sample sizes (500, 1000, and 2000) with multivariate normal distributions with various means, variances, and covariances were generated. One set of ability parameters was used for the base group and the other for the four equated groups (see Table 21 for the five gr oup ability distributions). PAGE 50 50 Second, two sets of item parameters (one with 20 items and another with 40 items) for each of the two test structures (approximate simple structure and complex st ructure) were created using the three MIRT item characteristics: MD ISC, MDIFF, and dire ction (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKi nley, 1991). Based on the pooled results from past empirical studies (Reck ase, 1985; Reckase, 1997a; R eckase & McKinley, 1991), the estimated MIDSC has a lognormal distribution with mean of 1.37 and standard deviation of 0.54 and the estimated MDIFF has a normal distribution with mean of 0.28 and standard deviation of 0.69. The item parameters of MDISC and MDIFF in this study were selected randomly from lognormal and normal distributions with the same value of means and standard deviations. The test structure was created by mani pulating the angle of each item with the first dimension. For the items that loaded on one dominant dimension, the angle between the item and its dominant dimension was selected from a lognormal distribu tion with mean of 10 and standard deviation of 2. For the items loaded heavily on both di mensions, the angle betw een each item and two dimensions were selected from a normal distributi on with mean of 45 and standard deviation of 10. Next, the discrimination parameters, a1, a2, and the difficulty parameter, d, were computed by the following formula ((Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991): 1 1cos* MDISC a (28) 2 2cos* MDISC a (29) MDIFF MDISC d (210) (See Table 22, 23, 24, and 25 for specific parameter values for different test structures with different test lengths) PAGE 51 51 Next, dichotomous item responses were created using the twoparameter and twodimension IRT model desc ribed by Equation 27. To produce more precise and stable results, replications were conducted for each of the combinations of testing cond itions. In IRT simulation studies the number of replications depends on the purpose of the study, the desire of minimizing the sampling variance of the estimated parameters, and the need for statistical tests of results (Harwell, St one, Hsu, & Kirisci, 1996). The previous studies on multidimensional lin king or equating methods used 0 (Hirsch, 1988), 20 (Oshima et al., 2000), 50 (Min, 2003), 100, a nd 200 (Li, 1997) replications to evaluate the accuracy and stability of linking or equa ting results. Based on Harwell and colleagues (1996) recommendation of using a minimum of 25 replications for IRT simulation studies and given the level of complexity of this study, 500 replications were used for each of the combinations of testing conditions to evalua te the accuracy and stability of the four multidimensional IRT linking methods. Parameter Estimation The parameters of MIRT models can be esti mated using different methods and computer programs. The often used estimation methods in clude unweighted least squares (ULS) factor analysis of tetrochoric correlations, weighted least squares (WLS) analysis of the matrix of polychoric correlations, and robust WLS analys is methods performed by MPLUS (Muthen & Muthen, 1998), least squares es timation method based on the matr ix of raw product moments of item scores by NOHARM (Fraser & McDona ld, 1988), marginal maximum likelihood estimation method by TESTFACT (Bock, Gibbons, Schilling, Muraki, Wilson, & Wood, 1999). The study focusing on model parameters recovery by Knol and Berger (1991) suggests that for multidimensional data a common factor analysis on the matrix of tetrachoric correlations performs at least as well as the theoretically appropriate multidimensional item response models PAGE 52 52 (p. 457). A study comparing TESTFACT and N OHARM (Gosz & Walker, 2002) found that NOHARM provided better soluti ons for predicting item performance. The comprehensive comparison study by Tate (2003) found that MPLUS, NOHARM, and TESTFACT performed reasonably well over a relatively wide range of conditions in assessing the test structure and estimating parameters. This result was confir med by another recent study (Stone & Yeh, 2006). Based on these studies, all these methods can provide satisfactory estimation for model parameters. NOHARM was used in this study du e to its consistently good performance in previous studies. After the MIRT item parameters were estimated by NOHARM, the linking parameters estimated by the four multidimensional IRT lin king methods (direct method, equated function method, test characteristic function method, and item characteristic function method) were obtained by the computer progr am IPLINK, which was developed by Lee and Oshima (1996). Result Analysis Some previous multidimensional linking studies used descriptive analysis (Hirsch, 1988; Oshima et al., 2000) and some st udies used both descriptive and inferential analysis (Li, 2000; Min, 2003). In this study, the m eans and standard deviations of differences between the item parameter estimates for base group (jjFFd a) and the transformed item parameter estimates for equated group (** jjkFFda) across 500 replications were co mpared under different testing conditions. Specifically, the accuracy and stabili ty of the four multidimensional IRT linking methods were evaluated by examining the mean differences and diffe rence variations of 1a, 2a, and d for all items in the test under di fferent testing conditions. Based on the experimental conditions describe d above, there are 5 factors in this study: multidimensional linking method (4), test structur e (2), test length (2), sample size (3), and PAGE 53 53 ability distribution (4). Therefore, the total number of e xperimental conditions is 422 3 4 = 192. Five hundred replications were conducted for each of the conditions. PAGE 54 54 Table 21. Ability distributions for examinee groups ________________________________________________________________________ Base Group Group1 Group2 Group3 Group4 ___________ _________ __________ __________ __________ ________________________________________________________________________ 0 0 5. 1 1 5. 0 0 5. 1 1 5. 5. 5. 5. 1 1 5. 0 0 4. 8. 8. 4. 5. 5. 4. 8. 8. 4. ________________________________________________________________________ Note: All the correlations between dimensions are .5. PAGE 55 55 Table 22. Item parameters for 20 items with approximate simple structure Item a1 a2 d MDISC MDIFF 1 2 1 1.12 0.18 0.70 1.13 0.62 9 81 2 2.23 0.52 0.23 2.29 0.10 13 77 3 1.39 0.24 2.19 1.41 1.55 10 80 4 1.02 0.16 0.75 1.03 0.73 9 81 5 1.68 0.33 1.18 1.71 0.69 11 79 6 0.98 0.15 0.77 0.99 0.78 9 81 7 1.24 0.22 1.12 1.26 0.89 10 80 8 0.94 0.13 1.26 0.95 1.33 8 82 9 1.65 0.32 1.86 1.68 1.11 11 79 10 2.01 0.46 1.46 2.06 0.71 13 77 11 0.30 1.30 0.72 1.33 0.54 77 13 12 0.17 1.09 0.17 1.10 0.15 81 9 13 0.33 1.86 0.51 1.89 0.27 80 10 14 0.08 0.63 0.07 0.63 0.11 83 7 15 0.14 0.99 1.38 1.00 1.38 82 8 16 0.17 1.09 2.66 1.10 2.42 81 9 17 0.20 1.15 0.48 1.17 0.41 80 10 18 0.13 0.84 0.68 0.85 0.80 81 9 19 0.38 2.37 0.77 2.40 0.32 81 9 20 0.22 1.04 0.73 1.06 0.69 78 12 PAGE 56 56 Table 23. Item parameters for 40 items with approximate simple structure Item a1 a2 d MDISC MDIFF 1 2 1 2.29 0.44 1.28 2.33 0.55 11 79 2 1.10 0.21 0.21 1.12 0.19 11 79 3 1.44 0.20 1.45 1.45 1.00 8 82 4 0.57 0.09 0.28 0.58 0.48 9 81 5 0.92 0.16 1.59 0.93 1.71 10 80 6 0.96 0.20 0.45 0.98 0.46 12 78 7 1.18 0.23 0.79 1.20 0.66 11 79 8 1.12 0.16 1.07 1.13 0.95 8 82 9 0.98 0.19 0.80 1.00 0.80 11 79 10 1.90 0.30 0.94 1.92 0.49 9 81 11 0.55 0.14 0.27 0.57 0.48 14 76 12 1.35 0.26 0.40 1.38 0.29 11 79 13 1.16 0.23 0.14 1.18 0.12 11 79 14 1.66 0.29 0.32 1.69 0.19 10 80 15 1.39 0.24 1.66 1.41 1.18 10 80 16 0.79 0.14 0.65 0.80 0.81 10 80 17 1.08 0.19 1.50 1.10 1.36 10 80 18 1.18 0.15 0.46 1.19 0.39 7 83 19 0.81 0.16 0.07 0.83 0.09 11 79 20 0.70 0.11 0.74 0.71 1.04 9 81 21 0.21 1.32 0.66 1.34 0.49 81 9 22 0.27 1.19 0.94 1.22 0.77 77 13 23 0.29 1.62 0.20 1.65 0.12 80 10 24 0.15 1.20 0.86 1.21 0.71 83 7 25 0.19 0.98 1.09 1.00 1.09 79 11 26 0.15 0.77 0.12 0.78 0.16 79 11 27 0.21 1.53 1.97 1.54 1.28 82 8 28 0.15 0.97 0.35 0.98 0.36 81 9 29 0.26 1.45 0.90 1.47 0.61 80 10 30 0.13 1.09 0.23 1.10 0.21 83 7 31 0.17 0.86 0.18 0.88 0.20 79 11 32 0.28 1.59 0.37 1.61 0.23 80 10 33 0.18 0.84 0.18 0.86 0.21 78 12 34 0.23 1.48 0.60 1.50 0.40 81 9 35 0.45 2.58 0.10 2.62 0.04 80 10 36 0.22 1.40 1.75 1.42 1.23 81 9 37 0.32 1.38 0.62 1.42 0.44 77 13 38 0.17 1.05 0.25 1.06 0.24 81 9 39 0.14 0.89 0.63 0.90 0.70 81 9 40 0.28 1.42 0.44 1.45 0.30 79 11 PAGE 57 57 Table 24. Item parameters for 20 items with complex structure Item a1 a2 d MDISC MDIFF 1 2 1 1.12 0.18 0.70 1.13 0.62 9 81 2 2.23 0.52 0.23 2.29 0.10 13 77 3 1.39 0.24 2.19 1.41 1.55 10 80 4 1.02 0.16 0.75 1.03 0.73 9 81 5 1.68 0.33 1.18 1.71 0.69 11 79 6 0.22 0.96 0.77 0.99 0.78 77 13 7 0.20 1.24 1.12 1.26 0.89 81 9 8 0.16 0.94 1.26 0.95 1.33 80 10 9 0.20 1.67 1.86 1.68 1.11 83 7 10 0.29 2.04 1.46 2.06 0.71 82 8 11 0.99 0.89 0.72 1.33 0.54 42 48 12 0.55 0.95 0.17 1.10 0.15 60 30 13 1.26 1.40 0.51 1.89 0.27 48 42 14 0.49 0.40 0.07 0.63 0.11 39 51 15 0.60 0.80 1.38 1.00 1.38 53 37 16 0.87 0.68 2.66 1.10 2.42 38 52 17 0.83 0.83 0.48 1.17 0.41 45 45 18 0.68 0.51 0.68 0.85 0.80 37 53 19 1.48 1.89 0.77 2.40 0.32 52 38 20 0.56 0.90 0.73 1.06 0.69 58 32 PAGE 58 58 Table 25. Item parameters for 40 items with complex structure Item a1 a2 d MDISC MDIFF 1 2 1 2.29 0.44 1.28 2.33 0.55 11 79 2 1.10 0.21 0.21 1.12 0.19 11 79 3 1.44 0.20 1.45 1.45 1.00 8 82 4 0.57 0.09 0.28 0.58 0.48 9 81 5 0.92 0.16 1.59 0.93 1.71 10 80 6 0.96 0.20 0.45 0.98 0.46 12 78 7 1.18 0.23 0.79 1.20 0.66 11 79 8 1.12 0.16 1.07 1.13 0.95 8 82 9 0.98 0.19 0.80 1.00 0.80 11 79 10 1.90 0.30 0.94 1.92 0.49 9 81 11 0.09 0.56 0.27 0.57 0.48 81 9 12 0.31 1.34 0.40 1.38 0.29 77 13 13 0.20 1.16 0.14 1.18 0.12 80 10 14 0.21 1.68 0.32 1.69 0.19 83 7 15 0.27 1.38 1.66 1.41 1.18 79 11 16 0.15 0.79 0.65 0.80 0.81 79 11 17 0.15 1.09 1.50 1.10 1.36 82 8 18 0.19 1.18 0.46 1.19 0.39 81 9 19 0.14 0.82 0.07 0.83 0.09 80 10 20 0.09 0.70 0.74 0.71 1.04 83 7 21 1.00 0.90 0.66 1.34 0.49 42 48 22 0.61 1.06 0.94 1.22 0.77 60 30 23 1.10 1.23 0.20 1.65 0.12 48 42 24 0.94 0.76 0.86 1.21 0.71 39 51 25 0.60 0.80 1.09 1.00 1.09 53 37 26 0.61 0.48 0.12 0.78 0.16 38 52 27 1.09 1.09 1.97 1.54 1.28 45 45 28 0.78 0.59 0.35 0.98 0.36 37 53 29 0.91 1.16 0.90 1.47 0.61 52 38 30 0.58 0.93 0.23 1.10 0.21 58 32 31 0.61 0.63 0.18 0.88 0.20 46 44 32 1.22 1.06 0.37 1.61 0.23 41 49 33 0.49 0.70 0.18 0.86 0.21 55 35 34 1.35 0.66 0.60 1.50 0.40 26 64 35 2.04 1.65 0.10 2.62 0.04 39 51 36 1.07 0.93 1.75 1.42 1.23 41 49 37 1.04 0.97 0.62 1.42 0.44 43 47 38 0.88 0.59 0.25 1.06 0.24 34 56 39 0.42 0.79 0.63 0.90 0.70 62 28 40 1.11 0.93 0.44 1.45 0.30 40 50 PAGE 59 59 CHAPTER 3 RESULTS As described in Chapter 2, the criterion us ed in this study to evaluate the four m ultidimensional IRT linking methods was based on the differences between the item parameter estimates for the base group and the transforme d item parameter estimates for the equated group across 500 replications. Specifical ly, after the item parameter estimates from the two groups were transformed to a common scale, the mean a nd standard deviation of their differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. For each of the 192 experimental cond itions, there were three parameter estimates 1a, 2a, and ;d therefore, the mean and standard devia tion of the differences were computed for 1a, 2a, and dacross 500 replications for each item of the te st. Then the distributions of the means and standard deviations of the differences of 1a, 2a, and dfor all items in the test were obtained. Based on the characteristic of item parameter invariance in IRT, the item parameter estimates from the base and equated groups should theoreti cally be equal after they are transformed to a common scale. So their differences, and accordingl y the means and standard deviations of their differences across 500 replications, should be 0. Therefore, the performance of the four multidimensional IRT linking methods can be eval uated by examining how close the means and standard deviations of the differences are to 0. There is currently no generally accepted crit erion about how close the item parameter estimates for the two groups should be in order fo r the linking to be considered accurate and stable. To describe the distributi on of difference, histograms of m eans and standard deviations of the differences of 1a, 2a, and dfor the 192 experimental condi tions were prepared. The appendix contains the histograms for all 192 conditions In this chapter, histograms selected to PAGE 60 60 illustrate the trends in the results will be presented. The following midpoints were used to construct the histograms for means: 0, .2, .4, .6. All va lues smaller than 0.5 and larger than +0.5 were included in the categories wi th midpoints of .6. For the histograms of the standard deviations, 0.1, 0.3, 0.5, 0.7, 0.9, 1. 1, and 1.3 were used as the midpoints. All values beyond 1.2 were classified into the category with midpoi nts of 1.3. If all or most of the items in the test had means and standard deviations close to 0, the linking method was considered accurate and stable. Otherwise, the linking method was inaccurate and unstable. The performance of the four multid imensional IRT linking methods was evaluated in this way under different testing conditions. On the histograms, the direct method, equate d function method, test characteristic function method, and item characteristic function method are labeled Link1, Link2, Link3, and Link4. For test structure, the approximate simple structure is abbreviated as APP and the complex structure as COM. For test length, the number of items in the test is indicated by n = 20 or n = 40. For sample size, the number of examinees is indicated by N = 500, N = 1000, or N = 2000. For ability distribution differences between the base and equated groups, the condition is abbreviated as G1 if the mean vectors and covariance matrices were equal for the two groups, G2 if only the mean vectors were different, G3 if only the cova riance matrices were different, and G4 if the mean vectors and covariance matrices were not equal for the two groups. As will be shown subsequently, inspection of the results indicated that the effects of linking methods depended on the test structure. Therefore, the decision was made to focus primarily on the effects of linking methods within each of the test structures. Inspection of the results for APP suggested that the interactions of all other factors were small in size, therefore the focus was on the main effects of the factors. Inspection of the COM results suggested that PAGE 61 61 there were twoway, threeway, or fourway inter actions of other factors, so the performance of the linking methods were described taki ng into account th ese interactions. This chapter consists of six sections. The firs t section compares the general performance of the four linking methods. The second section compares the four linking methods for different test structures. The third section compares linking methods for tests with different lengths. The fourth section compares linking methods for diffe rent sample sizes. The fifth section compares linking methods for groups with different ability distributions. The la st section shows the relationship between scale linking perf ormance and item parameter values. General Performance of the Different Linking Methods The performance of the four m ultidimensiona l IRT linking methods was first compared across all testing conditions by collapsing the means and standard deviations of 1a, 2a, and dfor all items under different testing conditions The histograms in Figure 31 show the distributions of means and standard deviations for 1a, 2a, and d across all item s and both test structures. Based on the percentage of items with the means and standard deviations of differences close to 0, Link1 (d irect method) produced more accurate and stable linking results than Link4 (item characteristic function method), and Link4 yielde d more accurate and stable results than Link3 (test characte ristic function method). Link2 (e quated function method) did not provide accurate and stable result s for a high percentage of items. The performance of the four linking methods wa s also examined separately for different test structures. Figure 32 show s the distributions of means and standard deviations for 1a, 2a, and dfor APP and COM conditions. Comparing the hi stograms for the four linking methods on the left side of the figures, one can see that there was no apparent difference among the four linking methods for APP conditions. The histograms on the right side of the figures show that PAGE 62 62 there was obvious difference among the four li nking methods for COM conditions. Specifically, based on the accuracy and stability of linking func tion, (a) Link1 (direct method) worked well, (b) Link2 (equated function method) worked poor ly, and (c) the perfor mance of Link3 (test characteristic function method) and Link4 (item characteristic functi on method) was between that of Link1 and Link2, with Link4 being slightly better than Link3. In sum, Link1 (direct method) was consis tently the best method and Link2 (equated function method) the worst method under most COM conditions; the four linking methods worked equally and consistently well under most APP conditions. Performance of Linking Methods for Different Test Structures In this section, the perform a nce of the four linking methods is compared between APP and COM conditions. Figure 32 shows di fferent linking results for the two test structures. Based on the histograms for APP and COM c onditions in the figure, all the four linking methods produced more accurate and more stable results for APP te sts than for COM tests, but the difference in quality of linking varied acro ss the linking methods. Specifically Link1 (direct method) results were slightly better for APP tests than fo r COM tests, especially for parameters 1aand 2a; Link3 (test characteristic func tion method) and Link4 (item character istic function method) results were much better for APP tests than for COM tests; Link2 (e quated function method) yielded very poor results for COM tests, but good results for APP tests. However, for the large sample size (N = 2000), Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic f unction method) worked almost equally well for APP tests and COM tests; the linking performance diff erence between APP and COM conditions still remained for Link2 (equated function method) due to its poor function for COM conditions. Figure 33 shows the results of four linking methods for APP and COM tests PAGE 63 63 when the sample size is 2000. With smaller sample sizes (N = 500 and N = 1000), the linking performance difference between APP and COM conditions increased for Link3, Link4, and Link1 (see specific histograms in the appendix). Therefore, test structure had its smallest e ffects on Link1 (direct method), larger effects on Link3 (test characteristic function method) and Link4 (item characteristic function method), and the largest effect on Link2 (equate d function method). Link2 worked well for all APP tests, but poorly for all COM tests in this study. Due to the strong influence of test structure on the function of the four linking methods most of the results in the following sections are presented separately for the APP and COM conditions. Performance of Linking Methods for Different Test Lengths Given the different perform ance of linki ng methods for APP and COM conditions, the influence of test lengths on the linking function was explored sepa rately for APP and COM tests. The distributions of means and standa rd deviations of differences for 1a, 2a, and dfor short and long tests under the APP conditions are presented in Figure 34. Based on the histograms in the figure, one can see that for APP tests, a lthough the linking performance was not strongly influenced by test length, all four linking me thods produced slightly more accurate and stable results with long tests. Inspection of the results indi cated that under COM condition s, the performance of the linking methods depended on the sample size and test length. Therefore, th e influence of test lengths was next explored separa tely for different sample sizes for COM tests. Figure 35 shows the linking results for short and long tests with sample size of 500. Although none of the four linking methods worked well, the histograms still show that the results for Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) for PAGE 64 64 short tests were better than those for long tests and that Link1 to some extent performed similarly for different test lengths. Figure 36 illustrates the linki ng results for short and long te sts with sample size of 1000. From the figure, it was difficult to state at which test length linking performance was better. Subsequently reported results will show that th e performance depended to some extent on the ability distribution difference between the base and equated groups. When the linking results for ability condition 2 (G2: unequal mean vectors) were excluded, the linking function for long test was obviously better than that for short test except for Link2 (equated function method), as presented in Figure 37. Theref ore, in Figure 36, the performance under G2 masks the positive effect of test length on the linking accuracy and stability. Shown in Figure 38 are linking re sults for different test lengths with sample size of 2000. It is very obvious that the linking results for long tests were better than those for short tests except for Link2 (equated function method). In sum, the linking results for long tests were better than those for short tests except in some COM conditions when the sample size was small. Performance of Linking Methods for Different Sample Sizes Inspection of results indicated that sample size had stable and cons istent influence on the linking performance, but with different degrees of influence for different test structures. Therefore, the effect of sample size is firs t shown across all other testing conditions then presented separately for APP and COM conditions Figure 39 contains the linking results for different sample sizes. Compari ng horizontally the histograms for different sample sizes, one can find that both the linking accuracy and stabilit y increased for Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic f unction method) with the sample sizes changing from 500 and 1000 to 2000. Howe ver, the linking perfor mance increased at PAGE 65 65 different degrees for different test structures. Figure 310 shows the linking results for different sample sizes for APP tests. Figure 311shows th e results for different sample sizes for COM tests., From Figure 310, it can be found that the accuracy of the four linking methods was fairly good at all sample sizes and the stability of the four linking methods increased when the sample size became large for APP tests. Figure 311 sugg ests that although the acc uracy and stability of Link1, Link3, and Link4 increased when the samp le size became large for COM tests, the linking performance was poor for sample sizes of 500 and 1000 especially for Link3 and Link4. In addition, for COM tests, the accuracy and stab ility for Link2 (equated function method) were very poor for all sample sizes and relatively unaffected by sample size. Based on these findings, (a) Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) for APP tests were less affected by different sample sizes than we re COM tests; (b) Link1 (direct method) was less affected by sample sizes than were the othe r linking methods for COM tests. Performance of Linking Methods for Gro ups w ith Different Ability Distributions Inspection of the results suggested that the linking results for different ability distributions depended on other testing conditions. Therefore, the influence of ability distribution was first explored separately for APP and COM tests. Figure 312 shows the linking results for groups with different ability distributions under the APP conditions. Comp aring horizontally the histograms across G1 (equal mean vectors, eq ual covariance matrices), G2 (unequal mean vectors, equal covariance matrices), G3 (equal mean vectors, unequal covariance matrices), and G4 (unequal mean vectors, unequal covariance matrices) indicat es that: (a) for 1a and 2a, the linking results for G1 were slightly better th an those for other abil ity conditions; (b) for d, the results for G2 were somewhat worse than thos e for other ability conditions; (c) Link2 (equated PAGE 66 66 function method) was least affected by ability di stributions. The results imply that a difference between groups in the mean vectors was more in fluential than a difference between the groups in the covariance matrices. Inspection of results for COM tests indicated th at the influence of ab ility distribution was moderated by sample size; Therefore, the effect of ability distribution was explored separately for N=500, N=1000, and N=2000 for COM tests with the concentra tion on Link1 (direct method), Link3 (test characteristic function met hod), and Link4 (item characteristic function method). Figure 313 shows the linking results for groups with different abil ity distributions with sample size of 500. One can see from the fi gure that although none of the linking methods worked well for the small sample size, the linki ng results for G2 (unequal mean vectors, equal covariance matrices) and G4 (unequal mean vect ors, unequal covariance matrices) were worse than for G1 (equal mean vectors, equal covari ance matrices) and G3 (equal mean vectors, unequal covariance matrices). Even though Link1 (direct method) was relatively unaffected by betweengroup difference in ability distributi ons in groups, it still did not work well in linkingdfor G2, which indicates the strong influence of the mean difference between groups. The linking results for different groups with sa mple size of 1000 presented in Figure 314 shows that linking methods did not work well under G2, especially for d. However, there was some interaction between group differences and test le ngth. For long test (n = 40), the linking methods did not work well for G2 (see Figure 315); for short test (n = 20), the linking methods worked relatively well for G2 (see Figure 316). The linking results for different groups with sample size of 2000 shown in Figure 317 suggest that the linking methods worked approximately equally well for the groups with differe nt ability distributions. PAGE 67 67 In sum, the influence of ability distributi ons on linking results de pended on other testing conditions: (a) betweengroup differences in ability distributions did not ha ve a strong influence on the performance of the four linking methods for APP conditions or for COM conditions with a large sample size; (b) mean difference between groups had negative influence on the linking results especially for conditions with small sample size. Performance of Linking Methods for Test Items with Different Parameter Values Two types of scatterplots were used to exam ine the relationship between linking performance and item parameter values under each of the 48 testing conditions. The first type of scatterplot was used to evaluate the effect of item parameter va lues on the accuracy of different linking methods, with y axis as the mean of the differences and x axis as the true parameter values which were used to generate the item re sponse data. The second type of scatterplots was used to evaluate the effect of item parameter va lues on the stability of different linking methods, with y axis as the standard deviation of the diffe rences and x axis as the true parameter values. These two scatterplots were constructed for each of the three parameter estimates (e.g., 1a, 2a, and d) under each of the 48 testing conditions. However, the results of Link2 (equated function method) were not included in th e scatterplots un der the COM conditions due to its consistently poor performance. Given the limitation of space, the main outcomes are illustrated by some representative examples. The results suggest that: (a) Under most of the testing conditions, the linking results tended to be less accurate for 1a and 2awhen the two parameters had extreme values, and (b) under most of the testing conditions, the linking results became less stable for 1a and 2aas the parameters values increased. The results also in dicate that: (a) The accuracy of linking results for dwas not closely related to their true parameter values under most of the testing conditions, and PAGE 68 68 (b) the stability of linking results for dwas also not closely related to their true parameter values when the sample size was not large. The scatte rplots for one testing condition (COM, n = 20, N = 1000, G3) illustrate these relationships between linking performance and item parameter values (see Figure 318). However, for large sample size (N = 2000) the stability of linking results for d was closely related to their absolute true parameter values. Specifically when the absolute parameter values of d were closer to 0, the linki ng results were more stable; when the absolute parameter values of d were farther away from 0, the linking resu lts were less stable. The scatterplots for another testing condition (APP, n = 40, N = 2000, G4) show that: (a) the linking results tended to be less accurate and less stable for 1a and 2a when the two parameters had extreme values; (b) the linking results tended to be less stable for das the absolute parameters values increased (see Figure 319). PAGE 69 69 Figure 31. Accuracy and stabil ity for different linking methods PAGE 70 70 Figure 31. Continued PAGE 71 71 Figure 31. Continued PAGE 72 72 Figure 32. Accuracy and stability by linking method and test structure PAGE 73 73 Figure 32. Continued PAGE 74 74 Figure 32. Continued PAGE 75 75 Figure 33. Accuracy and stability by linking method and test structure: N = 2000 PAGE 76 76 Figure 33. Continued PAGE 77 77 Figure 33. Continued PAGE 78 78 Figure 34. Accuracy and stabilit y by linking method and test length for approximate simple structure tests PAGE 79 79 Figure 34. Continued PAGE 80 80 Figure 34. Continued PAGE 81 81 Figure 35. Accuracy and stabilit y by linking method and test lengt h for complex structure tests: N = 500 PAGE 82 82 Figure 35. Continued PAGE 83 83 Figure 35. Continued PAGE 84 84 Figure 36. Accuracy and stabilit y by linking method and test lengt h for complex structure tests: N = 1000 PAGE 85 85 Figure 36. Continued PAGE 86 86 Figure 36. Continued PAGE 87 87 Figure 37. Accuracy and stabilit y by linking method and test length for complex structure tests when G2 was excluded: N=1000 PAGE 88 88 Figure 37. Continued PAGE 89 89 Figure 37. Continued PAGE 90 90 Figure 38. Accuracy and stabilit y by linking method and test lengt h for complex structure tests: N = 2000 PAGE 91 91 Figure 38. Continued PAGE 92 92 Figure 38. Continued PAGE 93 93 Figure 39. Accuracy and stabilit y by linking method and sample size PAGE 94 94 Figure 39. Continued PAGE 95 95 Figure 39. Continued PAGE 96 96 Figure 310. Accuracy and stab ility by linking method and sample size for approximate simple structure tests PAGE 97 97 Figure 310. Continued PAGE 98 98 Figure 310. Continued PAGE 99 99 Figure 311. Accuracy and stab ility by linking method and sample size for complex structure tests PAGE 100 100 Figure 311. Continued PAGE 101 101 Figure 311. Continued PAGE 102 102 Figure 312. Accuracy and stability by linking method and gr oup for approximate simple structure tests PAGE 103 103 Figure 312. Continued PAGE 104 104 Figure 312. Continued PAGE 105 105 Figure 313. Accuracy and stabil ity by linking method and group for co mplex structure tests: N = 500 PAGE 106 106 Figure 313. Continued PAGE 107 107 Figure 313. Continued PAGE 108 108 Figure 314. Accuracy and stabil ity by linking method and group for co mplex structure tests: N = 1000 PAGE 109 109 Figure 314. Continued PAGE 110 110 Figure 314. Continued PAGE 111 111 Figure 315. Accuracy and stability for di fferent linking methods: COM, n=40, N=1000, G2 PAGE 112 112 Figure 316. Accuracy and stability for di fferent linking methods: COM, n=20, N=1000, G2 PAGE 113 113 Figure 317. Accuracy and stab ility by linking method and group for complex structure tests: N = 2000 PAGE 114 114 Figure 317. Continued PAGE 115 115 Figure 317. Continued PAGE 116 116 Figure 318. Linking accuracy and stability and item parameter values: COM, n=20, N=1000, G3 PAGE 117 117 Figure 318. Continued PAGE 118 118 Figure 318. Continued PAGE 119 119 Figure 319. Linking accuracy and stability and item parameter values: APP, n=40, N=2000, G4 PAGE 120 120 Figure 319. Continued PAGE 121 121 Figure 319. Continued PAGE 122 122 CHAPTER 4 DISCUSSION By using simulated data, the perform ance of the four multidimensional IRT scale linking methods was evaluated under differe nt testing conditions, which incl ude different test structures, test lengths, sample sizes, and ability distributi ons. The results illustrated in Chapter 3 suggest that test structure had a str ong influence on the performance of the four linking methods. For approximate simple test struct ure, each of the four linking methods worked approximately equally well under all testing cond itions. For complex test struct ure, the equated function method did not work well under any testing conditions; th e performance of other three linking methods depended on different testing conditions; the dir ect method was the best linking procedure for most testing conditions. In addition, the ite m parameter values influenced the linking performance. The results are discusse d in this chapter by seven sections. Results from Previous Studies Theoretically, there are at least two m ain co mponents in linking errors: error caused by parameter estimation and error produced by scale transformation (Li, 1997). A simulation study (Kaskowitz & Ayala, 2001) found that linking was more accurate when there was less error in the item parameter estimates. Therefore, it is important to review prev ious studies about IRT parameter estimation and linking accuracy under different testing conditions, although it is difficult, if not impossible, to decompose the parameter estimation error from the linking error in testing practice. Based on Lis review (1997), the following fact ors can cause error in parameter estimation in IRT: (a) Examinees ability di stribution. Item difficulty for easy and hard items will not be well estimated when the examinees are normally distributed around their mean; Examinees with ability levels above or below the item difficulty are more informative for estimating item PAGE 123 123 discrimination parameter; (b) Item parameter value. Item difficulty parameters that are small or large and discrimination parameters that are sma ll or large produce larger estimation error; (c) Sample size. Larger sample sizes reduce estima tion error. However, the standard error of parameter estimates depends on the combined eff ect of these factors (Thissen & Wainer, 1982). Using the bias and RMSE between the transformed linking parameter estimates and the true linking parameters across re plications as the criterion, Li (1997) found that the linking accuracy of his three methods improved as sample size or test length increases. In the second study, Li (1997) used the bias and RMSE between the transformed item parameter estimates and the true parameter values across replications as the criterion and found that one of his linking methods (e.g., the combination of procrustean ro tation approach for dimensional transformation, the ratio of trace procedure for dilation, and the least square procedure fo r translation) produced accurate linking of items. In addition, the positiv ely skewed distribution of the second dimension in equated group did not negatively influence the linking accuracy and stability. To evaluate the performance of the four multidimensional IRT scale linking methods, Oshima et al. (2000) used differe nt criteria, including mean and standard deviation of the linking parameter estimates over 20 replications, bias and mean square error (MSE) between the estimated and true linking parameters, correla tion and mean absolute difference of linking parameter estimates across different methods, and minimized function values by different methods. The results indicate that : (a) The direct method and e quated function method tended to yield similar linking results and the test characteristic function method and item characteristic function method tended to produce similar results, (b) the test and item characteristic function methods were more accurate and stable than the other two methods, and (c) the accuracy and stability decreased as ability differences between the groups increased. PAGE 124 124 Min (2003) used the bias and RMSE between transformed item parameter estimates and the initial item parameters acro ss the common items as the crit erion to compare Lis (1997) composite procedure, Oshima and colleagues test characteristic function method, and Mins extended composite procedure. Base d on the repeated measures analysis of variance for bias and log transformed RMSE, The author found that: (a) The ability distribution, test structure, and linking method accounted for large portion of the va riation in bias for discrimination parameter estimates but only linking method was an important factor for the va riation in bias of difficulty parameter estimates, (b) the sample size, ability distribution, and linki ng method were important for linking stability of discrimination paramete r estimates and sample size and linking method were critical for linking stability of difficulty pa rameter estimates, (c) as the sample size became larger and the two groups were more similar, the linking results became more accurate and stable, and (d) linking methods had significant intera ction effects with testing conditions. In sum, the linking methods and the three testing conditions, e.g., the ability distribution, test structure, and sample size, significantly affected the linking accuracy and stability. Effects of Different Test Structures In the present study, the perform ance of all four linking methods worked much better for APP than for COM tests. This is consistent w ith the fact that the test structure and item parameters are typically more easily and accurately estimated for APP test than for COM test. As Tate (2003) found and discussed in a study comparing different estimation methods, including NOHARM, for assessing the test st ructure of item responses, th e default rotation methods in exploratory analysis are usually developed to transform the initia l solution to simple structure, therefore the procedures may not always successfully describe nonsimple test structure. A study (Gosz & Walker, 2002) comparing the perf ormance of TESTFACT and NOHARM found that the item parameter estimation of NOHARM depends heavily on the number of bidimensional PAGE 125 125 items in the test with its better performa nce for fewer bidimensional items and worse performance for more bidimensional items. In addition, NOHARM is good at estimating items with very low values on one discriminati on parameter and high values on the other discrimination parameter. In this study, for APP tests, all items ha d higher values on one discrimination parameter and lower values on the other discrimination parameter; for COM tests, half of the items had approximately similar va lues on both discrimination parameter values. An investigation of item parameter estimation for the simulated data used in the present study indicated that the NOHARM program provided bette r item parameter estimation for APP tests than for COM tests. The superior estimation for the APP tests is likely the source of the superior linking results for the APP tests. However, in a study (Min & Kim, 2003) comparing Lis composite procedure and Oshima and colleagues test ch aracteristic function method under different testing conditions, no apparent linking difference was found between APP and COM tests (see Figure 27, Min & Kim, 2003). One possible reason is that the items loadings on the two dimensions in COM tests in this study were more similar than those in Mi n and Kims study. Specifically the heavily crossloaded items in their study had the direction of 50 65 and 25 40, and the direction of heavily crossloaded items in this study was sele cted from a normal distribution with mean of 45 and standard deviation of 10. According to the finding by Gosz and Walker (2002), the item parameter estimates for COM tests in this study we re less accurate, so that the linking results were more different between APP and COM test s. Another possible reas on is related to the different criteria used to desc ribe the linking performance. This study used the percentage of items with different means and standard deviations between the item parameter estimates for the base group and the transformed item paramete r estimates for the equated group over 500 PAGE 126 126 replications to evaluate linki ng results. Min and Kims study (2003) used the bias and RMSE between true parameter values and transforme d parameter estimates over both 20 items and 50 replications, which may have difficulty in identif ying the differential in fluence of APP and COM tests on the linking results. As shown in the Results chapter, the linki ng results (except for e quated function method) were very similar for APP and COM tests when the sample size became large (N = 2000). This may be related to the possible improved item parameter estimation for larger sample size for both APP and COM tests. However, the attribu tion of different linking performance for the two types of tests to estimation error needs to be investigated by more controlled st udies in the future. Effects of Different Test Lengths It was illustrated in the last chapter that the linking results for long tests were typically better than those for short tests, which is cons istent with Lis finding (1997) that the linking accuracy of his three m ethods improved as test length increases. This result was not unexpected since more items can provide more information to set up the linkage betw een the scales for the base and equated groups. The positive effect of large number of items on linking and equating performance has already been found in various unidimensional equating conditions (Budescu, 1985; Fitzpatrick & Yen, 2001; Kaskowitz & Ayal a, 2001; Kim & Cohen, 2002; Peterson, Cook, & Stocking, 1983; Swaminathan & Gifford, 1983; Wingersky, Cook, & Eignor, 1987). Therefore, this effect of the number of items can be extended from unidimensional to multidimensional linking and equating situations. However, there was an exception that the linki ng results for short CO M tests were better than those for long COM tests when the sample size was small (N = 500). Li could not find this exceptional result because he used sample sizes of 1000, 2000, and 4000 in his study. One possible reason for this exceptional result is th at small sample size was not large enough to PAGE 127 127 produce accurate item parameter estimates for long test because more item parameters needed to be estimated, which accordingly affected the linki ng performance for long COM tests. Therefore, the strength of large number of items in scale linking and equating depends on the quality of the item parameter estimation, which in turn require s enough sample size. Lord (1980) stated that it is test length in combination with sample size that affects the quality of parameter estimates. Compared with unidimensional IRT models, a la rger number of examinees are required for MIRT calibration because more parameters need to be estimated. In addition, this study found th at the effects of test leng th on scale linking performance also depended on the ability dist ributions for the two groups. As de scribed in Results chapter, the long test (n = 40) did not improve linking perfor mance when the means of ability distributions were different for base and equated groups for COM test when the sample size was 1000. This phenomenon confounded the general positive effect of large number of items on linking results. Klein and Kolens study (1985) suggests that test length has little effect on the equating quality when groups are similar in ability, but becomes very important when two groups differ in ability level. They found that a larger number of co mmon items did improve equating when groups were dissimilar. However, the exceptional result from this study mentioned above did not confirm their finding. Further studies are n eeded to examine the conflicting findings by controlling more conditions. Effects of Different Sample Sizes Based on the results from this study, the e ffects of sa mple size were very obvious and straightforward. Generally speaki ng, the linking accuracy and stab ility improved with the sample size increasing. This is consistent with the fact that large sample size can improve item parameter estimates. The same pattern was also found in the other two multidimensional scale linking studies (Li, 1997; Min & Kim, 200 3). In addition, the positive effect of large sample size has PAGE 128 128 also been found in unidimensional linking and equating studies (Fitzpatrick & Yen, 2001; Hanson & Beguin, 2002; Kim & Cohen, 2002; Pete rson, Kolen, & Hoover, 1989; Ree & Jensen, 1983). However, the linking performance improved at different degrees for APP tests and COM tests. The performance of direct method, test characteristic function method, and item characteristic function method in creased much more rapidly for COM tests than for APP tests when the sample sizes became larger. In fact, th e linking results for APP tests were consistently good for different sample sizes. However, the linki ng results for COM tests were very different for different sample sizes, although the linking function improved with the sample sizes increasing. This result was not found in Min and Kims study (2003). They showed similar effect of sample size on linking accuracy and stability for APP and COM tests (see Figure 27, Min & Kim, 2003). As we discussed for the effect of test structures on linking performance, this may be related to the different manipulatio ns of COM test items and differe nt evaluation criteria used in these two studies. Effects of Different Ability Distributions Based on this study, for all APP conditions a nd the COM conditions w ith a large sample size, betweengroup differences in ability distributions did not have a large influence on the performance of the four linking methods. For CO M conditions with small and medium sample size (N=500, N=1000) betweengroup differences in mean ability had a negative influence on the linking results. It seems that mean difference was more important than variance difference. These results were consistent with what Oshima et al. found in their study using very similar ability conditions (see Table 5 and Figure 1, Os hima et al., 2000), although they did not divide tests into APP and COM tests. However, we n eed to be very cauti ous about the possible differential effect of mean and variance differences on scale linking in both studies because they PAGE 129 129 were controlled at different degrees, with mean difference at 0.5 and the variance difference at 0.2. Based on the study by Min and Kim (see Figure 27, Min & Kim, 2003), it seems that the influence of ability distributi ons on scale linking by the test ch aracteristic function method was approximately similar for APP and COM conditi ons (see the above explanation for possible reasons for this conflicting findings between thei r study and this study). However, they did find that the influence of betweengroup differences in ability distribution on scale linking depended on sample size, with less influence for large sa mple size (N = 2000) and more influence for small sample size (N = 500). This is consiste nt with the results from this study. Li (1997) used a different manipulation of the betweengroup difference in ability distribution than was used in the present study: for the base group both ability distributions were normal; for the equated group one ability distribution was normal and the other was positively skewed. No negative effect was found on the linking performance by us ing his three methods. The reason may be that although th e second ability had positively sk ewed distribution, the mean and standard deviation were stil l controlled at 0 and 1, which were the same as for the based group for the second dimension. It seems that mean and standard deviation were more important than the normality of the distribution. However, this conclusion needs to be confirmed for the MIRT linking methods. Based on the research on unidimensional scal e linking and test equa ting (see the review by Kolen and Brennan, 2004), the similarity betw een two groups of examinees affects linking and equating performance: the more similar the groups are, the more adequate the linking and equating will be; large differences between gr oups may produce signifi cant problems. Based on results from multidimensional scale linking, this conclusion can be extended to the PAGE 130 130 multidimensional cases, but with cautious consid eration of the interaction between ability distribution, test structure, and sample size. Effects of Different Item Parameter Values As m entioned in the first section, estimation of the item difficulty parameter is less accurate when the parameters are small or large, estimation of discrimination parameter is less accurate when discrimination parameters are sm all or large, and error in item parameter estimation affects scale linking perf ormance. Therefore, linking quality is likely to be influenced by the item parameter values, especially by th e extreme parameter values. This conceptual inference and conclusion were confirmed in this study: under most of th e testing conditions, the linking results tended to be less accurate when th e absolute item parameters had extreme values and less stable when the absolute item parameter values became large. This pattern of results was more apparent when (a) the test had approximate simple structure, (b) the sample size was larger, and (c) the linking performance for discrimination was evaluated. The only other multidimensional scale linking study evaluating the effects of different item parameter values was conducted by Li (1997). Ba sed on that study (see Figure IV116, Li, 1997), the linking results for difficulty were more accurate and stable when the absolute item parameter values became larger; the linking results for discrimination did not change consistently with the item parameter values. Therefore, the effects of different item parameter values on scale linking were more apparent for discrimination in this study and more obvious for difficulty in Lis study. This is reasonable given Min and Kims (2003) conclusion that Lis method worked better than Oshima and colleagues test characteris tic function method (2000) for difficulty parameters and Oshima and colleagues method worked better for the two discrimination parameters. PAGE 131 131 Performance of Differe nt Linking Methods The effects of test structure, tes t length sample size, ability distribution, and item parameter values on scale linking performance were separately discussed above. However, these factors interacted with each ot her and had both main and combined effect on the performance of the four linking methods. As summarized at the begi nning of this chapter, genera lly speaking, all four linking methods worked approximately equally well under all testing conditions for approximate simple tests. For complex tests, the direct method was the best linking procedure; the item characteristic function method and the test char acteristic function method we re the second and third; the equated function method did not work well for co mplex tests. These results were based on the differences between the item parameter estimat es for base group and the transformed item parameter estimates for equate d group for the common items. It is not entirely su rprising that the direct method, which minimizes the sum of squared differences between the two sets of item parame ter estimates over items, was the best one across different testing conditions because the evaluatio n criterion was consistent with the method. However, the equated function method estimates the linking parameters by minimizing the sum of squared difference between the means of the two sets of selected item parameter estimates in the test. It uses the accumulative information of some items. Therefore, it is possible that even though the mean parameter estimates were simi lar for the two groups, individual parameter estimates were not. In the same way, item characteristic function method uses the combined information of discrimination, difficulty, and ability item by item. The test characteristic function method uses the accumulative information of discri mination, difficulty, and ability over all items in the test. Therefore, item characteristic func tion method was better than test characteristic function method using the criterion based on differe nce between item parameter estimates. PAGE 132 132 Why do the four linking methods worked equa lly well for approximate simple tests but differentially poor for complex test? One of possible reason is that there is complicated interaction between item parameter estimation er ror and the characterist ics of the four linking methods. More simulation studies need to be conduc ted to differentiate the two types of effect on the performance of scale linking. PAGE 133 133 CHAPTER 5 CONCLUSIONS The purpose of this study was to use sim ulate d data to examine the performance of four multidimensional linking methods under different testing conditions. There were one hundred and ninetytwo experimental c onditions in this study: four linking methods (direct method, equated function method, test ch aracteristic function method, a nd item characteristic function method) by two test structures (approximate simple test structure and complex test structure) by two test lengths (20 items and 40 items) by three sample sizes (500, 1000, and 2000), and by four different ability distributions between two gr oups (no difference, only mean difference, only variance difference, and both mean and variance difference). Five hundred replications were conducted for each of the experimental conditions. The linking performance evaluation was based on the differences between the item pa rameter estimates for base group and the transformed item parameter estimates for equa ted group for the common items. The mean and standard deviation of the differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Conclusions Conclusion 1: The performance of the four linki ng methods. Generally speaking, the direct method was the best linking method; the item characteristic function method and test characteristic function method were the second and th ird best method; the equated function method was the last method. Howeve r, their linking performance depended on the following testing conditions. Conclusion 2: The effects of test structure. For approxi mate simple test structure, each of the four linking methods worked approximately equally well for all testing conditions; For complex test structure, the equated f unction method worked poorly under all testing conditions; the performance of the other thr ee linking methods depended on other testing conditions; the direction method was the be st method for most testing conditions. Conclusion 3: The effects of test length. The linking performance for long tests was typically better than that for short tests except for complex tests when the sample size was small. PAGE 134 134 Conclusion 4: The effects of sample size. The lin king performance improved when the sample size became larger, especially for complex tests. Conclusion 5: The effects of ability distribution. Quality of linking performance declined when there was difference in ability distri bution between the two groups, especially for complex tests; however, it seems that a betw eengroup difference in the means was more important than a difference in the variance. Conclusion 6: The effects of item parameter va lues. Under most of the testing conditions, the linking results for the discrimi nation parameter tended to be less accurate and less stable when the item parameter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depende d on the item parameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values. Future Research In this study, there are a num b er of limitations, which should be considered for making the conclusions described above. For example: (a) Although the item parameters for short and long approximate simple tests and complex tests were randomly created in the same way and from the same distributions, they did not have the same exact values. This should be considered when comparing the linking results for the four type s of tests; (b) The test structure was not constructed by randomly arranging th e items in the test. For the approximate simple test, the first half of the items had higher discrimination values for the first ability and lower values for the second ability, and the second half of the items had lower discrimination values for the first ability and higher values for th e second ability. The equated function method in this study used the means of the first half of items (all with lower or higher values), second half of items (all with lower or higher values) as the function to estimate the linking parameters. This may affect the linking performance of the equated function method; (c) This study used the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group as the criterion, which is consistent with th e minimized function of PAGE 135 135 the direct method and accordingly may favor this method. The linking performance should be evaluated using other criteria which are consistent with the other methods to examine the possible dependence of the results on the criteri a used. These criteria include the differences between the means of the selected item parame ter estimates obtained from the two groups, the differences between the test characteristic f unctions for a given range of ability, and the differences between item characteristic functions for a given range of ability. As mentioned in the first ch apter, the development of mu ltidimensional linking methods is just at the infancy stage and more research is needed to obtain definitive results. Therefore, a substantial research needs to be conducted to explore and evaluate di fferent procedures for multidimensional scale linking. Here are some fu ture research topics on multidimensional IRT scale linking. First of all, different specific procedures within each of the four linking methods need to be explored, compared, and evaluated so that the best method can be chosen for some specific purpose. For example: (a) For the test characte ristic function and item characteristic function methods, how should the theta region or levels be chosen? Should we use the equally spaced grid theta method or empirical theta method? If we choose empirical theta method, which examinee group, base group, equated group, or combined group, should be used? Which method is better? Should we give different weights to different theta regions and how to choose different weights? (b) For equated function method, which item pa rameter estimates shou ld be used? What characteristics should be considered to choose the appropriate sets of items? What function should be used to produce good linking performance? Second, what kind of criteria should be used to evaluate the performance of different linking methods? Within the multidimensional IRT linking and equating studies, different PAGE 136 136 criteria have been used. Even within one study, different criteria have been used. For example, Li (1997) used bias and RMSE between the tran sformed linking parameter estimates and the true linking parameters across replications in his fi rst study and then used bias and RMSE between true item parameter values and the transformed ite m parameter estimates and ability recovery in his second study. Oshima et al. (2000) used mean and standard deviation of the linking parameter estimates over 20 replications, bias and MSE between the estimated and true linking parameters, correlation and mean absolute diffe rence of linking parameter estimates across different methods, and minimized function values by different methods. Min (2003) used bias and RMSE between transformed item parameter estimates and the initial item parameters across common items for the simulated data, and used the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group across the common items for the real data. Given these criteria, which one should we use for which purpose for scale linking? This is a critical issue in evalua ting different methods. Third, as we discussed in last chapter, there are at least two main components in linking errors: error caused by parameter estimation a nd error produced by scale transformation. The problem is how to differentiate the estimation e rror from the linking error when scale linking is conducted? To answer this question, many st udies need to be conducted to evaluate the performance of different estimation programs for multidimensional IRT. In addition, some methods need to be developed to differenti ate the estimation error from linking error and evaluate the effects of estimation erro r on the performance of scale linking. Finally, the two approaches, multidimensional IRT approach and factor analysis approach, have different strengths and weaknesses in linking different scales. As Min and Kim (2003) found in their study that Lis method worked bett er for difficulty parameters and Oshima and PAGE 137 137 colleagues method (e.g., test characteristic function method) worked better for the two discrimination parameters. Therefore, how to use the strengths of the two approaches to develop a combined method for multidimensional scale li nking is an important topic in the future research. PAGE 138 138 APPENDIX ACCURACY AND STABILITY FOR DIFFERENT LINKING METHODS Figure A1. Accuracy and stab ility for different linking m ethods (APP, n=20, N=500, G1) PAGE 139 139 Figure A2. Accuracy and stab ility for different linking methods (APP, n=20, N=500, G2) PAGE 140 140 Figure A3. Accuracy and stab ility for different linking methods (APP, n=20, N=500, G3) PAGE 141 141 Figure A4. Accuracy and stab ility for different linking methods (APP, n=20, N=500, G4) PAGE 142 142 Figure A5. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G1) PAGE 143 143 Figure A6. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G2) PAGE 144 144 Figure A7. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G3) PAGE 145 145 Figure A8. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G4) PAGE 146 146 Figure A9. Accuracy and stab ility for different linking methods (APP, n=20, N=2000, G1) PAGE 147 147 Figure A10. Accuracy and stability for diffe rent linking methods (APP, n=20, N=2000, G2) PAGE 148 148 Figure A11. Accuracy and stability for diffe rent linking methods (APP, n=20, N=2000, G3) PAGE 149 149 Figure A12. Accuracy and stability for diffe rent linking methods (APP, n=20, N=2000, G4) PAGE 150 150 Figure A13. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G1) PAGE 151 151 Figure A14. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G2) PAGE 152 152 Figure A15. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G3) PAGE 153 153 Figure A16. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G4) PAGE 154 154 Figure A17. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G1) PAGE 155 155 Figure A18. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G2) PAGE 156 156 Figure A19. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G3) PAGE 157 157 Figure A20. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G4) PAGE 158 158 Figure A21. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G1) PAGE 159 159 Figure A22. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G2) PAGE 160 160 Figure A23. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G3) PAGE 161 161 Figure A24. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G4) PAGE 162 162 Figure A25. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G1) PAGE 163 163 Figure A26. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G2) PAGE 164 164 Figure A27. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G3) PAGE 165 165 Figure A28. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G4) PAGE 166 166 Figure A29. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G1) PAGE 167 167 Figure A30. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G2) PAGE 168 168 Figure A31. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G3) PAGE 169 169 Figure A32. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G4) PAGE 170 170 Figure A33. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G1) PAGE 171 171 Figure A34. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G2) PAGE 172 172 Figure A35. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G3) PAGE 173 173 Figure A36. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G4) PAGE 174 174 Figure A37. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G1) PAGE 175 175 Figure A38. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G2) PAGE 176 176 Figure A39. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G3) PAGE 177 177 Figure A40. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G4) PAGE 178 178 Figure A41. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G1) PAGE 179 179 Figure A42. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G2) PAGE 180 180 Figure A43. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G3) PAGE 181 181 Figure A44. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G4) PAGE 182 182 Figure A45. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G1) PAGE 183 183 Figure A46. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G2) PAGE 184 184 Figure A47. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G3) PAGE 185 185 Figure A48. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G4) PAGE 186 186 LIST OF REFERENCES Ackerm an, T. A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255278. Ackerman, T. A. (1996). Graphic representati on of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311329. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561573. Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological Measurement, 16, 8796. Baker, F. B. (1993). Equating tests under the nominal response model. Applied Psychological Measurement, 17, 239251. Bateley, R. M., & Boss, M. W. (1993). Th e effects on parameter estimation of correlateddimensions and a distributionrestrict ed trait in a multidimensional item response model. Applied Psychological Measurement, 17, 131141. Bedescu, D. (1985). Efficiency of linear equating as a function of the length of the anchor test. Journal of Educational Measurement, 22, 1320. Bock, R. D. (1972). Estimating item parameters and latent ability when the responses are scored in two or more nominal categories. Psychometrika, 37, 2951. Bock, R. D., & Aitkin, M. (1981). Marginal maxi mum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443459. Bock, R. D., Gibbons, R., & Muraki, E. (1988) Fullinformation item factor analysis. Applied Psychological Measurement, 12, 261280. Bock, R. D., Gibbons, R., Schilling, S. G., Muraki, E., Wilson, D. T., & Wood, R. (1999). TESTFACT 3: Test scoring, items statistics, and fullinfo rmation item factor analysis. Chicago: Scientific Software International. Carlson, J. E. (1987). Multidimensional item response theory estimation: A computer program. Unpublished manuscript. Cattell, R. B. (1978). The scientific use of factor analysis. New York: Plenum. Cohen, A. S., & Kim, S. H. (1998). An investigation of linking methods under the graded response model. Applied Psychological Measurement, 22, 116130. Comery, A. L., & Lee, H. B. (1992). A First course in factor analysis. Hillsdale, NJ: Erlbaum.Cook, L. L., Eignor, D. R. (1991). IRT equating methods. Educational Measurement: Issues and Practice, 10, 3745. PAGE 187 187 Cook, L. L., Eignor, D. R., & Taft, H. (1981, April). A comparative study of curriculum effects on the stability of IRT and conventional item parameter estimates (RR8538). Princeton, NJ: Educational Testing Service. Cook, L. L., & Petersen, N. S. (1987). Problem s related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225244. Cureton, E. E., & DAgostino, R. B. (1983). Factor analysis: An applied approach. Hillsdale, NJ: Erlbaum. Davey, T. C., Oshima, T. C., & Lee, K. (1996). Linking multidimensional item calibrations. Applied Psychological Measurement, 20, 405416. Divgi, D. R. (1985). A minimum chisquare met hods for developing a common metric in item response theory. Applied Psychological Measurement, 9, 413415. Fitzpatrick, A. R., & Yen, W. M. (2001). The effects of test length and sample size on the reliability and equating of tests composed of constructedresponse items. Applied Measurement in Education, 14, 3157. Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor analysis.Multivariate Behavioral Research, 23, 267269. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum. Gosz, J. K. & Walker, C. M. (2002, April). An empirical comparison of simple versus complex multidimensional item response data. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Gosz, J. K., Walker, C. M. (2002, April). An empirical comparison of multidimensional item response data using TESTFACT and NOHARM. Paper presented at the Annual Meeting of the National Council for Measurement in Educ ation (NCME), New Orleans, Louisiana. Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw Hill. Haebara, T. (1980). Equating l ogistic ability scales by a we ighted least squares method. Japanese Psychological Research, 22, 144149. Hason, B. A., & Beguin, A. A. (2002). Obtaini ng a common scale for item response theory item parameters using separate versus concurre nt estimation in the commonitem equating design. Applied Psychological Measurement, 26, 324. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff Publishing. Harwell, M., Stone, C. A., Hsu, T. C., & Kirisc i, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20, 101125. PAGE 188 188 Hirsch, T. M. (1989). Multidimensional equating. Journal of Educational Measurement, 26, 337349. Hirsch, T. M. (1988). Multidimensional equating. Unpublished doctoral di ssertation, Florida State University, Tallahassee, FL. Kaskowitz, G. S., & De Ayala, R. J. (2001). The effect of error in item parameter estimates on the test response func tion method of linking. Applied Psychological Measurement, 25, 3952. Kelderman, H. (1997). Loglinear multidimensional item response models for polytomously scored items. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. Kim, H. (1994). New techniques for the dimensionality a ssessment of standardized test data. Unpublished doctoral dissertation. University of Illinois at Urbana Champaign. UrbanaChampaign, IL. Kim, S. H., & Cohen, A. S. (1995). A mini mum chisquare method for equating tests under the graded response model. Applied Psychological Measurement, 19, 167176. Kim, S. H., & Cohen, A. S. (2002). A comparis on of linking and concu rrent calibra tion under the graded response model. Applied Psychological Measurement, 26, 2541. Klein, L. W., & Kolen, M. J. (1985, April). Effect of number of common items in commonitem equating with nonrandom groups. Paper presented at the annual meeting of American Educational Research Association, Chicago. Knol, D. L., & Berger, M. P. F. (1991). Empiri cal comparison between factor analysis and multidimensional item response models. Multivariate Behavioral Research, 26, 457477. Kolen, M. J. (2004a). Population invariance in equating and linking: Concept and history. Journal of Educational Measurement, 41, 314. Kolen, M. J. (2004b). Linking assessment: Concept and history. Applied Psychological Measurement, 28, 219226. Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and Practices (2nd ed.). New York, NY: Springer. Lautenschlager, G. J., Flaherty, V. L., & Park D. G. (1994). IRT differential item functioning: An examination of ability scale purifications. Educational and Psychological Measurement, 54, 2131. Lee, K., & Oshima, T. C. (1996). IPLINK: Multid imensional and unidimensional item parameter linking in item response theory. Applied Psychological Measurement, 20, 230. PAGE 189 189 Li, Y. H. (1997). An evaluation of multidimen sional IRT equating methods by assessing the accuracy of transforming parameters onto a ta rget test metric (Doctoral dissertation, University of Marryland, 1997). Dissertation Abstract International, UMI Number 9816494. Li, Y. H., & Lissitz, R. W. (2000). An evalua tion of the accuracy of multidimensional IRT linking. Applied Psychological Measurement, 24, 115138. Lord, F. M. (1980). Applications of item response theo ry to practical testing problems. Hillsdale, NJ: Erlbaum. Loyd, B. H., & Hoover, H. D. (1980). Ve rtical equating usin g the Rasch model. Journal of Educational Measurement, 17, 179193. Marco, G. L. (1977). Item characteristic curve so lutions to three intrac table testing problems. Journal of Educational Measurement, 14, 139160. Masters, G. N. (1982). A Rasch mo del for partial credit scoring. Psychometrika, 47, 149174. McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100117. McDonald, R. P. (1997). Normalogive multidimensional model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah, NJ: Erlbaum. McDonald, R. P. (2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99114. McKinley, R. L., & Reckase, M. D. (1983). An extension of the twoparameter logistic model to the multidimensional latent space (Research Report, ONR 832). Iowa City, IA: American College Testing Program. Millsap, R. E. (2005). Four unr esolved problems in studies of factorial invariance. In A. MaydeuOlivares & J. J. McArdle (Eds.), Contemporary Psychometrics. Mahwah, NJ: Lawrence Erlbaum Associates. Min, K. S. (2003). The impact of scale dilation on the quality of the linking of multidimensional item response theory calibrations. Unpublished Dissertation, Michigan State University, East Lansing, MI. Mroch, A. A., & Bolt, D. M. (2006). A simulati on comparison of parametric and nonparametric dimensionality detection procedures. Applied Measurement in Education, 19, 6791. Muraki, E. (1992). A generalized partial cred it model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159176. PAGE 190 190 Muraki, E. (1999). POLYFACT version 2 [Computer program]. Princeton, NJ: Educational Testing Service. Muraki, E, & Carlson, J. E. (1995). Fullinfo rmation factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 7390. Muraki E., & Engelhard, G. (1985). Fullinformation item factor analysis : Applications of EAP scores. Applied Psychological Measurement, 9, 417430. Muthen, L. K., & Muthen, B. (1998). MPLUS: The comprehensive modeling program for applied researcher: Users guide. Los Angeles: Muthen & Muthen. Oshima, T. C., Miller, M. D. (1992). Multidimensi onality and item bias in item response theory. Applied Psychological Measurement, 16, 237248. Oshima, T. C., Davey, T. C., & Lee, K. (2000). Multidimensional linking: Four practical approaches. Journal of Educational Measurement, 37, 357373. Park, D. G., & Lautenschlager, G. J. (1990). Im proving IRT item bias detection with iterative linking and ability sc ale purification. Applied Psychological Measurement, 14, 163173. Peterson, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137156. Peterson, N. S., Kolen, M. J. & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L. Linn (Ed.), Educational measurement (pp. 221262). New York: American Council on Education and Macmillan. Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401412. Reckase, M. D. (1991). The discriminating power of items that measure more than onedimension. Applied Psychological Measurement, 15, 361373. Reckase, M. D. (1997a). A linear logistic multidimensional model for dichotomous item response data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. Reckase, M. D. (1997b). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 2536. Rechase, M. D., & Martineau, J. (2004, October). The vertical scaling of Science Achievement Tests. Paper commissioned by the Committ ee on Test Design for K12 Science Achievement, Center for Educati on, National Research Council. PAGE 191 191 Ree, M. J., & Jensen, H. E. (1983). Effects of samp le size on liner equating of item characteristic curve parameters. In D. J. Weiss (Ed.), New horizons in testing (pp. 135146). New York: Academic. Roussos, L. A., Stout, W. F., & Marden, J. I. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 130. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17. Schonemann, P. H. (1966). A generalized soluti on of the orthogonal procrustes problem. Psychometrika, 31, 110. Schonemann, P. H., & Carroll, R. M. (1970). Fitt ing one matrix to another under choice of a central dilation and a rigid motion. Psychometrika, 35, 245255. Simpson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computeriz ed Adaptive Testing Conference (pp. 8298). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program. Stocking, M. L., & Lord, F. M. (1983). Developi ng a common metric in item response theory. Applied Psychological Measurement, 7, 201210. Stone, C. A., & Yeh, C. C. (2006). Assessing the dimensionality and factor structure of multiplechoice exams. Educational and Psychological Measurement, 66, 193214. Swaminathan, J., & Gifford, J. A. (1983). Estimati on of parameters in the threeparameter latent trait model. In D. J. Weiss (Ed.), New horizon in testing (pp. 1330). New York: Academic. Tate, R. (2003). A comparison of selected empi rical methods for assessing the structure of responses to test items. Applied Psychological Measurement, 27, 159203. Thissen, D. (1991). MULTILOG users guide: multiple, categorical item analysis and test scoring using item response theory [Computer program]. Chicago, IL: Scientific Software International. Thissen, D., & Wainer, H. (1982). Some standard errors in item response theory. Psychometrika, 47, 392412. Thompson, T. D., Nering, M., & Davey, T. (1997, June). Multidimensional IRT scale linking without common items or common examinees. Paper presented at the annual meeting of the Psychometric Society, Gatlinburg, TN. Thurstone, L. L. (1947). Multiple factor analysis. Chicago: University of Chicago Press. PAGE 192 192 Vandenberg, R. J.,& Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, pract ice, and recommendations for organizational research. Organizational Research Methods, 3, 470. Wang, M. (1985). Fitting a unidimensional model to mu ltidimensional item response data: The effects of latent space misspecification on the application of IRT. Unpublished manuscript. Wilson, D., Wood, R., & Gibbons, R. D. (1987). TESTFACT: Test scoring, item statistics, and item factor analysis. Mooresville, IN: Scientific Software. Wingersky, M. S., Cook, L. L., & Eignor, D. R. (1987). Specifying the characteristics of linking items used for item response theory item calibration (Research Report 8724). Princeton, NJ: Educational Testing Service. Wingersky, M. S., & Lord, F. M. (1984). An i nvestigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347364. Yen, W. M., & Fitzpatrick, A. R. (2006). It em response theory. In R. L. Brennan (Eds.), Educational Measurement (4th ed.). West Port, CT: Praeger. Zimowski, M. F., Muraki, E., Misle vy, R. J., & Bock, R. D. (1996). BILOGMG: Multiplegroup IRT analysis and test main tenance for binary items [Computer program]. Chicago, IL: Scientific Software International. PAGE 193 193 BIOGRAPHICAL SKETCH Youhua W ei was born in China. He received his B.Ed. in school education from Nanjing Normal University in 1992 and his M.Ed. in ps ychology from East China Normal University in 1995. From 1995 to 1997, he worked as a psychol ogical counselor at Sout heast University in Nanjing. From 1997 to 2001, he worked as a re search associate at Shanghai Academy of Educational Sciences. In 2004, he earned his M.S. in research, meas urement, and statistics from Texas A&M University in College Station. He began his doctoral study in research and evaluation methodology in the Department of E ducational Psychology at the University of Florida in fall 2004. He was awarde d the Ph.D. degree in August 2008. 