<%BANNER%>

A Simulation Study on the Performance of Four Multidimensional IRT Scale Linking Methods

Permanent Link: http://ufdc.ufl.edu/UFE0022538/00001

Material Information

Title: A Simulation Study on the Performance of Four Multidimensional IRT Scale Linking Methods
Physical Description: 1 online resource (193 p.)
Language: english
Creator: Wei, Youhua
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: equating, linking, multidimensional, scale
Educational Psychology -- Dissertations, Academic -- UF
Genre: Research and Evaluation Methodology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Scale linking is the process of developing the connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. It is the prerequisite for many applications of IRT, such as test equating and differential item functioning analysis. Unidimensional scale linking methods have been studied and applied frequently over the past two decades. The development of multidimensional linking methods is at the infancy stage and more research is needed to obtain definitive results. As an extension of previous research, the purpose of this study was to use simulated data to evaluate the performance of four multidimensional IRT scale linking methods, the direct method, equated function method, test characteristic function method, and item characteristic function method, under various testing conditions, which include different test structures, test lengths, sample sizes, and ability distributions. There were one hundred and ninety-two experimental conditions in this study and five hundred replications were conducted for each of the conditions. The linking performance evaluation was based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for the equated group across the test items. The mean and standard deviation of the differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Our results indicate that for approximate simple test structure, each of the four linking methods worked approximately equally well under all testing conditions. The results also suggest that for complex test structure: (a) The equated function method did not work well under any testing conditions, (b) the performance of other three linking methods depended on other testing conditions including sample size, test length, and ability distribution difference between groups, and (c) the direct method was the best linking procedure for most testing conditions. In addition, the study shows that the item parameter values influenced the linking performance. Under most of the testing conditions, the linking results for the discrimination parameter tended to be less accurate and less stable when the item parameter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depended on the item parameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Youhua Wei.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Algina, James J.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2010-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0022538:00001

Permanent Link: http://ufdc.ufl.edu/UFE0022538/00001

Material Information

Title: A Simulation Study on the Performance of Four Multidimensional IRT Scale Linking Methods
Physical Description: 1 online resource (193 p.)
Language: english
Creator: Wei, Youhua
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2008

Subjects

Subjects / Keywords: equating, linking, multidimensional, scale
Educational Psychology -- Dissertations, Academic -- UF
Genre: Research and Evaluation Methodology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Scale linking is the process of developing the connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. It is the prerequisite for many applications of IRT, such as test equating and differential item functioning analysis. Unidimensional scale linking methods have been studied and applied frequently over the past two decades. The development of multidimensional linking methods is at the infancy stage and more research is needed to obtain definitive results. As an extension of previous research, the purpose of this study was to use simulated data to evaluate the performance of four multidimensional IRT scale linking methods, the direct method, equated function method, test characteristic function method, and item characteristic function method, under various testing conditions, which include different test structures, test lengths, sample sizes, and ability distributions. There were one hundred and ninety-two experimental conditions in this study and five hundred replications were conducted for each of the conditions. The linking performance evaluation was based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for the equated group across the test items. The mean and standard deviation of the differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Our results indicate that for approximate simple test structure, each of the four linking methods worked approximately equally well under all testing conditions. The results also suggest that for complex test structure: (a) The equated function method did not work well under any testing conditions, (b) the performance of other three linking methods depended on other testing conditions including sample size, test length, and ability distribution difference between groups, and (c) the direct method was the best linking procedure for most testing conditions. In addition, the study shows that the item parameter values influenced the linking performance. Under most of the testing conditions, the linking results for the discrimination parameter tended to be less accurate and less stable when the item parameter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depended on the item parameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Youhua Wei.
Thesis: Thesis (Ph.D.)--University of Florida, 2008.
Local: Adviser: Algina, James J.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2010-08-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2008
System ID: UFE0022538:00001


This item has the following downloads:


Full Text
xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101107_AAAABY INGEST_TIME 2010-11-07T18:45:03Z PACKAGE UFE0022538_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 41241 DFID F20101107_AABIHD ORIGIN DEPOSITOR PATH wei_y_Page_037.pro GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
805d3241b236e97a3c920b306331b612
SHA-1
84a4d7e67fcce1a991e9f2d4d44622008c790650
39273 F20101107_AABIGO wei_y_Page_022.pro
9bab7c8ff0bb183be652af6eed1f47ae
cf697ad09d881b382e3daa9192bcb1fef4cfbfdf
35352 F20101107_AABIHE wei_y_Page_038.pro
a1b41b44acdc29cc79bb6a092984e8c5
b3319c6e3922f51217b5a8c52be2cc0472d8b1ad
31528 F20101107_AABIGP wei_y_Page_023.pro
0474e61d2ae45aa9df12655a02a7cb78
5c6d9d8e5e2cf8b086b42ba94d946b3e2f68a765
49365 F20101107_AABIHF wei_y_Page_039.pro
d05e2d89f239e69a0ba93d4104aa195e
4105b8d20478832f499b2d7177b0df38b29f43cc
13298 F20101107_AABIGQ wei_y_Page_024.pro
718d1a9aeeaaa3f492fc10e857e5f751
9b964e02ff2369ab11e0670bd8df7566bf1f3de1
56870 F20101107_AABIHG wei_y_Page_040.pro
479d2aea6f79770c9340664c398a887b
04c41a69020374c80872807e048a0b74df72caf2
43331 F20101107_AABIGR wei_y_Page_025.pro
41614239a4727d12ec3d9ebf97bf0dec
f261287784a7312600940e4eff556ba7cd4ee866
13385 F20101107_AABIHH wei_y_Page_041.pro
0c47e14f64a40a3ef1428022eaffc4d5
5715b481a25793ca457e4c35fefc5736ddea0548
36042 F20101107_AABIGS wei_y_Page_026.pro
f231de7967000d60117b4ea248f50eb8
a5227598f1656c7d73a6b725bc37d835add22ebd
49163 F20101107_AABIHI wei_y_Page_042.pro
d358d8055f647a68961b713d71b27fb2
c52fea2e434e7fa30fac1447193de458c741d845
49847 F20101107_AABIGT wei_y_Page_027.pro
be4e6001da4f7469c67e5307950ea415
dc635674fc1269189aa886ef3a93262433b29156
56218 F20101107_AABIHJ wei_y_Page_043.pro
1b2bb26ac92d2a11307c48a3909f85e7
8b531f43a532cb05c087b75bd995ffe984ff50a1
31773 F20101107_AABIGU wei_y_Page_028.pro
069dc3caf01d63f7980dc63fce754fdf
3ac32508e631dfde7570d299d46f9c2d0c19f2a1
55382 F20101107_AABIHK wei_y_Page_044.pro
6952f5b6cf38d03eab7922e64985e139
aa77ee9e52bcd5afc3079927de87b8e07b8de153
22414 F20101107_AABIGV wei_y_Page_029.pro
0764b3261745072ebe41e4dcdc8cb409
00fdf9e794cc9cf51e93982e75ace2a0f8a0d73c
53015 F20101107_AABIGW wei_y_Page_030.pro
cd5dc24767cc55a8ed46cf6a796b74e0
f5918075386e059597b47269272a5f8a440f50d1
55004 F20101107_AABIHL wei_y_Page_045.pro
cdeb1ae15ccdf1434b7feea8be9e1b00
451841b8d88f5426e6a1ba5e8ca9b38a042a2079
30243 F20101107_AABIGX wei_y_Page_031.pro
b2d437660ac05297851c34e4a47f04a8
f93273c8fbd8e1c04562a85ffc8f6aa97ef80cdf
55264 F20101107_AABIIA wei_y_Page_060.pro
6dfbd1849413f0b6a9407c6a57f3ebd7
799ce1a82f375baca2b306d55f65cb9d307b9c52
56731 F20101107_AABIHM wei_y_Page_046.pro
292d6d84dab801bc33f3f28e01e4e4b6
0137a127ff9f6b20b0d4bd68dd35ce40e65cb85e
42905 F20101107_AABIGY wei_y_Page_032.pro
fd8c6decb12d6e86ef46fd203aa201e8
b5d5748f85763aa21417700fb90832c48d58d94f
54332 F20101107_AABIIB wei_y_Page_061.pro
04d5e3f5f464e34b34b63d4a29dd0993
dd4ccab303b053191302cc0ff670969530c37021
53080 F20101107_AABIHN wei_y_Page_047.pro
a4f4a0809364d5a1d4024a2eda06eddb
4b2268b6737d83aadb36bb53333365e1bcd76a00
42050 F20101107_AABIGZ wei_y_Page_033.pro
2fe99b3641bfdb876e83c4dcd64983b7
3c799e0102d6c0bde246d7e98e80da53c72dc3c1
53650 F20101107_AABIIC wei_y_Page_062.pro
788993ec74a429e4a9404ff546f2f9f7
2ee970467eee042b4061bd9aa8add7df773e76fe
50091 F20101107_AABIHO wei_y_Page_048.pro
bc6c5986c2294899858a60694caaa8f7
b42d5a023b64be0b6e6b8c185d77a49cc58ddacd
52274 F20101107_AABIID wei_y_Page_063.pro
d641aa23bcee9d70a11f36c8b67fc155
d87907a6249874add229133eb7b51ad97bf8a29d
22211 F20101107_AABIHP wei_y_Page_049.pro
239f9eacc08a66bb49307f1c30b1dfd2
3869e92971c4ae98da7190910c22a3195877c7c4
54637 F20101107_AABIIE wei_y_Page_064.pro
d4e6e6ed0008ac0858f3cb8d41da4990
0010af81fa440ab5e0a8298e267d060fccbf6662
45733 F20101107_AABIHQ wei_y_Page_050.pro
271dad37a3e0a931d45de0f772d1ffa7
934f1468b29ecde29586ba0751247b53d6c7fd45
55927 F20101107_AABIIF wei_y_Page_065.pro
921fbe57a6d4d04b4e35d62458da8c3d
3ee33ef0ff22fddbc27f11592efd5306c5b6f3fa
53270 F20101107_AABIHR wei_y_Page_051.pro
6e2cb6fb6caa441edea27724f772070e
b9aad57f25eec13b2f3fb2efe4c5abfa5830c8a3
52007 F20101107_AABIIG wei_y_Page_066.pro
d513e8b1fa7d926201282768fe10652d
fda6a40511f6e2371255dac5b9e492798225dcce
50421 F20101107_AABIHS wei_y_Page_052.pro
c54133d58138c0d8afd0b9463e05bf99
3c62d9f37720c94e77196b7c57d907d369ab9446
55009 F20101107_AABIIH wei_y_Page_067.pro
e47c64a779036d3878c269ce34bdbe4a
e3a470011bd3b35f7962875ad7672729d3813849
4849 F20101107_AABIHT wei_y_Page_053.pro
02acc34c7a2701fb071d44988c77a614
b899ce15469fb493fc5d29474f7f5f1661c4da06
26397 F20101107_AABIII wei_y_Page_068.pro
336ce04773db950da5d3bd69ace33495
e733979e09bbd30f82ed6f12d882ac54f9758b4d
7350 F20101107_AABIHU wei_y_Page_054.pro
6ec7c171565590a73898ea744e2d8bf4
bfd00222903ab7024e2884e7d1fa7b91740a78f5
11267 F20101107_AABIIJ wei_y_Page_069.pro
f57c97b3fe9cc68c601490f99e370b3d
3951fa204a1b254da0de773c0ce168bbd0527037
14117 F20101107_AABIHV wei_y_Page_055.pro
ef4c2d2d2053d77a2a99871ace279384
f25be0287d410a3f64637ff47905fad54dff6b25
10929 F20101107_AABIIK wei_y_Page_070.pro
0715756efe315a99957df7a1156e5d68
5874c576756a5fb0db616d1f206d6643f1dda871
25440 F20101107_AABIHW wei_y_Page_056.pro
78345f823a7a120620e8ced64465dc7a
f5fafee45a198177045beec4de75ead8109c5616
9914 F20101107_AABIIL wei_y_Page_071.pro
8b69a784bff67992f21438cdb0e141bc
a349b7af3fdcfcd720b2c20ae7d8fb28a143f315
13935 F20101107_AABIHX wei_y_Page_057.pro
3d8149d5a80b95ef885b80032874731d
3e71f0cadeb07661d00aba867dfb9a24596951dd
13100 F20101107_AABIJA wei_y_Page_086.pro
9d3d7f31fabd75a7d25343dd3ab07ff1
1690fe534708019ac95cd0aab140c91935bf4c29
16951 F20101107_AABIJB wei_y_Page_087.pro
836c53967c778a21f25cf72c9692cd85
df5495bb136c928e4e65fe88ad62586fbda3f18e
12538 F20101107_AABIIM wei_y_Page_072.pro
7facf8e4a0cbd163ae3046d973b85e17
35c717adb61c481cd93ef95222133079608369f0
25418 F20101107_AABIHY wei_y_Page_058.pro
8486199adbee8379035718d437cc628c
b12c95ab55803cd3dc4d5399971ff9d43c91e683
11992 F20101107_AABIJC wei_y_Page_088.pro
3750229e31f38f88f8fb33cf2e3f303c
078bc9edf1e4b45bb1b00ab8711cb8f49c78e8c1
13782 F20101107_AABIIN wei_y_Page_073.pro
5c6754a03c6375aff2b0a7529e7ba236
4507cd7958a15ce79bcd47af6ac080071e391128
51589 F20101107_AABIHZ wei_y_Page_059.pro
e1a99105a1cb5cf15833699deb8c2b99
a4bc429f99d5307025b9608b430ff2acae58b07c
12588 F20101107_AABIJD wei_y_Page_089.pro
bfd7d4ffe8f362fc50ce7e1d72292d35
fbae68c6681be9487707d081cefe8bd2d79c60b9
12451 F20101107_AABIIO wei_y_Page_074.pro
cef8dddcc5c120228bb02bb4f18d67b1
bc9dc645c8c012092da8b6cd1d0fe4dbe744c84e
16682 F20101107_AABIJE wei_y_Page_090.pro
6608d8732c0890acad35fba2ffc719b4
a3951912dbc0a48c80e2e3b15035f36ffe918e4b
15448 F20101107_AABIIP wei_y_Page_075.pro
ee8691c9180a39bc571eafb82f3f06da
b049919037110fe98a89de8dbc4dc25a3b3a2a55
12970 F20101107_AABIJF wei_y_Page_091.pro
67416c017b0f7cf88f12dfb3802a9c20
0723d1219bc14af6d2e97a997d5a8dee0c3f7fac
14609 F20101107_AABIIQ wei_y_Page_076.pro
e9d61abcda1c558e06453841386a6b8b
47f9b36143a84b25118347d41dab9d58075c5156
13362 F20101107_AABIJG wei_y_Page_092.pro
11a731a851856c61392c82c9645fba82
ff8271783307ec97218483937156951f8629a7de
13873 F20101107_AABIIR wei_y_Page_077.pro
cf6d8d566dcb74f9d4e6b8ba1e80f536
59df833093a3f165df25dd73734727ef9be14230
18138 F20101107_AABIJH wei_y_Page_093.pro
03513e316522621a3b8b6bed18d30985
6458f6c0f1e379b4e10c6a9c178a748cd30717f7
16546 F20101107_AABIIS wei_y_Page_078.pro
5233e8aa0c8fea257fe148d7f7bb9ed2
4ab5b72e9ef35730b423dd589a04b8b2a4cd991b
20228 F20101107_AABIJI wei_y_Page_094.pro
07712eb6bb90d8ddaa10809cf07ade65
7b499c56fa5fa5dd9f731ba8bb439de0496743cf
11583 F20101107_AABIIT wei_y_Page_079.pro
cfa18170ee9589f74aad1a5aef25e9b2
1dd27028e869c4be8e13f89f3f3c151308d7f205
12205 F20101107_AABIJJ wei_y_Page_095.pro
77a9c1d85516d1277c0f7b7185fa4540
8f6b0a2e8f3e63a6c7ff9141189d9d18adb6300b
12248 F20101107_AABIIU wei_y_Page_080.pro
90448a4528fb912479eb89a8200d2ce4
b96f5d464c0f54c5b5e7f0817bbde208ad30f9b1
17558 F20101107_AABIJK wei_y_Page_096.pro
2d4ca099ed0436cf7d78b101e95c6038
1b716a3645f147792c5c7af4312988b4952b404f
16532 F20101107_AABIIV wei_y_Page_081.pro
96d63f9161f8bdcaf01e61501b2bdcab
47df0691de79fc9e50e092aff999cc327ecda021
14320 F20101107_AABIJL wei_y_Page_097.pro
9ce7bcf82bbfd1041a8db1142e4daa0f
16e936f10cfe534fada848c1d7a1debf2991fc32
15995 F20101107_AABIIW wei_y_Page_082.pro
ba9ec1e748eb54637458b1d911b53fec
53f8575ad5e98b2e387d00a77cf848cb83a5ab83
15468 F20101107_AABIJM wei_y_Page_098.pro
309c769a9c6655d5c011449a221cf15a
2eeab6511f07a423333abeca18f33026b15a1aac
13911 F20101107_AABIIX wei_y_Page_083.pro
a0396a22751fe56a59dd759356925db2
ab3d5c30d1ca5bd1d4afe129613e4e34688491a3
15150 F20101107_AABIKA wei_y_Page_112.pro
70ad01a3d80c314deb2ffb0eba232a9e
a653cac7f5dfecaafb2e9750bddc8d1b013d929b
17131 F20101107_AABIIY wei_y_Page_084.pro
4760dbc944b0989ab86a5a4543a9ad30
fce2d5a5dc8fc54a5fc8da0e560e5d1d54712720
19852 F20101107_AABIKB wei_y_Page_113.pro
68971dade229e5e3d863c6d4753cfcfe
d829eb7273ca53a406468b61c8b66fddae4d5c88
18570 F20101107_AABIJN wei_y_Page_099.pro
75f794535f61dc357bde69a4f9aec759
89c196300f00cbe48898fe87fbe7da6289e5e25f
12252 F20101107_AABIIZ wei_y_Page_085.pro
029d02bff08e6e4bfa1f4a3eabac3d94
361e43825c2354429863a19e2fd67de9e242a94a
16193 F20101107_AABIKC wei_y_Page_114.pro
dbf8937fc652c35a9273c66fc5d4f951
b9339454118af516c8ef5daace524616a148ddb6
20386 F20101107_AABIJO wei_y_Page_100.pro
88044482becd79267e160b17c467370f
8240caf94720187772861787f638151af5dd9ad5
18527 F20101107_AABIKD wei_y_Page_115.pro
a598ec2ff5808313a8f46bb1138a2cab
60dde795e5051530615b318b2dee91f11c2f2454
18865 F20101107_AABIJP wei_y_Page_101.pro
322bf4e43821703fd2c8bc8ae6f55fde
44afd6b36365e7bba3bde38114e8cbf77235b910
16901 F20101107_AABIKE wei_y_Page_116.pro
7a9f289877e8c6b10a241c2d742da64b
8b286a87a510c0025f25867bd99b9396fc9a83c6
13844 F20101107_AABIJQ wei_y_Page_102.pro
4ff5eaf4f12d63aeada00623fbbebf14
304e5377c55e163b3d7cb0d8b6fd7b29cd4a8cbf
17387 F20101107_AABIKF wei_y_Page_117.pro
abfba1e77e08cb626f1cfb6646b40eec
3d8a061217491c892d425731ba63f890c52df8fc
14970 F20101107_AABIJR wei_y_Page_103.pro
ba0a5d17d41810a0adc215bd3a492316
0234a2c9f845e079b78b816c50385b6db2832868
14567 F20101107_AABIKG wei_y_Page_118.pro
348e1757d956f5483f89cc317683d4a8
0f5e8db08f6bb7bbfe3f3e17a9c7b203246556b7
11599 F20101107_AABIJS wei_y_Page_104.pro
b50a72eac836398e021591dc1a73cecd
29f164fac6ee076d0ea2b13a5df35ddc78326f62
19179 F20101107_AABIKH wei_y_Page_119.pro
9d4e870b5a16a35b3c6abb881303c9fc
0b405454590d11251b9bdf60c3749ae9a6d1951f
13080 F20101107_AABIJT wei_y_Page_105.pro
83e94d9ad2576744f4c025742335407c
e787c2f47be073a575968d268eb2b1f5d4e80e6b
11373 F20101107_AABIKI wei_y_Page_120.pro
2aebea74165be39f9a170ec555aeae74
733458d1f41af7df20c2119b61b417b981037f51
11657 F20101107_AABIJU wei_y_Page_106.pro
81248046d8374a4d52ad37236af5f44a
d23d1888ec80ab1e35bfe4c4c25d09778bd584b1
15222 F20101107_AABIKJ wei_y_Page_121.pro
4432cf9129dbdfdde6cb276b1700a8d2
1f9ab98946dd6c8197ae560cef57ee718a453a07
16326 F20101107_AABIJV wei_y_Page_107.pro
7324423462630ee2d64a0d744f086a99
42f2d7f5baa6b0d59a352dbc709e847921901fde
51399 F20101107_AABIKK wei_y_Page_122.pro
0757e6f7a74b207aade43222b322e19f
3dda80bb577f14fc177ccfb52a92ae87c52814e5
12350 F20101107_AABIJW wei_y_Page_108.pro
215e5416664286ff2a50d7cf391a812b
1ecd5f1ab3eb0602ac65ab77bb6b60597e3054b3
56451 F20101107_AABIKL wei_y_Page_123.pro
1f2bc5585829b150c2a8f1eca259abd8
0a38b4a316a5ca03013d9aaa664169cbdb5eb817
16699 F20101107_AABIJX wei_y_Page_109.pro
f05a257d1ebb8e3fe3a7f1a7c6514fff
dd9acd0800085c2ae970ee89d0706b5347d353ba
19715 F20101107_AABILA wei_y_Page_138.pro
c38d86ef14c1040532bd16b8cc1389ab
4e6bc286207bd35d1a75699243362b5a5301491e
58088 F20101107_AABIKM wei_y_Page_124.pro
37dc0fdefa4e41d2418a6962a9dd103c
9b997fac5db60b09f79f0e6242e0de9a8875b9aa
13825 F20101107_AABIJY wei_y_Page_110.pro
023a33f2c6fdb2deffbf12bb0cfd2e74
2c7542c182abf468ec8adc1fe208e0217e157bd8
18296 F20101107_AABILB wei_y_Page_139.pro
0b305c7dba735809c5e783dfe52d282f
84eef85d0d44ed09c5ccd7571bcf4f1266a6405d
54919 F20101107_AABIKN wei_y_Page_125.pro
554acd586b3b1c71a4983199823e8b5d
3ee5b2cfb081cdb0d51af337dddc2f08d0a68fae
15363 F20101107_AABIJZ wei_y_Page_111.pro
5425ffe16381c08c728d82af32f53758
59964ae87c7f659c0726a5daed9af9fe4e6e860b
15872 F20101107_AABILC wei_y_Page_140.pro
38ac0894e7e0a341f4ca76c57ab5dbe8
41a6479042088072650bc318ebeec47df09f9ee3
16930 F20101107_AABILD wei_y_Page_141.pro
f46e94e864d7266833951f1fae464806
51f30381754e2a9d452b439e96da138b5a81b47b
54895 F20101107_AABIKO wei_y_Page_126.pro
2b6fb55f2d4a9dd74e00adad193b4d50
93da9261c769d08ef0d479a00b9cc96a4260c75e
16523 F20101107_AABILE wei_y_Page_142.pro
be0ccc39ea2e5c40573695dea923fa3d
d7fde8af00ac640879d47f35ea9664f338e34a49
56120 F20101107_AABIKP wei_y_Page_127.pro
8e91e0a9d5beaa285fee6bf88d8e3761
e0a30392d1fee2cff7a7020fe5705208701a3913
18109 F20101107_AABILF wei_y_Page_143.pro
a133ff78c2918777cc1c19d21ee97cf0
3a9ff9c977e5fbc60aa4879ae299f7e797396a43
53903 F20101107_AABIKQ wei_y_Page_128.pro
f5107641b42d63aab7e043c2b0efad98
d018f2937aa657ded00886063068091dba9a47bf
16612 F20101107_AABILG wei_y_Page_144.pro
72006155e719ff8b264143047836758b
45374ba995842673337c21ef3ccaba64069c154b
52478 F20101107_AABIKR wei_y_Page_129.pro
4f36cc608e0ed36e1b2317ea766618a1
e06fe751556c0d2395d6d3081e979bc7e4962ac1
13559 F20101107_AABILH wei_y_Page_145.pro
84f826ac0ed49922dca02d4cc121da47
1dc73e9a8d909aaed21cb31ef3617965d115451d
52027 F20101107_AABIKS wei_y_Page_130.pro
fb0a76a93579b272b9ffdcebc6e3a3d3
c9298275b29f48f5ddda153dc9f50d85a2d4cd2e
16734 F20101107_AABILI wei_y_Page_146.pro
596015966cb64bd1a4dacf2fca221073
2c1fa0e7a2270ef4b4b0faf5d969325edf566fcb
55874 F20101107_AABIKT wei_y_Page_131.pro
c70f85bdd22e7bd57ceacc5bfe9d7388
d31105209a3df220dcc3c38d2bd7c33f85be0911
17712 F20101107_AABILJ wei_y_Page_147.pro
dfbaf570110b589ac49a3ea1f5115504
efc8c7b53cbb4a5f603df4662631cfc3ae08acad
10796 F20101107_AABIKU wei_y_Page_132.pro
f5c9aa4b22ef495cba218260bd00d3e8
ca7e202a5d14e27ac5c3b6951695adf42bc8e554
15493 F20101107_AABILK wei_y_Page_148.pro
ebe4005097d6d9bf19b14452f5d4c1e2
33870de6d9889a60dcf86cde9700d53cef7af68e
58852 F20101107_AABIKV wei_y_Page_133.pro
6c95c9481c36e62c8a5affb7c0d3ba23
5280c3e5372c6d332d8d39dd1b6b36b5bed1d563
15251 F20101107_AABILL wei_y_Page_149.pro
bff134fd16de5ab4ad22a8aa0f35fa28
bf9797e127c71ef2f4e4add7fa8ca6e6df3a22e6
62244 F20101107_AABIKW wei_y_Page_134.pro
6d93738939437c37df4da1f348d84953
3fb3886d5c5a20d32a8336fc1d7257ee315b2ae5
13975 F20101107_AABIMA wei_y_Page_164.pro
7e19cd42bb8946fd5a84e48954124936
613cc088cf083aa969109592fdb013b6f5096b42
17100 F20101107_AABILM wei_y_Page_150.pro
32c7d8eece243fb1b1421832757670d4
05d4b2add903810327ed07a685a67584b4699b73
52996 F20101107_AABIKX wei_y_Page_135.pro
107477640f78df8d5cfcdca769ff1dbf
33f51382090f56a5c8ef9b3f82025be78b5e5dbd
13909 F20101107_AABIMB wei_y_Page_165.pro
ac09f589b8e1fb35456427c55324c9bd
9f6c6d41eb94ca561e59ac642aaa34c4315f1353
20300 F20101107_AABILN wei_y_Page_151.pro
472ab68c078a62450d5e03bbf9da178e
2c22f5b4ffdf591ad75912695a2471ab82b1bfb5
55745 F20101107_AABIKY wei_y_Page_136.pro
d56f24a407783c01bf757c7b4fc53acb
30bc38b1980e62a9397eaea4af468ed401924544
35240 F20101107_AABIMC wei_y_Page_166.pro
4400320ba739db013ad100edaf12a872
2341f5d64207bdda7734970ca7d617a3d2755542
18135 F20101107_AABILO wei_y_Page_152.pro
a15acb3809ea6f1c3978d88aeee79c48
0154ffda790272fc79872e506ad1511cdb64cb56
7687 F20101107_AABIKZ wei_y_Page_137.pro
609ae331bf09fd29d285df70a3a08d2b
5edfba2c20707d12b0f5fbe5f040c2bb497059ae
15123 F20101107_AABIMD wei_y_Page_167.pro
d56c29da08ee78707c40e6b1159ee6c4
f758ae72315932f99219812b68c3f453603e6e35
15883 F20101107_AABIME wei_y_Page_168.pro
3fa74393f28979cd3f8d5afff7c8e07a
68f9bc097cfabe10cce9b7113888a82fd93488e3
16785 F20101107_AABILP wei_y_Page_153.pro
0be64b159166b63f182a3ca4050004d3
7975b8dd79a11bcc534d67645a267c2bff65c059
13534 F20101107_AABIMF wei_y_Page_169.pro
956d9f6f9018f0f6ba39e89c4cfe6ef6
8dd1c3df1f3935c28685815a17f18748e7858eca
16535 F20101107_AABILQ wei_y_Page_154.pro
4051f23ea10d875c091a7e24ab17098c
632d030ac5cef52e1c18ed4662596b754a4e98ab
14459 F20101107_AABIMG wei_y_Page_170.pro
d68e517e5f46498c6c82640851fd55e6
0d106a52d4e35eb2e91d8b9a242b10328ed8bef5
16242 F20101107_AABILR wei_y_Page_155.pro
a5cdfb5b0d88ee3158abfd00bff91e52
1516b1d28dc756eef4f952847c6078b78ae005d5
15093 F20101107_AABIMH wei_y_Page_171.pro
8707961f7a43f93b6efba8f1bb1f2c35
802d5e414055ac12c48f0463e3c3131ef8b8ac00
16578 F20101107_AABILS wei_y_Page_156.pro
4b44db1d60a043142ea3e38db163810e
e6624504d917d959ff17b343c6d36e7124db57d2
13672 F20101107_AABIMI wei_y_Page_172.pro
be7cd00c0a1f2c45e44eb239ae20e7fd
72f4abba63d4bfd4a77eef0d3105cc3698b6ba3c
16555 F20101107_AABILT wei_y_Page_157.pro
9d7d3524c326eb33890ddb4352cc6ee5
be19d51a65e43daeafba55a8aa0e1302f9518b54
15415 F20101107_AABIMJ wei_y_Page_173.pro
c093a31664e7d811d4d5317e68bbad5c
f87e5aedf175820c7010004faef1f3b9dc3175ec
17022 F20101107_AABILU wei_y_Page_158.pro
bee6e4ead22f8aac5049490f4c45c368
4402df3e24ea14bb9acd515024ca970be0c78ca3
17359 F20101107_AABIMK wei_y_Page_174.pro
33da854eb43fe3e56586f5cb6d93bcae
9b020d551919b4823c3bb399c2ee16af18446a51
17245 F20101107_AABILV wei_y_Page_159.pro
8ebf5e0cf85354527a40712aca322032
646439eab3c6680f15b649dfa4cb095b318eca73
24939 F20101107_AABHJJ wei_y_Page_132.jpg
75f8f648e5816ee59c5266fdfc6b1083
ab712498bff2eb32c0ec1a4c1f426ed65564acd1
13724 F20101107_AABIML wei_y_Page_175.pro
097ee48e3a6ba4f5a775b76c74c83fae
8d00e4e0cd04478fe1620997a3ae0ebbc34a9282
17418 F20101107_AABILW wei_y_Page_160.pro
6b679835952bd4d5c3c1f9570ff29d04
2c5adc2abed83a7bf06675a23760423391473372
219145 F20101107_AABHJK UFE0022538_00001.mets FULL
65a0aefefc6d3126af7f07db469cf5e2
1f0d0b6f3514ff9749662692a7d8896b777330fd
14086 F20101107_AABIMM wei_y_Page_176.pro
f39f1c7920997311538e8f6b5c2982b2
b7c5e328907bc85baa87b72cc76270a7573f7405
18760 F20101107_AABILX wei_y_Page_161.pro
ac3774c77165b1e0c4de9c402dda279b
266771775fb5376d3862274ae4b18950fd95bd90
59997 F20101107_AABINA wei_y_Page_190.pro
d6abab18a4be0735115b0f09715eac1d
d821c951a755d89a9ebe2ab8e4b082c8d8acfd14
15080 F20101107_AABIMN wei_y_Page_177.pro
e3b54051dd1ca40f41c9a49cf94c5cb4
19d1137fd32bb32ef95e016657536bb9aea379a6
16250 F20101107_AABILY wei_y_Page_162.pro
9ce95d63509287a135aa045a5e5f2012
c66c7695d516b0ca7259567a191c1451cc0731fb
61596 F20101107_AABINB wei_y_Page_191.pro
292131c908bbfb7dfa6e0bc57151a138
fcc01e7d532a934d2d5abb334ad7e6c0fe307cef
13785 F20101107_AABIMO wei_y_Page_178.pro
d36c8a45c2dcfdf8f93a526731962ed5
867c401a48d0e1c0d2446df9c7cfd19c73b435d1
15316 F20101107_AABILZ wei_y_Page_163.pro
3dce39ee5bf074b738413852558dbd3e
dfa34fd216234fa551e121499493cc7df8ef2f3d
63912 F20101107_AABHKA wei_y_Page_014.jpg
613c0b5d389a233160106b2d4b567dc9
9cb9446c2de694744f65e60677b59441c87646d1
34328 F20101107_AABINC wei_y_Page_192.pro
5d5345d803d180fd0b51e3734f762400
47069305e8d662ed6a38ae6bbb088cdc22a92948
13777 F20101107_AABIMP wei_y_Page_179.pro
dec1dcf36bae9d01ac26ab88dd9ee491
48990952e0980f6c6e7a687ec7bea9b8819bed33
54302 F20101107_AABHKB wei_y_Page_015.jpg
1ad02ca16c372b76012320bd5c970f19
eca2f9430fcb34d3c474fea451579552a0a61a58
18864 F20101107_AABIND wei_y_Page_193.pro
e4ecb8f890640ca5a45c26ab58d39a1f
878716cc6ca948bc599501e852ec555fe303ede6
26649 F20101107_AABHJN wei_y_Page_001.jpg
77794096c1969daad1a946e48ceb9932
8f910edb27b1e02f65acb57465f69ce36a6fbc0d
82860 F20101107_AABHKC wei_y_Page_016.jpg
332fe35cf1f584a44aa85caf9ff5e167
4c2f97ab9706799279b3503d79aebef601546cc3
452 F20101107_AABINE wei_y_Page_001.txt
b398d5027ac28a2d5cec5c7053caa5af
98d2df69b88750b643906e2ab4d2540f26a848eb
13839 F20101107_AABIMQ wei_y_Page_180.pro
bf18bb8935281ed492062feb4ee25a06
905e833504a35efac2c7c64da3aced4ee16ef9ca
94579 F20101107_AABHKD wei_y_Page_017.jpg
e9fef0e5ebd2ef70e93a8af602182943
74b32bf8409eb774b8d184621a15627daa85a307
86 F20101107_AABINF wei_y_Page_002.txt
0ea2353ceadd1a54cc3535649540fe8a
a4d2c80791dafb619915c0d8978029834e2f763b
3539 F20101107_AABHJO wei_y_Page_002.jpg
5343707a14a0d1270fcb7f58323e81c4
05138aec02076ca6c51772b54d7fc2c1a7f955a1
15345 F20101107_AABIMR wei_y_Page_181.pro
e8ee2c6b3030a6ff0fa7c18cce44d079
9722e6f7d306883f7f98089495545085e13798b8
86202 F20101107_AABHKE wei_y_Page_018.jpg
71bd2c9a4321b43a1804da54aa841122
fc0757757112c4bbfbf49d9361797c829b92f73b
578 F20101107_AABING wei_y_Page_003.txt
9e990e515e59ddc0d5529c964cea021b
70ec34b50a62e049ea21f86612f8c2af2202125e
30761 F20101107_AABHJP wei_y_Page_003.jpg
15d9b2b952ada319cd3aab4b62b31ce9
bdbbd88077e751d07e817a922ade64d4b0cd1586
14058 F20101107_AABIMS wei_y_Page_182.pro
9410d15450c025ebd6d3cf33454af75c
b849e1aaf924bc2ff49a50b16933862e10a6cbee
93099 F20101107_AABHKF wei_y_Page_019.jpg
dbe5127e36f225b30e2d598bcdbd1d3f
03e5686e29a1ac815db254426da67f301662ebdb
3798 F20101107_AABINH wei_y_Page_004.txt
789d8afc20fbffd662bbca97f7520f34
d23396c34e6addb1afd7445052804489cd9f21d7
117374 F20101107_AABHJQ wei_y_Page_004.jpg
9e44e49a5640b120b2ce966434e5ba29
f1189df84ef848defbcdd016b4101ed29813b162
15206 F20101107_AABIMT wei_y_Page_183.pro
3c08ea82ce8a9b77c30ff406f0e84ed4
db91f754e7ebd343c544571ff8bb13b1554a17c1
93401 F20101107_AABHKG wei_y_Page_020.jpg
c5e99bb675a514626c3b9181102c093c
17d9abc2dbeca85a14a4fd6cf0b0da48048b6280
1955 F20101107_AABINI wei_y_Page_005.txt
f95782df83d9cd5bd9b674a916bf02b5
da84d3baeaa9981e9be11216da3686b125c4c572
70229 F20101107_AABHJR wei_y_Page_005.jpg
99928dae5a35c7ab684778236ac42820
be3de4df11e0c66927aa190aad56a14a317c6491
18183 F20101107_AABIMU wei_y_Page_184.pro
ca752f022f05c9918798892ab8b2ad71
fe40e7a82fafce4683fa787726e9576ae94ca57f
51257 F20101107_AABHKH wei_y_Page_021.jpg
d25d5623f23635b71b225456962d6694
942037074938a49d4ef168a2842d9bc7d17859d6
684 F20101107_AABINJ wei_y_Page_006.txt
4a983c998836b12683f3c05c839f69aa
b9bcce35cb0cd36343e64917d0c15bee83de11bf
30549 F20101107_AABHJS wei_y_Page_006.jpg
919f23fa5a4769e7edea3e5a7680ee07
ceec4c1dfb4986f0bfb4257df89aa60a091e80f1
78236 F20101107_AABHKI wei_y_Page_022.jpg
8828eb7eda9f7c4f1ed26ffa57efcf3b
4e5301e585a76e1546728da38446cedf151330f1
3053 F20101107_AABINK wei_y_Page_007.txt
a489b94f55523050a6f400321b4a3918
d28a5a4b64e4c27da854addb130a27f664f04659
132100 F20101107_AABHJT wei_y_Page_007.jpg
80bc8a4fa1a692461b0d5c2544f2352f
bf033dbade34da8d7d7285adf2513d69d90649dc
14549 F20101107_AABIMV wei_y_Page_185.pro
c8dc0a5eb71648e2c871533dc50711c9
c18089de960e6446656f6b5631ae162dbfb8e4b3
71032 F20101107_AABHKJ wei_y_Page_023.jpg
b563fcdc2e3561ef90dfa5452a925f48
3c5e089dffc7b1cf1faf1a2d275f9a81f9c507b9
111 F20101107_AABINL wei_y_Page_008.txt
b8a166934800970ac1f97ef09e3d17d4
66e5db6c87b263709f6fc88aa239c58e99945ce0
10480 F20101107_AABHJU wei_y_Page_008.jpg
c2b081fd073e12fb9e2c0ef7dcf534dc
ec2666c04fa64beb1fd3a67b4f74f780d4b90d32
60248 F20101107_AABIMW wei_y_Page_186.pro
f5f280623de31d40ad5271668d2ff9fc
573e9d1603738980656cc8fc0221cdeea9536b05
1530 F20101107_AABIOA wei_y_Page_023.txt
ae1cd3146d6c9bc9b3e16661c721c97b
47469868fa4829e2eea2db9b4c9a04b600ca14ad
32068 F20101107_AABHKK wei_y_Page_024.jpg
09f0467e5d75e4a08c9ec44d4f7bc175
9d9edcb9f559f5eb028be2c4fd068f77bdbec198
1916 F20101107_AABINM wei_y_Page_009.txt
3f9c1848aa4a92943a35e170b9e02284
2560de470c6a0521b43f179131a14610628ca355
92442 F20101107_AABHJV wei_y_Page_009.jpg
94b5c4513badd3c29a4a780201bad38e
1772c32f15a256591088f3661d265115ff54de7b
65332 F20101107_AABIMX wei_y_Page_187.pro
ca72cd8597958e2e1b585e94b21ea316
40b89406929b751c021dfcc6367c4bdb8237279e
1075 F20101107_AABIOB wei_y_Page_024.txt
c6adf17a562ec35d77d7473cfe9893bd
fd51863d34718581f399fbe8c42b7d83478a55b4
88061 F20101107_AABHKL wei_y_Page_025.jpg
c1a64ef4425f147fba01dd4502d5c2d7
26b19eb8af95984371385d44c74f04017699562a
1391 F20101107_AABINN wei_y_Page_010.txt
e87695abdd2ae772eb3c34b09f197cd4
a8620941792247239c3656eab36a69915143f5ce
72245 F20101107_AABHJW wei_y_Page_010.jpg
b20e2ac70599e5ced1ed0287e761d2b0
6d23bc8717ac56cb1dc47faa1234a041498ebd93
61091 F20101107_AABIMY wei_y_Page_188.pro
fc7d8093aeaf53bd33d2be2d63db6a95
649dd285c0826a93b9fbcdd0932732b01f96f805
2016 F20101107_AABIOC wei_y_Page_025.txt
fd2719b63988b68a740c7fa071680d8c
fe2073ff47462416e49003d3c59ef46ca61699af
116121 F20101107_AABHLA wei_y_Page_040.jpg
af52dc97c885c6ba2709d8cc1161176a
5f8e1e2f548cf09da134ae75077d4af1e95b70d4
72956 F20101107_AABHKM wei_y_Page_026.jpg
6055c066ab6e4c875765eb8d95bf44cb
30da4206291e45d33daa83bf36933fc0a93c00a1
2165 F20101107_AABINO wei_y_Page_011.txt
bbee1915655fec3edbedfc563f9b7f95
e46a95393fb187f533fb9c4c5dbb9057128271eb
105362 F20101107_AABHJX wei_y_Page_011.jpg
4ac3b7a23a3ff3ba121878244e0a430d
6de77a6182759094ee7669957b2002f7b6f15db5
61895 F20101107_AABIMZ wei_y_Page_189.pro
5d37e36b0aa7677c45642c6adb3bc048
16a9cba41b5cda2221c8c4290bb18e8086faa654
1700 F20101107_AABIOD wei_y_Page_026.txt
7f0459d5b1c7f009223e161970654a63
739147b60e83807667413e388143370a05c7edc0
2028 F20101107_AABINP wei_y_Page_012.txt
155c145535f4fd891cf21d8f8db39290
817a0792198fe77f62dcff8c083471b0117e8896
29502 F20101107_AABHLB wei_y_Page_041.jpg
9ff9fad6a30396ca3b51a0a46bab9657
4942c5a4e7b48e07e771fbf202f34fc155b8db40
101655 F20101107_AABHKN wei_y_Page_027.jpg
3eb5f979bb5a2d6524c1aeed89002148
6fb5be573e62b2d63ef0cb635b627b99b3ca34d6
104042 F20101107_AABHJY wei_y_Page_012.jpg
b9c1a945731c382e642dfb9872cfa6b7
37264ae3ea09ba9ae671b47051d76f66e4e5aee0
1958 F20101107_AABIOE wei_y_Page_027.txt
11db8335ff9d47edceb115e62ab2b005
c9988917239504c9686e069836b9eee8468b5059
1533 F20101107_AABINQ wei_y_Page_013.txt
7c68598f6e9a03adde249b925138449e
1f47eca8b119f6b9f105c4e6f211538404400f1f
102660 F20101107_AABHLC wei_y_Page_042.jpg
a327c196bfaea5bceacd6df14994cd09
15ddcf8cb149115b3a2ba6d01942dc3e8fc5e9af
65602 F20101107_AABHKO wei_y_Page_028.jpg
b51157c95bd54d183a6afd917915e6d9
6f6b4eec2e4e86157354300b105b07c7d55ed852
65545 F20101107_AABHJZ wei_y_Page_013.jpg
207fc89961fee27d4f64b95b005b67de
aa745b80e4d2d3cb45f6df809ab91302b70a69b0
1755 F20101107_AABIOF wei_y_Page_028.txt
934b65a261cb5ee976b27cdbd7425878
307b93bbc1c20fc75c4ceb51ba76b529f07185ef
112174 F20101107_AABHLD wei_y_Page_043.jpg
ac45717d10ba0a462b5245c52f2253d3
3d42b40becef319e83875450366c44e0b364f119
1534 F20101107_AABIOG wei_y_Page_029.txt
585cf32505e996c00b9e31589117e0c8
425b6d91523eadd8af8c71d71cf588371f316acf
1505 F20101107_AABINR wei_y_Page_014.txt
a9cb503c8f58c17630aaaf77d1b9c7ab
a6a216ee7e52085b89c715734590ba15cc02597e
112694 F20101107_AABHLE wei_y_Page_044.jpg
b8dcfa999f100275777df443587f221b
5cab87657fcee14d39ecf85414c5140860b69d89
45847 F20101107_AABHKP wei_y_Page_029.jpg
e0a301e161cb3261194525f0d2e9a206
61073466f17e803471828574c11b427ec21c1b84
2084 F20101107_AABIOH wei_y_Page_030.txt
6e63a3b3f07526bae1824fba437c14c4
ad21865c9ef9da32478125f688f8bb7f4bace9c4
1929 F20101107_AABINS wei_y_Page_015.txt
e04e823968d97713f78c4de599b9c1fe
bbbec32a4ae6a6995c66bd47354a5881e4d19028
113832 F20101107_AABHLF wei_y_Page_045.jpg
cb3aa9a5b1fea045f9cece80ffd21491
ada7e5fa69c15e67a830363bdcab2439d9d89f06
107552 F20101107_AABHKQ wei_y_Page_030.jpg
163c5d9e892cc52114574d714cbc8589
78f77048c7dce0d8c5fd3e88d94cbb32dafee7e1
1726 F20101107_AABIOI wei_y_Page_031.txt
ebaccee6633b24777893d63c41f5d66b
bc45fee108c702d186dda7b5ceedfe9e8a6f78f1
1774 F20101107_AABINT wei_y_Page_016.txt
02fe57ff22ed2a925eefdc3f2142f7e0
b0a6e54d7e3e751c6490f0e769884d416f0bea80
113922 F20101107_AABHLG wei_y_Page_046.jpg
65adccf8b5d0b6f3f33c89dff9a029f4
2e54520509f58a8042164581d918f22c5f4f4cb6
62488 F20101107_AABHKR wei_y_Page_031.jpg
76811fac45cfabfae8c9babeb7dad0cd
d73605c4d79f75955067d3b30545dbcefaf8367d
F20101107_AABIOJ wei_y_Page_032.txt
e44faa77624ec267ac5701382696c24b
35270e1cf163d5884be09fdaae842ece3090d9d2
1842 F20101107_AABINU wei_y_Page_017.txt
09fd297c4cf5259ae4450ae092695e71
ebd60b5ded751aa8c79cd5e8182f0f173ede1d90
106461 F20101107_AABHLH wei_y_Page_047.jpg
97b2e00f9d4fda0e9b690442ed8ce6e4
fbe742c8e33527fa2ec11cef9d6e667adfe8c962
84987 F20101107_AABHKS wei_y_Page_032.jpg
e1b2a7811e49858afb63cead5c58a735
4f05c55afa159440d9761abc3e0cd9d992065b14
1903 F20101107_AABIOK wei_y_Page_033.txt
f9650936209ad1a81cc5298ed8851a0e
56c15f2d71ba82e7c32d2c49364254a088875119
1607 F20101107_AABINV wei_y_Page_018.txt
cac46e847d7cca749b516eb42c61e1e8
0efb3b14ef2a07e6094469dfca4b9fdfd915dde3
101491 F20101107_AABHLI wei_y_Page_048.jpg
38d5a68181c59e1347893f1f96cf8265
a0d701fe227404dd54ed12b936afd627af0eb262
82491 F20101107_AABHKT wei_y_Page_033.jpg
5ccf6ecc8a978e227b9d3bcdcd3fd14c
009e5f7a79ff1a641b0124af824b248d6f491953
1632 F20101107_AABIOL wei_y_Page_034.txt
35d2dffaa8cf52a0f6dec81efb03cc09
3640386a7ea8e721723cd6e7caca9b2b5600ccc8
1741 F20101107_AABINW wei_y_Page_019.txt
36e1e270fa7c932ae3f9ea303d1f251d
ad53802ea8a25dfcaa452aca8014f5d735283900
46165 F20101107_AABHLJ wei_y_Page_049.jpg
edd704c8c0c777aa8d3a42273df5af54
25bccd1f4852f38957209805b6ba93d6ef907300
62404 F20101107_AABHKU wei_y_Page_034.jpg
854c5893a1d36b12d2e1d3f54ad450c2
a215316cf7125e82fc8b099112c78a4b61314700
1430 F20101107_AABIPA wei_y_Page_049.txt
2ce8b034d912b678ee5e156e3d19a607
2b6128f6a100f5a731f5e53b55168aabf24519a2
F20101107_AABIOM wei_y_Page_035.txt
de0dba5e16dbd72bc426accbb4b0ad45
170b8cccd194f2a93a1288dc37d421d1390898fb
1770 F20101107_AABINX wei_y_Page_020.txt
8499377022b9e603111a5d555d5e1127
0bd683b44566ede4c659c3d28cadcc2646e3a08b
94201 F20101107_AABHLK wei_y_Page_050.jpg
5c775478e9fbf23c07634fa636c16c0c
9063cabe1d8993e6344e1adeeb88304d784a9502
70647 F20101107_AABHKV wei_y_Page_035.jpg
4bc6962982de42db1057395d4e28bf7f
addf7987a91e2cc3a57544b2d342f1188e0043c9
1960 F20101107_AABIPB wei_y_Page_050.txt
5cea4f4f02a4074aaf1ea76a776fb9a0
219c0e0dcb00716c1c4070994c24a1ff0a2d0dc2
1866 F20101107_AABION wei_y_Page_036.txt
8061f8eda33b51fcf7f6fd706c32f95c
c93f8159f0d210aafb973029c9a8efb3f4a59cfa
1620 F20101107_AABINY wei_y_Page_021.txt
7bef6581e2107376717067c17915a00e
3bc9b54dabdc04a56e9b9a720c50798c18b983a5
109799 F20101107_AABHLL wei_y_Page_051.jpg
c061a5d2bc1d206a810afd9f56648f3e
75e563a8e7f3bcec15c13ec937eb060d6e88daa6
82548 F20101107_AABHKW wei_y_Page_036.jpg
53d33b9003b81dea16bc8ed837a41063
260fab70d4aa4df8d22a781188c9be2e41ea051b
2098 F20101107_AABIPC wei_y_Page_051.txt
a06f35f8c0241dbd765a184fd6d0b701
2abd060387970932d8f83c26b8e5bfc63a74e393
F20101107_AABIOO wei_y_Page_037.txt
a7025384e45fe11944dd2f09c5de9920
abc7e6ca71265fbaffd138d7a7d1879da4e2f37f
2211 F20101107_AABINZ wei_y_Page_022.txt
9421788f9856f20eac5ec9ba72e7cd03
89d6369e415a05ec0d0551d845c096011664271c
106934 F20101107_AABHLM wei_y_Page_052.jpg
b608a2821d3eb9b51fd6bc3a4bbd47b7
f84b3a1cbe6b0fd294bb5bb8ac19ae7b80f6f560
86636 F20101107_AABHKX wei_y_Page_037.jpg
9167c632745ec8e25f3e8a6386e0460d
2475ac85012d4b31a87f51f1a541efd59f1be0aa
105905 F20101107_AABHMA wei_y_Page_066.jpg
e54c544f2af3a8e5820a7a16d3a7971a
1826928f7947a5c76b1db3c769a23320aad483f3
1994 F20101107_AABIPD wei_y_Page_052.txt
eb4de7844f567474aca33befbb9e0abb
ec058d44e61ef5d4aad6e2713484af7b6176ca6b
1595 F20101107_AABIOP wei_y_Page_038.txt
7040945917d6eb331ec23d29265d57e7
c86f75b23ce5a7feefbab6ffa89898dc811809c0
12677 F20101107_AABHLN wei_y_Page_053.jpg
b37bf9eff60694f5302087048bc7fdcc
629953265ba6ff851c6ca7e2fcece55ec0717455
74547 F20101107_AABHKY wei_y_Page_038.jpg
7ced44ef5ecec1d996c31d068f56afcc
e3f44790bc13e11d9d7beb4b5486463ea1d338c6
108082 F20101107_AABHMB wei_y_Page_067.jpg
042ecb565f77ee0600212d9d3ac3eaa6
bc0d1a99e89f0e977812bfbae4dfe50ab79e61b5
195 F20101107_AABIPE wei_y_Page_053.txt
90f629fc9f94931ba6e6a2418877824c
cb7bcb62e080509bd65ed5721d7635b7bdf59947
2029 F20101107_AABIOQ wei_y_Page_039.txt
892450d86f9ef8aa0301399be43a4bde
bb1d8384583944a87bd939844e6df4b7c6d98f92
21210 F20101107_AABHLO wei_y_Page_054.jpg
ebdb5a00a3a6e04eadf6d9a08dd626ea
223c3652dd8a820755244da1343ca1d1496a8646
102547 F20101107_AABHKZ wei_y_Page_039.jpg
2d284c2512534bc31a6b13555c6ad7bd
efa504f177b89200ad6b9caaedfde2fc57967630
53801 F20101107_AABHMC wei_y_Page_068.jpg
43fec9559e6b41c26e00a71152319be1
cfdf401b7f66572ffc826fe4ee7e77ff38b4fc5b
364 F20101107_AABIPF wei_y_Page_054.txt
39e4e951fc8f0f7c3abf3a8206f517b0
439a3c41e1eea25e188749b9b09ae1a363de5373
2256 F20101107_AABIOR wei_y_Page_040.txt
7217eb91b0fcdedbd1f6bf8fa7c31ea7
be36af83aec6d264f8154356bdba6744cfb26db8
44440 F20101107_AABHLP wei_y_Page_055.jpg
976879137cb64140237f71c072897767
6949db89471e144ee069eec82c8d5667ef3c55f5
48787 F20101107_AABHMD wei_y_Page_069.jpg
2c565adcf361680dd328331844575d8e
ec5da180c25c659d71ff2b271e767c48b1b9221e
767 F20101107_AABIPG wei_y_Page_055.txt
7abf97af9998bdfaa2e4cb4764e71124
39f0b704f3fdd73ec29d57b35d14940cd3bf3adb
46470 F20101107_AABHME wei_y_Page_070.jpg
b715b71e78117034c679853823272b24
73805f4949355efb020b9c83bfbae9ad420cbf8c
1369 F20101107_AABIPH wei_y_Page_056.txt
479c08585759d1c8afebec5bf35cd665
42d4f60febeb4dfb2ca985d3e37b0c9fc86edbb3
533 F20101107_AABIOS wei_y_Page_041.txt
e1c0b3df0ec1bc6757a968478a4f4e52
b3dca327ff8fea1b08341dcbd5d9c67abaf2be51
77073 F20101107_AABHLQ wei_y_Page_056.jpg
b0c0e280f4fe6da13bc0f97a40413acc
d0be6c4e55b3f2255f97320395ae804b7dfb9051
46701 F20101107_AABHMF wei_y_Page_071.jpg
05950d62e14bf745458c9cca1e828191
6c600d76f6b4e351a7f69d73e40f450e38b98c61
756 F20101107_AABIPI wei_y_Page_057.txt
bb00c3a1b4e803bcccd66663cd6f6a04
302d70b89ef9e3c2a14ab90d7a2ee0802541d96c
2049 F20101107_AABIOT wei_y_Page_042.txt
00353682ab509cb5259d56afcd5004c0
e13bd45382957cf17d1bf048e161a45a51d2e191
44265 F20101107_AABHLR wei_y_Page_057.jpg
e178b38a2f69bfffa8bea85caea7455d
be7bff8b97cdc0a800485ee44e7037d1b0f4a0bb
57536 F20101107_AABHMG wei_y_Page_072.jpg
29f817d544f15d3fd32957001198939c
5d32d52fc6652f085765ae75b795c86f560f2f19
1358 F20101107_AABIPJ wei_y_Page_058.txt
d3c3aa7c2165d3931fc1670b551da59d
16075a92a620c4933a840774e8372e6504539530
2200 F20101107_AABIOU wei_y_Page_043.txt
515cab80fc4cb5a549f58861f781ad21
4e600762c975e26821f71a5604aac8e414d3ab80
76782 F20101107_AABHLS wei_y_Page_058.jpg
967de1dafb31a0c62aca12cfe263fb4e
adcc50c31603c4dab5be7656e26f606e57a9a354
54561 F20101107_AABHMH wei_y_Page_073.jpg
49399d5611df4103a3972a4ff0060248
b9e6aa2f62cd165ec61ea10f2cbba2e3db55cb19
2106 F20101107_AABIPK wei_y_Page_059.txt
1aef788945ea95057d632f70b4551864
87b9d5ec8900fd70e509708e6e9baef55a3b0931
2169 F20101107_AABIOV wei_y_Page_044.txt
10570f6dd92948d6c2ad964da37d6dae
dc31d4761cfa7fabca48e11cecdb5cf7113318c6
103829 F20101107_AABHLT wei_y_Page_059.jpg
463eda8e263c21f1b3b60b7fbf43b300
277d05cbcfc1f5e1ec024c5a912cdc9cfb6009d1
54372 F20101107_AABHMI wei_y_Page_074.jpg
979bfe1bdc25d7ed03284e8cd5120812
692ed9c007ef9045697d459f3fe0e65cb61284a4
2174 F20101107_AABIPL wei_y_Page_060.txt
8acf93377fb0e7cf976cc2fdcb53d735
1037b322950affc7071ee159e04e68a486eed707
2155 F20101107_AABIOW wei_y_Page_045.txt
10495f8e2f0185df251f4a8d0292d571
3ad665f7f8558b0099f1dc666365e7e6cafd6690
109197 F20101107_AABHLU wei_y_Page_060.jpg
98ddbf2841df0ff46dc4070384f26571
f532f182b7a2e871d95513394e7314c6958bf7bf
55571 F20101107_AABHMJ wei_y_Page_075.jpg
e882d97bc5c02e38b24050ace57c9521
9243c76b03957e46f7de3e9e019884faadd44899
773 F20101107_AABIQA wei_y_Page_075.txt
aa658b0f65893c7c3b5ad19b234e54bc
7bec0398a4075ba35aa2740304a77032791ee4b9
2156 F20101107_AABIPM wei_y_Page_061.txt
46051df14516d63c6b1cac79559ffbce
f626777034c9cb6f91c12e6a6be66b7a4a284a7a
2221 F20101107_AABIOX wei_y_Page_046.txt
8ee70516b594d47895a258757d42bc90
a5f335ec0c0147a6ea54d643dbae5c1a18ed70bc
109289 F20101107_AABHLV wei_y_Page_061.jpg
191ebbe96ebfc5f31d6ed99d7f2b20b6
4fe45848ca0b6fc5250bf498fca49e5bf19ab34e
52773 F20101107_AABHMK wei_y_Page_076.jpg
6331f06cc7515a9089231b4b443c64de
31e0b784329ece1104436cbfdfabfae95009b2e6
746 F20101107_AABIQB wei_y_Page_076.txt
e1f3dc79973206606f46d991d14d4e14
8e1ac66bf3cd29b606f4c37e9c29b863903fc7b4
2126 F20101107_AABIPN wei_y_Page_062.txt
3673611f6e1b9fe491f534a8c86370d6
ea557b2ce885af25f2b4928a3709a4b09b4d8859
2088 F20101107_AABIOY wei_y_Page_047.txt
8b6658325d38d704c77572b9bb6ef017
ab8743265968a91b0ddab5d6bce165dd9a4de4e2
111244 F20101107_AABHLW wei_y_Page_062.jpg
c00bf48fc2a30fcf60ff25b8c6075422
bb428b353a6df9d1c52fdd09ac35940d4007c29d
52182 F20101107_AABHML wei_y_Page_077.jpg
31c1abf191cb6afc025b3e6507121595
f199f9a7c00adbc2087b11c2b940089753b84a1e
764 F20101107_AABIQC wei_y_Page_077.txt
e99d4ee6e9dfef55359e6b049d253feb
916e39a707f20415f923ae162fadd6de393e3005
2074 F20101107_AABIPO wei_y_Page_063.txt
3e1aa9786d0a4acd0d9fb794bbf24638
87368540b09818def312e515b846570a8f6fe22b
1975 F20101107_AABIOZ wei_y_Page_048.txt
47f11f20fd47120ec98ce7536c141d4c
a2ca525ea886267ef79d7de3142269af1c414670
106131 F20101107_AABHLX wei_y_Page_063.jpg
d7b789426fe9f6573195810217d7fe8d
af37246169c8a78234cdc390d4da55201aa44be6
52159 F20101107_AABHNA wei_y_Page_092.jpg
1cec54fe089178e8dbcbc30032191e5d
01a3b9779a12992c25344bd1611298bafddf2f55
57728 F20101107_AABHMM wei_y_Page_078.jpg
0ea76ecd55a914e3082a37b3fa279837
375c44633f229d5a05b51aac8037bc6f032307a6
871 F20101107_AABIQD wei_y_Page_078.txt
9404f016c922d00c71774999e5fc0f22
e65a97a692f579329fbddf927f427f42e7175bb3
F20101107_AABIPP wei_y_Page_064.txt
1e998006696f822f1fd54292335e1e0c
b92283ddb63d28ba7861ae4666271b66a624ff86
107775 F20101107_AABHLY wei_y_Page_064.jpg
d1771269e84994b31fd8c8e7457c30e1
923e30f0b3204fceae0103d54988f95e3d3c8143
68539 F20101107_AABHNB wei_y_Page_093.jpg
45ead9cee61d50e8d336054bfbda20ad
453699848770d41ad50d826110555bb66726c0f5
53383 F20101107_AABHMN wei_y_Page_079.jpg
96ebc6325bd2ce4eb9e2e074a8d24b4a
3404bd9bb35d6cab66d83fca3d93a21b73c12c4c
656 F20101107_AABIQE wei_y_Page_079.txt
702d9df271f6d6aa8c4361da7446c3b1
652c6baeb80e81b72239f4f1f780f106df428d50
2198 F20101107_AABIPQ wei_y_Page_065.txt
1bc0fd1976d6323c67fa9047cd3bc277
afc398d785e1f01d05fecd4eee5a671605a48d60
112889 F20101107_AABHLZ wei_y_Page_065.jpg
6f24dee827d28705c17ad6ce3c4306c0
caeb6ed85ff8e5ff8df07c313da5116783de12fb
66008 F20101107_AABHNC wei_y_Page_094.jpg
eee700b662e049b1a702900cd3faa47f
de9c9b09679fe02dff75725dae395d0f6257d218
52753 F20101107_AABHMO wei_y_Page_080.jpg
88d6dbf254660c230670204476922bb5
fb3c157683959dc88b443e966b8b453e24dedfdc
651 F20101107_AABIQF wei_y_Page_080.txt
289a39488767131837be263ce665585f
6cf1c981dfa7df0ce9f91e43676c06a795b43966
2045 F20101107_AABIPR wei_y_Page_066.txt
fc6fb2442cdf8265eb4e3f378dc0d9d6
f90448dccb216e526cf345c122507200a0fd381d
66266 F20101107_AABHND wei_y_Page_095.jpg
660f744a29c68f2e8a8e2cfcc6b41ec2
554888671fe6ee480edc252bd9ee8c89873c474d
60571 F20101107_AABHMP wei_y_Page_081.jpg
b91e4a12e5c18c87e0d12a752150d99f
f92edf67e8ceb0a1334d361a62aacf45b1c4783c
851 F20101107_AABIQG wei_y_Page_081.txt
295091e22dded60aee48fa06ec638814
4fdb0165954834c53f43174f61438b71067b915c
2167 F20101107_AABIPS wei_y_Page_067.txt
b20ddeeb48470cf15b2e1fc5413ba747
92b5ba2ee4efeba518660d9f4a9e33612480e420
68164 F20101107_AABHNE wei_y_Page_096.jpg
3069d9a3764672d2def8f23f2a59c1a0
d6cf27031bf19b74452f2f3e07426431f08c80c2
55853 F20101107_AABHMQ wei_y_Page_082.jpg
077178a9a240b1b34515d696da9dbf41
9cbfca4f37d0cecc0263a322bc0dd84f05f3d98f
900 F20101107_AABIQH wei_y_Page_082.txt
3a076beba73a68ac8cbff3828b6523c4
3d7c95f6fd77c7e5ef5fb030e030f6852535eb06
64217 F20101107_AABHNF wei_y_Page_097.jpg
542b417b47443de50c1a23fdd5a51aa6
9f4474bbf8e4385dec9506c5ab46a3051a59b2cf
698 F20101107_AABIQI wei_y_Page_083.txt
fc67db7838c0a08abd5d81546e4c4692
06d0a9717b28c0c02252cef057a2a202592c5804
1055 F20101107_AABIPT wei_y_Page_068.txt
2dd81c9a074fe8bc035cc6562a0eea15
a3c6627d8bccba401bcad633bb47053eb4c1ae63
63525 F20101107_AABHNG wei_y_Page_098.jpg
7fee8e4a2c4748682bbed9c364df09f9
2ebbdc3804b9e02cee353d588b62c633b42e329f
55533 F20101107_AABHMR wei_y_Page_083.jpg
2d800056b8ea7a680099a2d63caf8259
5ee2551b368efd80f4d1a6b56f51709af1530a24
888 F20101107_AABIQJ wei_y_Page_084.txt
ac976a85be406f9800b5b9c63b126e3a
831756f589a71a0e767027f9fab7799621aa5607
607 F20101107_AABIPU wei_y_Page_069.txt
4d7988a0d47db3f6fb3eb884cf63a719
20828f2be587c20fd407b157999803c58caa5a9c
70052 F20101107_AABHNH wei_y_Page_099.jpg
d2291be45afb03a9dd1dd5b21c6be7d9
e3fbdfb6af57198d71023ce94db1d190b11bc2b8
58851 F20101107_AABHMS wei_y_Page_084.jpg
668cf5019ac6939e2deed7251ed13ad0
a17364aba2a644423d063ee6e1bfe23509219f61
660 F20101107_AABIQK wei_y_Page_085.txt
3a79568e28108c3d5f79f1101bad1a45
acd22afbde5045a2a7e223aed7def300bbb0ac19
704 F20101107_AABIPV wei_y_Page_070.txt
850f2b366a71a7bf04848748630c2ba6
ff35ef381c9493da7965c983391cad641a4310aa
66452 F20101107_AABHNI wei_y_Page_100.jpg
9e24dde00fecd643c9a130d73720aea3
9bcb10ec1427df135b1c160416004fd8c3487e34
54707 F20101107_AABHMT wei_y_Page_085.jpg
d30c0f7ab56ef409b94faeb17c478b4c
05d608a7449f37a28ade6ef55b739ff7d116cce5
697 F20101107_AABIQL wei_y_Page_086.txt
2da1d8b0c4b55d1ddffa38f9053f83b7
12ab92ad72e7de1645a1a21a8fcb278571fbeb83
666 F20101107_AABIPW wei_y_Page_071.txt
29e1791f12764d6f6df59fc08abcce9e
e38cc80d143f1733617a938ebde1e3e0c49a989d
66336 F20101107_AABHNJ wei_y_Page_101.jpg
3398d769b729490be7f0a1e5cc1ee004
36f1020c3943ed32ac41280f4b39e96d0e41ae34
54046 F20101107_AABHMU wei_y_Page_086.jpg
fe52bc829da2f6cc9226d70b4db6957c
90df5c026cb5da78ddddbfec43fb14f17472ac64
969 F20101107_AABIRA wei_y_Page_101.txt
4d55204f734d906f3e96eba1e2a58d0e
17743ce1682d949169cf3e6f305f0046d0362f5a
883 F20101107_AABIQM wei_y_Page_087.txt
6d6ffc57b71866a8cb3b6ffcce5fbd16
a60b41827cdb6934e377f0f51b594b90b2c8faf9
600 F20101107_AABIPX wei_y_Page_072.txt
fbd6d1f8ec0ec1d8ab371e8667ffa219
4153b418dd6527dbf059f7676b16f25dcb9ec5f5
77122 F20101107_AABHNK wei_y_Page_102.jpg
636bba303accd736b7f0992e3aba8391
6c11ef501b33619a648a255e72232c0c99a3f71c
58940 F20101107_AABHMV wei_y_Page_087.jpg
857da1aaf6da4cd590f63468bff9912e
37a6af0f60a2edc712b38ccb0530ebe333511222
954 F20101107_AABIRB wei_y_Page_102.txt
34a31beabd7045279d4b8bdcc74b9be8
4c078c681e3f3132f3a304e81918e7ebadfdd638
672 F20101107_AABIQN wei_y_Page_088.txt
03c94319862f954fcc857f754bab5e8c
728f0ad1d34368d6a3169af7c750bddde3f7b012
742 F20101107_AABIPY wei_y_Page_073.txt
f98b0b8e7e4629c899aa7a26674704d9
713450f6a5beef0421dc90fcb074ea00cd5bc039
72960 F20101107_AABHNL wei_y_Page_103.jpg
aa1a8e4e8986e367b9c43fdefd8922bb
40b3344ea9a74a243db11f073af7eecccd78c690
53867 F20101107_AABHMW wei_y_Page_088.jpg
e3bbbbff5f6d26f84f586a6292e82382
e7b533589d837e568dc1cb26ff99694607a139e3
779 F20101107_AABIRC wei_y_Page_103.txt
7b49b6cd5f651ba28ddcd0081dc982ef
531c78afa6a92072d4f5f7c1d69925719f6745ce
653 F20101107_AABIQO wei_y_Page_089.txt
7237e3422995480b68a08907ba29ce30
30d27f648a49707d51eb4e3fbb32b71180e0deee
652 F20101107_AABIPZ wei_y_Page_074.txt
d82434fba6750d1ee0daff80348006fe
4ee2c6808bd4547094940a4b24d7fd0af8573198
40393 F20101107_AABHOA wei_y_Page_118.jpg
e22cece7d4f2cf1cd0a9f208e61bd2bb
02b1b7fcb8d835fc0cfc94164b24f2925cb73e0e
72512 F20101107_AABHNM wei_y_Page_104.jpg
cb707a65a7d7950c81771d534f5f7de2
5db83124095d7dfbd94a8e70a7a5a977da72b5a9
53018 F20101107_AABHMX wei_y_Page_089.jpg
366a8c1de6091a044747d3b858022625
688ff3549e607e5a681811058d3a936d91945c68
794 F20101107_AABIRD wei_y_Page_104.txt
a0c4e8feadf3ec5c9ab7990535c3a568
fdb346ce1053adf8dd7309502fdd35c427907602
905 F20101107_AABIQP wei_y_Page_090.txt
bff024339a8814b7dc4917670e7ac3ea
333cf3713073e4fd679d7515fd67c49134bf058e
46050 F20101107_AABHOB wei_y_Page_119.jpg
0cebf5a1f60dfcfdf6642b185835aa77
e75a37de4d0cedead2e737744a97f1eba98706d6
80375 F20101107_AABHNN wei_y_Page_105.jpg
91eebd79de54ffa11fc0dec160334a88
a47c6da63b5a6f59c02fcf80aecd253e72c4c888
58371 F20101107_AABHMY wei_y_Page_090.jpg
2357d59e1b44331b4987791f845241d9
1b1424cc8cf877b8ab1af2f5719c4fd30591094d
801 F20101107_AABIRE wei_y_Page_105.txt
65c6e49fb7edeb94245b1a0ce568a0a0
9ad4950af45a1e3f093a8924a29c2e97939f2e72
712 F20101107_AABIQQ wei_y_Page_091.txt
b2abb9163878b8125cb6ce1d957bc2b3
d2bc614ebd0824be47a2c92b98e8aa3428c0c13c
36074 F20101107_AABHOC wei_y_Page_120.jpg
3156fd6e01dce00d23c2a1ccca334f36
ae751789ed756e36a2ae1c4b306dba102e4dc508
75025 F20101107_AABHNO wei_y_Page_106.jpg
c283a2e0fcaf3e78136d310cb802b03b
c0d2a2442df0cc7a30e9a1a464b10463595ebc37
53057 F20101107_AABHMZ wei_y_Page_091.jpg
a4d1de4ef7eb3bff9ed7159c62dba57b
d930ce39831b56d1751eba8f6c26c85807e37f7b
946 F20101107_AABIRF wei_y_Page_106.txt
d9f120b36e51d31cd5f719212179916a
ca9084f4fd7af516d3947c7d1519159dd9fe770f
678 F20101107_AABIQR wei_y_Page_092.txt
3626b80db8d1f72bc30c270f2a8745d3
477b9e31f017e2bf0961fb65c2fc551d65becd42
43276 F20101107_AABHOD wei_y_Page_121.jpg
4daa229eb3097250dd51f982fbadaeaa
7a1695afa35a58f1c40d06b70b4874f063de90ca
76954 F20101107_AABHNP wei_y_Page_107.jpg
75ed3dde6e937cd26a971c094b49d266
472a21015eaf0a51f6f676f9f56bcdc144ac80ba
1249 F20101107_AABIRG wei_y_Page_107.txt
c06e19cf6348f1ffe4680b28d1134027
1f95d76fa0f051ff16d05974c6c9cb0958a3667c
1319 F20101107_AABIQS wei_y_Page_093.txt
4a70e14135db08fdbd99a9175ab4b503
655cf71524abefe3ed7ae909295820837e03856c
104659 F20101107_AABHOE wei_y_Page_122.jpg
fb4f435e14e4fd1ec4de26e1466b6710
753f1d7cb3a84c18a147f4489caeb2997d14ad27
78233 F20101107_AABHNQ wei_y_Page_108.jpg
bde371a8472d4ee0361602115af11bfa
487dff48423f3cc3c11713027190c2aef218bf59
784 F20101107_AABIRH wei_y_Page_108.txt
659672e686f6fd56f2641d68eae152a7
a857fcc9f37f99dfc508febf9fbad9ea8ce3e180
981 F20101107_AABIQT wei_y_Page_094.txt
7064b6147802d1c10fec46aada36940f
690d27da4ec165a29d4153d8bb67aadacb9c7bbd
112638 F20101107_AABHOF wei_y_Page_123.jpg
9c729842637815246d9ffd18278cc2c7
84605d37ffe4bd9b90e0031e74809748e686a08e
74810 F20101107_AABHNR wei_y_Page_109.jpg
346c0c68191a4215752fb838acdc14f9
d022867da8e16a07452c39c81e0a04203caa40d9
922 F20101107_AABIRI wei_y_Page_109.txt
ad45f7de726253bd69924f304cdd1e0f
c79dbcacbde760a5eb05a637f1ce907ede45c1cc
116983 F20101107_AABHOG wei_y_Page_124.jpg
2edc0838a80c66b1cbb1baa6e9b0bca7
9f3bf21058b1177efdc5b135716cdde5feadf2c1
913 F20101107_AABIRJ wei_y_Page_110.txt
abad9815a76b21338618a9eef8217b28
fd66bfa1a346f249060f628bf6a8c1834dbaabd9
613 F20101107_AABIQU wei_y_Page_095.txt
1558c29efc4b67f7e0d35dbd9d62a21a
5d3a3f8e1173ec3fd122b37a1abec3598dea52d7
111007 F20101107_AABHOH wei_y_Page_125.jpg
f2b9d7149d2e16d34ee8585b11529176
cef306ddfbffdaebfffece0bf13fe559ff40a0dc
74415 F20101107_AABHNS wei_y_Page_110.jpg
07c79529109687d1e9945b95a4091f46
bc34fe4727ce4b86e8a596f5a6cc068d5eead829
780 F20101107_AABIRK wei_y_Page_111.txt
123bb6abc04bc66e04c906bb340f63cc
f62febac624cc68cd851208c2270654a614448a2
902 F20101107_AABIQV wei_y_Page_096.txt
8c9ec2d0f814805814cfa51a50ebf52e
3caca096e05c465a9639be939a1a1a68ca579215
86846 F20101107_AABHNT wei_y_Page_111.jpg
9e029b80261f6eb95a70cfc624a0439b
832177b36e20ddd575b353de87491b3006c7d228
112825 F20101107_AABHOI wei_y_Page_126.jpg
d60f81fafc36f3cbecd7c2058100120a
118b19a391a088e7353fab5aaab63f9447376703
804 F20101107_AABIRL wei_y_Page_112.txt
e5cecdca295824e0782c0eb625f2e29a
61301c0fde937d17891fd40bbde8fce2ae132b14
716 F20101107_AABIQW wei_y_Page_097.txt
11898de1548f061a8368b05ef5dfd948
91b80ccc011d66b4150cf549b9f8588f80178cc3
86530 F20101107_AABHNU wei_y_Page_112.jpg
cc6d97e7b68b829246294f82bb3b1f0c
723853e46acec166d6c42187bcd7416a85033cef
111972 F20101107_AABHOJ wei_y_Page_127.jpg
e2204fff1ed64e0d001f81ad936c6841
9e134fb15eecccb742ebb0dac555a31afc1dd65a
2226 F20101107_AABISA wei_y_Page_127.txt
8fe0d14134246fc18c9b54cecc568536
066cb94117e265fb0e5d1df0e7e5953966a73eca
1187 F20101107_AABIRM wei_y_Page_113.txt
da79d26066724c54e9bad0816deed982
63c71d26f08c216b3d70d899438150061d2fd97e
863 F20101107_AABIQX wei_y_Page_098.txt
429218dfc44328d5c22cc263ab2ca448
bc009e03b90f0be9381f4d811d7c09156f267cc2
77214 F20101107_AABHNV wei_y_Page_113.jpg
b2fd5cccebcdf73367b4f75ad9bcf16c
78b7bb177ea014bd7bf6bc8335c46232820565ff
110072 F20101107_AABHOK wei_y_Page_128.jpg
66d0709e91012adde5674e4a7277a58f
7e064e836d0fbca0e4c30095a515d05a9b268774
2137 F20101107_AABISB wei_y_Page_128.txt
b171661533e02b95627a533287775869
5f17e979081e5347cf82c9eaef7f9679f0f0f229
869 F20101107_AABIRN wei_y_Page_114.txt
384adabbb22ad36e10bb6c5eba82b04d
e89bf816f1e5716488eef5d9ba37f8a120cfecdf
975 F20101107_AABIQY wei_y_Page_099.txt
a51e130faec586a9d3e3afa2694d88a5
d97ce1a6c72c63a004bf6ce9adab5e52c7ab4ed2
73476 F20101107_AABHNW wei_y_Page_114.jpg
b74a7149f85ff55e981cf429a602ebc6
1154f847b6947d92c937145b3f6007e80856f44c
105371 F20101107_AABHOL wei_y_Page_129.jpg
1a8692ae82a2e6ebf32f85317f656a5a
3eadc41ed5cb3b15d32a2ed513d731ed1675ac40
2069 F20101107_AABISC wei_y_Page_129.txt
b5d6f9381ec2f580de346797760ad065
77ae15edf6e77b0cfb309912617a49aa544e0153
F20101107_AABIRO wei_y_Page_115.txt
d425614fe175a5cb2911e7e922f8f8c7
93c4d4db152f1e8eb1066fa4d8d47d5e0140b44e
994 F20101107_AABIQZ wei_y_Page_100.txt
03635da03bc58240d5af4149e9cf7d92
39081a9b7d7ea6f262a2abad6cd06ef63ee490eb
72161 F20101107_AABHNX wei_y_Page_115.jpg
a8ea627640b6b11a93375e7a4689019b
bfb00e2b5fb667a764e0289e00e8c8d2eeae0c7e
84884 F20101107_AABHPA wei_y_Page_145.jpg
7f45b2f6cfd314206974dc30f3f35d4c
7dce48cdc5dce2ec2ebcb706964534626261c5ba
104641 F20101107_AABHOM wei_y_Page_130.jpg
b09d2cc81dce61fa83c63fb31be62fb5
4f613252f6613c752e367103253cfc2891598efb
2076 F20101107_AABISD wei_y_Page_130.txt
321c0da0da80db42ad0fead955719e78
2e964910a19d5491222a4f4353babfbcd4a8850d
1439 F20101107_AABIRP wei_y_Page_116.txt
5e6b8fb18330fe55192ce7a5e8e84261
10c2349a1df4ed8e828aee9d427315c5d55b48a5
40288 F20101107_AABHNY wei_y_Page_116.jpg
09590136a7add1feba00d98153f04202
89458bc96f1176da671b583beec382c94d956ec7
83739 F20101107_AABHPB wei_y_Page_146.jpg
864bb061a02409f4c1f929a8d9daf39d
3aea44aa0aca6fb8682bce19b295c2efadd6a592
111561 F20101107_AABHON wei_y_Page_131.jpg
8896bfbb1a2e3d290f1ad80d751924b6
036161998f07f0c21a41f9eeab59e67220645c43
2218 F20101107_AABISE wei_y_Page_131.txt
f555b2182cc78c336a77921903027d7d
c705c721a7d64eaffdb08c52f0510f4aa2561a22
1255 F20101107_AABIRQ wei_y_Page_117.txt
79d64b84a9903c605f84cc32e6dbc613
49328aed49c1b6f3b44d7ecdca9f5069f6cf6623
45967 F20101107_AABHNZ wei_y_Page_117.jpg
0e87b5abe5c85981bf6287c12f2ad2e2
04ecb2947a02ac15c686bf4c54ccc22e2547e297
83781 F20101107_AABHPC wei_y_Page_147.jpg
d21e8ed262b5631d43da5a6e433cff10
9b441a3412637ac6133b3967e42f6965650fb1de
114260 F20101107_AABHOO wei_y_Page_133.jpg
49f8e90d29ed31e346b196cd04d48072
ac22fbbb183fe82382ed745edde061e3a63f7159
437 F20101107_AABISF wei_y_Page_132.txt
a70ebb2ea0d0251e680e344e120c3e13
8d62dc1604d4cd95fa5799d9c574295e9d3bc49d
1184 F20101107_AABIRR wei_y_Page_118.txt
10286317ccfd1ae249071289ef4e46d8
e00c5788b9425b73661c4f2d2b8bf8c7d91dd107
84116 F20101107_AABHPD wei_y_Page_148.jpg
544f0a313ef5c3ca72dfa0fc6f5febfc
bab47e65ff4815aa20d983e073a92d09dd25df7c
119279 F20101107_AABHOP wei_y_Page_134.jpg
9d949043c18fff9231e9659bfb8c4ac3
8c01f253c6015efc94084b8fba44c3c2d63b4b90
2480 F20101107_AABISG wei_y_Page_133.txt
b655fcd97251603d51d7a004d804f4ea
10dbc82472667a47f8177bf6673047db1159d748
1320 F20101107_AABIRS wei_y_Page_119.txt
4f9045dbf715b7af5fd946ca1ba73857
a24433c9f9f1ecbbdabc138d10cbfd833a95bb58
84080 F20101107_AABHPE wei_y_Page_149.jpg
696403c2fdfd84ce7e5e6de9ef2d4243
02c04d1b0be2e8ee00a723f61e002424629d9bd2
107291 F20101107_AABHOQ wei_y_Page_135.jpg
d1cd6be40988a81f05046dc588cf8a88
ceb573c4bdd7afb72276d6382fb7e7b4c7e97cb6
2544 F20101107_AABISH wei_y_Page_134.txt
7d0c8869bb65f258c8e3a9482b7dc199
2fed9217ae865bc4ff52d5d4781ecb1e50674836
F20101107_AABIRT wei_y_Page_120.txt
8f51b4ce996a6f6b5c4fc27b1ca456b6
ba187906748f7fd0eff1c89bcdf06fed461e3a9e
84888 F20101107_AABHPF wei_y_Page_150.jpg
2150b5f0dd61dd6a72f1904b2e9537cd
8de3d6a7c58a09be927d59236bba095ce369cda1
112450 F20101107_AABHOR wei_y_Page_136.jpg
e693dc2f82287d0f3f3d6ef6306f810b
3611f572020fea419b670d0e2a497006288b5f7c
2092 F20101107_AABISI wei_y_Page_135.txt
435f06e58e28e9092e420299a94d5eb6
c693dc83f26bb327dccd70ab379ece60fe11dd02
1568 F20101107_AABIRU wei_y_Page_121.txt
603e21eccfac3324de6521f3aa811af8
01e2681f73c8f1efa6ec6043933195a5aedd2b5d
86849 F20101107_AABHPG wei_y_Page_151.jpg
5cc9fa8bbc77809a5f5d6d9b97758ece
8b9b8ed1673f26056f8fd4327541f5c7790474c2
18424 F20101107_AABHOS wei_y_Page_137.jpg
42277810eea190634c87d25764de8d83
d879842706661652f501f1e0c6fdcf44beb35ca9
2189 F20101107_AABISJ wei_y_Page_136.txt
82ac07756d99a5a12bf2e9e412d0ec46
7d45f49d15315fb7b9c46c06b25c53063ffbe090
85044 F20101107_AABHPH wei_y_Page_152.jpg
513f0bda26ceb5c275fd720b4e90d81a
a495ca2d9f1e98b3a74733841fad226435869fbd
309 F20101107_AABISK wei_y_Page_137.txt
0333faa129241a8e8eb68009e8561ee0
681d3f89fd6113b170179d12aa142973a6bfecbe
F20101107_AABIRV wei_y_Page_122.txt
093d2823f1e8202e5337ffd7f161cb30
fc7b6bf0f3b6c89792824bf8b5ad290fe71bc587
85545 F20101107_AABHPI wei_y_Page_153.jpg
98b65478478fc6bf4770fad1aedd7011
6a753f3991d1305bda80f53e027e4fe7ba9d53c2
88915 F20101107_AABHOT wei_y_Page_138.jpg
2dffbf7841404cda3c24f133a6d80280
7ceed38c8701efffa0cd174d083e8ce9902f222b
1256 F20101107_AABISL wei_y_Page_138.txt
5cc00712d9aa4f330db7c8f776bb22e2
fdcfb79d81168d5dc112128c65733daf294071c1
2216 F20101107_AABIRW wei_y_Page_123.txt
411d0e5bda9cd512a8c24021860eea81
c627f78ffbe1e8a5595a7f5ef496784024fe8373
84001 F20101107_AABHPJ wei_y_Page_154.jpg
28b01ad413c1cffaad70cb35117307c8
656d9a81e51e69457e1e1b4b39d2584002469a91
85906 F20101107_AABHOU wei_y_Page_139.jpg
be4efa37adc41afe49be0cf0e7a0ddc8
5d06a29b87eaee4261e89a2d71b0cdec99e2854a
925 F20101107_AABITA wei_y_Page_153.txt
a0ac30c741f1a9123e76fa58f771f995
7772b1f2e39925affdd9e823ef10fe3be7386cee
865 F20101107_AABISM wei_y_Page_139.txt
e5932a13ee79622243d56df18786ffec
39b79fd0a4b2207c3666c78fb8a485c903144a8e
2302 F20101107_AABIRX wei_y_Page_124.txt
f522c5ab93d5179b8e15eecd8367a973
8dea30b2f47ec02686447a8ca4be9b87a9d27d1e
84825 F20101107_AABHPK wei_y_Page_155.jpg
2ea4c35af25001d09b79dde40bd6d92b
236e25f28245a566ad5883e5103096432c855974
86839 F20101107_AABHOV wei_y_Page_140.jpg
62ad83da3ba229577a7987cd418dfc50
f0cf99585a5b7bec22b451d4032e722d969d4509
836 F20101107_AABITB wei_y_Page_154.txt
642890cabcaf4dc9aba2652364eda080
a5450651cf1ea0eda1670512181c07c68121f93d
829 F20101107_AABISN wei_y_Page_140.txt
e501e0dca71c3ca795af2694f1ea41bd
02d3f3adcbee8fd1fceb01d1f3a863f1bcd5c519
2159 F20101107_AABIRY wei_y_Page_125.txt
d1a3d1c41b1491328f6fc04b5f326b78
9a8da46911fc596800870101f7ff659ce9ec5f2a
83848 F20101107_AABHPL wei_y_Page_156.jpg
0adc3116ca473edea7138cb23d140a3a
161699d456ba2e2d224cd562e3f21a48ffae8408
86604 F20101107_AABHOW wei_y_Page_141.jpg
dac146669244afccea685b68a90df2d0
43a36de2e0f38f4f0ddde5d1e60b459d738dcba9
862 F20101107_AABITC wei_y_Page_155.txt
9e3e20bd91bd8bd9191a3e4953fb084b
bdeec14106f33566d60e9deaafefd909515018ce
828 F20101107_AABISO wei_y_Page_141.txt
f35db62b5593c3ef84dbdaa48cc5c1ec
6530ab8baa8e883bf5ccff463cddb887ae794a5c
2190 F20101107_AABIRZ wei_y_Page_126.txt
dd0f78a7f7665d3bcf4715d53a3b1844
8ace42369bb1740c93aad8b1e52f7b9ee269f0ff
85453 F20101107_AABHQA wei_y_Page_171.jpg
420a860199f0405ade03fc8c479a6634
35d4e5f378ff9388616cb881abae16c3ce678c94
84186 F20101107_AABHPM wei_y_Page_157.jpg
3ab648435d47e8db33c6183b3c08aaaf
80ec837f7378344e345f3a76cb6d412853590203
84449 F20101107_AABHOX wei_y_Page_142.jpg
16505491b73c1c56c1b25842cbb1bd54
0b772d552664ee9ff3a7aa51856da51d63bc2553
903 F20101107_AABITD wei_y_Page_156.txt
ce414d78efe49018fba49356c7a4e9be
4b2ad0ec50c1b2a8bc9ccdb20865afe6c5b46faf
856 F20101107_AABISP wei_y_Page_142.txt
8c2fe0d7a5c12c9f4e90aad77fc73fe6
e2c96480a2fb6711da7d47c4b9aa46afa0857996
85209 F20101107_AABHQB wei_y_Page_172.jpg
66f10b0d9adaea831f320a0b9adc02e3
f6e8fc9db3d58fd46f235c62158068aa1a0d30bb
83200 F20101107_AABHPN wei_y_Page_158.jpg
312bd4d36abbcb1901b4ddfe0349d7a8
24eb7a315f7e78da05e7fe1981f60938a3a08fce
83719 F20101107_AABHOY wei_y_Page_143.jpg
5cbcde946dd88838af97d146199b27dc
93d6c8862de466a7947d7be35d3ba57f955e61a2
835 F20101107_AABITE wei_y_Page_157.txt
41e886f0a2ecc96e31c66f4a64645bf8
01bd60180328454db8ef0095b7de36844ce5c959
1134 F20101107_AABISQ wei_y_Page_143.txt
dde764437b2d84ad837c6ea800aac62a
405729edfbdc24550be7afcd4bfb9407337925a7
85509 F20101107_AABHQC wei_y_Page_173.jpg
b92b7a991c410772096de2b97d18e622
fd0244cd5b5f2c5a4dda24fbdbd4538635b69532
83511 F20101107_AABHPO wei_y_Page_159.jpg
b89f4954cc44830761a1243d7260f4b1
1d305ff19c70443d8b7b35eb82ab3a5296c0eb41
85194 F20101107_AABHOZ wei_y_Page_144.jpg
be3b52b035f886054fc7ec681867495a
066571c4bdcb438c586845319f1170964ee2a978
876 F20101107_AABITF wei_y_Page_158.txt
ccb8c6ea0632943d5939d9813248f07b
44933bed6e362d6861e14fb79593c12a01e3dfbb
1028 F20101107_AABISR wei_y_Page_144.txt
88a970958c6807584b4dd3977b33431f
715e7087dd735729e557b721fe6757ca3c5e32f1
87596 F20101107_AABHQD wei_y_Page_174.jpg
7b962255856b8175bd67418fadeeabd8
427c1e74e34fcfa2e7a7e85cb3badc05c102f578
83347 F20101107_AABHPP wei_y_Page_160.jpg
96541863d6cb21cfbff35af7f6878435
cd4a033fbba37877ebcf675b5938ca32da562f15
955 F20101107_AABITG wei_y_Page_159.txt
ffc41b6a0feca7584448638d360f4f53
542d9f2d487b19fefd7fd2f278442420db4da604
751 F20101107_AABISS wei_y_Page_145.txt
04dcd74f0dd22c7a7f91cc28e403cea7
c42570dc83e03f576a1e40b29c8d270ec36aaec2
87294 F20101107_AABHQE wei_y_Page_175.jpg
909f875c77a2539276cb700886f42d6b
eaee619c56def26798cd9a0b8993eb22a47a823c
83573 F20101107_AABHPQ wei_y_Page_161.jpg
f36b5d113b967fa76c9375fbdd2cca3e
23f2ffbebcf1f7c0a553953d6758dc90b1ff662d
870 F20101107_AABITH wei_y_Page_160.txt
09c8ad7c9049770464840a2f8976653b
1fff1608f600afa79d28116dd25ca8f8a4a27ec7
1030 F20101107_AABIST wei_y_Page_146.txt
96f56a3ecd5039e0798c76a5c7df7e7c
f9f59e822c09b9c0204b4743fe032000126ae10d
87097 F20101107_AABHQF wei_y_Page_176.jpg
055220816a970a771bca5c2d3a6509c9
30558635eec302f563ceb5dc47903fc39ec76f34
87682 F20101107_AABHPR wei_y_Page_162.jpg
a7cb8874fa64b705019546406113c6b5
0d1af334cfb1c70b759b5caa0877f1221cb05314
1032 F20101107_AABITI wei_y_Page_161.txt
48009d9efa19ff6f284cbd4b0bf673ed
56be387fbfa0191ff759ef4dcd2db9fc1235c053
858 F20101107_AABISU wei_y_Page_147.txt
77d9eb5a35d0a000dcd58b709cb21efd
703fdd882cd2b53b49fdbf5220c5ee4eedb5c2ea
87106 F20101107_AABHQG wei_y_Page_177.jpg
8421312606cb0136035050dd308befc7
ed31758c4a528af824134ea8276aed3228bbf343
87888 F20101107_AABHPS wei_y_Page_163.jpg
8f61af8727322b8ddf6ac2e020c64494
bb47d509c059643169cda5d9586c62d2d0a1f468
827 F20101107_AABITJ wei_y_Page_162.txt
d0275a12717d8ff0b965e22bea79613d
eaa2e682a55ddc0be3d1787aa89f5621440b21c2
745 F20101107_AABISV wei_y_Page_148.txt
8530924ce26f94ecb71b732de9b5ce37
4e4773fdd8073b9fcd85f9b16caa2eca3b78ddbe
85794 F20101107_AABHQH wei_y_Page_178.jpg
959d2e0de963339bb81aa6030fd0843d
93ae281fbd19651edb118093dda2fc3f9ad6c475
87405 F20101107_AABHPT wei_y_Page_164.jpg
2c10c7ce241f66181b692fd5fab4cb39
9b8dae900f7990e90df3302a761006bd40f8905e
F20101107_AABITK wei_y_Page_163.txt
652da84b6e16fc042e8ddf480112c837
50c75a2cbbe4197df72dcacf7818ff1ffd585bad
85954 F20101107_AABHQI wei_y_Page_179.jpg
3e7be00bb42c8d118b853ee40b9942be
94fd4f286a311d12799e4eef12e1e6293d5e6120
677 F20101107_AABITL wei_y_Page_164.txt
1df546e619ebaf7b3f26fd19b3c00e0a
079321db4c2a6099c1c34bb9d97c69840a5d05cd
F20101107_AABISW wei_y_Page_149.txt
3044aeda59350aa4be8c6b274920bf43
a73603525a753a37ecf19be3f10de6d92c3351be
85327 F20101107_AABHQJ wei_y_Page_180.jpg
9d7f4c6cc4b11f05d806750c1d9bf4c2
ece16ca43e1c261afdf37681b45beff33933b801
87750 F20101107_AABHPU wei_y_Page_165.jpg
58d4a26e81b98e4cdbee9fc01c7cb94a
b6769a68bad3a85c601fce26f23a4209e764de7c
F20101107_AABIUA wei_y_Page_179.txt
d0cb0a8fa2369c7e2228268846684a63
3ff46898c5f376dd3930c2b1126a26dcee2d76b2
F20101107_AABITM wei_y_Page_165.txt
3bef15807df7be39e958877bbcca6d27
14fb0df07b0c83af208b893fcc3844c68fa4b4f0
899 F20101107_AABISX wei_y_Page_150.txt
3b46900c31672dcb20c68af540d5012e
aa941a6fa8b1f8dfa508ad8632229abf6c5af33a
85013 F20101107_AABHQK wei_y_Page_181.jpg
a4c4b6bf51e993cf4fc984c98eb5f49c
e12c5d05f23c14eaab27a7b3a97c8011abfa09d1
85851 F20101107_AABHPV wei_y_Page_166.jpg
5956cdf1f903a60f247425fa3769b1d3
88feb8846e2b6e993c2c1671bdb430970abcb4a2
686 F20101107_AABIUB wei_y_Page_180.txt
71d228701d43eac624d55e9f05814672
2b60e5e75c35e7c0b9a73a25d4addb690afb7ba8
1698 F20101107_AABITN wei_y_Page_166.txt
ff3fc455f4d7ff3591506cc5c06b732c
449d32b254d3651a6971941818910d06e5ff320f
1041 F20101107_AABISY wei_y_Page_151.txt
7779fd768962d581047ce3442b05ec86
08f76005b6f2c62c9e2f2a5f3ef6e121cba046a3
84774 F20101107_AABHQL wei_y_Page_182.jpg
b6157f5850a6be616835f9670761931b
db595716e7d095108f34a37251d1ac4c56e8af4d
86407 F20101107_AABHPW wei_y_Page_167.jpg
8ec3f9eea8bc01a6e78cbeed89d49c46
c41bde0b7a749221290e16fc932c13f104c8ba55
807 F20101107_AABIUC wei_y_Page_181.txt
919e005d403eef16c8b9836e6fec4354
0eaf44d7b6265a2c0acec551e070ad19b5775429
826 F20101107_AABITO wei_y_Page_167.txt
6f55e909ab270386e5281391d19cd5a8
1d59b7bae8067cd09de5614a43abc6ec6b145dd5
889 F20101107_AABISZ wei_y_Page_152.txt
90c1a9bd4456d4810e063c06a59d6769
49f6dcce1de342a0a43c2de8ca81fd6f87f99948
84879 F20101107_AABHQM wei_y_Page_183.jpg
1d32bf1ece3c898a2e04834ec34abf25
b4d12c598baf55184947795a54708e203f3a9328
86517 F20101107_AABHPX wei_y_Page_168.jpg
5c255763735e4463ce00617ac64c7b07
8e1ea665bf1d5292b022fdb261c0b1d763bd7cff
1051983 F20101107_AABHRA wei_y_Page_004.jp2
f9225a3c7128bb0fe1630d3aed5fac60
30d28f3695202822bfc92f80438cb5cc5f82da57
703 F20101107_AABIUD wei_y_Page_182.txt
0e1d9c3457542e2776adb1f9d024e393
3fd4969a933343888379c07cc4c36d1a593c7ec9
832 F20101107_AABITP wei_y_Page_168.txt
d1c8c1e6d1b97463c109d8a22d8491c1
16cb7020230cafd40fe6572b53101b6de24eba71
79085 F20101107_AABHQN wei_y_Page_184.jpg
dac9e3219ceebcc6e91dbc314bde8309
1d8e89a3f492e212d1bd987e0a1b0d5029ffa550
86314 F20101107_AABHPY wei_y_Page_169.jpg
08bc3bb4c4b633ae21515670dda123f0
b66f8cd9ba9da4ef17ed6451215cf0215c8e6bac
1051978 F20101107_AABHRB wei_y_Page_005.jp2
1f00a5571ef920ed678203fe1b84343e
965e2a892fc02f8240245d7feb391fd1c2d1d787
741 F20101107_AABIUE wei_y_Page_183.txt
b6b6669ceb055e69c6c1732808308e23
589aca27a9fd9c0af4b361fe75cd636be17981d8
673 F20101107_AABITQ wei_y_Page_169.txt
817c38dc2f40e07f4d84dc9f206e72c4
f9c6c76af19ac6316f68572bf74ef75218d87750
85367 F20101107_AABHQO wei_y_Page_185.jpg
42938378eb89f76908be885b536eab57
64d7265c0384237cebb37d9c7f08f15a94dc09bd
85333 F20101107_AABHPZ wei_y_Page_170.jpg
d6e41747a59dd1c8e1ec1e7bbccca388
4439dcfc604dee16182e1f636382a47a11407b0f
519907 F20101107_AABHRC wei_y_Page_006.jp2
859c0aa5442e52109afa199fa57b33bc
b7ea501df902ce3aded402b33c6e81e5500f7b9b
9119 F20101107_AABJAA wei_y_Page_045thm.jpg
9ced25251ba645e995490ee0696de47d
e08a5fbac8bcdc1e126da830a8c679d278fa93d8
852 F20101107_AABIUF wei_y_Page_184.txt
414e573d2c43f9ac559c6f5d9b3530e0
8a3ae57c94c1fbb99f85fa0b628f1b2b11abe9bf
F20101107_AABITR wei_y_Page_170.txt
be09590c1a0763a979e3337f4d4cc0b1
6e064f0f9742d01dcc5e1b865472cfdac24dab5c
122570 F20101107_AABHQP wei_y_Page_186.jpg
ba62dba549b40ba9c59f8d3b2c390603
6e16aeffdd75e74b2cca6029e2f5058233726923
1051980 F20101107_AABHRD wei_y_Page_007.jp2
2f6c9a32c9da38b6381cfb4e6eaf4951
f9c14bdcf9d27aa2a741d2d98afabf1b448c63db
36893 F20101107_AABJAB wei_y_Page_046.QC.jpg
a13ca8d5262fa5a5bbe4321fd1732f87
4f99ca0344fbd17f4859e1d3afeb5ad1a8f378bc
713 F20101107_AABIUG wei_y_Page_185.txt
8347e432c21f440f08d4212ef9921025
e1be4f6253ae612d0d50015400bceb2c728d473e
682 F20101107_AABITS wei_y_Page_171.txt
84af2faa22e26a1c3902800df25f3dc7
ac2cad8ce2e6ef14c84984f33b5240b107638da6
131461 F20101107_AABHQQ wei_y_Page_187.jpg
6fea5f6794d51817afb5cb6e9eb70a70
0c296a8c6a87e96a046180fbd76aa31324448a46
137722 F20101107_AABHRE wei_y_Page_008.jp2
b13b26ec5436f08e4d52ca0de024784c
802bf129bd184f29d58198ff5cbb1b0fadd6078e
9124 F20101107_AABJAC wei_y_Page_046thm.jpg
75d8677c0704747ad972f2512aa8de31
cfb2f6953997ce154875c949f6cc1dbb7f22ca48
2453 F20101107_AABIUH wei_y_Page_186.txt
911e4157712f3effeb816a9359044bd9
3df420a8de7df1f8050297b1eb82afc372713c9d
646 F20101107_AABITT wei_y_Page_172.txt
b96a10eb559e565670b0f419ec58bebd
dade81ce43821276f7ef106236a0ddaf58d1f42f
124900 F20101107_AABHQR wei_y_Page_188.jpg
40a3cd49f06ac0cf627d0321700e325c
b74c31c532f9224aeec4cccb0bacc56e77b99a70
983332 F20101107_AABHRF wei_y_Page_009.jp2
b4f02cea683bbcc1a204687db4aab07f
8aa301ed04c5fd378ad3bac80a75e434dfa4537c
2625 F20101107_AABIUI wei_y_Page_187.txt
e25e4694143efb1d7778f2675ca51a3e
8625d417ff2f0077869d0093bea2d7ddd62c4986
760 F20101107_AABITU wei_y_Page_173.txt
79031d00b8657f9c73b8b773f7c449e5
d0f1da1025cf7df556f002126357c063025bc02f
123067 F20101107_AABHQS wei_y_Page_189.jpg
9c8740f27f741aa86da507cfb64868f7
845945d0d8656ddb7883229e0d151fc06c1125cd
785677 F20101107_AABHRG wei_y_Page_010.jp2
0203923baf2467a14a3b1df63ca9bf6d
5b4829f25fd6f64eda118b33dbd661228020a0df
34445 F20101107_AABJAD wei_y_Page_047.QC.jpg
0bff13689fcc884ef986984fc37bea3f
914ecf0f8a76021b74fc4172fa1e8963d874f4ae
2465 F20101107_AABIUJ wei_y_Page_188.txt
45d5b322c4bb8536898980a8f2da9156
f829f58a854d517dda91bfd9df83e903ba0437a0
926 F20101107_AABITV wei_y_Page_174.txt
ff5ac9d71c4d472698d51729051b8a98
ab3a05917d934eb4e12f0a6ec7a468d25caab5fa
116770 F20101107_AABHQT wei_y_Page_190.jpg
2fc995a763170282447a508da8830e24
a4f5a468da1232d56485f2580d111f8bb3c6e0c1
F20101107_AABHRH wei_y_Page_011.jp2
c6c5155c22985fdc203f2d149cb55401
e57b4a319e76dd96955cdfa2e9dd9f782d0ac867
8622 F20101107_AABJAE wei_y_Page_047thm.jpg
d0751c1c8a102d6e241e109a78f6132e
246ea06c5a952bdffb5e1d5010ffd4f0b6eda73f
2493 F20101107_AABIUK wei_y_Page_189.txt
74d08687d880819bf1bd3e07e319e78a
9211b1c928d37ccc8323cc1611be2ad36cda4bff
729 F20101107_AABITW wei_y_Page_175.txt
75e5f2844e97a83ee5f647110e79c39c
cd3a5ff71eae51788f692fcd0cdde3118e5d5ca1
129725 F20101107_AABHQU wei_y_Page_191.jpg
37f2b5ef8f1f9ba50f09f797a705d183
2bc1b963fbf025fba81c26281b13bd132c846d85
1051939 F20101107_AABHRI wei_y_Page_012.jp2
3be923df10ca748bd863a9ec9232bdc3
5fcbf923b4b1fa6c3985531084e1aa0318e20233
33600 F20101107_AABJAF wei_y_Page_048.QC.jpg
756f26ccd5dc5820e044dafc28075dfc
b18912549b54f5edf0b186a1fa3c0e4d9e7c8c64
2422 F20101107_AABIUL wei_y_Page_190.txt
163e82f29ac1b68b76f34cecc17999d1
a1baa4f03d51873fcc9a0171edc88be06861ef5d
694844 F20101107_AABHRJ wei_y_Page_013.jp2
8adc76d95d49730efae68f02e8f9f0c8
f3467ac491d871ec9d319f1046e36a38972df201
8075 F20101107_AABJAG wei_y_Page_048thm.jpg
a7cd5840e2340bf2085d46b7da91edf9
41ff1695eba457470f32d4d49aa57ed6cb641aff
9739 F20101107_AABIVA wei_y_Page_142thm.jpg
c30cc1a1bfd20058a5dcf9b471c09b44
68d8d2aeb0fc09d588b2f81abb83539eab48b42a
2490 F20101107_AABIUM wei_y_Page_191.txt
9b7161014abb101e192e95559610ee49
795f71408298d710b62bfd7d26ceeb54df828519
F20101107_AABITX wei_y_Page_176.txt
d4c275de32cdd92c002ee5f3d86a1221
cb52168704434c591cccc963ef9b46b9b5a8e72d
75931 F20101107_AABHQV wei_y_Page_192.jpg
e10cbf723d914f96cf05e0ce3f18edc6
a110314b2bef8669f1fc8d2443fc4488a22a1668
666516 F20101107_AABHRK wei_y_Page_014.jp2
e855eecb3ae9da7612b9f048cf4d82e3
416a9686fffe914b0520c1adcbc7932d2f1e8f55
15322 F20101107_AABJAH wei_y_Page_049.QC.jpg
2db2b231b6e41085ed56511c33998655
ce607cf3517bd3e14d262bfb751c56df15fdfa47
1795 F20101107_AABIVB wei_y_Page_137thm.jpg
c30809b6e7b8e785aeaf4382ccf012f4
3fc3cf4fb0056def364389f4b4d796c5ded871f6
1402 F20101107_AABIUN wei_y_Page_192.txt
1f39d0e6eb120d247492fe6659b5fd93
27710ed5c74549710c73eaff6d2faab74985671d
F20101107_AABITY wei_y_Page_177.txt
f2741a252fa17fd13dac698c27a27a75
7b3ded23751f30e64f6d7cbcd99325626d50ef77
43989 F20101107_AABHQW wei_y_Page_193.jpg
769855eca5c4d279e23b52d07bd0d4e6
8233ae3dd070e9ad1d2d18a366f054d72aaebd61
560383 F20101107_AABHRL wei_y_Page_015.jp2
cb5ecb77a962a1b1b6d8e6c3dce058d9
22a1bba229e25952c510911215c8f5b3eb579f4f
4681 F20101107_AABJAI wei_y_Page_049thm.jpg
eff070b699151a7771af436909c66dcc
4f99fcf09f3afe7f674a0fdb4165f6b02f7002cb
9715 F20101107_AABIVC wei_y_Page_185thm.jpg
09fbcb3a6ec0fbcff3ec2e61a95d30ac
491c757fe013f1a2b6a31b3ee82193074d9c470e
788 F20101107_AABIUO wei_y_Page_193.txt
aff475d95e2ec5333f46b9712588ab8d
584cdf88b40e70b7176cade0191e214a73e689c4
F20101107_AABITZ wei_y_Page_178.txt
dab0455a9c819a5c55461ba434e64bc7
e613a82c8c6b40fc806fdb595c8c07c5a03c55e1
248761 F20101107_AABHQX wei_y_Page_001.jp2
ca957400f32c430dc5bc96870cb3274f
396a9a2a1ddee7d3c9b0c7c870b087d5909f9407
1051967 F20101107_AABHSA wei_y_Page_030.jp2
7c262fe6568b4c10afe6f1db09d1189f
c621c9d6f6a737b613e7b30f625e0ffda931f7ee
909873 F20101107_AABHRM wei_y_Page_016.jp2
1047b403a456e3682fd89995f40d2c50
037f48d137f812524c9a764556fc4db7a4d8c84f
31623 F20101107_AABJAJ wei_y_Page_050.QC.jpg
09ae25fb16db299d50d66c7720148454
160fb4e5731cc0a891b866fb167ae55988a5bdb3
9688 F20101107_AABIVD wei_y_Page_157thm.jpg
84c40706798d6aa9bc57170965e861de
a7d2c96cca1d70d93c9f06c9c6500345370cde63
1985 F20101107_AABIUP wei_y_Page_001thm.jpg
e7eafa3c42fe2ea3eee27d7071eb1040
221b336534d7890bbd4d2469b38b33e240586d03
22843 F20101107_AABHQY wei_y_Page_002.jp2
8794ca15e60c6d573997439686653c3f
eea1a3c35edac5a55b1ce02c9f8b240d453f50e1
652833 F20101107_AABHSB wei_y_Page_031.jp2
bb29b7e5e84c628027eb25ac82e83051
73bc9309ac577c0f1319be0ed0bd0f49650343c2
1018880 F20101107_AABHRN wei_y_Page_017.jp2
3f2287080d8cb7d4d41759870af789b7
c99138b655421a181fab55632adeed9c3d990c55
7835 F20101107_AABJAK wei_y_Page_050thm.jpg
098735bed40e771d9986934aab830cd3
72e6152481c69ea80c93686eb8933a5ecf514308
9627 F20101107_AABIVE wei_y_Page_108thm.jpg
baf1b9aeb8e1a0ad572651f2f1d47653
9005e385c975b2a67c384b9442748035ab03f742
4411890 F20101107_AABIUQ wei_y.pdf
bd68cac58c4726daee6288977bcfc556
4c9fb4c464e3e47d379daf78d2c4869e0abc2068
308460 F20101107_AABHQZ wei_y_Page_003.jp2
7d2ece8f88efe50eac3bc7a018e6e50c
bb4cef96a982e88beb39c1f685cc4bb6171f25cb
933001 F20101107_AABHSC wei_y_Page_032.jp2
46a019aa8cc2f4ebb4f84d512b890c36
42913e3a29dd141f86b1f318357ed0b68586b475
910056 F20101107_AABHRO wei_y_Page_018.jp2
bb9e15e307e7219f6b667c90445f215b
3ea712b95ac9045fa6bacdc76f79cf559f893bf1
31266 F20101107_AABJBA wei_y_Page_059.QC.jpg
ba55a78c5c06c848b67272ce70451ee6
78ff555541746f26c7b024af4de26575eebd77a3
35994 F20101107_AABJAL wei_y_Page_051.QC.jpg
9b5760dd75f3505baea8326e8107398b
56cf71adc45216da04259c4e3dee91193939cedc
5856 F20101107_AABIVF wei_y_Page_028thm.jpg
3f73497adb28cf7fcff56b81e70b8bce
ae934b4d36d8a6e6d2e30e920cb98c5f7d04e525
9935 F20101107_AABIUR wei_y_Page_151thm.jpg
57c98066ebc8e3cf37e1ba4c463b4a01
2b5130c6b7cb9ac62548f655a45200b2e993daf3
914365 F20101107_AABHSD wei_y_Page_033.jp2
821c3488bd160531e3a03212ee0b4093
fff43d25e714ca0cb0df2b352afd257d4628ab16
1024672 F20101107_AABHRP wei_y_Page_019.jp2
499e0c71f111d5c3cd339f6d4172ff8a
6205fffd82e47adf40df43090668a73b2af59de6
36228 F20101107_AABJBB wei_y_Page_060.QC.jpg
900c552edb15868c82a21076e2ab2c5e
974a7ba525fcf60f09dacbb996e266cd2640dc36
8450 F20101107_AABJAM wei_y_Page_051thm.jpg
e43998e7927099b54bc61518c8f2b63e
a95fa9c602d8a3895eb225055b8598b52e1d2fec
20390 F20101107_AABIVG wei_y_Page_076.QC.jpg
1ed2f0593316940a459bc21d6f90e3ff
7503ca439d2597a1c55958bb1337e506041f67e3
21054 F20101107_AABIUS wei_y_Page_088.QC.jpg
f0bc7908365737711fcc43c65ed3658f
00b87bd975daa39a904ead483570c9dcba063341
627839 F20101107_AABHSE wei_y_Page_034.jp2
04fdac0da0db8f1686b9d03cf0e8a2bf
bcb30ca8091d2cda03e72f381b9a2ca458424be3
1019919 F20101107_AABHRQ wei_y_Page_020.jp2
c1d893ac54660dcb4fdd696f68ef6695
c4591ed2d50d9659198eebc8d0d1a1ed3a098dcd
8992 F20101107_AABJBC wei_y_Page_060thm.jpg
d45ff0ef839dfccb8ff411c1b2a7cc1f
c7285faf65004d02b462740c83fef6c37b759d38
35948 F20101107_AABJAN wei_y_Page_052.QC.jpg
2c27aa5afdc51b320ed1dc6b025d80b3
c0711e85cf0b569560ffe9ecdc6056b89393d2f5
9247 F20101107_AABIVH wei_y_Page_184thm.jpg
40f9d9bfed1314895c9eca1e0e3358f5
cce905b5922ea3bd1034ca3a01a4ad91ff972503
36070 F20101107_AABIUT wei_y_Page_191.QC.jpg
34f33cb8a332f4ff3eaa1bc37f98e814
1ac3be242aac708d5431fd123c5604cf868ec625
764629 F20101107_AABHSF wei_y_Page_035.jp2
a818dd149641d46cb3ce9b14a9819e4b
c5da63bda49469851cd6ac74ba4b0656394f9433
528099 F20101107_AABHRR wei_y_Page_021.jp2
67f29c6b6dd78bd5a73afb5630d886ca
781aa6fed174000ed5c3885424621f1240ba1b4a
36208 F20101107_AABJBD wei_y_Page_061.QC.jpg
c59919938085bb1fee5c6671f7000c57
efcff4046149bd5152b5496dcbabc7ea9dbd5287
8270 F20101107_AABJAO wei_y_Page_052thm.jpg
ab18d79cabea8adad2f23b163c3d5806
cf5a227e1dee57a02ab4527ed6dc4eeff5a12daf
9116 F20101107_AABIVI wei_y_Page_110thm.jpg
561742a417899d8238dbde76aa5b8bf5
8eaa960bd3bf85221c3326b95637de7d0f076400
9886 F20101107_AABIUU wei_y_Page_180thm.jpg
1c05844e7069b0410b82cf53cbfafa68
0cd0677e41b97038659f3cceb7d1d1b064203979
862290 F20101107_AABHSG wei_y_Page_036.jp2
fb3cbbacd7140e729bc819b53c15a345
ed7c37ade38973061efdbdc76aaf6916edc4ed09
832474 F20101107_AABHRS wei_y_Page_022.jp2
1d84144529c05ac2726e92688aba188a
209e46b4193a5e24650f54476aefb21e79c2f3fc
4804 F20101107_AABJAP wei_y_Page_053.QC.jpg
9464f1a4911dc073c8028fcf5d39f820
b068a8143955102b849d96ab7b912a20068a60b1
1115 F20101107_AABIVJ wei_y_Page_002.QC.jpg
198a0d6f6fa9e83ee1b3d58cfc8b538f
ee30a664da43d0815475f494b806724e0410b0b0
10093 F20101107_AABIUV wei_y_Page_139thm.jpg
dabad021365fbfff1bcf1727518f614f
a346da00fdae35d83c5de79d835516396ce458cf
927416 F20101107_AABHSH wei_y_Page_037.jp2
37b367b462600161598de730e56e6f04
1a1d69a118d5e75a8cb26d112ab013a62e905527
737868 F20101107_AABHRT wei_y_Page_023.jp2
0586accec8789c2b2c7662dd6a42f35f
4ef7a773937e509fc1a2682a8996fcedd877a81c
9117 F20101107_AABJBE wei_y_Page_063thm.jpg
0163693da8210473091216a44a7d1777
54f2fc59543c8c178f5068890d599c56fa396be4
1210 F20101107_AABJAQ wei_y_Page_053thm.jpg
eaac16a9ae0ef0f216ed0f0e9d946d45
2cedc6e533666387a34432eaa65d07a22839a09b
6666 F20101107_AABIVK wei_y_Page_080thm.jpg
7e56be260765f7d5c97eda910945b621
7e0a57b87205304003d40d41297753262dc9071b
35604 F20101107_AABIUW wei_y_Page_189.QC.jpg
3b6fe5a3d058ca0e311ba1b3271e93cd
48f3b60b919df6db99ef33d918e041adbae28892
792004 F20101107_AABHSI wei_y_Page_038.jp2
6d842e37f3a44453a24cae57d4e4d866
e8a7a7340b8d4f2815e7b4b1248ce3185bf29c59
310390 F20101107_AABHRU wei_y_Page_024.jp2
c019e8f874a2fd2e23c271aa4eb7eec3
487282e8a239860deed171d363f8c24aca10eb5e
8757 F20101107_AABJBF wei_y_Page_064thm.jpg
675c213a1d9dd94851ce0aa3f0fa388b
3eae4152506ba348b4fec28794ea55f315578d89
7696 F20101107_AABJAR wei_y_Page_054.QC.jpg
59fcde4b09e575ae20d33f03075b278b
8830a78055fd6db5be4df0af17bff8c04101b771
8543 F20101107_AABIVL wei_y_Page_059thm.jpg
f59e1e0dcaa8659ac1de67cd9c110285
db94bd68d440d6f0986633ff770d4604ff650024
9658 F20101107_AABIUX wei_y_Page_156thm.jpg
71cf333df9e75e31f55f5275a2c2569b
5d8d0ea9b08c8d0bfa44a5e054e2c499bbd0146c
1051976 F20101107_AABHSJ wei_y_Page_039.jp2
972e80722b4f6d05853a5fdc76afe872
47c1ea11b87c4998e7f90ac00be8d0cad6ee7b24
956047 F20101107_AABHRV wei_y_Page_025.jp2
684a4392a2aeccadc41958cdbe53f0ec
174fe9068ac94284f7270ebfddd3bb3785b6a6b0
36584 F20101107_AABJBG wei_y_Page_065.QC.jpg
4b5026ef9882aaceaf0004ac28320244
61ee5f7a209c4adc570b06ba21845d74ba812537
2195 F20101107_AABJAS wei_y_Page_054thm.jpg
e45a8f6da8f72224edd3d9d042fb422c
f98cd0b2f64c0b84743647a3d93f79dbd8aca47f
6994 F20101107_AABIWA wei_y_Page_088thm.jpg
69f428c80f976cf616bf526ff78ca314
5458cc453550113c755cf03797dc2ad9ef95f235
6193 F20101107_AABIVM wei_y_Page_058thm.jpg
a7453fc166f7924196973aec7f0a89c5
2e131094aacb283a2ab687c2bd9cd0db39d02e42
1051942 F20101107_AABHSK wei_y_Page_040.jp2
fa2a6cc9618384aa8d23a8c2c0f6518f
d072180b04c04deb42c0aed824a3dcf86e45b650
9007 F20101107_AABJBH wei_y_Page_065thm.jpg
b89a4a5e1b8f5e5fcaff153c50c8e2e3
c4243e4450f2ba1568e1383315c91047def340e0
13147 F20101107_AABJAT wei_y_Page_055.QC.jpg
53c594a18b1dabab2f7f765637ce7b9b
840c6a8b1a9f83af2df72bd90a902c1bfa2e4791
35034 F20101107_AABIWB wei_y_Page_030.QC.jpg
e0473863c4be1078ad871396ce313fd9
aba873342ae83aae8b683287d2a8c0b746f23a90
9925 F20101107_AABIVN wei_y_Page_178thm.jpg
1ae5b88ec4e3c284ec7050f59c335f9b
b9e4e1264c172f63bdcd709411d01a6f45c358f2
9059 F20101107_AABIUY wei_y_Page_061thm.jpg
def9c9853ff1802a3db34e70fa08e162
4934fe90a826f46deb8d7d3636d4c2fa2a2e54fa
303847 F20101107_AABHSL wei_y_Page_041.jp2
980def7b65d5e8f813abb32b6e0e1ff8
818369a810222a8e158e42bd92880d46eb5241fd
779926 F20101107_AABHRW wei_y_Page_026.jp2
2b08368f2014e20b831dcbc2f1a77b22
82e41f72282156888a5ba09ed95262c2eb41ea1d
34781 F20101107_AABJBI wei_y_Page_066.QC.jpg
c7bdd6c6c780a41ca4636650d7376f73
70321309bcfd4ba093d909fc970c8e368e5f0367
3685 F20101107_AABJAU wei_y_Page_055thm.jpg
cb839f287e21b6d2c191dc8ab4e1d355
701f1669d3ae2f9f42578af5ead6eebe768a11da
9907 F20101107_AABIWC wei_y_Page_177thm.jpg
ada8f6ef80f2d16ee299131e792b7723
3db35d5dab78af90bcf05fc7125eebf887565993
26615 F20101107_AABIVO wei_y_Page_096.QC.jpg
23422151e02ab30abda95c526f4699bd
0695e7494bb61346c16a719a0c45031cb6574cca
10082 F20101107_AABIUZ wei_y_Page_174thm.jpg
1dd6f02ac7913d99eb13adf4d5cf8d55
bda52d3f7a16d35803d2b88792199637bc048739
701282 F20101107_AABHTA wei_y_Page_056.jp2
de68ed79898396cd40fa328d0ab7aa05
bb22a197759f95c797ecee1543eb1548187134fb
1051986 F20101107_AABHSM wei_y_Page_042.jp2
8b18a5762d4edbdb2c2a764a641d5ba5
d22d6cc2f9b8efba4d1895ab9b1fdfef5dbf2590
1051985 F20101107_AABHRX wei_y_Page_027.jp2
23f49346e860599553320417014ecbf1
dcfd93781c455f6e122a5d0ec0414f3e8ec18203
8561 F20101107_AABJBJ wei_y_Page_066thm.jpg
ab9788bdeab7c70e88e611a625b25216
ed0b71a74a8c0e9336d0b50e33749233124e7fb4
22344 F20101107_AABJAV wei_y_Page_056.QC.jpg
2ecab62ccbc4083adc22dcb2d0d5b3a3
eb01772778ac3b337ded4b5ee8c228c596aae184
F20101107_AABIWD wei_y_Page_062thm.jpg
6bb62c186780c1243ed61e23b036c95e
70f80057cbba8f15386f40f01099b8cd57700cd2
9608 F20101107_AABIVP wei_y_Page_146thm.jpg
788b0066cf818559ee088a839464b90b
8361c77d6fe9a82274ab618901ab96f1cf6cc33f
390940 F20101107_AABHTB wei_y_Page_057.jp2
786974fe7be59324b7523d36ee8220fe
6f3f0b7001ac37a7508229b7bbce71e470e990b5
1051879 F20101107_AABHSN wei_y_Page_043.jp2
cb229b3db106c521a707484db5d3edbb
47a0d7315bb9580aaa5e59a06f7c7f9751bf5f93
688306 F20101107_AABHRY wei_y_Page_028.jp2
8b64af526a2c832169212f96a2b097e9
92467eec60d3e9bc6bfcafd985c05f221cb6206c
37035 F20101107_AABJBK wei_y_Page_067.QC.jpg
f1969624a36477f78971473440f13210
78a938a9a1fd32d7706ac3729fb48ae9d822892d
6206 F20101107_AABJAW wei_y_Page_056thm.jpg
3d7d8f39b120b785fc1d7d2a0dadd04a
ad67854e92a67e78bc7629dafe0f1c06693bf2ec
7333 F20101107_AABIWE wei_y_Page_072thm.jpg
3fedb42d9893527be64d903e92bf695f
01e59f0e2c21329b9c978f389e50904d3fdf5463
9894 F20101107_AABIVQ wei_y_Page_173thm.jpg
58bb094f033e3207237ab15b8118bc4e
30bc25795169df2fa841c2a255f489b9efe25aed
703617 F20101107_AABHTC wei_y_Page_058.jp2
3d682e406a7cacfa8b9d70d3461a7694
13a8fce6b3dd0f993c3cb216ebea66744b326b8e
1051945 F20101107_AABHSO wei_y_Page_044.jp2
c2f1b04e9e81b446562a76eea7f0c53f
72809cc0ee143f87000ed164d0071e2d7784312e
475592 F20101107_AABHRZ wei_y_Page_029.jp2
8a71afd1ec4303e648986713a02a5eaa
3d6d91339aed37bb38c2f5517094a46e9fd4ffb5
6873 F20101107_AABJCA wei_y_Page_075thm.jpg
ec10c89ae8517679827018b9617baaf9
6ac0361b7a3a7ce6436e57add3ad3244a677a25c
9271 F20101107_AABJBL wei_y_Page_067thm.jpg
c0a7501c285f3611d4bc453c824a3844
61a1a3b7255461576916bedc2972a4bc13288e2a
13098 F20101107_AABJAX wei_y_Page_057.QC.jpg
7e031d61cf5450fbc658a12c8b0b962c
97ab6bd452fb5de033701ce9a821f67db519b5dd
6720 F20101107_AABIWF wei_y_Page_092thm.jpg
4138c465610e2d437258d772ab3167eb
fbda4f7c7735105c57c0b820bd68720915b67be9
16860 F20101107_AABIVR wei_y_Page_119.QC.jpg
cc47233f5b5b038863df007b4f41886e
88bd7044e6f223e10b54d49fa8ffb47dac8afa7e
1051958 F20101107_AABHTD wei_y_Page_059.jp2
913e659f95aaaede8812231bed1426cc
49803f8d73181e0fe9a7e47f498be8df52e8a226
1051964 F20101107_AABHSP wei_y_Page_045.jp2
6d9f2aae8990bd05d588e558b89ca106
70990bf06ad371835d47373e87a7781c238cacf5
6560 F20101107_AABJCB wei_y_Page_076thm.jpg
292a8c299ce1a9efea8ea842a5b5d91a
7b22290ae23ae0d69779abedfca31aa15bef9c47
18759 F20101107_AABJBM wei_y_Page_068.QC.jpg
5aa8ff8752e9b58ade1f6f963fcbca1d
c0d82d6201de6b2cbab9ac93c47aa14d93a4a683
3698 F20101107_AABJAY wei_y_Page_057thm.jpg
e332ea2f8ca49e43ba1dba0f86fc2107
102446b4814ab95bf678dae62c4e10a761d2c823
8678 F20101107_AABIWG wei_y_Page_191thm.jpg
250a6b5d511ac168c5218ce0545d6450
dc76a42105277e4edf68b02e8ef0033fe7af950a
9980 F20101107_AABIVS wei_y_Page_175thm.jpg
b3543226e145373ae1ac277ff13b64db
ea84b193aefa845839bb98ec61be4ad1c988b6e0
1051981 F20101107_AABHTE wei_y_Page_060.jp2
41a7394c4f21dd5d2107c0e28509865a
f2cadfb23ba562e085a2340c28ddbfa3c40603e0
1051936 F20101107_AABHSQ wei_y_Page_046.jp2
14ed4bc8371a3fbb8e0c2673df8462c2
d467602e9c3aa7c5be8469b495093a0146e4c9bc
19997 F20101107_AABJCC wei_y_Page_077.QC.jpg
25cb4b4ded66861ba43a723b65e34a5d
48d0acae26a969a4eb17e4da69aab72a62e52222
4989 F20101107_AABJBN wei_y_Page_068thm.jpg
3e19d1a4468875ae2fbdde54b0f00693
bcd39a6dcac0620d591c94db562fffefa50fdbf5
22274 F20101107_AABJAZ wei_y_Page_058.QC.jpg
126cc5092429ae17811d2f9d6ceda61d
87d516a5981bbb813ee8a2fc9e9f0c402627ab55
36758 F20101107_AABIWH wei_y_Page_062.QC.jpg
37850796695a0b394f82fc3efe49c9f1
5234898e4991207848913362d227644ee7d92992
10097 F20101107_AABIVT wei_y_Page_141thm.jpg
e407c3a7012e5777220da02f37d92b68
e269c0bbfb3d2061360e42dd98e0db5e7a697d6a
1051970 F20101107_AABHSR wei_y_Page_047.jp2
5027711c6ca07779a41f9e9b5192b393
2d497a65a4fbc8e612631c3aaa288e0f62210a39
1051965 F20101107_AABHTF wei_y_Page_061.jp2
394c1f69dfe0ffb4a34e32cbb9778232
d784cc652f35d6caf0d9695c22444f8cae466968
6375 F20101107_AABJCD wei_y_Page_077thm.jpg
95692c62569dd3c3b007183d12140390
9744ac1825db4789a9a02fc45a0d1ff26a490192
19742 F20101107_AABJBO wei_y_Page_069.QC.jpg
c0fb2bc26580cbe6867574f78eca8872
035c6aa77a55afea346d18d58498ef81d723c151
35600 F20101107_AABIWI wei_y_Page_064.QC.jpg
9aaa598c05a78b596f2025975885fe10
777d627f14d9a14cccef41c27040f59e08838067
34462 F20101107_AABIVU wei_y_Page_063.QC.jpg
0094f6fc9fae9a9fadb53926feafac0d
ddbed0db3cb9f2d915163d3c9b7493ea35c97162
1051957 F20101107_AABHSS wei_y_Page_048.jp2
dd091e03ceb42644cca2014040aaa087
f24df94ef7fb10d8744c9bd131647affacf28c4e
1051975 F20101107_AABHTG wei_y_Page_062.jp2
3515b82f261224b5395684f00ed5ebee
15101749f7275691dd4af6fa31474f8f951011f6
22314 F20101107_AABJCE wei_y_Page_078.QC.jpg
aa9a6904cd9291c2446d0340c18e2c58
1ed3192e84d23042d050bb2415f8198ec48a5411
7042 F20101107_AABJBP wei_y_Page_069thm.jpg
a8f91839c588673d11f582ea41fea22d
6676c800db5882953a5c4bc542d89e3274e2536f
9990 F20101107_AABIWJ wei_y_Page_167thm.jpg
8bf8ba83b3f29e0e6cd863806269b74c
df892c11cbd2839fb591929b40f57915946a75f9
21140 F20101107_AABIVV wei_y_Page_086.QC.jpg
3325a84d252ab56d6c18d1528239e099
630496925b72718261de14e25d3ba8464efb31b2
488841 F20101107_AABHST wei_y_Page_049.jp2
ab403f921863f19a574314ca5b38feec
8695874309dd3036f506ada62217b5e758b4e001
1051950 F20101107_AABHTH wei_y_Page_063.jp2
f51057c342d2e5f27531648ab4357b61
410ea97153d7f4f48cc3bada69130931772c5ec1
18747 F20101107_AABJBQ wei_y_Page_070.QC.jpg
a17c400ec275afaa8059e6606878c37a
6489c423433c409c7278410b97be65931fc274d3
30635 F20101107_AABIWK wei_y_Page_157.QC.jpg
bb13559dd26e9912f8320b4bbab94cc9
d2be141181592cb008e17952b299a2bc2b368292
9175 F20101107_AABIVW wei_y_Page_102thm.jpg
fc853c306fe4005a87af4522145d02a7
1697c7e99ee99b2500f1e70fd5808e265047be3a
1024689 F20101107_AABHSU wei_y_Page_050.jp2
c59f111db850a0734f8bd109796a0040
579ad45a4ed818f601494252cdfd4cb927c12619
1051972 F20101107_AABHTI wei_y_Page_064.jp2
0626e21b0790d76f7485b7af69ad9b81
db3504978237ce104f5afdd16f17e3156cee0c95
7190 F20101107_AABJCF wei_y_Page_078thm.jpg
69dd529961ed589d6d94477617bb3675
205bf59f6fb08e3d58d02ba802842c3a7aa0f7ee
6885 F20101107_AABJBR wei_y_Page_070thm.jpg
ce5edc51cdcf307fb5ee08d836ac3345
4d60c94a2ded1369f052f5f18ffabb608f09784e
20438 F20101107_AABIWL wei_y_Page_089.QC.jpg
5d5d99e90dc02ccf252179f584acd295
4de96b225375c017c8ddab5e81ee6b5e17de50c2
9711 F20101107_AABIVX wei_y_Page_155thm.jpg
a35b6143aec469bb3ed2367e728324e2
6f017bb4122c48b6f1f91d4745e717759704db76
1051961 F20101107_AABHSV wei_y_Page_051.jp2
3a3bdcaebc161cd95a6e22fd96e9c6f7
e012672851ed2de46dfaa2c377fd556396574ecc
1051947 F20101107_AABHTJ wei_y_Page_065.jp2
7ab4d7739b9d757b3fe34f018974d6b8
23ee42a1c72f0c38e69ab626a8a0d76b19633436
21019 F20101107_AABJCG wei_y_Page_079.QC.jpg
a9c888992ae9155c42f4b2b25c8e51f3
b192598fec287da13c7ecf22decb4f74f2266d1e
19050 F20101107_AABJBS wei_y_Page_071.QC.jpg
e5340512c78ece8615f3299144bab0ab
40e2245646783faa570b40bfdab9f8b55cf2dec8
25557 F20101107_AABIXA wei_y_Page_004.QC.jpg
9221461b71adfe5ce75c90b724b94815
8d8f06c4384224f29f196c0ddae4092aa5800ebb
24934 F20101107_AABIWM wei_y_Page_098.QC.jpg
49b261068349af7aa38cb46723b9f720
39ce45c94f2279dd01699cbef6e533c47632ebf1
7472 F20101107_AABIVY wei_y_Page_025thm.jpg
273b811423cb258a04d00854d441daf7
97041ae6310e65e0612ca07376b681bc21f7e685
F20101107_AABHSW wei_y_Page_052.jp2
2de9595aab3dc8d2c5f3ce4a79ebc852
77357f286c4f5e26f58fae2b0979905a58a3d176
1051930 F20101107_AABHTK wei_y_Page_066.jp2
09179db9c4804329efa4b8e277e4b636
e9fc3058abfaef01dedc374258ce610bbf3f3dbc
6654 F20101107_AABJCH wei_y_Page_079thm.jpg
7f724cc6cb5504a1c89d1752ed5a6c29
3e0df543055e45e52642675e78018a5a7d362ade
6863 F20101107_AABJBT wei_y_Page_071thm.jpg
03662d61193db284b9a664f6f26e403b
1ed8389ba520333faac510c908d725cee79b3254
6279 F20101107_AABIXB wei_y_Page_004thm.jpg
1525f6aaf4a6118453ad6d996b9b9341
288d48b86d35b887334ba6d6be2dd8c2954c1ca7
7326 F20101107_AABIWN wei_y_Page_019thm.jpg
7c92a41540b928e9a0e20fb7b2007c31
e7fc7a546b1fdf8646bac9330a7403ebf0b7695f
F20101107_AABHTL wei_y_Page_067.jp2
a01e2e5667b158dd01dd2b1c4a8509da
13834c8aee9213076558e054f576b2aa99a04642
20544 F20101107_AABJCI wei_y_Page_080.QC.jpg
cf704f7034564528fa700b1cd907559a
070567734f7b15361a43c22a68df836211226946
22109 F20101107_AABJBU wei_y_Page_072.QC.jpg
8f94eb7640acc9e4dc9289a5f9f95809
6541fea5d55e16f91e1d8ee48981bff88a9981ac
16743 F20101107_AABIXC wei_y_Page_005.QC.jpg
956608ab6e1803e3ba52d7c70d95fc8d
2f2e5aede20dcad5aae570ba1f4696dc63cece76
10104 F20101107_AABIWO wei_y_Page_163thm.jpg
0f1e33186fa34fce5efc6012167742aa
1f7b67f6708344eb6d61aefe10c10304ff639345
5369 F20101107_AABIVZ wei_y_Page_021thm.jpg
aec009436bdb15f53f8056ee45053e73
0eddc83640f144186af18e2a0ff1ae82c6e8bb12
110305 F20101107_AABHSX wei_y_Page_053.jp2
3f5e93e2e4a7315f77a88959ec7e42c5
cb5a1ba94505e1fbf7f95089fb8766080b9c2804
431096 F20101107_AABHUA wei_y_Page_082.jp2
1e892c61098e0c9005a621c670ab0efc
4349a798b1d417e04368da5042cbae599eff5542
587310 F20101107_AABHTM wei_y_Page_068.jp2
78125e2120a2075004533bdc38c6abf5
3ed656fa5866ca30fe61786065f83f020c418471
23488 F20101107_AABJCJ wei_y_Page_081.QC.jpg
b0a1aed359fd5d46d6277ebf7be1df3f
1f32a943678f09797dabc4f09c260a0867aefa37
21170 F20101107_AABJBV wei_y_Page_073.QC.jpg
48363664f953d6db42ac5ae1b58609f7
b7daefb810486405bb5a12483aa8c6e73c321a25
4143 F20101107_AABIXD wei_y_Page_005thm.jpg
fdc8aaf149ea82e064b616dd58bd065a
f20fd3488e845281fd7746d479497a1c63952941
8518 F20101107_AABIWP wei_y_Page_096thm.jpg
5c507fa25bb7196c623d9cb79a75dbc8
0e20655090790e36112fff015949524f70b812eb
178992 F20101107_AABHSY wei_y_Page_054.jp2
805a0dd314e5cbe7e28c2cb24ad85d90
47a68be0e1d29f59b456aaabcfd96c4a633bc7bf
427376 F20101107_AABHUB wei_y_Page_083.jp2
07851a31d0a4e0e13b1b1b5114cf3f87
1e5db8cfea1da5e7b91fec3c9fc5a10a6506a5c8
383866 F20101107_AABHTN wei_y_Page_069.jp2
21f7211edbc466b526982bc21ea52b2b
506081e8e0dafaec8d2c91fcbd8ea2fd03ae8665
7521 F20101107_AABJCK wei_y_Page_081thm.jpg
c48a4ed7d74d91b158207d6402df7654
90115db7eb41af0e9d257678789cc6a4b412c7e6
F20101107_AABJBW wei_y_Page_073thm.jpg
10ed55dab377de2a5a7e10520d7f1499
e260b430e3283d32cf484442a96b4b74d4891ad8
9236 F20101107_AABIXE wei_y_Page_006.QC.jpg
69343bc0f49960e43a7097d44bd24bbc
26ec9ef765eb947c9789be7355030e46e0c08cee
9055 F20101107_AABIWQ wei_y_Page_099thm.jpg
4eebdccedf46207829d89f57a0e9d359
4d9e1b4810cacb50e5660d854de2cbcbbdf36cc9
390163 F20101107_AABHSZ wei_y_Page_055.jp2
9851c45c5d19081aceafe08418e2d3ba
849d0bc25d3dd2d1b6c40934a6d669b29b6a2c6a
470743 F20101107_AABHUC wei_y_Page_084.jp2
fd24f36e9121f2142ec33dd851ffe6f6
51dd7af15c9fe43a3698055258a7b1d8a639566e
358070 F20101107_AABHTO wei_y_Page_070.jp2
4f78f87ce752d14b66e41602ee3e8c1e
321dde431c22ff053ff50b05325a2d2d6410521d
27252 F20101107_AABJDA wei_y_Page_093.QC.jpg
e454d3256a51de1a8b4563b08f703106
6ea9242efbb74ff70d3ad4f042dd51b9b6a14142
7053 F20101107_AABJCL wei_y_Page_082thm.jpg
f86fee667a16d154acdd04ba82a0213e
4daa20b547313a6e611dc51185eefe001162d0c2
21102 F20101107_AABJBX wei_y_Page_074.QC.jpg
d05455163dc081dd18a81007cf506486
bdcacf1ace73518a5c484576ab77b4201bda9b67
2593 F20101107_AABIXF wei_y_Page_006thm.jpg
c060b9a57fd5850db2b4e6192297bcce
53015b90404c85877be6b1ab871633b230c2e3ed
7110 F20101107_AABIWR wei_y_Page_086thm.jpg
7712dfcda6f621981e3824840e8a13bd
0f3c367072a9edcfe84dc915066e0da63e8dc36c
421548 F20101107_AABHUD wei_y_Page_085.jp2
724f968bd2a58fea55cd0a8589d3e60f
bb9047fb8157466373a3f2a67ae2837fdb146462
357736 F20101107_AABHTP wei_y_Page_071.jp2
c136e76cf569ee1daded84512215c8e6
c0d2b0bfe92d1cb6f6be0d7e6fe89cd64bdbc304
8834 F20101107_AABJDB wei_y_Page_093thm.jpg
1fe82bfb01008a9172227ec1b58a71de
bd6dad3a07585d9dd099b4b3eeae18e4979a31ad
21144 F20101107_AABJCM wei_y_Page_083.QC.jpg
42d3715bd3c3cef0a4c9647a6f2c9ccd
c576e65347793caccf137fc9824ef81b68fd188d
6963 F20101107_AABJBY wei_y_Page_074thm.jpg
80bc4fb800651700f66ba24faa43217c
b09fb04ce47c6c8186be4d2a273ad321812555e6
37278 F20101107_AABIXG wei_y_Page_007.QC.jpg
85e05dd7ee3f46ee5c6c959be2ecf9d3
1b9ff3b9a393fd76de20428ad44c3672d3e1ccab
27472 F20101107_AABIWS wei_y_Page_099.QC.jpg
eecdc668066cde14fb74d6d691fb3cfe
2a995734f96526ada28f0f134ad0a9b16bf07930
420971 F20101107_AABHUE wei_y_Page_086.jp2
7130318a88828474c4f6de8ad3588b9d
ce969775e0b4a4d5d718238e17e83632dd4922a6
454884 F20101107_AABHTQ wei_y_Page_072.jp2
4e7d70e5813cdbf08806ee0c97662ca7
18e0fb061186f389cb6406ca0a4f5620c132f215
26328 F20101107_AABJDC wei_y_Page_094.QC.jpg
838db6c0fbd2322b94e3358948c487a6
dc056b63f29b22980c83d5559d81f14432fce8ec
6956 F20101107_AABJCN wei_y_Page_083thm.jpg
8ff7cddac95a7b252f9eb10c172faab8
f7e69338c175749e72b05c218def56e7693ec2ea
21063 F20101107_AABJBZ wei_y_Page_075.QC.jpg
18e6ac604b99d99c1447d4b4fa49d559
dfa453565a84cb066a245f40dcbb6d23b42aa907
9314 F20101107_AABIXH wei_y_Page_007thm.jpg
ccc7d07ee94e6247c0bb2b30c2978887
f3f87a818db191a8531587619e62c287178ff1f4
21563 F20101107_AABIWT wei_y_Page_082.QC.jpg
13b578934b804ee9f847f62c9cb45c62
15d4ceb0842b90cc759b89200b9eb5a1f66f27b2
25271604 F20101107_AABIAA wei_y_Page_045.tif
b8a6a83710de5b95250dbee4fcd66208
264ddd61ddd5f0a2743b52f666c5dc3108a3cd90
480287 F20101107_AABHUF wei_y_Page_087.jp2
50170aa81c2dc1aa91e30e1162efeedc
2fbdb375fc8919a6864e978a7e987471934fe1af
426008 F20101107_AABHTR wei_y_Page_073.jp2
7fa4455473469d2b2c79474a9fbd7912
12a74fedee653ccf430ed7fff6b5bfe01abbb192
8491 F20101107_AABJDD wei_y_Page_094thm.jpg
659b852f34065def94217789e9d01121
e7e627f736d00d6ef72039bbe076345f797e5eb7
22857 F20101107_AABJCO wei_y_Page_084.QC.jpg
d178d9100efd095ffb15e8a7042ccb02
067be8c81fd1f24221a277619f26ddc47a5e6bd0
3654 F20101107_AABIXI wei_y_Page_008.QC.jpg
a60915dbfca7d9dc27da730a1c61725a
0327e40595338065cf3f8c4078dd6ec5740a4d28
9069 F20101107_AABIWU wei_y_Page_126thm.jpg
6c40a62b4135893d724aed0305a6577d
3b24b7917fc8639e833a21ec6cd336c0fbb2ec7a
F20101107_AABIAB wei_y_Page_046.tif
9b45dbe5ba4b289ae94a8ddc791586db
fce3186a65fb60bccf89b76db33c451ecfa3a74d
417704 F20101107_AABHUG wei_y_Page_088.jp2
0aa24a27e19c7bf09fa75721279d5603
4c87169426885bdbac77346f09ccc656be90251a
425604 F20101107_AABHTS wei_y_Page_074.jp2
5b18029efa6d88de9e02ca40f91375df
201c179b7b565a4605198d36bd7b6ce729b769e9
26342 F20101107_AABJDE wei_y_Page_095.QC.jpg
6b9bd5de6ddd552081cbd1fe48c7ff74
762105a4ba21cab368d9c23c473accf3b5019bef
7731 F20101107_AABJCP wei_y_Page_084thm.jpg
67cfa86d7543381c66edf3cf30c9fd47
264a9f1cdf43f358019e19468536fe4642566695
1160 F20101107_AABIXJ wei_y_Page_008thm.jpg
d430e7ca401dc2ba0e9bb9e4403ead50
21ae202ad4d825b5b70139e235a12a7c819c5e95
283566 F20101107_AABIWV UFE0022538_00001.xml
2a1e9de692dadae9adb38ebe2a0aee7c
d58f9c5de0f11f705f8f95b861d1b226b97eac65
F20101107_AABIAC wei_y_Page_047.tif
5a8019e59476deacfd9c900f64649f5b
828338d78075dde8e5b2c5290f21f5c2ca4f277a
413264 F20101107_AABHUH wei_y_Page_089.jp2
44046b5b7b74a826e75d38bbaad2ac35
247e1f89f959d11f959fb01a0bfa1dd8f3247853
448568 F20101107_AABHTT wei_y_Page_075.jp2
ee7d7b81991bb802d220b47f3e0c91b6
b2aa7091412629d0900b6d1c04fd5a882f67122c
8464 F20101107_AABJDF wei_y_Page_095thm.jpg
ee0ee09342627742b4209f39e4f46603
8bf0d1d2fbcb6e7c07f9e5b6803682404e112b1e
21113 F20101107_AABJCQ wei_y_Page_085.QC.jpg
2b88b74161db3317ed627773303bc036
3b3f6d0baaf2f9aea2544ea90ec94791f79aaf00
28020 F20101107_AABIXK wei_y_Page_009.QC.jpg
1ae4d3788cd141c3735fee077d8125fe
4e289ee7a3524caa252f401a0da937d1408c60d4
8205 F20101107_AABIWW wei_y_Page_001.QC.jpg
dc64f581decaf7e0b87903f445659535
550ff19e85870caa467a80abbad596eab6254e30
468364 F20101107_AABHUI wei_y_Page_090.jp2
ad4256d29cd2e21596817efe87a45319
4369d0d062e29dbaa3789ecc3790d5f91b4b6441
414088 F20101107_AABHTU wei_y_Page_076.jp2
5507ee3245afea885269b866287f53c2
6385555f26ac4d7b87e10431ff8f48eb8ad28331
F20101107_AABIAD wei_y_Page_048.tif
1780acaec5a2ca159f72c0e7e244fee0
034e0d58145f1d90bc77dc5652a366a161ba2c6f
7120 F20101107_AABJCR wei_y_Page_085thm.jpg
e23412c1424dceb65185f5cc51ed120c
51e77257d5fcb8a6f60aabcd45f3eb859ab9a063
6935 F20101107_AABIXL wei_y_Page_009thm.jpg
6b1c7d938bb5ece7d7265c8617e79603
d8c00d0c702f4ebf95c4828f9ebd37c923124288
497 F20101107_AABIWX wei_y_Page_002thm.jpg
80a7c91092f383d242262b698a95c5c8
1e49437b2ad6c808d0928f68f233dc87be17c3d8
413901 F20101107_AABHUJ wei_y_Page_091.jp2
321fbb25f99ad1d1c1bbe11810608fee
d55eea733ea13d93fc6b16b7a63596741da09852
408208 F20101107_AABHTV wei_y_Page_077.jp2
39dcc4a1ac97e7b74d34166463658009
89714130361c30fb4f514483e53fdaf1c86792b3
25480 F20101107_AABJDG wei_y_Page_097.QC.jpg
2f75bb0c7e59c899568356bd0ac2e2a4
147e43840b9329acda4a6660eab36addaa7c5ca0
22636 F20101107_AABJCS wei_y_Page_087.QC.jpg
977bbb9f095e07f3dbac256a8424c4cb
91be14a5a96b5501a52994074aea19c46b111d1c
31033 F20101107_AABIYA wei_y_Page_017.QC.jpg
b24fe98b290e5b65d3edfe717d116a44
57d8819cd4d2a6349b84e9f06f696a960985a1ad
23543 F20101107_AABIXM wei_y_Page_010.QC.jpg
85c0c6310beda25cabdbfefe876afa8b
de1c1e118c40e811e3120b26e224fa8df6cc49e5
9960 F20101107_AABIWY wei_y_Page_003.QC.jpg
7aa5ce712a227d814befbf82f456986c
2897b60f8c8477493a09fffb3cb5445b5a4a1657
409661 F20101107_AABHUK wei_y_Page_092.jp2
2b6ffcbe5e8b8cfe6d12b0ca8cc9e795
4c7042ea3dbefe845a55e053fbeb2d10e5a4f9c9
461399 F20101107_AABHTW wei_y_Page_078.jp2
439b66a68f89131fc75fa45d29b18b02
ef8850312250036c6aae2866314e2f0ec2918584
F20101107_AABIAE wei_y_Page_049.tif
5732d41adda74994e56a6acb181b53dd
39df374ebcaf4bed82ff0d02f9b0b74d1506cab8
7996 F20101107_AABJDH wei_y_Page_097thm.jpg
ddcb14476380e12ce9d85669737cd687
f4c83bdc07c6ad64d0c46cf7a63806381a147457
7658 F20101107_AABJCT wei_y_Page_087thm.jpg
6d1ef9a8c1cbd41346a26a5fc7faa6fa
10249d869d830f4d19912c3970bf87b05b575844
7793 F20101107_AABIYB wei_y_Page_017thm.jpg
fcc9c3d348a60044720f809713ca074d
afc5d77f9597729c9bc1e01eef511d18c82fbc38
5819 F20101107_AABIXN wei_y_Page_010thm.jpg
dfc1c7afe00737f5f5f72fe8fc74c660
fdc7cc991ac86ff88cd9a608f7150b848a16ebb0
2588 F20101107_AABIWZ wei_y_Page_003thm.jpg
88e81044e60fee92c8396d78f7c58e26
3edcf9e2aa0a89be146ad9746f500fca15e3afa6
526184 F20101107_AABHUL wei_y_Page_093.jp2
4df0da68e8364758b562b3b21909a75f
13b7b881ad1333f9294173cc3407429dad3c91d1
413544 F20101107_AABHTX wei_y_Page_079.jp2
d5fb8e8eb1405150a7c77c0a8c01fb95
6275c8dfddc7e87fb5b97beee7b4cbe79f676cef
F20101107_AABIAF wei_y_Page_050.tif
998b5de969d632e073a03121d07461ef
c0b9ff386e8f9bc34b631da13830a69a0ae2a862
7961 F20101107_AABJDI wei_y_Page_098thm.jpg
b0e2c6a50fda60d74ec7bf430a4d2e1f
83a7c30a3e6f4adb600b1e16c91376146613eee6
6850 F20101107_AABJCU wei_y_Page_089thm.jpg
f2ec77d185d8cba68c115f0e50b09e00
47586af9710a0e00f94bc21cc707eda63075ecbf
28130 F20101107_AABIYC wei_y_Page_018.QC.jpg
a4f01a82fcb9b0fab9ec8e9d77468681
d4e4264b11d031abe4d18d44754aa0bd9601f238
33632 F20101107_AABIXO wei_y_Page_011.QC.jpg
3304aef0e5ceca0df4cc41296eb4cc99
c656de04c62670dd618e27ca3acb2c62c25791ff
597326 F20101107_AABHVA wei_y_Page_108.jp2
0c79d164a7f6fc9583f46af77f59fca1
95914a5687b9df69a931619ec2279cf87ee474ea
498382 F20101107_AABHUM wei_y_Page_094.jp2
e66e5523d97d6126ad01c7b370113cd0
afb9a4d058c8d2f41d727a8a08443c85c46a64f3
F20101107_AABIAG wei_y_Page_051.tif
5344c549bd13046743ac9cdadb5225a3
c8c7ef523769ebc71a92ecd5ec7aced6a932b30b
26100 F20101107_AABJDJ wei_y_Page_100.QC.jpg
31d74f555f301f34a2f8259fbfba9ba3
cd172e37befe0fe18e0f3c4ffaa39d4825ad28ce
21797 F20101107_AABJCV wei_y_Page_090.QC.jpg
2e06a4bbba89aaef7787eb7981b0983c
5b22d70fa0640904981de1ba814f45ab4c3954af
7691 F20101107_AABIYD wei_y_Page_018thm.jpg
92052508c77f1936e3d15601f5e3fbc2
ac3606427766c8d1de1bdb1f2f2b3586e02a2da1
8258 F20101107_AABIXP wei_y_Page_011thm.jpg
bc306b6cba44a43c6534e65c55236a37
dc3d837222809e9c88dfec97d675045788d33b2d
549768 F20101107_AABHVB wei_y_Page_109.jp2
8ce0b2c5b0e0c383eb6c0a9416457bd6
e9fb3e93e12e1a113a95e2497e18f4cb8da701ef
497401 F20101107_AABHUN wei_y_Page_095.jp2
6ab62785eb1a0fe044fd0a07353faf44
cc718a9985e262ff842efc92a72060d08d6d6e1a
409828 F20101107_AABHTY wei_y_Page_080.jp2
2f0c5a850ea3c5c9c7ee9cf232ea63e1
ac4fed553a8941191f3360ce77eb7a7682a1531e
F20101107_AABIAH wei_y_Page_052.tif
b4fe29af317cfd00c7554491cd3c0171
beac25323474fe7e0f00c9a440f10984c7149239
F20101107_AABJDK wei_y_Page_100thm.jpg
c13eef6b6ee7c4d1c20895c0340c2600
0bbbd64728419c837aa829e1199f501f6fdd2008
7336 F20101107_AABJCW wei_y_Page_090thm.jpg
58121c47a0ff95b37e2f7a2961df8ee5
a4c375cbf3ececd18d984c78c12fa3d114441b55
30731 F20101107_AABIYE wei_y_Page_019.QC.jpg
e3bfa557e2ca0a700e31dd5d0c01680a
105bc9dc4c5bf5415b9bbf5cc77f2c02925a9738
34035 F20101107_AABIXQ wei_y_Page_012.QC.jpg
48bc46e262641897efa2aa6c04e38699
208e9e5e906480d82dc2557a9eb808c77e44c2e8
549220 F20101107_AABHVC wei_y_Page_110.jp2
b7b6e4bf7c5e1c30e32bea1344a8d0e0
a4691a0a1ae4868b4456f9e187bdd4488c0de7f2
530352 F20101107_AABHUO wei_y_Page_096.jp2
08d6b5ff95b3a9a11ca97e370f2ab86b
c92836e0c4bd0caa6f0e007c95bbd4026896960d
479044 F20101107_AABHTZ wei_y_Page_081.jp2
beb958a04d31e9ab7480acf964c32088
1291fb92ee1779502f0ecfc31f108c8f5a13227d
F20101107_AABIAI wei_y_Page_053.tif
2decd3e1e4b5bce0bca03a436f12e5c0
60e360d0f15c5a3c9bfedbc8c7edc4282c449dae
9054 F20101107_AABJEA wei_y_Page_109thm.jpg
27f21fcb2ade9d70818576b8077c19c8
ddcb9dbd9a6f469e2e7395f02f35a574e6a623b0
25904 F20101107_AABJDL wei_y_Page_101.QC.jpg
ff55fd9e42dd18d4cd337b962d9d857a
3494f4c1e51fddf73eea0405f5a48650165179bb
20549 F20101107_AABJCX wei_y_Page_091.QC.jpg
c6e60f1b0f1c0efe6c1aab37a0e73bea
078afeeb67c15465440ee17aff27b9b9a9192556
29930 F20101107_AABIYF wei_y_Page_020.QC.jpg
28160a3eda8d39d2a1232ab20b9aa919
de7bbea29ba8640b60753c171a0e25edeb7772ba
8919 F20101107_AABIXR wei_y_Page_012thm.jpg
e77d41f85633b1824dcae025f2c2f464
622a108ec1ce3f8437cd47cccc20baf30b24fc86
720846 F20101107_AABHVD wei_y_Page_111.jp2
cbb165a381ca40187b68f7fb007d6556
493a1779e55e2f2daaa2c7f1cb32e41dbd1a6d28
481642 F20101107_AABHUP wei_y_Page_097.jp2
8d4a7de819d66f63ef20e189a1e21eb4
80a7850295f83fbde31635b0d1fd7dd54136b026
F20101107_AABIAJ wei_y_Page_054.tif
2f3467bd78e3dc07b2f7192271e13c0b
0fd9832711e0e378e0b990bd83901e3bf0051893
28119 F20101107_AABJEB wei_y_Page_110.QC.jpg
ba6c226842ccf144f28a38c8235dc04a
8f8305dd77fa61167a18b2ae9f2751f81d9ec0ad
8487 F20101107_AABJDM wei_y_Page_101thm.jpg
173873228b0836bf60455c4c962c6e95
3d6d55f78075c368625105b33484dbf8063ee01b
6814 F20101107_AABJCY wei_y_Page_091thm.jpg
3aa3a59c9efee7af23e056b47ca9e783
3c439d8e209deb8db24074ddddd1553770a8705b
7517 F20101107_AABIYG wei_y_Page_020thm.jpg
4b3883a4cbf662ba9e1671c32d7c91c3
96e2a3fd3230e6229d578e4d752876f529bb3cf0
22142 F20101107_AABIXS wei_y_Page_013.QC.jpg
10eac8a5ddde731952fb6432a783fec9
c6b3665aebba3ec8b40713e885153ff427b72a5e
720090 F20101107_AABHVE wei_y_Page_112.jp2
5c87470c93668b92dcec7fb4192cfae4
e533e52dbbabb9c21f6d8caadb1dae3952829ded
477595 F20101107_AABHUQ wei_y_Page_098.jp2
6417cfb077877f94586b6a501836d15b
70fddb058e84678db7337ad94b7a86a9083b88d4
F20101107_AABIAK wei_y_Page_055.tif
afb2a149ee52e66430fec2c39b0b371d
f6db2422de21f67a156fd144e6fae3a9aed116b2
31700 F20101107_AABJEC wei_y_Page_111.QC.jpg
46ef922b242ebe79b6a40e48f5dff84e
7e03902422164cace04bd3ea636f329c1f042450
29167 F20101107_AABJDN wei_y_Page_102.QC.jpg
35e8ab26245074cfe3ae6a6160e95d8b
e2b47ae5c00dbc574e2856024583673db5ef5dc3
20305 F20101107_AABJCZ wei_y_Page_092.QC.jpg
bbd4056d8baa377dbc93c20b6d555eba
fb49fbac7f6c58be0ff2f6c9a9d61180fe5000e0
17073 F20101107_AABIYH wei_y_Page_021.QC.jpg
a9f9f1b04183fcc736a78304419f9e75
69bd9eaab318c023ccddea1c59ca852ca0664a5a
6194 F20101107_AABIXT wei_y_Page_013thm.jpg
ffbe6a192b05b008b27f66f956c27003
30f7a46128219453f788242461745791ac983f6f
588072 F20101107_AABHVF wei_y_Page_113.jp2
c82ea5a3b3b7685005da4a578826fe15
4550c8b142bfeeddf3224fa6a2b4e20efca8eab2
545241 F20101107_AABHUR wei_y_Page_099.jp2
447dc533bfad0136e92edccb18a072a8
d6f9ad81371e8191b0b49ef3ee737d3b6e44dd34
F20101107_AABIBA wei_y_Page_071.tif
32527a3db2ac239b89349db5610d76e3
67058ec22c3fdc564fc4502081d8ab754431ef4f
F20101107_AABIAL wei_y_Page_056.tif
03e7b4311ec838bc066e0d1d5edf28b1
566d1f9a495bd77e52d099bc0f2751bb9e415821
9946 F20101107_AABJED wei_y_Page_111thm.jpg
30eb109ddcf6fdf58cb716f5d8936268
53e4864f565b3f7053bfdd5ab39cf32a6f818034
28150 F20101107_AABJDO wei_y_Page_103.QC.jpg
98ef2b9e0eb77f94167836b0ff4c452c
db5ab097282496dd2af14042bf85f232ff02e678
23218 F20101107_AABIYI wei_y_Page_022.QC.jpg
54286f107f7bf28b1fa431844a589d35
adaa044c40413c2e13d3cd44c13602a5b945458f
20589 F20101107_AABIXU wei_y_Page_014.QC.jpg
3f1bd55c331c3c18bdf4b32516af1441
08c021f5f8ee5e93f0ca2845c40219385b141750
541029 F20101107_AABHVG wei_y_Page_114.jp2
188222c1afae5b09a7370d0387911a7c
a1f687d2932580e9b14b4a80da9db40a971e32a4
499567 F20101107_AABHUS wei_y_Page_100.jp2
171b002527bc217e6028aab00d2023fa
b50a4c40f3a981849f9acda76767da7b2ce2e270
F20101107_AABIBB wei_y_Page_072.tif
2cd27ae75796cf9650e036618af2b988
564bcc7a17c97e665f796df6e06070dd18e866c0
F20101107_AABIAM wei_y_Page_057.tif
69279410f3134077915001a7495deb52
b918a28641218c140acbe6ad7e1037a2801ea7bc
31416 F20101107_AABJEE wei_y_Page_112.QC.jpg
2a0b84d2c6e7b81141257923c28549df
48c934eeacbcf0192152c8dd6c85efb88967a881
8651 F20101107_AABJDP wei_y_Page_103thm.jpg
5726c070a54eeb1d4d73b48946111ca1
c070fbdf0872643a265740dc3482700aef7e08a4
6968 F20101107_AABIYJ wei_y_Page_022thm.jpg
1147fd35a87517505e78d73f5af423a7
892010eb8e0a861d90916780484e995fc63cc58c
6380 F20101107_AABIXV wei_y_Page_014thm.jpg
201b4473316cf0bcc18a61a2af480228
30e5881f397b9010a7233ecd86c39fec94317560
532739 F20101107_AABHVH wei_y_Page_115.jp2
fbd5cfcda76d0b5651cac9abc9a84e66
661f8ac08488bab76c48a28e4a354e8c3ea29213
498498 F20101107_AABHUT wei_y_Page_101.jp2
cbc12424097283c68c8a44554b584314
2b7ba1487a618414d6e3c84ca09f17c3cadf85d9
F20101107_AABIBC wei_y_Page_073.tif
11771b3d84fdb73a719383b8813da4e5
8542758f6edd4f4c39a7cd9034ade1dc973da96f
F20101107_AABIAN wei_y_Page_058.tif
9d2c36f99743fd65423614ce46924b1b
a7dc2ddac6edf51c354ed2d7ecac27f05a3cf7a6
9970 F20101107_AABJEF wei_y_Page_112thm.jpg
9954f5f4701f3275757c9f5e6f2dcb7b
5685d9729f2d85a68c4112149d7957802efb7dbd
28065 F20101107_AABJDQ wei_y_Page_104.QC.jpg
440c38aa22c8124496d801a01e9c57ce
b97bd7af9711f4e4ba28fc3ece11285e4bf5a408
26057 F20101107_AABIYK wei_y_Page_023.QC.jpg
457f54e5adfd321f5beedd96f5765bd6
273ebed1fd5ee5023d88d9277b0a97d44bd4c3b8
19257 F20101107_AABIXW wei_y_Page_015.QC.jpg
2fb147a0640f197d96452be40ac603fb
eedfaf291b1d2bed655859fe07fa8edf75b89eb4
390661 F20101107_AABHVI wei_y_Page_116.jp2
c4fb17a12d6051d305d1719979611305
d0f22abe68a01d06c20b032e1b7a22ba8c29ed3a
584571 F20101107_AABHUU wei_y_Page_102.jp2
7904086319cbb4050c0c722e8d116bf5
3a856a065e3bfd694f03d3c7833d4ee91b50b192
F20101107_AABIBD wei_y_Page_074.tif
d3660206df2e34eb612a66183030dbad
230e7a3bf48c732edddc8532f84997d7174a88a4
F20101107_AABIAO wei_y_Page_059.tif
9231bc96b033e04a6dd41b45135c8ee9
6a0b3a59b23adc13e111ad8cf42cde72efff65ae
28917 F20101107_AABJEG wei_y_Page_113.QC.jpg
9166cdfbec55e3829655db6a1edf000c
8da9a50fb6b6ca7f5b8643131631786cfed8669e
8742 F20101107_AABJDR wei_y_Page_104thm.jpg
c2262fd8f6b9b2749631d00887d4a0fe
bdea8e1a9552c1bc5bd5eac84e881848883f501d
6969 F20101107_AABIYL wei_y_Page_023thm.jpg
95074861779b739de00372e87fae6574
123dc0172ffe62a6b11730091dec680eb855c476
5666 F20101107_AABIXX wei_y_Page_015thm.jpg
7b8ec3639f0a4060c417503358cbd7cc
6ed991faf7e67184b52b0c367d687cde22c701ad
416819 F20101107_AABHVJ wei_y_Page_117.jp2
3693be3e1a8540b064e59fb25fbc7a54
b0cdfa66a9ec1f76aa1f69bc57cbb4b711498d9a
537605 F20101107_AABHUV wei_y_Page_103.jp2
d78bb7d3a697550c1d67ace06b73e6a9
e017682e66dc14fd008c004d19b3f55ce5c1b4f3
F20101107_AABIBE wei_y_Page_075.tif
01486052d806a2e15e233edb04f9417c
766543c4aa277b62c00a56751da1e2fd4b6416a1
F20101107_AABIAP wei_y_Page_060.tif
a21d34b4279aa16cf208bb182b8efcb0
c8733cc5de494fad5679b25bf5b0f144c5b5b1ca
30421 F20101107_AABJDS wei_y_Page_105.QC.jpg
0c73762670bf5f592a3ca15c4215f72c
40de0c6215943af3db657fc4cf26ce5c58cafbcb
7670 F20101107_AABIZA wei_y_Page_032thm.jpg
588d40fb070739aa01745261be5307fa
8f15ce2115b862d8f06e130322313fd7ef7b5d9b
11356 F20101107_AABIYM wei_y_Page_024.QC.jpg
4342457d58aa21e6ef161c2c8bd82ae7
182129dc642ff41e066c3cea51e379343a63de68
28426 F20101107_AABIXY wei_y_Page_016.QC.jpg
bc1f87619663661ebcadd1d0ab53259b
ece83af37b5c19f33d6fb754351fcbf0ef713304
369787 F20101107_AABHVK wei_y_Page_118.jp2
10073ef1118780557ab6d0a62402f2ad
b91cd8dbb61802b6a5fb9aeae829f2228175ca00
536381 F20101107_AABHUW wei_y_Page_104.jp2
d517ac2e1670ce3fde533e30f3fbfbf0
78c218d40075455d815080432a8d0b463b540a88
F20101107_AABIAQ wei_y_Page_061.tif
9ad9669cd6859d92dbd1ee34a3e2c670
0b676a13ef3848e41cd7b7b30c5048a73adb9d77
9507 F20101107_AABJEH wei_y_Page_113thm.jpg
94e453e1f4c5c36bea477e86a69c96e3
c4dcdca37f03b15e4d83ffc2b2c046ef53ea3a45
F20101107_AABJDT wei_y_Page_105thm.jpg
def14f54ba493411a7ba65ac88d0a88b
4560f0839c7a0a3d883c07ff5f09d79eb548669d
28269 F20101107_AABIZB wei_y_Page_033.QC.jpg
67dd349946756380aec91978633815a6
2708288dc1611c79ccb804938f08d290ec2c20d7
F20101107_AABIYN wei_y_Page_024thm.jpg
8ebad1f0ea7840c0048cc7d5f7a31913
3fcad3bff691cb18e4b395cbd6f27784555a6d9a
7169 F20101107_AABIXZ wei_y_Page_016thm.jpg
a28015f4873342cf32b35f68fd704a75
6ed13af9087881e1fcca068a013acccc59e67920
464910 F20101107_AABHVL wei_y_Page_119.jp2
fb609eb36b07f5335d01ebf2b5c5c410
6831748c5806fcdd88b4584dbceb82b8726685f5
608470 F20101107_AABHUX wei_y_Page_105.jp2
36b09d0f54cc2bdbe6e71bdc06faecc2
e215da9eb8fb861427db54b168d3db49fc7dad5e
F20101107_AABIBF wei_y_Page_076.tif
a240eff20404a3095a42ee890bc04a83
e771eb8dbb3132cfab9126283e59a2c3ddfd8c7c
F20101107_AABIAR wei_y_Page_062.tif
7092b500d4effdae092efacc6b1b778d
2b952fe7204a5903cf4e426ad0c9ed24069e9172
27555 F20101107_AABJEI wei_y_Page_114.QC.jpg
20e205445ddcea2f6f75887189414d3d
3b437fd0f46ab7a034e51ff0a2401b8e44e7796d
29065 F20101107_AABJDU wei_y_Page_106.QC.jpg
8f19f7d2137fd106851613bfd9f7b5bd
4af3c37fc6c25e4b1095714aa807e1e2ed459c44
7137 F20101107_AABIZC wei_y_Page_033thm.jpg
be035131d7f8b3d48c3d383c83387b2e
0d0768d4e43e6a94dfa1d56987c298b5256adb36
28289 F20101107_AABIYO wei_y_Page_025.QC.jpg
c452dd371719c23ff832bd6acf15a177
2b71ac63bb7c7c026dc6d71378f9d7abd850a42e
387467 F20101107_AABHVM wei_y_Page_120.jp2
e6ffbeb47f1ae01c99bdc0ba2b3289b7
45ca54478ee5cfd8df640cfd6d9e42ffb4898529
565934 F20101107_AABHUY wei_y_Page_106.jp2
6514adbcbcbf61a13782baa1275218d5
b3972ab9a0a7059a368d45956972c6a6d67bb501
F20101107_AABIBG wei_y_Page_077.tif
60930f3b5de65abf8dc285f59cf6f06c
957ca57b6022c01f0d8ba7fb9171d76470714bfb
1051954 F20101107_AABHWA wei_y_Page_134.jp2
6a1ec2af5d3dcc3144d24004a32d32a4
822c2efb285f570574883d52918d9859d5269e80
F20101107_AABIAS wei_y_Page_063.tif
2416a4193ef0199eb440a61b1cec5fed
8b937167879fd7c50c48638c4c23663316d9d22a
8807 F20101107_AABJEJ wei_y_Page_114thm.jpg
f312afc7eb60f8dd633bd76a60399a5c
afb51cee18d717101e9c982c8e82509ef55a96d6
8955 F20101107_AABJDV wei_y_Page_106thm.jpg
90b10544ae5a0a77fad3035efa284f44
c815407d7e1929c84d3beb8b8e6dc5ae6c95095b
20560 F20101107_AABIZD wei_y_Page_034.QC.jpg
07ff9a27fa64946900e9c01fc325383d
1b8d7bb09adcfca8f27ed18e9abc39305c1de427
22972 F20101107_AABIYP wei_y_Page_026.QC.jpg
7882489395dc528a24b3689dd2b18c31
69765f9f0ab6e0ad76ceee31b0359991d79237b8
471611 F20101107_AABHVN wei_y_Page_121.jp2
8cf718fd062d51c239a31f5ae5e01f73
9b0b335549478992405a420510e676899f45f325
F20101107_AABIBH wei_y_Page_078.tif
8775a86f8492edfef497fd4f86be7ac2
8589360368b4825474534f88ea06d5e03e6f9755
1051977 F20101107_AABHWB wei_y_Page_135.jp2
2e6771e6ed43830cf55b280de97ee02c
587a451595ada8a1df89ee64b821cdfc13bf8208
F20101107_AABIAT wei_y_Page_064.tif
e30553438716e1a34bbb1918615b5a3e
83f4accc548e94ccaf8ac509754fd4171d7d60d1
27147 F20101107_AABJEK wei_y_Page_115.QC.jpg
97773113223aaa77c4da0ab96225d0c2
c25214e23e19964da35708734fce621c1a2f79ae
29121 F20101107_AABJDW wei_y_Page_107.QC.jpg
1010bdf485a6ebf65b5a673d615b7c7e
029f1d48946b691f0d61362d27f4a355fc25d40b
5783 F20101107_AABIZE wei_y_Page_034thm.jpg
cbfe59a82478aa3bc80034188fc3b8d3
3d9831cecfbb962a34246ae437ea1da49003d22f
6414 F20101107_AABIYQ wei_y_Page_026thm.jpg
b39b63143d008ea6868130b74fe98039
6939c56cfbc1dcc7d3910c1b1cb8beb36b92a4fc
F20101107_AABHVO wei_y_Page_122.jp2
fc54cb4dbdb8a119746b204299dc8bea
6874b5fe0fb2214c1aac1e641614534b4a9ea564
564280 F20101107_AABHUZ wei_y_Page_107.jp2
b56af7d7e4a9407a29e9fef6d989e155
8d9a2a2443afc91949838a00050fd75abbb45416
F20101107_AABIBI wei_y_Page_079.tif
4cc5a43abb7416561b4c7d55cd4b2023
ff7234587475929a0b0dcbe0014d2b76a767e6c8
1051973 F20101107_AABHWC wei_y_Page_136.jp2
39cc1b95884b8c76ec523a33be53ba6f
d02def4d87170bc2cf62eebbcaf369f8b6132b58
F20101107_AABIAU wei_y_Page_065.tif
d724add5a5d2e31266aace3dd4133ec5
9ea3298be7fd2c68fd3063a957f89a0409097fc5
8945 F20101107_AABJFA wei_y_Page_123thm.jpg
3404b36b1cb6cc1fbe271b424ebebaf5
59fe04a455356bff0e6eb2768e1fe2e3bc8a2b96
8882 F20101107_AABJEL wei_y_Page_115thm.jpg
c02ce4b6b4e82852eb929d00c8563090
f43e1ad4967c178fb24ec652f03e8ad885612736
9248 F20101107_AABJDX wei_y_Page_107thm.jpg
fe4d3188676ca5fa8af99f8c7f4ff947
9a8b9fd1749163572b2f7f86086015580fe30a86
22883 F20101107_AABIZF wei_y_Page_035.QC.jpg
847a764d8db13d45a174cb5ec009c05f
8052f7ee072103dd434fb6f09629b706d34f37c2
32664 F20101107_AABIYR wei_y_Page_027.QC.jpg
3272b8f6346f1436ee4e932ab441a5b4
75a8524eefa34bb4467cdc15edaebda5358222d7
F20101107_AABHVP wei_y_Page_123.jp2
87e0af650d3fe7a4f719fd1e28e14a74
6336798a90680d965a2a7f66c8a68b9973a2c116
F20101107_AABIBJ wei_y_Page_080.tif
b2ed692a6022b8b8d01f763f269988e8
012c2963f09f16c51257cced317e54f6efbcd0b9
179112 F20101107_AABHWD wei_y_Page_137.jp2
c070b48d8534da1abc6129dcac377624
4c0a811a382284d67ad2510e1aa9a64e63512c36
F20101107_AABIAV wei_y_Page_066.tif
392f69593a28c4a01d96390a2af1cbb6
25f7f1e040f3a88cf8d966e77f53fca411ff6bd5
38134 F20101107_AABJFB wei_y_Page_124.QC.jpg
140aa34eb87fd985d588ad93862a70dd
29adb06e7984684f9e68a07dc759b1fb199cd1b0
15294 F20101107_AABJEM wei_y_Page_116.QC.jpg
963c2283b7282458804de975748c6c64
6de4c392cc97b5e397ec31b28385a51d399d2efc
29513 F20101107_AABJDY wei_y_Page_108.QC.jpg
f066fb9370166474654cbf96b095f8a0
03cc0bfbbcd0c0e79a93f21f8807529e787891ad
5979 F20101107_AABIZG wei_y_Page_035thm.jpg
8c0eac8a36b0052c1fcaebc2ae009e5c
33d9bf6a7275f30ef04e896d85359eecdf2a9630
8725 F20101107_AABIYS wei_y_Page_027thm.jpg
6ddc4767863d1ca61b47845da9c43679
a16185f40cb61a609fd34fdd2cd6b67c8e19ea51
1051982 F20101107_AABHVQ wei_y_Page_124.jp2
9f7e7d781170b5bb0b68322b7764ddd3
f70731f9d9a89f6db8538717ec6131403ce31cff
F20101107_AABIBK wei_y_Page_081.tif
fa9e50c58ece6be1a9dd5c2ebfefe361
d6028c1481b9147918b56640eb7f2534d6605889
710817 F20101107_AABHWE wei_y_Page_138.jp2
b05e46de8ef223f8f70c4e2ee8047f6a
1ce7a46d7a66fae06bc9fe3afa4ca09f017e1cc6
F20101107_AABIAW wei_y_Page_067.tif
5b5b2641972a7ff93ad9715be750ceef
80fe60f153f89f82860dac36d51dcf4c89208eeb
F20101107_AABJFC wei_y_Page_124thm.jpg
c8aa81278e62072a56b603edd058c545
8964f4bc06cf41db2025a3ef1882ac7a05449a6d
5173 F20101107_AABJEN wei_y_Page_116thm.jpg
6504f2befdefb4374f02014d9dc505c8
1125591e72e6ca069df47a5e2efef0839736e2cf
28164 F20101107_AABJDZ wei_y_Page_109.QC.jpg
dcae8e16c883d193b2d6837c2048486a
c651d916615aea4edf2625d2cb5d55acc3e5ff60
25428 F20101107_AABIZH wei_y_Page_036.QC.jpg
4300a677b97e5b28a7cd2bb330781218
6a2e6343921f61c9dc24f704786b5fac8213b4c1
22197 F20101107_AABIYT wei_y_Page_028.QC.jpg
47c6c29b82016d1ebc6aa7f85eba5265
fe9edef2fa6f911f33830363f7bb84dcdef226de
1051946 F20101107_AABHVR wei_y_Page_125.jp2
d4d2f505144acef44aea32d11a893225
2c8cd7b5eabbdd4d9cef15a897f6ad7cce391af3
F20101107_AABICA wei_y_Page_097.tif
c1c34a9da367f011ee578fb4224a960a
1ef2911720924a3ff73397e16cca4924f68a0184
F20101107_AABIBL wei_y_Page_082.tif
4b8d8745d000abd4e930b6f590663567
46e1d21b1416afbbcc7dd55e1303833f4d224350
721224 F20101107_AABHWF wei_y_Page_139.jp2
ee33b0902bcf9e42a8c68d10d065e865
2a412d4d5d94f67e3a9e7072f9a7d7413ff5a200
F20101107_AABIAX wei_y_Page_068.tif
d2961023ec0a1d86adddc871660ec8ce
cdfdc833e14d39ddbbca712ead11fffbc5171cad
35694 F20101107_AABJFD wei_y_Page_125.QC.jpg
43bbb7c003cfea0cea5514f317b6c093
723a6185324a0ef5efad93a93d180c9c13122c6e
15847 F20101107_AABJEO wei_y_Page_117.QC.jpg
37c1cd9bdeeb1289eaa1fe4947034043
5392343011b5a08761d362afd2536b7d0e51c0a5
7108 F20101107_AABIZI wei_y_Page_036thm.jpg
299457f704fe9a316b568023b6a13ed9
4a2b5cb16af46f44080ca01a14180966a96eaf41
15177 F20101107_AABIYU wei_y_Page_029.QC.jpg
3b0cdc2663d9529910dccbfd97e6b9ab
d94c92fad0a289ec269ecf89c9c19ad911613e0f
1051979 F20101107_AABHVS wei_y_Page_126.jp2
40bf35fa64d65855aaaa00f6752a5860
2eb57023fd2c07c9e45ed3a0bde9842524c830b2
F20101107_AABICB wei_y_Page_098.tif
c6ee6666d0940f2f958b6f4ed16bf5c3
0e870a442feb8c582f69695e867aab984ee78976
F20101107_AABIBM wei_y_Page_083.tif
8ee7cf90da6173ca4fc4ab988096553f
e863667fff0830b1a18b432f96aa7d8b20fc4420
722711 F20101107_AABHWG wei_y_Page_140.jp2
3a4b7df7f232bae2c8db5c516483446c
4654a027e09b7d476d5aa61ce1788c0788ab34bc
F20101107_AABIAY wei_y_Page_069.tif
bb9709b0a4cde4afbcabc0f5c41078af
660f62e64369672e4add1612ab3a753f30664f1a
9096 F20101107_AABJFE wei_y_Page_125thm.jpg
cf19b22cb39e0765fb26be9adca4908c
c95ea76478c1b9732d06e7b46d14af08f0ff308f
5240 F20101107_AABJEP wei_y_Page_117thm.jpg
0472b0dc0588dda9c186397edf7bf6a6
402e54adc9fc3472ed6d0e4181c3c8161eeab8e9
28023 F20101107_AABIZJ wei_y_Page_037.QC.jpg
278519faa582c5252b0cbd396c8fe9ce
255c5f3ba367311fb648ab516462f43102ac3c31
4453 F20101107_AABIYV wei_y_Page_029thm.jpg
7439a6fea15c6163fac5851ddb496df1
b9980b014e83e4eb058d52eefc1c2e49dae0d7bd
F20101107_AABHVT wei_y_Page_127.jp2
9c9e9030d1a9baecc8e58bb36c36911a
fe02dae5a70d5864ef8d9014c933442705f5ef1e
F20101107_AABICC wei_y_Page_099.tif
93104688e79c09d973d387cb03dc20fb
eaf034c6f1b883bc930ec4e22863f842f21a8285
F20101107_AABIBN wei_y_Page_084.tif
d8d34b5e5e7b0b5f3021cce0769f2970
697cfd90b8fa076a49d2e04f636b0b0c4d8495af
722487 F20101107_AABHWH wei_y_Page_141.jp2
2c10adb786a7ba285e18a62c4ff6f534
b95e5c7c52038271e26f9da16e366ee22ca27d84
F20101107_AABIAZ wei_y_Page_070.tif
d43326caa5a5c99843d7e72d2d6847b0
dcb491b622cce9ced10b00a78b5d0c3c6ec6a65f
36954 F20101107_AABJFF wei_y_Page_126.QC.jpg
4af8a0aa3b6b975dce54a4a6c57ba702
9fa441a7327202fa69b2743ab724fb9dad82ef14
14608 F20101107_AABJEQ wei_y_Page_118.QC.jpg
d71c6e6f2f44e793d31d94c7495c87d9
c4ad1d696c659f60d75f012c345fc9ba70dd3673
F20101107_AABIZK wei_y_Page_037thm.jpg
8c11524d5bd8f4a3add90a6752038d09
dc89ca1d77d919fbfb1d2fa71c89b4c2fa843a54
8436 F20101107_AABIYW wei_y_Page_030thm.jpg
52fb4fc603a93577d5d3007053e2f8b4
d00341c4d868ad933c2f1e2bc07abfac9319d476
1051969 F20101107_AABHVU wei_y_Page_128.jp2
01f6ec23dc151400c15e8f3c405d1097
3dd20d70bc1dc50c21730a19af334eec5b638056
F20101107_AABICD wei_y_Page_100.tif
3509b2e43b60bedb015bcfb7274c1d92
f63bcc2b98bf75b37189e6b9e7a815912416ce45
F20101107_AABIBO wei_y_Page_085.tif
fa8adae45f883ff8ff15136340d49cf0
33f39c528cc19f5a16ef46aaf82fdf7b49acbd8d
708198 F20101107_AABHWI wei_y_Page_142.jp2
6b0835ddaf877452df129b1af575039c
27603097e87496f427e12a5c11af713316d6f7c1
36993 F20101107_AABJFG wei_y_Page_127.QC.jpg
0ba71424fd54ea31daa9bf450f131090
bb8b409f739202a57a9506ee88891577666b341a
5106 F20101107_AABJER wei_y_Page_118thm.jpg
d774ff554e28ea63508e8c099b76c2f1
1d868f18d22396016a8d36c523abeefc339754a1
23764 F20101107_AABIZL wei_y_Page_038.QC.jpg
ad6fd3578c5664cae5fcc6ce6d52c154
18e716f04d5c9f76a55c0273c791fbbe368c3639
20550 F20101107_AABIYX wei_y_Page_031.QC.jpg
e15c62ae50b2b0bfe8e38cdcce80c3c9
1eb57064baafbabd74c2a75eb8fde8221fcf6cb7
F20101107_AABHVV wei_y_Page_129.jp2
aee56e4053945d984a806bba940aa034
01113fbf355f1c5b7b6e0cbe1c5cabbc1b804847
F20101107_AABICE wei_y_Page_101.tif
ad528199c3dd9cba94ec87233cfbe54e
24a4e6f9d1d5b42b55b77f1080e281483703be7a
F20101107_AABIBP wei_y_Page_086.tif
d8d5e0eafc44d9a87df3856c4c7303b8
4c4ec9ccabdb523c100a9fd1c288fb1d4ca24b0c
686928 F20101107_AABHWJ wei_y_Page_143.jp2
b1efc6e94bbc9b0aca2b51cc757fd1bf
24f90f4aa7736925e3cb3319d95920e3ba80bd39
8886 F20101107_AABJFH wei_y_Page_127thm.jpg
39e6cdc137427930a2c4e382928a72c8
5e5237938ea47c11de0bb98b1d04ad37db71dd8b
5487 F20101107_AABJES wei_y_Page_119thm.jpg
f121f6b8f5d077c8fdfd8bdbf7955a71
b501c1eb88bd0b5c8d0c779dc3575e190b6c8727
6717 F20101107_AABIZM wei_y_Page_038thm.jpg
cc401602362ea9841ca66192126c91da
414d27608c3c255bdbfaf56e2c807940e8442a7b
5894 F20101107_AABIYY wei_y_Page_031thm.jpg
c245d295cb594f329be8e1167dbe1e9d
764ab8d53394403826e6dbc698b1486c6c5880db
1051919 F20101107_AABHVW wei_y_Page_130.jp2
0acd7bea6205eb55151729aeaf343fc9
2418d1b289a65da9462dfce12dd76a70b07262b8
F20101107_AABICF wei_y_Page_102.tif
fd786c85aeaffa9c2427c9181cdfcf13
1e2a9ce9cfae35666198c4506cce3b242c003ab1
F20101107_AABIBQ wei_y_Page_087.tif
ba4ce25ae3a83a232af25ab6ba04886f
3fc475ac14ed7d0bc31291449186386e3f9909b4
710531 F20101107_AABHWK wei_y_Page_144.jp2
370e06447f4e693f200b99eea48ce179
16bfc6cc2a8347ee45cf5f5f23c66269bf3743ee
14491 F20101107_AABJET wei_y_Page_120.QC.jpg
e642a7e418863fd2f68fa34a9f3b8d88
01afae5f541778c443fb1b0eee626ea40e850bd9
33829 F20101107_AABIZN wei_y_Page_039.QC.jpg
183d93ecb71b54d940dc4e5c671ba006
a590ce85a3432c0935c8135f0e3878051e163b40
28250 F20101107_AABIYZ wei_y_Page_032.QC.jpg
e073dc0633d97c7fcc9551c9189ca09b
6f325bafe991879a4a776e8cd2b1eaf265bcfb0c
F20101107_AABHVX wei_y_Page_131.jp2
5df2d5bdf9f7cfe2544c8cc03b4cf1d6
2c01ff6d4578b79955adbd3974d86eca6f750f40
F20101107_AABIBR wei_y_Page_088.tif
86204c02941fd162cb8f03bd00ae0499
e7c1b053b950b2fea06f043f62ac6175b90e31b2
713726 F20101107_AABHWL wei_y_Page_145.jp2
f4ee1bb2ff85cd184421e1537690d765
de519e7155f89487ed06167945d32a78df455a1d
36282 F20101107_AABJFI wei_y_Page_128.QC.jpg
c432bc82364ed52c1fea9104fd5dddbf
24c33ec174f1085908bc33879ee209cc9099b0e8
5177 F20101107_AABJEU wei_y_Page_120thm.jpg
1fa8d4dd197f8032976b805c698be0a8
ee8bcc97d887ed09c0db673722bcd663da77018e
8476 F20101107_AABIZO wei_y_Page_039thm.jpg
2989f199ca30368a9fead790c13f6fc8
0a8a545fc373c559f5f139bce680d239d3141fb9
249719 F20101107_AABHVY wei_y_Page_132.jp2
3399fcc1c8cd66d05904251f984fa6fb
6a7dd1ab762c50baa3d07b572cf3be5b058d7ef7
F20101107_AABICG wei_y_Page_103.tif
c8ca09537e6d0d4c5297a509e5970497
6d71a9d35463030dfb743bef0bf26dbc3aea5da9
702380 F20101107_AABHXA wei_y_Page_160.jp2
503161bb97d5cf911d3a3853316fe8ef
d0414f9eb1d137a3ee4a45859c90a96c2013d10b
F20101107_AABIBS wei_y_Page_089.tif
507aa5bb8109802df511424dc876ca07
66fa1abe85fb61fbb4f239d86f1ae5c02e064c8d
700766 F20101107_AABHWM wei_y_Page_146.jp2
4c66f30b75a42ace00c8bc81e5c7ea22
d74c0744e12720bd9d8ea4900af22d622e77fb9e
8925 F20101107_AABJFJ wei_y_Page_128thm.jpg
b555b62f9caf53dcf9c18e96ed90b94c
d3e51d1dbc0f7e7956c4bc808e401fe07aadcbb2
17115 F20101107_AABJEV wei_y_Page_121.QC.jpg
942d270bce179ec9c17c849e78bda574
90107312fd88a1159229e5bbd8ea8ba656297eae
37806 F20101107_AABIZP wei_y_Page_040.QC.jpg
4bc8ec329565467a7e47d29d86002157
7f4013a87a2c1c4f0d5cce17ce200f51f64418a9
F20101107_AABHVZ wei_y_Page_133.jp2
f89ff0dd15bba3574ca4fbbdab3fb9a1
7849e2caff697a7778e53a70e924105b397d6dff
F20101107_AABICH wei_y_Page_104.tif
1fb0296ddaa5f513dbe48f6dd1878542
59f1a27fd70d40b37882c70af1cbfa268b620338
698130 F20101107_AABHXB wei_y_Page_161.jp2
a52235eae16bd8aabb700aed97352abf
7b5d3fd74a77930d4f2b392de0c03301565f607f
F20101107_AABIBT wei_y_Page_090.tif
3a0ae67406d7becb5cc4d64c5c9b24ae
52ada4382b42b076bd89a371adf35488f1d7f9c9
702886 F20101107_AABHWN wei_y_Page_147.jp2
7716174f837e1c35636208567676556f
056ca6309a430f34262fcd542e22ceed69fc8575
34672 F20101107_AABJFK wei_y_Page_129.QC.jpg
6ab0880380dce3111304bd7b5e8dc267
e2fd8fc4724cc39c69aed6bb664e978b14e33061
5607 F20101107_AABJEW wei_y_Page_121thm.jpg
535dce019fa8de53239aa06060ef235f
3ebc97ddb314f9372cc2eaaea9820d2f41d65729
8980 F20101107_AABIZQ wei_y_Page_040thm.jpg
3cd24ef94ba153e19ee1006f135a5b6d
9828f53476d4e895e31ea093dfcc5c8a44e60871
F20101107_AABICI wei_y_Page_105.tif
e6d7254f76f2432bedc90265f9677dbd
3d5f9087ddab3b6feb3ba9965821ae4978bf4c55
728394 F20101107_AABHXC wei_y_Page_162.jp2
a9c43d8eebdf8dd7b0df3af24a9d5e50
b13c10567ce7b8d45a6e30d553a65e7ed8f64936
F20101107_AABIBU wei_y_Page_091.tif
448d44f9b73abb27c68243aeec9818a3
8cf3f9e0d50721e3257ff317c6337e65440c4a90
705254 F20101107_AABHWO wei_y_Page_148.jp2
aea8ee7438dad1d5b98fb4e15d30b65b
8ecbcda147ad87e74da73fd0a88b04f528a823dd
6631 F20101107_AABJGA wei_y_Page_137.QC.jpg
9500da0dad9f91e910b6982c9fe2a0d4
d2dab99ab8da55825450be4a447ff7cabc54f787
8795 F20101107_AABJFL wei_y_Page_129thm.jpg
e8bec1345966f4a36f28a34780b79f1c
2a3db9aac1fc4027cd00e349accb01b8190ad464
33787 F20101107_AABJEX wei_y_Page_122.QC.jpg
94f7739d85cfd5c5ee3cfaf833c054bb
e72fa4d88f86c89b07fed965ebd1f812c529f22b
9810 F20101107_AABIZR wei_y_Page_041.QC.jpg
0c639234bfad00e4b0d538c3e390b1ca
3923e432b70127092efd0fd3d1e1045572df8ea2
F20101107_AABICJ wei_y_Page_106.tif
3054d6b9a198bb1964050dfa330b2a52
e14be029f183cd8b03858b90b0062fc7dee9075f
729776 F20101107_AABHXD wei_y_Page_163.jp2
9e52c5de85af361340dc2030d23e38e9
fb37d78104497e36569b37f6075908c0ea0f2d84
F20101107_AABIBV wei_y_Page_092.tif
4b33b694c1c51a57ac5eae00842ad48e
93cea8db69ad92a5c8d8b4a3c285d3c87c81934e
705940 F20101107_AABHWP wei_y_Page_149.jp2
a46d211a16616198ab62148aefc5795a
b772ea00aaed27cbbfb7ae5c949079329bc1a151
30438 F20101107_AABJGB wei_y_Page_138.QC.jpg
49fe303217566467a2e993be0c303591
32d2aaf1b885d97d2fa87d47dd983230a079232d
33960 F20101107_AABJFM wei_y_Page_130.QC.jpg
dee794649f136baf3b2c470b35c58a45
a6ce5754da6327d9cca35eeb15735c07f81dade0
8522 F20101107_AABJEY wei_y_Page_122thm.jpg
f70ec2a9bf4d8fdf0e9c27fee372220d
643349f565f56b3c7a32dc4e3769c3c0ffe11e3a
2463 F20101107_AABIZS wei_y_Page_041thm.jpg
36fcc5e6bbbd38f165f0fcaa9a28ffea
b7049e63b0416e7a0cf1668577783f7a451fb9da
F20101107_AABICK wei_y_Page_107.tif
30032cd911b9f584a1574868e5b17fae
95905a19d086b88bd7dcbb5de158d3c71f3c64c0
729203 F20101107_AABHXE wei_y_Page_164.jp2
5e0cb3df4b4d5269b0616b792bd89415
7ec153ef7f101c1e46d3e03110c965067a08be84
F20101107_AABIBW wei_y_Page_093.tif
3bf59c0b76daa162d954afce9278f53e
2af65b0b993cb9ea6658413032bc8bcfb298c0fb
706063 F20101107_AABHWQ wei_y_Page_150.jp2
ea16c8dade21345d9362ad419758507b
55bf8b61300ef195a438798c24d3e5924f24cb4c
9486 F20101107_AABJGC wei_y_Page_138thm.jpg
38518549b7ca2c0898dab1592ea74343
9833bab3e39139ed2ff41c4a93fe46036de22eb2
8511 F20101107_AABJFN wei_y_Page_130thm.jpg
d8838512de378901c24f8d947751ffc9
41338125aa8cf8665d296e608979bb1a0179140c
36616 F20101107_AABJEZ wei_y_Page_123.QC.jpg
8266e10ad9e70dbd031ad65cc3ede664
2889e6f0124769dad1967e055fff66b091c6974e
31548 F20101107_AABIZT wei_y_Page_042.QC.jpg
83814adcf4b1fc355c07872e931a341f
9897bb74c3920d6b6ecfb2ebbefcb5878ba5566a
F20101107_AABICL wei_y_Page_108.tif
f93a58d44195a4c29008eb6edc4eed56
f6f6896fb17f13c9753a9fb5ee51a0ca4035d5f4
731963 F20101107_AABHXF wei_y_Page_165.jp2
f2a9b6170b76b119ab536b90e852c8e1
6bf64125314c244be0e0645d2ecfbab6854031ce
F20101107_AABIBX wei_y_Page_094.tif
170f5aa8f2f799456e725e9027830b11
69a017a58fb720861fea94961806b099a5a8da26
714390 F20101107_AABHWR wei_y_Page_151.jp2
7b8c2ab964b87987d2d7c6e77c8782d2
8f3f0e8f229c2531d0ba52ede9d910bff2af4a6a
F20101107_AABIDA wei_y_Page_123.tif
cc589d99d6612639ad2bf8611698935e
2b0881a1afb21640e83b0c74aab6158b693b934c
31738 F20101107_AABJGD wei_y_Page_139.QC.jpg
c06e31e2721af68c37f32c298b60cac8
bc83218f4fffa759851848ad1a14056215b6acde
36385 F20101107_AABJFO wei_y_Page_131.QC.jpg
c032c909f7ce14c466d7484cfaade32e
a2e5e45113cdaf9a5714fbb792c231b11f24447e
8321 F20101107_AABIZU wei_y_Page_042thm.jpg
b19944702aa11d5b42d82124fc931fb6
4d625ab0dc63598ba81fe49db87ecfdfc38ecc2e
F20101107_AABICM wei_y_Page_109.tif
e58407540cb6d5bc8cffbf33340b419d
cc1b39ecf81e4d2573e62cea67294cf490ba937c
719216 F20101107_AABHXG wei_y_Page_166.jp2
59ef94cec74cedf7206ead798f24f5b9
bf8cd92c1959b98d3aed41dbc89a413e728f99e4
F20101107_AABIBY wei_y_Page_095.tif
27be5fc3b0f0236f36f94e4f9fbdda80
7c7741c3616193ab6becc41afc796e12330c40ce
708273 F20101107_AABHWS wei_y_Page_152.jp2
571d9b9f1fb51c87d56c0e60d4dd4be6
f1f0147c7a57a232faac6f89e71e1e3361197e88
F20101107_AABIDB wei_y_Page_124.tif
fb4fcb44afb15c62f02506a3daaadb06
b21f9985b53a779f907434df223a6845bec37e26
32011 F20101107_AABJGE wei_y_Page_140.QC.jpg
a0e81d5c7bef5b11732bbaab500b38b9
0d395d3b997706e6aa6ac6775c9621705e40f5fd
8588 F20101107_AABJFP wei_y_Page_131thm.jpg
94bf2f7e2def9f1c0a6b207682e3ae7b
6f41b25d7f2fab4d3f35508e3eb6ca8f3724c9bc
35985 F20101107_AABIZV wei_y_Page_043.QC.jpg
00fdf77f869d78160c047b999de71db6
bbaf4735a5e104cac2a627e4a16286ee8ec3cadd
F20101107_AABICN wei_y_Page_110.tif
abcaa9c92e216e599de1476accfb47b3
6555d5b1403e74a16bf41d0fc07bc8b695d2efa5
722864 F20101107_AABHXH wei_y_Page_167.jp2
b565ef0eb6ca9f0d8c6dbc0984dd5aef
a01f13c8ca288f9e23f8d4be31f6aaac3ec04dcf
F20101107_AABIBZ wei_y_Page_096.tif
6d055cfebc16a0cb5a3123429d79d169
5327d53ca9547270a0347247f3916dca285c625c
710612 F20101107_AABHWT wei_y_Page_153.jp2
0c2c80b6878d7f698ca55d178049e26d
beff1c0103be5c178fcc064eab404849ee7ce779
F20101107_AABIDC wei_y_Page_125.tif
2cb81974a1cf2e6bced5b14abd361090
1c6ff6881b413b383e57dfd130963c12bc0c5dbe
10134 F20101107_AABJGF wei_y_Page_140thm.jpg
62baf35042c7a9e9d3bf00c846f037d0
9427f70e5297277dabb35b2452c9e96a0f5a6d81
8150 F20101107_AABJFQ wei_y_Page_132.QC.jpg
65a234c94fa53ea44c2844f8f4afb4d9
63f8385dfe0da1cef801a6a2370a5426a91d1e68
8963 F20101107_AABIZW wei_y_Page_043thm.jpg
b779fca26bcf2723a788a696ef1de569
b2ca698082db6412fd1e5ac430433bdf7cc55c52
F20101107_AABICO wei_y_Page_111.tif
b0e976bc4c9f0ae90d8fdfe53df4f98d
19d04ba8d7ebaed3788aad06de228802f7b34ce1
724204 F20101107_AABHXI wei_y_Page_168.jp2
919286e42e2823d29a237ed3d95ffaf7
17448fefd968c936848f4f7a6b6bb069f03f48f2
703931 F20101107_AABHWU wei_y_Page_154.jp2
c2cadbc9d4cd53445aa47bc6d6387d27
78ec221eb229ef67d5b47ce4fe1f005e9af7e4b8
F20101107_AABIDD wei_y_Page_126.tif
0a1a8182ad25374701aa96dc50a2326f
f4cc3b35e9826a01e0b0cd6549890f0ebee97dc4
32015 F20101107_AABJGG wei_y_Page_141.QC.jpg
b09f68acd459f7416d687fc44ee0890c
e1fcf634a3b329738aa17bb0895b76a256838332
2191 F20101107_AABJFR wei_y_Page_132thm.jpg
9c1e49fe6d15106241885b945b520735
20378470c57bb58dcdc6ad10d6896c6d6e2318bd
35987 F20101107_AABIZX wei_y_Page_044.QC.jpg
0d1ac4b3ff3fed82aea39bf1b2423526
d248aea2ce988a8beea8884dbae685926e5c27e7
F20101107_AABICP wei_y_Page_112.tif
b128eeb2123b6680dee800d0f42bb623
0f9dcc5564b0faa433f6ed38ee67ca5038c124a6
723965 F20101107_AABHXJ wei_y_Page_169.jp2
a8fe4a29fb92ba92ef5f434d95824d44
732d8af866d50e04c5077a17832f377d9aa96fe1
706572 F20101107_AABHWV wei_y_Page_155.jp2
f8c1ad7cf0f35be772af6961a17b860d
891f47ec7e677e4dece43737c9945385d79d4f0f
F20101107_AABIDE wei_y_Page_127.tif
914149fc7fd7ba67c0866a19c5aaea4c
7a7159f36ae74a7c653d1e37b5b688330b512f90
30792 F20101107_AABJGH wei_y_Page_142.QC.jpg
d234a6f696926629ac270ca41956ff95
1aee9852cf6b25cba1f555db3ab8feb6e3af8474
34097 F20101107_AABJFS wei_y_Page_133.QC.jpg
28cc18165d26736a2a287e02a651f9fa
d9fed5085aa3e79a68eba9bb220c18c9347de71b
9107 F20101107_AABIZY wei_y_Page_044thm.jpg
75cf80441c382a194ae87e6bdbc78c5d
bcd27f70f4cc5e8b5b078f21ba79bc194c738918
F20101107_AABICQ wei_y_Page_113.tif
61fec49fc73c49126a8d001d2755dcfb
e6558ed00c51dca521d714992c36ba9235346616
716300 F20101107_AABHXK wei_y_Page_170.jp2
b1c873edb3a499bf061dc2bbd171fdf2
241bec1558b49698aa898df03b333e6218685079
702792 F20101107_AABHWW wei_y_Page_156.jp2
32266d5a6a4275a284daa68ea092b861
34e258ce09fe5bc817bc84f11c5748846002f965
F20101107_AABIDF wei_y_Page_128.tif
9e16c7e56f1011586677c9f345f10f6e
20e3ef41475f8f4cc0f994c731e334091815c8bc
31265 F20101107_AABJGI wei_y_Page_143.QC.jpg
022d2043ae83cd6e5385f2365f5dbdc2
4d174fd5d3ba38f6b31c5ee465d7899c26ac56db
8439 F20101107_AABJFT wei_y_Page_133thm.jpg
1fb3bcf20fe74383adc7af43e5851bac
605f6f14f765340cb1806014696996ef05684a8a
36898 F20101107_AABIZZ wei_y_Page_045.QC.jpg
ef7f827ddcfdce34a3d4c126d7986e7d
49a48b60eada2323067b424855b27fc6cbd34ea8
F20101107_AABICR wei_y_Page_114.tif
97bbf3ced9d81b49c7e22302e965dfe1
a4248587e580a82041ba5dc5a3f2720f5692f58f
717706 F20101107_AABHXL wei_y_Page_171.jp2
82df9ab50e52b9a76e0b726720366492
c12893363424412f140ebb102098d563e9211660
704970 F20101107_AABHWX wei_y_Page_157.jp2
977392a93cee5901804ab75e9ad36ce0
21ce3879c35ce62bcfd6ef9a11e4abca897d2455
F20101107_AABIDG wei_y_Page_129.tif
06ed03fefdf5aa08f2389d66d3ece93d
c0e2fdb37a2e30a26e3bdc2aff41bcea61d3f602
36180 F20101107_AABJFU wei_y_Page_134.QC.jpg
b85911dff5a92b509f7589724a3f475b
b620db04ccf835548d2f6415c2c7aa066a776efd
F20101107_AABHYA wei_y_Page_186.jp2
864ff85b30f0dfc54d2e76ad08db291b
3dddce570c4f65314d0665920e49917db646282b
F20101107_AABICS wei_y_Page_115.tif
072c6f68dd8dc3782c90201badbc0eda
57849eadac66774e698aec34eb86462e422120c0
716123 F20101107_AABHXM wei_y_Page_172.jp2
6365b205e6c28f4c2fa59aa2836ee6c4
bd94448c5f942302abe309e9cc2595b0bcbaa9a0
699160 F20101107_AABHWY wei_y_Page_158.jp2
d3994221ba08ee198b629cdc9f336447
4eb9b02dc60a46b3c401b8daaf0fc98e8d674c0a
9900 F20101107_AABJGJ wei_y_Page_143thm.jpg
d075ca098cef58816207f9b598b2acad
a7ceabf69a2b988467d37ce80f4b3f889024f334
9079 F20101107_AABJFV wei_y_Page_134thm.jpg
fd05d248ceffef07c611fdccabe4f19a
b5b8f739888ed9c5f72cb85478ab38087fc44518
1051959 F20101107_AABHYB wei_y_Page_187.jp2
b6fa292cd24a81c7bbe08438ca95d94a
220553a25c9bd7f1be26f093bf240130872346da
F20101107_AABICT wei_y_Page_116.tif
25ce2f24aa7062b7108db9caa6f1de71
1feb8d937bc45b5e955addd4c556922f1674a327
719914 F20101107_AABHXN wei_y_Page_173.jp2
d8c2d9f995fad36089d23473053f2b8d
5af9c68e3916d84e7a0b8944694b38b8175bf50c
702132 F20101107_AABHWZ wei_y_Page_159.jp2
8fad21e597787593ced26b6986e74106
affb6f00a7b6d2a292528cde6688065233674ad0
F20101107_AABIDH wei_y_Page_130.tif
28c671b32c5e122123f0e2f388b847be
45a209d73d1ab3d019f4da6b02306958662857cb
31013 F20101107_AABJGK wei_y_Page_144.QC.jpg
493b7666710935bfc3a23785b65a0b62
c9f018dea2efa4f0e3771a7110055613f84a78e1
34516 F20101107_AABJFW wei_y_Page_135.QC.jpg
effc9a1a382c7fe9afb2ffa6a2351bd5
b1e06119420c531d2e5d95ae6d1a15c0f286f005
F20101107_AABICU wei_y_Page_117.tif
b6599f0b514da82e7655de75c830bac3
32122612c549cd6cfe9a6cc780ecaebe04b684fa
727309 F20101107_AABHXO wei_y_Page_174.jp2
3659d280893dac15fcf2d09876bb0e86
0aa46f1916d63fa455b89c27a125dc1b78c36729
F20101107_AABIDI wei_y_Page_131.tif
900ed8e08c57242c17c1785184cfc405
d6b268c280a4f9c25af4b5fe3e7dc3d20b0f3b02
F20101107_AABHYC wei_y_Page_188.jp2
f3a058b08af81757a293c73dfb180ce9
df25b2edf0603a202b5931056b1e1761c0d0ed62
30875 F20101107_AABJHA wei_y_Page_153.QC.jpg
b48fb05e66189826b80f50a6bfa39fc8
2f00f503844b11decc3f38317771e244e89e7e23
9691 F20101107_AABJGL wei_y_Page_144thm.jpg
7b416cc42cff27a0143b5e7a8791d2f5
905c237d18b4af7a2ed291ce2b0a9b9a148d054c
8721 F20101107_AABJFX wei_y_Page_135thm.jpg
b863eaf25c2bb903c2f68a6ba275f8b3
2068304d0a33e31ac9804034267ddc97531f12d9
F20101107_AABICV wei_y_Page_118.tif
05660cd95c902dc23d513d6a41568b5b
860aaec1e00d6f05dd188eeccf80ce11e6fbb447
725452 F20101107_AABHXP wei_y_Page_175.jp2
908c98321bef29d5c027263025ed29e3
dd425e933d663a633c7b8e62fdf34128e173eced
F20101107_AABIDJ wei_y_Page_132.tif
cb8f81f483bfd93924b94f61cc8b2f15
233a100325d1b96d610e9217592d63108ce373ef
F20101107_AABHYD wei_y_Page_189.jp2
3f75c8bbc2527b5fd6ff38aa18322e87
9df07806d431a6f7c2f3c39d6d5560fecb2e4d9f
9717 F20101107_AABJHB wei_y_Page_153thm.jpg
0668043c4e89f7ec285e3c309b04646a
5087b4d4ba4bd04d0792d114f3e4f3edf0da5c8c
31174 F20101107_AABJGM wei_y_Page_145.QC.jpg
af8e5e6496d5e52e7c5a16b5c7b4a4db
843b1fe87279de9e00da2928620859c667dde4d0
36435 F20101107_AABJFY wei_y_Page_136.QC.jpg
237557e12bf80f58b29746db3a553705
863fc9758c0214c2d85939b09e74b6e82b43aa7d
F20101107_AABICW wei_y_Page_119.tif
c9bbf7e38b8d8afbbd5cb587d7d5e98e
c89664eefb17da79565b0e658f03e01052144542
728744 F20101107_AABHXQ wei_y_Page_176.jp2
955b9c36fead9223746ff5e9a7b94e08
64cbdd710fa77577ffa2b1ebc33e5e486c905e27
F20101107_AABIDK wei_y_Page_133.tif
fc18efe7fe425dbb352ac11ad07f0d0e
36d4ae0c02390b195c0767ab246f5112ba7db402
F20101107_AABHYE wei_y_Page_190.jp2
313545e843199a602bc46f5ffb81684a
d46343d776199376f8df688a8d2c305a64ca91cd
30441 F20101107_AABJHC wei_y_Page_154.QC.jpg
db7fa15e23f6df563983e4705c74c2ec
c6b74a7a4d558e66dc8f8ebdefcfb2fddce4147f
9750 F20101107_AABJGN wei_y_Page_145thm.jpg
c5900f8b9a6b3f5ff16a0530758c652e
834b75725958b6b9c94a77563f35f650b982002e
F20101107_AABJFZ wei_y_Page_136thm.jpg
dfbe2564a52b9230a0f78071ff64e71b
3d1905f150e7696ba846b5f00322b669c3a4430c
F20101107_AABICX wei_y_Page_120.tif
141c630ea055edf5d243a6017e9c005d
cced2b614662300eef8183282c63916163017645
722346 F20101107_AABHXR wei_y_Page_177.jp2
a483bcc16d7a6d675f506861a400e2b2
55144d65f208f56cc51a2f627d26637e373b8d38
F20101107_AABIEA wei_y_Page_149.tif
f8ae5d872a6e5709d6ffdefaf0cd254d
cea5808857688238959608de75ed061a81009902
F20101107_AABIDL wei_y_Page_134.tif
72ae300736d90b4ba3cfde688fac1f8a
37ba800d88e74f98ddc828e461a552470688e827
F20101107_AABHYF wei_y_Page_191.jp2
8a52262845a35de9508720f53a068a81
4a257947e78f7ac410c51e296c37973514801c0a
9653 F20101107_AABJHD wei_y_Page_154thm.jpg
22c64cac2b586cc013288d69db78b7aa
efb6f4edf78cab7ba624c2e7800c4f7bac162a71
30418 F20101107_AABJGO wei_y_Page_146.QC.jpg
d5e524c3ad17cb694250ce2e4373a044
b3a59aa663ca08e5223ac2cef3196fec999aa937
F20101107_AABICY wei_y_Page_121.tif
38587ea47178b63bf2519511c8e8a6ea
6541394deca91faaa5e7dc9ad61f15a4a94cdb86
719580 F20101107_AABHXS wei_y_Page_178.jp2
a458bc0bd03739756a85c810ba0ece5c
61b2cd985546fd73d294e86a2aad2c45ed256c6d
F20101107_AABIEB wei_y_Page_150.tif
f0853b22a71a47fb3ec341fafccd8115
a43a9b6140d7178066167c009c3c73958c9b893e
F20101107_AABIDM wei_y_Page_135.tif
46455ca00c4bcc550ae79bc512bcd135
71b7fdabd730baefa0813730ed8ef9e708df21c8
802897 F20101107_AABHYG wei_y_Page_192.jp2
9d4b7026331eb00f5552fc98236b0947
9fffd87bba67dfbd897d11fc6e23b9cde3b0bc50
30672 F20101107_AABJHE wei_y_Page_155.QC.jpg
2a031242939f8d66eb6e46b9c3f11555
6618950a4dc5c33d477f55a8b175f7b2d3e26f05
30563 F20101107_AABJGP wei_y_Page_147.QC.jpg
e6d197673eedb5672cbb36532326b13f
3e868f501e0e6a5ef846ed60dfbaa3b4dcaddc03
F20101107_AABICZ wei_y_Page_122.tif
23fc6a56d0d928f74cf40ffff09bd0fd
dad832c56d35addc4ffa8bcef394b059ed324eb7
719028 F20101107_AABHXT wei_y_Page_179.jp2
01f72c773b80975c0bf3105eee94657f
49f50e58e207928ecd85f8c09717e720d7f7d4d1
F20101107_AABIEC wei_y_Page_151.tif
3cf58cfd986048f9a020776289157ba9
7478adf491709520268ca867bf88c2e62a9c9792
F20101107_AABIDN wei_y_Page_136.tif
c3b421e58bf2ca14ff917de78866ac6e
79ce01e9458f790b43344c81e3252cccaa62b035
448385 F20101107_AABHYH wei_y_Page_193.jp2
bc40227a8e37863ae53c5f7ca576f030
d934ceedaafe94a0ec46db908a25e5557332f037
30630 F20101107_AABJHF wei_y_Page_156.QC.jpg
2ed49ba9109cc9a2e3c0241838f3fb32
5dcd8005c07a9fc07fd18b125b22e0223c9c9f19
9621 F20101107_AABJGQ wei_y_Page_147thm.jpg
79316525463cb2409cd53f8f5f959cb0
b24aaaf83acfe324b5334313e1905c384113e1b6
715686 F20101107_AABHXU wei_y_Page_180.jp2
db2bd01eb36fa56b684cc4bc59e18467
267fee197207779237b585dfe4a16a0109b4ec41
F20101107_AABIED wei_y_Page_152.tif
80caad96f7ab6ff16f7791e48ad0f3de
c6033dfb628fcd288e898e6119861d1b8c41f7d6
F20101107_AABIDO wei_y_Page_137.tif
7b679168edf063f810e2d5cf112e4af9
ec8175874c09eb1501708db86f70442fb99abe07
F20101107_AABHYI wei_y_Page_001.tif
ba83cb86c4049c7b5c79607664b83d41
991afef56a7cf17a7f5b75be793e15359ae2a61c
30347 F20101107_AABJHG wei_y_Page_158.QC.jpg
38d8d04feadab768502a7ca87ac75bd4
a8aa51aaa3924d39f9d94542dc579fbf1529a62d
30710 F20101107_AABJGR wei_y_Page_148.QC.jpg
d4787f5763cfb1e6aa020f74d0206c2a
775bde66efa6f7fed2bdcbd92c56b5455883cdbd
713996 F20101107_AABHXV wei_y_Page_181.jp2
bd92315f4d41a124ba1b925aefc866db
1f89f79d58092e8859023a192e211115ac1ba0e0
F20101107_AABIEE wei_y_Page_153.tif
d5f0e97abfb0f381a65f5b00802764dc
14907e50dfd31d78bca9720664ec8b4c433a9f66
F20101107_AABIDP wei_y_Page_138.tif
c0dc00182b7eaec95c99684846c00c7b
369767fffdb11afab7769e5e4ca72bb3dc8fe526
F20101107_AABHYJ wei_y_Page_002.tif
d4fb526729bb095520ab4247172f6724
723b1ef3df98bbd4b561b4beaeab6a27d0579d9c
9601 F20101107_AABJHH wei_y_Page_158thm.jpg
46b46ab6dadefff4003a6cc09b226e17
7d05482110b2ebc01a1e8ebef2268c5f27ebc7bf
9648 F20101107_AABJGS wei_y_Page_148thm.jpg
6df1ceba5803579bf71c8005841b5b63
e941cf3282d7c89ede0ffc359d846eea1a1f66f8
711416 F20101107_AABHXW wei_y_Page_182.jp2
25b73e06b89361b40e56e0e507d96bb4
3ef51d747096fdac9ecbd11bcd06a60b7835722e
F20101107_AABIEF wei_y_Page_154.tif
93da63aeabeb69942a685fc02cdb989e
780f8c1e3f270b8292429618eda3a33bf028b787
F20101107_AABIDQ wei_y_Page_139.tif
53394f839dae999e8b154a856e2e2f97
46ad4f5c8d879f9e94c20e1aa79c0206edd2693b
F20101107_AABHYK wei_y_Page_003.tif
53e223e9a2322ce3daf59835fac330f5
f3a65b715df8a1e9911570970556bba3a4b97dd3
30531 F20101107_AABJHI wei_y_Page_159.QC.jpg
8d7c9777a4673f7ba8fbd1ee64889c5a
e6f33c9e56914f3941b8626cdf94f98729c86599
30770 F20101107_AABJGT wei_y_Page_149.QC.jpg
c485c3c72252aac4f6d77d20f15e757a
3f0401ec40960f13e7713cf3843df7b954c9668b
715561 F20101107_AABHXX wei_y_Page_183.jp2
a042733405399c9afed20275876804ba
c2912581b02c4d53cb70786a90f53d496b7e146e
F20101107_AABIEG wei_y_Page_155.tif
ffc0a7d3b32d03548125f2199083b9c4
0a8d5ef8af31bdad281350bc6f7b32fc72ddf29e
F20101107_AABIDR wei_y_Page_140.tif
5ea4d5c58b4357f8fa655f5f2f5509c3
cbe327003664dbb03d711272900a0da2506313c3
F20101107_AABHYL wei_y_Page_004.tif
234c5001d6658b1dfcdca9c039045b6f
ce0859532b2a0540eaf4a477e35aa15b136dc9bf
9662 F20101107_AABJHJ wei_y_Page_159thm.jpg
505a66262588f204ee4fc40311d86d5e
1d3772c1c5a989ea03b91e1131e37ffa15e2b7b5
F20101107_AABJGU wei_y_Page_149thm.jpg
ddcce38ca21360b44810a81313940874
074a89d12d383b4949cdbcd3b85ba3b9e2ac1306
680650 F20101107_AABHXY wei_y_Page_184.jp2
5840dde53dbbd4e9af2a983b1322dc74
04c45291f4de87f38107824b63b1465897af0e56
F20101107_AABIEH wei_y_Page_156.tif
3442c82d7be4741fed6c9c8d61012253
a29febafea3f9162c3443c9c343cb6e7863640f6
F20101107_AABHZA wei_y_Page_019.tif
c470d20f8ef115e3f0c08e021a9cf5ac
9a038677a9de7a0a1a529aa19112c189a22ea32e
F20101107_AABIDS wei_y_Page_141.tif
8bf1d123c4110a0b8c1b38fcdfef6be6
9329da16fbcb885106d99389fdf3a44b9d394034
F20101107_AABHYM wei_y_Page_005.tif
c540e781f94f4734980d36879a5b5f1b
7228ebd17ae5f42e401cc508d88180bd051de580
30669 F20101107_AABJGV wei_y_Page_150.QC.jpg
496bcfc40ee692a21d2a27d16df6a0cd
b0194cfe03be907e735976666eeee02bacdeb87b
715355 F20101107_AABHXZ wei_y_Page_185.jp2
55ad394bd2c051f56728143d44518dfe
819486164a43bccffe5bddd7b2f9b62c800387c4
F20101107_AABHZB wei_y_Page_020.tif
7b9bb94d1f9216472e2708314154e19a
d012d7149c4b8a10ca7df90904f47f857678c0f2
F20101107_AABIDT wei_y_Page_142.tif
897e4c22f48b2cef60f829516fe7e3e5
e14da3fe39bf899eb6429ff94c00fad7d2cf8a0c
F20101107_AABHYN wei_y_Page_006.tif
b653c77c2670f46fd54691c8adbf42e1
7a2a2cc07cf40afc10f9c69293de34457b819c8f
30402 F20101107_AABJHK wei_y_Page_160.QC.jpg
2c2df5f1233ee5f39c22e3af5d7e6733
5bc2a82c0c937d0997a695335f83ec7b67b6687f
9631 F20101107_AABJGW wei_y_Page_150thm.jpg
2f6993aec2fdb01a8740919d6c183068
83ce0f56b052270484cbe6f8b342e1adb341b640
F20101107_AABIEI wei_y_Page_157.tif
9aad48f07e035612facf1863aeee55c6
a986d458c5dadbf38f3a29ed4659d1f775788d16
F20101107_AABHZC wei_y_Page_021.tif
a46d1d3487e0aeb945f73688199092e6
360a468c0f82963a85f78700c6464747176bc6e4
F20101107_AABIDU wei_y_Page_143.tif
ec992bf7eb00ba1a9c0786c8128bfc5a
baa20cef10716355df2b64765e9264266a4fc8cd
F20101107_AABHYO wei_y_Page_007.tif
bcf13c29fb79474adba66eedc8ef5bc3
bca7f66249929ef4e8792181452321abdd816585
31702 F20101107_AABJIA wei_y_Page_169.QC.jpg
19cc8c1bc132e177f35a93516f730ac4
f30725c55158c5fa6a0b20073d844f05c771287e
9577 F20101107_AABJHL wei_y_Page_160thm.jpg
b00930ee6af4b3eb7994259e81b5a960
ef147909d1b3fe02c73ba81f002ced7929b87e9c
30889 F20101107_AABJGX wei_y_Page_151.QC.jpg
c8b33f0ca99989ccb012f6b27336ea2d
b5945634774a63db78cb7a92f58444b21964843e
F20101107_AABIEJ wei_y_Page_158.tif
bdad0d06758aa7fe4ee4c12caf329d0c
1765a302b1cddf41deba07b35ec46c48b20bba14
F20101107_AABHZD wei_y_Page_022.tif
d2fcfb85a6b1edeccae543076cde004c
54fd502d04a236167bc148f852b6c1c1af35953d
F20101107_AABIDV wei_y_Page_144.tif
d64f291aa1531342e1fedef59c8bd386
d34dd2e342d6fa90a5ae91bbc03b3ee33db76c23
F20101107_AABHYP wei_y_Page_008.tif
cbebfb00e103e7f1181f8dc3216a26de
036e00791e0e8162ee375f3a8302122059d94c0d
10022 F20101107_AABJIB wei_y_Page_169thm.jpg
4e438d2e9f5dd2cb34cef918ba53c007
46688aa8ed450478749f69808b8957c4ef56b75a
30282 F20101107_AABJHM wei_y_Page_161.QC.jpg
7f1956ff234d750987a8babf33eb539a
41323a7041003e0edfe219ef1b1445d362ff5cb5
30790 F20101107_AABJGY wei_y_Page_152.QC.jpg
77d2afb01afe0b018964c7937e7e8ab2
1ad2871e0cdd42de4ae16b7e60222185a3074248
F20101107_AABIEK wei_y_Page_159.tif
7193ad942f6e2405123aaf13bc706815
c7687532b2b0006d969de933c26d9e6be55ecf84
F20101107_AABHZE wei_y_Page_023.tif
00edd5686b7099a36eee4064d1abd73b
0bf30480d441e0258bb81ded9ac22a1730f97b5f
F20101107_AABIDW wei_y_Page_145.tif
832e234e7cc408c5161d8b9ab30fa59e
d022156a2227dd850683471fbbf0847c0da0114c
F20101107_AABHYQ wei_y_Page_009.tif
6b1c8d453b3165a28ff915c7bb50f4f7
f4f84d3c51bcff61bf1549ac5e65106acc4ed861
31141 F20101107_AABJIC wei_y_Page_170.QC.jpg
a63bb94e1f7dceb65790d4da834d2d0a
64f63b9f90433477fdde0e37bda1ec59b707c028
9602 F20101107_AABJHN wei_y_Page_161thm.jpg
c223b2aaa4045c6fb78696bbf3ec600c
c6bb6d0d7faeca1ab80e4619aabcceb72166b638
9681 F20101107_AABJGZ wei_y_Page_152thm.jpg
11d83a6e6e46461c93a325e8fc4514cf
7e38bc69a10a8175615f392b00b1680f2320123f
F20101107_AABIEL wei_y_Page_160.tif
f1025350a641835f8d82e29eb05e241d
acdb635c1808941300f5c9413329fe8558ab1f7b
F20101107_AABHZF wei_y_Page_024.tif
75636aec281217dd361cf4dac2b5ef6a
f74a88a6a59ddc158f5b11f720fbe117f41b91b0
F20101107_AABIDX wei_y_Page_146.tif
0a4001572a92129d71ad4dbfa6c74336
d0126e0d84d85d9fed83f6ac279f8e7035c147fb
F20101107_AABHYR wei_y_Page_010.tif
3be02b5911b7989a3808a3de468d92f4
847a31404c928995b1c818796c52f19cd8aaae23
F20101107_AABIFA wei_y_Page_175.tif
c2e12f11afdf6334c9bc7f165e627192
cae77a429248e0b28162ed9c436645093654cf74
9881 F20101107_AABJID wei_y_Page_170thm.jpg
da316b049bcacd261d566cd47984b8cb
722e79870db2814e456e0d733fa8f20a4b165ffe
32226 F20101107_AABJHO wei_y_Page_162.QC.jpg
bad2cab0aa009442e35a8fd7f1d13fe9
ef3c83c88bbcd8a055ebc81b0a44fb2348a67e9b
F20101107_AABIEM wei_y_Page_161.tif
5090a972b48bb69d183451aafa48bbea
ca96b67d9a6a00cb41a2fbb965375e41a4272778
F20101107_AABHZG wei_y_Page_025.tif
24ce5af71a9a4ab95999e378b22823e0
68e28c69f3f6f815f0eadb1b86c49edb39d73e3f
F20101107_AABIDY wei_y_Page_147.tif
0912c0a81801d280cd8c1a16c2200757
5fa12d8f293b8cf66cb1abef8d5aa7f6463e146a
F20101107_AABHYS wei_y_Page_011.tif
72a323f49b9d66404802d2177ca4b269
4292ce2cd17f8fedddc057ced2b7f5c3cbe85af2
F20101107_AABIFB wei_y_Page_176.tif
0f377028267eb87adfcc5b2876f15c4d
50e496eadca6890e48104eda564bc431cbe5cb3a
31312 F20101107_AABJIE wei_y_Page_171.QC.jpg
46792c92f0efcb1cbcb7753d3c83f232
a3ccc97f18d35459564ca1852f13d8cc07e3d0e1
10188 F20101107_AABJHP wei_y_Page_162thm.jpg
35647c9bd60733eeb67a5544adacab7a
dacaf5f4feb5bcb249a534ab63838a69bcacacd4
F20101107_AABIEN wei_y_Page_162.tif
7e25087b7d37c3cca0c14c8857ddc087
e5604e619de955b9e9142f3c543e73c6b3cb7a9e
F20101107_AABHZH wei_y_Page_026.tif
662acfa706eb608d7004aad198f30a7b
1e510296feac98adb4bdad2751fe4ca4fa84ba0c
F20101107_AABIDZ wei_y_Page_148.tif
66e13b42ab8e0b8d93124d7aee15c1df
6b81b2a8e359bd31872f15e2e7cc012edf83e953
F20101107_AABHYT wei_y_Page_012.tif
ed6e566357788c7d1427c595de0d9605
c50641dbe82930bb49cafb21b3623a5a196b5719
F20101107_AABIFC wei_y_Page_177.tif
a8d9cf760aa63496dcd203222671d073
2401a38e6d8271408efa668543fb1d2cce1f6f59
9924 F20101107_AABJIF wei_y_Page_171thm.jpg
ba88c156b7e4c74ca8226a26301cf534
3a8cce32ba28ec664e1c5c0efcccff4a23cd5c26
32218 F20101107_AABJHQ wei_y_Page_163.QC.jpg
d789eb4684fc9170176439de256941c6
0c74d2dd56bd1dd78f66e28482b3b53173988072
F20101107_AABIEO wei_y_Page_163.tif
32a69f9db1e463347287348c2ecf4b9d
8e43c32f1fac882e63916dfa091a1ad00d3b410d
F20101107_AABHZI wei_y_Page_027.tif
0f959db3d06797b6c63f47c28bc9705a
0adbdd7a8b91623c07b954cf077d8657bbbf4831
F20101107_AABHYU wei_y_Page_013.tif
696a7df1efa25261d7d9f03be31cac32
c44663ee601a7d3f0300093bd2c0d5240a246fda
F20101107_AABIFD wei_y_Page_178.tif
d81c4f084c5678dcbb0d23b0e52270f1
6e5a5bd42f1552f20ebe4c7cf53dbfabbac67113
F20101107_AABJIG wei_y_Page_172.QC.jpg
8c24db9b3eff553898f64cd01ac14d77
9ebde612103e789c1b4d91fd7597725afd711258
32375 F20101107_AABJHR wei_y_Page_164.QC.jpg
d58fd95b9b69c3edc56b29c1924ed1ba
21e672b41e0fbf2e42f2ee7818bdbdba093ff353
F20101107_AABIEP wei_y_Page_164.tif
47cdd8a00f1ea8eaea6de4cd84161b6c
1e2fa90ad12de63671631c6dfbd2cea8c6314086
F20101107_AABHZJ wei_y_Page_028.tif
80c26ef270b57dfc8f76313b499dad21
24ee6fd3cc3b1589fd9f54de566961fa5008e3a9
F20101107_AABHYV wei_y_Page_014.tif
bcddcdf36a618de403d591e212c6f138
99153c4522b7b53a245e0856c4919e3f82340513
F20101107_AABIFE wei_y_Page_179.tif
6c918f0520d373112864e242fce5b21b
c81f2a7e256b38284e8c085befa85600eeae5d69
9855 F20101107_AABJIH wei_y_Page_172thm.jpg
4366270bba0fc3e378abd1c4764f9fcc
24f281768e7fe6cc6616f48cfaf122862efe8689
10186 F20101107_AABJHS wei_y_Page_164thm.jpg
8a034c967bc8d60d7b055763a7796ae7
171dc4e67872445db278b4408fc0b953d265e346
F20101107_AABIEQ wei_y_Page_165.tif
5a7062289e7e834a8e83642a47057b82
4da5791c249704c99d4cbadddf8802685d162e76
F20101107_AABHZK wei_y_Page_029.tif
51063553ab4a6f5d78dc88a744c932b7
2a9c5325a0f5f78055c1d6d86d7107cc62e5ed08
F20101107_AABHYW wei_y_Page_015.tif
3f8fd0f8e5b364b2d408f3f386875577
27c25056c8ed5720ea8b15e5ac6f727a50b90fa6
F20101107_AABIFF wei_y_Page_180.tif
83c967fda61776aa37bbca495782eef4
52f29a77393f14633e662bcadfde56b2ede54125
31264 F20101107_AABJII wei_y_Page_173.QC.jpg
bed9b96a4d8031310b3615616f3279ae
5525abcdd085fb6b6c7a0532d16f1b5832c8f3d4
32144 F20101107_AABJHT wei_y_Page_165.QC.jpg
86acaa515499dd8dee28a0f3c4eae8c1
4cdcbe4aee71c2bbbd29eac7021b6c801a0f0f5b
F20101107_AABIER wei_y_Page_166.tif
cdcc0fce905303c98f1a34919c8cdb5b
181250e30c4231e0b0a1d1a47f083ab90d396dec
F20101107_AABHZL wei_y_Page_030.tif
9f2b6621a2b83e5f67dc232488e2f1f5
f67d532541f2e7bbdf9c740051502b0a0d15da8b
F20101107_AABHYX wei_y_Page_016.tif
ac81752d22b27eaaff84f01c59aed390
9d8212610203c12add7a39733d24b3cef02cb05c
F20101107_AABIFG wei_y_Page_181.tif
23bf846aa88c4dca64072fbaeebe11c5
50196e5f5b4d2de7d6f762bcdb5d6a5ce637bdef
32082 F20101107_AABJIJ wei_y_Page_174.QC.jpg
1dd4266f5c39a8b6c0b9dbd75ecd1385
86347c5ae23538564e68ebfebe93d133e653ed4d
10108 F20101107_AABJHU wei_y_Page_165thm.jpg
6ec0a3e62469351df49c23f94c008a92
8000bb89c6136fe9c4397cfd1b93bbe297be9d1d
F20101107_AABIES wei_y_Page_167.tif
bb9edc031cc90a674dc313b744939d8b
c2a7b52ab8b1c17891505322452a1fc5c60ef4de
F20101107_AABHZM wei_y_Page_031.tif
f7bac70f05095275f7b78791c069725a
05d584fb9288e96a2a29bcc4fda703b5ef133f28
F20101107_AABHYY wei_y_Page_017.tif
a29d33ce1be37d6deb9ee1ab9a2c94b7
5c73beddcca8cd52a1c0e4c0e42f41b437a2fe02
F20101107_AABIFH wei_y_Page_182.tif
8adb492d132f0a9a6ca791f093a72e7d
f56a2d6107fad06789d77a908e38e572ef60d96b
32281 F20101107_AABJIK wei_y_Page_175.QC.jpg
65b6c0753ac455c4f76151feb758ed23
369a94fd89884d726c9abed292452e7b49cf28b7
31443 F20101107_AABJHV wei_y_Page_166.QC.jpg
54f13e032ec305b15a8b9cd4607a6de5
f886e25f9c5bbb991ee0d3129559a6d05278a0b4
F20101107_AABIET wei_y_Page_168.tif
3a2bbf77f060547a1a449d2817bf83e4
69b24b49f44702a3f1cdcb613acb7dd6617c5bfe
F20101107_AABHZN wei_y_Page_032.tif
f5c33c39ed2dc401707fa946e106e90f
67af8c503279b72afb7d973547216ba0e29b3389
F20101107_AABHYZ wei_y_Page_018.tif
d0946cf2f8723a1bc7db4ea2e37faa89
96b0ed9deb1fe74da5d33c428ad2e0e840fe782c
F20101107_AABIFI wei_y_Page_183.tif
13b5e833b2859a59280727218ac7a13b
774c2088c3f0fe15e79c4edf3bc022c3fe554fc3
10027 F20101107_AABJHW wei_y_Page_166thm.jpg
052dc3b9b9bff2ab47fc3618667aab58
d629658e317d9fa3cbc9d7be09f7750eff0fa221
F20101107_AABIEU wei_y_Page_169.tif
337e1f442d9aa53e5bb8bb19246c8a2b
d695f88ed390ac0fd9ba1bc3598704274906d3b8
F20101107_AABHZO wei_y_Page_033.tif
ecc295f177cb1f47d9126f36071e0fa6
e2de9fbdb7d9d32f0a5737dbfb1f8acddde0df4a
35611 F20101107_AABJJA wei_y_Page_186.QC.jpg
cffdbd5628f380f1a21d47fbd732d875
66d7589eacc7911a92e5be39e022244eb1f820c8
31962 F20101107_AABJIL wei_y_Page_176.QC.jpg
56525698613c8e9adb6ac5217f921f0b
260262d72c1a8eb7b28ad6699a9d7ed3d9e33c89
31560 F20101107_AABJHX wei_y_Page_167.QC.jpg
13ecfcdbd50b403693591ffb66369eb5
6431e2f21ad2247cea9ae477fdad4c5e9921e79e
F20101107_AABIEV wei_y_Page_170.tif
b7699b314a16f66e3ed58083d8613b3a
9357ee065e5f10c1ab28d30d61d70970546aad35
F20101107_AABHZP wei_y_Page_034.tif
d0635d2f32de75d88a31c5e58aa5117a
89c5d1ea9c2ba54f97420f0001f8f09422723520
F20101107_AABIFJ wei_y_Page_184.tif
622f16a0ebd7168382de653c2481dace
08e40169874debbb579a3944ee8f14c18cf21d30
8883 F20101107_AABJJB wei_y_Page_186thm.jpg
f74f5fd33f8e2a95ee74d3d1bd227518
7ddf57746a5cf7675c0812ccc283f60a31194bed
10065 F20101107_AABJIM wei_y_Page_176thm.jpg
815f15fd34fa32b2098e87300e67246a
f863fdc686c441a53248bda0dd4b8b2a2abe36cc
31675 F20101107_AABJHY wei_y_Page_168.QC.jpg
8b0e22b9254f6c0f9a8c7eeaa46dda30
82b4eb0ec9bd4ae8487aab36af0fb1a1f2490b1e
F20101107_AABIEW wei_y_Page_171.tif
d077e8db3613574dd2c6c6d632afd130
d44d236b5f07b7d4434127801cfa64c04a8ada72
F20101107_AABHZQ wei_y_Page_035.tif
27b18919cf3ee06beb391ba26fdb33d7
63b21f87c34196f3f4a1b3d067c5539adfeeb5c7
F20101107_AABIFK wei_y_Page_185.tif
d6f9f217ee525f54e3ef2401237010f5
9d4c2f9d48689e6c23f840e4476b983a9f75588a
37523 F20101107_AABJJC wei_y_Page_187.QC.jpg
397f6e03c5fc8919e49dd8b6de44b505
6a2cc1cc21169d696341fbb53915912c62d92287
F20101107_AABJIN wei_y_Page_177.QC.jpg
f578bb8a19a07e64c3c76adacf98a1bf
5dd684f3ee760c44f702cae005239a29799a61d6
9989 F20101107_AABJHZ wei_y_Page_168thm.jpg
90df07ddcbe75f2b48f0bf75734733b1
5c981f3d419595def719a141da43c63d998f3816
F20101107_AABIEX wei_y_Page_172.tif
8d7ac910378d95320c77feea66166d97
22700ba31b6bd7f452e7cf6d8d1116ec13f5721a
F20101107_AABHZR wei_y_Page_036.tif
3b00ebfbed945a6f1463c7722960aef2
e91d0f771566d5a763daaed54de52c76c93b01d2
2690 F20101107_AABIGA wei_y_Page_008.pro
a73dee609c9f60ab64bd440b61f0f011
50f9c5685ed948dbda685e6e397b2338b51f142f
F20101107_AABIFL wei_y_Page_186.tif
aa78910368de91ae81b6188ee1c6082d
36f8b59026f6ff5355328516d7c15001a771a28c
9026 F20101107_AABJJD wei_y_Page_187thm.jpg
526fef2f95bb8db836caef3027da511f
c248af19a83cf7abc1c2ef710aa6b30d1bf2ace2
31409 F20101107_AABJIO wei_y_Page_178.QC.jpg
55ec8fef93c4ab30b1a2cac83271a0b7
78a42066a7b2b5f9553778ede5811e2af79098f9
F20101107_AABIEY wei_y_Page_173.tif
b00236e901002be380ee0ec78d0b3594
2298e6be615655e5d02fae3f436ae1b2d4f59736
F20101107_AABHZS wei_y_Page_037.tif
75a98eed3fbf797a853e603e418981eb
e5b187b9d3e72efedd04f5cc4be302ffeafd54e6
43257 F20101107_AABIGB wei_y_Page_009.pro
4dfb8423c0b487d762f5b3ce2c38dd53
b16f102342d67f8f8d9976e1a9a2b353fb599a93
F20101107_AABIFM wei_y_Page_187.tif
71845b36a26fb45bbb78e87e3e0ad955
3355af75bb9d507adf71998e1a5862c6c2af5fc1
35699 F20101107_AABJJE wei_y_Page_188.QC.jpg
7eb73130ff2067cfeb3646d0b40b88b0
e63474902beff905609968a2b54059b17f51b220
31530 F20101107_AABJIP wei_y_Page_179.QC.jpg
f8f4c8a330187ddd1ed7c5c62a65bc02
7293e79deff15cf46a4d12e2df03d522d4c23299
F20101107_AABIEZ wei_y_Page_174.tif
fad1c3364f33095df397d043ee4a531d
3c507eb5c6d54c5c898bcd830ed36ad6c9830843
F20101107_AABHZT wei_y_Page_038.tif
efa4faf07d8c3c6a97ee78815a7bb9dc
803d067f0a3ec7dbf629642c7fe997b08015e912
F20101107_AABIGC wei_y_Page_010.pro
25edbfea16ccf347934f81b58e11281d
963ae7c3b36c11bfb29f59e3bb2d1dbbf17fe4ed
F20101107_AABIFN wei_y_Page_188.tif
fab8d4836cd3e5cea09014868c075431
b3b0684a8ca8ce4aa7ba50919d0935edfddad45e
8893 F20101107_AABJJF wei_y_Page_188thm.jpg
5624fcf1a88f82a7865eaad39f3b6496
81f4dd319d7fb934ab77b8ce1d6720992a721f58
10272 F20101107_AABJIQ wei_y_Page_179thm.jpg
473a07daf58d5d8712762f95e0539361
b620160cbae945c527cebf30036dc2abae0a3b47
F20101107_AABHZU wei_y_Page_039.tif
3fcd96aa13e9df39a9fed22edfcea6e2
595b994538e85a3c15660d505567bc3faab95305
53042 F20101107_AABIGD wei_y_Page_011.pro
249df77759883dee466f1df6b14337f2
1d16fc54e84ce92c0334d94594d8e9fbe9f97dd0
F20101107_AABIFO wei_y_Page_189.tif
e93229d687d92556276caaf956cd98e7
22821d1c7a3dd893757d3661d5dcfe888aba9fe1
9134 F20101107_AABJJG wei_y_Page_189thm.jpg
11a975a7b2d9922de5fb3249e2594ed5
21d7f620b4953734b3a5e2189d82b2fb274e629a
31244 F20101107_AABJIR wei_y_Page_180.QC.jpg
ae300879ed6bcfac1dd307087f6bc840
4f62d7c50f80ea2d5be6d258e3ee356e8f2e8db7
F20101107_AABHZV wei_y_Page_040.tif
2267b2c2ccd98fe302cc91b7ec99179b
f09352b5ab732f9e6730e786fa60510e5f5758a5
51343 F20101107_AABIGE wei_y_Page_012.pro
8ffd36b4610d46574889473f27039a05
eb8a7e7a324acce4826700e99c9f22799d9da469
F20101107_AABIFP wei_y_Page_190.tif
e15db0fe27e4e9d6807f451b38a80c6a
864c8c085f336a38adbd874138a5eb8e909bc0ef
34785 F20101107_AABJJH wei_y_Page_190.QC.jpg
6317a7c2466a8aad4bbb6eecb9cbbd1b
334da345d1d11827e92e0e2ff4979f05cade04cb
30833 F20101107_AABJIS wei_y_Page_181.QC.jpg
18e3c23a0b21f9029199d8d86e6f7227
785c0bfde3bc991e287b49f00b22eeb8266dbd41
F20101107_AABHZW wei_y_Page_041.tif
d09f1310131bef915e10935d3b717986
c95a365aa7780790307974bed1733b28b189e66e
30827 F20101107_AABIGF wei_y_Page_013.pro
577500bf690fab48c345ed77e8e6fe81
1b30d02378a1459b7f559df36fb788618ee1eb8a
F20101107_AABIFQ wei_y_Page_191.tif
e13763e6ffcdc3d332c4c5336b7834f8
b33eeaaf201a7d5b1f46d3325fbba911ae376aa3
8705 F20101107_AABJJI wei_y_Page_190thm.jpg
b02ab1b22db13705cdee630a993d3b94
bcad47d92db617195db6f019fdf94cfbc96508ce
9817 F20101107_AABJIT wei_y_Page_181thm.jpg
77552c3b93cf894a40b9729115287a9f
c4272dcbaf58f24ae0697e646a2c9b1a2ebbf1a3
F20101107_AABHZX wei_y_Page_042.tif
fc602b8040cc7d6dc64704955bedc599
b6f59dc8f76977691864399fe5442eabecc9b0fc
30317 F20101107_AABIGG wei_y_Page_014.pro
c5b5b7205ae8f7523e513377872bc19e
b565aef9e9fc01920e64bf7fe9a5723fc0cb4047
F20101107_AABIFR wei_y_Page_192.tif
ac6f2a298c05a429667ef9fb90ac9f62
909aabf0824d6ef13c246a761cb7a527065eb01c
21183 F20101107_AABJJJ wei_y_Page_192.QC.jpg
dd8f4ffca8ece0c0cc556b7aa18dec47
1d5216a7f5e69290ccfff61ee39f3f7ae83c85d8
30872 F20101107_AABJIU wei_y_Page_182.QC.jpg
49e97c36bb9a0f29d02db8e95f330112
f243e0c65eb541a1010562d8ef12f003cf3421e6
F20101107_AABHZY wei_y_Page_043.tif
5d56a26f89448bf9ed58e7cef68afcc8
bb7e0326fe2ec1d7535ca25e1441a00f6a71e5a8
29791 F20101107_AABIGH wei_y_Page_015.pro
e3e90b3b263e1228625c8c2455b8d3f2
3b1d26045404697135f953e50409fa8503216933
F20101107_AABIFS wei_y_Page_193.tif
26a5456cf4d1501bd4d30007bf4b14f2
4b58f1cbe153deb371f090a6562dd21ef3982cef
5076 F20101107_AABJJK wei_y_Page_192thm.jpg
9a2cd4a92025e1fcccad427ae3acc0df
91f20fee3d981610aa5744cce5c8170ecf09bf13
9816 F20101107_AABJIV wei_y_Page_182thm.jpg
998a58ec16f7d7e7580194baae9a854e
4a9efdc0c321a0a896a60fa9ba9799f42174b933
F20101107_AABHZZ wei_y_Page_044.tif
b33be9dd96257a2dac3abe7efc426c73
86cb76dc8e788e1bb371b9a6f9bf22c7c6fa1d0b
39475 F20101107_AABIGI wei_y_Page_016.pro
5b832f517f372c07779cd386e33879f0
6074885cebc5955a0707bdafde99eb0558e1f027
8344 F20101107_AABIFT wei_y_Page_001.pro
c53372603dcf20097f6e31fb8d5267b9
ba6eb54d8405df9686cd60180dd4bb745336a567
13985 F20101107_AABJJL wei_y_Page_193.QC.jpg
f5a88a569d506938070c0058ab90b5f0
1461a1849acb7fc89855a245bc470d3465ec3ce7
31043 F20101107_AABJIW wei_y_Page_183.QC.jpg
ebdd15f2160417afdf5cec2fac9ab280
c7dd6cfdd469159cdd63d36f4a12009d49370b7b
44144 F20101107_AABIGJ wei_y_Page_017.pro
b7cff22d376eac410d0bca224043ecaa
9ac0f504f34604bba50d835ce7cf0630a6c0c82a
728 F20101107_AABIFU wei_y_Page_002.pro
708311055e426923adf46266d935d3cd
94d939fc3eb1743200b2e6e1facce7bc081c950b
9932 F20101107_AABJIX wei_y_Page_183thm.jpg
14bc1f5bd71b7780e3b9274e1462508a
289072cc8343ec6e6342ea9a652e1bc4cf5873bf
13322 F20101107_AABIFV wei_y_Page_003.pro
b32616f037ea6b7a253586edfde8e643
f1b2931974cff4add6bcdc2cf936b0805f61f97e
3663 F20101107_AABJJM wei_y_Page_193thm.jpg
1eff32dae778753e7f92cc9071a522bf
c967e43555f97c91afeb4baa68785cff8523e4c3
29702 F20101107_AABJIY wei_y_Page_184.QC.jpg
99324a2543ccd507bd167907bfef82b8
3060e2d56bf5b05e373579e788de1410630bcf46
38286 F20101107_AABIGK wei_y_Page_018.pro
069dc853a82f57d1a2edbe3f338fc833
fb8969c6f74b3b3bbe3603b630ee595a89b2fbf7
91322 F20101107_AABIFW wei_y_Page_004.pro
367bb101bbcef9e251c50f642a91b764
9d6f4ba3b43e1862d2d367520a69aa45cf5dd4e3
31067 F20101107_AABJIZ wei_y_Page_185.QC.jpg
de9fe8421cc72b6cf4bde1e72b1565d0
cf138f8e9dd318f9fec95151438ac982d2baca81
28724 F20101107_AABIHA wei_y_Page_034.pro
5667d362bd78d983209519879baed720
9c0c767ea1e0f2ee48c4685ca8b8d057a300071a
43647 F20101107_AABIGL wei_y_Page_019.pro
502a7ec300aa035fe9b70a62a97ecf3b
1ba4f9b2e0870cd56c5df926816e03b1ce5d9555
48834 F20101107_AABIFX wei_y_Page_005.pro
8bc1387199ad7f4b676c4c9923ef7f9f
8769691246255527b863969580a7d99f1e5c07a2
34036 F20101107_AABIHB wei_y_Page_035.pro
8c07853fca439d02b3224a23582ced39
788fe9d0f1d24c906938cbdbda3193dc2651556d
44285 F20101107_AABIGM wei_y_Page_020.pro
182c148ede19f7b695059ce9bacb5be6
99e7fac47478bf08d094af8806c80c81d1c4bc12
16315 F20101107_AABIFY wei_y_Page_006.pro
64aa4242ea1f95eda468a2bcf2d82042
b6add4897c5ae2d87690f0cef947b6cf28f2fbf1
38252 F20101107_AABIHC wei_y_Page_036.pro
53319618ff1057aac83ee29150bcdd8c
392d1a0beb58ec473d973c9055c68af3bda69053
24591 F20101107_AABIGN wei_y_Page_021.pro
e3b64ecbda2f174bca4a483097c04e5f
3bd98f1341345d111795b2a9003c4171d731f20f
75711 F20101107_AABIFZ wei_y_Page_007.pro
017ae02ca1ad06d66d7e1ab17c528f94
8da162122e8a5dd936f72abf03d4c3d112a01d0e







A SIMULATION STUDY ON THE PERFORMANCE OF
FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS

















By

YOUHUA WEI


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2008


































2008 Youhua Wei









ACKNOWLEDGMENTS

I would like to express my sincere appreciation to Dr. James J. Algina, my supervisory

committee chair, for providing valuable guidance and support. I would also like to thank other

committee members, Dr. M. David Miller, Dr. Walter L. Leite, and Dr. Zhihui Fang, for their

time and effort on this project.

I thank my parents and my brothers and sisters for their continuous and unconditional

support and encouragement. Finally, I thank my wife, Yan Zhang, for her love and support.









TABLE OF CONTENTS

page

A C K N O W L E D G M E N T S ...............................................................................................................3

L IS T O F T A B L E S ................................................................................. 6

LIST OF FIGURES .................................. .. ..... ..... ................. .7

ABSTRAC T ..........................................................................................

CHAPTER

1 INTRODUCTION ............... ................. ........... ......................... .... 11

U nidim ensional IR T M models ......................................................................... ................... 13
L ogistic M odel ................. ......... .........................................13
N orm al O give M odel ........... .. ....................... ........ ...... ........ .... 14
U nidim ensional IRT Scale Linking ......... .. .................. ........... ................................... 14
Scale Transformation........ .......... ..... ... ... .... ....... ...... ....... 14
Scale Linking .............. ........................................... 16
M ultidim ensional IR T M models ....................................................................... ..................20
Logistic M odel ................. ......... .........................................20
N orm al O give M odel ........... .... ............................. ........ .. ........ .... 23
Multidimensional IRT Scale Linking .................................................. 25
H irsch's M ethod ........................................... ........................... 25
L i's M eth o d ........................................................................... 3 0
M in's M ethod ......... ..... ..................................................................... ..3........ ........ 33
Oshim a and Colleagues' M ethod ............................................................................. 35
Purpose of the Study ........... ............................... .......... ................... 40

2 METHODOLOGY ............................. ...................... ........42

Design .................... .... ........... ...................... 42
Independent Variables or Experimental Conditions............................................ 42
Dependent Variables or Evaluation Criteria......................................... ............... 47
P ro c ed u re ......................................................... ................................... 4 9
D ata G en eration .................................................... ................ 4 9
P aram eter E stim action ............. .. ...... .......................................................... 51
Result Analysis ................................................52

3 R E SU L T S .............. ... ................................................................59

General Performance of the Different Linking Methods................................................61
Performance of Linking Methods for Different Test Structures .............. ............ ......62
Performance of Linking Methods for Different Test Lengths.................. ..... .............63
Performance of Linking Methods for Different Sample Sizes .............................................64









Performance of Linking Methods for Groups with Different Ability Distributions .............65
Performance of Linking Methods for Test Items with Different Parameter Values .............67

4 D ISC U S SIO N ............................................... .......................................... 122

R results from Previous Studies ....................................................... ............ ..................122
E effects of D different Test Structures......................................................................... ...... 124
Effects of Different Test Lengths ................................................. ........................ 126
Effects of D different Sam ple Sizes......................................... ........................ ............... 127
Effects of D different Ability D istributions..................................... ......................... ......... 128
Effects of Different Item Param eter Values .................................... .................................. 130
Perform ance of D different Linking M ethods .................................. .................................... 131

5 C O N C L U SIO N S ................. ......................................... .......... ........ .. ............... .. 133

C onclu sions.......... .............................. ...............................................133
F u tu re R research .......................................................................... 134

APPENDIX: ACCURACY AND STABILITY FOR DIFFERENT LINKING METHOD S..... 138

L IST O F R E F E R E N C E S .................................................................................. ..................... 186

B IO G R A PH IC A L SK E T C H ......................................................................... ... ..................... 193









LIST OF TABLES


Table page

2-1 Ability distributions for exam inee groups .............................................. ............... 54

2-2 Item parameters for 20 items with approximate simple structure............ ...............55

2-3 Item parameters for 40 items with approximate simple structure............ ...............56

2-4 Item parameters for 20 items with complex structure ................................................. 57

2-5 Item parameters for 40 items with complex structure ................................................. 58









LIST OF FIGURES


Figure p e

3-1 Accuracy and stability for different linking methods ................................................69

3-2 Accuracy and stability by linking method and test structure...........................................72

3-3 Accuracy and stability by linking method and test structure: N = 2000..........................75

3-4 Accuracy and stability by linking method and test length for approximate simple
stru ctu re te sts ........................................................................... 7 8

3-5 Accuracy and stability by linking method and test length for complex structure tests:
N = 5 0 0 ......................................................................................... 8 1

3-6 Accuracy and stability by linking method and test length for complex structure tests
N = 10 0 0 : ....................................................................................... 8 4

3-7 Accuracy and stability by linking method and test length for complex structure tests
w hen G2 w as excluded: N =1000 ....................................................................... 87

3-8 Accuracy and stability by linking method and test length for complex structure tests:
N = 2 0 0 0 .................................................................................9 0

3-9 Accuracy and stability by linking method and sample size.............................................93

3-10 Accuracy and stability by linking method and sample size for approximate simple
stru ctu re te sts ........................................................................... 9 6

3-11 Accuracy and stability by linking method and sample size for complex structure tests ...99

3-12 Accuracy and stability by linking method and group for approximate simple
stru ctu re tests ............................................................................102

3-13 Accuracy and stability by linking method and group for complex structure tests: N =
5 0 0 .............. ...................... ........................................ ......... ...... 10 5

3-14 Accuracy and stability by linking method and group for complex structure tests: N =
10 0 0 .............. ..................... ....................................... ......... ..... 10 8

3-15 Accuracy and stability for different linking methods: COM, n=40, N=1000, G2...........111

3-16 Accuracy and stability for different linking methods: COM, n=20, N=1000, G2...........112

3-17 Accuracy and stability by linking method and group for complex structure tests: N =
2000.......... ... ....................... .............................................. ...... 113

3-18 Linking accuracy and stability and item parameter values: COM, n=20, N=1000, G3 ..116









3-19 Linking accuracy and stability and item parameter values: APP, n=40, N=2000, G4 ....119









Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

A SIMULATION STUDY ON THE PERFORMANCE OF
FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS

By

Youhua Wei

August 2008

Chair: James J. Algina
Major: Research and Evaluation Methodology

Scale linking is the process of developing the connection between scales of two or more

sets of parameter estimates obtained from separate test calibrations. It is the prerequisite for

many applications of IRT, such as test equating and differential item functioning analysis.

Unidimensional scale linking methods have been studied and applied frequently over the past

two decades. The development of multidimensional linking methods is at the infancy stage and

more research is needed to obtain definitive results.

As an extension of previous research, the purpose of this study was to use simulated data to

evaluate the performance of four multidimensional IRT scale linking methods, the direct method,

equated function method, test characteristic function method, and item characteristic function

method, under various testing conditions, which include different test structures, test lengths,

sample sizes, and ability distributions. There were one hundred and ninety-two experimental

conditions in this study and five hundred replications were conducted for each of the conditions.

The linking performance evaluation was based on the differences between the item parameter

estimates for base group and the transformed item parameter estimates for the equated group









across the test items. The mean and standard deviation of the differences across the 500

replications were computed to examine the accuracy and stability of the four linking methods.

Our results indicate that for approximate simple test structure, each of the four linking

methods worked approximately equally well under all testing conditions. The results also suggest

that for complex test structure: (a) The equated function method did not work well under any

testing conditions, (b) the performance of other three linking methods depended on other testing

conditions including sample size, test length, and ability distribution difference between groups,

and (c) the direct method was the best linking procedure for most testing conditions. In addition,

the study shows that the item parameter values influenced the linking performance. Under most

of the testing conditions, the linking results for the discrimination parameter tended to be less

accurate and less stable when the item parameter had extreme values. The linking accuracy for

the difficulty parameter was not dependent on the item parameter values. The linking stability for

the difficulty parameter depended on the item parameter values only when the sample size was

large. Then, the linking results were less stable when the item parameter had extreme values.









CHAPTER 1
INTRODUCTION

Suppose a set of test items is administered to non-equivalent groups of examinees and

item response theory (IRT) is used to estimate the item parameters for each of the groups. The

parameter estimates will be on different scales because the metric defined by each separate

calibration is different (Stocking & Lord, 1983). Specifically, IRT parameter estimation

procedures often scale the ability for each group with mean of 0 and standard deviation of 1,

although the actual ability distributions of the two groups may be different (Kolen & Brennan,

2004). Therefore, to compare the parameter estimates from different IRT calibrations, they

should be transformed on the same scale. Scale linking is the process of developing the

connection between scales of two or more sets of parameter estimates obtained from separate test

calibrations. The objective is to establish a common metric for all sets of parameter estimates.

Scale linking is an important issue in psychometrics, and many applications of IRT

require that item parameter estimates from independent calibrations be expressed on the common

metric, including test equating and differential item functioning (DIF) (Stocking & Lord, 1983).

Based on Kolen and Brennan (2004), equating is "a statistical process that is used to adjust

scores on test forms so that scores on the forms can be used interchangeably." (p. 2), and linking

refers to relating scores on tests which are not built to the same content or statistical

specifications. Different terminologies have been used to describe the process of establishing

relationship between scores on two or more tests (for a complete review, see Kolen, 2004a,

2004b). Scale linking is used in this study to refer to the process of linking different scales rather

than the process of linking test scores. However, scale linking is the prerequisite for establishing

the connection between different test scores. Therefore, scale linking is an important step in test









equating (Cook & Eignor, 1991; Kolen & Brennan, 2004) and satisfactory equating results

require successful scale linking.

If different groups of examinees have different probabilities of success on an item after

they have been matched on the ability of interest, the item has differential functioning. In IRT,

DIF is defined as the differences in the model parameters for the comparison groups (Clauser &

Mazor, 1998). The item parameters for different groups should be compared only after they are

placed on a common metric. Therefore, DIF identification depends heavily on the quality of

scale linking. Some procedures have been developed to detect DIF by improving scale linking

(Candell & Drasgow, 1988; Lautenschlager & Park, 1988; Lautenschlager, Flaherty, & Park,

1994; Park & Lautenschlager, 1990).

In addition to psychometrics, scale linking is also very important to educational and

psychological studies. Multi-group confirmatory factor analysis or mean and covariance

structure analysis has been increasingly used to compare constructs across different groups (for a

comprehensive review, see Vandenberg, 2000) and some unresolved issues are closely related to

the difficulty of linking scales across groups (Millsap, 2005). Therefore, successful scale linking

has the potential to produce satisfactory comparison studies on psychological constructs across

different groups.

In sum, scale linking is very important for educational measurement to be fair and

objective for different groups of examinees. Unidimensional scale linking methods have been

studied and applied frequently over the past two decades (for more information, see Kolen &

Brennan, 2004; Yen & Fitzpatrick, 2006). The development of multidimensional linking

methods (Davey, Oshima, & Lee, 1996; Hirsch, 1988, 1989; Li, 1997; Li & Lissitz, 2000; Min,

2003; Oshima, Davey, & Lee, 2000) is just at the infancy stage and more research is needed to









obtain definitive results (Yen & Fitzpatrick, 2006). In this chapter, unidimensional and

multidimensional models and linking methods are reviewed and the purpose of the current study

is presented.

Unidimensional IRT Models

Logistic Model

The three-parameter logistic (3PL) model (see Hambleton & Swaminathan, 1985; Lord,

1980) assumes that the probability of a correct answer to a dichotomously scored item j by an

examinee with ability ,O is


P,; (X 1 = l10,; aj,bj,cj
a, (0, -b)
l \ e -
1+e

c+ +1-c +e e_[ bj)],

where x, is the item response (0 or 1) for person i on test item j,

a, is the item discrimination parameter,

bi is the item difficulty parameter, and

c, is the guessing parameter or the pseudo-chance score level, representing the

probability of correct response when the ability assessed by the item is very low.

Sometimes the 3PL model is expressed as


^(x, =1,;a,,b,,c)= +1 + l-c 1, (1-2)


with D=1.701, so that a normal ogive model item characteristic curve (ICC) and a logistic model

ICC with the same item parameters are almost identical.

If ci is 0, the 3PL model becomes two-parameter logistic (2PL) model:









x = 1;a,b) = +e 1 j]. (1-3)

For 2PL model, if ai is 1, it becomes one-parameter logistic (1PL) model or Rasch

model:


pX = 11;b )= 1e,-b (1-4)

Normal Ogive Model

There are also three normal ogive models or cumulative normal distribution models in

IRT: one parameter model:

0(8,-b,} 1 1 t(2
P ;b)=7 e 2 dt; (1-5)

two parameter model:


S1e 2 dt; (1-6)

and three parameter model:
(x,, =1 ,,c )= c, -c (8,-b,) 1 d (1-7)
Pz(X I 11Ob c) (i- b e 2 dt (1-7)

Many IRT models have been developed for test items that are polytomously scored using

ordered categories, including graded response model (Samejima,1969), partial credit model

(Masters, 1982), generalized partial credit model (Muraki, 1992), rating scale model (Andrich,

1978), and nominal response model (Bock, 1972).

Unidimensional IRT Scale Linking

Scale Transformation

The IRT parameter estimates produced from independent calibrations using data obtained

from different groups of examinees are often on different metrics. Lord (1980) demonstrated that









the relationship between the metrics of any two independent item calibrations is linear.

Therefore, a linear equation can be used to transform the IRT parameters on scale E

(representing the linked scale or equated scale) to scale F (representing the base scale). For

person i and item j,

F = AE +B, (1-8)

aE- (1-9)
aF = A-


bF = AbE + B, (1-10)

CF = CE, (1-11)

where 6 aF b* and c* represent the transformed values from the linked scale to the base

scale. A is the slope and B is the intercept. The constants A and B can be expressed as


A = (1-12)
aF

B= AOE = b AbE (1-13)

A and B can also be expressed for any two individuals i and i* or two items j and j*

-OF, bF -bF
A = (1-14)
E, OE bE bE E

B=bFb -AbE= 0F AOE, (1-15)

or expressed for groups of items or examinees (see Kolen & Brennan, 2004):

c(b) (0) (aE)(1-16)
A= = (1-16)
a(bE) c(O,) (aF)

B p=(bF)- A(bE,)= (0)- Ap(0). (1-17)









The E U (O, ) value for the original parameters on scale E will be the same as the Pj (oF ) value

for the transformed parameters on scale F as demonstrated by



-C + (1 1


1+e '


=C 1
-CD -(C ) D OE +B-AbE -Bb

1+e



Therefore, the logistic function is invariant under a linear transformation of item and ability

parameters. Most of the unidimensional IRT scale linking methods are based on this important

feature.

Scale Linking

In practice, both test item parameters and examinees' ability parameters need to be

estimated and the ability estimates are often scaled to have means of 0 and standard deviations of

1. Parameter estimates obtained from different groups of examinees are often on different scales

due to nonequivalence of the groups even though all ability estimates are scaled with means of 0

and standard deviations of 1. Therefore, some data collection procedures are required to establish

the connection between different scales by using the linear transformations mentioned above. In

test equating, three data collection designs are often used, including random groups design,

single group design, and common-item nonequivalent groups design. The IRT parameter

estimates for the first and second designs are assumed to be on the same scale because of the

randomly equivalent groups of examinees and single group of examinees (Kolen & Brennan,

2004) if random sampling errors are ignored. For the third design, the parameter estimates are









assumed to be on different scales due to the nonequivalent groups. The third design is the most

often used equating design (Kolen & Brennan, 2004) and it is very similar to the design used for

exploring DIF. Two approaches have been used to establish a common scale for parameter

estimates for this design. One is to estimate parameters for all items on both test forms together.

This method is often called concurrent calibration (Wingersky & Lord, 1984). Both BILOG-MG

(Zimowski, Muraki, Mislevy, & Bock, 1996) and MULTILOG (Thissen, 1991) have the function

of simultaneously obtaining parameter estimates for two test forms and two groups on the same

scale. The second approach is to link the two scales by using the parameter estimates for the

common items. This study will focus on the second approach. The following IRT linking

methods have been developed to establish a common metric for parameter estimates.

Mean/sigma method. This method (Marco, 1977) uses the means and standard

deviations of the b parameter estimates for the common items to calculate the constants A and B

in the linear transformation equation:


A= B= (bF A (bE(1-18)


Mean/mean method. This method (Loyd & Hoover, 1980) uses the means of a

parameter estimates for the common items to calculate A and the means of b parameter estimates

for the common items to calculate B in the transformation equation:


A=/a B= (bF)-A/(bE). (1-19)


Item response function method. In this procedure (Haebara, 1980), the constants A and

B are estimated by minimizing the sum of the squared difference between the item characteristic

curves for the common items over examinees:










Hd =y zF F, FbF F aF,," AbE, +,C, j (1-20)


Test response function method. The constants A and B are estimated by minimizing the

sum of the squared difference between the test characteristic curves for the common items for

examinees (Stocking & Lord, 1983):


SLd z Zr (F,a J ,bF,) Z O ;E ,AbE, +BEJ (1-21)


Item response function method and test response function method are often referred as the

characteristic curve methods (Stocking & Lord, 1983). Specifically, the former is called item

characteristic method and the latter test characteristic curve method.

Minimum ,2 method. This method (Divgi, 1985) combines information of each item's

parameter estimates and the variance-covariance matrix of sampling errors for each item from

the item parameter estimation procedure. The constants A and B are estimated by minimizing the

following quadratic function:


X2 = a, ,FA, A B)] [Bi1F +iF F ,J (AbE +B (1-22)


where Fj is the estimated variance-covariance matrix of the sampling errors for the item

parameter estimates for itemj on the F scale and is the estimated variance-covariance matrix

of the sampling errors for the item parameter estimates for itemj which are transformed from the

E scale to the F scale.

Comparison studies have been conducted for these methods with dichotomous IRT

model. Based on a comprehensive literature review (Kolen & Brennan, 2004): (a) The









characteristic curve methods produced more stable and accurate results than the mean/mean and

mean/sigma methods, (b) the mean/mean method was more stable than the mean/sigma method,

(c) the concurrent calibration method yielded more accurate results than the test characteristic

curve method for a small number of common items and both procedures had the similar accuracy

for a larger number of common items, and (d) the concurrent calibration method might be less

robust to violations of the IRT assumptions than characteristic curve methods.

These methods have been extended to link scales with polytomous IRT models. For

example, Cohen and Kim (1998) extended mean/mean and mean/sigma methods to the graded

response model and Kolen and Brennan (2004) suggested using mean/mean and mean/sigma

methods for the generalized partial credit model. Baker (1992) generalized the test response

function method to the graded response model and Baker (1993) used the item response function

for the nominal response model. Kim and Cohen (1995) tried the minimum -2 method for the

graded response model.

There are also some comparison studies for these methods with polytomous IRT models.

A simulation study (Cohen & Kim, 1998) comparing the mean/mean method, mean/sigma

method, weighted mean/sigma method, test response function method, and minimum 2 method

for the graded response model found that all methods produced similar results. Another

simulation study (Kim & Cohen, 2002) comparing the test response function method and the

concurrent calibration method for the graded response model found that the concurrent

calibration was relatively more accurate.









Multidimensional IRT Models


Logistic Model

Unidimensional IRT models appear to be adequate for scaling achievement test items in

most practical situations (Yen & Fitzpatrick, 2006). However, it is reasonable to believe that the

performance of examinees on some test items depends on more than one trait or ability and some

consequences of applying unidimensional models to multidimensional data have been identified

(see Yen & Fitzpatrick, 2006). The number of dimensions necessary to model the test item

responses depends not only on the number of ability dimensions and the level on those

dimensions exhibited by the examinees but also on the number of skills to which the test items

are sensitive (Reckase, 1997a). Therefore, multidimensionality can occur in different ways

depending on the interaction between a specific group of examinees and certain set of test items.

There are two types of multidimensional IRT (MIRT) models for dichotomously scored

item response data: the compensatory model and the noncompensatory model. In the

compensatory model, a low 0 value on one dimension can be compensated for by a high 0

value on another dimension (McKinley & Reckase, 1983; Reckase, 1997a). In the

noncompensatory model, an increase in the 0 value on one dimension cannot compensate for a

lower value on another dimension (Simpson, 1978). Since estimation programs and linking

methods have not been well developed for noncompensatory model, the most often used

compensatory model is discussed and used in this study.

The compensatory multidimensional three-parameter logistic (M3PL) model is a direct

generalization of the unidimensional 3PL model (Reckase, 1997a):









P(xJ = 10,;a,,djcj)
(a,0,+d,
= C +(1- CJ, e (a, +d,) (1-23)


1+e

whereP(x, = 10,;a, dj, cj) is the probability of a correct response (x, =1) for person i on

test item j,

x, is the item response (0 or 1) for person i on test item j,

a, is the vector of item discrimination parameters,

d, is the scalar parameter related to the difficulty of the item,

c, is the lower asymptote or guessing parameter, and

0, is the vector of ability parameters for person i.

This model can be expressed in the following scalar form:

P(x, = 10,; a,d ,c,)

j ark k+d, j
= c + (1 -c) e k(1-24)
Z "jkOk+dj
1+e
= c + 1 -c 1
a "lOkk+dj'
l+e

where m is the number of dimensions. When c is 0, the compensatory M3PL model becomes

the compensatory multidimensional two-parameter logistic (M2PL) model (McKinley &

Reckase, 1983):

e(a,,+ d,) 1
p(xj =10,;aj,d)= e(a,+d,) -1a,,+d,) (1-25)
1+e(a+j) +e(JO+d










This model can also be expressed as the following scalar form:


a > O,kk + d,
e = 1
P(x, = 1O,; a,d)= -e 1- (1-26)
Z "jakk+dj a"jkk+d,
k=l kl=
1+e 1+e

Compared with unidimensional IRT models, multidimensional discrimination and ability

parameters are described in the form of vectors instead of scalars. If the 0, dimensions are

orthogonal, the observed correlations among the item scores will be accounted for by the a,


parameters. Otherwise, the item correlations will reflect both the aj parameters and correlated

0 dimensions. In MIRT, the probability of a correct response to an item depends on

multidimensional ability and is defined as an item characteristic surface (ICS). Assuming

orthogonal axes of dimensions in the surface, an itemj can be described by the following three

characteristics (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991):

multidimensional discrimination (MDISCj):


MDISC, = a., (1-27)


which is the discrimination power of the item for the most discriminating combination of

dimensions; multidimensional difficulty (MDIFFj):

-d
MDIFF = (1-28)
SMDISC

which, similar to the difficulty parameter in unidimensional model, is the distance from the

origin of the 0 space to the point of steepest slope in a direction from the origin; and direction

(a jk) of the greatest slope from the origin:


ajk = arccos a (1-29)
MDISC
1









which is the angle that the line from the origin of the space to the point of steepest slope makes

with the kth axis for the item.

Normal Ogive Model

By adapting Thurstone's multiple factor model (1947) to dichotomous item response

data, Bock and Aitkin (1981) proposed a multidimensional normal ogive model by firstly

assuming that an unobserved continuous response variable, y,, for person i and item j is a

linear combination ofm latent variables, 0, weighted by the factor loadings, a:

yj = ajlO,1 + a 2,O2 +-O-+ a-- ,,,, +dj (1-30)

where 0- N(0, I), y ~ N(0,1), and 6- N(O,G,) (Note that the as in Equation 1-30 are not the

as in Equation 1-29). It is assumed that there is an underlying process which generates a correct

observed response, x = 1, when y, equals or exceeds a threshold, y,, and produces an incorrect

observed response, xj =0, otherwise. Then the probability of obtaining a correct item score is


Pj(x, =110,; j,a)I


1 1exp k


(1-31)

Yj ZcjkOk,


m
wh=r (D --------


where 1- a k This is a compensatory model because greater ability on one dimension
k=1

can make up for lesser ability on other dimensions. This model can be reparameterized to










produce similar parameters in multidimensional logistic model (Bock, Gibbons, & Muraki 1988;


Muraki & Engelhard, 1985) by


y 2



J0 ') (t)dt


(1-32)


where


m
Z(o,)= ZaJkO + d
k=l


aO + dj,


ajk ctjk
C U
J



d j
(71


It can also be shown that


ajk
a]k k
q,


di
r-'-
7 '-,
qj


with

m
q = 1+ ak .
k=l


When Os are correlated with covariance matrix O, it can be shown that

ajk 1 _
(DCFc


(1-33)



(1-34)



(1-35)


(1-36)



(1-37)


(1-38)


(1-39)










d = (1-40)


ak k (1-41)
k 1 + aa

7, = d/ (1-42)
1+a'#a

where a is vector of factor loading for itemj and a is the vector of discrimination for itemj.

Multidimensional IRT models for polytomously scored test items have also been

developed, including multidimensional logistic models and multidimensional normal ogive

models (Kelderman, 1997; Muraki, 1999; Muraki & Carlson, 1995).

Multidimensional IRT Scale Linking

The multidimensional scale linking is more complicated than the unidimensional scale

linking because it involves the transformations of scale locations, variances, and covariances of

several ability dimensions obtained from different calibrations and more technical problems need

to be resolved. Just as MIRT can be considered either as a special case of factor analysis or an

extension of unidimensional IRT, the multidimensional scale linking can be realized either by

borrowing methods from factor analysis (Hirsch, 1988; Li, 1997; Min, 2003) or by extending the

unidimensional IRT linking methods to the multidimensional situations (Oshima et al., 2000).

Hirsch's Method

Hirsch (1988) is possibly the first author to explore the feasibility and effectiveness of

multidimensional linking and equating by using the common-examinee design. Hirsch presented

three technical issues in multidimensional linking and provided three possible resolutions. The

first issue is to establish scale transformations to keep the M2PL function invariant. The

following transformation equations can be used for a two-dimension (dimension 1 and 2) M2PL

model:










O, 1t A 02 _82- (1-43)


a = aOoal, a2 = U20a2, (1-44)


d* = d +a iu + +a 2u20, (1-45)

where parameters with superscript "* are transformed parameters on a new scale. The M2PL

function is invariant by this transformation:

P(x, = 1 ;aJ, )


7- t0i Pl- +o-20 '2 a 2 2 0 +(dd+al,1Oa 01j220)


1+e [a j(ll1- o)+ .- 2)+(d+al1lO+j2e20)


1+ e [al, j +a2 ~z2+ d]
=P(x = 10,;aj,dj)

This scale transformation method can be extended to M2PL models with more than two

dimensions. Hirsch's multidimensional scale linking method was based on the invariance of

multidimensional function under the above transformations of item and ability parameters.

The second technical issue is that the correlation between dimensions obtained from the

first calibration may be somewhat different from the correlation estimated from the second

calibration due to some non-parallel items for common-examinee design. If this occurs, the

parameter estimates from two calibrations are composites or linear combinations of different

basis vectors. Therefore, it is necessary to transform the basis vectors from one calibration to

those of the second calibration. This can be realized by transforming the two sets of ability

parameter estimates of the common examinees from two calibrations so that they are as similar

as possible.









The third technical issue is the joint rotational indeterminacy of the item discrimination

and ability parameters. That is, the dimensions can be rotated and produce many possible sets of

0, and aj parameter estimates without affecting the M2PL item characteristic function. As

suggested by Wang (1985), the procrustean rotation in factor analysis (Schonemann ,1966) can

be used to transform the parameter estimates from one calibration to those from the other

calibration.

Hirsch's linking method for the common-examinee design includes four steps. In the first

step, two sets of item and ability parameters (OF, F,) and (E, aE, ) for the common examinees

but on different metrics are estimated from two independent calibrations. In (0F, F ), 0~ is a

Nxm matrix, where N is the number of examines, and aFi is a nxm matrix, where n is the

number of items. In the second step, three transformations are used to obtain common basis

vectors for the two sets of parameter estimates. The first transformation by T1 refers the

discrimination parameter estimates from the first calibration (aF, ) to a set of orthogonal basis

vectors instead of the basis vectors defined by the ability estimates (0, ). The second

transformation by T2-1 refers the discrimination parameter estimates from the second calibration

(aE, ) to a set of orthogonal basis vectors instead of the basis vectors defined by the ability

estimates (E, ). The third transformation by T3= Ti* T2-1 refers the discrimination parameter

estimates from the first calibration (F, ) to a set of common basis vectors for both calibrations.

In the third step, orthogonal procrustean transformation is used to rotate the ability estimates

from the first calibration (F ) to those from the second calibration (0E,). This fourth

transformation matrix T4 can be found by minimizing the sum of squared difference between









each element of the two sets of ability parameter estimates (OF,) and (E, ). The method was

called orthogonal procrustean transformation developed by Schonemann (1966). Specifically,

suppose S = F'0E, SS' = PDP', and S'S = QDQ', then T4 =PQ Given the above four

transformations, the means and standard deviations of the ability parameters for the common

examinees from the two calibrations are estimated in the fourth step. For the common-examinee

design, the linking parameters can be estimated by equating the means and standard deviations of

the ability estimates from the first calibration (F, ) and those transformed from the second

calibrations (1 ). The linking parameter estimates are then used to transform the parameter

estimates which have already been transformed by the procedure described in the second step.

For example, suppose one uses the common examinee design and the M2PL model with two

dimensions, the following relations exist:

OF,, UF,, OEl -- UlEl
Fl1o C 10

'2, -_'U'20 Op2, -'Up'2
F2 -F20 E2 -E (1-46)
OF20 OE2

So


oe1 l El-/ \--6 71
0 E, L FlO/

F0l o


0E2 E22 20 F20
60 = (1-47)
OF( 2
0'S26













M 10\ -= 1o I



M20 =/iE2 /F2 (1-48)



10 10
F2
0F,,


S2 = E2 (1-49)
CF20

Then the transformed parameters from E scale to F scale are





0 -M
S 21 (1-50)
SF2 2

aF1p = SioaE,


aF =S2 (1-51)

dF* =d + aE MlO + a M2, (1-52)

where the parameters with "*" as superscript and "F" as subscript on the left side of equations

are the final transformed parameters on F scale, and the parameters with as superscript and

"E" as subscript on the right side of equations are the transformed parameters on E scale by the

first three transformations.

The function of this four-step scale linking procedure for M2PL model was evaluated by

test equating results performed on both simulated and real data sets using the common-examinee









design (Hirsch, 1989). The equating results were examined by comparing the mean differences

and the mean absolute differences of the true scores and ability estimates between the base tests

and equated tests. Satisfactory equating was found for true scores but not for ability estimates.

Hirsch's linking method was originally developed for the common-examinee design.

However, it can easily be modified to conduct scale linking for common-item nonequivalent

groups design which is most usually used in test equating and DIF study. As Hirsch (1988)

suggested, the basis vector transformation would be the same. The procrustean transformation

would use the common item discrimination parameters instead of the ability parameters. The

item difficulty parameter for each item would need to be regressed onto each of the ability

dimension parameters and therefore produce one unique difficulty parameter for each of the

dimensions (Reckse, 1985). Then the mean and sigma method would be used for the common

item difficulty parameters for the final transformation. However, more study is needed to verify

the adequacy of this modified linking procedure.

Li's Method

Compared with Hirsch's procedure, Li's (1997) multidimensional linking methods are

more straightforward and consistent with MIRT computer estimation programs. Most MIRT

programs solve the identification problem by requiring multidimensional abilities be distributed

as multivariate normal MVN (0, 1). Therefore, the metric of the item parameter estimates is

typically referred to orthogonal reference axes with unit length. Given this condition, one

reference system can be transformed onto the other reference system by a composite

transformation: an orthogonal procrustean transformation for re-rotating the reference system, a

translation transformation for shifting the point of origin, and a single dilation for re-scaling unit

length. Specifically the following equations are used in the reference system transformation:









aF =kT aE, (1-53)

d = d +(aT (1-54)

OF, = (1/k)(T E -m). (1-55)

It can be shown that the M2PL function is invariant to these transformations:

P(X,, =19*;a*,^
P(x, = 10;aFF ,dF)


1+ [kTaE ][(1/k I ',. -m)][d +(a T)m


1 +e kaEjTl(1/k)(T 10Ez m)[dE ajTn]
1
1+e aJE 0 -aEJ Tm][dE +aEJTm]
1
1+e -;EJo +dE
=P(x, = 10E;aE,dE).

The question is how to find T, m, and k. Li (1997) proposed several methods to estimate

the scale linking parameters. The rotation matrix T can be estimated by orthogonal procrustean

transformation procedure as mentioned in Hirsch's method above. Let S = a aEj SS' = PDP

and S'S = QDQ', then T = PQ'

The origin shift coefficient m and unit change coefficient k can be estimated

simultaneously by minimizing the sum of squared difference between test characteristic

functions for the common items obtained from the two calibrations, which was originally

developed by Stocking and Lord (1983) for the unidimensional linking:


f(m,k)= i PF (0;;F ,dF )- PF (;aF J (1-56)
e 1N t=1 j=1

where N is the number of grid points of values.









The origin shift and unit change coefficients can also be estimated separately by different

procedures. For example, the origin shift coefficient can be estimated by minimizing the sum of

squared difference between the two difficulty parameter estimates obtained from two

calibrations:


f (m) = (F d F-), (1-57)
J-1

where n is the number of common items. This was called least squares procedure (Li, 1997). The

unit change coefficient can be estimated as the ratio of the square root of the maximum

eigenvalues of the matrices aFp aF and aE aE, obtained from the two calibrations:

Maximum sig( apF, Fj
k = (1-58)
Maximum sigi HE H

where sig() represents the singular value or the nonnegative square roots of the eigenvalue. This

was called ratio of eigenvalues procedure (Li, 1997). Similar to the least squares procedure for

the estimation of origin shift coefficient, the unit change coefficient can also be estimated by

minimizing the sum of squared difference between the two sets of discrimination parameters

estimated from two calibrations. This is also referred as least squares procedure (Li, 1997):


f(k)= FC, -F,. (1-59)
J=1

The rotation matrix T and unit change coefficient k can also be estimated simultaneously

by a least squares method developed for fitting one matrix to another through a rotation matrix, a

translation vector, and a central dilation vector (Schonemann & Carroll, 1970). In this case, the

rotation matrix and dilation scalar were estimated by minimizing the sum of squared errors of the

following residual matrix:









E=(kaFi T)- a, (1-60)

It can be shown that

trace (T a )acE
k = (1-61)
trace (aC aF

where

aF = iF aFaCE, = iE aE, (1-62)

with aiF as the mean of iF and a-, as the mean of aE This was called ratio of trace procedure

(Li, 1997). The translation vector was not estimated by this method because item discrimination

can not provide information about origin shift.

Comparing the effect of different combinations of reference, translation, and dilation

transformation procedures on the multidimensional linking parameters estimation, Li (1997)

found that the most appropriate MIRT linking method is the combination of procrustean rotation

approach (for dimensional transformation), the ratio of trace procedure (for dilation), and the

least square procedure (for translation). This linking method could produce accurate estimation

of item parameters, approximately equivalent estimation of ability parameters, but unsatisfactory

true score estimation.

Min's Method

Min (2003) challenged Li's (1997) two reasons for using a single dilation parameter, that

is, mathematical tractability and the assumption of constant variance across dimensions, and

argued that one single dilation is insufficient for describing the scale unit changes for multiple

dimensions. Two independent calibrations may change the scales of the multidimensional

dimensions to different degrees. To address this problem, Min (2003) modified Li's (1997)

method by replacing the single dilation parameter with a diagonal dilation matrix to model









different unit changes on different dimensions. The reference system transformations are

performed as follows:

aF = K'Ta (1-63)

dF =dE + (aET)n, (1-64)

6 = K I(T 10 m), (1-65)

where K is a diagonal dilation matrix. It can be shown that the M2PL function is invariant to

these transformations:

P(x, = 10;aF,d)
1
l+e [KTaE [(K 1XT 10E- m)[dE d(aET'jT
1
1 +e r'EjTKIK 1T 1E m)dEj (aEjT)

1
1 +e rEaEj ETKm +.aEjTm]
1
1+e Eo, +dEj I
=P(x, = 10,;a, dE).


For two-dimensional model, K becomes where k1 is the dilation parameter for the first


dimension, and k2 for the second dimension. The least square method (Schonemann & Carroll,

1970) of estimating a rotation matrix, a translation vector, and a central dilation vector for fitting

two matrixes can be followed to find T, K, and m in the transformation equations (Min, 2003).

Mathematically Li's (1997) method and Min's (2003) method produce the same solution for T

and m and the only difference of linking results comes from the different dilation parameters.









Reckase and Martineau (2004) identified an important weakness in Li's (1997) and Min's

(2003) method for MIRT models with high dimensionality and provided a solution to the

problem by employing a non-orthogonal procrustean transformation. However, this approach

needs to be examined by further empirical studies.

Oshima and Colleagues' Method

All multidimensional linking methods mentioned above borrowed an important

procedure, procrustean rotation, from factor analysis to transform the dimensional axes. Oshima

et al. (2000) extended four scale linking methods within IRT from unidimensional to

multidimensional models. According to their methods, the following equations were used to

transform the IRT parameters on one scale E to another scale F (to distinguish IRT linking

methods from the factor analysis methods described above, different indices for linking

parameters are used). For person i and item j,

a =(A- 1a, (1-66)

dF =dE- a'EA P, (1-67)

O, AOE, +, (1-68)

where the rotation matrix Am m adjusts the variances and covariances of the ability dimensions

(scale), and the translation vector P,,, changes the means of the ability dimensions (location) on

the two scales. The model indeterminacy can be shown as the following:









P(x, = OF ;a ,dF)
1

(+e- [( )'a [AOE +P[E -E aEA-ip]
1+e

1+e faEA 1AoEz 'E A ip]}
1

1+e IaEz+ EA dE aE-A 1]}
1
l+e E'+d i
=P(x = 1O,; aE, d,)

As in the unidimensional IRT, suppose that two nonequivalent groups of examinees take

common test items and independent calibrations produce two sets of parameter estimates

(siF ,d, ) and (El ,d, ). These two sets of parameter estimates are on different scales F and E,

and scale linking needs to be conducted to place the two sets of parameter estimates on a

common scale. Using the above equations, (. dE ) on E scale can be transformed to the F


scale (aF ,d ). The values of the two sets of item parameter estimates (a ,dF ) and (a. ,d* )

should be similar due to the invariance of common item characteristic in IRT. Unidimensional

IRT linking methods can be extended to multidimensional IRT model to minimize some

functions of the difference between the two sets of item parameters. Again, the question is how

to find the values of A and P so that the connection between the two scales can be established.

The direct method. This method was a multivariate extension of the minimum chi-

square linking method for unidimensional IRT model (Divgi, 1985). The values of A and P are

estimated by minimizing the sum of squared difference between the two sets of item parameter

estimates over all items. However, the direct method is different from the original method in that









it does not consider the variance-covariance matrix of sampling errors for item parameter

estimates in the function:


f(A, p)=
j(A, 0) ] + dF (1-69)
n(!mn + 1) j=- k=\ j-1

where n is the number of items,

m is the number of ability dimensions, and

(a k d, ) are transformed parameter estimates from E scale to F scale.

The equated function method. This method is the multidimensional extension of the

mean and sigma methods for the unidimensional IRT model (Loyd & Hoover, 1980; Marco,

1977). A more general system of scale linking equations is used to specify that some functions of

the common item parameters from the first calibration (a ,dF ) are equal to the same functions


of the transformed common item parameters from the second calibration (a di ). The

transformed item parameter estimates can be obtained by using the above scale transformation

equations with the linking parameters A and P. The values of A and P are estimated by

minimizing the sum of squared difference between the same functions of the two sets of selected

elements of the estimated (aF, ,dF ) and (aF dF ). The number of functions needed (p) depends

on the number of dimensions (m) or elements in A and p, with p = m2 + m. For example, in the

two dimensional case (m = 2), four parameters in A and two parameters in P need to be

estimated. Therefore, six functions are required to estimate the six linking parameters and they

could be the means of a1,, aj2, and c for the first and second halves of the common items (or

other block of items).









The scale linking functions are flexible in terms of which item parameter estimates to use

and what function to use. Different systems of scale linking functions may produce different

values of the linking parameters A and P. The quality or appropriateness of linking functions can

be evaluated by their stability across random examinee samples, the character of the common

item sets, and the true values of the linking parameters (Davey, et al., 1996). For example, if the

mean is the chosen linking function, the function to be minimized is

f(A, p) 1 pF ) 2, (1-70)
P u=1

where HiF, I pF, ... are the estimated means ofp separate sets of elements of

the estimated (aF, dF ), and


Fu, ,, up are the estimated means ofp separate sets of elements of the

estimated (aF d* ).

The test characteristic function method. This method is an extension of the test

response function method developed by Stocking and Lord (1983) for the unidimensional IRT

model:


f(A,p)=I W ;aF, P ;aFI (1-71)
qo O FwF J-F

where q is the number of matching 0 vectors,

W. is the weight taken at different 0 values.

The W. is used to emphasize that some 0 values are more important than others to estimate the

linking parameters. The weight can also be considered equal along the ability scale.









The item characteristic function method. This method is the multidimensional

generalization of the item response function method for unidimensional IRT model (Haebara,

1980):


f(A,p)= 1 W [P(O;a,dF-P(O;aFdF)1. (1-72)
nxq 1

Based on a simulation study comparing the four IRT linking methods under different

ability distributions (Oshima et al., 2000), all of the four methods were acceptable under almost

any of the minimization criteria and offered dramatic improvement over not linking at all. It was

also found that the test characteristic function method and item characteristic function method

were more stable and recovered the true linking parameters better than the direct method and

equated function method.

The multidimensional linking methods developed by Hirsch (1988), Li (1997), Oshima et

al. (2000), and Min (2003) can all be directly or indirectly performed for the common-item

nonequivalent groups design, which have been a widely used in test equating (Kolen & Brennan,

2004). Accordingly those methods have the potential for establishing calibrated item pool and

exploring DIF. Another multidimensional linking method proposed by Thompson, Nering, and

Davey (1997) can be used for test equating in a design without common items or examinees.

With the assumption of the same origin, axes, and correlation between axes for the two randomly

equivalent groups of examinees, this method solve the rotational indeterminacy by identifying

similar item content clusters on different tests and then rotating them in the same

multidimensional-reference system. Further studies need to be conducted to evaluate the

performance of this method.

Multidimensional scale linking is a new research area. There have been very few studies

conducted for each of the proposed methods (Hirsch, 1988; Li, 1997; Min, 2003; Oshima et al.,









2000) and even fewer studies for comparing different methods in the literature. Therefore, it is

currently difficult to evaluate the function of different methods. The only comparison study by

Min (2003) compared Li's method, Min's method, and Oshima and colleagues' test

characteristic function method in terms of accuracy and stability of scale transformations under

different conditions varying in sample size, structure of dimensions, and ability distribution. The

results indicate that both Oshima and colleagues' and Min's methods were better in transforming

discrimination parameters than Li's method, and Min's and Li's methods performed better than

Oshima and colleagues' method in transforming the difficulty related parameters. In addition,

Oshima and colleagues' method performed better than Min's and Li's methods in transforming

test true scores, and Li's and Min's methods were better than Oshima and colleagues' method in

maintaining the structure of dimensions through orthogonal rotation.

Purpose of the Study

Based on the literature review of the multidimensional linking methods, Li's methods have

been evaluated under various circumstances such as different linking procedures, sample sizes,

equating situations, number of anchor items, linking situations, and ability distributions (Li,

1997). Min's method has also been examined with comparison with other methods under

different conditions including different sample sizes, dimensional structures, and ability

distributions. The performance of Oshima and colleagues' four IRT linking methods has been

examined under fewer testing conditions, that is, for different ability distributions, using

simulation study with only 20 replications (Oshima et al., 2000). A comparison study (Min,

2003) indicates that one of the four IRT linking methods, that is, the test characteristic function

method, outperformed other methods in transforming item discrimination parameter estimates

and equating true score estimates. This suggests that the IRT procedures are promising methods

for multidimensional linking and equating. Further studies are needed to examine the









performance of these four methods under more testing conditions. As an extension of previous

research (Oshima et al., 2000), the purpose of this study was to evaluate the performance of the

four multidimensional IRT scale linking methods, the direct method, equated function method,

test characteristic function method, and item characteristic function method, under various

testing conditions, which include different test structures, test lengths, sample sizes, and

examinees' ability distributions.









CHAPTER 2
METHODOLOGY

A comprehensive review of the unidimensional scale linking and test equating (Cook &

Petersen, 1987) provides us a framework for exploring the performance of multidimensional

scale linking methods. According to Cook and Petersen's discussion, the results of linking and

equating depend on linking or equating methods, sample characteristics, and properties of the

common items. In addition, the multidimensional structure underlying the test item responses

makes scale linking more complicated and should be considered as one important testing

condition. In this simulation study, the performance of the four MIRT scale linking methods

(Oshima et al., 2000) for the common-item nonequivalent groups design was evaluated with the

compensator compensatory M2PL model under different testing conditions, including different

test structures, test lengths, sample sizes, and examinees' ability distributions. The M2PL model

had two dimensions.

Design

Independent Variables or Experimental Conditions

IRT linking method. This study was to evaluate the performance of the four

multidimensional IRT scale linking methods proposed by Oshima et al. (2000): the direct

method, equated function method, test characteristic function method, and item characteristic

function method (see the section Multidimensional IRT Scale Linking in Chapter 1 for detailed

description). The equated function, test characteristic function, and item characteristic function

methods were implemented in a manner consistent with the implementation in Oshima and

colleagues' study (2000). For the equated function method, the means of ajl, Cj2, and dc for the

first and second halves of the items were used as the equated function. For the test and item

characteristic function methods, seven equally spaced 01 points from -4 to 4 and seven equally









spaced 02 points from -4 to 4, making 7 x 7 = 49 grid points, were used with equal unit weight

along the ability scale. The four IRT linking methods have been compared under different ability

distributions (Oshima et al., 2000). It is unknown how they perform under other circumstances.

Therefore, this study can be considered as an extension of Oshima and colleagues' study (2000)

from one testing condition (ability distribution) to various testing conditions (see the following

for the detail).

Test structure. In IRT, the test dimensionality for a particular population is the

minimum number of latent abilities required to produce a monotone and locally independent

model (McDonald, 1981, 1997; Stout, 1990). In the geometrical representation of a test structure,

the coordinate axes of a multidimensional space is defined by a complete set of latent abilities

examined by the test, and each item is described by a vector in the space with its orientation

representing the ability composite that is best measured by the item (Ackerman, 1994, 1996;

Reckase, 1985, 1991). According to the literature review by Tate (2003), based on the number

and nature of the abilities required for the response to each item in the test, there are three types

of test structure: simple structure, approximate simple structure, and complex structure. In the

simple structure, all item vectors are exactly aligned with one of the axes in the multidimensional

space after an appropriate rotation, so all the items under each dimension measure the same

ability. If all item vectors are approximately aligned with one of the multiple axes and therefore

the contribution of one ability is dominant over the contribution of all other abilities, the test has

approximate simple structure. In complex test structure, the response to one item depends on

more than one ability. The first type of structure has been considered as an ideal one and the

second and third types as more realistic item structures (Kim, 1994; Roussos, Stout, & Marden,

1998). Following the method in previous studies (Batley & Boss, 1993; Min, 2003; Mroch &









Bolt, 2006; Oshima et al., 1997; Oshima & Miller, 1992; Tate, 2003), two types of two-

dimension test structure were created by using the three MIRT item characteristics: MDISC,

MDIFF, and direction (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley,

1991). In the approximate simple structure, there were two sets of items: The responses to the

first half items depended on one composite ability with the first dimension as the dominant

dimension and the second dimension as the minor dimension; The responses to the second half

items depended on another composite ability with the second dimension as the dominant

dimension and the first dimension as the minor dimension. In the complex test structure, there

were four sets of items with equal number of items in each of the set. Two sets of items loaded

heavily on one of the two dimensions and lightly on the other dimension, and the remaining two

sets loaded heavily on both dimensions.

Test length. The test items are used to establish the common metric for the two sets of

parameter estimates obtained in separate calibrations. Therefore, the feature of items is very

important for sale linking. The estimation of linking parameters depends not only on the number

of items, but also on the characteristics of the item parameters. Based on some literature reviews

(Brennan 1987; Cook & Petersen 1987; Kolen & Brennan, 2004), 15-30 common items are

necessary for unidimensional IRT linking, although the required number also depends on other

conditions, such as the linking methods, examinees' ability distributions, and characteristics of

the items. Different numbers of items have been used in multidimensional linking studies. Li

(1997) used 15 and 25 items in his study and found that the number of items had a significant

influence on the stability of transformation parameter estimates for multidimensional linking.

Oshima et al., (2000) created 40 item parameters to examine the performance of their four IRT

multidimensional linking procedures. Twenty items were used in Min's study (2003) to compare









three multidimensional linking methods. In this study, 20 and 40 items were used to evaluate the

four MIRT linking methods under different numbers of items with the consideration that more

items may be needed for MIRT than unidimensional IRT linking.

Sample size. Theoretically the performance of linking methods depends on the accuracy

of parameter estimates and parameter estimation is affected by the sample size (Li, 1997). So the

linking function depends in some extent on different sample sizes. Compared with

unidimensional IRT models, a large number of examinees are required for MIRT calibration

because more parameters need to be estimated. Based on some MIRT researchers' (Ackerman,

1994; Carlson, 1987) recommendations, 2000 examinees is a reasonable sample size to obtain

satisfactory item parameter estimates for compensatory multidimensional model. Reckase

(1997a) reported that NORHARM (Fraser & McDonald, 1988) and TESTFACT (Wilson, Wood,

& Gibbons, 1987) generally produced stable parameter estimates for long tests and sample sizes

exceeding 1000 cases. A comprehensive study (Tate, 2003) using both simulated and real data

found that most of the often-used multidimensional computer programs performed well for the

sample size of 2000 examinees. To acquire stable item parameter estimates, Hirsch (1988) used

2000 examinees to evaluate the proposed multidimensional equating. In his first study, Li (1997)

used three different sample sizes, 1000, 2000, and 4000, to examine the performance of three

multidimensional linking methods and found that the sample size had a prominent role in

estimating transformation parameters. Li used 2000 examinees to evaluate the best linking

method in his second study (Li, 1997). Min (2003) also found the significant effect of sample

sizes, 500, 1000, and 2000 on the accuracy and stability of different multidimensional linking

methods and suggested that the sample of 500 examinees showed unreliable results and the

sample of 1000 showed somewhat acceptable outcomes (note that approximate simple structure









and complex structure were used in the study). It is not unusual in testing practice that the sample

size is less than 1000 especially in non-achievement area and the performance of the four IRT

linking methods need to be evaluated under this condition. In this study, three different sample

sizes, 500, 1000, 2000, were used to examine the robustness of the four multidimensional IRT

scale linking methods against parameter estimation errors. As defined in other studies (Li, 1997;

Min, 2003), the sample size of 2000 is the base for comparing the effect of different parameter

estimation errors. The sample size of 500 can be used to examine the robustness of IRT scale

linking for small sample size. The sample size of 1000 was used to examine the effect of sample

size between 500 and 2000 on scale linking and it was also consistent with a study using

multidimensional linking for identifying differential item functioning (Oshima et al., 1997).

Examinees' ability distribution. Based on the review by Kolen and Brennan (2004), the

performance of scale linking also depends on the similarity between the two groups of

examinees. The more similar the groups are, the more adequate the linking will be. Large

difference between groups may produce significant problems in estimating scale linking

parameters. Groups of examinees may differ in many characteristics, such as cultural

background, attitude, motivation, and personality. A comprehensive review on population

invariance in equating and linking (Kolen, 2004) found that equating is population dependent

except under highly restrictive conditions, such as two test forms with similar content, difficulty,

and reliability. This suggests that scale linking parameters that are used to obtain the equivalent

scores should also be dependent on the populations used in the estimation. The ability

distribution is an important characteristic of the examinees and has a significant influence on test

equating and scale linking under both unidimensional (Cook et al., 1985) and multidimensional

circumstances (Li, 1997; Min, 2003). As summarized by Cook and Petersen (1987), the









similarity of ability distribution between groups also affects other conditions required for test

equating, such as the number of common items.

Groups of examinees may differ from each other in terms of mean, variance, and

covariance of the dimensions. Oshima et al. (2000) examined the four multidimensional IRT

scale linking methods under six conditions of the ability distributions across two groups: no

difference at all; differences in 0 variances; differences in 0 correlations; differences in0

means; differences in 0 means and variances; differences in 0 means and correlations. Min

(2003) used four conditions similar to those investigated by Oshima et al, such as differences in

0 correlations; differences in8 correlations and means; and differences in 0 correlations,

means, and variances. However, in all conditions the ability dimensions were uncorrelated in the

base group. In education and psychology, most constructs and dimensions within a construct are

correlated. Two groups should have similar structure of construct before scale linking and

equating are conducted. Given these two considerations and to keep the scope of the study

manageable, correlations between dimensions were set at the same level, but not zero, across all

groups and the two groups varied only in ability level and variance. One purpose of this study

was to explore two-dimensional linking methods under the following four ability distributions:

no difference at all, differences in 0 means, differences in 0 variances, and differences in 0

means and variances (See Table 2-1 for the detail).

Dependent Variables or Evaluation Criteria

Different statistics have been used to evaluate multidimensional linking methods. Bias

and root mean square error (RMSE) are often used to evaluate the accuracy and stability of

results across replications of experiment in IRT simulation studies. For example, using a

common examinee design, Hirsch (1988) evaluated the effectiveness of multidimensional linking









and equating by examining the means and standard deviations of the differences and absolute

differences between the true scores, ability estimates, test characteristic response surfaces, and

contour plots of the common examinees on the base and equated tests. Li (1997) used bias and

RMSE to evaluate three multidimensional linking methods, but he used both the bias and RMSE

of linking parameters and item and ability parameters over replications in his study. Oshima et

al. (2000) compared the means, standard deviations, bias, and RMSE of linking parameters for

different methods.

Another criterion for common item scale linking in IRT framework is to evaluate how

small the differences are between the item parameter estimates for base group and the

transformed item parameter estimates for equated group across the common items (Min, 2003;

Min & Kim, 2003). This criterion was used in this study. Specifically, the common item

nonequivalent groups design was used and simulation was performed to create the data for both

base and equated groups. The parameters for the two groups were estimated and then

transformed onto a common scale. Specifically, the parameter estimates for equated groups were

transformed onto the scale for the base groups by using the transformation equations described

before. The linking coefficients in the transformation equations, A and P, were estimated through


the four IRT multidimensional linking methods. After the common item parameters estimated

from base and equated groups were placed on the same scale, the performance of the four linking

methods were evaluated by examining the differences between the two sets of item parameter

estimates. The mean difference and difference variation across replications (r) for each item were

used to evaluate the accuracy and stability of the four linking methods, as described by the

following statistics:















SD (aJ ) = di (2-2)
r

where

diff = a F, (2-3)


rdiff
idff (dj) r= (2-4)



SD )= dJ df (2-5)
r

where

diff = d dF. (2-6)

Procedure

Data Generation

The following compensatory, two-parameter, two-dimension IRT model was used to create

the item responses with different testing conditions described above:

P(x, = l0,;aj,d) d D aD(all+a22+d (2-7)
l+e

First, five sets of ability parameters for each of the three sample sizes (500, 1000, and

2000) with multivariate normal distributions with various means, variances, and covariances

were generated. One set of ability parameters was used for the base group and the other for the

four equated groups (see Table 2-1 for the five group ability distributions).









Second, two sets of item parameters (one with 20 items and another with 40 items) for

each of the two test structures (approximate simple structure and complex structure) were created

using the three MIRT item characteristics: MDISC, MDIFF, and direction (Ackerman, 1994;

Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991). Based on the pooled results from

past empirical studies (Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991), the

estimated MIDSC has a lognormal distribution with mean of 1.37 and standard deviation of 0.54

and the estimated MDIFF has a normal distribution with mean of 0.28 and standard deviation of

0.69. The item parameters of MDISC and MDIFF in this study were selected randomly from

lognormal and normal distributions with the same value of means and standard deviations. The

test structure was created by manipulating the angle of each item with the first dimension. For

the items that loaded on one dominant dimension, the angle between the item and its dominant

dimension was selected from a lognormal distribution with mean of 100 and standard deviation

of 20. For the items loaded heavily on both dimensions, the angle between each item and two

dimensions were selected from a normal distribution with mean of 450 and standard deviation of

100. Next, the discrimination parameters, al, a2, and the difficulty parameter, d, were computed

by the following formula ((Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase &

McKinley, 1991):

a, = MDISC* cos a,, (2-8)

a2 = MDISC cos a2, (2-9)

d = -MDISC MDIFF (2-10)

(See Table 2-2, 2-3, 2-4, and 2-5 for specific parameter values for different test structures with

different test lengths)









Next, dichotomous item responses were created using the two-parameter and two-

dimension IRT model described by Equation 2-7.

To produce more precise and stable results, replications were conducted for each of the

combinations of testing conditions. In IRT simulation studies, the number of replications

depends on the purpose of the study, the desire of minimizing the sampling variance of the

estimated parameters, and the need for statistical tests of results (Harwell, Stone, Hsu, & Kirisci,

1996). The previous studies on multidimensional linking or equating methods used 0 (Hirsch,

1988), 20 (Oshima et al., 2000), 50 (Min, 2003), 100, and 200 (Li, 1997) replications to evaluate

the accuracy and stability of linking or equating results. Based on Harwell and colleagues'

(1996) recommendation of using a minimum of 25 replications for IRT simulation studies and

given the level of complexity of this study, 500 replications were used for each of the

combinations of testing conditions to evaluate the accuracy and stability of the four

multidimensional IRT linking methods.

Parameter Estimation

The parameters of MIRT models can be estimated using different methods and computer

programs. The often used estimation methods include unweighted least squares (ULS) factor

analysis of tetrochoric correlations, weighted least squares (WLS) analysis of the matrix of

polychoric correlations, and robust WLS analysis methods performed by MPLUS (Muthen &

Muthen, 1998), least squares estimation method based on the matrix of raw product moments of

item scores by NOHARM (Fraser & McDonald, 1988), marginal maximum likelihood

estimation method by TESTFACT (Bock, Gibbons, Schilling, Muraki, Wilson, & Wood, 1999).

The study focusing on model parameters recovery by Knol and Berger (1991) suggests that "for

multidimensional data a common factor analysis on the matrix of tetrachoric correlations

performs at least as well as the theoretically appropriate multidimensional item response models"









(p. 457). A study comparing TESTFACT and NOHARM (Gosz & Walker, 2002) found that

NOHARM provided better solutions for predicting item performance. The comprehensive

comparison study by Tate (2003) found that MPLUS, NOHARM, and TESTFACT performed

reasonably well over a relatively wide range of conditions in assessing the test structure and

estimating parameters. This result was confirmed by another recent study (Stone & Yeh, 2006).

Based on these studies, all these methods can provide satisfactory estimation for model

parameters. NOHARM was used in this study due to its consistently good performance in

previous studies.

After the MIRT item parameters were estimated by NOHARM, the linking parameters

estimated by the four multidimensional IRT linking methods (direct method, equated function

method, test characteristic function method, and item characteristic function method) were

obtained by the computer program IPLINK, which was developed by Lee and Oshima (1996).

Result Analysis

Some previous multidimensional linking studies used descriptive analysis (Hirsch, 1988;

Oshima et al., 2000) and some studies used both descriptive and inferential analysis (Li, 2000;

Min, 2003). In this study, the means and standard deviations of differences between the item

parameter estimates for base group (aF ,dF ) and the transformed item parameter estimates for


equated group (aF ,dF ) across 500 replications were compared under different testing

conditions. Specifically, the accuracy and stability of the four multidimensional IRT linking

methods were evaluated by examining the mean differences and difference variations of al, a2,

and d for all items in the test under different testing conditions.

Based on the experimental conditions described above, there are 5 factors in this study:

multidimensional linking method (4), test structure (2), test length (2), sample size (3), and









ability distribution (4). Therefore, the total number of experimental conditions is 4 x 2 x 2 x 3 x 4

192. Five hundred replications were conducted for each of the conditions.








Table 2-1. Ability distributions for examinee groups

Base Group Group 1 Group2 Group3 Group4


0] 1 .5 0 1 .5 .5 1 .5 0 .8 .4] .5] L .8 .4
0 .5 1 0 .5 1 .5 .5 1 0 .4 .8 .5 .4 .8

Note: All the correlations between dimensions are .5.











Table 2-2. Item parameters for 20 items with approximate simple structure


Item
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20


d
-0.70
-0.23
-2.19
-0.75
-1.18
0.77
1.12
-1.26
-1.86
-1.46
-0.72
0.17
0.51
-0.07
-1.38
-2.66
-0.48
-0.68
-0.77
-0.73


MDISC
1.13
2.29
1.41
1.03
1.71
0.99
1.26
0.95
1.68
2.06
1.33
1.10
1.89
0.63
1.00
1.10
1.17
0.85
2.40
1.06


MDIFF
0.62
0.10
1.55
0.73
0.69
-0.78
-0.89
1.33
1.11
0.71
0.54
-0.15
-0.27
0.11
1.38
2.42
0.41
0.80
0.32
0.69











Table 2-3. Item parameters for 40 items with approximate simple structure


Item
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


d
1.28
0.21
-1.45
0.28
-1.59
0.45
-0.79
-1.07
0.80
-0.94
0.27
0.40
-0.14
-0.32
-1.66
-0.65
-1.50
-0.46
-0.07
-0.74
0.66
-0.94
0.20
0.86
-1.09
0.12
1.97
0.35
-0.90
-0.23
0.18
-0.37
0.18
-0.60
-0.10
-1.75
-0.62
-0.25
0.63
0.44


MDISC
2.33
1.12
1.45
0.58
0.93
0.98
1.20
1.13
1.00
1.92
0.57
1.38
1.18
1.69
1.41
0.80
1.10
1.19
0.83
0.71
1.34
1.22
1.65
1.21
1.00
0.78
1.54
0.98
1.47
1.10
0.88
1.61
0.86
1.50
2.62
1.42
1.42
1.06
0.90
1.45


MDIFF
-0.55
-0.19
1.00
-0.48
1.71
-0.46
0.66
0.95
-0.80
0.49
-0.48
-0.29
0.12
0.19
1.18
0.81
1.36
0.39
0.09
1.04
-0.49
0.77
-0.12
-0.71
1.09
-0.16
-1.28
-0.36
0.61
0.21
-0.20
0.23
-0.21
0.40
0.04
1.23
0.44
0.24
-0.70
-0.30











Table 2-4. Item parameters for 20 items with complex structure


Item
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20


d
-0.70
-0.23
-2.19
-0.75
-1.18
0.77
1.12
-1.26
-1.86
-1.46
-0.72
0.17
0.51
-0.07
-1.38
-2.66
-0.48
-0.68
-0.77
-0.73


MDISC
1.13
2.29
1.41
1.03
1.71
0.99
1.26
0.95
1.68
2.06
1.33
1.10
1.89
0.63
1.00
1.10
1.17
0.85
2.40
1.06


MDIFF
0.62
0.10
1.55
0.73
0.69
-0.78
-0.89
1.33
1.11
0.71
0.54
-0.15
-0.27
0.11
1.38
2.42
0.41
0.80
0.32
0.69











Table 2-5. Item parameters for 40 items with complex structure


Item
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


d
1.28
0.21
-1.45
0.28
-1.59
0.45
-0.79
-1.07
0.80
-0.94
0.27
0.40
-0.14
-0.32
-1.66
-0.65
-1.50
-0.46
-0.07
-0.74
0.66
-0.94
0.20
0.86
-1.09
0.12
1.97
0.35
-0.90
-0.23
0.18
-0.37
0.18
-0.60
-0.10
-1.75
-0.62
-0.25
0.63
0.44


MDISC
2.33
1.12
1.45
0.58
0.93
0.98
1.20
1.13
1.00
1.92
0.57
1.38
1.18
1.69
1.41
0.80
1.10
1.19
0.83
0.71
1.34
1.22
1.65
1.21
1.00
0.78
1.54
0.98
1.47
1.10
0.88
1.61
0.86
1.50
2.62
1.42
1.42
1.06
0.90
1.45


MDIFF
-0.55
-0.19
1.00
-0.48
1.71
-0.46
0.66
0.95
-0.80
0.49
-0.48
-0.29
0.12
0.19
1.18
0.81
1.36
0.39
0.09
1.04
-0.49
0.77
-0.12
-0.71
1.09
-0.16
-1.28
-0.36
0.61
0.21
-0.20
0.23
-0.21
0.40
0.04
1.23
0.44
0.24
-0.70
-0.30









CHAPTER 3
RESULTS

As described in Chapter 2, the criterion used in this study to evaluate the four

multidimensional IRT linking methods was based on the differences between the item parameter

estimates for the base group and the transformed item parameter estimates for the equated group

across 500 replications. Specifically, after the item parameter estimates from the two groups

were transformed to a common scale, the mean and standard deviation of their differences across

the 500 replications were computed to examine the accuracy and stability of the four linking

methods. For each of the 192 experimental conditions, there were three parameter estimates a,,

a2, and d; therefore, the mean and standard deviation of the differences were computed for a,,

a2, and d across 500 replications for each item of the test. Then the distributions of the means

and standard deviations of the differences of a,, a2, and d for all items in the test were obtained.

Based on the characteristic of item parameter invariance in IRT, the item parameter estimates

from the base and equated groups should theoretically be equal after they are transformed to a

common scale. So their differences, and accordingly the means and standard deviations of their

differences across 500 replications, should be 0. Therefore, the performance of the four

multidimensional IRT linking methods can be evaluated by examining how close the means and

standard deviations of the differences are to 0.

There is currently no generally accepted criterion about how close the item parameter

estimates for the two groups should be in order for the linking to be considered accurate and

stable. To describe the distribution of difference, histograms of means and standard deviations of

the differences of a,, a2, and d for the 192 experimental conditions were prepared. The

appendix contains the histograms for all 192 conditions. In this chapter, histograms selected to









illustrate the trends in the results will be presented. The following midpoints were used to

construct the histograms for means: 0, 0.2, 0.4, 0.6. All values smaller than -0.5 and larger

than +0.5 were included in the categories with midpoints of +0.6. For the histograms of the

standard deviations, 0.1, 0.3, 0.5, 0.7, 0.9, 1.1, and 1.3 were used as the midpoints. All values

beyond 1.2 were classified into the category with midpoints of 1.3. If all or most of the items in

the test had means and standard deviations close to 0, the linking method was considered

accurate and stable. Otherwise, the linking method was inaccurate and unstable. The

performance of the four multidimensional IRT linking methods was evaluated in this way under

different testing conditions.

On the histograms, the direct method, equated function method, test characteristic function

method, and item characteristic function method are labeled Linkl, Link2, Link3, and Link4. For

test structure, the approximate simple structure is abbreviated as APP and the complex structure

as COM. For test length, the number of items in the test is indicated by n = 20 or n = 40. For

sample size, the number of examinees is indicated by N = 500, N = 1000, or N = 2000. For

ability distribution differences between the base and equated groups, the condition is abbreviated

as Gl if the mean vectors and covariance matrices were equal for the two groups, G2 if only the

mean vectors were different, G3 if only the covariance matrices were different, and G4 if the

mean vectors and covariance matrices were not equal for the two groups.

As will be shown subsequently, inspection of the results indicated that the effects of

linking methods depended on the test structure. Therefore, the decision was made to focus

primarily on the effects of linking methods within each of the test structures. Inspection of the

results for APP suggested that the interactions of all other factors were small in size, therefore

the focus was on the main effects of the factors. Inspection of the COM results suggested that









there were two-way, three-way, or four-way interactions of other factors, so the performance of

the linking methods were described taking into account these interactions.

This chapter consists of six sections. The first section compares the general performance of

the four linking methods. The second section compares the four linking methods for different test

structures. The third section compares linking methods for tests with different lengths. The

fourth section compares linking methods for different sample sizes. The fifth section compares

linking methods for groups with different ability distributions. The last section shows the

relationship between scale linking performance and item parameter values.

General Performance of the Different Linking Methods

The performance of the four multidimensional IRT linking methods was first compared

across all testing conditions by collapsing the means and standard deviations of a,, a2, and

d for all items under different testing conditions. The histograms in Figure 3-1 show the

distributions of means and standard deviations for al, a2, and d across all items and both test

structures. Based on the percentage of items with the means and standard deviations of

differences close to 0, Linki (direct method) produced more accurate and stable linking results

than Link4 (item characteristic function method), and Link4 yielded more accurate and stable

results than Link3 (test characteristic function method). Link2 (equated function method) did not

provide accurate and stable results for a high percentage of items.

The performance of the four linking methods was also examined separately for different

test structures. Figure 3-2 shows the distributions of means and standard deviations for a,, a,,

and d for APP and COM conditions. Comparing the histograms for the four linking methods on

the left side of the figures, one can see that there was no apparent difference among the four

linking methods for APP conditions. The histograms on the right side of the figures show that









there was obvious difference among the four linking methods for COM conditions. Specifically,

based on the accuracy and stability of linking function, (a) Linki (direct method) worked well,

(b) Link2 (equated function method) worked poorly, and (c) the performance of Link3 (test

characteristic function method) and Link4 (item characteristic function method) was between

that of Linki and Link2, with Link4 being slightly better than Link3.

In sum, Linki (direct method) was consistently the best method and Link2 (equated

function method) the worst method under most COM conditions; the four linking methods

worked equally and consistently well under most APP conditions.

Performance of Linking Methods for Different Test Structures

In this section, the performance of the four linking methods is compared between APP and

COM conditions. Figure 3-2 shows different linking results for the two test structures. Based on

the histograms for APP and COM conditions in the figure, all the four linking methods produced

more accurate and more stable results for APP tests than for COM tests, but the difference in

quality of linking varied across the linking methods. Specifically, LinkI (direct method) results

were slightly better for APP tests than for COM tests, especially for parameters a, and a2; Link3

(test characteristic function method) and Link4 (item characteristic function method) results

were much better for APP tests than for COM tests; Link2 (equated function method) yielded

very poor results for COM tests, but good results for APP tests.

However, for the large sample size (N = 2000), LinkI (direct method), Link3 (test

characteristic function method), and Link4 (item characteristic function method) worked almost

equally well for APP tests and COM tests; the linking performance difference between APP and

COM conditions still remained for Link2 (equated function method) due to its poor function for

COM conditions. Figure 3-3 shows the results of four linking methods for APP and COM tests









when the sample size is 2000. With smaller sample sizes (N = 500 and N = 1000), the linking

performance difference between APP and COM conditions increased for Link3, Link4, and

Linki (see specific histograms in the appendix).

Therefore, test structure had its smallest effects on Linki (direct method), larger effects on

Link3 (test characteristic function method) and Link4 (item characteristic function method), and

the largest effect on Link2 (equated function method). Link2 worked well for all APP tests, but

poorly for all COM tests in this study. Due to the strong influence of test structure on the

function of the four linking methods, most of the results in the following sections are presented

separately for the APP and COM conditions.

Performance of Linking Methods for Different Test Lengths

Given the different performance of linking methods for APP and COM conditions, the

influence of test lengths on the linking function was explored separately for APP and COM tests.

The distributions of means and standard deviations of differences for al, a2, and dfor short and

long tests under the APP conditions are presented in Figure 3-4. Based on the histograms in the

figure, one can see that for APP tests, although the linking performance was not strongly

influenced by test length, all four linking methods produced slightly more accurate and stable

results with long tests.

Inspection of the results indicated that under COM conditions, the performance of the

linking methods depended on the sample size and test length. Therefore, the influence of test

lengths was next explored separately for different sample sizes for COM tests. Figure 3-5 shows

the linking results for short and long tests with sample size of 500. Although none of the four

linking methods worked well, the histograms still show that the results for Linki (direct method),

Link3 (test characteristic function method), and Link4 (item characteristic function method) for









short tests were better than those for long tests and that Linki to some extent performed similarly

for different test lengths.

Figure 3-6 illustrates the linking results for short and long tests with sample size of 1000.

From the figure, it was difficult to state at which test length linking performance was better.

Subsequently reported results will show that the performance depended to some extent on the

ability distribution difference between the base and equated groups. When the linking results for

ability condition 2 (G2: unequal mean vectors) were excluded, the linking function for long test

was obviously better than that for short test except for Link2 (equated function method), as

presented in Figure 3-7. Therefore, in Figure 3-6, the performance under G2 masks the positive

effect of test length on the linking accuracy and stability.

Shown in Figure 3-8 are linking results for different test lengths with sample size of 2000.

It is very obvious that the linking results for long tests were better than those for short tests

except for Link2 (equated function method).

In sum, the linking results for long tests were better than those for short tests except in

some COM conditions when the sample size was small.

Performance of Linking Methods for Different Sample Sizes

Inspection of results indicated that sample size had stable and consistent influence on the

linking performance, but with different degrees of influence for different test structures.

Therefore, the effect of sample size is first shown across all other testing conditions then

presented separately for APP and COM conditions. Figure 3-9 contains the linking results for

different sample sizes. Comparing horizontally the histograms for different sample sizes, one can

find that both the linking accuracy and stability increased for Linki (direct method), Link3 (test

characteristic function method), and Link4 (item characteristic function method) with the sample

sizes changing from 500 and 1000 to 2000. However, the linking performance increased at









different degrees for different test structures. Figure 3-10 shows the linking results for different

sample sizes for APP tests. Figure 3-11 shows the results for different sample sizes for COM

tests., From Figure 3-10, it can be found that the accuracy of the four linking methods was fairly

good at all sample sizes and the stability of the four linking methods increased when the sample

size became large for APP tests. Figure 3-11 suggests that although the accuracy and stability of

Linkl, Link3, and Link4 increased when the sample size became large for COM tests, the linking

performance was poor for sample sizes of 500 and 1000 especially for Link3 and Link4. In

addition, for COM tests, the accuracy and stability for Link2 (equated function method) were

very poor for all sample sizes and relatively unaffected by sample size.

Based on these findings, (a) Linki (direct method), Link3 (test characteristic function

method), and Link4 (item characteristic function method) for APP tests were less affected by

different sample sizes than were COM tests; (b) Linki (direct method) was less affected by

sample sizes than were the other linking methods for COM tests.

Performance of Linking Methods for Groups with Different Ability Distributions

Inspection of the results suggested that the linking results for different ability distributions

depended on other testing conditions. Therefore, the influence of ability distribution was first

explored separately for APP and COM tests. Figure 3-12 shows the linking results for groups

with different ability distributions under the APP conditions. Comparing horizontally the

histograms across G1 (equal mean vectors, equal covariance matrices), G2 (unequal mean

vectors, equal covariance matrices), G3 (equal mean vectors, unequal covariance matrices), and

G4 (unequal mean vectors, unequal covariance matrices) indicates that: (a) for a, and a2, the

linking results for G1 were slightly better than those for other ability conditions; (b) for d, the

results for G2 were somewhat worse than those for other ability conditions; (c) Link2 (equated









function method) was least affected by ability distributions. The results imply that a difference

between groups in the mean vectors was more influential than a difference between the groups in

the covariance matrices.

Inspection of results for COM tests indicated that the influence of ability distribution was

moderated by sample size; Therefore, the effect of ability distribution was explored separately

for N=500, N=1000, and N=2000 for COM tests with the concentration on Linki (direct

method), Link3 (test characteristic function method), and Link4 (item characteristic function

method). Figure 3-13 shows the linking results for groups with different ability distributions with

sample size of 500. One can see from the figure that although none of the linking methods

worked well for the small sample size, the linking results for G2 (unequal mean vectors, equal

covariance matrices) and G4 (unequal mean vectors, unequal covariance matrices) were worse

than for G1 (equal mean vectors, equal covariance matrices) and G3 (equal mean vectors,

unequal covariance matrices). Even though Linki (direct method) was relatively unaffected by

between-group difference in ability distributions in groups, it still did not work well in

linking d for G2, which indicates the strong influence of the mean difference between groups.

The linking results for different groups with sample size of 1000 presented in Figure 3-14 shows

that linking methods did not work well under G2, especially for d. However, there was some

interaction between group differences and test length. For long test (n = 40), the linking methods

did not work well for G2 (see Figure 3-15); for short test (n = 20), the linking methods worked

relatively well for G2 (see Figure 3-16). The linking results for different groups with sample size

of 2000 shown in Figure 3-17 suggest that the linking methods worked approximately equally

well for the groups with different ability distributions.









In sum, the influence of ability distributions on linking results depended on other testing

conditions: (a) between-group differences in ability distributions did not have a strong influence

on the performance of the four linking methods for APP conditions or for COM conditions with

a large sample size; (b) mean difference between groups had negative influence on the linking

results especially for conditions with small sample size.

Performance of Linking Methods for Test Items with Different Parameter Values

Two types of scatter-plots were used to examine the relationship between linking

performance and item parameter values under each of the 48 testing conditions. The first type of

scatter-plot was used to evaluate the effect of item parameter values on the accuracy of different

linking methods, with y axis as the mean of the differences and x axis as the true parameter

values which were used to generate the item response data. The second type of scatter-plots was

used to evaluate the effect of item parameter values on the stability of different linking methods,

with y axis as the standard deviation of the differences and x axis as the true parameter values.

These two scatter-plots were constructed for each of the three parameter estimates (e.g., a,, a2,

and d) under each of the 48 testing conditions. However, the results of Link2 (equated function

method) were not included in the scatter-plots under the COM conditions due to its consistently

poor performance. Given the limitation of space, the main outcomes are illustrated by some

representative examples.

The results suggest that: (a) Under most of the testing conditions, the linking results

tended to be less accurate for a, and a2 when the two parameters had extreme values, and (b)

under most of the testing conditions, the linking results became less stable for a, and a2 as the

parameters values increased. The results also indicate that: (a) The accuracy of linking results for

d was not closely related to their true parameter values under most of the testing conditions, and









(b) the stability of linking results for dwas also not closely related to their true parameter values

when the sample size was not large. The scatter-plots for one testing condition (COM, n = 20, N

= 1000, G3) illustrate these relationships between linking performance and item parameter

values (see Figure 3-18).

However, for large sample size (N = 2000) the stability of linking results for d was

closely related to their absolute true parameter values. Specifically when the absolute parameter

values of d were closer to 0, the linking results were more stable; when the absolute parameter

values of d were farther away from 0, the linking results were less stable. The scatter-plots for

another testing condition (APP, n = 40, N = 2000, G4) show that: (a) the linking results tended to

be less accurate and less stable for a, and a2 when the two parameters had extreme values; (b)

the linking results tended to be less stable for d as the absolute parameters values increased (see

Figure 3-19).











100

80

60

40

20

0
100

80


BO
40

20

0
100

80

60


40

20

0
100


so


40

20

0


II


-0.6 -OA -02 0 0.2 OA 0.6
Mean of Diferences for al


Ee
'18


100

so
80*

do

40

20

0
100

80


jo

40

20

0
100

80

*6o

40

20

0
100

80




40

20

0


Figure 3-1. Accuracy and stability for different linking methods


1!


0.1 03 0.5 0.7 0.9 1.1 1.3
Standard Deviation of Diferences for al


eII











100

80

i60

40

20

0
100

80

so

40

20

0
100

80

s60

40

20

0
100

80

is0

40

20

0


I I
-0.6 -OA -0.2 0 0.2 OA 0.6
Mean of Differences for a2


-1


II


Ee
'18


100

so
80

do

40

20

0
100

so

jo

40

20







60
40










20
80






100

80
so






40


20

100
80






20

0


0.1 03 0.5 0.7 0.9 1.1 1.3
Standard Deviation of Diferences for a2


Figure 3-1. Continued


a


~1~~


eII











100 I

80
BC



40

20

0
100 I

I

80

40

20





0



I I
100










40

20
C













-0.6 -OA -0.2 0 0.2 OA 0.8
Mean of Diferences for d


-1


II


Ee
'18


100




60

40

20

0
100

80


sol

40

20

0
100

80

60

40

20

0
100

80




40

20

0


Figure 3-1. Continued


0.1 03 0.5 0.7 0.9 1.1 1.3
Standard Deviation of Diflerences for d


III












IAPP I


100
Va o
leo
1 o
S40
20
0
100
E 80

40
20
0
100


P 40
20
0
100
S80
S40
20
0


-OB -OA -02 0


0.2 0.4 0.6 -0. -OA -0.2 0
Mean of DIferences fbr al


0.2 0.4 0.6


IAPP I


COM


0.1 0.3 05 0.7 0.9 1.1 1.3 0.1 03 0.5 0.7 0.9 1.1 13
Standard Devialon of Differences for al


Figure 3-2. Accuracy and stability by linking method and test structure


COM


I
I
I
I
I












IAPP I


100
V 0

140
20
0
100
E 0

40
20
0
100


P 40
20
0
100
s 80
S80
S40
20
0


-OB -OA -02 0


IAPP I


0.2 0.4 0.6 -0. -OA -0.2 0
Mean of Dllerences fbr a2


0.2 0.4 0.6


cOM


0.1 0.3 05 0.7 0. 1.1 1.8 0.1 03 0.5 0.7 0.
Standard Deviation of Differences r a2


Figure 3-2. Continued


100
I 80
S60
140
20
0
100



S80
so
140
20
0
100


S40
20
0
100
so
60
S40
20
0


1.1 13


I COM


I
I
I
I
I











IAPP I


100
1a 0
eeo
S40
0

100

BO
60
a40
20
0
100
80
S80
I 40
20
0
100
1so

S40
20
0


-0.6 -OA -02 0 0.2 0.4 0.6 -0. -OA -0.2 0 0.2 0.4 0.6
Mean of Derences for d


IAPP


I COM

IF


0.1 0.3 05 0.7 0.9 1.1 1.3 0.1 03 0.5
Standard Deviation of Dllerences lor d


0.7 0.9 1.1


Figure 3-2. Continued


100
V 8o
slo
1 640
20
0
100

a40
20
0
100

S40
20
0
100
80
S40
20
0


13


COM


-.











IAPP I


100
I 80
Sleo
40
20
0


80
S40
20
0
100

s co
S40
20
0
100
so
60
S40
20
0


-08 -04 -0.2 0


IAPP I


0.2 0.4 0.6 -0.6 -0.4 -0.2 0
Mean of Differences for al


02 0.4 0.6


cOM


0.1 0.3 0

0.1 0.3 05


I I I I
0.7 0. 1.1 1.
Standard Deviation


I I
0.1 0.
of DIferences


0.5 0.7 0. 1.1 13
br al


Figure 3-3. Accuracy and stability by linking method and test structure: N = 2000


100
I 80
e 60
140
20
0
100
80
S40
20
0
100
80

P 40
20
0
100
so
60
S40
20
0


I COM









IAPP I


100
I 80
Sleo
40
20
0
80
100
seo
S40
20
0
100

S80
S40
20
0
100
so
60
S40
20
0


-08 -OA -0.2 0


IAPP I


1 0

0.1 0.8 05 0.7


0.2 0.4 0.6 -0.6 -0.4 -0.2 0
Mean of Dfferences for a2


0.2 0.4 0.


cOM


0. 1.1 1. 0.1 03 0.5
Standard Deviation of Dfferences r a2


0I I I I1
0.7 0.9 1.1 1i


Figure 3-3. Continued


100
I 80
e 60
140
20
0
100
eso
S40
20
0
100

S40
20
0
100
so
60
S40
20
0


I



I


ii


COM









IAPP I


100
I 80
e eo
40
20
0
80
100
seo
140
20
0
100

S80
140
20
0
100
60
S40
20
0


-0. -A0 -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dlfferencs for d


IAPP I


I

- I


0.1 0.3 05


II


LI









0.7 0.9 1.1 1.3 0.1 0o 0.5
Standard Deviation of Dflerences for d


I 0 I
0.7 0.9 1.1 1i


Figure 3-3. Continued


cOM


I


100
a 60
140
20
0
100

40
20
0
100

S40
20
0
100
60
S40
20
0


ii


COM











20 1


100
I 80
Sleo
140
20
0
100
eso
J 80
S40
20
0
100

s 6o
S40
20
0
100
s 80
60
S40
20
0


-0. -OA -0.2 0 0.2 0.4 0.6 -0.-0.4 -0.2 0 0.2 0.4 0.6
Mean of Differences for al


140 |


0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7
Standard Devialon of Dfferences br al


0. 1.1 13


Figure 3-4. Accuracy and stability by linking method and test length for approximate simple
structure tests


100
I 80
e 60
40
20
0


80
S40
20
0
100


140
20
0
100
so
60
S40
20
0


140 |


|











120 1


100
Va o
1 40
20
0
100

160
a40
20
0
100

P 40
20
0
100
s 80
S80
S40
20
0


-OB -OA -02 0 0.2 0.4 0.6 -0.6 -OA -0.2 0 0.2 0.4 0.6
Mean of Dllerences fr a2


140


100
I 80
S60
40
20
0
100



S80
40
20
0




80
40
20
0
100
so
60
S40
20
0


1.1 13


0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7 0.
Standard Deviation of Dfferences br a2


Figure 3-4. Continued


140


|











120 1


100
I 80
60eo
140
20
0
100

60
140
20
0
100

s ao
P 40
20
0
100
so
60
S40
20
0


-0.6 -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dflerences for d


140


100
I 80
o60
140
20
0
100
Eso

S40
20
0
100


S40
20
0
100
so
60
S40
20
0


0.1 0.3 05 0.7 0. 1.1 1. 0.o 003 0.5 0.7 0.
Standard Deviatn of Diearences for d


Figure 3-4. Continued


1.1 1l


140


|














100
I 80
e eo
40
20
0



0
100
S40
20
0
100


140
20
0
100
so
60
S40
20
0


-08 -OA -0.2 0


U


-nn


0.2 0.4 0.6 -0.6 -0.4 -0.2 0
Mean of Differences for al


02 0.4 0.6


140 |


100
I 80
e 60
40
20
0
100

60
140
20
0
100


S40
20
0
100
s 80
S60
S40
20
0


-l


0.3 05 0.7 0. 1.1 1. 0.1 03
Standard Devialon of Differences


0.5
br al


0.7 0. 1.1


Figure 3-5. Accuracy and stability by linking method and test length for complex structure tests:
N= 500


-l


0.1


I I I





I I


I I


I
I
I
I











|40 |


a


100
I 80
o60
40
20
0
100
J 80
seo
40
20
0
100

S40
20
0
100
so
60
S40
20
0


140


-l


-



I I -


0.1 0.3 05 0.7 0. 1.1 1.8 0.1 03 0.5 0.7 0. 1.1 13
Standard Deviation of Differences r a2


Figure 3-5. Continued


I A


-0B -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dfferences for a2


100
I 80
o60
40
20
0
100
80
seo
860
140
20
0
100
60
140
20
0
100
so
60
S40
20
0


-l


II


L~L


m


m


m


m










20 1


a


100
I 80
e eo
40
20
0
100
Seso
140
20
0
100

S40
20
0
100
so
60
S40
20
0


0.1 0.3 05 0.7


a


a


0.2 0.4 0.6 -0.6 -0.4 -0.2 0
Mean of Dflerences for d


0. 1.1 1. 0.1 03 0.5
Standard Deviatn of Diffrences for d


0.2 0.4 0.6


0.7 0. 1.1


Figure 3-5. Continued


-08 -OA -0.2 0


100
V 80
l 60
140
20
0
100
60
140
20
0
100

S40
20
0
100
s 80
S60
S40
20
0


140


L


|


I


I


JL


I


114l











20 1


100
I 80
Seo
140
20
0



80
100


S40
20
0
100

s co
S40
20
0
100
so
60
S40
20
0


ItI


-0. -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Differences for al


140 |


100
I 80
o60
40
20
0
100

60
140
20
0
100


S40
20
0
100
so
60
S40
20
0


0.1 0.3 05 0.7 0. 1 1. 1 0.1 03
Standard Devialon of Differences


0.5 0.7 0. 1.1 1
br al


Figure 3-6. Accuracy and stability by linking method and test length for complex structure tests:
N= 1000


140 |


I
I
I
I
I


|












100
I 80
leo
140
20
0

80
100
seo
140
20
0
100


S80
240
20
0
100
so
60
S40
20
0


120 1


1-r




Il-


-0.6 -A0 -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dfferences for a2


40


100
V 80
slo
140
2 40
0

100
160
a40
20
0
100


80
0

1 40
20
0


0.1 0.3 0.5 0.7 0.9 1.1 1.3 0.1 03 0.5 0.7 0.9
Standard Devlation of Differences for a2


1.1 1s


Figure 3-6. Continued


140


|









120 1


- -


- -


100
I 80
Seo
140
20


so
0
100

140
20
0
100

S40
20
0
100
s 80
60
1 40
20
0


140


0.1 0.3 05 0.7 09 1 1.3 1 0.1 0 5
Standard Deviatln of Dellrences for d


0.7 0. 1.1


Figure 3-6. Continued


-0B -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dfferences for d


100
I 80
o60
40
20
0
100
80
so
140
20
0
100

S40
20
0
100
so
60
S40
20
0


25


140


L



C


|


m











20 1


100
I 80
Seo
140
20



so
0
100


140
20
0
100


S40
20
0
100
s 80
60
1 40
20
0


-08 -OA -0.2 0


0.2 0.4 0.6 -0.6 -0.4 -0.2 0
Mean of Differences for al


0.2 0.4 0.


140 |


0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7
Standard Devialon of Dferences br al


0. 1.1 12


Figure 3-7. Accuracy and stability by linking method and test length for complex structure tests
when G2 was excluded: N=1000


100
I 80
S60
140
20
0
100
80
so
140
20
0
100


S40
20
0
100
so
60
S40
20
0


140 |


|










120 1


1-r



L~Z


100
I 80
leo
140
20
0
100
60
S40
20
0
100


S80
240
20
0
100
so
60
S40
20
0


140


0.1 0.3 05 0.7 0. 1.1 1.8 0.1 03 05 0.7 0.
Standard Deviation of Differences r a2


1.1 12


Figure 3-7. Continued


-0.6 -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dfferences for a2


100
I 80
o60
40
20
0
100
,e4O
60
140
20
0
100




180
P 40




20
0
0


140


|










120 1


z


a


100
I 80
Seo
140
20
0
100
S80
so
140
20
0
100
s co
S40
20
0
100
60
S40
20
0


140


0.1 0.3 0.5 0.7 0.9 1.1 1.8 0.1 03 0.5
Standard Deviation of Dilerences for d


I7 I I I
0.7 0.9 1.1 1i


Figure 3-7. Continued


-0. -A0 -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dfferences for d


100
0so
slo
140
2 40
0
100
160
a40
20
0
100


80
0


1 40
20
0


140


|











20 1


100
I 80
Seo
140
20
0



80
100

S40
20
0
100

s co
S40
20
0
100
so
60
S40
20
0


-0. -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Differences for al


140 |


100
I 80
o60
40
20
0
100
,e o
60
140
20
0
100

S40
20
0
100
so
60
S40
20
0


-I -








0.1 0.3 05 0.7 0. 1.1 1. 0.1 03 0.5 0.7 0.
Standard Deviatlon of Dfferences br al


1.1 12


Figure 3-8. Accuracy and stability by linking method and test length for complex structure tests:
N = 2000


140 |


|











120 1


-
-
-

-

-
-
-


100
V 80
S60eo
S40
20
0
100
J 80
S60
140
20
0
100

s 6o
S40
20
0
100
s 80
60
S40
20
0


140


0.1 0.3 05 0

0.1 0.8 05 0.7


0. 1.1 15 0.1 03 0.5 0.7 0.9
Standard Deviation of Differences r a2


Figure 3-8. Continued


-0. -OA -0.2 0 0.2 0.4 0.6 -0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of fferences for a2


100
I 80
o60
140
20
0
100
eso

S40
20
0
100


140
20
0
100
so
60
S40
20
0


1.1 13


140


I


|


I


II-









120 1


- I

- i


1_


100
V 80
Sleo
140
20
0
100
80
140
20
0
100
s co
S40
20

0
100
so
60
S 40
20
0


0.2 0.4 0.6 -0.6 -0.4 -0.2 0
Mean of Dlfferences for d


0.2 0.4 0.


140


Li0.1 0.3 05
1 I



0.1 0.3 05


0.7 0. 1.1 15 0.1 05 05 0.7 0.
Standard Deviatin of Diffrences lor d


Figure 3-8. Continued


-0B -OA -0.2 0


100
I 80
e 60
140
20
0
100
80
140
20
0
100

S40
20
0
100
so
60
S40
20
0


1.1 12


140


|











1000


I so


-Oh -0.2


100
t 80



100

so
40
20
0
100
S80
s0
40
20
0
100
80
s0
S40
20
0
100
80
60
I 40
20
0


I I






















O. 0.6


I
















-0.8 -0.2 0.2 0.6
Mean of DIerences for al


Jl000
LI-















]-0.






-0.6


1500
I
I
I
I
I

I
I
I
I
I

I
I
I
I
I


0.1 0.5


i


11000oo II

II II




















0.8 13 0.1 0.5 0.9 1.3
Standard Devialtn of Dlferences for al


LI




LI


0.1 0.5 0.9 1.3


Figure 3-9. Accuracy and stability by linking method and sample size


I


I-













-02 0.2


100
t 80
s0
40
20
0
100
80
80
40
20
0
100
80
o0
S40
20
0
100
80
60
P 40
20
0


I


I


a








11100011200


I so0
100

40-
20-
0


100 -
80 -
40
20
100-

0-
0


20-
-0100 -02


I5o


100 -
t 80 -
40
20
100-
80 -
100
40-
0
100-
80-
80 _

100-
80 -
60-
S40-
20
0.1 05


~I


]


02 0.6 -0.6 -02 0.2 0.6
Mean of Differences for a2


111000


I
I
I
I
I
I
I
I
I
I


1I


[

[


ii


I


I I I I I
-0.6 -02 0.2 0.6



11I2o I


I
I
I
I
I
I
I
I
I
I


uL


I I I I I I I I I
1 0.1 0.5 0.9 1.3 0.1 0.5 0.9 1.3
Standard Deviatlon of Differences for a2


Figure 3-9. Continued








i


tI


I





"II


.


- &


1 mi


Ito


i i


111000


112000O


i


,L









100
t 80 -
0 -
40-
20-


-Oh -0.2


ILI I
JLw







0.2 0.6 -0.6 -0.2 0.2 0.6
Mean d DIIerences for d


]LI





-0.6


I







2 0.2
-0.2 0.2


Ism lI 110
I I



I I
I I




_ I I


0.1 0.5 0.9 13 0.1 0.5 0.9 1.3
Standard Deviation d DIerences for d


= I



11111
zzz


0.5 0.0 1.3


Figure 3-9. Continued


100
t 80
20
40
20


0.6









I 500
100-
80 -
0 -
40-
20-
0
100-
80 -
0-
40-
20-
0
100
80-

0
100
80-
60-
40
20
0 -


0 [

I[



Il
0I I




02 0.6


1000
















I I
-0.6 -02 0.2 0.6
Mean of DIerences for al


JL


Iloon
I I I

,
I __]I


0.1 0.5 0.9 13 0.1 0.5 0.9 1.3
Standard Deviatin of DIIerences for al


2m I


LI




LI
I
I

I


0.1 0.5 0.9 1.3


Figure 3-10. Accuracy and stability by linking method and sample size for approximate simple
structure tests


-0.6


-0.6 -0.2


I










,-02 02

-0_2 0.2


S1500
100-
80 -
40
20
0
100
80-
40
20
100
80-
40
0
100
80-

20
0-











I


I








1500

so-
so
40
20
100 -
80 -
so0 -
40
20
0
100
80 -
so60 -
40
20
0
100 -
so-
40
20
0


0 0 [

I[


I[


0I I
02 0.6


1000












I
zII



-0.6 -02 0.2 0.6
Mean of Differences for a2


-I


-0.8


J L2oon


100 -
BO I
80


100 -
40
20
I
100-



80-
40







0.1 0.5 0. 1
St


II MOI
II I



I
I IB


I-


0.1 0.5 0.9 1.3
andard Deviatlon of Differences fr a2


0.1 0.5 0.9 1.3


Figure 3-10. Continued


I





-02 0.2


-0.6 -02


I.
0.6


I12ooo


i








limo 1200


I


I I I I I I
02 0.6 -0.6 -02 0.2 0.6
Mean d Dlerences for d


111000


100 -
loo
t 80 -
So0 -
40-
20-
0


-0.8 -02 0.2 0.6


11oo I


100 -
t80-
40
20



0
100-
E 80 -
40

0
100

j40
20
100-
80~
20
0 L^^^


-itL


I
I
I
I
I
I
I
I
I
I


-L


0.1 0.5 0. 13 0.1 0.5 0.9 1.3
Standard Devialton d Dlerences for d


Figure 3-10. Continued


0.1 0.5 0.9 1.3


Il1000


-O. -02


I 0o








I I


I L


I I


-


I mm I


B


~I






11I


112000


IL











1500 1100120


- -


a


-0.6 -02 0.2 0.6
Mean of DIerences for al


111000


1I


I


I I I I I
-0.8 -02 0.2 0.6



112o0


t80I-


20- I I




80

60[-
100 -
o






0.1 0.5 0.9 13 0.1 0.5 0.9 1.3 0.1 0.5 0.9 1.3
Standard Devialti of DIlrences fr al


Figure 3-11. Accuracy and stability by linking method and sample size for complex structure
tests


100 -
t 80 -
80 -
40-
20-
0a-
100-
E 80 -
0 -
40-
on -


I
I
I
I
I
I


2


-Oh -0.2


I I
0.2 0.6


I 500o











Ij


I
I
I
I


M


I
I
I

L



I


50so


11 1000


112000o


I
I
I
I


i_


_I_












""I'


100
t 80-
0 -
20 -
100-
0 80 -
40-
!-


100
80
O0
S40
20
0
100
80
60
S40
20
0


iii


-O. -02 0.2 0.6


|aao n


-0.6 -0.2 0.2 0.6
Mean of Differences fr a2


11000


I
I
I
I
I
I


I


1I


C


G


r


I


I I I I I
-0.8 -0.2 0.2 0.6


112000


100
80 -
0 -
40-
20
0
100-
80-
0 -
40-
20



0-
100-

80 -
60 -
40-

20 -
0
80-


I I I
I I I
I _I

IIIJLII


-'Es


I I I I I I I I I
13 0.1 0.5 0.9 1.3 0.1 0.5 0.9 1.3
Standard Deviallon of Differences for a2


Figure 3-11. Continued


I'


as














I


I


0





111000


1120oo00


1


-I


ii











100 -
t 80 -
So0 -
40-
20-
100-
So-
20 -


Ill""


100 I
so-
60 I
SI40
-o0 -02 0.2 0.6


112000


S


M-


-E


-I
-0.8


I I
-0.2 0.2


Mean d Dlerences for d


Ioo


111000


I


I


I I I I
-0.8 -0.2 0.2 0.6



112000 I


-I


I
I
I
I
I


It.


I I


0.9 13 0.1 0.5 0.9 1.3
Standard Deviatlon d Dlerences for d


0.1 0.5 0.9 1.3


Figure 3-11. Continued


I1.


0.1 0.5











I
I
I
I
II


,,


r


~I I











I
I
I
I


.I


Im


1


! I


-- -


. .


--











IG1 G l2 O |G4




100


-0
40
20




40





20-






0 gQ -I I __ |








100I I I I
00 I -




S0 I I

40

100 I I I I
20




100 --I I I I




0.1 05 09 1. 0.1 0 -0. 0.1 0.35 .0 13 0. 1 1 0.5 0. 1.
Standard Deviaton of DIerences for al




Figure 3-12. Accuracy and stability by linking method and group for approximate simple
structure tests









iG G2 ll iG4







ii L A iII









-0.6 -0.2 02 OB -0.6 -0.2 0.2 0.6 -0.6 -02 0.2 0.6 -0.6 -02 02 0S
Mean of Dilerences for a2
100










5I I I I
40



















100I I I
I0 I I I
20











00 -
-0











100








0 I I I
so
w:3www






















0.1 05 09 1.3 0.1 0.5 0.9 1.3 0.1 0.5 0.2 13 0.1 05 09 1.3
S'andeard Devialon of Dierences far a2
Figure 3-12. Continued
20












IG1 11 2 11 l I

























Mean dl Dllfeences for d
100
40


a-




100-



















80I I
20





0




100 -








0.1 05 02 1.8 0.1 0.52 0.20.8 -0.8 -02 0.2 0.8 -O -02 02 Os
Standard Deviation D Ierences far d
0Figure 3-12. Continued

so


20
100


0







0.1 0. 0.9 1.3 0. 0.1 1. 6 0.1 0. 0. 9 1. 0.i Oh 0.1 105
Standard Deviation 1 DIlrences fo r d




Figure 3-12. Continued











IG1 11 2 11 l I



lo



100- I







I0I- 1 l
40













100-
20





0










100-
-0. -02 02 OS 1 -0. -0.2 0.2 O. -0. -0.2 0.2 0.8 -0. -02 0.2 O.








I I I I
20








I I I I
0





























00 500
20 -
-0.1 -0. 0.2 18 0.6 -0.2 0.2 0.8 -0.1 -02 0.2 0. 1 -OA -0. 0.2 0.
Mman a f DM Irences tfr al








M I I I









20

















N = 500













II 11 11- Il I


lo

20
100 I
i 80- I
I I 0 I

40
20




40













100 -I
0







100-
-0. -0 02 O1.3 -0.6 -0.2 0.2 0.6 -0. 2 0.2 0.6 0. -02 0. 1.3
MeStan dard Deviaon of Differences r








e 3-1. C I I
20-

100- I i I I

-0



100-I I I I
20





60 i










Figure 3-13. Continued










IG1 11G 2 lG4 I

100 i






Sg I
40










100 I
80 -


40-











-0.6 -02 02 OB -0.6 -0.2 0.2 0.6 -0.6 -02 0.2 0.6 -06 -02 02 Oh
Mean d Dilbrences far d
20
0






100 I I
I I I I











100 I
20
I I I I

20








100I I I I

l0
t I 80









60-









0.1 05 09 1.3 0.1 0.5 0.1 1.8 01 0.5 0. 1 0. 5 O0. 1.3
Standard Devialton d DIlrences for d



Figure 3-13. Continued











G-1 G2 O |G4






100 i i

wz 80w

20




0
100




-0. -02 0.2 OS -0. -0.2 0.2 O. -0.8 -0.2 0.2 0.8 -Oh -02 0.2 O.h
Mean of D ibrences for al
20- I









0
100
s0




















100- I
-0. -05 02 1. -0.1 0.5 0.2 0.6 -0.6 -02 0.2 0. -0. 092 0 1.
MSt an ofat Dll fences for al



el G2 S A I




20
0























N 1000
Bo-







I I
20








N = 1000









G1 G lG2 IG4 I



100 -

10 -
40

100 -
SI 80
20

I 0

100



i i iww
I I I I I I
60-

-0.6 -0.2 0.2 0 -0.6 -0.2 0.2 0.6 -0. -0.2 0.2 0.8 -0. -02 0.2 0.
Mean of Differences for a2


Igr Iw GS 11Gnue

SOI I I I

20
0!
so
OI I I I
20 I i I I

so
20
IL
100 I I I I


0o 80-
0.1 0.5 0 1.3 0.1 0.5 OA 1.3 0.1 05 0. 1 0.5 09 1.3
Standard Deviallon of DlfAerences for a2



Figure 3-14. Continued









|G1 G 12 |G4 I

o

40

1 I I I
I I I I

S100 1 a
20


40 -




i0 iW I ] W
20








O I I I I I II I I I I I

40
Mean d Dierences for d





F00 -14I I I IC n
80I I
40

20
40


20
100







0.1 0. 0 1.6 0.1 0.5 0. 1.3 0.1 0.5 0.9 10 0.i 0. 0 1.3
Standard Dvialn d D~llenances for d


Figure 3-14. Continued
Figure 3-14. Continued























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Dllfrences for al


-0.6 -0.4 -0.2 0 0.2 A 0.6
Mean of Dferences for a2






I ,


EI i



-0.6 -0.4 -0.2 0 0.2 .A 0.
Mean of Difrences for d


so
0


80
40
0

80
40




0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Diferences for al


40








80:
40
II
0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dferences or a2



0
0

0
0

,0
80
40

0.1 0.3 0.5 0.7 0.9 1.1 1.3
Standard Devlaian of DIlfrences fr d


Figure 3-15. Accuracy and stability for different linking methods: COM, n=40, N=1000, G2


























-0.6 -0.4 -0.2 0 0.2 0A 0.
Mean of Dfferences for al


-0.6 -0.4 -0.2 0 0.2 0.4 B
Mean of Dfferences for a2



40
.. ....








40



-0.6 -0.4 -0.2 0 02 A0.4 O
Mean of DIffrences for d


80

40
0
I 0

a0



0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences fr al







40
0

80
0

I80


0.1 0.8 05 0.7 0.9 1.1 1.3
Standard Deviation of Dfferences tr a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d


Figure 3-16. Accuracy and stability for different linking methods: COM, n=20, N=1000, G2









IG1 11 2 11 l I



40
100 i I I
84E I I I

20
S40
10



S0 I I I I I

-0.6 -02 02 OB -0.6 -0.2 0.2 0.6 -0.8 -02 0.2 0.8 -06 -02 02 06
Mean of Dllerences for al


Ie 11G 12 G CA I
100 I I I I

40
20

0 I1 1 1 1


100 I I



2100 -
20 i
so-




0.1 05 09 1.3 0.1 0.5 0.a 1.3 0.1 0.5 o0. 13 0.1 05 09 1.3
Standard Deviaoln of Dllferences for al


Figure 3-17. Accuracy and stability by linking method and group for complex structure tests: N
= 2000












100
W lW W
40
100 I I I













Mean of Dlirerences for a2
8EI I I I
40




S0
100


20-

-0. -02 0.2 0 -0.8 -0.2 0.2 0. -0.8 -02 0.2 0.8 -Oh -02 02 Oh
Mean of D differences fbr a2


IFg11. 117 Cnl I
I I I I


0
100I I I I
60



20

0



100 I



0.1 05 0.9 1.3 0.1 0.5 0.9 1.3 0.1 0.5 0.9 12 0. 05 0.9 1.3
Standard Deviallon of Diferences fo r a2



Figure 3-17. Continued









IG1 11 2 11 l I

100
40
a-
100- I I I I
I I I I

2 0
o

SIIso- I
I 0



0 I I I I I II II II I
10





100I



0 I I I
60
1I I I I
20
0 1 -I ,I -I I -1



100- I I I I
I : I : I

-I


-: I I I I

I I I I
20
0.1 05 09 1.3 0.1 0.5 0.g 1.3 0.1 0.5 0.9 15 0.1 05 09 1.3
Standard Devialton d Dllrences for d


Figure 3-17. Continued












Mean of
Differences
0.17

0.12

0.07-
0.07 A

0.02- ** -.
.--. ------- ------------ -
-0.03 8A
0 13
-0.08
A
-0.138 D 0

-0.18

-0.23
-028
-0.28



,-0 *** Unki AA.A Unk3 0 D 0 Unk4

0 1 2 3

Parameter Value: al


SDof
Differences
Unki A A A UnkS O ULnk4

0.8


0.7
*
0.6


0.5 A
*
OA


0.3 A


A D
0.2-


0.1

0 1 2 3

Parameter Value: al




Figure 3-18. Linking accuracy and stability and item parameter values: COM, n=20, N=1000, G3





116












Mean of
Differences
0.12

0.07

0.02

-0.03

-0.08-

-0.13-

-0.18

-0.23-

-n-
-0.28



-0.28
-0.8-

-m-


0.1 0.2 0.8 04 0.5 0.6 0.7 OB 09 1.0 1.1 1.2 1. 1.4 15 1.6 1.7 1.8 19 2.0 2.1

Parameler Value: a2


SOD
Differences
0.8


0.7


0.6


OA-






0.2


0.2


0.1


0.1 0.2 0 OA0


'1' '1' I ' ''' I .'.' I''1 I''' 1 I '''.1' '1 '' '1 'I''1 1 1 1 '
0.5 0.6 0.7 02 09 1.0 1.1 1.2 1. 1.4 15 1.6 1.7 1.8 19 2.0 2.1

Parameter Value: a2


a


B 6




El A
AA
E S











D
SUnk nknk


* Unki AA"A UnkS DO Link4







A
S











I


Figure 3-18. Continued











Mean of
Differences
0.14
0.13

0.12
0.11
0.10
0.09
0.08 A
0.07
0.086
0.05 0
0.04
0.03 *A
0.02 -
0.01 a

-0.01
-0.02
O ** Unki AA Unk8 Do UInk4
-0.03

-3 -2 -1 0 1 2

Parameter Value: d




SD f
Differences
07 Unk AA Unlk D O L nk


0.6


0.5


0.4 6


0.8
OAO




0
0.2


0.1 *


0.0

-3 -2 -1 0 1 2

Parameter Value: d




Figure 3-18. Continued



118











Mean of
Difference
0.10-
0.09
0.08
0.07
0.08
0.05
0.04-
0.03
0.02-
0.01- .h. .
0.00 -- -- --- ----
-0.01-
-0.02 8 a
-0.043
-0.05
-0.06
-0.07
-0.08
*-0 UnkI *** I.nlk2 AA A InS 0 3 Unk4
-0.09

0 1 2 3

Parameter Value: al



SOof
Differences
0 .-e UnkI ** hIk A A Unlk U Linknk4
0.32
0.30 *
0.28
0.28
0.24
0.22A
0.20 *
0.18
0.16
0.14
0.121 Ns
0.10 f
0.08
0.068

0 1 2 3

Parameter Value: 81


Figure 3-19. Linking accuracy and stability and item parameter values: APP, n=40, N=2000, G4




119













Mean of
Differences
0.03

0.02

0.01

0.00

-0.01

-0.02

-0.03

-0.04

-0.05

-0.06


Parameter Value: a2


SDo
Dillerences
0.6



0.5
O -





a-
0.4



0.9



0.2 -
-







0.1



0.0


Parameter Value: a2


Figure 3-19. Continued


a A A A




- ------- -A- -- ----- ---------------------------
a A





*

*
**








* Uink *** i Unk2 AAA UinkS Unk4


* nkl *** Link2 A A Link3 D D Unk4















I

a~~ rjs a













Mean of
DIIerences
0.05


0.04


0.03



:*

0.021

0.01 A





-0.02-
Ii ** Lnlk2 Ln nk4
-0.08 ------fl -




. I .. I .. Il* ^

-2 -1 0 1 2
0 Aink Lnk A inkS .
Pamme:r Value: d










0.142-
-0.02







S*** Unl^ ***n Unk2 AAA ELln l Urnk4








0.103 -
S oo

0.12
D
0.12 [
;A




0.11














-2 -1 1 2

Parameter Value: d
0.10
D U





0.0 A

I0.07' I A E







Paramet:r Value: d



Figure 3-19. Continued



121









CHAPTER 4
DISCUSSION

By using simulated data, the performance of the four multidimensional IRT scale linking

methods was evaluated under different testing conditions, which include different test structures,

test lengths, sample sizes, and ability distributions. The results illustrated in Chapter 3 suggest

that test structure had a strong influence on the performance of the four linking methods. For

approximate simple test structure, each of the four linking methods worked approximately

equally well under all testing conditions. For complex test structure, the equated function method

did not work well under any testing conditions; the performance of other three linking methods

depended on different testing conditions; the direct method was the best linking procedure for

most testing conditions. In addition, the item parameter values influenced the linking

performance. The results are discussed in this chapter by seven sections.

Results from Previous Studies

Theoretically, there are at least two main components in linking errors: error caused by

parameter estimation and error produced by scale transformation (Li, 1997). A simulation study

(Kaskowitz & Ayala, 2001) found that linking was more accurate when there was less error in

the item parameter estimates. Therefore, it is important to review previous studies about IRT

parameter estimation and linking accuracy under different testing conditions, although it is

difficult, if not impossible, to decompose the parameter estimation error from the linking error in

testing practice.

Based on Li's review (1997), the following factors can cause error in parameter estimation

in IRT: (a) Examinees' ability distribution. Item difficulty for easy and hard items will not be

well estimated when the examinees are normally distributed around their mean; Examinees with

ability levels above or below the item difficulty are more informative for estimating item









discrimination parameter; (b) Item parameter value. Item difficulty parameters that are small or

large and discrimination parameters that are small or large produce larger estimation error; (c)

Sample size. Larger sample sizes reduce estimation error. However, the standard error of

parameter estimates depends on the combined effect of these factors (Thissen & Wainer, 1982).

Using the bias and RMSE between the transformed linking parameter estimates and the

true linking parameters across replications as the criterion, Li (1997) found that the linking

accuracy of his three methods improved as sample size or test length increases. In the second

study, Li (1997) used the bias and RMSE between the transformed item parameter estimates and

the true parameter values across replications as the criterion and found that one of his linking

methods (e.g., the combination of procrustean rotation approach for dimensional transformation,

the ratio of trace procedure for dilation, and the least square procedure for translation) produced

accurate linking of items. In addition, the positively skewed distribution of the second dimension

in equated group did not negatively influence the linking accuracy and stability.

To evaluate the performance of the four multidimensional IRT scale linking methods,

Oshima et al. (2000) used different criteria, including mean and standard deviation of the linking

parameter estimates over 20 replications, bias and mean square error (MSE) between the

estimated and true linking parameters, correlation and mean absolute difference of linking

parameter estimates across different methods, and minimized function values by different

methods. The results indicate that: (a) The direct method and equated function method tended to

yield similar linking results and the test characteristic function method and item characteristic

function method tended to produce similar results, (b) the test and item characteristic function

methods were more accurate and stable than the other two methods, and (c) the accuracy and

stability decreased as ability differences between the groups increased.









Min (2003) used the bias and RMSE between transformed item parameter estimates and

the initial item parameters across the common items as the criterion to compare Li's (1997)

composite procedure, Oshima and colleagues' test characteristic function method, and Min's

extended composite procedure. Based on the repeated measures analysis of variance for bias and

log transformed RMSE, The author found that: (a) The ability distribution, test structure, and

linking method accounted for large portion of the variation in bias for discrimination parameter

estimates but only linking method was an important factor for the variation in bias of difficulty

parameter estimates, (b) the sample size, ability distribution, and linking method were important

for linking stability of discrimination parameter estimates and sample size and linking method

were critical for linking stability of difficulty parameter estimates, (c) as the sample size became

larger and the two groups were more similar, the linking results became more accurate and

stable, and (d) linking methods had significant interaction effects with testing conditions. In sum,

the linking methods and the three testing conditions, e.g., the ability distribution, test structure,

and sample size, significantly affected the linking accuracy and stability.

Effects of Different Test Structures

In the present study, the performance of all four linking methods worked much better for

APP than for COM tests. This is consistent with the fact that the test structure and item

parameters are typically more easily and accurately estimated for APP test than for COM test. As

Tate (2003) found and discussed in a study comparing different estimation methods, including

NOHARM, for assessing the test structure of item responses, the default rotation methods in

exploratory analysis are usually developed to transform the initial solution to simple structure,

therefore the procedures may not always successfully describe non-simple test structure. A study

(Gosz & Walker, 2002) comparing the performance of TESTFACT and NOHARM found that

the item parameter estimation of NOHARM depends heavily on the number of bi-dimensional









items in the test with its better performance for fewer bi-dimensional items and worse

performance for more bi-dimensional items. In addition, NOHARM is good at estimating items

with very low values on one discrimination parameter and high values on the other

discrimination parameter. In this study, for APP tests, all items had higher values on one

discrimination parameter and lower values on the other discrimination parameter; for COM tests,

half of the items had approximately similar values on both discrimination parameter values. An

investigation of item parameter estimation for the simulated data used in the present study

indicated that the NOHARM program provided better item parameter estimation for APP tests

than for COM tests. The superior estimation for the APP tests is likely the source of the superior

linking results for the APP tests.

However, in a study (Min & Kim, 2003) comparing Li's composite procedure and

Oshima and colleagues' test characteristic function method under different testing conditions, no

apparent linking difference was found between APP and COM tests (see Figure 2-7, Min & Kim,

2003). One possible reason is that the item's loadings on the two dimensions in COM tests in

this study were more similar than those in Min and Kim's study. Specifically the heavily cross-

loaded items in their study had the direction of 50 65 and 25 40, and the direction of

heavily cross-loaded items in this study was selected from a normal distribution with mean of

450 and standard deviation of 100. According to the finding by Gosz and Walker (2002), the item

parameter estimates for COM tests in this study were less accurate, so that the linking results

were more different between APP and COM tests. Another possible reason is related to the

different criteria used to describe the linking performance. This study used the percentage of

items with different means and standard deviations between the item parameter estimates for the

base group and the transformed item parameter estimates for the equated group over 500









replications to evaluate linking results. Min and Kim's study (2003) used the bias and RMSE

between true parameter values and transformed parameter estimates over both 20 items and 50

replications, which may have difficulty in identifying the differential influence of APP and COM

tests on the linking results.

As shown in the Results chapter, the linking results (except for equated function method)

were very similar for APP and COM tests when the sample size became large (N = 2000). This

may be related to the possible improved item parameter estimation for larger sample size for

both APP and COM tests. However, the attribution of different linking performance for the two

types of tests to estimation error needs to be investigated by more controlled studies in the future.

Effects of Different Test Lengths

It was illustrated in the last chapter that the linking results for long tests were typically

better than those for short tests, which is consistent with Li's finding (1997) that the linking

accuracy of his three methods improved as test length increases. This result was not unexpected

since more items can provide more information to set up the linkage between the scales for the

base and equated groups. The positive effect of large number of items on linking and equating

performance has already been found in various unidimensional equating conditions (Budescu,

1985; Fitzpatrick & Yen, 2001; Kaskowitz & Ayala, 2001; Kim & Cohen, 2002; Peterson, Cook,

& Stocking, 1983; Swaminathan & Gifford, 1983; Wingersky, Cook, & Eignor, 1987).

Therefore, this effect of the number of items can be extended from unidimensional to

multidimensional linking and equating situations.

However, there was an exception that the linking results for short COM tests were better

than those for long COM tests when the sample size was small (N = 500). Li could not find this

exceptional result because he used sample sizes of 1000, 2000, and 4000 in his study. One

possible reason for this exceptional result is that small sample size was not large enough to









produce accurate item parameter estimates for long test because more item parameters needed to

be estimated, which accordingly affected the linking performance for long COM tests. Therefore,

the strength of large number of items in scale linking and equating depends on the quality of the

item parameter estimation, which in turn requires enough sample size. Lord (1980) stated that it

is test length in combination with sample size that affects the quality of parameter estimates.

Compared with unidimensional IRT models, a larger number of examinees are required for

MIRT calibration because more parameters need to be estimated.

In addition, this study found that the effects of test length on scale linking performance

also depended on the ability distributions for the two groups. As described in Results chapter, the

long test (n = 40) did not improve linking performance when the means of ability distributions

were different for base and equated groups for COM test when the sample size was 1000. This

phenomenon confounded the general positive effect of large number of items on linking results.

Klein and Kolen's study (1985) suggests that test length has little effect on the equating quality

when groups are similar in ability, but becomes very important when two groups differ in ability

level. They found that a larger number of common items did improve equating when groups

were dissimilar. However, the exceptional result from this study mentioned above did not

confirm their finding. Further studies are needed to examine the conflicting findings by

controlling more conditions.

Effects of Different Sample Sizes

Based on the results from this study, the effects of sample size were very obvious and

straightforward. Generally speaking, the linking accuracy and stability improved with the sample

size increasing. This is consistent with the fact that large sample size can improve item parameter

estimates. The same pattern was also found in the other two multidimensional scale linking

studies (Li, 1997; Min & Kim, 2003). In addition, the positive effect of large sample size has









also been found in unidimensional linking and equating studies (Fitzpatrick & Yen, 2001;

Hanson & Beguin, 2002; Kim & Cohen, 2002; Peterson, Kolen, & Hoover, 1989; Ree & Jensen,

1983).

However, the linking performance improved at different degrees for APP tests and COM

tests. The performance of direct method, test characteristic function method, and item

characteristic function method increased much more rapidly for COM tests than for APP tests

when the sample sizes became larger. In fact, the linking results for APP tests were consistently

good for different sample sizes. However, the linking results for COM tests were very different

for different sample sizes, although the linking function improved with the sample sizes

increasing. This result was not found in Min and Kim's study (2003). They showed similar effect

of sample size on linking accuracy and stability for APP and COM tests (see Figure 2-7, Min &

Kim, 2003). As we discussed for the effect of test structures on linking performance, this may be

related to the different manipulations of COM test items and different evaluation criteria used in

these two studies.

Effects of Different Ability Distributions

Based on this study, for all APP conditions and the COM conditions with a large sample

size, between-group differences in ability distributions did not have a large influence on the

performance of the four linking methods. For COM conditions with small and medium sample

size (N=500, N=1000) between-group differences in mean ability had a negative influence on the

linking results. It seems that mean difference was more important than variance difference.

These results were consistent with what Oshima et al. found in their study using very similar

ability conditions (see Table 5 and Figure 1, Oshima et al., 2000), although they did not divide

tests into APP and COM tests. However, we need to be very cautious about the possible

differential effect of mean and variance differences on scale linking in both studies because they









were controlled at different degrees, with mean difference at 0.5 and the variance difference at

0.2.

Based on the study by Min and Kim (see Figure 2-7, Min & Kim, 2003), it seems that the

influence of ability distributions on scale linking by the test characteristic function method was

approximately similar for APP and COM conditions (see the above explanation for possible

reasons for this conflicting findings between their study and this study). However, they did find

that the influence of between-group differences in ability distribution on scale linking depended

on sample size, with less influence for large sample size (N = 2000) and more influence for small

sample size (N = 500). This is consistent with the results from this study.

Li (1997) used a different manipulation of the between-group difference in ability

distribution than was used in the present study: for the base group both ability distributions were

normal; for the equated group one ability distribution was normal and the other was positively

skewed. No negative effect was found on the linking performance by using his three methods.

The reason may be that although the second ability had positively skewed distribution, the mean

and standard deviation were still controlled at 0 and 1, which were the same as for the based

group for the second dimension. It seems that mean and standard deviation were more important

than the normality of the distribution. However, this conclusion needs to be confirmed for the

MIRT linking methods.

Based on the research on unidimensional scale linking and test equating (see the review by

Kolen and Brennan, 2004), the similarity between two groups of examinees affects linking and

equating performance: the more similar the groups are, the more adequate the linking and

equating will be; large differences between groups may produce significant problems. Based on

results from multidimensional scale linking, this conclusion can be extended to the









multidimensional cases, but with cautious consideration of the interaction between ability

distribution, test structure, and sample size.

Effects of Different Item Parameter Values

As mentioned in the first section, estimation of the item difficulty parameter is less

accurate when the parameters are small or large, estimation of discrimination parameter is less

accurate when discrimination parameters are small or large, and error in item parameter

estimation affects scale linking performance. Therefore, linking quality is likely to be influenced

by the item parameter values, especially by the extreme parameter values. This conceptual

inference and conclusion were confirmed in this study: under most of the testing conditions, the

linking results tended to be less accurate when the absolute item parameters had extreme values

and less stable when the absolute item parameter values became large. This pattern of results was

more apparent when (a) the test had approximate simple structure, (b) the sample size was larger,

and (c) the linking performance for discrimination was evaluated.

The only other multidimensional scale linking study evaluating the effects of different item

parameter values was conducted by Li (1997). Based on that study (see Figure IV-1-16, Li,

1997), the linking results for difficulty were more accurate and stable when the absolute item

parameter values became larger; the linking results for discrimination did not change consistently

with the item parameter values.

Therefore, the effects of different item parameter values on scale linking were more

apparent for discrimination in this study and more obvious for difficulty in Li's study. This is

reasonable given Min and Kim's (2003) conclusion that Li's method worked better than Oshima

and colleagues' test characteristic function method (2000) for difficulty parameters and Oshima

and colleagues' method worked better for the two discrimination parameters.









Performance of Different Linking Methods

The effects of test structure, test length, sample size, ability distribution, and item

parameter values on scale linking performance were separately discussed above. However, these

factors interacted with each other and had both main and combined effect on the performance of

the four linking methods.

As summarized at the beginning of this chapter, generally speaking, all four linking

methods worked approximately equally well under all testing conditions for approximate simple

tests. For complex tests, the direct method was the best linking procedure; the item characteristic

function method and the test characteristic function method were the second and third; the

equated function method did not work well for complex tests. These results were based on the

differences between the item parameter estimates for base group and the transformed item

parameter estimates for equated group for the common items.

It is not entirely surprising that the direct method, which minimizes the sum of squared

differences between the two sets of item parameter estimates over items, was the best one across

different testing conditions because the evaluation criterion was consistent with the method.

However, the equated function method estimates the linking parameters by minimizing the sum

of squared difference between the means of the two sets of selected item parameter estimates in

the test. It uses the accumulative information of some items. Therefore, it is possible that even

though the mean parameter estimates were similar for the two groups, individual parameter

estimates were not. In the same way, item characteristic function method uses the combined

information of discrimination, difficulty, and ability item by item. The test characteristic function

method uses the accumulative information of discrimination, difficulty, and ability over all items

in the test. Therefore, item characteristic function method was better than test characteristic

function method using the criterion based on difference between item parameter estimates.









Why do the four linking methods worked equally well for approximate simple tests but

differentially poor for complex test? One of possible reason is that there is complicated

interaction between item parameter estimation error and the characteristics of the four linking

methods. More simulation studies need to be conducted to differentiate the two types of effect on

the performance of scale linking.









CHAPTER 5
CONCLUSIONS

The purpose of this study was to use simulated data to examine the performance of four

multidimensional linking methods under different testing conditions. There were one hundred

and ninety-two experimental conditions in this study: four linking methods (direct method,

equated function method, test characteristic function method, and item characteristic function

method) by two test structures (approximate simple test structure and complex test structure) by

two test lengths (20 items and 40 items) by three sample sizes (500, 1000, and 2000), and by four

different ability distributions between two groups (no difference, only mean difference, only

variance difference, and both mean and variance difference). Five hundred replications were

conducted for each of the experimental conditions. The linking performance evaluation was

based on the differences between the item parameter estimates for base group and the

transformed item parameter estimates for equated group for the common items. The mean and

standard deviation of the differences across the 500 replications were computed to examine the

accuracy and stability of the four linking methods.

Conclusions

Conclusion 1: The performance of the four linking methods. Generally speaking, the
direct method was the best linking method; the item characteristic function method and
test characteristic function method were the second and third best method; the equated
function method was the last method. However, their linking performance depended on
the following testing conditions.

Conclusion 2: The effects of test structure. For approximate simple test structure, each of
the four linking methods worked approximately equally well for all testing conditions;
For complex test structure, the equated function method worked poorly under all testing
conditions; the performance of the other three linking methods depended on other testing
conditions; the direction method was the best method for most testing conditions.

Conclusion 3: The effects of test length. The linking performance for long tests was
typically better than that for short tests except for complex tests when the sample size
was small.









Conclusion 4: The effects of sample size. The linking performance improved when the
sample size became larger, especially for complex tests.

Conclusion 5: The effects of ability distribution. Quality of linking performance declined
when there was difference in ability distribution between the two groups, especially for
complex tests; however, it seems that a between-group difference in the means was more
important than a difference in the variance.

Conclusion 6: The effects of item parameter values. Under most of the testing
conditions, the linking results for the discrimination parameter tended to be less accurate
and less stable when the item parameter had extreme values. The linking accuracy for the
difficulty parameter was not dependent on the item parameter values. The linking
stability for the difficulty parameter depended on the item parameter values only when
the sample size was large. Then, the linking results were less stable when the item
parameter had extreme values.

Future Research

In this study, there are a number of limitations, which should be considered for making the

conclusions described above. For example: (a) Although the item parameters for short and long

approximate simple tests and complex tests were randomly created in the same way and from the

same distributions, they did not have the same exact values. This should be considered when

comparing the linking results for the four types of tests; (b) The test structure was not

constructed by randomly arranging the items in the test. For the approximate simple test, the first

half of the items had higher discrimination values for the first ability and lower values for the

second ability, and the second half of the items had lower discrimination values for the first

ability and higher values for the second ability. The equated function method in this study used

the means of the first half of items (all with lower or higher values), second half of items (all

with lower or higher values) as the function to estimate the linking parameters. This may affect

the linking performance of the equated function method; (c) This study used the differences

between the item parameter estimates for base group and the transformed item parameter

estimates for equated group as the criterion, which is consistent with the minimized function of









the direct method and accordingly may favor this method. The linking performance should be

evaluated using other criteria which are consistent with the other methods to examine the

possible dependence of the results on the criteria used. These criteria include the differences

between the means of the selected item parameter estimates obtained from the two groups, the

differences between the test characteristic functions for a given range of ability, and the

differences between item characteristic functions for a given range of ability.

As mentioned in the first chapter, the development of multidimensional linking methods is

just at the infancy stage and more research is needed to obtain definitive results. Therefore, a

substantial research needs to be conducted to explore and evaluate different procedures for

multidimensional scale linking. Here are some future research topics on multidimensional IRT

scale linking.

First of all, different specific procedures within each of the four linking methods need to be

explored, compared, and evaluated so that the best method can be chosen for some specific

purpose. For example: (a) For the test characteristic function and item characteristic function

methods, how should the theta region or levels be chosen? Should we use the equally spaced grid

theta method or empirical theta method? If we choose empirical theta method, which examinee

group, base group, equated group, or combined group, should be used? Which method is better?

Should we give different weights to different theta regions and how to choose different weights?

(b) For equated function method, which item parameter estimates should be used? What

characteristics should be considered to choose the appropriate sets of items? What function

should be used to produce good linking performance?

Second, what kind of criteria should be used to evaluate the performance of different

linking methods? Within the multidimensional IRT linking and equating studies, different









criteria have been used. Even within one study, different criteria have been used. For example,

Li (1997) used bias and RMSE between the transformed linking parameter estimates and the true

linking parameters across replications in his first study and then used bias and RMSE between

true item parameter values and the transformed item parameter estimates and ability recovery in

his second study. Oshima et al. (2000) used mean and standard deviation of the linking

parameter estimates over 20 replications, bias and MSE between the estimated and true linking

parameters, correlation and mean absolute difference of linking parameter estimates across

different methods, and minimized function values by different methods. Min (2003) used bias

and RMSE between transformed item parameter estimates and the initial item parameters across

common items for the simulated data, and used the differences between the item parameter

estimates for base group and the transformed item parameter estimates for equated group across

the common items for the real data. Given these criteria, which one should we use for which

purpose for scale linking? This is a critical issue in evaluating different methods.

Third, as we discussed in last chapter, there are at least two main components in linking

errors: error caused by parameter estimation and error produced by scale transformation. The

problem is how to differentiate the estimation error from the linking error when scale linking is

conducted? To answer this question, many studies need to be conducted to evaluate the

performance of different estimation programs for multidimensional IRT. In addition, some

methods need to be developed to differentiate the estimation error from linking error and

evaluate the effects of estimation error on the performance of scale linking.

Finally, the two approaches, multidimensional IRT approach and factor analysis approach,

have different strengths and weaknesses in linking different scales. As Min and Kim (2003)

found in their study that Li's method worked better for difficulty parameters and Oshima and









colleagues' method (e.g., test characteristic function method) worked better for the two

discrimination parameters. Therefore, how to use the strengths of the two approaches to develop

a combined method for multidimensional scale linking is an important topic in the future

research.










APPENDIX
ACCURACY AND STABILITY FOR DIFFERENT LINKING METHODS



I I I
80





S 40


i so


0
a
40


si


a I
a


SI I

4 i


4 0

0


a
-0.6 -0.4 -0.2 0 0.2 0. 0.6
Mean of Dlferences for a2




4


a

40




-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of Dilhrences for d


80
40

0

so


40


0
880










40
80



o
040





0
80
40
0









0


I


0.1 03 05 0.7 0.9 1.1 1.
Standard Deviation of DIlarences for al















0.1 03 05 0.7 0.9 1.1 13
Standard Deviation d Dilerences for a2















0.1 03 05 0.7 0.9 1.1 13
Standard Deviation of Dferences for d
Standard Dvaio tDilrncsfr d


Figure A-1. Accuracy and stability for different linking methods (APP, n=20, N=500, Gl)




138


-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of DIlerences for al










80


40
a

0 /

I40





-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences for al


so
40


40


40
s o

40

-0.6 -0.4 -0.2 0 0.2 0.4A 0.6
Mean of Dfferences fbr a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences tr al


480
40


0


80



0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Dferences tor a2


ao
40
0


0


0o
0
0:


-0.6 -0.4 -0.2 0 0.2 0.4 B
Mean of DIferences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatan of Dlherences for d


Figure A-2. Accuracy and stability for different linking methods (APP, n=20, N=500, G2)


]


0
40
a
80
40


40


40
0a -- I


J-


-


- ,I


OPEL-.











80




40
i-u
40







-0.6 -0.4 -0.2 0 0.2 0A OB
Mean of Dlferences for al




40

0








-0.6 -0.4 -0.2 0 0.2 0.4A 0.
Mean of Differences br a2

0

40

80

0

so






-0.8 -0.4 -0.2 0 0.2 0A OB
Mean of DIffrences for d


80


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences fr al



40



40

O'



80
40

0.1 0.8 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences br a2


8o
40
0

80
S.40:M^^^
0-^


0I



80


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-3. Accuracy and stability for different linking methods (APP, n=20, N=500, G3)











80



40
a

0


40

40




-0.6 -0.4 -0.2 0 0.2 0A OB
Mean of Differences Mr al


0
40
a
8I

40







-0.6 -0.4 -02 0 02 0.4 0.6
Mean of Dfferences for a2






8K-
40
A 0

4
I ll II0







-0.6 -0.4 -0.2 0 02 A0.4 O
Mean of Dfferences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences fr al


0
140W



0


40
0.^^^^^


80


0.1 0S 05 0.7 0.9 1.1 1.
Standard Deviation of DIlerences fr a2


80
40
0


S40
0*^^^


I-




0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatian of Dllerences or d


Figure A-4. Accuracy and stability for different linking methods (APP, n=20, N=500, G4)












80
40
a1
8:


4 .
8 0

40





-0.8 -0.4 -0.2 0 0.2 0.4A OB
Mean of Differences or al


0
4:

80
40






















-0.6 -0.4 -0.2 0 0.2 0. B
Mean of Dflferences for a2


80
40














0 ,
40








40


s o



-0.6 -0.4 -0.2 0 0.2 0.4A 0.6
Mean of DllIrences for da
Mean of Dlffrences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaion of DIfferences or al


0.1 0.8 05 0.7 0.9 1.1 1.8
Standard Devialon of DIfferences for a2


0.1 0. 05 0.7 0.9 1.1 1.8
Standard Deviation of Dllerences br d


Figure A-5. Accuracy and stability for different linking methods (APP, n=20, N=1000, Gl)














so
i40










-0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dierences for al



0
so







40
0
P40

















0 .H |
-0.6 -0.4 -02 0 0.2 0.4 0.8
Mean of Differences for a2
4


0







P4 I

1:4














-0. -0.4 -0.2 0 0.2 0.4 OB
Mean of Dlffrences for d


80
40j


0
a
sao
440
0
80

40
a


0.1 0. 05 0.7 0.9 1.1 1.
Standard Deviation of Diferences for al


80
40


40
0
40
sio

40
0JI


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dferences for a2



80
40



i: I
0
0





0.1 0.3 0.5 0.7 0.9 1.1 1.8
Standard Devlaton of Dlfferences for d


Figure A-6. Accuracy and stability for different linking methods (APP, n=20, N=1000, G2)




143












80

40











-0.8 -0.4 -0.2 0 0.2 0.4 0.B
Mean of Differences or al1
a


40



















0

Ma o
40
40




















-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of Dfferences tr a2

s i
40
40

















0





-0.6 -0.4 -0.2 0 0.2 0A OB
Mean of Dfferences for da2
Mean of DNlerencas for d


40



0







0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences tr al




O'






o'


0I
40
0




0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dfferences for a2

iII


0


O'



40




0.1 0.8 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-7. Accuracy and stability for different linking methods (APP, n=20, N=1000, G3)











80

40
a




80U
I
80
40


40

-0.6 -0.4 -0.2 0 0.2 4A OB
Mean of Dllerences for al


80




40

80|






-0.6 -0.4 -0.2 0 02 CA 0.
Mean of Dtlleence for a2
















-0.6 -0.4 -0.2 0 02 CA OS
Mean of Dlflrena for d


0



0
4'





0

0.1 0.3 05 0.7 0.- 1.1 1.8
Standard Deviation o DIferences for al


Standard DeviatA n of DIferences for a2
















0.1 0. 05 0.7 05 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-8. Accuracy and stability for different linking methods (APP, n=20, N=1000, G4)











80

40










-0.8 -0.4 -0.2 0 0.2 0.4 0.B
Mean of Differences or al
a








0 !
40

















O t
0 a
40



















-0.6 -.4 -0.2 0 0 0.4 0.
Mean of Dfferences or a2
40
40














o. |
0 ,

4


0 ,
80







40







0 ,



-0. -0.4 -0.2 0 02 0.4 OB
Mean of Differences for d
Mean of DNlerencas for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al


0.1 0.3 05 0.7 0.9 1.1 1.3

















0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIferences hbr d
I









I


Figure A-9. Accuracy and stability for different linking methods (APP, n=20, N=2000, Gl)









80
40
ao
0 U
40
. .
80
40__

8O
40
-0.6 -0.4 -0.2 0 0.2 0. OB
Mean of Differences for al


0 ,

s:
40
s:
40



-0.6 -0.4 -0.2 0 0.2 0.4A 0.6
Mean of Differences for a2
: i

4

: i
8

s o




-0.6 -0.4 -0.2 0 0.2 0.4A 0.6
Mean of DIferences for d


&ILI_




Ei



0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences for al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences or a2












0.1 0. 05 0.7 0. 1.1 1.
Standard Deviation of Differences r d


Figure A-10. Accuracy and stability for different linking methods (APP, n=20, N=2000, G2)











80
40
a

8
40






40

-0.6 -0.4 -0.2 0 0.2 0. OB
Mean of Dfferences for al



40

0 ,
40






40

-0.6 -0.4 -0.2 0 02 0A .B
Mean of Differences for a2






40
s:

40
s:


40

-0.6 -0.4 -0.2 0 02 OA OB
Mean of DIferences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences for al


I i
0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIfferences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaton of DIerences for d


Figure A-11. Accuracy and stability for different linking methods (APP, n=20, N=2000, G3)










a80

40


80: _
0







-0.6 -0.4 -0.2 0 0.2 0.4 .OB
Mean of Differences for al


80


80
40







-0.6 -0.4 -0.2 0 02 0A .B
Mean of Differences for a2



40



s:
40



s:


40

-0.6 -0.4 -0.2 0 02 OA OB
Mean of Differences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences br al


I i
0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIfferences tr a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaton of Dlherences or d


Figure A-12. Accuracy and stability for different linking methods (APP, n=20, N=2000, G4)














80



40

0


40

0




-0.6 -.4 -0.2 0 02 .A OB
Mean of Differences or al




40




0
40







40

-0.6 -0.4 -0.2 0 02 OA .B
Mean of Differences for a2




40



40
0 ,



40




-0.6 -0.4 -0.2 0 02 OA OB
Mean of Dfferences for d


80
40


0

. 40


0


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaton of Differences ar al


oi
40





40



0.


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIfferences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatin of Dllerences Ifr d


Figure A-13. Accuracy and stability for different linking methods (APP, n=40, N=500, Gl)





_
I



-
I


I
I



I
I



I














O80
40
0


S8o0
j40
0

80
40
0

I80

0


-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of Differencs for a2


-0.6 -0.4 -0.2 0 0.2 A0.4 0
Mean of Diflrences for d


80
j40


0

0
j40
0

80
j40
0



0


40
0



80
0

80


j40
a

I80
0
0


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation df Diferencs for a2


-I







0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Difrences for d


Figure A-14. Accuracy and stability for different linking methods (APP, n=40, N=500, G2)


-I I
-I I

-I I
-I I


-I I
-I I


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al















80

80
40

SI


0




-0.6 -0.4 -0.2 0 02 0A OB
Mean of Differences or al



80
40






80
80
40


40


-0.6 -0.4 -0.2 0 02 OA .B
Mean of Differences or a2








0 ,
s:


40

:o
0 ,


40

-0.6 -0.4 -0.2 0 02 OA OB
Mean of Dfferences for d


80
40


80
400
0

80

40


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences fr al


80
40

0
80
40
40

80
0-.4


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIfferences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatin of Dlherences or d


Figure A-15. Accuracy and stability for different linking methods (APP, n=40, N=500, G3)





_
I


I
I


I
I


I
I


I













80

40



8 0



40










80




40





0
-0.6 -0.4 -0.2 0 0.2 CA 0.
Mean of Differences hor al








40








40












-0.6 -0.4 -0.2 0 02 0A 0.
Mean of Differences for da


40



40

-0.6 -0.4 -0.2 0 0.2 A 0.6
Mean od Dlbranoes Ibr d


80

40

80

0

80

a-^B


80
40

0.1 0. 05 0.7 0. 1.1 1.3
Standard Devlatlon of Dlerences or al


40

0'

40

0
80
40.
0-^^-


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIferences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatlon of Dlerences for d


Figure A-16. Accuracy and stability for different linking methods (APP, n=40, N=500, G4)





I



I
I


I



I
I



-









80
40

40

0
40



-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences for al

I 0


:... .
80


40

80

-0.6 -.4 -0.2 0 0 .2 A 0.B
Mean of D flerences or a2


0 ,


40

s:

40
-0.6 -0.4 -0.2 0 02 0A OB
Mean of Dlberences for d


I








mm i .
Ei





0.1 0.8 05 0.7 0.0 1.1 1.8
Standard Devilion of DIferences hr al
I






K.

K.
LI


Ei











0.1 0.8 05 0.7 0.0 1.1 1.8
Standard Devllon of DIerences hr a2


0.1 0. 05 0.7 0. 1.1 1.
Standard Devlation of DIbrences for d


Figure A-17. Accuracy and stability for different linking methods (APP, n=40, N=1000, Gl)










80

40
80

40


m .
40


40

-0.6 -0.4 -0.2 0 0.2 0.4 A
Mean of Differences for al



40
o i


: i






-0.6 -0.4 -0.2 0 0.2 A 0.
Mean of Differences for a2






0 ,


40


40

-0.6 -0.4 -0.2 0 02 0.4 0.8
Mean of Dlflerences for d


E-L
I










0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaion of DIfferences tr al




U-










0.1 0.8 05 0.7 0. 1.1 1.8
Standard Deviallon of DIMerences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaton of Dllerences for d


Figure A-18. Accuracy and stability for different linking methods (APP, n=40, N=1000, G2)










80

40
80

40


m .



-0.8 -0.4 -0.2 0 02 0.4 .OB








Mean of DIfferences or a2
40


0


s:
40


s:
40









-0.6 -0.4 -0.2 0 02 0.4
Mean of Dlferences for a2
; iI


i















-0.6 -0.4 -0.2 0 0.2 OA B
Mean of DIferences for a2







Mean of Dlflerences for d


I










0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences tor al
I













0.1 0.38 05 0.7 0.9 1.1 1.38
Standard Deviation of DIfferences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-19. Accuracy and stability for different linking methods (APP, n=40, N=1000, G3)











80
40


40
8 0





0... ....
40


40

-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences for al


80
40

0

40
80
40

80
40


-0.6 -.4 -0.2 0 02 OA OB
Mean of Dflerences or a2




0 ,
4:
0 ,


s:
40

8s
40

-0.6 -0.4 -0.2 0 02 0A OB
Mean of Dllerences for d


HaL







0.1 0.8 05 0.7 0.0 1.1 1.8
Standard Deviatilon of DIferences hr al
I



U-









0.1 0. 05 0.7 0. 1.1 1.
Standard Deviation of Dlerences tor a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatlan of Dllferences bfr d


Figure A-20. Accuracy and stability for different linking methods (APP, n=40, N=1000, G4)










80
I |
40

0
40


0 -
40

80
40
-0.6 -0.4 -0.2 0 0.2 0. OB
Mean of Dlfferences for al

80
40
0 ,

0 ,
0 ,
40

80
40
-0.6 -0.4 -0.2 0 02 0.4A OB
Mean of Dlferences for a2


40
0 ,

0 a

s:
0 a

80
40
0
-0.6 -0.4 -0.2 0 0.2 0.4A 0.6
Mean of DIAerences bfor d


I I


uI







Standard Deviation of Differences or a
I












0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIIerences or a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlherences or d


Figure A-21. Accuracy and stability for different linking methods (APP, n=40, N=2000, Gl)










80

40
a1

40



80
.A .

40








-0
-0.6 -.4 -0.2 0 02 A 0.B
Mean of Dfferences or al

0



8 0










0 ,
40
40

-0.6 -0.4 -0.2 0 0.2 0A OB
Mean of Dferences for a2




40


0 ,



80


-0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of DIllerences Ibr d


I
I



E i



0.1 0. 05 0.7 0.9 1.1 1.
Standard Deviallon of DIferences f a1
I
HI










I i










0.1 0. 05 0.7 0. 1.1 15
Standard Deviatlon of DIferences for a2
I |











StnardDm l atbn of lernce ar


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatlin of Dllferences br d


Figure A-22. Accuracy and stability for different linking methods (APP, n=40, N=2000, G2)










80
40
a0
o80
40

0
o80
40

80
40
0 -


-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of Differences for al

80
40
80




40
0





-0.6 -0.4 -0.2 0 0.2 0. O
Mean of Dfferences bor a2



80
0 ,

s:
0 ,
0 ,



40

-0.6 -0.4 -0.2 0 0.2 0.A O
Mean of DIffrences for d


I


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dllerences Ir d


Figure A-23. Accuracy and stability for different linking methods (APP, n=40, N=2000, G3)





uI


uI



I IK

0.1 0.8 05 0.7 0.9 1.1 1.8
Standard Deviation of DIerences for al








I l
*,_



0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences tr a2










so
40
i
4s
40

80

40
0 0

-0.6 -.4 -0.2 0 0.2 0.4 0
Mean of Differences or al




0 ,
40

0 ,
40
80

40
-0.8 -0.4 -0.2 0 02 0.A4 B
Mean of Dlferences hor a2
so






0 ,



40
: i



-0.6 -0.4 -02 0 0.2 4A 0.
Mean of DIferences for d


uI
ILI
I

I

I
I
I i






0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences or al















Standard Deviagion of DIfferences tor a2
I

I
I

I
I


I



0.1 0.8 5 0.7 0.9 1.1 1.8
Standard Devlatilon of DIlerences for a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dllerences for d


Figure A-24. Accuracy and stability for different linking methods (APP, n=40, N=2000, G4)
























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences or al


80
40
a
80
40
a
80
40
a
80
40
a


1 *
I

I
I







0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al




I









0.1 0.3 0.5 0.7 09 1.1 1.3
Standard Devlation of DIIffrences for a2
~Emli-


_p


Ii a


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Differences for a2


-0.6 -0.4 -0.2 0 02 0.4 0.6
Mean of Dilerences for d


III







0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Derences for d


Figure A-25. Accuracy and stability for different linking methods (COM, n=20, N=500, Gl)














p



1.rrn~nrrn.


02 0.4 O


Mean of Dlferences for al


-0.6 -0A -0.2 0 0.2 0.4 0.6
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 0A 06
Mean of Dfferences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devation of Differences for al


I H






0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for a2
II















0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d
Stadad Dvll~n o Dl~nce frI


Figure A-26. Accuracy and stability for different linking methods (COM, n=20, N=500, G2)


-0.6 -0.4 -0.2 0



























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Dfferences for a2


-0.6 -0.4 -0.2 0 0.2 0.4 B
Mean of Dfferences for d


S!-












0.1 0. 05 0.7 0.9 1.1 1.
Standard Deviatdon of Diferences for al


0.1 0.8 05 0.7 0.9 1.1 1.3
Standard Deviation of DIferences for a2









i i_







0.1 0. 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for d


Figure A-27. Accuracy and stability for different linking methods (COM, n=20, N=500, G3)






















-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Dllerences for al


-0.6 -0.4 -0.2 0 0.2 0.4 .B
Mean of Dfferences for a2


0.1 0. 05 0.7 0.9 1.1 1.
Standard Devlallon of Dferences for al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Dfferences tor a2




^^3

l__I


0.1 0.3
Standard


05 0.7 0.9 1.1 1.3
Deviation of DIereances for d


Figure A-28. Accuracy and stability for different linking methods (COM, n=20, N=500, G4)


-0.6 -0.4 -0.2 0 0.2 0A 0.
Mean of Dfferences for d










BnI

= 0 0

404







-0. -0.4 -0.2 0 02 CA 0.8 0.1 0.3 05 0.7 0.9 1.1 1.3
Mean of Differences hor al Standard Deviation of Differences hor al
S 80 80
4 40 -i 40













S a80 ___,0m
0 = 0
0 0







-0.6 -0.4 -0.2 0 02 0A .B 0.1 0.3 05 0.7 0.9 1.1 1.3
Mean of Differences or a2 Standard Deviation of Differences tbr al









.- o __ _o
0 i I o :
= 0 = 40
0 0









S | o so
S 40 40
= 0 = 0












-0.6 -.4 -0.2 0 0.2 A04 0. 0.1 0.3 05 0.7 0.9 1.1 1.3
Mean of Dllerences for d Standard Deviation of Diferences for d





Figure A-29 Accuracy and stability for different linking methods (COM, n20, N 1000,
a 0 I
-0.6 -OA -0.2 0 0.2 0.4 0. 0.1 0.8 05 0.7 0-9 1.1 1.8
Mean of Dillerances for d Standard Devialan of DIerences for d

Figure A-29. Accuracy and stability for different linking methods (COM, n=20, N=1000, GI)




























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences or al


-0.6 -0.4 -0.2 0 0.2 0.4 .B
Mean of Differences for a2





"40







0 ,
40



40

-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Dillerences for d


80
40
a






40
I






a
0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al



0



40
80


40





0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences tr a2


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviaton of Dlerences for d


Figure A-30. Accuracy and stability for different linking methods (COM, n=20, N=1000, G2)



























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences or al


-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences for a2


0





oi Io
40
0

AII






0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences br al


80


0

80:
0








0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences br a2


40 401

-0.6 -0.4 -0.2 0 0.2 0A 0. 0.1 0. 05 0.7 0.9 1.1 1.
Mean of Dfferences for d Standard Deviatln of Diferences for d


Figure A-31. Accuracy and stability for different linking methods (COM, n=20, N=1000, G3)




























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 4A 0.
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 A 0.6
Mean of DIfferences bfr d


80

40
aI





80



80



Standard Deviallon of Dflerences for al









40








0.1 a 05 0.7 09 1.1 1.3
Standard Deviaton of DIferences for a2


so






6I


0.1 0. 05 0.7 0. 1.1 1.3
Standard Deviatin of Dlferences for d


Figure A-32. Accuracy and stability for different linking methods (COM, n=20, N=1000, G4)
























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Dlferences for al


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Dfferences for d


,*











0.1 0.3 0.5 0.7 0.9 1.1 1.3
Standard Devlaton of Differences for al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation do DIfferences for a2
*1 I





I -I







0.1 0.S 5 0.7 0.9 1.1 1.3
Standard Deviation of Diflerences for d


Figure A-33. Accuracy and stability for different linking methods (COM, n=20, N=2000, Gl)
























-0.6 -0.4 -0.2 0 0.2 0. OB
Mean of Diferences or al


-0.6 -0.4 -0.2 0 0.2 0. OB
Mean of Dierences for a2


0.1 0.3 05 0.7 09 1.1 1.3
Standard Deaaion of Differences for al


i

0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation f DIfferences or a2


0 40

440

-0.6 -0.4 -0.2 0 0.2 0A 0. 0.1 OA 05 0.7 0. 1.1 1.
Mean of Dfferences for d Standard Devation of Differences for d


Figure A-34. Accuracy and stability for different linking methods (COM, n=20, N=2000, G2)




























-0.6 -0.4 -0.2 0 0.2 OA 0.
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 0A 0.
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 OA 0.
Mean of Dfferences for d


0.1 0S 05 0.7 0.9 1.1 15
Standard Devlatdon of Diferences for al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Diferences for a2




!













0.1 0.8 05 0.7 0.9 1.1 1.8
Standard Deviation of Dlhrences for d


Figure A-35. Accuracy and stability for different linking methods (COM, n=20, N=2000, G3)






















-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences or al


-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences or a2


-0.6 -0.4 -0.2 0 0.2 A0.4 O
Mean of Dferences for d


&


*

I
U-i-



0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devation of Differences for al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIfferences or a2


!











0.1 aS 05 0.7 0. 1.1 15
Standard Deviation of Dilerences for d
0.1 08 05 0.7 .0 11 1.
Stadad Dvitio &D~henes r!


Figure A-36. Accuracy and stability for different linking methods (COM, n=20, N=2000, G4)
























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences or al


-0.6 -0.4 -0.2 0 0.2 A 0.6
Mean of Differences for a2


40
0


o
40
0







0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences tr al



0

40

o







0.1 0.83 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences br a2


- I ii01 '
S, I .
-0.6 -0.4 -0.2 0 0.2 0.4 0. 0.1 0.3 05 0.7 0.9 1.1 1.3
Mean of Differences for d Standard Deviatan of DIerences or d


Figure A-37. Accuracy and stability for different linking methods (COM, n=40, N=500, Gl)
























-0.8 -0.4 -0.2 0 0.2 0A B0
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 0.4 0.
Mean of Dfferences for a2



I -











-0.6 -0.4 -0.2 0 0.2 .A 0.
Mean of Dffrences for d


i____I













0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIfferences hor al















0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIIerences or a2















0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIerences bfr d


Figure A-38. Accuracy and stability for different linking methods (COM, n=40, N=500, G2)



























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 A 0.6
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 A0.4 0
Mean of Dfferences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Diferences for al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of Differences for a2


I
I I




I I

0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of DIerences for d


Figure A-39. Accuracy and stability for different linking methods (COM, n=40, N=500, G3)





















-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences or al


-0.6 -0.4 -0.2 0 0.2 0.4 0
Mean of Dfferences fr a2


E



-0.6 -0.4 -02 0 0.2 0A 06
Mean of DIffrences for d


Iu.









0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al


1 5









0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dfferences br a2
I


Standard Devlation of DIbrence~s br a2


I*


I


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-40. Accuracy and stability for different linking methods (COM, n=40, N=500, G4)




























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Dfferences for d


0.1 0S 05 0.7 0.9 1.1 15
Standard Devoaton of Difterences for al


















0.1 0. 0.5 0.7 0.9 1.1 1.3
Standard Devilatlon of Dlerences for a2


SI




0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devialon of DIllreances for d


Figure A-41. Accuracy and stability for different linking methods (COM, n=40, N=1000, Gl)

























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences for al
















-0.6 -0.4 -0.2 0 0.2 0.4 06
Mean of Differences for a2


I 80


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Dfferences for d


0.1 0 05 0.7 0. 1.1 1
Standard Devialon of DIHerences or al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation f Differences for a2

I___














0.1 0.3 05 0.7 0. 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-42. Accuracy and stability for different linking methods (COM, n=40, N=1000, G2)




























-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Differences for al


-0.6 -0.4 -0.2 0 02 0.4 0.6
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 4A 0.
Mean of Dfferences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al


















0.1 0s 05 0.7 A 1.1 1.3
Standard Devialation DIMerences for a2

I
















0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviatlan of DMerences for d


Figure A-43. Accuracy and stability for different linking methods (COM, n=40, N=1000, G3)



























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Dlferences for al


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Dfferences for d


40
0


40
0

80

40
0


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Differences for al






80
40



0
0








0.1 0.3 05 0.7 0.9 1.1 1.3









40








0.1 0.8 05 0.7 0.9 1.1 1.8
Standard DevIation of DIlFerences for da2



a




0.1 0.8 05 0.7 0.9 1.1 1.8
Standard Devialan of DIlerances Ior d


Figure A-44. Accuracy and stability for different linking methods (COM, n=40, N=1000, G4)




























-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Dlferences for al


-0.6 -0.4 -0.2 0 0.2 0.4 B
Mean of Dllerences for a2


80
40

80
4 0
40





so
40




-0.6 -0.4 -0.2 0 02 A0.4 O
Mean of DIlbrences for d


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIfferences fr al


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Devlation of DIferences fr a2


















0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dlerences for d


Figure A-45. Accuracy and stability for different linking methods (COM, n=40, N=2000, Gl)



















-0.6 -0.4 -0.2 0 02 0.4 OB
Mean of Differences for al


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Differences for a2


-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Diferences for d


5-;I


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Diferences for al





.1





0.1 0S 05 0.7 0.9 1. 5
Standard Deviaton of DIMerences for a2
5!


















0.1 0a 05 0.7 O0 1.1 15
Standard Deviaton of Dilaerences for d
MI III
0.1 0. 5070.!. .
Stadar Dvialo ofDIerece Ib a

HI|


Figure A-46. Accuracy and stability for different linking methods (COM, n=40, N=2000, G2)






























80
40


-0.8 -0.4 -02 0 0.2 0.4 0.6
Mean of Diferences for al


I


0
80
40

80
40
0 i -- ,-- i -- i
-0.8 -0.4 -0.2 0 0.2 0.4 0.8
Mean of Differences for a2


80
4 n
0 I
80
40
0
-0.6 -0.4 -0.2 0 0.2 0.4 0.6
Mean of Diferences for d


i
I
I
I
I


0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation f DiWerences for a2


a...
0.1 0.3 05 0.7 0.9 1.1 1.
Standard Deviation of Differences for d


Figure A-47. Accuracy and stability for different linking methods (COM, n=40, N=2000, G3)


I


UI



Ei
-, II






0.1 0.3 05 0.7 0.o 1.1 1.
Standard Deviation of Diferences for al


. .


n


I
I
I
I
I























-0.6 -0.4 -0.2 0 0.2 A0.4 0
Mean of Differences or al


-0.6 -0.4 -0.2 0 0.2 0.4 OB
Mean of Dfferences for a2


-0.6 -0.4 -0.2 0 0.2 0.4 O
Mean of Dfferences for d


*E i










0.1 0. 05 0.7 0.9 1.1 1.
Standard Deviallon of Derences for al


0.1 0.3 05 0.7 09 1.1 1.3
Standard Deviation Dirkrences for a2














0.1 0.3 05 0.7 0.9 1.1 1.3
Standard Deviation of Dferences for d
!-








Standard Devilian of DIHrences for d


Figure A-48. Accuracy and stability for different linking methods (COM, n=40, N=2000, G4)









LIST OF REFERENCES

Ackerman, T. A. (1994). Using multidimensional item response theory to understand what items
and tests are measuring. Applied Measurement in Education, 7, 255-278.

Ackerman, T. A. (1996). Graphic representation of multidimensional item response theory
analyses. Applied Psychological Measurement, 20, 311-329.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43,
561-573.

Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological
Measurement, 16, 87-96.

Baker, F. B. (1993). Equating tests under the nominal response model. Applied Psychological
Measurement, 17, 239-251.

Bateley, R. M., & Boss, M. W. (1993). The effects on parameter estimation of
correlateddimensions and a distribution-restricted trait in a multidimensional item response
model. Applied Psychological Measurement, 17, 131-141.

Bedescu, D. (1985). Efficiency of linear equating as a function of the length of the anchor test.
Journal of Educational Measurement, 22, 13-20.

Bock, R. D. (1972). Estimating item parameters and latent ability when the responses are scored
in two or more nominal categories. Psychometrika, 37, 29-51.

Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters:
Application of an EM algorithm. Psychometrika, 46, 443-459.

Bock, R. D., Gibbons, R., & Muraki, E. (1988). Full-information item factor analysis. Applied
Psychological Measurement, 12, 261-280.

Bock, R. D., Gibbons, R., Schilling, S. G., Muraki, E., Wilson, D. T., & Wood, R. (1999).
TESTFACT 3: Test scoring, items statistics, and full-information item factor analysis.
Chicago: Scientific Software International.

Carlson, J. E. (1987). Multidimensional item response theory estimation: A computer program.
Unpublished manuscript.

Cattell, R. B. (1978). The scientific use offactor analysis. New York: Plenum.

Cohen, A. S., & Kim, S. H. (1998). An investigation of linking methods under the graded
response model. Applied Psychological Measurement, 22, 116-130.

Comery, A. L., & Lee, H. B. (1992). A First course in factor analysis. Hillsdale, NJ:
Erlbaum.Cook, L. L., Eignor, D. R. (1991). IRT equating methods. Educational
Measurement: Issues and Practice, 10, 37-45.









Cook, L. L., Eignor, D. R., & Taft, H. (1981, April). A comparative study of curriculum effects
on the stability of RT and conventional item parameter estimates (RR-85-38). Princeton,
NJ: Educational Testing Service.

Cook, L. L., & Petersen, N. S. (1987). Problems related to the use of conventional and item
response theory equating methods in less than optimal circumstances. Applied
Psychological Measurement, 11, 225-244.

Cureton, E. E., & D'Agostino, R. B. (1983). Factor analysis: An applied approach. Hillsdale,
NJ: Erlbaum.

Davey, T. C., Oshima, T. C., & Lee, K. (1996). Linking multidimensional item calibrations.
Applied Psychological Measurement, 20, 405-416.

Divgi, D. R. (1985). A minimum chi-square methods for developing a common metric in item
response theory. Applied Psychological Measurement, 9, 413-415.

Fitzpatrick, A. R., & Yen, W. M. (2001). The effects of test length and sample size on the
reliability and equating of tests composed of constructed-response items. Applied
Measurement in Education, 14, 31-57.

Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor
analysis.Multivariate Behavioral Research, 23, 267-269.

Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum.

Gosz, J. K. & Walker, C. M. (2002, April). An empirical comparison of simple versus complex
multidimensional item response data. Paper presented at the annual meeting of the
American Educational Research Association, New Orleans, LA.

Gosz, J. K., Walker, C. M. (2002, April). An empirical comparison of multidimensional item
response data using TESTFACT andNOHARM. Paper presented at the Annual Meeting of
the National Council for Measurement in Education (NCME), New Orleans, Louisiana.

Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw Hill.

Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method.
Japanese Psychological Research, 22, 144-149.

Hason, B. A., & Beguin, A. A. (2002). Obtaining a common scale for item response theory item
parameters using separate versus concurrent estimation in the common-item equating
design. Applied Psychological Measurement, 26, 3-24.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and
applications. Boston: Kluwer Nijhoff Publishing.

Harwell, M., Stone, C. A., Hsu, T. C., & Kirisci, L. (1996). Monte Carlo studies in item response
theory. Applied Psychological Measurement, 20, 101-125.









Hirsch, T. M. (1989). Multidimensional equating. Journal of Educational Measurement, 26,
337-349.

Hirsch, T. M. (1988). Multidimensional equating. Unpublished doctoral dissertation, Florida
State University, Tallahassee, FL.

Kaskowitz, G. S., & De Ayala, R. J. (2001). The effect of error in item parameter estimates on
the test response function method of linking. Applied Psychological Measurement, 25, 39-
52.

Kelderman, H. (1997). Loglinear multidimensional item response models for polytomously
scored items. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern
item response theory. New York, NY: Springer.

Kim, H. (1994). New techniques for the dimensionality assessment of standardized test data.
Unpublished doctoral dissertation. University of Illinois at Urbana-Champaign. Urbana-
Champaign, IL.

Kim, S. H., & Cohen, A. S. (1995). A minimum chi-square method for equating tests under
the graded response model. Applied Psychological Measurement, 19, 167-176.

Kim, S. H., & Cohen, A. S. (2002). A comparison of linking and concurrent calibration under the
graded response model. Applied Psychological Measurement, 26, 25-41.

Klein, L. W., & Kolen, M. J. (1985, April). Effect of number of common items in common-item
equating i i/th nonrandom groups. Paper presented at the annual meeting of American
Educational Research Association, Chicago.

Knol, D. L., & Berger, M. P. F. (1991). Empirical comparison between factor analysis and
multidimensional item response models. Multivariate Behavioral Research, 26, 457-477.

Kolen, M. J. (2004a). Population invariance in equating and linking: Concept and history.
Journal of Educational Measurement, 41, 3-14.

Kolen, M. J. (2004b). Linking assessment: Concept and history. AppliedPsychological
Measurement, 28, 219-226.

Kolen, M. J., & Brennan, R. L. (2004). Test equating,. scaling, and linking: Methods and
Practices (2nd ed.). New York, NY: Springer.

Lautenschlager, G. J., Flaherty, V. L., & Park, D. G. (1994). IRT differential item functioning:
An examination of ability scale purifications. Educational and Psychological
Measurement, 54, 21-31.

Lee, K., & Oshima, T. C. (1996). IPLINK: Multidimensional and unidimensional item parameter
linking in item response theory. Applied Psychological Measurement, 20, 230.









Li, Y. H. (1997). An evaluation of multidimensional IRT equating methods by assessing the
accuracy of transforming parameters onto a target test metric (Doctoral dissertation,
University of Marryland, 1997). Dissertation Abstract International, UMI Number
9816494.

Li, Y. H., & Lissitz, R. W. (2000). An evaluation of the accuracy of multidimensional IRT
linking. Applied Psychological Measurement, 24, 115-138.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale,
NJ: Erlbaum.

Loyd, B. H., & Hoover, H. D. (1980). Vertical equating using the Rasch model. Journal of
Educational Measurement, 17, 179-193.

Marco, G. L. (1977). Item characteristic curve solutions to three intractable testing problems.
Journal of Educational Measurement, 14, 139-160.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

McDonald, R. P. (1981). The dimensionality of tests and items. British Journal ofMathematical
and Statistical Psychology, 34, 100-117.

McDonald, R. P. (1997). Normal-ogive multidimensional model. In W. J. van der Linden & R.
K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY:
Springer.

McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah, NJ: Erlbaum.

McDonald, R. P. (2000). A basis for multidimensional item response theory. Applied
Psychological Measurement, 24, 99-114.

McKinley, R. L., & Reckase, M. D. (1983). An extension of the two-parameter logistic model to
the multidimensional latent space (Research Report, ONR 83-2). Iowa City, IA:
American College Testing Program.

Millsap, R. E. (2005). Four unresolved problems in studies of factorial invariance. In A.
Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary Psychometrics. Mahwah, NJ:
Lawrence Erlbaum Associates.

Min, K. S. (2003). The impact of scale dilation on the quality of the linking of multidimensional
item response theory calibrations. Unpublished Dissertation, Michigan State University,
East Lansing, MI.

Mroch, A. A., & Bolt, D. M. (2006). A simulation comparison of parametric and nonparametric
dimensionality detection procedures. Applied Measurement in Education, 19, 67-91.

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied
Psychological Measurement, 16, 159-176.









Muraki, E. (1999). POLYFACT version 2 [Computer program]. Princeton, NJ: Educational
Testing Service.

Muraki, E, & Carlson, J. E. (1995). Full-information factor analysis for polytomous item
responses. Applied Psychological Measurement, 19, 73-90.

Muraki E., & Engelhard, G. (1985). Full-information item factor analysis: Applications ofEAP
scores. Applied Psychological Measurement, 9, 417-430.

Muthen, L. K., & Muthen, B. (1998). MPLUS: The comprehensive modeling program for
applied researcher: User's guide. Los Angeles: Muthen & Muthen.

Oshima, T. C., Miller, M. D. (1992). Multidimensionality and item bias in item response theory.
Applied Psychological Measurement, 16, 237-248.

Oshima, T. C., Davey, T. C., & Lee, K. (2000). Multidimensional linking: Four practical
approaches. Journal ofEducational Measurement, 37, 357-373.

Park, D. G., & Lautenschlager, G. J. (1990). Improving IRT item bias detection with iterative
linking and ability scale purification. Applied Psychological Measurement, 14, 163-173.

Peterson, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating
methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137-
156.

Peterson, N. S., Kolen, M. J. & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L.
Linn (Ed.), Educational measurement (pp. 221-262). New York: American Council on
Education and Macmillan.

Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied
Psychological Measurement, 9, 401-412.

Reckase, M. D. (1991). The discriminating power of items that measure more than
onedimension. Applied Psychological Measurement, 15, 361-373.

Reckase, M. D. (1997a). A linear logistic multidimensional model for dichotomous item
response data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern
item response theory. New York, NY: Springer.

Reckase, M. D. (1997b). The past and future of multidimensional item response theory.
Applied Psychological Measurement, 21, 25-36.

Rechase, M. D., & Martineau, J. (2004, October). The vertical scaling of Science Achievement
Tests. Paper commissioned by the Committee on Test Design for K-12 Science
Achievement, Center for Education, National Research Council.









Ree, M. J., & Jensen, H. E. (1983). Effects of sample size on liner equating of item characteristic
curve parameters. In D. J. Weiss (Ed.), New horizons in testing (pp. 135-146). New York:
Academic.

Roussos, L. A., Stout, W. F., & Marden, J. I. (1998). Using new proximity measures with
hierarchical cluster analysis to detect multidimensionality. Journal of Educational
Measurement, 35, 1-30.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores.
Psychometrika Monograph, No. 17.

Schonemann, P. H. (1966). A generalized solution of the orthogonal procrustes problem.
Psychometrika, 31, 1-10.

Schonemann, P. H., & Carroll, R. M. (1970). Fitting one matrix to another under choice of a
central dilation and a rigid motion. Psychometrika, 35, 245-255.

Simpson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.),
Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82-98).
Minneapolis: University of Minnesota, Department of Psychology, Psychometric
Methods Program.

Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory.
Applied Psychological Measurement, 7, 201-210.

Stone, C. A., & Yeh, C. C. (2006). Assessing the dimensionality and factor structure of multiple-
choice exams. Educational and Psychological Measurement, 66, 193-214.

Swaminathan, J., & Gifford, J. A. (1983). Estimation of parameters in the three-parameter latent
trait model. In D. J. Weiss (Ed.), New horizon in testing (pp. 13-30). New York: Academic.

Tate, R. (2003). A comparison of selected empirical methods for assessing the structure of
responses to test items. Applied Psychological Measurement, 27, 159-203.

Thissen, D. (1991). MULTILOG user's guide: multiple, categorical item analysis and testscoring
using item response theory [Computer program]. Chicago, IL: Scientific Software
International.

Thissen, D., & Wainer, H. (1982). Some standard errors in item response theory. Psychometrika,
47, 392-412.

Thompson, T. D., Nering, M., & Davey, T. (1997, June). Multidimensional IRTscale linking
1 ithi,,t common items or common examinees. Paper presented at the annual meeting of the
Psychometric Society, Gatlinburg, TN.

Thurstone, L. L. (1947). Multiple factor analysis. Chicago: University of Chicago Press.









Vandenberg, R. J.,& Lance, C. E. (2000). A review and synthesis of the measurement invariance
literature: Suggestions, practice, and recommendations for organizational research.
Organizational Research Methods, 3, 4-70.

Wang, M. (1985). Fitting a unidimensional model to multidimensional item response data: The
effects of latent space misspecification on the application of RT. Unpublished
manuscript.

Wilson, D., Wood, R., & Gibbons, R. D. (1987). TESTFACT. Test scoring, item statistics, and
item factor analysis. Mooresville, IN: Scientific Software.

Wingersky, M. S., Cook, L. L., & Eignor, D. R. (1987). Specifying the characteristics of linking
items usedfor item response theory item calibration (Research Report 87-24). Princeton,
NJ: Educational Testing Service.

Wingersky, M. S., & Lord, F. M. (1984). An investigation of methods for reducing sampling
error in certain IRT procedures. Applied Psychological Measurement, 8, 347-364.

Yen, W. M., & Fitzpatrick, A. R. (2006). Item response theory. In R. L. Brennan (Eds.),
Educational Measurement (4 ed.). West Port, CT: Praeger.

Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). BILOG-MG: Multiple-
group IRT analysis and test maintenance for binary items [Computer program]. Chicago,
IL: Scientific Software International.









BIOGRAPHICAL SKETCH

Youhua Wei was born in China. He received his B.Ed. in school education from Nanjing

Normal University in 1992 and his M.Ed. in psychology from East China Normal University in

1995. From 1995 to 1997, he worked as a psychological counselor at Southeast University in

Nanjing. From 1997 to 2001, he worked as a research associate at Shanghai Academy of

Educational Sciences. In 2004, he earned his M.S. in research, measurement, and statistics from

Texas A&M University in College Station. He began his doctoral study in research and

evaluation methodology in the Department of Educational Psychology at the University of

Florida in fall 2004. He was awarded the Ph.D. degree in August 2008.





PAGE 1

1 A SIMULATION STUDY ON THE PERFORMANCE OF FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS By YOUHUA WEI A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2008

PAGE 2

2 2008 Youhua Wei

PAGE 3

3 ACKNOWLEDGMENTS I would lik e to express my sincere appreciati on to Dr. James J. Algina, my supervisory committee chair, for providing valuable guidance and support. I would also like to thank other committee members, Dr. M. David Miller, Dr. Walter L. Leite, and Dr. Zhihui Fang, for their time and effort on this project. I thank my parents and my brothers and sisters for their continuous and unconditional support and encouragement. Finally, I thank my wife, Yan Zhang, for her love and support.

PAGE 4

4 TABLE OF CONTENTS page ACKNOWLEDGMENTS...............................................................................................................3 LIST OF TABLES................................................................................................................. ..........6 LIST OF FIGURES.........................................................................................................................7 ABSTRACT.....................................................................................................................................9 CHAP TER 1 INTRODUCTION..................................................................................................................11 Unidimensional IRT Models.................................................................................................. 13 Logistic Model.................................................................................................................13 Normal Ogive Model.......................................................................................................14 Unidimensional IRT Scale Linking........................................................................................ 14 Scale Transformation....................................................................................................... 14 Scale Linking...................................................................................................................16 Multidimensional IRT Models............................................................................................... 20 Logistic Model.................................................................................................................20 Normal Ogive Model.......................................................................................................23 Multidimensional IRT Scale Link ing..................................................................................... 25 Hirschs Method.............................................................................................................. 25 Lis Method.....................................................................................................................30 Mins Method..................................................................................................................33 Oshima and Colleagues Method....................................................................................35 Purpose of the Study........................................................................................................... ....40 2 METHODOLOGY................................................................................................................. 42 Design.....................................................................................................................................42 Independent Variables or Experimental Conditions........................................................42 Dependent Variables or Evaluation Criteria .................................................................... 47 Procedure................................................................................................................................49 Data Generation...............................................................................................................49 Parameter Es tim ation....................................................................................................... 51 Result Analysis................................................................................................................ 52 3 RESULTS...............................................................................................................................59 General Performance of the Different Linking Methods........................................................ 61 Performance of Linking Methods fo r Different Test Structures ............................................ 62 Performance of Linking Methods for Different Test Lengths ................................................ 63 Performance of Linking Methods for Different Sam ple Sizes...............................................64

PAGE 5

5 Performance of Linking Methods for Groups with Different Ability Distributions .............. 65 Performance of Linking Methods for Test Item s with Different Parameter Values.............. 67 4 DISCUSSION.......................................................................................................................122 Results from Previous Studies.............................................................................................. 122 Effects of Different Test Structures...................................................................................... 124 Effects of Different Test Lengths.........................................................................................126 Effects of Different Sample Sizes.........................................................................................127 Effects of Different Ability Distributions.............................................................................128 Effects of Different Item Parameter Values......................................................................... 130 Performance of Different Linking Methods.........................................................................131 5 CONCLUSIONS.................................................................................................................. 133 Conclusions...........................................................................................................................133 Future Research....................................................................................................................134 APPENDIX: ACCURACY AND STABILITY FOR DIFFERENT LINKING METHODS .....138 LIST OF REFERENCES.............................................................................................................186 BIOGRAPHICAL SKETCH.......................................................................................................193

PAGE 6

6 LIST OF TABLES Table page 2-1 Ability distributions for examinee groups......................................................................... 54 2-2 Item parameters for 20 items w ith approxim ate simple structure...................................... 55 2-3 Item param eters for 40 items with ap proximate simple structure...................................... 56 2-4 Item parameters for 20 items with complex structure....................................................... 57 2-5 Item parameters for 40 items with complex structure....................................................... 58

PAGE 7

7 LIST OF FIGURES Figure page 3-1 Accuracy and stability for different linking m ethods........................................................69 3-2 Accuracy and stability by li nking m ethod and test structure............................................. 72 3-3 Accuracy and stability by linking m ethod and test structure: N = 2000............................75 3-4 Accuracy and stability by linking method and test length for approxim ate simple structure tests................................................................................................................ .....78 3-5 Accuracy and stability by linking method and test length for com p lex structure tests: N = 500..............................................................................................................................81 3-6 Accuracy and stability by linking method and test length for com plex structure tests N = 1000:...........................................................................................................................84 3-7 Accuracy and stability by linking method and test length for com plex structure tests when G2 was excluded: N=1000....................................................................................... 87 3-8 Accuracy and stability by linking method and test length for com p lex structure tests: N = 2000............................................................................................................................90 3-9 Accuracy and stability by linking m ethod and sample size............................................... 93 3-10 Accuracy and stability by linking method and sample size for approxim ate simple structure tests................................................................................................................ .....96 3-11 Accuracy and stability by linking method and sample size for com plex structure tests... 99 3-12 Accuracy and stability by linking me thod and group for approxim ate simple structure tests................................................................................................................ ...102 3-13 Accuracy and stability by linking method and group for com plex structure tests: N = 500....................................................................................................................................105 3-14 Accuracy and stability by linking method and group for com plex structure tests: N = 1000..................................................................................................................................108 3-15 Accuracy and stability for differe nt linking m ethods: COM, n=40, N=1000, G2...........111 3-16 Accuracy and stability for differe nt linking m ethods: COM, n=20, N=1000, G2...........112 3-17 Accuracy and stability by linking method and group for com plex structure tests: N = 2000..................................................................................................................................113 3-18 Linking accuracy and stability and item para meter values: COM, n=20, N=1000, G3.. 116

PAGE 8

8 3-19 Linking accuracy and stability and item para meter values: APP, n=40, N=2000, G4.... 119

PAGE 9

9 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy A SIMULATION STUDY ON THE PERFORMANCE OF FOUR MULTIDIMENSIONAL IRT SCALE LINKING METHODS By Youhua Wei August 2008 Chair: James J. Algina Major: Research and Evaluation Methodology Scale linking is the process of developing th e connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. It is the prerequisite for many applications of IRT, such as test equa ting and differential item functioning analysis. Unidimensional scale linking methods have been studied and applied frequently over the past two decades. The development of multidimensional linking methods is at the infancy stage and more research is needed to obtain definitive results. As an extension of previous research, the purpos e of this study was to use simulated data to evaluate the performance of f our multidimensional IRT scale linking methods, the direct method, equated function method, test ch aracteristic function method, and item characteristic function method, under various testing cond itions, which include different test structures, test lengths, sample sizes, and ability distri butions. There were one hundred and ninety-two experimental conditions in this study and five hundred replicat ions were conducted for each of the conditions. The linking performance evaluation was based on the differences between the item parameter estimates for base group and the transformed item parameter estimates for the equated group

PAGE 10

10 across the test items. The mean and standard deviation of the diffe rences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Our results indicate that for approximate simp le test structure, each of the four linking methods worked approximately equally well under all testing conditions. The results also suggest that for complex test structure: (a) The equated function method did not work well under any testing conditions, (b) th e performance of other three linki ng methods depended on other testing conditions including sample size, test length, and abil ity distribution difference between groups, and (c) the direct method was the best linking pr ocedure for most testing conditions. In addition, the study shows that the item para meter values influenced the li nking performance. Under most of the testing conditions, the li nking results for the discriminati on parameter tended to be less accurate and less stable when the item parame ter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depended on the item pa rameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values.

PAGE 11

11 CHAPTER 1 INTRODUCTION Suppose a set of test items is administered to non-equivalent groups of examinees and item response theory (IRT) is used to estimate the item parameters for each of the groups. The parameter estimates will be on different scales because the metric defined by each separate calibration is different (Stocking & Lord, 1983). Specifically, IRT parameter estimation procedures often scale the ability for each group with mean of 0 and st andard deviation of 1, although the actual ability distributions of the two groups may be different (Kolen & Brennan, 2004). Therefore, to compare the parameter esti mates from different IRT calibrations, they should be transformed on the same scale. Sc ale linking is the pro cess of developing the connection between scales of two or more sets of parameter estimates obtained from separate test calibrations. The objective is to establish a comm on metric for all sets of parameter estimates. Scale linking is an important issue in psychometrics, and many applications of IRT require that item parameter estimates from i ndependent calibrations be expressed on the common metric, including test equating and differentia l item functioning (DIF) (Stocking & Lord, 1983). Based on Kolen and Brennan (2004), equating is a statistic al process that is used to adjust scores on test forms so that scores on the forms can be used interchangeably. (p. 2), and linking refers to relating scores on tests which are not built to the same content or statistical specifications. Different terminologi es have been used to descri be the process of establishing relationship between scores on two or more te sts (for a complete re view, see Kolen, 2004a, 2004b). Scale linking is used in this study to refer to the process of linking different scales rather than the process of linking test scores. However, scale linking is the prer equisite for establishing the connection between different test scores. Therefore, scale linking is an important step in test

PAGE 12

12 equating (Cook & Eignor, 1991; Kolen & Brennan, 2004) and satisfactory equating results require successful scale linking. If different groups of examinees have diffe rent probabilities of success on an item after they have been matched on the ability of intere st, the item has differential functioning. In IRT, DIF is defined as the differences in the model parameters for the comparison groups (Clauser & Mazor, 1998). The item parameters for different groups should be compared only after they are placed on a common metric. Therefore, DIF iden tification depends heavily on the quality of scale linking. Some procedures have been developed to det ect DIF by improving scale linking (Candell & Drasgow, 1988; Lautenschlager & Pa rk, 1988; Lautenschlager Flaherty, & Park, 1994; Park & Lautenschlager, 1990). In addition to psychometrics, scale linking is also very important to educational and psychological studies. Multi-group confirmatory factor analysis or mean and covariance structure analysis has been increas ingly used to compare constructs across different groups (for a comprehensive review, see Vandenberg, 2000) and so me unresolved issues are closely related to the difficulty of linking scales across groups (Millsap, 2005). Therefore, successful scale linking has the potential to produce sa tisfactory comparison studies on ps ychological constructs across different groups. In sum, scale linking is very important for educational measurement to be fair and objective for different groups of examinees. Unid imensional scale linking methods have been studied and applied frequently over the past tw o decades (for more information, see Kolen & Brennan, 2004; Yen & Fitzpatrick, 2006). Th e development of multidimensional linking methods (Davey, Oshima, & Lee, 1996; Hirs ch, 1988, 1989; Li, 1997; Li & Lissitz, 2000; Min, 2003; Oshima, Davey, & Lee, 2000) is just at the infancy stage and more research is needed to

PAGE 13

13 obtain definitive results (Yen & Fitzpatrick, 2006). In this chapter, unidimensional and multidimensional models and linking methods are reviewed and the purpose of the current study is presented. Unidimensional IRT Models Logistic Model The three-parameter logistic (3PL) model (see Hambleton & Swaminathan, 1985; Lord, 1980) assumes that the probability of a correct answer to a dichotomously scored item j by an examinee with ability i is 1 1 1 1 1 ,,;1][jij jij jijba j j ba ba j j jjjiijije cc e e cc cbaxP (1-1) where ijx is the item response (0 or 1) for person i on test item j jais the item discrimination parameter, jbis the item difficulty parameter, and jcis the guessing parameter or the pseudochance score level, representing the probability of correct res ponse when the ability assessed by the item is very low. Sometimes the 3PL model is expressed as ] [1 1 1 ,,;1jijbDa j jjjjiijije cccbaxP (1-2) with D=1.701, so that a normal ogive model item ch aracteristic curve (ICC) and a logistic model ICC with the same item parameters are almost identical. If jc is 0, the 3PL model becomes twoparameter logistic (2PL) model:

PAGE 14

14 ][1 1 ,;1jijba jjiijije baxP (1-3) For 2PL model, if ja is 1, it becomes one-parameter logistic (1PL) model or Rasch model: jib jiijije bxP 1 1 ;1. (1-4) Normal Ogive Model There are also three normal ogive models or cumulative normal distribution models in IRT: one parameter model: jib t jiijdte bP 22 12 1 ; ; (1-5) two parameter model: jjiijijbaxP ,;1 = jijba tdte22 12 1 ; (1-6) and three parameter model: jjjiijijcbaxP ,,;1 = jijba t j jdte cc22 12 1 1 (1-7) Many IRT models have been developed for te st items that are polyt omously scored using ordered categories, including graded response model (Samejima,1969), partial credit model (Masters, 1982), generalized partial credit mo del (Muraki, 1992), rating scale model (Andrich, 1978), and nominal response model (Bock, 1972). Unidimensional IRT Scale Linking Scale Transformation The IRT parameter estimates produced from in dependent calibrations using data obtained from different groups of examinees are often on di fferent metrics. Lord (1980) demonstrated that

PAGE 15

15 the relationship between the metrics of any two independent item calibrations is linear. Therefore, a linear equation can be used to transform the IRT parameters on scale E (representing the linked scale or equated scale) to scale F (representing the base scale). For person i and item j BAi iE F*, (1-8) A a aj jE F*, (1-9) BAbbj jE F*, (1-10) j jEFcc *, (1-11) where *iF, *jFa, *jFb, and *jFc represent the transformed values from the linked scale to the base scale. A is the slope and B is the intercept. The constants A and B can be expressed as *j jF Ea a A (1-12) j j i iE FE FAbbAB *. (1-13) A and B can also be expressed for any two individuals i and *ior two items j and *j: * j j j j i i i iEE FF EE FFbb bb A (1-14) i i j jE FE FA AbbB (1-15) or expressed for groups of items or examinees (see Kolen & Brennan, 2004): F E E F E Fa a b b A (1-16) E F E FA bAbB (1-17)

PAGE 16

16 The iEijP value for the original parameters on scale E will be the same as the iF ijP* value for the transformed parameters on scale F as demonstrated by 1 1 1 1 1 1 1 1 1**** *i j E i E j E j j j E i E j E j j j F i F j F j j iEij b Da E E BAbBA A a D E E b Da F F F ijP e cc e cc e cc P Therefore, the logistic functi on is invariant under a linear transformation of item and ability parameters. Most of the unidimensional IRT scal e linking methods are based on this important feature. Scale Linking In practice, both test item parameters and examinees ability parameters need to be estimated and the ability estimates are often scaled to have means of 0 and standard deviations of 1. Parameter estimates obtained from different groups of examinees are often on different scales due to nonequivalence of the groups even though a ll ability estimates are s caled with means of 0 and standard deviations of 1. Therefore, some da ta collection procedures are required to establish the connection between different scales by using the linear transformations mentioned above. In test equating, three data collection designs ar e often used, including random groups design, single group design, and common-item nonequi valent groups design. The IRT parameter estimates for the first and second designs are assu med to be on the same scale because of the randomly equivalent groups of examinees and single group of examinees (Kolen & Brennan, 2004) if random sampling errors are ignored. For the third design, the parameter estimates are

PAGE 17

17 assumed to be on different scales due to the nonequivalent groups. The third design is the most often used equating design (Kolen & Brennan, 2004) an d it is very similar to the design used for exploring DIF. Two approaches have been used to establish a comm on scale for parameter estimates for this design. One is to estimate parameters for all items on both test forms together. This method is often called concurrent calibration (Wingersky & Lord, 1984). Both BILOG-MG (Zimowski, Muraki, Mislevy, & Bock, 1996) an d MULTILOG (Thissen, 199 1) have the function of simultaneously obtaining parameter estimates for two test forms and two groups on the same scale. The second approach is to link the two scales by using the parameter estimates for the common items. This study will focus on the second approach. The following IRT linking methods have been developed to establish a common metric for parameter estimates. Mean/sigma method. This method (Marco, 1977) uses the means and standard deviations of the b parameter estimates for the common items to calculate the constants A and B in the linear transformation equation: E Fb b A E FbAbB (1-18) Mean/mean method. This method (Loyd & Hoover, 1980) uses the means of a parameter estimates for the common items to calculate A and the means of b parameter estimates for the common items to calculate B in the transformation equation: F Ea a A E FbAbB (1-19) Item response function method. In this procedure (Haebara, 1980), the constants A and B are estimated by minimizing the sum of the s quared difference between the item characteristic curves for the common items over examinees:

PAGE 18

18 ij E E E FijFFFFij diffj j j i jjjicBbA A a PcbaP H2 ; ; (1-20) Test response function method. The constants A and B are estimated by minimizing the sum of the squared difference between the test characteristic curves for the common items for examinees (Stocking & Lord, 1983): ij E E E Fij FFFF j ij diffj j j i jjjicBbA A a PcbaP SL2 ; ; (1-21) Item response function method and test resp onse function method are often referred as the characteristic curve methods (Stocking & Lord 1983). Specifically, the former is called item characteristic method and the latter test characteristic curve method. Minimum 2method. This method (Divgi, 1985) combines information of each items parameter estimates and the variance-covariance matrix of sampling errors for each item from the item parameter estimation procedure. The c onstants A and B are estimated by minimizing the following quadratic function: 1 2 j E F E F FF E F E FBbAb A a a BbAb A a aj j j j j j j j j j, (1-22) where jF is the estimated variance-covariance matrix of the sampling errors for the item parameter estimates for item j on the F scale and *jFis the estimated variance-covariance matrix of the sampling errors for the item parameter estimates for item j which are transformed from the E scale to the F scale. Comparison studies have been conducte d for these methods with dichotomous IRT model. Based on a comprehensive literature review (Kolen & Brennan, 2004): (a) The

PAGE 19

19 characteristic curve methods produced more stable and accurate results than the mean/mean and mean/sigma methods, (b) the mean/mean method was more stable than the mean/sigma method, (c) the concurrent calibration method yielded more accurate results than the test characteristic curve method for a small number of common items and both procedures had the similar accuracy for a larger number of common items, and (d) the concurrent ca libration method might be less robust to violations of the IRT assumptio ns than characteristic curve methods. These methods have been extended to li nk scales with polytom ous IRT models. For example, Cohen and Kim (1998) extended mean/mean and mean/sigma methods to the graded response model and Kolen and Brennan (2004) suggested using mean/m ean and mean/sigma methods for the generalized partial credit mode l. Baker (1992) generalized the test response function method to the graded response model a nd Baker (1993) used the item response function for the nominal response model. Kim and Cohen (1995) tried the minimum 2method for the graded response model. There are also some comparison studies for these methods with polytomous IRT models. A simulation study (Cohen & Kim, 1998) comparing the mean/mean method, mean/sigma method, weighted mean/sigma method, te st response function method, and minimum 2 method for the graded response model found that all methods produced similar results. Another simulation study (Kim & Cohen, 2002) compari ng the test response function method and the concurrent calibration method for the graded response model found that the concurrent calibration was relatively more accurate.

PAGE 20

20Multidimensional IRT Models Logistic Model Unidimensional IRT models appear to be adequate for scaling achievement test items in most practical situations (Yen & Fitzpatrick, 2006). However, it is reasonable to believe that the performance of examinees on some test items depends on more than one trait or ability and some consequences of applying unidimensional models to multidimensional data have been identified (see Yen & Fitzpatrick, 2006). Th e number of dimensions necessa ry to model the test item responses depends not only on the number of ability dimensions and the level on those dimensions exhibited by the examinees but also on the number of skills to which the test items are sensitive (Reckase, 1997a). Therefore, mu ltidimensionality can occur in different ways depending on the interaction betw een a specific group of examinees and certain set of test items. There are two types of multidimensional IRT (MIRT) models for dichotomously scored item response data: the compensatory mode l and the noncompensatory model. In the compensatory model, a low value on one dimension can be compensated for by a high value on another dimension (McKinley & Reckase, 1983; Reckase, 1997a). In the noncompensatory model, an increase in the value on one dimension cannot compensate for a lower value on another dimension (Simpson, 1978). Since estimation programs and linking methods have not been well developed for non compensatory model, the most often used compensatory model is discussed and used in this study. The compensatory multidimensional three-pa rameter logistic (M3PL) model is a direct generalization of the unidimensi onal 3PL model (Reckase, 1997a):

PAGE 21

21 1 1 1 1 1 ),1(' 'jij jij jijd j j d d j j jjjiije cc e e cc c,d xP a a aa; (1-23) where ),,1(jjjiijcd xPa; is the probability of a correct response (1 ijx) for person i on test item j ijx is the item response (0 or 1) for person i on test item j ja is the vector of item discrimination parameters, jd is the scalar parameter related to the difficulty of the item, jc is the lower asymptote or guessing parameter, and i is the vector of ability parameters for person i. This model can be expressed in the following scalar form: 1 1 1 1 1 ),,;1(1 1 1 j d m k ikjk a j d m k ikjk a j m k ikjke cc e e cc cdaxPj j da j j jjjiij (1-24) where m is the number of dimensions. When jc is 0, the compensatory M3PL model becomes the compensatory multidimensional two-parame ter logistic (M2PL) model (McKinley & Reckase, 1983): jij jij jijd d d jjiije e e d xP a a aa; '1 1 1 ),1 ( (1-25)

PAGE 22

22 This model can also be expressed as the following scalar form: j d m k ikjk a j d m k ikjk a j m k ikjke e e daxPda jjiij1 1 11 1 1 ),;1( (1-26) Compared with unidimensional IRT models, multidimensional discrimination and ability parameters are described in the form of vectors instead of scalars. If the i dimensions are orthogonal, the observed correlations among th e item scores will be accounted for by theja parameters. Otherwise, the item co rrelations will reflect both the japarameters and correlated dimensions. In MIRT, the probability of a correct response to an item depends on multidimensional ability and is defined as an item characteristic surface (ICS). Assuming orthogonal axes of dimensions in the surface, an item j can be described by the following three characteristics (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991): multidimensional discrimination (MDISCj): m k jk ja MDISC1 2, (1-27) which is the discrimination power of the item for the most discriminating combination of dimensions; multidimensional difficulty (MDIFFj): j j jMDISC d MDIFF (1-28) which, similar to the difficulty parameter in unidimensional model, is the distance from the origin of the space to the point of steepest slope in a direction from the origin; and direction (jk ) of the greatest slope from the origin: j jk jkMDISC a arccos (1-29)

PAGE 23

23 which is the angle that the line from the origin of the space to the point of steepest slope makes with the k th axis for the item. Normal Ogive Model By adapting Thurstones multiple factor model (1947) to dichotomous item response data, Bock and Aitkin (1981) proposed a multidimensional normal ogive model by firstly assuming that an unobserved continuous response variable, ijy for person i and item j is a linear combination of m latent variables, weighted by the factor loadings, : ijmijm ijijijy 2211, (1-30) where ~, 0IN, 10y,~N, and 2,~jN 0 (Note that the s in Equation 1-30 are not the s in Equation 1-29). It is assumed that there is an underlying process which generates a correct observed response,1ijx when ijy equals or exceeds a threshold, j and produces an incorrect observed response, 0 ijx otherwise. Then the probability of obtaining a correct item score is 2 1 exp 2 1 ,;12 ij j m k kijk j ij j m k kijk ij j jjiijijdy y xPj (1-31) where m k jk j 1 2 21 This is a compensatory model because greater ability on one dimension can make up for lesser ability on other dimens ions. This model can be reparameterized to

PAGE 24

24 produce similar parameters in multidimensional logistic model (Bock, Gibbons, & Muraki 1988; Muraki & Engelhard, 1985) by 2 exp 2 1 ,;12 ij z Z jjiijiji idtt dt t d xP a (1-32) where jjj m k ikjk iddaZ a' 1, (1-33) j jk jka (1-34) j j jd (1-35) It can also be shown that i jk jkq a, (1-36) j j jq d (1-37) with m k jk ja q1 2 21. (1-38) When s are correlated with covariance matrix it can be shown that ` 1jk jka (1-39)

PAGE 25

25 ` 1j jd (1-40) aa`1 jk jka, (1-41) aa`1 j jd, (1-42) where is vector of factor loading for item j and a is the vector of discrimination for item j Multidimensional IRT models for polytomously scored test items have also been developed, including multidimensional logistic models and multidimensional normal ogive models (Kelderman, 1997; Muraki 1999; Muraki & Carlson, 1995). Multidimensional IRT Scale Linking The multidimensional scale linking is more complicated than the unidimensional scale linking because it involves the tran sformations of scale locations, variances, and covariances of several ability dimensions obtained from differen t calibrations and more technical problems need to be resolved. Just as MIRT can be considered ei ther as a special case of factor analysis or an extension of unidimensional IRT, the multidimensional scale linking can be realized either by borrowing methods from factor an alysis (Hirsch, 1988; Li, 1997; Min, 2003) or by extending the unidimensional IRT linking methods to the multi dimensional situations (Oshima et al., 2000). Hirschs Method Hirsch (1988) is possibly the first author to explore the feasibility and effectiveness of multidimensional linking and equating by using the common-examinee design. Hirsch presented three technical issues in multidimensional linking and provided three possible resolutions. The first issue is to establish scale transformati ons to keep the M2PL function invariant. The following transformation equations can be used for a two-dimension (dimension 1 and 2) M2PL model:

PAGE 26

26 ,1 11 1 i i 2 22 2 i i, (1-43) ,11 1 j jaa 22 2 j jaa, (1-44) 2211 j jjjaadd (1-45) where parameters with superscript are transformed parameters on a new scale. The M2PL function is invariant by this transformation: .,;1 1 1 1 1 1 1 ),1(2211 2211 222111 2211 2 22 22 1 11 11) ()()( ) ( *** jjiij daa aad a a aad a a jjiijd xP e e e d xPjijij j jj ij ij j jj i j i ja a; This scale transformation method can be exte nded to M2PL models with more than two dimensions. Hirschs multidimensional scale li nking method was based on the invariance of multidimensional function under the above transf ormations of item and ability parameters. The second technical issue is that the corre lation between dimensions obtained from the first calibration may be somewhat different fr om the correlation estim ated from the second calibration due to some non-parallel items for common-examinee design. If this occurs, the parameter estimates from two calibrations are co mposites or linear combinations of different basis vectors. Therefore, it is necessary to tr ansform the basis vectors from one calibration to those of the second calibration. This can be realized by transf orming the two sets of ability parameter estimates of the common examinees from two calibrations so that they are as similar as possible.

PAGE 27

27 The third technical issue is the joint rota tional indeterminacy of the item discrimination and ability parameters. That is, the dimensions can be rotated and produce many possible sets of i and ja parameter estimates without affecting th e M2PL item characteristic function. As suggested by Wang (1985), the procrustean rotation in factor analysis (Schonemann ,1966) can be used to transform the parameter estimates from one calibration to those from the other calibration. Hirschs linking method for the common-examin ee design includes four steps. In the first step, two sets of item and ability parameters ) (jiFFa and jiEEa for the common examinees but on different metrics are estimated from two independent calibrations. In (,), aijFF iF is a Nm matrix, where N is the number of examines, and ajF is a nm matrix, where n is the number of items. In the second step, three tr ansformations are used to obtain common basis vectors for the two sets of paramete r estimates. The first transformation by T1 refers the discrimination parameter estimates from the first calibration (jFa) to a set of orthogonal basis vectors instead of the basis vector s defined by the ability estimates (iF ). The second transformation by T2 -1 refers the discrimination parameter estimates from the second calibration (jEa) to a set of orthogonal basis ve ctors instead of the basis ve ctors defined by the ability estimates (iE ). The third transformation by T3= T1* T2 -1 refers the discrimination parameter estimates from the first calibration (jFa) to a set of common basis vectors for both calibrations. In the third step, orthogonal proc rustean transformation is used to rotate the ability estimates from the first calibration ( iF) to those from the second calibration (iE ). This fourth transformation matrix T4 can be found by minimizing the sum of squared difference between

PAGE 28

28 each element of the two sets of ability parameter estimates (iF ) and (iE ). The method was called orthogonal procrustean transformation developed by Sc honemann (1966). Specifically, suppose S = iiEF ', 'PDPSS and 'QDQSS, then 4PQ T Given the above four transformations, the means and standard deviatio ns of the ability parameters for the common examinees from the two calibrations are estimated in the fourth step. For the common-examinee design, the linking parameters can be estimated by equating the means and standard deviations of the ability estimates from the first calibration (iF ) and those transformed from the second calibrations (*iF). The linking parameter estimates are th en used to transform the parameter estimates which have already been transformed by the procedure described in the second step. For example, suppose one uses the common examinee design a nd the M2PL model with two dimensions, the following relations exist: 1 1 1 1 1 1E EE F FFi i 2 2 2 2 2 2E EE F FFi i (1-46) So 1 1 1 1 1 1 1 1F E F F E E E Fi i 2 2 2 2 2 2 2 2F E F F E E E Fi i (1-47)

PAGE 29

29 Let 1 1 1 11F F E EM 2 2 2 22F F E EM (1-48) 1 11 F ES 2 22 F ES (1-49) Then the transformed parameters from E scale to F scale are 1 1 *1 1S Mi iE F 2 2 *2 2S Mi iE F (1-50) 1 *1 1 j jE FaSa, 2 *2 2 j jE FaSa, (1-51) 2 1 ***2 1MaMaddj j j jE EEF, (1-52) where the parameters with as superscript and F as subscr ipt on the left side of equations are the final transformed parameters on F scale, and the parameters with as superscript and E as subscript on the right side of equations are the transformed parameters on E scale by the first three transformations. The function of this four-step scale linki ng procedure for M2PL model was evaluated by test equating results performed on both simulate d and real data sets us ing the common-examinee

PAGE 30

30 design (Hirsch, 1989). The equati ng results were examined by comparing the mean differences and the mean absolute differences of the true sc ores and ability estimates between the base tests and equated tests. Satisfactory equating was found for true scores but not for ability estimates. Hirschs linking method was originally developed for the common-examinee design. However, it can easily be modified to c onduct scale linking for common-item nonequivalent groups design which is most usually used in test equating and DIF st udy. As Hirsch (1988) suggested, the basis vector tran sformation would be the same. The procrustean transformation would use the common item discrimination parameters instead of the ability parameters. The item difficulty parameter for each item would need to be regressed onto each of the ability dimension parameters and therefore produce one unique difficulty parameter for each of the dimensions (Reckse, 1985). Then the mean a nd sigma method would be used for the common item difficulty parameters for the final transforma tion. However, more study is needed to verify the adequacy of this modified linking procedure. Lis Method Compared with Hirschs procedure, Lis (1997) multidimensional linking methods are more straightforward and consis tent with MIRT computer es timation programs. Most MIRT programs solve the identification problem by requi ring multidimensional abilities be distributed as multivariate normal MVN (0, 1). Therefore, the metric of the item parameter estimates is typically referred to orthogonal reference axes with unit lengt h. Given this condition, one reference system can be transformed onto the other reference system by a composite transformation: an orthogonal pr ocrustean transformation for re-rotating the reference system, a translation transformation for shifting the point of origin, and a single di lation for re-scaling unit length. Specifically the following equations are used in the reference system transformation:

PAGE 31

31j jE FkaTa' *, (1-53) mTa' *j j jEEFdd, (1-54) m T i iE Fk1 */1. (1-55) It can be shown that the M2PL function is invariant to these transformations: ).,;1( 1 1 1 1 1 1 1 1 ),;1(' ' 1 1 '/1 /1 ***jji j E i E j E j E j E j E i E j E j E j E i E j E j E j E i E j E jjiEEEij d d d k k d k k FFFijd xP e e e e d xP a a a Tma Tma a mTa m TTa mTa m T aT The question is how to find T, m, and k. Li (1997) proposed seve ral methods to estimate the scale linking parameters. The rotation matrix T can be estimated by orthogonal procrustean transformation procedure as mentioned in Hirschs method above. Let S = jjEFaa', 'PDPSS and 'QDQSS then 'PQ T The origin shift coefficient m and unit change coefficient k can be estimated simultaneously by minimizing the sum of squa red difference between test characteristic functions for the common items obtained from the two calibrations, which was originally developed by Stocking and Lord (19 83) for the unidimensional linking: N i n j n j FFF FFFjj j jj jdPdP N kf1 2 11 *** ; ; 1 a a m, (1-56) where N is the number of grid points of values.

PAGE 32

32 The origin shift and unit change coefficients can also be estimated separately by different procedures. For example, the origin shift coeffi cient can be estimated by minimizing the sum of squared difference between the two difficulty parameter estimates obtained from two calibrations: n j FFj jddf1 2 m, (1-57) where n is the number of common items. This was called least squares procedure (Li, 1997). The unit change coefficient can be estimated as th e ratio of the square root of the maximum eigenvalues of the matrices jjFFaa' and jjEEaa'obtained from the two calibrations: jj jjEE FFsig Maximum sig Maximum kaa aa ', (1-58) where sig represents the singular va lue or the nonnegative square r oots of the eigenvalue. This was called ratio of eigenvalues procedure (Li, 19 97). Similar to the leas t squares procedure for the estimation of origin shift coefficient, the unit change coefficient can also be estimated by minimizing the sum of squared difference betwee n the two sets of discrimination parameters estimated from two calibrations. This is also referred as least squares procedure (Li, 1997): n j FFj jkf1 2 aa. (1-59) The rotation matrix T and unit change coefficient k can also be estimated simultaneously by a least squares method developed for fitting one matrix to another through a rotation matrix, a translation vector, and a central dilation vector (Schonemann & Ca rroll, 1970). In this case, the rotation matrix and dilation scalar were estimate d by minimizing the sum of squared errors of the following residual matrix:

PAGE 33

33 'j jE FkaTaE (1-60) It can be shown that '' 'Taa aajj jjCFCE CFCFtrace k trace, (1-61) where j j j j j jEE CEFF CFaaaaaa (1-62) with jFaas the mean of jFa and jEaas the mean of jEa This was called ratio of trace procedure (Li, 1997). The translation vector was not estimat ed by this method because item discriminations can not provide information about origin shift. Comparing the effect of different combinati ons of reference, tran slation, and dilation transformation procedures on the multidimensiona l linking parameters estimation, Li (1997) found that the most appropriate MIRT linking met hod is the combination of procrustean rotation approach (for dimensional transf ormation), the ratio of trace pr ocedure (for dilation), and the least square procedure (for tr anslation). This linking method could produce accurate estimation of item parameters, approximately equivalent estimation of ability parameters, but unsatisfactory true score estimation. Mins Method Min (2003) challenged Li s (1997) two reasons for using a single dilation parameter, that is, mathematical tractability and the assumpti on of constant variance across dimensions, and argued that one single dilation is insufficient fo r describing the scale unit changes for multiple dimensions. Two independent calibrations may change the scales of the multidimensional dimensions to different degrees. To address this problem, Min (2003) modified Lis (1997) method by replacing the single d ilation parameter with a diagona l dilation matrix to model

PAGE 34

34 different unit changes on different dimensions The reference system transformations are performed as follows: j jE FaTKa''*, (1-63) mTa' *j j jEEFdd, (1-64) m TK i iE F 11 *, (1-65) where K is a diagonal dilation matrix. It can be shown that the M2PL function is invariant to these transformations: ).,;1( 1 1 1 1 1 1 1 1 ),;1(' ' 11 11 ''***jji j E i E j E j E j E j E i E j E j E j E i E j E j E j E i E j E jjiEEEij d d d d FFFijd xP e e e e d xPa a a Tma TKma a mTa m TKTKa mTa m TKaTK For two-dimensional model, K becomes 2 10 0k kwhere 1k is the dilation parameter for the first dimension, and 2k for the second dimension. The least square method (Schonemann & Carroll, 1970) of estimating a rotation matrix, a translation vector, and a central dilation vector for fitting two matrixes can be followed to find T, K, and m in the transformation equations (Min, 2003). Mathematically Lis (1997) method and Min s (2003) method produce the same solution for T and m and the only difference of linking results co mes from the different dilation parameters.

PAGE 35

35 Reckase and Martineau (2004) identified an important weakness in Lis (1997) and Mins (2003) method for MIRT models with high dime nsionality and provided a solution to the problem by employing a non-orthogonal procrustean transformation. However, this approach needs to be examined by further empirical studies. Oshima and Colleagues Method All multidimensional linking methods mentioned above borrowed an important procedure, procrustean rotation, from factor analysis to transf orm the dimensional axes. Oshima et al. (2000) extended four s cale linking methods within IRT from unidimensional to multidimensional models. According to their methods, the following equations were used to transform the IRT parameters on one scale E to another scale F (to distinguish IRT linking methods from the factor analysis methods de scribed above, different indices for linking parameters are used). For person i and item j j jE FaAa' 1*, (1-66) Aa1'j j jEEFdd*, (1-67) A *i iE F, (1-68) where the rotation matrix mm Aadjusts the variances and covarian ces of the ability dimensions (scale), and the translation vector 1 m changes the means of the ability dimensions (location) on the two scales. The model indetermin acy can be shown as the following:

PAGE 36

36 ).,;1( 1 1 1 1 1 1 1 1 ),;1(' 1' 1' 1' 1' 1' 1***jji j E i E j E j E j E j E i E j E j E j E i E j E j E j E i E j E jjiEEEij d d d d FFFijd xP e e e e d xPa a a Aa Aa a Aa A Aa Aa A aA As in the unidimensional IRT, suppose that two nonequivalent groups of examinees take common test items and independent calibrations produce two sets of parameter estimates (jjFFd a) and (jjEEd a). These two sets of parameter estimates are on different scales F and E, and scale linking needs to be conducted to pla ce the two sets of parameter estimates on a common scale. Using the above equations, (jjEEd a) on E scale can be transformed to the F scale (** jjFFda ). The values of the two sets of item parameter estimates (jjFFd a) and (** jjFFda) should be similar due to the i nvariance of common item character istic in IRT. Unidimensional IRT linking methods can be extended to multidimensional IRT model to minimize some functions of the difference between the two sets of item parameters. Again, the question is how to find the values of A and so that the connection between th e two scales can be established. The direct method. This method was a multivariate extension of the minimum chisquare linking method for unidimensional IRT model (Divgi, 1985). The values of A and are estimated by minimizing the sum of squared diffe rence between the two sets of item parameter estimates over all items. However, the direct method is different fr om the original method in that

PAGE 37

37 it does not consider the variance-covariance matr ix of sampling errors for item parameter estimates in the function: n j FF n j m k FFj j jk jkdd aa mn f1 2* 11 2*] [] [ 1 1 A, (1-69) where n is the number of items, m is the number of ability dimensions, and (** jjkFFda ) are transformed parameter estimates from E scale to F scale. The equated function method. This method is the multidimensional extension of the mean and sigma methods for the unidimensi onal IRT model (Loyd & Hoover, 1980; Marco, 1977). A more general system of s cale linking equations is used to specify that some functions of the common item parameters from the first calibration (jjFFd a) are equal to the same functions of the transformed common item paramete rs from the second calibration (** jjFFda). The transformed item parameter estimates can be obt ained by using the above scale transformation equations with the linking parameters A and The values of A and are estimated by minimizing the sum of squared difference between th e same functions of the two sets of selected elements of the estimated (jjFFd, a) and (**,jjFFda). The number of functions needed ( p ) depends on the number of dimensions ( m ) or elements in A and with p = m2 + m For example, in the two dimensional case (m = 2), four parameters in A and two parameters in need to be estimated. Therefore, six functions are required to estimate the six linking parameters and they could be the means of 1ja 2ja and jd for the first and second halves of the common items (or other block of items).

PAGE 38

38 The scale linking functions are flexible in terms of which item parameter estimates to use and what function to use. Different systems of scale linking functions may produce different values of the linking parameters A and The quality or appropriateness of linking functions can be evaluated by their stability across random examinee samples, the character of the common item sets, and the true values of the linking parame ters (Davey, et al., 1996). For example, if the mean is the chosen linking function, the function to be minimized is p j FFp pp f1 2 1 A, (1-70) where pF FF 21 are the estimated means of p separate sets of elements of the estimated (jjFFd, a), and *** 21pFFFare the estimated means of p separate sets of elements of the estimated (**,jjFFda). The test characteristic function method. This method is an extension of the test response function method developed by Stocking and Lord (1983) for the unidimensional IRT model: 2 11 *** ; ; 1 n j n j FFE FFFjj j jj jdPdPW q f a a A (1-71) where q is the number of matching vectors, W is the weight taken at different values. The W is used to emphasize that some values are more important than others to estimate the linking parameters. The weight can also be considered equal along the ability scale.

PAGE 39

39 The item characteristic function method. This method is the multidimensional generalization of the item response function me thod for unidimensional IRT model (Haebara, 1980): n j FFEFFFjj j jj jdPdPW qn f1 2 *** ; ; 1 a a A. (1-72) Based on a simulation study comparing the four IRT linking methods under different ability distributions (Oshima et al., 2000), all of the four methods were acceptable under almost any of the minimization criteria and offered dramatic improvement over not linking at all. It was also found that the test charac teristic function method and item characteristic function method were more stable and recovered the true linki ng parameters better than the direct method and equated function method. The multidimensional linking methods develope d by Hirsch (1988), Li (1997), Oshima et al. (2000), and Min (2003) can a ll be directly or indirectly performed for the common-item nonequivalent groups design, which ha ve been a widely used in te st equating (Kolen & Brennan, 2004). Accordingly those methods have the potential for estab lishing calibrated item pool and exploring DIF. Another multidimensional linking method proposed by Thompson, Nering, and Davey (1997) can be used for test equating in a design without common items or examinees. With the assumption of the same origin, axes, and correlation between axes for the two randomly equivalent groups of examinees, this method so lve the rotational inde terminacy by identifying similar item content clusters on different tests and then rotating them in the same multidimensional-reference system. Further stud ies need to be conducted to evaluate the performance of this method. Multidimensional scale linking is a new research area. There have been very few studies conducted for each of the proposed methods (Hirsch, 1988; Li, 1997; Min, 2003; Oshima et al.,

PAGE 40

40 2000) and even fewer studies for comparing different methods in the litera ture. Therefore, it is currently difficult to evaluate the function of different me thods. The only comparison study by Min (2003) compared Lis me thod, Mins method, and Oshima and colleagues test characteristic function method in terms of accur acy and stability of scale transformations under different conditions varying in sample size, structure of dimensions, and ability distribution. The results indicate that both Oshima and colleagues and Mins methods were better in transforming discrimination parameters than Lis method, an d Mins and Lis methods performed better than Oshima and colleagues method in transforming the difficulty related parameters. In addition, Oshima and colleagues method performed better than Mins and Lis methods in transforming test true scores, and Lis and Mins methods were better than Os hima and colleagues method in maintaining the structure of dime nsions through orthogonal rotation. Purpose of the Study Based on the literature review of the multidim ensional linking methods, Lis methods have been evaluated under various circumstances such as different linking procedures, sample sizes, equating situations, number of anchor items, li nking situations, and ability distributions (Li, 1997). Mins method has also been examined with comparison with other methods under different conditions including di fferent sample sizes, dimensi onal structures, and ability distributions. The performance of Oshima and colleagues four IRT linking methods has been examined under fewer testing conditions, that is for different ability distributions, using simulation study with only 20 replications (Oshima et al ., 2000). A comparison study (Min, 2003) indicates that one of the f our IRT linking methods, that is, th e test characteristic function method, outperformed other methods in transforming item discrimination parameter estimates and equating true score estimates. This suggests that the IRT procedures are promising methods for multidimensional linking and equating. Further studies are needed to examine the

PAGE 41

41 performance of these four methods under more tes ting conditions. As an extension of previous research (Oshima et al., 2000), the purpose of this study was to evaluate the performance of the four multidimensional IRT scale linking methods, the direct method, equated function method, test characteristic function method, and item characteristic function method, under various testing conditions, which include different test structures, te st lengths, sample sizes, and examinees ability distributions.

PAGE 42

42 CHAPTER 2 METHODOLOGY A com prehensive review of the unidimensi onal scale linking and test equating (Cook & Petersen, 1987) provides us a framework for exploring the performance of multidimensional scale linking methods. According to Cook and Petersens discussi on, the results of linking and equating depend on linking or equating methods, sa mple characteristics, and properties of the common items. In addition, the multidimensional structure underlying the test item responses makes scale linking more complicated and shoul d be considered as one important testing condition. In this simulation study, the perfor mance of the four MIRT scale linking methods (Oshima et al., 2000) for the common-item nonequiv alent groups design was evaluated with the compensator compensatory M2PL model under different testing conditions, including different test structures, test lengths, sample sizes, and examinees ability distributions. The M2PL model had two dimensions. Design Independent Variables or Experimental Conditions IRT linking method. This study was to evaluate the performance of the four multidimensional IRT scale linking methods propos ed by Oshima et al. (2000): the direct method, equated function method, test characteristic f unction method, and item characteristic function method (see the section Multidimensional IRT Scale Linking in Chapter 1 for detailed description). The equated function, test characte ristic function, and item characteristic function methods were implemented in a manner consiste nt with the implementation in Oshima and colleagues study (2000). For the equa ted function method, the means of 1ja, 2ja, and jd for the first and second halves of the items were used as the equated function. For the test and item characteristic function me thods, seven equally spaced 1 points from -4 to 4 and seven equally

PAGE 43

43 spaced 2 points from -4 to 4, making 7 x 7 = 49 grid points, were used w ith equal unit weight along the ability scale. The four IRT linking methods have been compared under different ability distributions (Oshima et al., 2000). It is unknow n how they perform under other circumstances. Therefore, this study can be considered as an extension of Oshima and colleagues study (2000) from one testing condition (ability distribution) to various testing condi tions (see the following for the detail). Test structure. In IRT, the test dimensionality for a particular population is the minimum number of latent abilities required to produce a monotone and locally independent model (McDonald, 1981, 1997; Stout, 1990). In the geom etrical representation of a test structure, the coordinate axes of a multidimensional space is defined by a complete set of latent abilities examined by the test, and each item is described by a vector in the spac e with its orientation representing the ability composite that is best measured by the item (Ackerman, 1994, 1996; Reckase, 1985, 1991). According to the literature review by Tate (2003), based on the number and nature of the abilities required for the respons e to each item in the test there are three types of test structure: simple structure, approximate simple structure, and complex structure. In the simple structure, all item vectors are exactly aligned with one of the axes in the multidimensional space after an appropriate rota tion, so all the items under each dimension measure the same ability. If all item vectors are approximately aligned with one of the multiple axes and therefore the contribution of one ability is dominant over the contribut ion of all other abilities, the test has approximate simple structure. In complex test structure, the respons e to one item depends on more than one ability. The first type of structur e has been considered as an ideal one and the second and third types as more realistic item structures (Kim, 1994; R oussos, Stout, & Marden, 1998). Following the method in previous studies (Batley & Boss, 1993; Min, 2003; Mroch &

PAGE 44

44 Bolt, 2006; Oshima et al., 1997; Oshima & Miller, 1992; Tate, 2003), two types of twodimension test structure were created by using the three MIRT item characteristics: MDISC, MDIFF, and direction (Ackerma n, 1994; Reckase, 1985; Reckas e, 1997a; Reckase & McKinley, 1991). In the approximate simple st ructure, there were two sets of items: The responses to the first half items depended on one composite ability with the firs t dimension as the dominant dimension and the second dimension as the minor dimension; The responses to the second half items depended on another composite ability with the second dimension as the dominant dimension and the first dimension as the minor di mension. In the complex test structure, there were four sets of items with equal number of ite ms in each of the set. Two sets of items loaded heavily on one of the two dimensions and lightly on the other dimension, and the remaining two sets loaded heavily on both dimensions. Test length. The test items are used to establish the common metric for the two sets of parameter estimates obtained in separate calibrati ons. Therefore, the feature of items is very important for sale linking. The estimation of linking parameters depends not only on the number of items, but also on the characteristics of the item parameters. Based on some literature reviews (Brennan 1987; Cook & Peters en 1987; Kolen & Brennan, 2004), 15-30 common items are necessary for unidimensional IRT linking, althou gh the required number also depends on other conditions, such as the linking methods, examinees ability distributions, and characteristics of the items. Different numbers of items have been used in multidimensional linking studies. Li (1997) used 15 and 25 items in his study and found that the number of items had a significant influence on the stability of transformation parameter estimates for multidimensional linking. Oshima et al., (2000) created 40 item parameters to examine the performa nce of their four IRT multidimensional linking procedures. Twenty items were used in Mins study (2003) to compare

PAGE 45

45 three multidimensional linking methods. In this study, 20 and 40 items were used to evaluate the four MIRT linking methods under different numbers of items with the cons ideration that more items may be needed for MIRT than unidimensional IRT linking. Sample size. Theoretically the performance of linking methods depends on the accuracy of parameter estimates and parameter estimation is affected by the sample size (Li, 1997). So the linking function depends in some extent on different sample sizes. Compared with unidimensional IRT models, a large number of examinees are required for MIRT calibration because more parameters need to be estimated. Based on some MIRT researchers (Ackerman, 1994; Carlson, 1987) recommendations, 2000 examinees is a reasonable sample size to obtain satisfactory item parameter estimates for compensatory multidimensional model. Reckase (1997a) reported that NORHARM (Fraser & Mc Donald, 1988) and TESTFACT (Wilson, Wood, & Gibbons, 1987) generally produced stable paramete r estimates for long te sts and sample sizes exceeding 1000 cases. A comprehensive study (Tate, 2003) using both simulated and real data found that most of the often-used multidimensi onal computer programs performed well for the sample size of 2000 examinees. To acquire stable item parameter estimates, Hirsch (1988) used 2000 examinees to evaluate the proposed multidimen sional equating. In his first study, Li (1997) used three different sample sizes, 1000, 2000, and 4000, to examine the performance of three multidimensional linking methods and found that the sample size had a prominent role in estimating transformation parameters. Li used 2000 examinees to evaluate the best linking method in his second study (Li, 1997). Min (2003) also found the significan t effect of sample sizes, 500, 1000, and 2000 on the accuracy and stability of different multidimensional linking methods and suggested that the sample of 500 examinees showed unrel iable results and the sample of 1000 showed somewhat acceptable outcom es (note that approximate simple structure

PAGE 46

46 and complex structure were used in the study). It is not unusual in testing practice that the sample size is less than 1000 especially in non-achievement area and the performance of the four IRT linking methods need to be evaluated under this c ondition. In this study, three different sample sizes, 500, 1000, 2000, were used to examine th e robustness of the f our multidimensional IRT scale linking methods against parameter estimation errors. As defined in other studies (Li, 1997; Min, 2003), the sample size of 2000 is the base fo r comparing the effect of different parameter estimation errors. The sample size of 500 can be used to examine the robustness of IRT scale linking for small sample size. The sample size of 1000 was used to examine the effect of sample size between 500 and 2000 on scale linking and it was also cons istent with a study using multidimensional linking for identifying differe ntial item functioning (Oshima et al., 1997). Examinees ability distribution. Based on the review by Ko len and Brennan (2004), the performance of scale linking also depends on the similarity between the two groups of examinees. The more similar the groups are, the more adequate the linking will be. Large difference between groups may produce sign ificant problems in estimating scale linking parameters. Groups of examinees may differ in many characteristics, such as cultural background, attitude, motivation, and persona lity. A comprehensive review on population invariance in equating and linking (Kolen, 2004) found that equa ting is population dependent except under highly restrictiv e conditions, such as tw o test forms with similar content, difficulty, and reliability. This suggests that scale linking para meters that are used to obtain the equivalent scores should also be dependent on the populat ions used in the estimation. The ability distribution is an important char acteristic of the examinees and ha s a significant influence on test equating and scale linking under both unidimensional (Cook et al., 1985) and multidimensional circumstances (Li, 1997; Min, 2003). As summarized by Cook and Petersen (1987), the

PAGE 47

47 similarity of ability distributi on between groups also affects ot her conditions required for test equating, such as the nu mber of common items. Groups of examinees may differ from each other in terms of mean, variance, and covariance of the dimensions. Oshima et al. (2000) examined the four multidimensional IRT scale linking methods under six conditions of th e ability distributions across two groups: no difference at all; differences in variances; differences in correlations; differences in means; differences in means and variances; differences in means and correlations. Min (2003) used four conditions similar to those investigated by Oshima et al, such as differences in correlations; differences in correlations and means; and differences in correlations, means, and variances. However, in all conditions the ability dimensions were uncorrelated in the base group. In education and psyc hology, most constructs and dimensions within a construct are correlated. Two groups should have similar stru cture of construct before scale linking and equating are conducted. Given th ese two considerations and to keep the scope of the study manageable, correlations between dimensions were set at the same level, but not zero, across all groups and the two groups varied only in ability le vel and variance. One purpose of this study was to explore two-dimensional linking methods under the followi ng four ability distributions: no difference at all, differences in means, differences in variances, and differences in means and variances (See Table 2-1 for the detail). Dependent Variables or Evaluation Criteria Different statistics have been used to evaluate multidimensional linking methods. Bias and root mean square error (RMS E) are often used to evaluate the accuracy and stability of results across replications of experiment in IRT simulation studies. For example, using a common examinee design, Hirsch (1988) evaluated the effectiveness of multidimensional linking

PAGE 48

48 and equating by examining the means and standard deviations of the differences and absolute differences between the true scor es, ability estimates, test char acteristic response surfaces, and contour plots of the common examinees on the base and equated tests. Li (1997) used bias and RMSE to evaluate three multidimensional linking methods, but he used both the bias and RMSE of linking parameters and item and ability paramete rs over replications in his study. Oshima et al. (2000) compared the means, standard deviat ions, bias, and RMSE of linking parameters for different methods. Another criterion for common ite m scale linking in IRT fram ework is to evaluate how small the differences are between the item pa rameter estimates for base group and the transformed item parameter estimates for equa ted group across the common items (Min, 2003; Min & Kim, 2003). This criterion was used in this study. Specifically, the common item nonequivalent groups design was used and simulati on was performed to create the data for both base and equated groups. The parameters fo r the two groups were estimated and then transformed onto a common scale. Specifically, th e parameter estimates for equated groups were transformed onto the scale for the base groups by using the transformation equations described before. The linking coefficients in the transformation equations, A and were estimated through the four IRT multidimensional lin king methods. After the common item parameters estimated from base and equated groups were placed on the sa me scale, the performance of the four linking methods were evaluated by examining the differe nces between the two se ts of item parameter estimates. The mean difference and diffe rence variation across replications (r) for each item were used to evaluate the accuracy a nd stability of the four linking methods, as described by the following statistics:

PAGE 49

49 r diff Mr r jdiff1a, (2-1) r diffdiff SDjdiff 2 a, (2-2) where j jFFdiffaa*, (2-3) r diff dMr r jdiff1, (2-4) r diffdiff dSDj 2 (2-5) where j jFFdddiff*. (2-6) Procedure Data Generation The following compensatory, two-parameter, tw o-dimension IRT model was used to create the item responses with different testing conditions described above: j d ij a ij aDe daxPjjiij 22111 1 ),;1(. (2-7) First, five sets of ability parameters fo r each of the three sample sizes (500, 1000, and 2000) with multivariate normal distributions with various means, variances, and covariances were generated. One set of ability parameters was used for the base group and the other for the four equated groups (see Table 2-1 for the five gr oup ability distributions).

PAGE 50

50 Second, two sets of item parameters (one with 20 items and another with 40 items) for each of the two test structures (approximate simple structure and complex st ructure) were created using the three MIRT item characteristics: MD ISC, MDIFF, and dire ction (Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKi nley, 1991). Based on the pooled results from past empirical studies (Reck ase, 1985; Reckase, 1997a; R eckase & McKinley, 1991), the estimated MIDSC has a lognormal distribution with mean of 1.37 and standard deviation of 0.54 and the estimated MDIFF has a normal distribution with mean of 0.28 and standard deviation of 0.69. The item parameters of MDISC and MDIFF in this study were selected randomly from lognormal and normal distributions with the same value of means and standard deviations. The test structure was created by mani pulating the angle of each item with the first dimension. For the items that loaded on one dominant dimension, the angle between the item and its dominant dimension was selected from a lognormal distribu tion with mean of 10 and standard deviation of 2. For the items loaded heavily on both di mensions, the angle betw een each item and two dimensions were selected from a normal distributi on with mean of 45 and standard deviation of 10. Next, the discrimination parameters, a1, a2, and the difficulty parameter, d, were computed by the following formula ((Ackerman, 1994; Reckase, 1985; Reckase, 1997a; Reckase & McKinley, 1991): 1 1cos* MDISC a (2-8) 2 2cos* MDISC a (2-9) MDIFF MDISC d (2-10) (See Table 2-2, 2-3, 2-4, and 2-5 for specific parameter values for different test structures with different test lengths)

PAGE 51

51 Next, dichotomous item responses were created using the two-parameter and twodimension IRT model desc ribed by Equation 2-7. To produce more precise and stable results, replications were conducted for each of the combinations of testing cond itions. In IRT simulation studies the number of replications depends on the purpose of the study, the desire of minimizing the sampling variance of the estimated parameters, and the need for statistical tests of results (Harwell, St one, Hsu, & Kirisci, 1996). The previous studies on multidimensional lin king or equating methods used 0 (Hirsch, 1988), 20 (Oshima et al., 2000), 50 (Min, 2003), 100, a nd 200 (Li, 1997) replications to evaluate the accuracy and stability of linking or equa ting results. Based on Harwell and colleagues (1996) recommendation of using a minimum of 25 replications for IRT simulation studies and given the level of complexity of this study, 500 replications were used for each of the combinations of testing conditions to evalua te the accuracy and stability of the four multidimensional IRT linking methods. Parameter Estimation The parameters of MIRT models can be esti mated using different methods and computer programs. The often used estimation methods in clude unweighted least squares (ULS) factor analysis of tetrochoric correlations, weighted least squares (WLS) analysis of the matrix of polychoric correlations, and robust WLS analys is methods performed by MPLUS (Muthen & Muthen, 1998), least squares es timation method based on the matr ix of raw product moments of item scores by NOHARM (Fraser & McDona ld, 1988), marginal maximum likelihood estimation method by TESTFACT (Bock, Gibbons, Schilling, Muraki, Wilson, & Wood, 1999). The study focusing on model parameters recovery by Knol and Berger (1991) suggests that for multidimensional data a common factor analysis on the matrix of tetrachoric correlations performs at least as well as the theoretically appropriate multidimensional item response models

PAGE 52

52 (p. 457). A study comparing TESTFACT and N OHARM (Gosz & Walker, 2002) found that NOHARM provided better soluti ons for predicting item performance. The comprehensive comparison study by Tate (2003) found that MPLUS, NOHARM, and TESTFACT performed reasonably well over a relatively wide range of conditions in assessing the test structure and estimating parameters. This result was confir med by another recent study (Stone & Yeh, 2006). Based on these studies, all these methods can provide satisfactory estimation for model parameters. NOHARM was used in this study du e to its consistently good performance in previous studies. After the MIRT item parameters were estimated by NOHARM, the linking parameters estimated by the four multidimensional IRT lin king methods (direct method, equated function method, test characteristic function method, and item characteristic function method) were obtained by the computer progr am IPLINK, which was developed by Lee and Oshima (1996). Result Analysis Some previous multidimensional linking studies used descriptive analysis (Hirsch, 1988; Oshima et al., 2000) and some st udies used both descriptive and inferential analysis (Li, 2000; Min, 2003). In this study, the m eans and standard deviations of differences between the item parameter estimates for base group (jjFFd a) and the transformed item parameter estimates for equated group (** jjkFFda) across 500 replications were co mpared under different testing conditions. Specifically, the accuracy and stabili ty of the four multidimensional IRT linking methods were evaluated by examining the mean differences and diffe rence variations of 1a, 2a, and d for all items in the test under di fferent testing conditions. Based on the experimental conditions describe d above, there are 5 factors in this study: multidimensional linking method (4), test structur e (2), test length (2), sample size (3), and

PAGE 53

53 ability distribution (4). Therefore, the total number of e xperimental conditions is 422 3 4 = 192. Five hundred replications were conducted for each of the conditions.

PAGE 54

54 Table 2-1. Ability distributions for examinee groups ________________________________________________________________________ Base Group Group1 Group2 Group3 Group4 ___________ _________ __________ __________ __________ ________________________________________________________________________ 0 0 5. 1 1 5. 0 0 5. 1 1 5. 5. 5. 5. 1 1 5. 0 0 4. 8. 8. 4. 5. 5. 4. 8. 8. 4. ________________________________________________________________________ Note: All the correlations between dimensions are .5.

PAGE 55

55 Table 2-2. Item parameters for 20 items with approximate simple structure Item a1 a2 d MDISC MDIFF 1 2 1 1.12 0.18 -0.70 1.13 0.62 9 81 2 2.23 0.52 -0.23 2.29 0.10 13 77 3 1.39 0.24 -2.19 1.41 1.55 10 80 4 1.02 0.16 -0.75 1.03 0.73 9 81 5 1.68 0.33 -1.18 1.71 0.69 11 79 6 0.98 0.15 0.77 0.99 -0.78 9 81 7 1.24 0.22 1.12 1.26 -0.89 10 80 8 0.94 0.13 -1.26 0.95 1.33 8 82 9 1.65 0.32 -1.86 1.68 1.11 11 79 10 2.01 0.46 -1.46 2.06 0.71 13 77 11 0.30 1.30 -0.72 1.33 0.54 77 13 12 0.17 1.09 0.17 1.10 -0.15 81 9 13 0.33 1.86 0.51 1.89 -0.27 80 10 14 0.08 0.63 -0.07 0.63 0.11 83 7 15 0.14 0.99 -1.38 1.00 1.38 82 8 16 0.17 1.09 -2.66 1.10 2.42 81 9 17 0.20 1.15 -0.48 1.17 0.41 80 10 18 0.13 0.84 -0.68 0.85 0.80 81 9 19 0.38 2.37 -0.77 2.40 0.32 81 9 20 0.22 1.04 -0.73 1.06 0.69 78 12

PAGE 56

56 Table 2-3. Item parameters for 40 items with approximate simple structure Item a1 a2 d MDISC MDIFF 1 2 1 2.29 0.44 1.28 2.33 -0.55 11 79 2 1.10 0.21 0.21 1.12 -0.19 11 79 3 1.44 0.20 -1.45 1.45 1.00 8 82 4 0.57 0.09 0.28 0.58 -0.48 9 81 5 0.92 0.16 -1.59 0.93 1.71 10 80 6 0.96 0.20 0.45 0.98 -0.46 12 78 7 1.18 0.23 -0.79 1.20 0.66 11 79 8 1.12 0.16 -1.07 1.13 0.95 8 82 9 0.98 0.19 0.80 1.00 -0.80 11 79 10 1.90 0.30 -0.94 1.92 0.49 9 81 11 0.55 0.14 0.27 0.57 -0.48 14 76 12 1.35 0.26 0.40 1.38 -0.29 11 79 13 1.16 0.23 -0.14 1.18 0.12 11 79 14 1.66 0.29 -0.32 1.69 0.19 10 80 15 1.39 0.24 -1.66 1.41 1.18 10 80 16 0.79 0.14 -0.65 0.80 0.81 10 80 17 1.08 0.19 -1.50 1.10 1.36 10 80 18 1.18 0.15 -0.46 1.19 0.39 7 83 19 0.81 0.16 -0.07 0.83 0.09 11 79 20 0.70 0.11 -0.74 0.71 1.04 9 81 21 0.21 1.32 0.66 1.34 -0.49 81 9 22 0.27 1.19 -0.94 1.22 0.77 77 13 23 0.29 1.62 0.20 1.65 -0.12 80 10 24 0.15 1.20 0.86 1.21 -0.71 83 7 25 0.19 0.98 -1.09 1.00 1.09 79 11 26 0.15 0.77 0.12 0.78 -0.16 79 11 27 0.21 1.53 1.97 1.54 -1.28 82 8 28 0.15 0.97 0.35 0.98 -0.36 81 9 29 0.26 1.45 -0.90 1.47 0.61 80 10 30 0.13 1.09 -0.23 1.10 0.21 83 7 31 0.17 0.86 0.18 0.88 -0.20 79 11 32 0.28 1.59 -0.37 1.61 0.23 80 10 33 0.18 0.84 0.18 0.86 -0.21 78 12 34 0.23 1.48 -0.60 1.50 0.40 81 9 35 0.45 2.58 -0.10 2.62 0.04 80 10 36 0.22 1.40 -1.75 1.42 1.23 81 9 37 0.32 1.38 -0.62 1.42 0.44 77 13 38 0.17 1.05 -0.25 1.06 0.24 81 9 39 0.14 0.89 0.63 0.90 -0.70 81 9 40 0.28 1.42 0.44 1.45 -0.30 79 11

PAGE 57

57 Table 2-4. Item parameters for 20 items with complex structure Item a1 a2 d MDISC MDIFF 1 2 1 1.12 0.18 -0.70 1.13 0.62 9 81 2 2.23 0.52 -0.23 2.29 0.10 13 77 3 1.39 0.24 -2.19 1.41 1.55 10 80 4 1.02 0.16 -0.75 1.03 0.73 9 81 5 1.68 0.33 -1.18 1.71 0.69 11 79 6 0.22 0.96 0.77 0.99 -0.78 77 13 7 0.20 1.24 1.12 1.26 -0.89 81 9 8 0.16 0.94 -1.26 0.95 1.33 80 10 9 0.20 1.67 -1.86 1.68 1.11 83 7 10 0.29 2.04 -1.46 2.06 0.71 82 8 11 0.99 0.89 -0.72 1.33 0.54 42 48 12 0.55 0.95 0.17 1.10 -0.15 60 30 13 1.26 1.40 0.51 1.89 -0.27 48 42 14 0.49 0.40 -0.07 0.63 0.11 39 51 15 0.60 0.80 -1.38 1.00 1.38 53 37 16 0.87 0.68 -2.66 1.10 2.42 38 52 17 0.83 0.83 -0.48 1.17 0.41 45 45 18 0.68 0.51 -0.68 0.85 0.80 37 53 19 1.48 1.89 -0.77 2.40 0.32 52 38 20 0.56 0.90 -0.73 1.06 0.69 58 32

PAGE 58

58 Table 2-5. Item parameters for 40 items with complex structure Item a1 a2 d MDISC MDIFF 1 2 1 2.29 0.44 1.28 2.33 -0.55 11 79 2 1.10 0.21 0.21 1.12 -0.19 11 79 3 1.44 0.20 -1.45 1.45 1.00 8 82 4 0.57 0.09 0.28 0.58 -0.48 9 81 5 0.92 0.16 -1.59 0.93 1.71 10 80 6 0.96 0.20 0.45 0.98 -0.46 12 78 7 1.18 0.23 -0.79 1.20 0.66 11 79 8 1.12 0.16 -1.07 1.13 0.95 8 82 9 0.98 0.19 0.80 1.00 -0.80 11 79 10 1.90 0.30 -0.94 1.92 0.49 9 81 11 0.09 0.56 0.27 0.57 -0.48 81 9 12 0.31 1.34 0.40 1.38 -0.29 77 13 13 0.20 1.16 -0.14 1.18 0.12 80 10 14 0.21 1.68 -0.32 1.69 0.19 83 7 15 0.27 1.38 -1.66 1.41 1.18 79 11 16 0.15 0.79 -0.65 0.80 0.81 79 11 17 0.15 1.09 -1.50 1.10 1.36 82 8 18 0.19 1.18 -0.46 1.19 0.39 81 9 19 0.14 0.82 -0.07 0.83 0.09 80 10 20 0.09 0.70 -0.74 0.71 1.04 83 7 21 1.00 0.90 0.66 1.34 -0.49 42 48 22 0.61 1.06 -0.94 1.22 0.77 60 30 23 1.10 1.23 0.20 1.65 -0.12 48 42 24 0.94 0.76 0.86 1.21 -0.71 39 51 25 0.60 0.80 -1.09 1.00 1.09 53 37 26 0.61 0.48 0.12 0.78 -0.16 38 52 27 1.09 1.09 1.97 1.54 -1.28 45 45 28 0.78 0.59 0.35 0.98 -0.36 37 53 29 0.91 1.16 -0.90 1.47 0.61 52 38 30 0.58 0.93 -0.23 1.10 0.21 58 32 31 0.61 0.63 0.18 0.88 -0.20 46 44 32 1.22 1.06 -0.37 1.61 0.23 41 49 33 0.49 0.70 0.18 0.86 -0.21 55 35 34 1.35 0.66 -0.60 1.50 0.40 26 64 35 2.04 1.65 -0.10 2.62 0.04 39 51 36 1.07 0.93 -1.75 1.42 1.23 41 49 37 1.04 0.97 -0.62 1.42 0.44 43 47 38 0.88 0.59 -0.25 1.06 0.24 34 56 39 0.42 0.79 0.63 0.90 -0.70 62 28 40 1.11 0.93 0.44 1.45 -0.30 40 50

PAGE 59

59 CHAPTER 3 RESULTS As described in Chapter 2, the criterion us ed in this study to evaluate the four m ultidimensional IRT linking methods was based on the differences between the item parameter estimates for the base group and the transforme d item parameter estimates for the equated group across 500 replications. Specifical ly, after the item parameter estimates from the two groups were transformed to a common scale, the mean a nd standard deviation of their differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. For each of the 192 experimental cond itions, there were three parameter estimates 1a, 2a, and ;d therefore, the mean and standard devia tion of the differences were computed for 1a, 2a, and dacross 500 replications for each item of the te st. Then the distributions of the means and standard deviations of the differences of 1a, 2a, and dfor all items in the test were obtained. Based on the characteristic of item parameter invariance in IRT, the item parameter estimates from the base and equated groups should theoreti cally be equal after they are transformed to a common scale. So their differences, and accordingl y the means and standard deviations of their differences across 500 replications, should be 0. Therefore, the performance of the four multidimensional IRT linking methods can be eval uated by examining how close the means and standard deviations of the differences are to 0. There is currently no generally accepted crit erion about how close the item parameter estimates for the two groups should be in order fo r the linking to be considered accurate and stable. To describe the distributi on of difference, histograms of m eans and standard deviations of the differences of 1a, 2a, and dfor the 192 experimental condi tions were prepared. The appendix contains the histograms for all 192 conditions In this chapter, histograms selected to

PAGE 60

60 illustrate the trends in the results will be presented. The following midpoints were used to construct the histograms for means: 0, .2, .4, .6. All va lues smaller than -0.5 and larger than +0.5 were included in the categories wi th midpoints of .6. For the histograms of the standard deviations, 0.1, 0.3, 0.5, 0.7, 0.9, 1. 1, and 1.3 were used as the midpoints. All values beyond 1.2 were classified into the category with midpoi nts of 1.3. If all or most of the items in the test had means and standard deviations close to 0, the linking method was considered accurate and stable. Otherwise, the linking method was inaccurate and unstable. The performance of the four multid imensional IRT linking methods was evaluated in this way under different testing conditions. On the histograms, the direct method, equate d function method, test characteristic function method, and item characteristic function method are labeled Link1, Link2, Link3, and Link4. For test structure, the approximate simple structure is abbreviated as APP and the complex structure as COM. For test length, the number of items in the test is indicated by n = 20 or n = 40. For sample size, the number of examinees is indicated by N = 500, N = 1000, or N = 2000. For ability distribution differences between the base and equated groups, the condition is abbreviated as G1 if the mean vectors and covariance matrices were equal for the two groups, G2 if only the mean vectors were different, G3 if only the cova riance matrices were different, and G4 if the mean vectors and covariance matrices were not equal for the two groups. As will be shown subsequently, inspection of the results indicated that the effects of linking methods depended on the test structure. Therefore, the decision was made to focus primarily on the effects of linking methods within each of the test structures. Inspection of the results for APP suggested that the interactions of all other factors were small in size, therefore the focus was on the main effects of the factors. Inspection of the COM results suggested that

PAGE 61

61 there were two-way, three-way, or four-way inter actions of other factors, so the performance of the linking methods were described taki ng into account th ese interactions. This chapter consists of six sections. The firs t section compares the general performance of the four linking methods. The second section compares the four linking methods for different test structures. The third section compares linking methods for tests with different lengths. The fourth section compares linking methods for diffe rent sample sizes. The fifth section compares linking methods for groups with different ability distributions. The la st section shows the relationship between scale linking perf ormance and item parameter values. General Performance of the Different Linking Methods The performance of the four m ultidimensiona l IRT linking methods was first compared across all testing conditions by collapsing the means and standard deviations of 1a, 2a, and dfor all items under different testing conditions The histograms in Figure 3-1 show the distributions of means and standard deviations for 1a, 2a, and d across all item s and both test structures. Based on the percentage of items with the means and standard deviations of differences close to 0, Link1 (d irect method) produced more accurate and stable linking results than Link4 (item characteristic function method), and Link4 yielde d more accurate and stable results than Link3 (test characte ristic function method). Link2 (e quated function method) did not provide accurate and stable result s for a high percentage of items. The performance of the four linking methods wa s also examined separately for different test structures. Figure 3-2 show s the distributions of means and standard deviations for 1a, 2a, and dfor APP and COM conditions. Comparing the hi stograms for the four linking methods on the left side of the figures, one can see that there was no apparent difference among the four linking methods for APP conditions. The histograms on the right side of the figures show that

PAGE 62

62 there was obvious difference among the four li nking methods for COM conditions. Specifically, based on the accuracy and stability of linking func tion, (a) Link1 (direct method) worked well, (b) Link2 (equated function method) worked poor ly, and (c) the perfor mance of Link3 (test characteristic function method) and Link4 (item characteristic functi on method) was between that of Link1 and Link2, with Link4 being slightly better than Link3. In sum, Link1 (direct method) was consis tently the best method and Link2 (equated function method) the worst method under most COM conditions; the four linking methods worked equally and consistently well under most APP conditions. Performance of Linking Methods for Different Test Structures In this section, the perform a nce of the four linking methods is compared between APP and COM conditions. Figure 3-2 shows di fferent linking results for the two test structures. Based on the histograms for APP and COM c onditions in the figure, all the four linking methods produced more accurate and more stable results for APP te sts than for COM tests, but the difference in quality of linking varied acro ss the linking methods. Specifically Link1 (direct method) results were slightly better for APP tests than fo r COM tests, especially for parameters 1aand 2a; Link3 (test characteristic func tion method) and Link4 (item character istic function method) results were much better for APP tests than for COM tests; Link2 (e quated function method) yielded very poor results for COM tests, but good results for APP tests. However, for the large sample size (N = 2000), Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic f unction method) worked almost equally well for APP tests and COM tests; the linking performance diff erence between APP and COM conditions still remained for Link2 (equated function method) due to its poor function for COM conditions. Figure 3-3 shows the results of four linking methods for APP and COM tests

PAGE 63

63 when the sample size is 2000. With smaller sample sizes (N = 500 and N = 1000), the linking performance difference between APP and COM conditions increased for Link3, Link4, and Link1 (see specific histograms in the appendix). Therefore, test structure had its smallest e ffects on Link1 (direct method), larger effects on Link3 (test characteristic function method) and Link4 (item characteristic function method), and the largest effect on Link2 (equate d function method). Link2 worked well for all APP tests, but poorly for all COM tests in this study. Due to the strong influence of test structure on the function of the four linking methods most of the results in the following sections are presented separately for the APP and COM conditions. Performance of Linking Methods for Different Test Lengths Given the different perform ance of linki ng methods for APP and COM conditions, the influence of test lengths on the linking function was explored sepa rately for APP and COM tests. The distributions of means and standa rd deviations of differences for 1a, 2a, and dfor short and long tests under the APP conditions are presented in Figure 3-4. Based on the histograms in the figure, one can see that for APP tests, a lthough the linking performance was not strongly influenced by test length, all four linking me thods produced slightly more accurate and stable results with long tests. Inspection of the results indi cated that under COM condition s, the performance of the linking methods depended on the sample size and test length. Therefore, th e influence of test lengths was next explored separa tely for different sample sizes for COM tests. Figure 3-5 shows the linking results for short and long tests with sample size of 500. Although none of the four linking methods worked well, the histograms still show that the results for Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) for

PAGE 64

64 short tests were better than those for long tests and that Link1 to some extent performed similarly for different test lengths. Figure 3-6 illustrates the linki ng results for short and long te sts with sample size of 1000. From the figure, it was difficult to state at which test length linking performance was better. Subsequently reported results will show that th e performance depended to some extent on the ability distribution difference between the base and equated groups. When the linking results for ability condition 2 (G2: unequal mean vectors) were excluded, the linking function for long test was obviously better than that for short test except for Link2 (equated function method), as presented in Figure 3-7. Theref ore, in Figure 3-6, the performance under G2 masks the positive effect of test length on the linking accuracy and stability. Shown in Figure 3-8 are linking re sults for different test lengths with sample size of 2000. It is very obvious that the linking results for long tests were better than those for short tests except for Link2 (equated function method). In sum, the linking results for long tests were better than those for short tests except in some COM conditions when the sample size was small. Performance of Linking Methods for Different Sample Sizes Inspection of results indicated that sample size had stable and cons istent influence on the linking performance, but with different degrees of influence for different test structures. Therefore, the effect of sample size is firs t shown across all other testing conditions then presented separately for APP and COM conditions Figure 3-9 contains the linking results for different sample sizes. Compari ng horizontally the histograms for different sample sizes, one can find that both the linking accuracy and stabilit y increased for Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic f unction method) with the sample sizes changing from 500 and 1000 to 2000. Howe ver, the linking perfor mance increased at

PAGE 65

65 different degrees for different test structures. Figure 3-10 shows the linking results for different sample sizes for APP tests. Figure 3-11shows th e results for different sample sizes for COM tests., From Figure 3-10, it can be found that the accuracy of the four linking methods was fairly good at all sample sizes and the stability of the four linking methods increased when the sample size became large for APP tests. Figure 3-11 sugg ests that although the acc uracy and stability of Link1, Link3, and Link4 increased when the samp le size became large for COM tests, the linking performance was poor for sample sizes of 500 and 1000 especially for Link3 and Link4. In addition, for COM tests, the accuracy and stab ility for Link2 (equated function method) were very poor for all sample sizes and relatively unaffected by sample size. Based on these findings, (a) Link1 (direct method), Link3 (test characteristic function method), and Link4 (item characteristic function method) for APP tests were less affected by different sample sizes than we re COM tests; (b) Link1 (direct method) was less affected by sample sizes than were the othe r linking methods for COM tests. Performance of Linking Methods for Gro ups w ith Different Ability Distributions Inspection of the results suggested that the linking results for different ability distributions depended on other testing conditions. Therefore, the influence of ability distribution was first explored separately for APP and COM tests. Figure 3-12 shows the linking results for groups with different ability distributions under the APP conditions. Comp aring horizontally the histograms across G1 (equal mean vectors, eq ual covariance matrices), G2 (unequal mean vectors, equal covariance matrices), G3 (equal mean vectors, unequal covariance matrices), and G4 (unequal mean vectors, unequal covariance matrices) indicat es that: (a) for 1a and 2a, the linking results for G1 were slightly better th an those for other abil ity conditions; (b) for d, the results for G2 were somewhat worse than thos e for other ability conditions; (c) Link2 (equated

PAGE 66

66 function method) was least affected by ability di stributions. The results imply that a difference between groups in the mean vectors was more in fluential than a difference between the groups in the covariance matrices. Inspection of results for COM tests indicated th at the influence of ab ility distribution was moderated by sample size; Therefore, the effect of ability distribution was explored separately for N=500, N=1000, and N=2000 for COM tests with the concentra tion on Link1 (direct method), Link3 (test characteristic function met hod), and Link4 (item characteristic function method). Figure 3-13 shows the linking results for groups with different abil ity distributions with sample size of 500. One can see from the fi gure that although none of the linking methods worked well for the small sample size, the linki ng results for G2 (unequal mean vectors, equal covariance matrices) and G4 (unequal mean vect ors, unequal covariance matrices) were worse than for G1 (equal mean vectors, equal covari ance matrices) and G3 (equal mean vectors, unequal covariance matrices). Even though Link1 (direct method) was relatively unaffected by between-group difference in ability distributi ons in groups, it still did not work well in linkingdfor G2, which indicates the strong influence of the mean difference between groups. The linking results for different groups with sa mple size of 1000 presented in Figure 3-14 shows that linking methods did not work well under G2, especially for d. However, there was some interaction between group differences and test le ngth. For long test (n = 40), the linking methods did not work well for G2 (see Figure 3-15); for short test (n = 20), the linking methods worked relatively well for G2 (see Figure 3-16). The linking results for different groups with sample size of 2000 shown in Figure 3-17 suggest that the linking methods worked approximately equally well for the groups with differe nt ability distributions.

PAGE 67

67 In sum, the influence of ability distributi ons on linking results de pended on other testing conditions: (a) between-group differences in ability distributions did not ha ve a strong influence on the performance of the four linking methods for APP conditions or for COM conditions with a large sample size; (b) mean difference between groups had negative influence on the linking results especially for conditions with small sample size. Performance of Linking Methods for Test Items with Different Parameter Values Two types of scatter-plots were used to exam ine the relationship between linking performance and item parameter values under each of the 48 testing conditions. The first type of scatter-plot was used to evaluate the effect of item parameter va lues on the accuracy of different linking methods, with y axis as the mean of the differences and x axis as the true parameter values which were used to generate the item re sponse data. The second type of scatter-plots was used to evaluate the effect of item parameter va lues on the stability of different linking methods, with y axis as the standard deviation of the diffe rences and x axis as the true parameter values. These two scatter-plots were constructed for each of the three parameter estimates (e.g., 1a, 2a, and d) under each of the 48 testing conditions. However, the results of Link2 (equated function method) were not included in th e scatter-plots un der the COM conditions due to its consistently poor performance. Given the limitation of space, the main outcomes are illustrated by some representative examples. The results suggest that: (a) Under most of the testing conditions, the linking results tended to be less accurate for 1a and 2awhen the two parameters had extreme values, and (b) under most of the testing conditions, the linking results became less stable for 1a and 2aas the parameters values increased. The results also in dicate that: (a) The accuracy of linking results for dwas not closely related to their true parameter values under most of the testing conditions, and

PAGE 68

68 (b) the stability of linking results for dwas also not closely related to their true parameter values when the sample size was not large. The scatte r-plots for one testing condition (COM, n = 20, N = 1000, G3) illustrate these relationships between linking performance and item parameter values (see Figure 3-18). However, for large sample size (N = 2000) the stability of linking results for d was closely related to their absolute true parameter values. Specifically when the absolute parameter values of d were closer to 0, the linki ng results were more stable; when the absolute parameter values of d were farther away from 0, the linking resu lts were less stable. The scatter-plots for another testing condition (APP, n = 40, N = 2000, G4) show that: (a) the linking results tended to be less accurate and less stable for 1a and 2a when the two parameters had extreme values; (b) the linking results tended to be less stable for das the absolute parameters values increased (see Figure 3-19).

PAGE 69

69 Figure 3-1. Accuracy and stabil ity for different linking methods

PAGE 70

70 Figure 3-1. Continued

PAGE 71

71 Figure 3-1. Continued

PAGE 72

72 Figure 3-2. Accuracy and stability by linking method and test structure

PAGE 73

73 Figure 3-2. Continued

PAGE 74

74 Figure 3-2. Continued

PAGE 75

75 Figure 3-3. Accuracy and stability by linking method and test structure: N = 2000

PAGE 76

76 Figure 3-3. Continued

PAGE 77

77 Figure 3-3. Continued

PAGE 78

78 Figure 3-4. Accuracy and stabilit y by linking method and test length for approximate simple structure tests

PAGE 79

79 Figure 3-4. Continued

PAGE 80

80 Figure 3-4. Continued

PAGE 81

81 Figure 3-5. Accuracy and stabilit y by linking method and test lengt h for complex structure tests: N = 500

PAGE 82

82 Figure 3-5. Continued

PAGE 83

83 Figure 3-5. Continued

PAGE 84

84 Figure 3-6. Accuracy and stabilit y by linking method and test lengt h for complex structure tests: N = 1000

PAGE 85

85 Figure 3-6. Continued

PAGE 86

86 Figure 3-6. Continued

PAGE 87

87 Figure 3-7. Accuracy and stabilit y by linking method and test length for complex structure tests when G2 was excluded: N=1000

PAGE 88

88 Figure 3-7. Continued

PAGE 89

89 Figure 3-7. Continued

PAGE 90

90 Figure 3-8. Accuracy and stabilit y by linking method and test lengt h for complex structure tests: N = 2000

PAGE 91

91 Figure 3-8. Continued

PAGE 92

92 Figure 3-8. Continued

PAGE 93

93 Figure 3-9. Accuracy and stabilit y by linking method and sample size

PAGE 94

94 Figure 3-9. Continued

PAGE 95

95 Figure 3-9. Continued

PAGE 96

96 Figure 3-10. Accuracy and stab ility by linking method and sample size for approximate simple structure tests

PAGE 97

97 Figure 3-10. Continued

PAGE 98

98 Figure 3-10. Continued

PAGE 99

99 Figure 3-11. Accuracy and stab ility by linking method and sample size for complex structure tests

PAGE 100

100 Figure 3-11. Continued

PAGE 101

101 Figure 3-11. Continued

PAGE 102

102 Figure 3-12. Accuracy and stability by linking method and gr oup for approximate simple structure tests

PAGE 103

103 Figure 3-12. Continued

PAGE 104

104 Figure 3-12. Continued

PAGE 105

105 Figure 3-13. Accuracy and stabil ity by linking method and group for co mplex structure tests: N = 500

PAGE 106

106 Figure 3-13. Continued

PAGE 107

107 Figure 3-13. Continued

PAGE 108

108 Figure 3-14. Accuracy and stabil ity by linking method and group for co mplex structure tests: N = 1000

PAGE 109

109 Figure 3-14. Continued

PAGE 110

110 Figure 3-14. Continued

PAGE 111

111 Figure 3-15. Accuracy and stability for di fferent linking methods: COM, n=40, N=1000, G2

PAGE 112

112 Figure 3-16. Accuracy and stability for di fferent linking methods: COM, n=20, N=1000, G2

PAGE 113

113 Figure 3-17. Accuracy and stab ility by linking method and group for complex structure tests: N = 2000

PAGE 114

114 Figure 3-17. Continued

PAGE 115

115 Figure 3-17. Continued

PAGE 116

116 Figure 3-18. Linking accuracy and stability and item parameter values: COM, n=20, N=1000, G3

PAGE 117

117 Figure 3-18. Continued

PAGE 118

118 Figure 3-18. Continued

PAGE 119

119 Figure 3-19. Linking accuracy and stability and item parameter values: APP, n=40, N=2000, G4

PAGE 120

120 Figure 3-19. Continued

PAGE 121

121 Figure 3-19. Continued

PAGE 122

122 CHAPTER 4 DISCUSSION By using simulated data, the perform ance of the four multidimensional IRT scale linking methods was evaluated under differe nt testing conditions, which incl ude different test structures, test lengths, sample sizes, and ability distributi ons. The results illustrated in Chapter 3 suggest that test structure had a str ong influence on the performance of the four linking methods. For approximate simple test struct ure, each of the four linking methods worked approximately equally well under all testing cond itions. For complex test struct ure, the equated function method did not work well under any testing conditions; th e performance of other three linking methods depended on different testing conditions; the dir ect method was the best linking procedure for most testing conditions. In addition, the ite m parameter values influenced the linking performance. The results are discusse d in this chapter by seven sections. Results from Previous Studies Theoretically, there are at least two m ain co mponents in linking errors: error caused by parameter estimation and error produced by scale transformation (Li, 1997). A simulation study (Kaskowitz & Ayala, 2001) found that linking was more accurate when there was less error in the item parameter estimates. Therefore, it is important to review prev ious studies about IRT parameter estimation and linking accuracy under different testing conditions, although it is difficult, if not impossible, to decompose the parameter estimation error from the linking error in testing practice. Based on Lis review (1997), the following fact ors can cause error in parameter estimation in IRT: (a) Examinees ability di stribution. Item difficulty for easy and hard items will not be well estimated when the examinees are normally distributed around their mean; Examinees with ability levels above or below the item difficulty are more informative for estimating item

PAGE 123

123 discrimination parameter; (b) Item parameter value. Item difficulty parameters that are small or large and discrimination parameters that are sma ll or large produce larger estimation error; (c) Sample size. Larger sample sizes reduce estima tion error. However, the standard error of parameter estimates depends on the combined eff ect of these factors (Thissen & Wainer, 1982). Using the bias and RMSE between the transformed linking parameter estimates and the true linking parameters across re plications as the criterion, Li (1997) found that the linking accuracy of his three methods improved as sample size or test length increases. In the second study, Li (1997) used the bias and RMSE between the transformed item parameter estimates and the true parameter values across replications as the criterion and found that one of his linking methods (e.g., the combination of procrustean ro tation approach for dimensional transformation, the ratio of trace procedure for dilation, and the least square procedure fo r translation) produced accurate linking of items. In addition, the positiv ely skewed distribution of the second dimension in equated group did not negatively influence the linking accuracy and stability. To evaluate the performance of the four multidimensional IRT scale linking methods, Oshima et al. (2000) used differe nt criteria, including mean and standard deviation of the linking parameter estimates over 20 replications, bias and mean square error (MSE) between the estimated and true linking parameters, correla tion and mean absolute difference of linking parameter estimates across different methods, and minimized function values by different methods. The results indicate that : (a) The direct method and e quated function method tended to yield similar linking results and the test characteristic function method and item characteristic function method tended to produce similar results, (b) the test and item characteristic function methods were more accurate and stable than the other two methods, and (c) the accuracy and stability decreased as ability differences between the groups increased.

PAGE 124

124 Min (2003) used the bias and RMSE between transformed item parameter estimates and the initial item parameters acro ss the common items as the crit erion to compare Lis (1997) composite procedure, Oshima and colleagues test characteristic function method, and Mins extended composite procedure. Base d on the repeated measures analysis of variance for bias and log transformed RMSE, The author found that: (a) The ability distribution, test structure, and linking method accounted for large portion of the va riation in bias for discrimination parameter estimates but only linking method was an important factor for the va riation in bias of difficulty parameter estimates, (b) the sample size, ability distribution, and linki ng method were important for linking stability of discrimination paramete r estimates and sample size and linking method were critical for linking stability of difficulty pa rameter estimates, (c) as the sample size became larger and the two groups were more similar, the linking results became more accurate and stable, and (d) linking methods had significant intera ction effects with testing conditions. In sum, the linking methods and the three testing conditions, e.g., the ability distribution, test structure, and sample size, significantly affected the linking accuracy and stability. Effects of Different Test Structures In the present study, the perform ance of all four linking methods worked much better for APP than for COM tests. This is consistent w ith the fact that the test structure and item parameters are typically more easily and accurately estimated for APP test than for COM test. As Tate (2003) found and discussed in a study comparing different estimation methods, including NOHARM, for assessing the test st ructure of item responses, th e default rotation methods in exploratory analysis are usually developed to transform the initia l solution to simple structure, therefore the procedures may not always successfully describe non-simple test structure. A study (Gosz & Walker, 2002) comparing the perf ormance of TESTFACT and NOHARM found that the item parameter estimation of NOHARM depends heavily on the number of bi-dimensional

PAGE 125

125 items in the test with its better performa nce for fewer bi-dimensional items and worse performance for more bi-dimensional items. In addition, NOHARM is good at estimating items with very low values on one discriminati on parameter and high values on the other discrimination parameter. In this study, for APP tests, all items ha d higher values on one discrimination parameter and lower values on the other discrimination parameter; for COM tests, half of the items had approximately similar va lues on both discrimination parameter values. An investigation of item parameter estimation for the simulated data used in the present study indicated that the NOHARM program provided bette r item parameter estimation for APP tests than for COM tests. The superior estimation for the APP tests is likely the source of the superior linking results for the APP tests. However, in a study (Min & Kim, 2003) comparing Lis composite procedure and Oshima and colleagues test ch aracteristic function method under different testing conditions, no apparent linking difference was found between APP and COM tests (see Figure 2-7, Min & Kim, 2003). One possible reason is that the items loadings on the two dimensions in COM tests in this study were more similar than those in Mi n and Kims study. Specifically the heavily crossloaded items in their study had the direction of 50 65 and 25 40, and the direction of heavily cross-loaded items in this study was sele cted from a normal distribution with mean of 45 and standard deviation of 10. According to the finding by Gosz and Walker (2002), the item parameter estimates for COM tests in this study we re less accurate, so that the linking results were more different between APP and COM test s. Another possible reas on is related to the different criteria used to desc ribe the linking performance. This study used the percentage of items with different means and standard deviations between the item parameter estimates for the base group and the transformed item paramete r estimates for the equated group over 500

PAGE 126

126 replications to evaluate linki ng results. Min and Kims study (2003) used the bias and RMSE between true parameter values and transforme d parameter estimates over both 20 items and 50 replications, which may have difficulty in identif ying the differential in fluence of APP and COM tests on the linking results. As shown in the Results chapter, the linki ng results (except for e quated function method) were very similar for APP and COM tests when the sample size became large (N = 2000). This may be related to the possible improved item parameter estimation for larger sample size for both APP and COM tests. However, the attribu tion of different linking performance for the two types of tests to estimation error needs to be investigated by more controlled st udies in the future. Effects of Different Test Lengths It was illustrated in the last chapter that the linking results for long tests were typically better than those for short tests, which is cons istent with Lis finding (1997) that the linking accuracy of his three m ethods improved as test length increases. This result was not unexpected since more items can provide more information to set up the linkage betw een the scales for the base and equated groups. The positive effect of large number of items on linking and equating performance has already been found in various unidimensional equating conditions (Budescu, 1985; Fitzpatrick & Yen, 2001; Kaskowitz & Ayal a, 2001; Kim & Cohen, 2002; Peterson, Cook, & Stocking, 1983; Swaminathan & Gifford, 1983; Wingersky, Cook, & Eignor, 1987). Therefore, this effect of the number of items can be extended from unidimensional to multidimensional linking and equating situations. However, there was an exception that the linki ng results for short CO M tests were better than those for long COM tests when the sample size was small (N = 500). Li could not find this exceptional result because he used sample sizes of 1000, 2000, and 4000 in his study. One possible reason for this exceptional result is th at small sample size was not large enough to

PAGE 127

127 produce accurate item parameter estimates for long test because more item parameters needed to be estimated, which accordingly affected the linki ng performance for long COM tests. Therefore, the strength of large number of items in scale linking and equating depends on the quality of the item parameter estimation, which in turn require s enough sample size. Lord (1980) stated that it is test length in combination with sample size that affects the quality of parameter estimates. Compared with unidimensional IRT models, a la rger number of examinees are required for MIRT calibration because more parameters need to be estimated. In addition, this study found th at the effects of test leng th on scale linking performance also depended on the ability dist ributions for the two groups. As de scribed in Results chapter, the long test (n = 40) did not improve linking perfor mance when the means of ability distributions were different for base and equated groups for COM test when the sample size was 1000. This phenomenon confounded the general positive effect of large number of items on linking results. Klein and Kolens study (1985) suggests that test length has little effect on the equating quality when groups are similar in ability, but becomes very important when two groups differ in ability level. They found that a larger number of co mmon items did improve equating when groups were dissimilar. However, the exceptional result from this study mentioned above did not confirm their finding. Further studies are n eeded to examine the conflicting findings by controlling more conditions. Effects of Different Sample Sizes Based on the results from this study, the e ffects of sa mple size were very obvious and straightforward. Generally speaki ng, the linking accuracy and stab ility improved with the sample size increasing. This is consistent with the fact that large sample size can improve item parameter estimates. The same pattern was also found in the other two multidimensional scale linking studies (Li, 1997; Min & Kim, 200 3). In addition, the positive effect of large sample size has

PAGE 128

128 also been found in unidimensional linking and equating studies (Fitzpatrick & Yen, 2001; Hanson & Beguin, 2002; Kim & Cohen, 2002; Pete rson, Kolen, & Hoover, 1989; Ree & Jensen, 1983). However, the linking performance improved at different degrees for APP tests and COM tests. The performance of direct method, test characteristic function method, and item characteristic function method in creased much more rapidly for COM tests than for APP tests when the sample sizes became larger. In fact, th e linking results for APP tests were consistently good for different sample sizes. However, the linki ng results for COM tests were very different for different sample sizes, although the linking function improved with the sample sizes increasing. This result was not found in Min and Kims study (2003). They showed similar effect of sample size on linking accuracy and stability for APP and COM tests (see Figure 2-7, Min & Kim, 2003). As we discussed for the effect of test structures on linking performance, this may be related to the different manipulatio ns of COM test items and differe nt evaluation criteria used in these two studies. Effects of Different Ability Distributions Based on this study, for all APP conditions a nd the COM conditions w ith a large sample size, between-group differences in ability distributions did not have a large influence on the performance of the four linking methods. For CO M conditions with small and medium sample size (N=500, N=1000) between-group differences in mean ability had a negative influence on the linking results. It seems that mean difference was more important than variance difference. These results were consistent with what Oshima et al. found in their study using very similar ability conditions (see Table 5 and Figure 1, Os hima et al., 2000), although they did not divide tests into APP and COM tests. However, we n eed to be very cauti ous about the possible differential effect of mean and variance differences on scale linking in both studies because they

PAGE 129

129 were controlled at different degrees, with mean difference at 0.5 and the variance difference at 0.2. Based on the study by Min and Kim (see Figure 27, Min & Kim, 2003), it seems that the influence of ability distributi ons on scale linking by the test ch aracteristic function method was approximately similar for APP and COM conditi ons (see the above explanation for possible reasons for this conflicting findings between thei r study and this study). However, they did find that the influence of between-group differences in ability distribution on scale linking depended on sample size, with less influence for large sa mple size (N = 2000) and more influence for small sample size (N = 500). This is consiste nt with the results from this study. Li (1997) used a different manipulation of the between-group difference in ability distribution than was used in the present study: for the base group both ability distributions were normal; for the equated group one ability distribution was normal and the other was positively skewed. No negative effect was found on the linking performance by us ing his three methods. The reason may be that although th e second ability had positively sk ewed distribution, the mean and standard deviation were stil l controlled at 0 and 1, which were the same as for the based group for the second dimension. It seems that mean and standard deviation were more important than the normality of the distribution. However, this conclusion needs to be confirmed for the MIRT linking methods. Based on the research on unidimensional scal e linking and test equa ting (see the review by Kolen and Brennan, 2004), the similarity betw een two groups of examinees affects linking and equating performance: the more similar the groups are, the more adequate the linking and equating will be; large differences between gr oups may produce signifi cant problems. Based on results from multidimensional scale linking, this conclusion can be extended to the

PAGE 130

130 multidimensional cases, but with cautious consid eration of the interaction between ability distribution, test structure, and sample size. Effects of Different Item Parameter Values As m entioned in the first section, estimation of the item difficulty parameter is less accurate when the parameters are small or large, estimation of discrimination parameter is less accurate when discrimination parameters are sm all or large, and error in item parameter estimation affects scale linking perf ormance. Therefore, linking quality is likely to be influenced by the item parameter values, especially by th e extreme parameter values. This conceptual inference and conclusion were confirmed in this study: under most of th e testing conditions, the linking results tended to be less accurate when th e absolute item parameters had extreme values and less stable when the absolute item parameter values became large. This pattern of results was more apparent when (a) the test had approximate simple structure, (b) the sample size was larger, and (c) the linking performance for discrimination was evaluated. The only other multidimensional scale linking study evaluating the effects of different item parameter values was conducted by Li (1997). Ba sed on that study (see Figure IV-1-16, Li, 1997), the linking results for difficulty were more accurate and stable when the absolute item parameter values became larger; the linking results for discrimination did not change consistently with the item parameter values. Therefore, the effects of different item parameter values on scale linking were more apparent for discrimination in this study and more obvious for difficulty in Lis study. This is reasonable given Min and Kims (2003) conclusion that Lis method worked better than Oshima and colleagues test characteris tic function method (2000) for difficulty parameters and Oshima and colleagues method worked better for the two discrimination parameters.

PAGE 131

131 Performance of Differe nt Linking Methods The effects of test structure, tes t length sample size, ability distribution, and item parameter values on scale linking performance were separately discussed above. However, these factors interacted with each ot her and had both main and combined effect on the performance of the four linking methods. As summarized at the begi nning of this chapter, genera lly speaking, all four linking methods worked approximately equally well under all testing conditions for approximate simple tests. For complex tests, the direct method was the best linking procedure; the item characteristic function method and the test char acteristic function method we re the second and third; the equated function method did not work well for co mplex tests. These results were based on the differences between the item parameter estimat es for base group and the transformed item parameter estimates for equate d group for the common items. It is not entirely su rprising that the direct method, which minimizes the sum of squared differences between the two sets of item parame ter estimates over items, was the best one across different testing conditions because the evaluatio n criterion was consistent with the method. However, the equated function method estimates the linking parameters by minimizing the sum of squared difference between the means of the two sets of selected item parameter estimates in the test. It uses the accumulative information of some items. Therefore, it is possible that even though the mean parameter estimates were simi lar for the two groups, individual parameter estimates were not. In the same way, item characteristic function method uses the combined information of discrimination, difficulty, and ability item by item. The test characteristic function method uses the accumulative information of discri mination, difficulty, and ability over all items in the test. Therefore, item characteristic func tion method was better than test characteristic function method using the criterion based on differe nce between item parameter estimates.

PAGE 132

132 Why do the four linking methods worked equa lly well for approximate simple tests but differentially poor for complex test? One of possible reason is that there is complicated interaction between item parameter estimation er ror and the characterist ics of the four linking methods. More simulation studies need to be conduc ted to differentiate the two types of effect on the performance of scale linking.

PAGE 133

133 CHAPTER 5 CONCLUSIONS The purpose of this study was to use sim ulate d data to examine the performance of four multidimensional linking methods under different testing conditions. There were one hundred and ninety-two experimental c onditions in this study: four linking methods (direct method, equated function method, test ch aracteristic function method, a nd item characteristic function method) by two test structures (approximate simple test structure and complex test structure) by two test lengths (20 items and 40 items) by three sample sizes (500, 1000, and 2000), and by four different ability distributions between two gr oups (no difference, only mean difference, only variance difference, and both mean and variance difference). Five hundred replications were conducted for each of the experimental conditions. The linking performance evaluation was based on the differences between the item pa rameter estimates for base group and the transformed item parameter estimates for equa ted group for the common items. The mean and standard deviation of the differences across the 500 replications were computed to examine the accuracy and stability of the four linking methods. Conclusions Conclusion 1: The performance of the four linki ng methods. Generally speaking, the direct method was the best linking method; the item characteristic function method and test characteristic function method were the second and th ird best method; the equated function method was the last method. Howeve r, their linking performance depended on the following testing conditions. Conclusion 2: The effects of test structure. For approxi mate simple test structure, each of the four linking methods worked approximately equally well for all testing conditions; For complex test structure, the equated f unction method worked poorly under all testing conditions; the performance of the other thr ee linking methods depended on other testing conditions; the direction method was the be st method for most testing conditions. Conclusion 3: The effects of test length. The linking performance for long tests was typically better than that for short tests except for complex tests when the sample size was small.

PAGE 134

134 Conclusion 4: The effects of sample size. The lin king performance improved when the sample size became larger, especially for complex tests. Conclusion 5: The effects of ability distribution. Quality of linking performance declined when there was difference in ability distri bution between the two groups, especially for complex tests; however, it seems that a betw een-group difference in the means was more important than a difference in the variance. Conclusion 6: The effects of item parameter va lues. Under most of the testing conditions, the linking results for the discrimi nation parameter tended to be less accurate and less stable when the item parameter had extreme values. The linking accuracy for the difficulty parameter was not dependent on the item parameter values. The linking stability for the difficulty parameter depende d on the item parameter values only when the sample size was large. Then, the linking results were less stable when the item parameter had extreme values. Future Research In this study, there are a num b er of limitations, which should be considered for making the conclusions described above. For example: (a) Although the item parameters for short and long approximate simple tests and complex tests were randomly created in the same way and from the same distributions, they did not have the same exact values. This should be considered when comparing the linking results for the four type s of tests; (b) The test structure was not constructed by randomly arranging th e items in the test. For the approximate simple test, the first half of the items had higher discrimination values for the first ability and lower values for the second ability, and the second half of the items had lower discrimination values for the first ability and higher values for th e second ability. The equated function method in this study used the means of the first half of items (all with lower or higher values), second half of items (all with lower or higher values) as the function to estimate the linking parameters. This may affect the linking performance of the equated function method; (c) This study used the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group as the criterion, which is consistent with th e minimized function of

PAGE 135

135 the direct method and accordingly may favor this method. The linking performance should be evaluated using other criteria which are consistent with the other methods to examine the possible dependence of the results on the criteri a used. These criteria include the differences between the means of the selected item parame ter estimates obtained from the two groups, the differences between the test characteristic f unctions for a given range of ability, and the differences between item characteristic functions for a given range of ability. As mentioned in the first ch apter, the development of mu ltidimensional linking methods is just at the infancy stage and more research is needed to obtain definitive results. Therefore, a substantial research needs to be conducted to explore and evaluate di fferent procedures for multidimensional scale linking. Here are some fu ture research topics on multidimensional IRT scale linking. First of all, different specific procedures within each of the four linking methods need to be explored, compared, and evaluated so that the best method can be chosen for some specific purpose. For example: (a) For the test characte ristic function and item characteristic function methods, how should the theta region or levels be chosen? Should we use the equally spaced grid theta method or empirical theta method? If we choose empirical theta method, which examinee group, base group, equated group, or combined group, should be used? Which method is better? Should we give different weights to different theta regions and how to choose different weights? (b) For equated function method, which item pa rameter estimates shou ld be used? What characteristics should be considered to choose the appropriate sets of items? What function should be used to produce good linking performance? Second, what kind of criteria should be used to evaluate the performance of different linking methods? Within the multidimensional IRT linking and equating studies, different

PAGE 136

136 criteria have been used. Even within one study, different criteria have been used. For example, Li (1997) used bias and RMSE between the tran sformed linking parameter estimates and the true linking parameters across replications in his fi rst study and then used bias and RMSE between true item parameter values and the transformed ite m parameter estimates and ability recovery in his second study. Oshima et al. (2000) used mean and standard deviation of the linking parameter estimates over 20 replications, bias and MSE between the estimated and true linking parameters, correlation and mean absolute diffe rence of linking parameter estimates across different methods, and minimized function values by different methods. Min (2003) used bias and RMSE between transformed item parameter estimates and the initial item parameters across common items for the simulated data, and used the differences between the item parameter estimates for base group and the transformed item parameter estimates for equated group across the common items for the real data. Given these criteria, which one should we use for which purpose for scale linking? This is a critical issue in evalua ting different methods. Third, as we discussed in last chapter, there are at least two main components in linking errors: error caused by parameter estimation a nd error produced by scale transformation. The problem is how to differentiate the estimation e rror from the linking error when scale linking is conducted? To answer this question, many st udies need to be conducted to evaluate the performance of different estimation programs for multidimensional IRT. In addition, some methods need to be developed to differenti ate the estimation error from linking error and evaluate the effects of estimation erro r on the performance of scale linking. Finally, the two approaches, multidimensional IRT approach and factor analysis approach, have different strengths and weaknesses in linking different scales. As Min and Kim (2003) found in their study that Lis method worked bett er for difficulty parameters and Oshima and

PAGE 137

137 colleagues method (e.g., test characteristic function method) worked better for the two discrimination parameters. Therefore, how to use the strengths of the two approaches to develop a combined method for multidimensional scale li nking is an important topic in the future research.

PAGE 138

138 APPENDIX ACCURACY AND STABILITY FOR DIFFERENT LINKING METHODS Figure A-1. Accuracy and stab ility for different linking m ethods (APP, n=20, N=500, G1)

PAGE 139

139 Figure A-2. Accuracy and stab ility for different linking methods (APP, n=20, N=500, G2)

PAGE 140

140 Figure A-3. Accuracy and stab ility for different linking methods (APP, n=20, N=500, G3)

PAGE 141

141 Figure A-4. Accuracy and stab ility for different linking methods (APP, n=20, N=500, G4)

PAGE 142

142 Figure A-5. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G1)

PAGE 143

143 Figure A-6. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G2)

PAGE 144

144 Figure A-7. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G3)

PAGE 145

145 Figure A-8. Accuracy and stab ility for different linking methods (APP, n=20, N=1000, G4)

PAGE 146

146 Figure A-9. Accuracy and stab ility for different linking methods (APP, n=20, N=2000, G1)

PAGE 147

147 Figure A-10. Accuracy and stability for diffe rent linking methods (APP, n=20, N=2000, G2)

PAGE 148

148 Figure A-11. Accuracy and stability for diffe rent linking methods (APP, n=20, N=2000, G3)

PAGE 149

149 Figure A-12. Accuracy and stability for diffe rent linking methods (APP, n=20, N=2000, G4)

PAGE 150

150 Figure A-13. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G1)

PAGE 151

151 Figure A-14. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G2)

PAGE 152

152 Figure A-15. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G3)

PAGE 153

153 Figure A-16. Accuracy and stability for diffe rent linking methods (APP, n=40, N=500, G4)

PAGE 154

154 Figure A-17. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G1)

PAGE 155

155 Figure A-18. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G2)

PAGE 156

156 Figure A-19. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G3)

PAGE 157

157 Figure A-20. Accuracy and stability for diffe rent linking methods (APP, n=40, N=1000, G4)

PAGE 158

158 Figure A-21. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G1)

PAGE 159

159 Figure A-22. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G2)

PAGE 160

160 Figure A-23. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G3)

PAGE 161

161 Figure A-24. Accuracy and stability for diffe rent linking methods (APP, n=40, N=2000, G4)

PAGE 162

162 Figure A-25. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G1)

PAGE 163

163 Figure A-26. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G2)

PAGE 164

164 Figure A-27. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G3)

PAGE 165

165 Figure A-28. Accuracy and stability for diffe rent linking methods (COM, n=20, N=500, G4)

PAGE 166

166 Figure A-29. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G1)

PAGE 167

167 Figure A-30. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G2)

PAGE 168

168 Figure A-31. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G3)

PAGE 169

169 Figure A-32. Accuracy and stability for diffe rent linking methods (COM, n=20, N=1000, G4)

PAGE 170

170 Figure A-33. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G1)

PAGE 171

171 Figure A-34. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G2)

PAGE 172

172 Figure A-35. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G3)

PAGE 173

173 Figure A-36. Accuracy and stability for diffe rent linking methods (COM, n=20, N=2000, G4)

PAGE 174

174 Figure A-37. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G1)

PAGE 175

175 Figure A-38. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G2)

PAGE 176

176 Figure A-39. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G3)

PAGE 177

177 Figure A-40. Accuracy and stability for diffe rent linking methods (COM, n=40, N=500, G4)

PAGE 178

178 Figure A-41. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G1)

PAGE 179

179 Figure A-42. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G2)

PAGE 180

180 Figure A-43. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G3)

PAGE 181

181 Figure A-44. Accuracy and stability for diffe rent linking methods (COM, n=40, N=1000, G4)

PAGE 182

182 Figure A-45. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G1)

PAGE 183

183 Figure A-46. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G2)

PAGE 184

184 Figure A-47. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G3)

PAGE 185

185 Figure A-48. Accuracy and stability for diffe rent linking methods (COM, n=40, N=2000, G4)

PAGE 186

186 LIST OF REFERENCES Ackerm an, T. A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255-278. Ackerman, T. A. (1996). Graphic representati on of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311-329. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573. Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological Measurement, 16, 87-96. Baker, F. B. (1993). Equating tests under the nominal response model. Applied Psychological Measurement, 17, 239-251. Bateley, R. M., & Boss, M. W. (1993). Th e effects on parameter estimation of correlateddimensions and a distribution-restrict ed trait in a multidimensional item response model. Applied Psychological Measurement, 17, 131-141. Bedescu, D. (1985). Efficiency of linear equating as a function of the length of the anchor test. Journal of Educational Measurement, 22, 13-20. Bock, R. D. (1972). Estimating item parameters and latent ability when the responses are scored in two or more nominal categories. Psychometrika, 37, 29-51. Bock, R. D., & Aitkin, M. (1981). Marginal maxi mum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459. Bock, R. D., Gibbons, R., & Muraki, E. (1988) Full-information item factor analysis. Applied Psychological Measurement, 12, 261-280. Bock, R. D., Gibbons, R., Schilling, S. G., Muraki, E., Wilson, D. T., & Wood, R. (1999). TESTFACT 3: Test scoring, items statistics, and full-info rmation item factor analysis. Chicago: Scientific Software International. Carlson, J. E. (1987). Multidimensional item response theory estimation: A computer program. Unpublished manuscript. Cattell, R. B. (1978). The scientific use of factor analysis. New York: Plenum. Cohen, A. S., & Kim, S. H. (1998). An investigation of linking methods under the graded response model. Applied Psychological Measurement, 22, 116-130. Comery, A. L., & Lee, H. B. (1992). A First course in factor analysis. Hillsdale, NJ: Erlbaum.Cook, L. L., Eignor, D. R. (1991). IRT equating methods. Educational Measurement: Issues and Practice, 10, 37-45.

PAGE 187

187 Cook, L. L., Eignor, D. R., & Taft, H. (1981, April). A comparative study of curriculum effects on the stability of IRT and conventional item parameter estimates (RR-85-38). Princeton, NJ: Educational Testing Service. Cook, L. L., & Petersen, N. S. (1987). Problem s related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225-244. Cureton, E. E., & DAgostino, R. B. (1983). Factor analysis: An applied approach. Hillsdale, NJ: Erlbaum. Davey, T. C., Oshima, T. C., & Lee, K. (1996). Linking multidimensional item calibrations. Applied Psychological Measurement, 20, 405-416. Divgi, D. R. (1985). A minimum chi-square met hods for developing a common metric in item response theory. Applied Psychological Measurement, 9, 413-415. Fitzpatrick, A. R., & Yen, W. M. (2001). The effects of test length and sample size on the reliability and equating of tests composed of constructed-response items. Applied Measurement in Education, 14, 31-57. Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor analysis.Multivariate Behavioral Research, 23, 267-269. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum. Gosz, J. K. & Walker, C. M. (2002, April). An empirical comparison of simple versus complex multidimensional item response data. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Gosz, J. K., Walker, C. M. (2002, April). An empirical comparison of multidimensional item response data using TESTFACT and NOHARM. Paper presented at the Annual Meeting of the National Council for Measurement in Educ ation (NCME), New Orleans, Louisiana. Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw Hill. Haebara, T. (1980). Equating l ogistic ability scales by a we ighted least squares method. Japanese Psychological Research, 22, 144-149. Hason, B. A., & Beguin, A. A. (2002). Obtaini ng a common scale for item response theory item parameters using separate versus concurre nt estimation in the common-item equating design. Applied Psychological Measurement, 26, 3-24. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff Publishing. Harwell, M., Stone, C. A., Hsu, T. C., & Kirisc i, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20, 101-125.

PAGE 188

188 Hirsch, T. M. (1989). Multidimensional equating. Journal of Educational Measurement, 26, 337-349. Hirsch, T. M. (1988). Multidimensional equating. Unpublished doctoral di ssertation, Florida State University, Tallahassee, FL. Kaskowitz, G. S., & De Ayala, R. J. (2001). The effect of error in item parameter estimates on the test response func tion method of linking. Applied Psychological Measurement, 25, 3952. Kelderman, H. (1997). Loglinear multidimensional item response models for polytomously scored items. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. Kim, H. (1994). New techniques for the dimensionality a ssessment of standardized test data. Unpublished doctoral dissertation. University of Illinois at Urbana -Champaign. UrbanaChampaign, IL. Kim, S. H., & Cohen, A. S. (1995). A mini mum chi-square method for equating tests under the graded response model. Applied Psychological Measurement, 19, 167-176. Kim, S. H., & Cohen, A. S. (2002). A comparis on of linking and concu rrent calibra tion under the graded response model. Applied Psychological Measurement, 26, 25-41. Klein, L. W., & Kolen, M. J. (1985, April). Effect of number of common items in common-item equating with nonrandom groups. Paper presented at the annual meeting of American Educational Research Association, Chicago. Knol, D. L., & Berger, M. P. F. (1991). Empiri cal comparison between factor analysis and multidimensional item response models. Multivariate Behavioral Research, 26, 457-477. Kolen, M. J. (2004a). Population invariance in equating and linking: Concept and history. Journal of Educational Measurement, 41, 3-14. Kolen, M. J. (2004b). Linking assessment: Concept and history. Applied Psychological Measurement, 28, 219-226. Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and Practices (2nd ed.). New York, NY: Springer. Lautenschlager, G. J., Flaherty, V. L., & Park D. G. (1994). IRT differential item functioning: An examination of ability scale purifications. Educational and Psychological Measurement, 54, 21-31. Lee, K., & Oshima, T. C. (1996). IPLINK: Multid imensional and unidimensional item parameter linking in item response theory. Applied Psychological Measurement, 20, 230.

PAGE 189

189 Li, Y. H. (1997). An evaluation of multidimen sional IRT equating methods by assessing the accuracy of transforming parameters onto a ta rget test metric (Doctoral dissertation, University of Marryland, 1997). Dissertation Abstract International, UMI Number 9816494. Li, Y. H., & Lissitz, R. W. (2000). An evalua tion of the accuracy of multidimensional IRT linking. Applied Psychological Measurement, 24, 115-138. Lord, F. M. (1980). Applications of item response theo ry to practical testing problems. Hillsdale, NJ: Erlbaum. Loyd, B. H., & Hoover, H. D. (1980). Ve rtical equating usin g the Rasch model. Journal of Educational Measurement, 17, 179-193. Marco, G. L. (1977). Item characteristic curve so lutions to three intrac table testing problems. Journal of Educational Measurement, 14, 139-160. Masters, G. N. (1982). A Rasch mo del for partial credit scoring. Psychometrika, 47, 149-174. McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117. McDonald, R. P. (1997). Normal-ogive multidimensional model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah, NJ: Erlbaum. McDonald, R. P. (2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99-114. McKinley, R. L., & Reckase, M. D. (1983). An extension of the two-parameter logistic model to the multidimensional latent space (Research Report, ONR 83-2). Iowa City, IA: American College Testing Program. Millsap, R. E. (2005). Four unr esolved problems in studies of factorial invariance. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary Psychometrics. Mahwah, NJ: Lawrence Erlbaum Associates. Min, K. S. (2003). The impact of scale dilation on the quality of the linking of multidimensional item response theory calibrations. Unpublished Dissertation, Michigan State University, East Lansing, MI. Mroch, A. A., & Bolt, D. M. (2006). A simulati on comparison of parametric and nonparametric dimensionality detection procedures. Applied Measurement in Education, 19, 67-91. Muraki, E. (1992). A generalized partial cred it model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.

PAGE 190

190 Muraki, E. (1999). POLYFACT version 2 [Computer program]. Princeton, NJ: Educational Testing Service. Muraki, E, & Carlson, J. E. (1995). Full-info rmation factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73-90. Muraki E., & Engelhard, G. (1985). Full-information item factor analysis : Applications of EAP scores. Applied Psychological Measurement, 9, 417-430. Muthen, L. K., & Muthen, B. (1998). MPLUS: The comprehensive modeling program for applied researcher: Users guide. Los Angeles: Muthen & Muthen. Oshima, T. C., Miller, M. D. (1992). Multidimensi onality and item bias in item response theory. Applied Psychological Measurement, 16, 237-248. Oshima, T. C., Davey, T. C., & Lee, K. (2000). Multidimensional linking: Four practical approaches. Journal of Educational Measurement, 37, 357-373. Park, D. G., & Lautenschlager, G. J. (1990). Im proving IRT item bias detection with iterative linking and ability sc ale purification. Applied Psychological Measurement, 14, 163-173. Peterson, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137156. Peterson, N. S., Kolen, M. J. & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L. Linn (Ed.), Educational measurement (pp. 221-262). New York: American Council on Education and Macmillan. Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412. Reckase, M. D. (1991). The discriminating power of items that measure more than onedimension. Applied Psychological Measurement, 15, 361-373. Reckase, M. D. (1997a). A linear logistic multidimensional model for dichotomous item response data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory. New York, NY: Springer. Reckase, M. D. (1997b). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25-36. Rechase, M. D., & Martineau, J. (2004, October). The vertical scaling of Science Achievement Tests. Paper commissioned by the Committ ee on Test Design for K-12 Science Achievement, Center for Educati on, National Research Council.

PAGE 191

191 Ree, M. J., & Jensen, H. E. (1983). Effects of samp le size on liner equating of item characteristic curve parameters. In D. J. Weiss (Ed.), New horizons in testing (pp. 135-146). New York: Academic. Roussos, L. A., Stout, W. F., & Marden, J. I. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 1-30. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17. Schonemann, P. H. (1966). A generalized soluti on of the orthogonal procrustes problem. Psychometrika, 31, 1-10. Schonemann, P. H., & Carroll, R. M. (1970). Fitt ing one matrix to another under choice of a central dilation and a rigid motion. Psychometrika, 35, 245-255. Simpson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computeriz ed Adaptive Testing Conference (pp. 82-98). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program. Stocking, M. L., & Lord, F. M. (1983). Developi ng a common metric in item response theory. Applied Psychological Measurement, 7, 201-210. Stone, C. A., & Yeh, C. C. (2006). Assessing the dimensionality and factor structure of multiplechoice exams. Educational and Psychological Measurement, 66, 193-214. Swaminathan, J., & Gifford, J. A. (1983). Estimati on of parameters in the three-parameter latent trait model. In D. J. Weiss (Ed.), New horizon in testing (pp. 13-30). New York: Academic. Tate, R. (2003). A comparison of selected empi rical methods for assessing the structure of responses to test items. Applied Psychological Measurement, 27, 159-203. Thissen, D. (1991). MULTILOG users guide: multiple, categorical item analysis and test scoring using item response theory [Computer program]. Chicago, IL: Scientific Software International. Thissen, D., & Wainer, H. (1982). Some standard errors in item response theory. Psychometrika, 47, 392-412. Thompson, T. D., Nering, M., & Davey, T. (1997, June). Multidimensional IRT scale linking without common items or common examinees. Paper presented at the annual meeting of the Psychometric Society, Gatlinburg, TN. Thurstone, L. L. (1947). Multiple factor analysis. Chicago: University of Chicago Press.

PAGE 192

192 Vandenberg, R. J.,& Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, pract ice, and recommendations for organizational research. Organizational Research Methods, 3, 4-70. Wang, M. (1985). Fitting a unidimensional model to mu ltidimensional item response data: The effects of latent space misspecification on the application of IRT. Unpublished manuscript. Wilson, D., Wood, R., & Gibbons, R. D. (1987). TESTFACT: Test scoring, item statistics, and item factor analysis. Mooresville, IN: Scientific Software. Wingersky, M. S., Cook, L. L., & Eignor, D. R. (1987). Specifying the characteristics of linking items used for item response theory item calibration (Research Report 87-24). Princeton, NJ: Educational Testing Service. Wingersky, M. S., & Lord, F. M. (1984). An i nvestigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347-364. Yen, W. M., & Fitzpatrick, A. R. (2006). It em response theory. In R. L. Brennan (Eds.), Educational Measurement (4th ed.). West Port, CT: Praeger. Zimowski, M. F., Muraki, E., Misle vy, R. J., & Bock, R. D. (1996). BILOG-MG: Multiplegroup IRT analysis and test main tenance for binary items [Computer program]. Chicago, IL: Scientific Software International.

PAGE 193

193 BIOGRAPHICAL SKETCH Youhua W ei was born in China. He received his B.Ed. in school education from Nanjing Normal University in 1992 and his M.Ed. in ps ychology from East China Normal University in 1995. From 1995 to 1997, he worked as a psychol ogical counselor at Sout heast University in Nanjing. From 1997 to 2001, he worked as a re search associate at Shanghai Academy of Educational Sciences. In 2004, he earned his M.S. in research, meas urement, and statistics from Texas A&M University in College Station. He began his doctoral study in research and evaluation methodology in the Department of E ducational Psychology at the University of Florida in fall 2004. He was awarde d the Ph.D. degree in August 2008.