UFDC Home  myUFDC Home  Help 



Full Text  
xml version 1.0 encoding UTF8 REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchemainstance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd INGEST IEID E20101212_AAAAAH INGEST_TIME 20101212T07:29:38Z PACKAGE UFE0022478_00001 AGREEMENT_INFO ACCOUNT UF PROJECT UFDC FILES FILE SIZE 474222 DFID F20101212_AAAIAE ORIGIN DEPOSITOR PATH min_y_Page_072.jp2 GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5 4a541fc58ed7dc632dddfe44c5e53d14 SHA1 c41d43a41a69cbe0f2510ca5b7a0321e32dc18e4 27568 F20101212_AAAHTH min_y_Page_052.jpg 87b7be159c1ba9924a7b0e459b00e767 d7f00d0715b83baea6d1f070bf4756061c2c8793 1523 F20101212_AAAHOK min_y_Page_041thm.jpg c916cdd985a949cb6a0dc67774775d6e efaf1a1e84e26f582e2607dbe61c9e94a13f135d 5305 F20101212_AAAHYE min_y_Page_047thm.jpg 4c506492ce16a490f9abdcc72157a61f 44846c4d5b1a114740b13d18c811df9af3af2f45 947346 F20101212_AAAIAF min_y_Page_075.jp2 f5f25e8895d96137a2c3e625b2a640ff 36de29549929a3cd20448709b8739e6bea35d4c4 29051 F20101212_AAAHTI min_y_Page_097.QC.jpg ccce068a6e113e2361fe8449fa41df8a b1372b69ae0c620432bbd6f5d314c0fd955437f9 83212 F20101212_AAAHOL min_y_Page_045.jpg 85062e3db1e795cca99078a346e58024 5ebd50c0dc4463b892d5b0d9eb8c130f2fadf833 5065 F20101212_AAAHYF min_y_Page_096thm.jpg cf9d05df7456533d652d17e126596d5b b70950eebef325dc617a437ae30e0f76853ff1ef 879819 F20101212_AAAIAG min_y_Page_079.jp2 aa0a7c0091d6e2792e128720b29ca3c5 209a4106e8dd765c902394b75249fb4c946d2ce2 26537 F20101212_AAAHTJ min_y_Page_050.QC.jpg 2e7cec44f0051a7576eb98691f568e48 08c15a48dc4aa7892a6e0e219ed33d318dd846b7 29609 F20101212_AAAHOM min_y_Page_098.QC.jpg fa4f294f38e2fd59978b025d1ea0dbb2 06f09fdbb6317e5ae8d1364ed03f1cf0c111ba07 25271604 F20101212_AAAHYG min_y_Page_039.tif de759d9e1eae23f5808697c42c6e1cc3 096e11af8d68ffe3861c30242964a9fd755b06f1 1051953 F20101212_AAAIAH min_y_Page_091.jp2 e053aebba2c18895deb48b1d5f9a83e5 8b225108eee7d77f9a4b76b0a2b49b438a42564e 90624 F20101212_AAAHTK min_y_Page_049.jpg 90c1061320f04a1720f321552873cfcd 3527de77a917c27a52191d20609ed9fc9298fd51 6717 F20101212_AAAHYH min_y_Page_049thm.jpg 175c2a12649fbff822f942bb665e2aba 28a7bb0b2fc05aac346b22588d806326e2aa52d3 1051968 F20101212_AAAIAI min_y_Page_102.jp2 6a12877bd230437c150c6c430547708e e6a4c565a919a331c2a18f9d6f7722122be4c40e 1051959 F20101212_AAAHTL min_y_Page_004.jp2 a79291a23a213ade2600862626a33196 99bb16e6c31b2bf7e3e0cd5fd4740dde5181c0ed 26165 F20101212_AAAHON min_y_Page_051.QC.jpg 8c76b3f6a91b515c449c77976f4a739d a75bf0dab1f5bab598a578720c863fdec42da674 6279 F20101212_AAAHYI min_y_Page_055thm.jpg b9ef73f9ae8d30e791f57d3224157370 151f69bf6faca03fa790d6a7e24c03b1c896b74b 633585 F20101212_AAAIAJ min_y_Page_103.jp2 a5b857f4730a5e91f08234e2696dd33f ebb8b02a020eee4659f74bc602c9ebc93076bd9a 6374 F20101212_AAAHTM min_y_Page_048thm.jpg eb21226b02e3e271c7e3ea3fb1026862 4234fa2a89f4d0b134fa46150bce3cac3384854e 29344 F20101212_AAAHOO min_y_Page_070.jpg f52dae2d4eebac6263942a5801d3b5fc 18f880745dd229a26ba6f434e444c90fc0a37f81 F20101212_AAAHYJ min_y_Page_024.tif c5f22f3535afeae9397b3d63bcc16fc0 61cfcd381d83350dc0ad80d4c9533527bd415400 F20101212_AAAIAK min_y_Page_001.tif bba80a51f8b5450bad161291d8ca9072 9031c60bd571a8d11dbfd28dc72d54b52492bd04 1051972 F20101212_AAAHTN min_y_Page_035.jp2 bc513ba352b825f9e344096f7ffbffa6 8627265d0be8f0b8d08f4086e15970a5351b0461 9450 F20101212_AAAHOP min_y_Page_010.QC.jpg 252369a8ad004afa384f06ccc043c35f a3ad3e2a25dda6fd1a170bde1a20889a12d2abc6 89140 F20101212_AAAHYK min_y_Page_034.jpg 27bcd37decae237d880fa2d9af619032 e40620d4ca499071065188c69bea3cf14efe3f51 F20101212_AAAIAL min_y_Page_003.tif 0e2187aae91daf1e4257203b4a9daa8f acbd87df028a52cf2842afb87f70594fe5bea3de F20101212_AAAHTO min_y_Page_030.tif 0370a8839f759faff84ada55c80bbbc6 e5bfc893e5fa6f190858bf9bb6e11791d54dde1e 16601 F20101212_AAAHOQ min_y_Page_038.QC.jpg 916fa72409fb251ec6265a8e63120f11 79ff2284a69118a94b811f1b77e4a292eb55d40b 88807 F20101212_AAAHYL min_y_Page_065.jpg 642019afe1f9e56da99b0cb3504089c4 ad542c5b9b1f466b1b292d91aa409fdf325de21d F20101212_AAAIAM min_y_Page_005.tif d9fb4632cb1fb44af6656cdce1ddc48d c9e97a6e0e8386dd581a1b0fb4472b27eaa11fa2 85834 F20101212_AAAHTP min_y_Page_004.jpg 26abffecc9143ef84f67208003dc19fc 406486d73d4b17d2f1b2adc34b711af68f046b27 77396 F20101212_AAAHOR min_y_Page_018.jpg 6927e6cdef3779aa9ee0f3989096ca2e 4a16114ce42082ae4ae69dfcf67ce4f46ea2a3d0 84115 F20101212_AAAHYM min_y_Page_061.jpg fb9470179838bd847b60c7766b236e41 65474bd7fb75e87595919d4a6d09c3a89afe3e17 F20101212_AAAIAN min_y_Page_006.tif 8dcfd9c2daac3896d7e0812ff55d1f17 6e45e3e66b6ae7a2e48d7764870fb8dd1ef2c61a F20101212_AAAHTQ min_y_Page_004.tif 80e68ab1e907274d072aa3470d5c8f4e 80003d31304e43756686ab07c94f3b0b8f93917e F20101212_AAAHOS min_y_Page_092.tif f5978b7fd7893ab5fb29c1eef4297fb6 60c58fa5678dba5b449286ef790890af2a7244f8 111378 F20101212_AAAHYN UFE0022478_00001.xml FULL 083ddd2699dfe7f058fa574e87d66e29 f08ea3ca55a49034ea7d0aea46baa5e2e4cee4f8 F20101212_AAAIAO min_y_Page_007.tif 0e3a6d235955de9a58f456eb6194125c 821ee2b64e7977d72092f278f2b7cf1b8b80846e F20101212_AAAHTR min_y_Page_101.tif 366cbd7ee01176064f6b047a886a35f4 1530eb272031bfc5d41a429376c6dcfbe67e6e37 1051917 F20101212_AAAHOT min_y_Page_058.jp2 e01d844e5c1f5a72287fb3c81d4acc40 16b2cb66edde24f643ed6beffde14eba48f1af82 F20101212_AAAIAP min_y_Page_012.tif 4356b881655983428ad054518fddd79d d7e5b67ace25c7fbd81ef1c56aa49ebb85720c36 1051926 F20101212_AAAHOU min_y_Page_098.jp2 55416595dfd048322cb766df84728125 a4a63379ad5a2cd1617ab19f2745cc2b1a9fa40f F20101212_AAAIAQ min_y_Page_013.tif 5c0ec1a99ff8c796c3110be1b8e05638 55342e318ce65bae3334eae7effce882e793d7bf 36833 F20101212_AAAHTS min_y_Page_007.jpg 064d79fbf7e954d0f2415af82c558319 eb26aa6a6a488944c335689f4bf3847a4d662337 6177 F20101212_AAAHOV min_y_Page_020thm.jpg aa68c89a833529be59516b3e33e1d137 caa429555f05558088465a80289f1f753835f1af 28760 F20101212_AAAHYQ min_y_Page_001.jpg 8e8f084865fcb5176bae1909384f3eaa a71ac896a2a9602a1957bec01b291a214460b001 F20101212_AAAHJY min_y_Page_099.tif 7a81256b23199ffe071a47e0d9f292dc 5d741ed093350086414ed6ee32a4b222cf1f3886 F20101212_AAAIAR min_y_Page_017.tif 0927f63b4694a3ec52007e837613605d ea97d2f8f5c3c4aa77f5b895657c493ad586ce1b 6512 F20101212_AAAHTT min_y_Page_004thm.jpg c61f35977dcd4861c40d3f61ab1984c5 8b0a1e8ed2dc903987d52a59b5ac4e00a2583006 6191 F20101212_AAAHOW min_y_Page_044.QC.jpg ee91fa1581420c75ad9efd2f29c59863 4205e48396d669f6df37e41bbc17cc35ca2d88f8 104579 F20101212_AAAHYR min_y_Page_008.jpg 0fc0ecb694ef810837f552f9a7498ecf d295ea8ec2d65fd562f73bee8ac4c0e3f1501651 1051971 F20101212_AAAHJZ min_y_Page_100.jp2 1792f766953359f2d3c84e2c6f9eead9 6b29fec2d3586c8640e8589465548888dc3a1284 F20101212_AAAIAS min_y_Page_018.tif 847cc92786c1401028c70c4617ffd617 6a18716670839a1f25ca4a2c424f9c0f78ba96ea 2154 F20101212_AAAHTU min_y_Page_052thm.jpg 95aa0407abc3984c8035c1d6946435c0 8b7fb2d5c0445a22c574cef88f24c916dd5fd2fa 6663 F20101212_AAAHOX min_y_Page_063thm.jpg 3f9d33214e926399894a3affb29e152e c5b3a5f1900adb67da43069e48d9639f9ef2c9cf 45618 F20101212_AAAHYS min_y_Page_011.jpg 565b46f92680719ecc2cc6e976b11bde 608007dcefa47a76fee41820b5a4445fd9d4516a F20101212_AAAIAT min_y_Page_020.tif ae906b17ea1d887cc1d034b807cf5d52 8d150e3435f0685235fa98bfacc8ff90a9b6aff2 90173 F20101212_AAAHTV min_y_Page_088.jpg 6045b721df209dcc7a2d12a1e5452c9d 93fee5231c97fc8d8aa8bc750d807600127adf39 1051986 F20101212_AAAHMA min_y_Page_008.jp2 0784a48021bd2de944becb36f6de2766 34b1cb721ef21a77a2708aae52a713593a79682c 45311 F20101212_AAAHOY min_y_Page_073.jpg 057a3d18d0fd195b5141d8b3ca3ea8c6 2e39e6f4ed3acc9ac0c7368d45d1fe232b799202 63341 F20101212_AAAHYT min_y_Page_013.jpg ed2f4c62dabb3c58a4c8622175403ba8 f34e9fc679723da0a4619a2bfaf8c9c5b4b43601 F20101212_AAAIAU min_y_Page_023.tif c8ba67576c02298861aeed042c5379a6 0a716072aadbdda5c6cef31966c6e88c884bc985 6307 F20101212_AAAHTW min_y_Page_022thm.jpg 5a818742b97b8ac56cc2d9a950d86fe9 0a955319c35f4840ace1a79e9d6d9a9bc610672f F20101212_AAAHMB min_y_Page_015.tif ac6a936b380bda838857cc921e300add aeb63b7f3578de2b6e9f0aff589f968411e4b807 F20101212_AAAHOZ min_y_Page_072.tif a5a93a4aeb167907e5641a619ee052d8 961312dbee164df2e67d602cbcce3d830ac67b33 73980 F20101212_AAAHYU min_y_Page_016.jpg 6cc479dc5ca9f66626ec514fbb7b7d53 8d53ec79198443b5d795a423d0a62dde213a6bb5 F20101212_AAAIAV min_y_Page_025.tif 2c668079e0401539c46052a1ef5fdc70 bae6a32f008e5a541a719e69f3d675880c61de9c 25780 F20101212_AAAHTX min_y_Page_064.QC.jpg c5dee1538f677803a0385a0018f186b8 5566ddf4dab6fa580bc1a971f69790c764a1f181 42983 F20101212_AAAHMC min_y_Page_077.jpg c0135d3c2870a363d953ceac57deaba7 0bc72cbc50bfcd2473ed1f087840f9a3f72f0b72 78777 F20101212_AAAHYV min_y_Page_019.jpg 27299ff974a561e6a205588690eb19a5 40e377cfce32d49168c0b3625800655ea5d7cae5 F20101212_AAAIAW min_y_Page_029.tif 4602079e8ba26598e7baa243bebee0d3 8c3299d8b8ef2f07967f1d165f7a40c1c3e01367 6920 F20101212_AAAHTY min_y_Page_098thm.jpg e408cf78e41055c8849eb68b6030a600 284b85a723b97ac00942f1efbfe990f6769def9e 44378 F20101212_AAAHMD min_y_Page_028.jpg 13a263b20f038a69d044ccd0eaaeaf6d 124f4a41cdaa1ef2406d1ac0d86ca3d4829828dd 88932 F20101212_AAAHYW min_y_Page_029.jpg 1af68cdf5db6c013936789ebb7b9064c 82e3ad963c0c14815abefc9bbdb7251e547686b6 80494 F20101212_AAAHRA min_y_Page_021.jpg 5eb3a24ced535cb1ffab0916cb179d4a eaaa2ea73e025285a5d75f8469efd903336753f4 F20101212_AAAIAX min_y_Page_033.tif d1cbcf5b22537d591c5ec0971aff1554 23b3dc202d8e612e0137f551be81d064587b66fa 6865 F20101212_AAAHTZ min_y_Page_091thm.jpg d15bdffbe75cd42ab7a7988867414dd0 587e492fa7642fb2ac849947fc5b70cea5ad7efb 6141 F20101212_AAAHME min_y_Page_056thm.jpg 691c45356f855a8c4a0a6e39cf9b2028 fcaccb6fb136366f3a7fa8c40a78d0bb2495ec7d 91278 F20101212_AAAHRB min_y_Page_063.jpg ddc7f5cbcaad0b3b631d396575add849 dcab5b6246d79eaa82f4a1b7247aac1d92e71df2 F20101212_AAAIAY min_y_Page_042.tif 70528df813608b4c6643b1f0e135ae14 06154d3333a34c9dca080c4d976350b444499274 273468 F20101212_AAAHMF min_y_Page_071.jp2 b2aaf7625b3ecfcec9d5ef0cd67097ae f1d1d3281ec69c8bf7fda06b11558c15e56a8f82 73140 F20101212_AAAHYX min_y_Page_030.jpg 1da783003ab2be966db844df40dd9f54 176210a7b95ad0b15d99938ae441e8e0ec14301e F20101212_AAAHRC min_y_Page_014.tif de8c7294d2c43062645c8a56fcd90c2a 1cbef98539bdc6584f59c79969e8445be0320cca F20101212_AAAIAZ min_y_Page_044.tif 4e822d8d0001ef85876c26cbdecc9ea5 cecb52a72a7a1a090fb1d040c9a3d2393ad2c8c7 55174 F20101212_AAAHMG min_y_Page_081.jpg 99155d74de7cc7d9c5d0b6156625841c bb1797be881688d937da698e481ba9832110bff3 75720 F20101212_AAAHWA min_y_Page_075.jpg c7457832cffcc06ef5e34b0dd6f4ee97 cb636a43f28e44acd3b8f15413c88944beae0ac2 90062 F20101212_AAAHYY min_y_Page_035.jpg 4dd272b070d50bc48fd4af10f6e75d26 ef03c083276fe7fe63f51c0746fa533b828091e4 1051960 F20101212_AAAHRD min_y_Page_064.jp2 dc36e3af1d1dd691a8282109bdba46eb 762cfa3485e1f83b58772b5e6fbb1236fa97c976 26851 F20101212_AAAHMH min_y_Page_025.QC.jpg ba1d98fe80b89da6b83282e19bdcd945 64e8dda0c6d105e39cec27ca3c2afe5099a59366 44758 F20101212_AAAHWB min_y_Page_026.jpg 41ab91a3bdf449755a95f5c6fea5e8e5 63e3953dbf8a7673fa8a55360d0f5e61b920d371 84509 F20101212_AAAHYZ min_y_Page_037.jpg 0d5e762491b48cd1b975fc6876d72132 af34e7aae7cfda76fbf9025046bd397c8a2bfe57 99242 F20101212_AAAHRE min_y_Page_100.jpg 9184c3cb8e56d19209fd21d2aa4a7e16 43b2113d58a3f85616d17fd9e5f0b7c0e8bcd65b 27954 F20101212_AAAIDA min_y_Page_067.QC.jpg a6eb680041e867ca94824c9fb4a6f7ad c89095219f85773f9087051e3607aa61561c5dbf 6609 F20101212_AAAHMI min_y_Page_021thm.jpg acf16fb47eb5a0d975194c9e89741725 5ff5d2c8b7356fcd22fa04086c95db9c7749b1de 5380 F20101212_AAAHWC min_y_Page_027thm.jpg 6e2ff0fd34d02c5d54300d550d027807 44210e9c5904ea5a67e1477bf876ded1e0c4a1a5 1051955 F20101212_AAAHRF min_y_Page_021.jp2 125d7bdad9c4baa95cee7efdc13336d1 2ecc9b09440194baeecf6dced594780ca1f47087 6935 F20101212_AAAHMJ min_y_Page_036thm.jpg a8e5f361435a18abb32a1c1a904dddfa 5db3b9728a7ba8791b6b334bc226403619979d47 27030 F20101212_AAAHWD min_y_Page_004.QC.jpg 788cd3eeb4ec85ad95447b0e80fc823e 944376a11f7dc1af1062339e99560e91a650959f 27780 F20101212_AAAHRG min_y_Page_021.QC.jpg 9ee89840ce63bada062cbedb603963b7 bc8ed115dfac0820ef39da2c47a4af071753dd63 1951 F20101212_AAAIDB min_y_Page_071thm.jpg 8d6c5942b234cb04ff2444d5060de1a5 bb63aaf747b822ab8ed2c5053dfcb55b60fedccb 15410 F20101212_AAAHMK min_y_Page_103.QC.jpg 10f3d37e6cce1728a676708d7c58ab96 f051a80e536bea5d743a3c277cbbe7a89d41d0b9 F20101212_AAAHWE min_y_Page_028.tif 96d49cc26404adbb754a545426caa2ee 76ea3d92275ced2d1f119ca7a9e6329fc22586ed F20101212_AAAHRH min_y_Page_078.tif 670c8b6e875533b585c381536c90e004 82823c7ef3a47bd8fda3ae67a5dc4ff04f3b8fe5 7356 F20101212_AAAIDC min_y_Page_071.QC.jpg 02b5c44b8655d87409fb767b829be102 da6b95fd7f525415d054567ad53a6e8fea5ca453 F20101212_AAAHWF min_y_Page_071.tif ef9bdf88a9cb1a75d3b50ee7456cc356 28a11f367ca5435719738766464524e25eae083e 89401 F20101212_AAAHRI min_y_Page_066.jpg 65f1981af78d2273fd28c6a266fcba24 b65601f2b628adad48727782846fa9b02b5ebe12 13388 F20101212_AAAIDD min_y_Page_073.QC.jpg 69ac3a0c3f3d9164b35feb6c85113a01 b428c5944d79958fca5287f5a23a693047b73fe7 584803 F20101212_AAAHML min_y_Page_043.jp2 0820a062f734b6bc461b4385ea4e874d d9382b988109898d93f2653271c8378643573b79 1051929 F20101212_AAAHWG min_y_Page_012.jp2 c28e87894debc4e512742d733dbb0543 8a5e2a34a26c905424b0e38bdb725684e3c0b81b 519862 F20101212_AAAHRJ min_y_Page_077.jp2 10c7d9c802193c7a63fea20dd0a5d45c 6707055b2ae20e1cc2a52f12b21391dcceffd368 5880 F20101212_AAAIDE min_y_Page_074thm.jpg 7d58c71180bf65dd3ed67b6a9a1d976f 5a4b1d6e16ff465f60b20b6648163362a3862041 F20101212_AAAHMM min_y_Page_011.tif ce8d3ab682f25a184f0838d663976cca 846c7281a0615222a46e19391a8027ccf89020a2 1051983 F20101212_AAAHWH min_y_Page_055.jp2 f6dd6794b35bb2dacc332a3d7e1e21a1 2b9a3dd2c84505dc8a8831c25d61250f9addb140 1051976 F20101212_AAAHRK min_y_Page_036.jp2 0f46562fccc7adeca6007ba8bdecc1d7 695ebe6adaeb32acca9f19231053c1eb4144dc7b 23458 F20101212_AAAIDF min_y_Page_075.QC.jpg 1de07845baea67004cac727863dc6d0a 968c7e850ed77ab0f036792529c687f7c6e69e76 77140 F20101212_AAAHMN min_y_Page_057.jpg f3d3d8e81cf0358773b495809edc04a5 8d505cc0a664a08049421949e727969181e4e8a4 95425 F20101212_AAAHWI min_y_Page_033.jpg 6f623f47a01bdb5ac8db9c691519251e e627ac1eb7ce1e8417da48d8e2af810d4f2cdc90 84913 F20101212_AAAHRL min_y_Page_085.jpg 6417460a6bdca96476d31514df3dcdd5 c5ac7d681c8ed7abf277e2a1ed47265734e348b1 15123 F20101212_AAAIDG min_y_Page_081.QC.jpg 50b0f05c78f4e81b091d3e8dcbb55feb 0bc3fca98f74deabd40634b40d54833801177165 F20101212_AAAHMO min_y_Page_019.tif 7bcc33fcacea311c39c8264cc8d47637 b19192d4b57ce0358828b7ec029fdcb1ce13627c 6726 F20101212_AAAHWJ min_y_Page_100thm.jpg aecfd79d7465e1d55a69c6cf51206868 4d835b26201c15b20dfb22cdb4f3e48d9acf1961 839836 F20101212_AAAHRM min_y_Page_010.jp2 31f68d6e1592c819fa368d2e713c9b25 666ba6d582f544dea5741fcfa9c3f9cc3b5280b8 18807 F20101212_AAAIDH min_y_Page_082.QC.jpg 476b4efb02f133423972dd30d5b0f30d e8a60af3c948097a86a1766d323bb05f485b5d1d 808806 F20101212_AAAHMP min_y_Page_039.jp2 14866249526010064611b9e3a5001cd7 399e35f314b83a45af627f0c56a7aab2b76497a7 11955 F20101212_AAAHWK min_y_Page_069.QC.jpg 811900dede0615a98274dfead40e1c0e 7dbb7021365d8e604e285d19e2951c267acd2458 916998 F20101212_AAAHRN min_y_Page_027.jp2 11e3dd89ca3ada145b5de4edf3426b38 48a487cde30e9c53ae2601a7dc2d0b3f146a6c3a 4509 F20101212_AAAIDI min_y_Page_084thm.jpg d2a99d63ae2fc5af6cfde9c488f2eecf 929bbcd8a4c22057d897d3a26c0223ea6cb75eb7 1051975 F20101212_AAAHMQ min_y_Page_054.jp2 0744447ce54c9b8e5bdeee7ff68b943a 8abca94cb036c87a037790feb1031dfbc591e54a 27496 F20101212_AAAHWL min_y_Page_014.QC.jpg c1e010ae1979cdf51b92e42169c638fa 6cea71063077b27da70e8a2dad4febc1bbb02603 78141 F20101212_AAAHRO min_y_Page_020.jpg dc60cd223f46e6128a3da4825c76d37d 36c2c8d89af9cea2dc22c417d7efa2dcabfaf8d2 18282 F20101212_AAAIDJ min_y_Page_084.QC.jpg 2a353bdd92cda13a10be9c57f878b51e 23e555cfce4be84946b782ea093b76a441f192c2 F20101212_AAAHWM min_y_Page_070.tif 7bc577d645690645cd5baefbdef606cd d2d3bd1cfa80d59e5ca2b5e2aa66d4c9edf51981 F20101212_AAAHRP min_y_Page_062.tif a7f690f8b2670a9bc202bb77262e00a6 c644233b0f60fb59c5eb55470987db872f14aa25 71739 F20101212_AAAHMR min_y_Page_059.jpg 8eecd62377ac11ee48150d9728120445 9f31775f267793adcb16e3acc0dd50cd48c4f171 6970 F20101212_AAAIDK min_y_Page_086thm.jpg 4b57af584c3431f5eff448763b05de76 867ef46740c9633b5f6432ebe292496f36acec1e 1051982 F20101212_AAAHWN min_y_Page_014.jp2 dc1948cc7080693069e3ec4481848481 578050f484a8b68c70e2de67666775b138c2a717 8211 F20101212_AAAHMS min_y_Page_052.QC.jpg e065425ae01ca51bc9dfb2956e348205 5ee7f17fd3eae32af8177a8e6bcc40951c4772d4 28297 F20101212_AAAIDL min_y_Page_086.QC.jpg 9523c2fa106c0f0385ceae601d1fd383 640382ba2de5e811fa829a3833382172f98f3457 6907 F20101212_AAAHWO min_y_Page_054thm.jpg 3f3cb69ab24ec02ba54364d44eafaf54 6d8c4a2f1561f78e4186cf0f90aeea758bc091fd 25348 F20101212_AAAHRQ min_y_Page_012.QC.jpg aedc25033fc9e73d3d743f950641d6f4 83fbc6244b92359a406e0354364c888e710f155b 2284 F20101212_AAAHMT min_y_Page_007thm.jpg 86baab40ccf12299eeccf7ad348aab64 6fd8097af7eb9bc0a328776cb1201c2bfea046c1 26330 F20101212_AAAIDM min_y_Page_087.QC.jpg d2fb66eb3fe1b02265de8fe4a74c2e3a f7a260b546417f9b8c5a692c306c15f1ba19f526 28276 F20101212_AAAHWP min_y_Page_066.QC.jpg b4997f09211e61a17f5c9850fcb0df64 38b732faf6973ee936563dab5f88bb53539600a5 F20101212_AAAHRR min_y_Page_040.tif 4d2747a0d4492959ed7de86e7235df1e bdc7efff07bd4b10c02442841305e4cad5ffaf8e 27528 F20101212_AAAHMU min_y_Page_063.QC.jpg e771ffc89156672590281ed69990dda1 549ad957ac363dd4e7f80da363aab7cb167eb919 5713 F20101212_AAAIDN min_y_Page_090thm.jpg 6d013d169a304741333fdf472d553a04 3a8d54b2ece0c149b3ff339a5023c8a3572db3e9 21913 F20101212_AAAHWQ min_y_Page_042.QC.jpg 359d996a0a0a1efe29dfca177fbada03 f0dda959af0b53f85c3228606cabd52195dd7204 9392 F20101212_AAAHRS min_y_Page_007.QC.jpg 0bf20943861a98054f531d70c260f9c7 c772473e6b96089bf3976c8b86a7005e931ae97e 77271 F20101212_AAAHMV min_y_Page_056.jpg 2dcf4a8cc02da8719806e735e2eef496 4e1fe194c986867292fc36fa86f160b828989367 4886 F20101212_AAAIDO min_y_Page_094thm.jpg 6ea4daee0c038807bd3483fea0d5207f afa95c33c4fdf3be5f24c2a10c64c587274b1c18 F20101212_AAAHWR min_y_Page_058.tif 138f9139af7ed580bdae5de3805acbab 2a2336cced3bbec518ff0da6f8e9581c540f9c23 F20101212_AAAHRT min_y_Page_021.tif f4a652256c0cf35247768f8e594dc224 591c293ef7f56a51e141a38842a9a8cec2720144 90875 F20101212_AAAHMW min_y_Page_024.jpg 3c4e6fafe677debd63788fc3f5654b64 1c18277fffbfd3baca828a8b15c6f348cff5bfb5 22099 F20101212_AAAIDP min_y_Page_094.QC.jpg 881e018cdccd3fa05e8647f2270babe0 9b920069c593c7c3e1b1139a1d38690c80047b04 6654 F20101212_AAAHWS min_y_Page_034thm.jpg bb0b024d7e505ee84b1703818d31ff80 6fbbc75d988684769f53b1cacfc2c2235d794e2d F20101212_AAAHRU min_y_Page_090.tif b03f5fc72ce3f32c072752a579e40e69 2022eeb5895b8fd899b3f5a910df475ebf1894e2 F20101212_AAAHMX min_y_Page_080.tif 7d691d80c494cff80ae6a19a28984faf 7c24864179d2531f0227c45cbe44728fc3bd48eb 4918 F20101212_AAAIDQ min_y_Page_095thm.jpg 2143a28597bbbaa3190d91af478941cd 19d4751ec9d1eb013f45e9fc197c9c5226ff972a 471 F20101212_AAAHWT min_y_Page_002thm.jpg 028229c082fa5d3fe8e2880e3f5d67e6 63c9d3366280eb9f2330e2eca6eb3c842bf3d94f 6503 F20101212_AAAHRV min_y_Page_014thm.jpg 71a2818f76fe3f90ededc79682e373a8 5bb9c36ecb7cb397345c4cc53d74b07d77c73c9c 88107 F20101212_AAAHKA min_y_Page_050.jpg 90f0fad6925da2221813e25396f92b7a a3f2ecb4cccff49cdf52613b2d44ae725b0f9446 59540 F20101212_AAAHMY min_y_Page_003.jp2 3eda6967aed19687bcace977602fb114 c52ed97cd8f318eb2d65d69341cef39d38ccfc30 22336 F20101212_AAAIDR min_y_Page_095.QC.jpg 2ea01be36738e8f46a8e6a1604e5f5ef f25646a44062126b245ed415e5f0049237701670 6817 F20101212_AAAHWU min_y_Page_068thm.jpg 8ad9659395e72f30c9dbb8a5930f7b41 de9c980fbf23032fb5d6a8189ec8fe7b06779421 1051954 F20101212_AAAHRW min_y_Page_045.jp2 376bcaf3c17732d5cc5aae8855242b77 d7a0f85a6e948ab5800801b2a009142ce7682d23 1051950 F20101212_AAAHKB min_y_Page_049.jp2 896a200d2d02688a442022daa77058c0 52fe3b43be3912c316597cb6d3215d03b31bdb26 6479 F20101212_AAAHMZ min_y_Page_015thm.jpg 3081967ffdb0af28db528f684e63c873 311dc65fafbcf850cd201a0aa232df59b30ce3e2 6850 F20101212_AAAIDS min_y_Page_097thm.jpg 4f4763cb4a1ccdd29b0bf5af5090a17a bc97de717ff3d7419b6fa87789d953c11ac7b92b 48078 F20101212_AAAHRX min_y_Page_040.jpg efe4b29b47e700708315bc480f45e717 58ffbfd7bee123df4a4d46bcaebc234aa5def60d 1051924 F20101212_AAAHKC min_y_Page_093.jp2 9362c0003d4f93d22a8bb95cf269d555 50427e2e7fc02fa761c69031614651c2ac5fa728 29118 F20101212_AAAIDT min_y_Page_099.QC.jpg 0077325f202977682c9f89a7b96905b6 591c777c8206cf23e7f768bd83e917405b8de643 90770 F20101212_AAAHWV min_y_Page_086.jpg 14fd1b996bff3d303f568badc1b7286f eee0b526aa0d220b1d17ece64ff23c3afd1bf060 20426 F20101212_AAAHPA min_y_Page_013.QC.jpg 84a021d40fac1d78353b70a2e5318133 4d9bed293155e2db501076c08cc268edd2374843 2054 F20101212_AAAHRY min_y_Page_001thm.jpg 27388ce62edc9ad0e19ff4143c1756fe 29772238f47dab6af95e69a0bd85eac79f33bf77 992245 F20101212_AAAHKD min_y_Page_016.jp2 0b0c501a1e24309ef4b5b620898c5318 63c83109baf5cce4d5f2fa0d401011d1a6ac3693 7072 F20101212_AAAIDU min_y_Page_101thm.jpg 70c8d250aea5931ffc57690c9e540050 97b31e685944d1ddaea27918cd1c1eaa4f61cfd8 22399 F20101212_AAAHWW min_y_Page_080.QC.jpg d8bfbb329168e5dd68eca49420a7f907 2265d5cad71cddbfbc3e474f479cdb5d2b2e4bb7 5845 F20101212_AAAHPB min_y_Page_059thm.jpg ae9530a0c50987cc7e8ed375e41a4f37 59b859a1ba1554beb1ee0240d3d3a37a434a4359 1051977 F20101212_AAAHRZ min_y_Page_022.jp2 4f2725aa957162056a0cc2dbdba2c05d 8e21a2f7f2f45bd186b5e3f230fcdfb8a209f5d0 1051888 F20101212_AAAHKE min_y_Page_095.jp2 678c82abf125ba0ca285ce976392c1bd 6c5d152a2f6e462959059512d74666237b19b134 30806 F20101212_AAAIDV min_y_Page_101.QC.jpg 83ddef1f1cbe11fa3511e9847acd7c83 f5a990b4e5c708b7ddcb84250e988abdf8d0757a F20101212_AAAHWX min_y_Page_035.tif fd91d4c46eb651e8aabe8cb50f1519ee 5f00d68abff1d4cea5c313df58e8d706e922b2f4 523116 F20101212_AAAHPC min_y.pdf 714e7e58b5fac77d4e98db2fb19b67a9 537f1f6983aeef6cadeeb090dde85b31a3401cea 554558 F20101212_AAAHKF min_y_Page_040.jp2 0114d442c59b908446be0d08ad72df81 85080df0c6aee54118bab2ef3afc163c097c5f28 3560 F20101212_AAAIDW min_y_Page_103thm.jpg c749f256acf89611ba6b70312a63cae1 5d1d04d651151f415aa235d16b37caa9a9ea0cc3 1051948 F20101212_AAAHWY min_y_Page_033.jp2 c81bfd4999cbbfce8c2956aef4e570f2 2933b56f1ffba9b15b4b0a3ddc68b7322533bcf6 1051966 F20101212_AAAHPD min_y_Page_032.jp2 5ad834144d91d5961ae6a80fccd0f33c edaa245fd564c8a7f73c0787391887ae67fd6999 696250 F20101212_AAAHKG min_y_Page_092.jp2 00ac12b79a8d729b538bb038f9edd876 18eb459938885024409cbbcb6f8cd3b5c5bfbd99 F20101212_AAAHUA min_y_Page_085.tif ddd36a96f819e35c343cb2aa9ecac576 3520fbaa6d0ea6aae2f3b0de1016b767c5f08be1 76763 F20101212_AAAIDX UFE0022478_00001.mets 0a04d339207b448ae3612c732523e76c d4e930dcca1bd4b88f195c6ef1aaf9cafdec5e42 F20101212_AAAHWZ min_y_Page_059.tif ec8da11484d45617a497b6288691fdf3 43c70f404919285eeaeede9a7abc4b382faf06e7 F20101212_AAAHPE min_y_Page_051.tif 77485a2f0792849c8525310b8530ddc1 c0b29c351a882045e70e509c838c76b78903f59e F20101212_AAAHKH min_y_Page_049.tif 59d5e733babf487aef447fc4b7e77123 2bd395acaa78edc91936af6bf98cbfe3a0e7357a 760901 F20101212_AAAHUB min_y_Page_084.jp2 b89bab631bec0defd4ba954c934a77cb f8df0fc8901d80fef841e79e3b7a1cc7402dc3f9 84642 F20101212_AAAHPF min_y_Page_015.jpg 909cca080a83ba9ca8ab8bb2c3e1a35b 90c6d0e78ede6de06cc5376f6733e4bae1173099 46260 F20101212_AAAHKI min_y_Page_043.jpg 581b1b27b826d235af77e466aa85642d 0b08713adfc8ac2c1ca40dd6c471127b28fa0b69 38147 F20101212_AAAHUC min_y_Page_069.jpg 4c90c23a286dc2d0b9a7fa718c2d0be0 5abfd544af44babfa37329eac43b275e57d4ad41 F20101212_AAAIBA min_y_Page_054.tif 23223e7cb64b2c7f63bdc37b2c37cf66 347e1414c22d62097ffe202f71d522bea2f995c1 F20101212_AAAHPG min_y_Page_037.tif 783956b25bd7df0ea88248b2ba9d2951 f0de3c64f65189e4aa9a0ff35929626e6324bc56 17537 F20101212_AAAHZA min_y_Page_041.jpg aa3ebee7e23a1d78acd8bd3d457e6a7b 72db3636470a027164c90644034a14204a9700c5 88630 F20101212_AAAHUD min_y_Page_067.jpg 7c0115b06fee470becea8b682a6f541d 56bc9b49b3dd2d4b1d78225325e967890e07a2ab F20101212_AAAIBB min_y_Page_055.tif f1d1e970bb52af7f4fada07086247b57 3061ddf7e7cc6d0afd34f6e7527594108fdaf554 6521 F20101212_AAAHPH min_y_Page_060thm.jpg e0d247f7674598ae8d0e5c6089a782bf 518657081c3917e15a5870dbb3dae690feba4c92 87678 F20101212_AAAHZB min_y_Page_042.jpg 142aeb76c98193f2bda1ebc5660d98b3 51fd3def73de6fd2b8aeff2cb05e91c3313dd4e6 28522 F20101212_AAAHKJ min_y_Page_068.QC.jpg 0f7b3968acb4130d9a54072168440be8 3e18484b6ce4e3596a62d670cb398004541d1cce 108012 F20101212_AAAHUE min_y_Page_097.jpg 364f2dfdd7782fb6f8771a77dcbf2695 a45b55e1b8e3265b5f0611ebf402d3354277bbde F20101212_AAAIBC min_y_Page_061.tif ec920222814a17064534a39911c80fc6 e10084f6b256015d46a4b90c96e0327493716781 F20101212_AAAHPI min_y_Page_056.tif 1ee0906690a445ca7367d3026b5aa506 25cddc10e885001d2d021f73bb84cea7c6e16022 72001 F20101212_AAAHZC min_y_Page_046.jpg 3a99830401c129dc8363d1bb20511529 89182b668694152e518ee2e8d49e5eb4f2124acc 5641 F20101212_AAAHKK min_y_Page_076thm.jpg 614547f1d8c6b9ea362cb423c11c72d5 6b26df150764621f2fc18860f901eae19cadea31 183879 F20101212_AAAHUF min_y_Page_041.jp2 8ef8bbdb11333da4411502dcda798e86 7a34383cc88b03db08bb8affe9399027cf22bce6 F20101212_AAAIBD min_y_Page_065.tif 578943d14ea4337379bc10d3b124cd65 89085764bb0e22238cefe5fdba7b2742a9b6c9ac 3154 F20101212_AAAHPJ min_y_Page_077thm.jpg 5180a3a0c6b1397d06749e7a0aa0670d b4de0cfdab8a2fdc785f6375123ae04ff930299f 66945 F20101212_AAAHZD min_y_Page_047.jpg 8771c009854cc7a3a3aee418cf960991 edf535c6c14c5ea87541514fd3e0a9d0092982fb 24661 F20101212_AAAHKL min_y_Page_074.QC.jpg 28046f75dfe9102fe70427699307af2b 14906aa6e9c70851966e47b6352fb64abd027279 6743 F20101212_AAAHUG min_y_Page_003.jpg 92f70ca39f949fb2a52ab88f18e26173 fc9412949704b34ad552ce8cf1c72e1cfdcb98ad F20101212_AAAIBE min_y_Page_067.tif 444fcae0a0c5a0df8292771035605b89 1ad9c3bd77db3f16f1a6e4eca2b6a2fb7cb7aa05 3958 F20101212_AAAHPK min_y_Page_092thm.jpg 716e0a5e93b5aed8cc3bc9c51866e48a f5504a86a1ad5f151195d295bdef68c44e188e69 98731 F20101212_AAAHZE min_y_Page_053.jpg 1d66822d4bed28806605a6620a864458 80d5437128b9120c457e23ce3f094ac4688af759 6545 F20101212_AAAHKM min_y_Page_029thm.jpg 920e227c45402234fcb23cc221cc5d2e ff37fd2519b19ae58d5e269ba62cb62c2b113d93 347900 F20101212_AAAHUH min_y_Page_070.jp2 3306c59481d9611e4188018ddb72bc93 7ec0765bb5bf50685bbff84bde5b1b6735cae717 F20101212_AAAIBF min_y_Page_073.tif 6ff512d8d3d285d6a6e026c23dc60e5b 69718dddb3c46350a801a92c70d5a101918639fd F20101212_AAAHPL min_y_Page_026.tif 12894c5a8a8c1716a0cd07a21c883472 47ebd4cf527945568075584dd2f9038fc3808967 89759 F20101212_AAAHZF min_y_Page_054.jpg a9dd623a24ebd4f39b71089e5df7fb8f 4be12f89e050bf17a4c0829cc67668956f741a32 3876 F20101212_AAAHKN min_y_Page_026thm.jpg cb4b37eab7d558078a84c6b74e87fd9c d37386bf4a0d65957962f7f1c84ebefbf9c91ebb F20101212_AAAHUI min_y_Page_076.tif ad70c612790c3d8dc4c0ba797cc6cf70 1cf84d8a02b73b4991f3f966e65df3f0f4a94184 F20101212_AAAIBG min_y_Page_079.tif 8c6605780aef6c1d102a27b29990b9e1 6050e485401ba1e876f141410246c69afa4ca0b7 F20101212_AAAHPM min_y_Page_097.tif 8e8afcdcf7c5c7d6023c17dca6984ede 6b527fbecc485896870fd23899b1984feb32bfe2 90686 F20101212_AAAHZG min_y_Page_068.jpg c1e5082b2349deb2fce6711a22387f27 50281159724c7e9e6d22a21dba16a13537bc9939 F20101212_AAAHKO min_y_Page_053.tif f2b6eb3bef14730317eca1c98ecbadd2 0f67957c2ad8dfec4e4ec26387b82c360b03eeb1 99262 F20101212_AAAHUJ min_y_Page_006.jpg e2088b81ebfa75fa08ad9b0ee94ef1e4 d4e9169b1251b10f91fffbcb9739d3c309115827 F20101212_AAAIBH min_y_Page_083.tif a3ff6bb21a47657c49729188dd72bc37 e45511a020df4c64d76f9046f0fdc5e3f08b843c 51936 F20101212_AAAHPN min_y_Page_038.jpg df7e0ce89f421ad895066f2902428269 05d3965261b588df5a1cb20a837c442087320ead 41994 F20101212_AAAHZH min_y_Page_072.jpg c9d9ee682c02bd0f279ac08a2de0f61f 7a02e5cd7526366b37b6cc06c8c08f61c6d422af 1051956 F20101212_AAAHKP min_y_Page_074.jp2 46f2ca4ee8ac64844a8563f78ae0c970 a8baf1522ac34772534fafe1ff3e4e221471137c 1051985 F20101212_AAAHUK min_y_Page_060.jp2 67d7214b188afd9192af2b16364a46ab 1fc4822c7cc50988d8d3cd516865dcb7951d9e9c F20101212_AAAIBI min_y_Page_086.tif 3c747238ee2f569367920cb0a1c6a0ab 5dbd2cb667fb3f62f45227bf239add28d96413ac 81776 F20101212_AAAHZI min_y_Page_074.jpg 83efdcd644b32511a01ef08b10a0e1bd 3b093d8147d9e25871272b894f034225d90b16ca F20101212_AAAHKQ min_y_Page_077.tif db147735dbee5bd18b69402b3f54f39e 59d831b7c47afefe844b27e34558bd4e2f3c0628 F20101212_AAAHUL min_y_Page_068.tif e6f55205c1f5718a7bf71749dc6dd9bd 043095330ed8bb93daea504de407add3381e8791 F20101212_AAAIBJ min_y_Page_088.tif 8f982a1d5a5f8644021abcd0c65e5ecc de1613c3396d76706043192572f67b780901c01f 71286 F20101212_AAAHZJ min_y_Page_079.jpg cd9c83d0e9459f90430afc107226086a a01c88db63debf0bc09d10ea42f9edffd4291c2e F20101212_AAAHKR min_y_Page_009.tif 7aafb7e5b6a25c3eb126759a962b9de6 d69ab4da09960cbcaeadbb754fd68aa787a7d3e3 1051984 F20101212_AAAHUM min_y_Page_062.jp2 9fd1351b16612ce02d221b9089194aac 17d83ae256d480f142ff7fcf78a1017af160132b 20356 F20101212_AAAHPO min_y_Page_079.QC.jpg 707075deaadf8754aa8716b27c896318 6d70c1679ab81b48a13d1766dc5549c857185ace F20101212_AAAIBK min_y_Page_091.tif e363c314fcba50c5b1dc286122b7582c 69ce021e9023c8df4d3e389aa261a072f0866ea2 77118 F20101212_AAAHZK min_y_Page_080.jpg 7d1b8e8b56feccbe6b9e6be1d557e274 5f57767e59953acf44d3e13bb139723c1f8db18e 4657 F20101212_AAAHKS min_y_Page_079thm.jpg d7f04a72e8de021bfd76909e0744f5f1 ae0b18b87d0ea354ad13b36d900940f2213ac7cf 27617 F20101212_AAAHUN min_y_Page_033.QC.jpg 1edb95ca0c3531642d8bad4420a250d3 d9363e04e210d3ad9e72fd540cb09f72bdfce8fd 111934 F20101212_AAAHPP min_y_Page_009.jpg a8f29fb34d3faa9604ace17ccbf9b43b 4709ea404e66c876668a856a02eb67a85d5fa2ee F20101212_AAAIBL min_y_Page_100.tif 02bf772f5d9f9fb68547afe965926519 988c39ac2cbb8e7cf06b957b5ebcc525ceaf7b30 82754 F20101212_AAAHZL min_y_Page_089.jpg 6c30c33f9d62367aa694f0688984a73e e9b1d46716879c2ed51c587731e773df71d3fe03 111340 F20101212_AAAHKT min_y_Page_099.jpg 2faffc4bca65a6839fd87644e0766d98 e28a5e5f998b94f64776ea45475d48412ca83ac2 F20101212_AAAHUO min_y_Page_010.tif 34ca6679ed926107599ac2d20d9172a8 978f2f787438b5da3bb6a9d2acaa87e01249e7bd F20101212_AAAHPQ min_y_Page_074.tif 795b25fc6c3be9eaa2a1e9e8fff8c6d5 0dff89958afd261e1a68e59f3fb5919e7eaa8c5a 609 F20101212_AAAIBM min_y_Page_003thm.jpg ec77923b5217a1b878416ed16b0438db 354a156a0d2930d52aacc95c676ff4964a67be78 52552 F20101212_AAAHZM min_y_Page_092.jpg 907b357abc11af39b2a011d0d9471c05 d8523dcdc9b452f1a825ac1c31e9dec93f600ed5 855180 F20101212_AAAHKU min_y_Page_013.jp2 920a664a7cc27658d9b40c80e4318d9a 1141002e258883cb6923de4b40c21acd4eb1cb16 F20101212_AAAHUP min_y_Page_087.tif f788a3ad9209dd79e8ffd66a5a57f6e3 da9f0c370bb4c931dea2de8e5ae37c9e775b8925 52970 F20101212_AAAHPR min_y_Page_031.jpg 30ecc3db0d4e32a9166f93237026971d 99a12082ccc1e013a9c0e901ecd9882408bef17e 1347 F20101212_AAAIBN min_y_Page_005thm.jpg 9d86be23e7564d088c85511f91aa6dac f8b3df695481184d1111e99632d811312e114615 99034 F20101212_AAAHZN min_y_Page_093.jpg 66f5c648cd3c2bd04af49c1678dd7f96 3b25cea9cdb90a85c6beb4fabbd41709cd4c5f4b 801108 F20101212_AAAHKV min_y_Page_082.jp2 122cad90e6e3ffe3603c1395f41b3c65 71bcc6d0863581d542bac37fd6f8ebb1f63f4460 86358 F20101212_AAAHUQ min_y_Page_014.jpg 73e457402c1fd1dd93d52e2ee1436663 f6fa3b0f2408cab165594edf172b402ebcb76058 F20101212_AAAHPS min_y_Page_096.tif f9dea0eab301a2f2d0a03882353e6dc3 6607fb70c697f05ca8e4f2f2c6a64df16c32e9c0 4729 F20101212_AAAIBO min_y_Page_005.QC.jpg e992b29d684ebbd16eaef6baad696ed1 593dc857a608f3296eede42eac99993c0e97cdf0 117029 F20101212_AAAHZO min_y_Page_101.jpg 830833f3b2321a435e3a9ab2f92a449d 3d1d7599011d1fc457692261145a5ce9840e025b 5413 F20101212_AAAHKW min_y_Page_075thm.jpg 3e743b56df1831d51497dd84baa636df de2f94f8ae2c6ad72d369d3ace30e5b17287c667 95253 F20101212_AAAHUR min_y_Page_102.jpg e767363a16872e4c8084b07ace5b4472 43a480c5bebf541f3276adc1622a4253a7388a60 F20101212_AAAHPT min_y_Page_060.tif fd2092089bf65617f05e91ced5307c53 79ad0252518411cc2260cd24199c55ca02775f66 5319 F20101212_AAAIBP min_y_Page_006thm.jpg 3de9b470777aacf5d721b61e77d30891 095e98bcee12aff61ce8dcce0ed5eba6c21c7cd3 28102 F20101212_AAAHZP min_y_Page_002.jp2 d129d77b3eec6242657f2d9386aaa688 8b11b6d6d50060fa30dddee3f80675c8edbbbdce 27978 F20101212_AAAHKX min_y_Page_049.QC.jpg 71406d2cd0e99ccf5f2711d16084677f 1d4976887c9576d619eedf94022a55a725508dc0 8350 F20101212_AAAHUS min_y_Page_001.QC.jpg aa5fb67bb324c14299dba6e39bde3d0b b7319a2e5567458f8641abf233191f7da75ce319 1805 F20101212_AAAHPU min_y_Page_044thm.jpg 08d47d5e71ce222d549856a5a9a77369 49f77b66d20eaaa0fc7147e1a1b7d2377603afcf 6713 F20101212_AAAIBQ min_y_Page_008thm.jpg af75ae1aa35cde13e3043089c9cff58e b69b6391e6d97c2c6951c3e16a6c5d2661a01e93 157922 F20101212_AAAHZQ min_y_Page_005.jp2 d6003f231c11dedca5d2d1cda9181f74 9cb37840a72392c8df26cc10ddaa2240b8ebe121 F20101212_AAAHKY min_y_Page_037thm.jpg 0fcbd2e29542b566e3d949e6f3972e44 6dc2572d8709d1b47d89378652072ae8ec99e120 4926 F20101212_AAAHPV min_y_Page_013thm.jpg 341a76bff9dfb46a8c48f3e70afa9190 6fd4353952d40daf24d2caf565b2ee5646570f64 7197 F20101212_AAAIBR min_y_Page_009thm.jpg 69916c0294ac6ca5aad46164d55d54b9 a409d1b3d491b3935638cb55ffc2943a89ab8815 1051981 F20101212_AAAHZR min_y_Page_009.jp2 3e4ff908d15f205212a4781af22d17cb dc7f90ba2d9d1b2911bc2f604bf87755e39aaa50 21812 F20101212_AAAHKZ min_y_Page_006.QC.jpg 4fa85a1c76009e991cc0971077030c28 b175bf932fd6ba1654177e7c1884262727251dce F20101212_AAAHUT min_y_Page_089.jp2 f945693121a2582c5a328151c0969f98 e5bd2cdcb2e65c94792546eb375c48d4d7de5830 F20101212_AAAHPW min_y_Page_064.tif 1abca6ef5b28c263ea5c8495bb457407 930af0975ec7e6e80d551d58bd290c7ffc51fb00 29987 F20101212_AAAIBS min_y_Page_009.QC.jpg 228c6b0b01fa1e8bc9a1c3c3998aaa43 339dbe04ff7ef0582666de56dd9132e2082f4865 975325 F20101212_AAAHZS min_y_Page_017.jp2 020621fae13ce2975b30ffc8c6b9a6d8 2b217225fd0e4f07fa1baa01d90972a2d603efc0 1734 F20101212_AAAHUU min_y_Page_003.QC.jpg 1a87174c875482e3099f0d22f434441d abf266ce26b4fc01d012dfef54f13058a402db6d 6318 F20101212_AAAHPX min_y_Page_061thm.jpg bd1bcc5ec8a800328754300cedbc87bf 52e562f98ba8978fcf62671c1264c48ddbf14204 6035 F20101212_AAAIBT min_y_Page_012thm.jpg acd53005de1b49e9ce0554f05787025e 69b00efa82e3498da545a1dc749cfc1e58f83cc6 1051908 F20101212_AAAHZT min_y_Page_024.jp2 f8099919526cf9df33bd3875fb749065 fa691d8d70e79940260f2160cbd5e1d219ce1867 F20101212_AAAHUV min_y_Page_094.tif 90ce9cd0c8713da9ee599ca7c6e8d3f9 65b01a711f2936c261aa35c52f8d3157f84711bc 5371 F20101212_AAAHNA min_y_Page_083thm.jpg 3c4fe9f8a02f938cde7c93ebad230ec2 3c171ff5d1d2d840b573b921f8c162cbf033b9e5 23984 F20101212_AAAHPY min_y_Page_071.jpg 888aff1090f42905b0711a55492c4f79 ed155b5527b78281a9829906315d67f5ec9c26f1 26989 F20101212_AAAIBU min_y_Page_015.QC.jpg a234a6d2d0c425045d04bdc01b176156 9f158c7c2be289bef991b452f44d6ec5eed2c6fa 1051946 F20101212_AAAHZU min_y_Page_029.jp2 df54b8cb346909e0de5a1fc183a07768 47ae3be969063d7054dd5bef15eddc7fd5925004 F20101212_AAAHUW min_y_Page_093.tif 5b3fb9afe1e4313e63287fa43ccafc8f f78265d72598c60c51f49634492811b1b79d32ad 6752 F20101212_AAAHNB min_y_Page_033thm.jpg b62f47f56fe12d54cc5012b07fc0177f 0d49fcbb8c09b2e9683ac651d844923858d79301 600588 F20101212_AAAHPZ min_y_Page_011.jp2 68163b61d6e682ec85a70faf07fbb9b9 00cb46205f4e5ac2cf494016971d2afedddd74af 24180 F20101212_AAAIBV min_y_Page_016.QC.jpg 02858a451e46258c3503da9f844913f4 540a7ff5786864491250dd2dced8cb599c0e758f 687602 F20101212_AAAHZV min_y_Page_038.jp2 09ac3547901f6785b04ec65977625d9e 3f935ba2e7a1aa3fa429369c48b9605532565808 F20101212_AAAHUX min_y_Page_027.tif 53e3e9af78053164caad24d3c54d738e 6449d112a9124d77cb38411eab1ab8a3ff5159f0 27670 F20101212_AAAHNC min_y_Page_008.QC.jpg 02e9ce0d5079b2b0fc20b335b6ee89f0 54492b978b3a3d1a647a382ad88075c6901efd62 6209 F20101212_AAAIBW min_y_Page_018thm.jpg 5737fc014bca585758eadc6d9baf4532 d2607905d0cdfbfe8b6e6690840f00c52f8e4d69 1051936 F20101212_AAAHZW min_y_Page_042.jp2 3c035a95af8c835d87a8cdb061536eaa 433bd82b602c267a42ace136dd94082d063fd74c 1051933 F20101212_AAAHSA min_y_Page_048.jp2 4415c307c884eca1c37a5ea421a99dc6 3a29ddaac6bff5ca3b0a802f5d5d69dcf888311e 989390 F20101212_AAAHUY min_y_Page_083.jp2 84fb8904571a707eafdd8c8906409ba9 4b97a97f5710b2ed8c695e0b1ad701d1ca99217d 4387 F20101212_AAAHND min_y_Page_039thm.jpg d10e202447ffa50deb68ed88d9dc5de8 44fabc2f89017e17bb6c101dda1f77b7b3004880 28608 F20101212_AAAIBX min_y_Page_024.QC.jpg 4b0b90c02817afe68cc2513f49ff7ff2 44e93027ff87a4d203be06403b66ad8e9367060b 1051979 F20101212_AAAHZX min_y_Page_050.jp2 c58cd3e53d247f8cc56a64fabe163e48 4d02dbb4b6dcae208b9f3dc05aa94a5baeec2366 3443 F20101212_AAAHSB min_y_Page_072thm.jpg dc6498a25f57692987fb089ab859b7af c1d296e717b03512c6ad04d196ffeef128a184a7 28334 F20101212_AAAHUZ min_y_Page_100.QC.jpg 894e923c7f15d16ccf9a23568defcc9e 0d301d329eca72716ec4f43c18a72a6850640692 26583 F20101212_AAAHNE min_y_Page_032.QC.jpg 7f0f08dc0a18e466d2995bf195e83d3d 4f23379462fb7e00176b826f51cfc81c19280780 6129 F20101212_AAAIBY min_y_Page_025thm.jpg 67ddc0aae4e6383c9db9330eb31ddbe3 42a5b17604347fbc3f3a478a62b749b75ee4c8c5 F20101212_AAAHSC min_y_Page_086.jp2 3d8e572c24a9001184ff263aa575ee58 7125961bea0a1921428497d93d4e7985942be57a F20101212_AAAHNF min_y_Page_022.tif a54ca3b7505003b9806b800de3ad7453 d1d65f641fe05f2df02a3f7f56418261df6857ad 22413 F20101212_AAAIBZ min_y_Page_027.QC.jpg 6804224a57661c825236d3d096786610 e4ba2398810f67acedf7aa05110676554ddbf058 F20101212_AAAHZY min_y_Page_053.jp2 df98ace6edef964b425ca8b922d439c0 abfa568d336d45ea07267a383e5aabf9574b4464 61000 F20101212_AAAHSD min_y_Page_082.jpg d25b34bd25638618de9407835659da14 7faabca0fe19cabbc31d8e30e7d86dd0d4b340fd 87172 F20101212_AAAHNG min_y_Page_095.jpg 5958a1a91ff3613f37b96c126529083d 96d52b2d031e910f90a8513c6c9a5a34a8e4c4d4 5912 F20101212_AAAHXA min_y_Page_016thm.jpg 7a2d544649edc7743e9fff82d83a1745 6f84941c807ee72da7adebef5d694742951f75dc 988388 F20101212_AAAHZZ min_y_Page_056.jp2 2f363d051a6a2e80cde49dd958d37d7a dfc70f2ed782d0a531933ebf03861272a51ab3d8 68721 F20101212_AAAHSE min_y_Page_027.jpg f3f1c04aa9f1c35d2c3022a6009ea725 0710c5316de0f6f5769b60b70a1ea9ab1ae14691 23536 F20101212_AAAHNH min_y_Page_056.QC.jpg e0c499153ffeb39eb68ed76a52fcaef6 27b30fbce8d788f56a248f9d6ba5276661ff20e5 13094 F20101212_AAAHXB min_y_Page_077.QC.jpg f0f285a0364aca4fdd591e794d2125be 782145c209b497e8275e88846df778133cb8be15 25501 F20101212_AAAHSF min_y_Page_093.QC.jpg ec6cadd72ad45d9362378c474c448075 6a339d629a3d78206b3d406cb1d18865b8c3fa69 F20101212_AAAHNI min_y_Page_075.tif 0b3d80bb73c5c415fec54bb2ce33ebad 54869d6c6dfe8ec4850c0d6d8dfdcce7a58d1d44 323931 F20101212_AAAHXC min_y_Page_001.jp2 5c26b16109801c4eecba6ccf03e70940 2de07f959ba82f860fc61fd339306dcf14d121e3 85953 F20101212_AAAHSG min_y_Page_032.jpg 7b07df7c99d3b585e1a0e50d150d5ab0 6b757334cb619d5f472c27341d009944bce20900 83631 F20101212_AAAHNJ min_y_Page_055.jpg 2f650a29f85422d980fbafeba3dd31d8 5b1c60b34f9c4884d014fbd0516d8aeaa630b3d0 2508 F20101212_AAAHXD min_y_Page_010thm.jpg ccf7e977605573302471bbe92a60f628 d4cad1ab226aad1acb934122c9502f67abe2d638 683703 F20101212_AAAHSH min_y_Page_031.jp2 cf64e53029cce3f875fd000dca6bd90c 431223453ef804b38b48b3095ed4646ca2ca6c7f 86427 F20101212_AAAHNK min_y_Page_094.jpg 417294dd920b72ccebc9f1d393075481 ce69c0a807e750144bb9e28911571e6efc8b91b0 85468 F20101212_AAAHXE min_y_Page_051.jpg c803cb07d71c9fb081de7b1a23689776 48708480c985454dd78ca3bb8aec8edab250ab5b F20101212_AAAHSI min_y_Page_094.jp2 a796307949b36b47816c8e3c6c0eafcf b9186ffd9d9fb595fe5fce939a99f4005c85ae52 3835 F20101212_AAAHNL min_y_Page_011thm.jpg ee23a7bdc685e9199e3682683411fc24 ab91ab6f143231404df7b9f29abfbd4ab7d3b3d6 25289 F20101212_AAAHXF min_y_Page_078.QC.jpg 860e573a6bfcb5d270e47787d217fb9e 8f1176fc9eec5a0cd953a8f444bd7975d5aa3a37 20278 F20101212_AAAHSJ min_y_Page_044.jpg 90d2fe4e6d5e973ae905421ccc996b31 98915b9851d1c09955c1b9a7cafdf2ebbfb12166 1051942 F20101212_AAAHXG min_y_Page_061.jp2 9d5b6f4a56b877b843d690b095e61d61 d946058844af83b7f52db6c26254344be3efac16 681675 F20101212_AAAHNM min_y_Page_081.jp2 025e33082bc2c689a022651c81418495 456fa7df073f8d6539365487a2556cd06beb28cd 84941 F20101212_AAAHXH min_y_Page_048.jpg 83e6af7bea46abed8335c2d1f59b98eb 4a7a161b3a873e8a7b551a41bd4bf73fb62ade71 892598 F20101212_AAAHSK min_y_Page_047.jp2 6ac68f5097d5fd6170331ea1e7dbeaa8 f03b1d98f1fa22177a50cce260325fed7535638a 85735 F20101212_AAAHNN min_y_Page_012.jpg 7855ae140c10055e308fc0a249b04285 7c3da59ebc799f6a0784155ce82bfe044b3736db 6458 F20101212_AAAHXI min_y_Page_019thm.jpg 9ae86c290ba7b73d3ee0231ab3e32a9b 2c48d733883967bc1937e8117d2e252316962649 25882 F20101212_AAAHSL min_y_Page_020.QC.jpg 1118f0623d6bd683058f9fc7c53051c1 bd38d28aa27d63b74a2cc5bc4ee6c272e5fb13c6 25813 F20101212_AAAHNO min_y_Page_055.QC.jpg faa7a8e8d60de7e7770733c7d53c4637 edc969d6bfb9ac92e5b71534e93f30a659801f74 F20101212_AAAHXJ min_y_Page_034.tif 346521bd2be408be3f90e33531c04ecf 53aedbdfdbc0416df5768cd9596e649cec1537a9 6691 F20101212_AAAHSM min_y_Page_062thm.jpg a063ddece2e360d6f0683a6a5ca8dc25 11dad8ebb77111a542e43268e36fc0f85b1b7889 F20101212_AAAHNP min_y_Page_082.tif 0011e46ff284987ed116c88ea81cca8b c2b1f2048a180ae1e81437be053992d20f9174e5 3507 F20101212_AAAHXK min_y_Page_073thm.jpg d8900ad81e7a458ab1ef70899058cb80 8be7c5bf6d291354edd6422961c54818052442f5 F20101212_AAAHSN min_y_Page_019.jp2 0959fb27ffeed1333b5ef92f653be47b 69efc8f121e42d21a6dcd014c541af39df5875f4 28240 F20101212_AAAHNQ min_y_Page_054.QC.jpg c4e1ddeecd601a53c30d706345514408 acfc522e9083ad1b802b1fc47f4dd5682e8deb4f F20101212_AAAHXL min_y_Page_069.tif abc5e1b8738ab14c28bd7b74856af1c5 f38f40c5ab7aaea5ea549d995b2663c22d079cbd 23814 F20101212_AAAHSO min_y_Page_076.QC.jpg 2cd1f2313e41b0660afa197a409e151e a375ddc2ed19ced27bef18932abffcc5e69cb977 F20101212_AAAHNR min_y_Page_015.jp2 6f46b24c04f363e8dfe905541ae8f334 a81cd067a5dc5791d4c2279e61b8d010907fe0c5 F20101212_AAAHXM min_y_Page_032.tif b033040a4d187e817a24b57a66d637c1 f8be1db0dcbe9a502865c21bc80962e0b8885347 6242 F20101212_AAAHSP min_y_Page_032thm.jpg 70e58b2e6bf447a63425702d367d5b9d b5a38195420b4111ecaff0283c037548e02c850f 85703 F20101212_AAAHNS min_y_Page_078.jpg f336969275b6c847b53f28f51cfe754c d516f7bb40007386da905c4a8f404a7c889e56c3 6728 F20101212_AAAHXN min_y_Page_024thm.jpg 5200e9ece5b9b86b7863a12a0a1fd95a b48d292a532366c14c6cc6d62312668ac033573a F20101212_AAAHSQ min_y_Page_103.tif 73088dd06c3487c63e93293c16e9ea5c ab25d77f0ad8462e2ccda03777bd1df795b6544b F20101212_AAAHNT min_y_Page_045.tif 51e622e3f4eb85f766efb6d39b9ec0ab 986960fbfcb9f4e4b88880ae32230bb3d8625f69 349908 F20101212_AAAHXO min_y_Page_052.jp2 f0879a91bb9bed79f2c21eb99850572f 67fd67f7369ea998b6af51fda0a6a4876a3a7267 1051973 F20101212_AAAHNU min_y_Page_097.jp2 82009f2a063c79576a448eee5b106e21 80c914040723dc7339118a40a8b7e615657be803 1051963 F20101212_AAAHXP min_y_Page_020.jp2 e0dd8f6173867f1ddcbee1bc9c7fef9b b6de46ec33a33cfa85036922c465ef3c5b01465d 23334 F20101212_AAAHSR min_y_Page_083.QC.jpg a9e61a251e577596b2b5ef77e46a0f67 d8afca6f1d6ffb40e0c68b580eb8c90e5ccd1509 F20101212_AAAHNV min_y_Page_081.tif 506ae492e6f2e8e994442bc0eb3fe0a9 8a3322178ac8ee82554ffb9e1dacb6f28a343af7 25443 F20101212_AAAHXQ min_y_Page_018.QC.jpg c5ce819493c69109c22e0b42a2b23b4e 8e7149c2a41cfc2a129c7d41f3defb92250ca5b6 87802 F20101212_AAAHSS min_y_Page_058.jpg c3ffe2566508f8cb9ca30f0095e71596 c043794bb2539eb35480d3967eaf24bf800781c2 F20101212_AAAHNW min_y_Page_102.tif 0613699279ac0151c472ff6c599916f3 54738eefc0bdb9e94d8449e67e8fe227e4727b96 991683 F20101212_AAAHXR min_y_Page_030.jp2 8e4b51904355df159230380eff69eca0 4511ccb3f2aa04933e6f04ca3cc7529db752344d 25150 F20101212_AAAHST min_y_Page_019.QC.jpg 0f524af3f1b2eba24cb86e6bfdff9d52 d2979340c1bc1fdd3673d57f7616bfae4caf279c 28238 F20101212_AAAHNX min_y_Page_091.QC.jpg edb620ff6cbcf6a149b0c63a537110e5 96d0ef611b34fc54046822f0844a96e797397870 F20101212_AAAHXS min_y_Page_063.tif 77a1366e2e31428e94d9065367feb834 c66fcde1cbbafd764d418dbc275c7e2f347d72c5 925326 F20101212_AAAHSU min_y_Page_046.jp2 90616efd3e4d9d1c0988b7ae6ddc59a4 88ee3475ac795bbcb507693f0bf00499980cfa8c 5963 F20101212_AAAHNY min_y_Page_102thm.jpg 2aa5b4d59b3fe758e0491be6176b8c4f a7bb364a41de204fb2eec5540a95378644921363 F20101212_AAAHXT min_y_Page_066.tif 8c24ebd4ec3413c7663c54ea0ae3818c a2f165a7b89eb37f7be73e10c821136d15d6f008 1028151 F20101212_AAAHSV min_y_Page_018.jp2 a995f86450b9777322c7324c6cebd90f cb51000c97329ae2094e82f3d2743a4c857b8814 22116 F20101212_AAAHLA min_y_Page_046.QC.jpg 367ffb21cc6a72604c28c90a27f30eb2 8f5301c4b260a1c2f17b42b05a9795dfaf5a1450 103345 F20101212_AAAHNZ min_y_Page_091.jpg 333a1e9c8ac084b94881d31bd0b6fec1 571466c56814611160dd426c8aaad607425956ad 85367 F20101212_AAAHXU min_y_Page_023.jpg b1dad6a884419077bb67242f317ee254 6477838d99f26974655caa8f37e4841d681024b9 F20101212_AAAHSW min_y_Page_095.tif b2f77d6e212d2e5d9d65a501f41bdaa1 dac793c3468e0d060933381e9dcb295c2e5bf802 1051914 F20101212_AAAHLB min_y_Page_087.jp2 b7a77694a9261d39869333595c30acdf 73c48a3ad1b87942f0f863ccf3e1b3e857d6989c 1051978 F20101212_AAAHXV min_y_Page_025.jp2 f94f6f653797db854e931ee7cf64d2a9 2b21069a5889a0ff1b731ad0205b6a162539f42c F20101212_AAAHSX min_y_Page_041.tif ecc630a9d10fc7ee8a9c6c73b3277f1c 079b54c8e86b3918c99300bfd9d3be4f6d4cd267 F20101212_AAAHLC min_y_Page_036.tif e102de590ef57dc6a5d40dec18eaee1b 925f995f3b4a418fbadd72f9c6782d4d11649415 F20101212_AAAHQA min_y_Page_090.jp2 60968cf2b874e1449adf7c9abde87532 8550f17b398770f00da7bf4ad304e5789651fc9c 24040 F20101212_AAAHSY min_y_Page_057.QC.jpg 036e32a08950b0c22f6c186e8b86bf7f 3be938626fa8aa4fe61cdc5ca55e17025cca1145 87376 F20101212_AAAHLD min_y_Page_062.jpg d69794db5747a45437c586bdeb5516fb 0bec84fd3d1efbbdb3e97255ca699b80d265d973 3709 F20101212_AAAHXW min_y_Page_002.jpg 888ca0c477c5e17e9165fc2b77f63547 5ae15b8c0b7ea221b5479dc97118cb19657a097c 1051898 F20101212_AAAHQB min_y_Page_096.jp2 2fce954de9637168092e879f237a1efd 51b0668a52dd70f55c53c1040fe154890f4ff212 1051938 F20101212_AAAHSZ min_y_Page_006.jp2 4e7536eaa895af81469169e9f0013ffd e7f411b9a5c5aee973b068d7396f09d635b5b36a F20101212_AAAHLE min_y_Page_002.tif ced2c964648b01aa5f545e0eafefff12 83454946cabc4abd4d459531c9c49682c4c94830 1070 F20101212_AAAHXX min_y_Page_002.QC.jpg 4bf909f2273f1fb564365c74e894ebb9 e9dc84c414c4f74023be3b34817dd0a0de55e0cc 33841 F20101212_AAAHQC min_y_Page_010.jpg 623c4c7bdad151fbbcaa569d0e6545ef 1ef50842f3ba0054efc400dc4002f8393564004e 1051944 F20101212_AAAHLF min_y_Page_078.jp2 d904b67a0fb55f40da2b37ef3714b353 5902fde25b54eb1b6c0b1c811c9bc8dcbd0cf4af F20101212_AAAHVA min_y_Page_051.jp2 560ddd7d37d6deab53d659727ac2dffd 4fbac5156e329d1a821ff427e00be73ab6cfda26 13026 F20101212_AAAHXY min_y_Page_072.QC.jpg f1fbd62315fb48e1137f0658630f88a7 847851edf5c105caa3e09aa39e7841bd260085aa 818633 F20101212_AAAHQD min_y_Page_007.jp2 11208fc4c0286277dcdd41af1449f527 7a3488b18ed2982b1b6746b441325d4f5889d9b9 5650 F20101212_AAAHLG min_y_Page_093thm.jpg 6509bcf6b4944e6c9d9bc5f57ffc5bb3 906d307fa566549d0117e34af1c3754dc0ab47ca 28034 F20101212_AAAHVB min_y_Page_036.QC.jpg 7e12e1b2e3bf9fde4330de74e32e4dcc 9f4e4685c1a1d2c5eb11127ff2b0b51c0cb244c5 5578 F20101212_AAAHXZ min_y_Page_046thm.jpg 2dd4c770119739baba926562b341239d f1d7860f7f7b147a234e2a8947d74dffe46ed9b9 79243 F20101212_AAAHQE min_y_Page_076.jpg db027f5d7035f16aa80cfccb8a0b1d38 1bef00f57786cf0eba0ac1acc875811cc0b7b899 14317 F20101212_AAAHLH min_y_Page_005.jpg 98998207fd03ef1d0758c0a20f5157bc 6ee29af23dbc2c97b130cc0579f820adcf1646e7 76510 F20101212_AAAHVC min_y_Page_090.jpg c19d6608315447ef09375245290e6cf9 8645a212a1174814f106744bcc2ee69719d564a1 15333 F20101212_AAAHQF min_y_Page_011.QC.jpg 5f7320b84d95529a6f4591d9190cff31 0412b6d752b2dfe4ab744ed4dd5baae7b8e318c7 1051951 F20101212_AAAHLI min_y_Page_063.jp2 450ec18253fe510d01bfea032f38af6d 13b997d59fe772335a093c2aa9bc47117a86c7bb 14903 F20101212_AAAICA min_y_Page_028.QC.jpg 040c2bd6cb9b51056c0f870c4268277b 1cfc083cd59dca08003d20510ad4c27f1f1ec2b9 85176 F20101212_AAAHVD min_y_Page_087.jpg 2bf17d55a672d89b28dfbc1b3e517d34 5d5cf9d64b4d861e7160d9e59da92c7724c5e488 26617 F20101212_AAAHQG min_y_Page_089.QC.jpg 5ef116464c8910664767b0aa78651b2b 32640ccf06bd05b234d10c74db4b022fe3fdaaff F20101212_AAAHLJ min_y_Page_085.jp2 ac3f7f9fa8a9892af6ca574b71d40109 0e75f96f8faa1eef611b99cccf9a135b5e94212a 5765 F20101212_AAAICB min_y_Page_030thm.jpg 553a9ca9bc6b6b676119b7184c97dea3 2667796da3e0334e3ba9f1f1f27c2fcee1a3ab7d 5539 F20101212_AAAHVE min_y_Page_080thm.jpg f45f61f578c19f7f0657e339e06b256b 277c5ffd2825b056742d00edaf608c7e3e1ced77 83348 F20101212_AAAHQH min_y_Page_022.jpg a13c355a11647aaa9ce899a2878723d5 35532dafb8e88d5ebf81ff98a704bb857ba49968 22672 F20101212_AAAICC min_y_Page_030.QC.jpg 25af00460533f001c346c9032ba3830b 92c33774cfa455a8838bbb816947d945b327b7bc 25264 F20101212_AAAHVF min_y_Page_102.QC.jpg 833ed8a2ea79e9520f8def8b01a9c871 72e2f7af31ccfd43b2bb7e9c96292d48d8a8e9b3 107089 F20101212_AAAHQI min_y_Page_098.jpg 9204dc476dc6357377bdedc58cb95672 a3f2033eef6bbac8e8b172214cbf9023f609594e 2931 F20101212_AAAHLK min_y_Page_069thm.jpg dd0c40692803c18662f4a49a9629e717 02a5191ffe33025305ebb97f24a01dcc11642431 4329 F20101212_AAAICD min_y_Page_031thm.jpg a6fa642282987298b066365c688acd86 7b8d25197f09901bb718dba297685dac4fa923a5 73869 F20101212_AAAHQJ min_y_Page_017.jpg 5753455e130a92272f5e2c2aa3e6cc66 21052b93831c1db0fd9f5d385274c901f9bf8947 27226 F20101212_AAAHLL min_y_Page_029.QC.jpg fcaada487a148e2a83ef89f3a6392b05 d90257bb9c25b3202e0a4321af0b472659d67dd3 6848 F20101212_AAAHVG min_y_Page_099thm.jpg 2929e616f9b717673143790fe3187b56 f94caf103af9402beb8d7ab9a834838dfe80db7e 17451 F20101212_AAAICE min_y_Page_031.QC.jpg dd935058bcebf8947810fb3b73f19c3b 345f77ed9fc28d209847b3b35d15bf5ad4dd5cb1 574715 F20101212_AAAHQK min_y_Page_028.jp2 d4eb91cbb4e37c0c7398b4dffc782e9e dbd59d67bc0d6ff3a83c906f5f5a3f107b6c8653 6456 F20101212_AAAHLM min_y_Page_085thm.jpg bf7cf3bd0d1bcf231ef02d8eda18a3ce 1ef8b297b29bdf54d33f6a8b59febf81a5e3ef1a 23396 F20101212_AAAHVH min_y_Page_059.QC.jpg 041a53720cbdb2dcaee1dd5dc4167e5b a734a4cb8a07f99edff1bc2929756ee10a88805d 27728 F20101212_AAAICF min_y_Page_034.QC.jpg 7125ec86b6c98b13f889f4fca4d2f4de 551f75a97e67012a6c7f3b9dc30ed7916e2d64b8 23779 F20101212_AAAHQL min_y_Page_090.QC.jpg e9c4f275c50f94d21decd76672997c5b 5e9aba58efde9d43db91e8b4a56893a4183d7fdd 5969 F20101212_AAAHLN min_y_Page_078thm.jpg 291c55c5c4399fe1c7cb13766d0a957a a807ccf004bd12c8b4e0bd3ecab7f9f74f1ea03e F20101212_AAAHVI min_y_Page_034.jp2 2dd67ea9dd2f57c720348e58e70b7e4d 2e72f661758135d12d2fca4894b1f34248158485 6711 F20101212_AAAICG min_y_Page_035thm.jpg 28940bcbc5a777458f48477093289bf6 551e89358a1ff0638f09c07b21a765453c56080b 996007 F20101212_AAAHQM min_y_Page_057.jp2 fb9ef76072382a1d960d2d0a5c9e8fea 6f89f1f477c60d3c2784a7bb6f7bdc8a6562f8d8 551544 F20101212_AAAHLO min_y_Page_073.jp2 7e4416e1950e2fde019853fd66c2521e 230eed21bb7e36a98faf440a60d5b1faa5ef9950 F20101212_AAAHVJ min_y_Page_084.tif 9883dd267c18de6c84557cfde1e6706e 1852033a88dbbd7cf1d80dc5c75dc72b45b23f6e 27966 F20101212_AAAICH min_y_Page_035.QC.jpg ca5e901d6b32ef5458a81ade8db1109f d1da7b9d69354c6de3379fe86fb87223e88e8f80 F20101212_AAAHQN min_y_Page_098.tif d4bed92ca36ebae149b8ffc074612d6d 5ae55ceaf2d9800d63b994dde35f8fbc53f561d2 22339 F20101212_AAAHLP min_y_Page_096.QC.jpg e9720240ac93bb802dadbf02865812a0 7e66c1f761c8be454fe82dd3883c823b9e321dfe 27105 F20101212_AAAHVK min_y_Page_037.QC.jpg 9b2e9d6789a9f20812becfe601358bc6 6f36ad3fb25a5b3edef610b6d2715abf8677f800 4100 F20101212_AAAICI min_y_Page_038thm.jpg 2f2f87169cf4dd5be7a2ddc1d133211f 79edcfdfdc77f0e7760199278b546c3d18d52a9c 9251 F20101212_AAAHQO min_y_Page_070.QC.jpg de8087033088594c511e85f3d088ab7b c05aac386ba14d57fe911f3e6ff99895e4f0a517 28002 F20101212_AAAHLQ min_y_Page_088.QC.jpg dc5fa033f9f365f039fe9968d3e85a19 7f125bb2d6bb8a2b0528c95a05ab0c6f7c083c94 F20101212_AAAHVL min_y_Page_047.tif 27ef32eea750fed757169dd854629992 b093dd488343fc2ee03ef990b0ec4573c4b5f2ef 17374 F20101212_AAAICJ min_y_Page_039.QC.jpg 9a9b65dabee90f0030da623ad9176b99 45ad742280c40971f4a0ae5c32ad34d47c8b21a4 87595 F20101212_AAAHLR min_y_Page_096.jpg 1c8dc6c643e8aa4e0bb7a1548e64fd06 213a6e284941eface802ab6644532ee9ff8a1a7a F20101212_AAAHVM min_y_Page_038.tif 9f4ddc8a1906fe932e0bfb1d29749bf0 b024b7932a4bd2eb06ae951e7b7b780bd4493aa4 12851 F20101212_AAAICK min_y_Page_040.QC.jpg 4122c5e4b5a0a6aa791246a12c8e0eca ff86eef963d2369dd2ca4fde5f7820622c863a13 4092 F20101212_AAAHQP min_y_Page_028thm.jpg 9419b789676cb681f63b7fff26475849 81153a217010fdaef25b58ea3bd5fefa6a9d2688 6420 F20101212_AAAHLS min_y_Page_087thm.jpg 7726c260fa88da6f61c13359153c5710 d037fd554d66f961994bbfb56e4b346e4185bad0 14074 F20101212_AAAHVN min_y_Page_026.QC.jpg b6e447d576e33bfa4e29dd7985a6540e da1c140376da267cf7576078db68a1d5f2166670 12850 F20101212_AAAICL min_y_Page_043.QC.jpg f70c0cf88b2678bdf2bb765db06d4bc4 d7d81dc48f5b0022cf2d19823593cce3e33bdf97 62244 F20101212_AAAHQQ min_y_Page_084.jpg 99abbc1c62d5a4e1f99f0ba446ccad2f abc8da26b9f66de7e6821fdd84cdb9c03aa74a2a 26644 F20101212_AAAHLT min_y_Page_023.QC.jpg 30b39e6a12b343fc3d8bcccf67600c6c 27beb8ac2f4f48e90475fcf541c4321fdad115e9 84507 F20101212_AAAHVO min_y_Page_025.jpg 4a0e4df1cb4d84dc81f5846d329ed3b7 02ed742a14116dd4f997ee6ba52f91679c19b903 6173 F20101212_AAAICM min_y_Page_045thm.jpg 3b2a4fa438bd321502ec2087f9ffc8a9 3fc6a870d59e835e72611cfc9aed25de0d6a6567 6694 F20101212_AAAHQR min_y_Page_058thm.jpg 161455e661c221dc0b1065cb224eb5b6 6483670bcb8b114a0c0400350cde407e4bf4857e 563355 F20101212_AAAHLU min_y_Page_026.jp2 f7578bbc83bc64030da02d9de060588a 7610342b326d068c3d167f7cebc6aa20d2f468eb 210563 F20101212_AAAHVP min_y_Page_044.jp2 db6e69ca7058b021ff89ce4be223c50c 68d6c15d4cde2d113e0871e79d603673afbe3c3f 25989 F20101212_AAAICN min_y_Page_045.QC.jpg 2a5ca74dc2cee3f25ae5e016c77b78d8 2a319928c316ddb979dc2f56dc12465482bc4cda F20101212_AAAHQS min_y_Page_046.tif 59a88479fdda4a355662a18cb16a959b b9cad15dc9580e71a93e3c6fd9218751d38304ad 6596 F20101212_AAAHLV min_y_Page_053thm.jpg 1faa94dfd0f585788bd5e9328ab81042 c05ea74ae4115aafd740736865b5a21d6986b715 F20101212_AAAHVQ min_y_Page_023thm.jpg e0105566f42fa8f7da1d3ae6ae51cd37 12ad5d6ad635201465db983f26ddfc53c271ecce 21975 F20101212_AAAICO min_y_Page_047.QC.jpg 62bef08aef28d7d667e5e2fed6bac17c 414a47e8a8964faaf377501afdeafda60e242c2c 6363 F20101212_AAAHQT min_y_Page_089thm.jpg 2d7a40fbdbc398ae83cd1221f78f05dd e40a1bdee72526b965c1112948f7fe5658b3e034 61676 F20101212_AAAHLW min_y_Page_039.jpg 50522c8bcd8d689fa45b639ee583eaa1 d895bc992abc73a3883331a044675a5e19ad4193 F20101212_AAAHVR min_y_Page_048.tif 7b488b0d019710a42b1d54261964e882 4fb48e15733cae726070170ba52ea898c8958ead 26407 F20101212_AAAICP min_y_Page_048.QC.jpg 9db6f4b5b991db1506c97edcd6a5cb3b bd504c2dbbdbe676f40dc8ff943e9606666d5555 F20101212_AAAHQU min_y_Page_031.tif a5c4fd6e418714979e0064b372bf3157 fb9825a9586c867fb133089deea21543f35e72f3 F20101212_AAAHLX min_y_Page_043.tif 91969de1f4c99234a87c45868fda28c8 7c8e174ea576d23578113064c312138222939061 5157 F20101212_AAAHVS min_y_Page_042thm.jpg 8e5bd0d8ac85a79dbc88a67d228c6ede 64d92ace1d954eedbc7e8ab5cad4f74d954ce233 6494 F20101212_AAAICQ min_y_Page_050thm.jpg 0fba5eed9feb671581806abc9899aebb dc0ad34a3f21ddfec1d12c2a4dada1ce0ebdcece 1051939 F20101212_AAAHQV min_y_Page_101.jp2 2e5f076a7ef407d6f28a8858bdb340ac 672c1f3515aab3d718626542cd7200c2ac8a7f3f F20101212_AAAHLY min_y_Page_088thm.jpg f79729aae617ca440218c06b75af6aff 4c9e004f321c3d20f87be99022e4c1eb2793a95d 972594 F20101212_AAAHVT min_y_Page_059.jp2 5b7aee5b80d75d101078b0c6de48c161 ea7a159cd5eaaa7b4af8e23c2c97adb90fe1c995 6232 F20101212_AAAICR min_y_Page_051thm.jpg 6d23ab2301f34ede730032eab4604f64 c1ea76f8266f6d99d6d35faca93d2b6b3c8b36e2 1002895 F20101212_AAAHQW min_y_Page_076.jp2 fc5a2ee6bfac746c1b05139ce394a2b2 d80b5612da643d02ba5a05f367f8d6be5d9ce520 27177 F20101212_AAAHLZ min_y_Page_062.QC.jpg e2b6b125ecaa5389e5f43607a832609e 5bb1ad2f36a6578a2a49ed703552ba50a2c3a0c6 27709 F20101212_AAAICS min_y_Page_053.QC.jpg 080b564a4e4b8056b8468cd3b4df1517 56810a15081d9948a71b5972e0e6a4d40fa0c065 3702 F20101212_AAAHQX min_y_Page_043thm.jpg f67d9db4b761754a33af4320a845c16b 8da86f16ebc4a99b5732498caf9b1428c2e55b26 4333 F20101212_AAAHVU min_y_Page_081thm.jpg bd06c4747b1f79b4ffae0c99693d7015 96dfc305cbdfebfcf064b8f0cb875b1d5642f075 6067 F20101212_AAAICT min_y_Page_057thm.jpg 81f85b17c928aee7ec6355c732ded19f 297b786062463aefd32b256248c3f5704d7f16f0 90940 F20101212_AAAHQY min_y_Page_036.jpg b1c819d01c107de633305045cc9b431e 729b46b4d90e0311c286c78c9d82427a93ec1974 2334 F20101212_AAAHVV min_y_Page_070thm.jpg ac2d1a46015c0f191eb50c12aa9a8e5e 71509b972df05fc5eb575c7afd78ec02b8c638b2 79565 F20101212_AAAHOA min_y_Page_083.jpg 51bbc6acb0ac6e2b4bc681175ca6cbae 6f184debbf9e3b67ebd3f2fbcc12cd3be4ac6f41 28105 F20101212_AAAICU min_y_Page_060.QC.jpg 663b8143776534f19fcd76a5ea2f6e7c f43031df18d10e26b91d3157cca8116ee857b6f0 83608 F20101212_AAAHQZ min_y_Page_064.jpg 013defb33aba0ce93192c21c8ac0c6d1 cc1cef3bff8546b8dd5b8eef27838ce6766e6e85 F20101212_AAAHVW min_y_Page_089.tif 54ff2b14d38594da8113b8415c27c675 7a70e14d9898bfe12a30a0201d8fbd926c184ace F20101212_AAAHOB min_y_Page_088.jp2 f92fcd12ca6cf12ff4ec2ed7571722c4 7defdc1c6e7a9a599e83f146de1a6315d6ad3af2 25845 F20101212_AAAICV min_y_Page_061.QC.jpg d34b113dc7461f0304cad4cbeea5dc6a 7d6bd690403bea52ef87ee830f257fa091d07d49 937383 F20101212_AAAHVX min_y_Page_080.jp2 7bc972bf3d98db22dc3879b8ae4f4ad5 7de8f8845b3e00ea5d3bc75a5597a0a760cfc5ea 16195 F20101212_AAAHOC min_y_Page_092.QC.jpg dc3c04d5e5996f3e8565aa695e6f90ca fa42572ba6a1a3223c622e79a9b88d9b72ddff07 6226 F20101212_AAAICW min_y_Page_064thm.jpg e43d9f3563bd4a2c52cdc2181568d08a 99fb3df93af53518b39c7e1198cc8d215c6163cf F20101212_AAAHTA min_y_Page_067.jp2 e6ad768a83c3a638a641a2e272983247 1a6f51e3a087bce4bdb515551c9e78829e8d4ed8 24167 F20101212_AAAHVY min_y_Page_017.QC.jpg dacb2eabc27cd8bbc01829e5efd88695 ab1e2866038090a6559412cbee890014977a1d2c 49421 F20101212_AAAHOD min_y_Page_103.jpg 0f7ace25074ab0038b3463a6e4f18308 4c0b5c6113018a31bb1679a42b1a636db3808c27 6826 F20101212_AAAICX min_y_Page_065thm.jpg 49ef0df0465d9202df2530d7803ffc49 4eef7e7f8722b21970bf8bf69ec1b57a42651c39 F20101212_AAAHTB min_y_Page_052.tif 4ac4599ca822905eec8e9b3c7e50c925 20bb69a49657d828a47401c44115f32b45303099 27480 F20101212_AAAHVZ min_y_Page_058.QC.jpg f854dfe897a34d56a29d701467b467c1 13091cd4a3d631795d661b068f65166c5db9ad3a F20101212_AAAHOE min_y_Page_057.tif dba26f23eeb46e28016d120b112f91ae df6c8e06a83a4be275e315db767974983605800d 6693 F20101212_AAAICY min_y_Page_066thm.jpg a2c248e9347ce8b46d2c7bef44b1bd29 624df597bc856bff87cba2c42d97cc39c48918a1 87869 F20101212_AAAHTC min_y_Page_060.jpg c39b0e810e6b8b77e905eda283be65d6 b6a45df3a3f1eebd1dc96590c48dca546a7e84cb F20101212_AAAHOF min_y_Page_008.tif efd7e2bd2078330ac9d8d5df58df8bff 99d4b7b577edb454ed584e638b420c1dc0b03ca5 6496 F20101212_AAAICZ min_y_Page_067thm.jpg 50b4e23ecabcfa850b3aa00891109d1b 21a17c511303657a6f1006800799a55709c06039 F20101212_AAAIAA min_y_Page_065.jp2 c37432385a903b0be260dedfd17de1cb 77de3b6636e7ccd507f282c6176c422bc878acf6 F20101212_AAAHYA min_y_Page_016.tif 8311adfb8eb6dad10969f99f4d4fac6f 95f0034a3ed831fbe1c9845ba23f6eb5ea285014 5742 F20101212_AAAHTD min_y_Page_041.QC.jpg 00be093b5c90f1da987e0a34082f3dc4 3758acee4d3b1c05e5742dbd6d3b776b0c291593 26514 F20101212_AAAHOG min_y_Page_022.QC.jpg fc043a9ac5bbe5344bd9066723bd31b0 f58d639b3067cd971044cfd0f606e694b3764ff3 1051974 F20101212_AAAIAB min_y_Page_066.jp2 6a441ed66b6ce283465413c0195f3ea0 f3977c524d98a21b7e9927aea51475127bb3fccf F20101212_AAAHYB min_y_Page_023.jp2 0d19c1188c0786c72fa06fae1541ed1e bac7e1e8d29850ccbf2252c1cd45663dbd39ad7f 26326 F20101212_AAAHTE min_y_Page_085.QC.jpg adab88413a497905086181d74d204df9 0cd4d4eea02548ca3d4c1c01db01f02193a6257c 1051958 F20101212_AAAHOH min_y_Page_037.jp2 ccc33071600db39bdf3f842f59d8323c 05f6d351c8bae037d7d8b7ec86ead079ca02824a F20101212_AAAIAC min_y_Page_068.jp2 b5bbf1cb4d525dcd8e116446681f02eb 7573be30ec8dfd5ec2e2a3d4e58b20a2bbe83ff4 3399 F20101212_AAAHTF min_y_Page_040thm.jpg b403fe2ef441d2ee2e79eb00f251c3d7 33bdb269dd2cc0c2b71045f9465d17be3aa8d5c7 27253 F20101212_AAAHOI min_y_Page_065.QC.jpg 932a2afd8f34f2965b1c640df6d5bdab 60631d107b9f0690294fa3a32ed7b6e69acd7d6a F20101212_AAAHYC min_y_Page_050.tif 9c3c438dbf37b12e09997ab3542592af c2cbd795af133f2b57e014411a39c709b8926b58 485991 F20101212_AAAIAD min_y_Page_069.jp2 aa9e571c90005ef12081050a0c073a78 d2ce93f88298956bdc1821cf67bb46765b548dc5 5864 F20101212_AAAHTG min_y_Page_017thm.jpg 01134696ed1deee553eb403b929ce235 095b1aa6a1d1a45c24e7c21f0fc840e0055fbf46 4693 F20101212_AAAHOJ min_y_Page_082thm.jpg 1a4fdb51cbf097e5d309c23d1837ad7d cd39dfe583f04d3bc6883205d37e41449252b9f1 1051967 F20101212_AAAHYD min_y_Page_099.jp2 1d60006be32ffcacb3776cc5c5a0a852 0d96964e183b8c2a7109b970770f6c137565b5ac PAGE 1 1 ROBUSTNESS IN CONFIRMATORY FACTOR ANALYSIS: THE EFFECT OF SAMPLE SIZE, DEGREE OF NONNORMALITY, MODEL, AND ESTIMATION METHOD ON ACCURACY OF ESTIMATION FOR STANDARD ERRORS By YOUNGKYOUNG MIN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2008 PAGE 2 2 2008 Youngkyoung Min PAGE 3 3 To my Mom, Taesun Jeong for her endless love, support, encouragement, and prayer PAGE 4 4 ACKNOWLEDGMENTS First of all, I would like to thank God and m y mother for every moment of my life and for providing the opportunity to gain this tremendous education. I am thankful to Dr. James Algina and Dr. David Miller for giving me the opportunity to study in Research and Evaluation Methodology Pr ogram, and for supporting me to finish my doctoral study. I owe much gratitu de to the faculty of Departme nt of Educational Psychology for his/her excellent lecture a nd thoughtful consideration. The completion of this dissertation would not be possible without the guidance of my dissertation committee. Each has helped me th roughout my dissertation in his/her own ways. I especially wish to thank Dr. James Algina, my supervisory committee chair and advisor, for his precious time, professional mentoring, and valuab le guidance during the last four years. I also wish to thank Dr. David Miller for his cons ideration and useful advice which made my confidence firm in the entire proc ess. I would like to thank Dr. Cynthia Garvan for her constant encouragement and challenging suggestions which made me stronger academically. She was also the committee chair of my thesis for M. S. in Statistics and encouraged me to pursue Ph. D. I also want to thank Dr. Walter Leite for his useful information on references and valuable suggestions which made this dissertation rich. I wish to give many thanks to Dr. Timothy Anderson, Associate Dean at the College of Engineering, for giving me the opportunity to wo rk at Engineering Education Center throughout my doctoral study, and for letting me obtain valuable research experiences in engineering college education field. I wish to thank Erica Hughes for her friendly tips a nd technical support. I would like to thank Dr. Mark Shermis, Ch air of Department of Educational Psychology, for making the departmental pro cess smooth and giving me practical tips for graduation. Lastly, I PAGE 5 5 want to express my gratitude to Elaine Green and Linda Parsons, for th eir kind support and help on the official process for a graduate student during my doctoral study. Give Thanks to the LORD, for he is good; his love endures forever. Psalms 107:1 PAGE 6 6 TABLE OF CONTENTS page ACKNOWLEDGMENTS...............................................................................................................4 LIST OF TABLES................................................................................................................. ..........8 LIST OF ABBREVIATIONS........................................................................................................ 11 ABSTRACT...................................................................................................................................12 CHAP TER 1 INTRODUCTION AND LITERATURE REVIEW..............................................................14 Introduction................................................................................................................... ..........14 Overview of Overall Robustness Studies in SEM.................................................................. 14 Covariance Structure Models a nd Estim ation Methods in SEM............................................ 16 Robustness against Nonnormal Distribution......................................................................... 24 Skewness and Kurtosis in Univariate Distribution.......................................................... 25 Skewness and Kurtosis in Mu ltivariate Distributions .....................................................27 Robustness against Small Sample Size................................................................................... 29 Research Summary Characteristics........................................................................................30 Findings from Overall Past Robustness Studies.....................................................................32 Direction for the Present Study............................................................................................... 33 2 DESIGN OF MONTE CARLO SIMULATION STUDY...................................................... 45 Data Generation......................................................................................................................45 Fleishmans Power Transformation Method................................................................... 45 Vale and Maurelli Data Generation Method ................................................................... 46 Estimation Methods................................................................................................................47 Sample Sizes...........................................................................................................................47 Distributional Characteristics................................................................................................. 48 Model Characteristics.......................................................................................................... ...48 Analytical Model Characteristics........................................................................................... 49 Number of Replications......................................................................................................... .50 Research Summary Statistics..................................................................................................50 Ways to Summarize and Present Results................................................................................ 51 3 MONTE CARLO STUDY RESULTS................................................................................... 53 Preliminary ANOVA Tests..................................................................................................... 53 WelchJames Tests and Et a Squared Statistics ....................................................................... 55 Effects on Parameter Estimates....................................................................................... 59 Effects on Standard Error Ratio Estimates...................................................................... 60 Practical Problems in Estimation............................................................................................ 67 PAGE 7 7 4 SUMMARY AND CONCLUSIONS.....................................................................................85 Comparison of Findings with Previous Studies......................................................................85 Parameter Estim ates........................................................................................................ 85 Standard Error Estimates................................................................................................. 87 Brief Summary........................................................................................................................90 Concluding Remarks............................................................................................................. .91 APPENDIX: STANDARD ERROR RESULTS FOR FACTOR CORRELATIONS: SCALE SET BY SPECIFYING FACTOR VARIANCES EQUAL TO ONE. ...................................93 LIST OF REFERENCES...............................................................................................................97 BIOGRAPHICAL SKETCH.......................................................................................................103 PAGE 8 8 LIST OF TABLES Table page 11 Overview of previous robustness st u dies against sample size and nonnormality............ 39 12 Estimation methods investigated in the past robustness studies ........................................ 40 13 When are ML, GLS, and ADF equivalent?....................................................................... 41 14 Degree of nonnormality examined in the past robustness studies.................................... 42 15 Range of sample size examined in the past robustness studies ......................................... 43 16 Number of replications in the past robustness studies....................................................... 44 31 Frequency of significance F tests on param eter estimates per 16 combinations of distribution and sample size: f actor variances equal to one.............................................. 70 32 Frequency of significant F tests on param eter estimates per 16 combinations of distribution and sample size: f actor loadings equal to one................................................ 71 33 Mean parameter estimate for = 0.6: Model 3F3I, GLS, N = 200, factor variances equal to one, and (0, 1.15)................................................................................................72 34 Frequency of significant F tests on standard error estim ates per 16 combinations of distribution and sample size: f actor variances equal to one............................................... 72 35 Frequency of significant F tests on standard error estim ates per 16 combinations of distribution and sample size: f actor loadings equal to one................................................ 73 36 Mean standard error estimates for = 0.3: model 6F3I, Robust ML, N = 1200, factor variances equal to one, and (2, 6)............................................................................ 73 37 F statistics (degrees of freedom ) for WJ tests for effects on parameter estimates: scale set by specifying factor variances equal to one........................................................ 74 38 Eta squared for significant effects on param eter estimates: scale set by specifying factor variances equal to one.............................................................................................. 75 39 Marginal means by estimation for estimates of = 0.8....................................................75 310 Mean estimates of residual variance param eters by parameter value, estimation method, and sample size.................................................................................................... 75 311 F statistics (degrees of freedom ) for WJ tests for effects on parameter estimates: scale set by specifying factor loadings equal to one.......................................................... 76 PAGE 9 9 312 Eta squared for significant effects on param eter estimates: scale set by specifying factor loadings equal to one............................................................................................... 77 313 Grand mean estimates of pa ram eter by parameter value................................................... 77 314 F statistics (degrees of freedom ) for WJ tests for effects on standard error ratio estimates: scale set by specifying f actor variances equal to one....................................... 78 315 Eta squared for significant effects on standard error ratio es timates: scale set by specifying factor variances equal to one............................................................................ 79 316 Marginal means by sample size for s tandard error ratio estimates of = 0.6................... 79 317 Mean estimates of standard error ratio for factor loading param eters by parameter value, estimation method, and distribution........................................................................ 80 318 Mean estimates of standard error ratio for factor corre lation parameters, (0.3) by model, sample size, and distribution.................................................................................. 80 319 Means estimates of standard error ratio for factor corre lation parameter, (0.3) by estimation and distribution................................................................................................. 80 320 Marginal means by sample size for s tandard error ratio estimates of = 0.36............... 81 321 Means estimates of standard error ratio for residual variance param eter, (0.36) by model and distribution....................................................................................................... 81 322 Means estimates of standard error ratio for residual variance param eters by parameter value, estimation method, and distribution....................................................... 81 323 F statistics (degrees of freedom ) for WJ tests for effects on standard error ratio estimates: scale set by specifying f actor loadings equal to one......................................... 82 324 Eta squared for significant effects on standard error ratio es timates: scale set by specifying factor loadings equal to one............................................................................. 83 325 Mean estimates of standard error ratio for factor loading param eters by parameter value, estimation method, and distribution........................................................................ 83 326 Marginal means by sample size for s tandard error ratio estimates of (i, j) = 0.108....... 83 327 Mean estimates of standard error ratio fo r factor corre lation parameters by parameter value, estimation method, and distribution........................................................................ 84 328 Frequency of nonconvergence (NC), improper estimates (IE), and nonpositive definite Hessians (NP) .......................................................................................................84 PAGE 10 10 A1 Mean standard errors, empi rica l standard errors, and standard error ratios by model, distribution, and sample size for ML estimation............................................................... 93 A2 Mean standard errors, empi rica l standard errors, and standard error ratios by model, distribution, and sample size for GLS estimation.............................................................. 94 A3 Mean standard errors, empi rica l standard errors, and standard error ratios by model, distribution, and sample size for Robust ML estimation...................................................95 A4 Mean standard errors, empi rica l standard errors, and standard error ratios by model, distribution, and sample size for Robust GLS estimation.................................................. 96 PAGE 11 11 LIST OF ABBREVIATIONS ML Maximum Likelihood GLS Generalized Least Squares ULS Unweighted Least Squares DWLS Diagonally Weighted Least Squares ADF Asymptotically Distribution Free RML Robust Maximum Likelihood / Robust ML RGLS Robust Generalized Least Squares / Robust GLS Factor Loading Factor Correlation (i, i) Factor Variance (i, j) Factor Covariance / Factor intercorrelation Residual Variance SEM Structural Equation Modeling 3F3I 3 Factors with 3 Indicators per factor 3F6I 3 Factors with 6 Indicators per factor 6F3I 6 Factors with 3 Indicators per factor (0, 0) Standard Normal Distribution w ith zero skewness and zero kurtosis (0, 1.15) Nonnormal Distribution (plat ykurtic) with zero skewness and equal negative kurtosis (1.15) (0, 3) Nonnormal Distribution (leptokur tic) with zero skewness and equal positive kurtosis (3) (2, 6) Nonnormal Distribution with high equal skewness (2) and high equal kurtosis (6) PAGE 12 12 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ROBUSTNESS IN CONFIRMATORY FACTOR ANALYSIS: THE EFFECT OF SAMPLE SIZE, DEGREE OF NONNORMALITY, MODEL, AND ESTIMATION METHOD ON ACCURACY OF ESTIMATION FOR STANDARD ERRORS By Youngkyoung Min August 2008 Chair: James Algina Major: Research and Evaluation Methodology A Monte Carlo approach was employed to exam ine the effect of model type, sample size, and characteristic of distribution on the Maxi mum Likelihood (ML), Generalized Least Squares (GLS), Robust ML, and Robust GLS estimates of parameters standard errors. The LISREL program was used for estimation, and the populati on covariance matrix and data were generated by using the SAS program. For each of four es timation methods (ML, GLS, Robust ML, and Robust GLS) the behavior of standard e rror ratio estimates was examined under each combination of four distributions, four sample sizes (200, 400, 800, and 1200), three CFA models, and two scalesetting methods (set by specifying factor varian ces equal to one and factor loadings equal to one). In addition, the bias of the parameter estimation procedures was investigated. The effects of four factors (estimation method, distributi on, model, and sample size) on parameter estimates and standard erro r estimates were examined within each scalesetting method with WelchJames test and eta squared. Results for parameter estimates indicate that ML estimates were almost unbiased at all sample sizes and ML estimation had less bias than GLS estimation, although the differences were trivial for factor loadings a nd factor correlations. Sample size played a more critical role in PAGE 13 13 GLS estimation than in ML estimation of residu al variance and, as a result, larger betweenmethod differences in bias were observed for estim ates of residual variance. When the scale was set by specifying factor loadings equal to one, th ere were no important effects of the factors on the factor loading, fact or variance, or factor covariance estimates. Results for standard error estimates indicate that Robust ML estimates were superior to the nonrobust estimates in the bias of th e standard error estimates for the nonnormal distributions, and the standard error estimates were underestimat ed for the distribution with positive kurtosis and overestimated for the distribution with negative kurtosis. From the results, it can be concluded that ML estimation method should be adopted for a normal distribution regardless of sample size, model, and scalesetting method to obtain less biased estimates of parameters and standard errors, and Robust ML should be used for nonnormal distributions to improve estimation of st andard errors. However, Robust ML estimation works very well even for a normal distribution a nd some cases better than GLS. It was also found that robust estimation generally worked better than nonrobust estimation for the nonnormal distributions regardless of the sample si ze and the model type. When the distribution is nonnormal, Robust GLS generally performs we ll, although Robust ML has less bias than Robust GLS. PAGE 14 14 CHAPTER 1 INTRODUCTION AND LITERATURE REVIEW Introduction Structural equation m odeling (SEM ) has been one of the importan t statistical approaches in the social sciences because it can be applied to nonexperimental and experimental data and offers a convenient way to differentiate between observed variables and latent variables. SEM is often conducted under the assumption of multivaria te normality, which implies that all univariate distributions are normal, the joint distribution of any pair of the va riables is bivariate normal, and all bivariate scatterplots are linear and homoscedastic (Kline, 2005). However, real data in social science are rarely normal. The viola tion of normality assumption can cause incorrect results in SEM. Checking the degree of nonnormality of the data and the use of the methods that are less reliant on normality are necessary. Over th e past years, the effects of nonnormality have been studied and different types of solutions have been proposed. Speci fically, it was recognized that the model, the estimation method, and data char acteristics such as sample size play critical roles under nonnormality. Therefore, the present study used simulation methods to investigate the impact of sample size on the accuracy w ith which standard errors are estimated under varying degrees of nonnormality, different estima tion methods, and different structural equation models which include latent variables. In th e present study, the effect s of model complexity, sample size, estimation method, scalesetting method, and degree of nonnormality on the accuracy of estimated standard errors were investigated. Overview of Overall Robustness Studies in SEM As the term has come to be used in statistics, robustness refers to th e ability to withstand violations of theoretical assumptions (Boo msma, 1983; Harlow, 1985). A common procedure for studying robustness is to generate data sets a nd examine the behavior of research summary PAGE 15 15 characteristics such as teststati stics, standard errors, etc. This procedure is known as the Monte Carlo simulation method and has been used in mo st of previous robustness studies in SEM (e.g., Anderson & Gerbing, 1984; Bearden et al., 1982 ; Boomsma, 1983; Browne, 1984; Muthen & Kaplan, 1985, 1992). The issues of robustness against small sample size, distributional viol ations, analysis of correlation matrices, model misspecification, and non linear structural equations under different estimation methods have been investigated over the past decade. Hoogland (1999) set forth the following five robustness questions: Do the maximum likelihood (ML) and generalized least squares (GLS) estimators possess the asymptotic statistical properties predicted under normal theory when variables are nonnormally distributed? Does a small sample size cause problems, because the statistical properties of the parameter estimators, standa rd error estimators and goodness offit test statistics are asymptotic properties? Does analysis of correlation ra ther than covariance matrices cause problems when a model is not scale invariant? Does model misspecification cause the incorrect results of analyses? Does ignoring the nonlinear relationships between latent variables cause the wrong solution? The problems of analyzing correlation matr ices, model misspecification, and nonlinear structure were excluded in the present study because covariance matrices were used, and the structural equation models we re specified to be linear. A review of past robustness studies was necessary to set up the purpose of the current study and the design of the Monte Carlo study. Tabl e 11 shows an overview of a collection of robustness studies for the effects of sample size and nonnormality. (The numbers in the Paper Number column will be used in Tables 12, 14, 15, and 16 to identify the studies listed in Table 11.) Of the 47 studies, a total of 41 studies investigated the effect of sample size on PAGE 16 16 robustness and a total of 34 st udied the effects of nonnormality on robustness in SEM. Other robustness issues (the problem of analyzing co rrelation matrices, model misspecification, and nonlinear structure) were excluded from Table 11 and the present study. Covariance Structure Models and Estimation Methods in SEM The funda mental hypothesis in c ovariance structure modeling is ),( where (k k) is the population c ovariance matrix of k observed variables,) ( (k k) is the population covariance matrix of k observed variables written as a hypothesized function of, and (t 1) is a vector of the model parameters. The sample estimator of the population covariance matrix in a sample of size N is )1/( NZZS, where Z is an ( N k) matrix of deviation scores of the observed variables and k is the number of the observed variables. A general formulation of a covariance struct ure model for confirmato ry factor analysis (CFA) with latent variables is as follows (Jreskog & Srbom 1996). Using LISREL notation, x where x is a ( k 1) vector of indicators (the obs erved or measured variables) of the m exogenous latent variables, is a ( k m ) matrix of the loadings of x on, and is a ( k 1) vector of measurement errors. It is assumed that thes and s are random variables with zero means, s are uncorrelated with s, and all observed variables are meas ured in deviations from their means. The measurement model represents the regression of x on and the element ij of is the partial regressi on coefficient of j in the regression of ix on 1 2 m The model implied covariance matrix for the x variables is defined as: PAGE 17 17 ) (, where is the covariance matrix for and is the covariance matrix for. Given a specified model,(), the unknown parameters of are estimated so that the discrepancy between the sample implied covariance matrix ) ( and the sample covariance matrix, S is as small as possible given some criterion, where is the vector of parameter estimates. A discrepancy function )) (,( SFis needed to quantify the fit of a model to the sample data. This function should have the following properties: 1. ))(,( SF is a scalar, and )) (,( SF 0. 2. ))(,( SF = 0 if and only if ( ) = S 3. ))(,( SF is twice differentiable in ) ( and S Minimizing a discrepancy function with these properties leads to consistent estimators of when the model is correctly specified and some regularity conditions are satisfied (Browne, 1984; Hoogland & Boomsma, 1998). With these definitions and properties, Browne (1982, 1984) framed the discrepancy function approach into a weighted least square s (WLS) approach and demonstrated that all existing discrepancy functions we re special cases of the follo wing WLS discrepancy function: )),(())(()( sV s F where ) ,...,,,,()(31222111 kksssssvecSKs is a vector consisti ng of the nonduplicated elements of S, )( S vec is a vector of order k2 1 consisting of the columns of S strung under each other, 1(), KDDD where D is the duplication matrix which transforms s to (),vecS K is the generalized inverse of D which transforms )( S vec to s and V is a specific positive definite weight matrix and is defined differently for different discrepancy functions: PAGE 18 18 Unweighted least squares (ULS): *IV ULS Generalized least squares (GLS): D DV )(11 GLS Maximum likelihood (ML): D DV )) () ((1 1 ML Asymptotically distribution free (ADF): 1 NNT NNTWV Diagona lly weighted least squares (DWLS):1 1][ NNT W DWLSdiagNNTW DV. In theses expressions,,...) 1,2,2,1,2,1(*diag is the symbol for a Kronecker product, and NNTW is the asymptotic covariance matrix of S estimated without assuming normality. DWLS can be formulated by using the diagonal elements of ,NTGLS WV but in this dissertation DWLS refers to estimates obtained using D WLSV defined above. As suggested by the expressions for the weight matrices for GLS and ML estimation, seve ral of the weight matrices are functions of population parameters and must, in practice, be estimated. The goal of estimation is to produce ) ( that is as similar as possible to the sample covariance matrix, S with similarity defined by the disc repancy function. The weight matrix, V in the discrepancy functions above, determines the estimation method chosen. Let U be the sampling covariance matrix of the nonredundant elements of Sand )]([ The optimal choice for the weight matrix is 1 U and this matrix provides best generalized least squares estimates. For example, if the data are multivariate normal GLSV or M LV provide optimal weight matrices. Browne (1982) has shown that if the optimal weight matrix is used, the asymptotic sampling covariance matrix for the parameter estimates is11 1)()1( U N. For any other choice of the weight matrix th e asymptotic sampling covariance matrix is1 1 1))(()()1( V VUV V N. In addition, Browne (1982) has shown that the diagonal PAGE 19 19 elements of 11 1)()1( U N are not smaller than th e diagonal elements of 1 1 1))(()()1( V VUV V N so that the standard errors of the best generalized least squares (BGLS) estimator cannot be larger than the standard errors when a suboptimal weight matrix is used. An important aspect of covariance structure an alysis is the evaluation of the fit of the model. A typical statisti c for such evaluation is ) ()1( )1( FNFN which is the so called chisquare goodness of fit statistic (Satorra, 1990). The residual vector, )) ( ( SK contains the nonredundant residuals and al so provides information for th e fit of the model (Satorra, 1990). It may be noted that under the assumption of the asymptotic normality of the residual vector, ), KS ( WLS estimation amounts to a problem in the distribution of a quadratic form in normal variables about which much is known. In fact, if one considers obtaining the symmetric square root 1/2V of V in the WLS discrepancy function above and multiplying the resulting matrix into the vectors on either side, it is apparent with an optimal V one obtains a vector of independent variates. Thus, the WL S discrepancy function can be considered to represent the sum of squares of independent normal variables which is intimately related to the chisquare distribution. In fact, with such an optimal weight matrix, the WLS estimator is a minimum chisquare estimator, or minimum modi fied chisquare estimator when the weight matrix is estimated (Bentler, 1983). The matrix 2,NNT WK CK where C is a 22kk matrix with elements that are fourth order cumulants of x the 1 k random vector for the data with mean and covariance matrix (Browne & Shapiro, 1988). That is ).2()()()(}]))({(}))({([1 KKKK x x x x C cvevec cve vecE PAGE 20 20 The matrix NNTNW1)1( is the asymptotic covariance matr ix of the nonredundant elements of the covariance matrix S and its elements will be finite assuming only that the variables have finite fourth order cumulants. Because ADF and DWLS are based on NNTW they do not make strong assumptions about the distribution of the data. ULS does not make any assumptions about the data but also uses less information about th e data than do ADF or DWLS. If the data are normal, C0 and NNTW simplifies to 2.NT WK K As GLS uses the inverse of NTW as its weight matrix, GLS is based on the normality assumption. ML estimates can be obtained by using two di fferent discrepancy functions. First ML estimates can be obtained by minimizing the following discrepancy function which is wellknown: k tr F log})({)(log)(1S S In addition ML estimates can be obtained by minimizing ))(())(()( sV s MLF. The ML weight matrix, MLV is derived as a function of elements of) ( This means that the ML weight matrix is effectively updated as the estimate of ) ( changes at each iteration in the estimation process. Both of these discrepancy func tions have a minimum at the same point in the parameter space, namely at the ML estimates, but the minimum value of the functions are not the same (Jreskog et al., 1996; Satorra, 1990). The first of the two functions is referred to as the ML discrepancy function and ) ()1( )1( FNFN is referred to as the minimum fit function chi square. The second function is referred to as th e normal theory WLS discrepancy function and ) ()1( )1( FNFN is referred to as the normal theory weighted least squares chi square. ML estimates assume the data are drawn from a multivariate normal distribution. PAGE 21 21 If an estimate of the optimal weight matrix is used, for example when the data are multivariate normal and GLS is used, then the sampling covariance matrix for is consistently estimated by 1 1)()1( V N where is the Jacobian matrix. However, if the weight matrix is not correct for the distribution of the data 1 1)()1( V N may not be a consistent estimator of sampling covariance matrix for Browne (1982) has shown that 1 1 1)( )()1( V V VW V NNTN is consistent even when the weight matrix is not correct for the distribution of the data. Using this expression, replacing V by D DV )) () ((1 1 ML and NNTW by a consistent estimator, to calculate standard errors of the ML estimates is referred to as Robust ML. The expression 1 1 1)( )()1( V V VW V NNTN can also be used with other weight matrices and will provide a consistent estimator of the sampling covariance matrix for the parameters. Replacing V by D WLSV ,GLSV and ULSV to calculate standard erro rs of the DWLS, GLS, and ULS estimates, respectively, are called R obust DWLS, Robust GLS, and Robust ULS, respectively. In particular applyi ng this procedure to the DWLS estimator may be attractive. The weight matrix in DWLS is N NTW diagonal. Thus, like ADF it is not based on the normality assumption, but unlike ADF will not be affected by sampling errors in the offdiagonal elements of an estimator of N NTW which is a 1212kkkk matrix and is likely to have substantial sampling error. Browne and Shapiro (1987) have shown that when the weight matrix for ML or GLS is used but the weight matrix is not optimal, under certain conditions1 1)()1( V MLN and 1 1)()1( V GLSN where 11,GLS VD D provides consistent estimates of the PAGE 22 22 standard errors of the estim ates of the elements of but not of the other parameters, even when the random vectors in the model are not normally di stributed. For the factor analysis model the required conditions are that (a) the various latent random vectors for a model are independent, not merely uncorrelated, (b) uncorrelated el ements within a latent random vector are independent, not merely uncorrelated, and (c) other than constraints that the offdiagonal elements of a covariance matrix for a random v ector are zero, the covari ance matrices must be unconstrained. For the factor analysis model the elements of the vectors which are specified to be uncorrelated in the CFA m odel, must be mutually statistically independent. Similarly the elements of the vectors which are specified in the CFA model to be uncorrelated with the elements of must be statistically inde pendent of the elements of In addition, with the exception of the constraints th at offdiagonal elements of are zero and of any constraints that offdiagonal elements of are zero, the elements of and must be unconstrained. The assumptions in Browne and Shapiro imply that Cis a function of the cumulant matrix for the random vectors in the model and of the matrix If the assumptions are not correct standard errors should be calculated by using 1 1 1)( )()1( V V VW V NNTN Table 12 lists the estimation methods investigated in the past robustness studies identified in the Table 11. Of the 47 studies ML was investigated in a total of 43 studies, GLS in a total of 16 studies, and ADF in a total of 22 studies. Es timation methods other than ML, GLS, ULS, DWLS, their robust versions, and ADF were not considered in preparing Table 12. The most popular techniques for estimating th e parameters in SEM are ML and GLS and there are three major assumptions for ML and GLS (Boomsma & Hoogland, 2001): PAGE 23 23 The sample observations are independently distributed. The sample observations are multivariate normally distributed. The hypothesized model is approximately correct. As was mentioned above, real data in social science are ra rely normally distributed. The violation of normality assumption raises a robustness question in terms of distributional violation. However, as sample size increases, the distribution of the estimator approximates a normal distribution, which is why many research ers have investigated the effect of nonnormality in SEM with different sample sizes to determine the minimum required sample size for valid parameter estimates. The restrictive characteristic of the normality assumption motivated the development of the WLS procedure, an asymptotically distribu tionfree (ADF) method. This method does not assume a specific distribution and can produce resu lts which are valid under a wide variety of distributions of the data. Howeve r, these ADF methods face some pr actical problems in that they are computationally expensive and they lack robu stness against small to moderate sample sizes (Satorra, 1990). That is, ADF methods need a su fficiently large sample size. Thus the normality assumption still plays a major role in the pr actice of structural equation modeling. Several issues in regard to robustness of estimation methods against nonnormality in SEM were introduced and reviewed and by Satorra (1990). He summarized several estimation methods (ML, ULS, DWLS, and ADF) and asympto tic robustness of normal theory inferences. Additionally, theore tical and empirical robustnes s to violation of assump tions were explained, a distinction was made between es timation methods that are eith er correctly or incorrectly specified for the distribution of data being analyzed, and a co mparison of ML, ADF, and Robust ML was reported using a real data example about teacher stress (Bentler & Dudgeon, 1996). PAGE 24 24 Moreover, there are factors affecting the eval uation and modification of a model such as nonnormality, missing data, specification error, sensitivity to sample size, etc. In regard to these aspects, general guidelines for covariance struct ural equation modeling ha ve been presented over the past decades (Kaplan, 1990; MacCallum, 1990; Ullman, 2006). Robustness against Nonnormal Distribution The param eter estimates are derived from info rmation in the sample covariance matrix and the weight matrix. When the obs ervations are continuous nonnor mal, the information in the sample covariance matrix or the weight matrix or both may be incorrect. Consequently, estimates based on the sample covariance matrix and the wei ght matrix may also be incorrect. The present study examined nonnormality of the observed co ntinuous variables. The variation in the measured variables is completely summarized by the sample covariances only when multivariate normality is present. If multivariate normality is violated, the variation of the measured variables will not be completely summarized by the sample covariances, so information from higher order moments is lost. In this situat ion, the parameter estimates do remain unbiased and consistent as sample size grows larger, but they are no longer efficient. These re sults suggest that theoretically important problems will occur with normal theo ry estimators such as ML and GLS when the observed variables do not have a multivariate norm al distribution. The im pacts of nonnormality are that chisquare statistics, standard errors, or tests of all parameter estimates can be biased (West et al., 1995). Most of previous studies of violation of the normality assumption are for ML, GLS, and ADF (e.g., Muthen & Kaplan, 1985, 1992; B oomsma, 1983; Hoogland, 1999; Boomsma & Hoogland, 2001; Olsson et al., 2000; Lei & Lomax, 2005). Table 13 provides a simple comparison of ML, GLS, and ADF, taken from Olsson, Foss, Troye, a nd Howell (2000). As shown in Table 13, when the models are incorrectly specified and the data are not multivariate PAGE 25 25 normal, the methods should give different results. With multivariate normal data but a misspecified model, ADF and GLS will be equivalent (Olsson et al., 2000). If the variables are highly nonnormal, it is still an open question whether to use ML, GLS, or ADF. Previous studies have not given a clearcut answer as to when it is necessary to use which estimation method and it is possible that standard erro rs produced by ML, GLS, ULS, DWLS may be underestimated when the observed variables deviate far from normality (LISREL 8 Users Reference Guide). Table 14 shows the degree of nonnormality investigated in the past robustness studies. The present study examined robustness against violations of the assumption of normality using Monte Carlo methods. The degree of nonnormality can be specified by skewness and kurtosis. A brief review of skewness and kurtosis is presented below before discussing the range of nonnormality utili zed in this study. Skewness and Kurtosis in U nivariate Distribution Univariate skewness can be viewed as how much a distribution de parts from symmetry, and univariate kurtosis has been described the extent to which the height of the probability density differs from that of the normal density curv e, that is, kurtosis measures the peakedness or flatness of the probability dens ity function (Casella & Berger, 2002; Harlow, 1985; West et al., 1995). Negative skewness indicates a distribution with an elongated lefthand tail and positive skewness indicates a distribution w ith an elongated righthand tail relative to the symmetrical normal distribution. Zero skewness indicates symmetry around the mean. Negative kurtosis indicates flatness and short tails relative to a normal distribution, whereas positive kurtosis indicates peakedness and long tails relative to a normal curve (Bentler & Yuan, 1999; Harlow, 1985; Olsson et al., 2000; West et al., 1995). PAGE 26 26 Measures of skewness and kurtosis can be defined by using central moments or by using cumulants. The nth central moment of X,n is ])[(n nXE or dxxfxX n n)()( where the first noncentral moment is the mean: EX 1 and )( xfX is the probability density function of a con tinuous random variable, X The second central moment is the variance: 22 2[()](). E XVarXEXEX Kendall, Stuart, and Ord (1987) pr esented several measures of sk ewness and kurtosi s in terms of the moments of a distribution. Skew ness and kurtosis can be defined as 1 and 2 respectively, where 3 2 2 3 1 and 4 2 2 2. In the univariate normal case, where the variables are assumed to have symmetrical, bellshaped distributions, 1 = 0 and 2 = 3. Measures of skewness and kurtosis can also be defined by using cumulants. Formally, the cumulants 1 2 n are defined by the identity in t !/)!/ exp(0 1ntntn n n n n n PAGE 27 27 and it should be observed that there is no 0 (Kendall et al., 1987). The cumulant generating function is simply the logarithm of the moment generating function. The first order cumulant is simply the mean (expected value); the second or der and third order cumulants are respectively the second and third central moments; the hi gher order cumulants are neither moments nor central moments, but rather more complicated polynomial functions of the moments. For example the fourth order cumulant is 2 4423. Kendall, Stuart, and Ord (1987) used cumulants to also define alte rnative measures with the of skewness and kurtosis, respectively 33 1 3/23/2 22 44 22 22 2233 where 1 = 0, and 2 = 0 for univariate normal distributions. Skewness and Kurtosis in Mu ltivariate Distributions Examinations of the skewness and kurtosis of the univariate distributi ons provide only an initial check on multivariate normality. If any of the observed variables deviate from univariate normality, the multivariate distribution cannot be multinormal (West et al., 1995). While univariate measures of skewness and kurtosis are informative regarding the marginal distribution of a variable, it is also of interest to have info rmation on the joint distribu tion of a set of variables (Halrow, 1985). As a result, it is important to examine multivariate measures of skewness and kurtosis developed by Mardia (1970, 1974). The Mardia measures are functions of the third order moments and the fourth order moments, which possess approximate standard normal PAGE 28 28 distributions permitting tests of multivariate skewness and multivariate kurtosis (Harlow, 1985; Mardia, 1970, 1974; West et al., 1995). Let ) ,...,(1 pXXX be a random vector with mean vector ) ,...,(1 p and covariance matrix ) (rs Mardia (1970, 1974) defined the measures of multivariate skewness and kurtosis as 3 1 ,1)}(){( Y X Ep, and 3 1 ,2)}(){( X X Ep, where X and Y are independent and identical random vect ors. Mardia (1974) also suggested the alternative expression for these measures in terms of cumulants and the expressions corresponding to p ,1 and p ,2 are )( 111111 ,1 tsrrstttssrr p and rstuturs p 1111 ,2 where )...( ...1 1 s srr ii denote the cumulant of order ) ,...,(1 sii for the random variable ) ,...,(1 sr rXX where r1, r2, rs are s integers taking values 1, 2, p and ) ()(1)( 11 rs rs. The relations between p ,1 and p ,1 and between p ,2 andp ,2 (Mardia, 1974; Kendall et al., 1987) are p p ,1,1 and )2(,2,2 ppp p PAGE 29 29 In the multivariate normal case, if Xi has a normal distribution then 1,()i = 0 and2,()i = 0 and if X has a pvariate normal distribution p ,1 = 0 and also all fourth order cumulants are zero with the result that p ,2 = 0 (Browne, 1982). Robustness against Small Sample Size In SEM or any other procedure for fitting mode ls, inferences are made from observed data to the model believed to be generating the observa tions. These inferences are dependent in large part on the degree to which the information availa ble in a sample mirrors the information in the complete population. This depends on the obtained sample size. To the extent that samples are large, more information is available and more confidence can be expressed for the model as a reflection of the population process. Thus, sample size has always been an issue in SEM. To get the correct answer to the question: What is the minimum required sample size for each combination of estimation methods, distributiona l characteristics, and model characteristics? many studies have been conducted as shown in the list of the past robustn ess studies in Table 11. The range of sample sizes examined in the pa st robustness studies is shown in Table 15. The typical requirements for a covariance stru cture statistic to be trustworthy under the null hypothesis )( are that the sample observations are independently distributed and multivariate normally distributed in addition to identification of parameters. Identification and independence were assumed and not considered in the present study because they can often be arranged by design. Generally, real data do not meet the assumption of multivariate normality, so sample size turns out to be critic al because all of the goodness of f it statistics and standard errors used in covariance structure an alysis are asymptotic based on the assumption that sample size becomes arbitrarily large. Since this situation can rarely be obtained, it becomes important to evaluate how large the sample size must be in pr actice for the theory to work reasonably well. PAGE 30 30 Different data and discrepancy functions have different robustness prope rties with respect to sample size. Basically, sample size requirement s increase as data become more nonnormal, models become larger, and assumptionfree disc repancy functions are used (Bentler & Dudgeon, 1996; Chou et al., 1991; Hu et al., 1992; Muthen &Kaplan, 1 992; West et al., 1995; Yung & Bentler, 1994). Sample size also affects the likelihood of nonconvergence and improper solution. Nonconvergence occurs when a minimum of the discre pancy function cannot be obtained. Improper solutions refer to estimates that are outside of their proper range, for example estimated variances that are less than zero. The like lihood of nonconvergence decreases with larger sample size; N >200 is generally safer. Nonconvergence rates also decrease with larger factor loadings and larger k/ m ratio (Boomsma & Hoogland, 2001; Marsh et al., 1998), where k/ m is ratio of the number of observed variables to the number of common factors. The likelihood of improper solutions is reduced with a larger k/ m ratio and increased N (Boomsma & Hoogland, 2001; Marsh et al., 1998). Thus sample size plays a crucial role in robus tness, nonconvergence, and improper solutions. Research Summary Characteristics Research summary characteristics determine how the quality of the simulation results is assessed. The following research summary statistic s have been used in the previous robustness studies of the Table 11. Bias of parameter estimates Bias of standard error estimates Standard deviation of parameter estimates Percentage of replications that lead to nonconvergence PAGE 31 31 Percentage of convergent replicatio ns that give improper solutions Rejection rate, mean, and standard de viation of a chisquare statistic p value of the KolmogorovSmirnov test fo r a chisquare distribution Statistics The relative bias of paramete r estimators is defined as i i i iBias ) (, where i is the population value of the ith parameter (0 i ) and i is the mean of the estimates for the ith parameter across the total number of replications. The mean absolute relative bias for parameter estimation is t i iBias t1) ( 1 where t is the number of parameters in the model. According to Boomsma and Hoogland (2001) the mean absolute relative bias for parameter estimation should be less than 0.025. The relative bias of estimators for the standard error of parameter estimates i is defined as ) ( ) () ( )) ((i i i isd sdse seBias where ) (isd is the standard deviation of the estimates for parameter i and ) (ise is the mean of the standard error estimates regarding parameter i across the total number of replications. The mean absolute relative bias for standard error estimation is t i iseBias t1)) (( 1, PAGE 32 32 which is considered to be accep table, if it is smaller than 0.05 (Boomsma & Hoogland, 2001; Harlow, 1985; Hoogland, 1999). Specially, standard errors of parameter estimates are important in many applications to judge the significance of the parameter estima tes. LISREL gives an estimated asymptotic standard error for each parameter. It is well known that these asymptotic standard errors for ML may be incorrect when the observed variable s are not multivariate normal (Browne, 1984; Jonsson, 1997). Findings from Overall Past Robustness Studies Findings from Boomsma (1983) and Muthen a nd Kaplan (1985) sugge st that estimated standard errors did not show bias when using ML, GLS, and ADF with approximately normal data. In nonnormal samples, there is some evidence of negative bias in estimated standard errors when using ML with continuous data (Browne, 1984; Tanaka, 1984) as well as with ADF in sample sizes of 100 (Tanaka, 1984). In studies w ith 400 or more subjects, estimated standard errors also showed some bias, relative to empirical standard errors, with nonnormal samples using ML with categorical data (Boomsma, 1983; Muthen & Kaplan, 1985) and ADF with categorical data (Mut hen & Kaplan, 1984). Hoogland summarized (1999) his conclusions ab out parameter estimators in his research with the following points: An important finding is that th e ML estimator of the model pa rameters (5 factors with 3 indicators per factor, and 4 f actors with 3 indicators per fa ctor) is almost unbiased when the sample size is at least 200. In the case of a small sample the GLS paramete r estimator has a much larger bias than the ML parameter estimator when the model ha s at least twelve obs erved variables. The bias of the ADF parameter estimator increases when the kurtosis increases. With positive kurtosis the bias of the ADF parame ter estimator is larger than that of the GLS parameter estimator. PAGE 33 33 Hoogland made (1999) the followi ng points about the standard error estimators based on the results of his research and the previous studies: The ML and GLS estimators of the standard erro rs are biased when the average kurtosis of the observed variables de viates from zero. The standard errors are underestimated in th e case of a positive average kurtosis and overestimated in the case of negative average kurtosis. The ADF estimation method results in a large un derestimation of the standard errors when the sample size is small relative to the num ber of observed variab les in the model. The robust ML standard error estimator has a smaller bias than the other standard error estimations when the average kurtosis is at l east 2.0 and the sample size is at least 400. From the past robustness studies, th e following points can be set forth: The sample sizes 200 and 400 were often investigated and were minimum sample sizes, though it depends on the models investigated. If the variables are highly nonnormal it is st ill an open question whether to use ML, GLS, ADF, or other methods in regard to bias of standard errors. Robust ML standard errors were rarely inves tigated for the effect of sample size and nonnormality compared with robustness studies in ML. DWLS estimation method was not examined for continuous nonnor mal distribution. Standard errors for Robust DWLS, Robust GLS, and Robust ULS were not investigated in previous studies, though GLS and ML estimatio n methods have quite often been studied. The numbers of replications, 100, 200, and 300 have often been used in previous robustness studies (Table 16). The number of replications is chosen to be reasonable because it is a trade off between precision a nd the amount of information to be handled (Hoogland, 1999). Effects of different scale setting methods on st andard error estimates in robust procedures were not investigated. Direction for the Present Study The present study examined the effect of sa mple size on the accuracy of estimation of standard errors under varying degrees of nonnorma lity with four estimation methods (ML, GLS, Robust ML, and Robust GLS) and three CFA mo dels. The reasons ML, GLS, Robust ML, and PAGE 34 34 Robust GLS were chosen as estimation methods are that standard errors of Robust ML and Robust GLS have not been very thoroughly investigated for effect of sample size, nonnormality, and scalesetting method. One of the main purpos es of the present study is to answer the question: Which estimation method is better to get asymptotically correct st andard errors? A review of the previous studi es of Robust ML standard erro rs provides a clearer idea for the simulation design of the present study. Chou et al. (1991) investigat ed the effect of nonnormality on standard errors varying estimatio n methods (ML, Robust ML, and ADF) with two different versions of the CFA model, a sample size of 400, and 100 replications. For nonnormality condition, they chose four nonnormal conditions: (a) symmetric with equal negative kurtosis, (b) symmetric with unequal kurtosis, (c ) unequal negative skewness with zero kurtosis, and (d) unequal skewness with unequal kurtosis to study the behavior of the robust standard errors. For the CFA model with two factors and three indicators per factor, there were two different versions. The first version contai ned 13 parameters, six factor loadings, six measurement error variances, and one factor co variance. The second version only included the measurement error variances and the factor covari ance as free parameters while fixing the factor loadings at the true values (C hou et al., 1991). In case of the first version of the model, for nonnormality conditions (a) and (c), all three types of estimated standard errors were very similar, mean ADF and robust standard errors were closer to the expected values. Under condition (b) and (d), all three types of estimated standard e rrors were negatively bias ed with robust standard errors performing slightly better than ADF and ML. Similar conclusions were drawn from the results for the second version of the CFA model. Robust stan dard errors provided better estimates than both ML and ADF, although some robus t standard errors were not as close to the PAGE 35 35 expected values for the second version of the mo del as they were for the first version of the model. Finch et al. (1997) studied the effect of sa mple size and nonnormality on the estimation of mediated effects in latent variable models. They investigated the mediated effects varying the structural regression coefficients in a model with three factors with three items per factor and concluded that the magnitude of the observed bias for estimating standard errors varied little across differing ratios of direct to indirect effects for ML, Robust ML, and ADF. First, ML, ADF, and Robust ML standard errors were exam ined to determine the range of nonnormality conditions under which these standard errors ar e accurate, varying sample size (150, 250, 500, and 1000), the population values for the hybrid model parameter, and the degree of nonnormality (normal [0,0], moderate nonnormality [2,7], and extreme nonnormality [3, 21]) with 200 replications. The partially mediated struct ural model included th ree factors and three indicators per factor and had four different mediated effect size s. For Robust ML, weaker effects of nonnormality on the standard e rrors were observed. With regard to the robust estimates of the standard error of the indirect effect, there wa s minimal bias under normality, and bias increased under severe nonnormality. Weak effects of samples size were also observed with some decrease in relative bias associated with la rger sample sizes. Under nonnormality, the Robust ML standard errors performed much better at all sample sizes. For all three methods of estimating standard errors, the magnitude of the observed bias varied li ttle across differing ratios of direct to indirect effects. The pattern of bias in the standard erro rs of direct and indirect effects was also not influenced by varia tion in the population values of the factor loadings. In the second study, they extended the generality of the findi ngs by examining bias under conditions in which the degree of nonnormality differed across observed (manifest) variables. The population values PAGE 36 36 of skewness and kurtosis were specified in th e three distributional c onditions. In the first distributional condition, the nine measured variab les were mildly to moderately nonnormal; in the second the nine manifest variables were mode rately to severely nonn ormal; and in the third the measured indicators of the final outcome c onstruct were severely nonnormal, whereas the other six variables were normally distributed. They adopted one specific latent variable model for the second study with 400 replications and the sa me conditions for sample size that they used in their first study. Under the first distributiona l condition, there were no appreciable effects of sample size on the estimated standard errors of the indirect effect using ML or Robust ML. Under the second distributional condition, Robust ML estimated standard errors of the structural coefficients showed a general tendency to beco me more accurate with increasing sample size. Under the third distributional condition, there we re large effects of nonnormality on the ML and Robust ML standard errors of the latent variab le coefficients. The Robust ML standard errors provided more accurate estimates of sampling variability than the normal theory standard errors as nonnormality increased. The Robust ML standard errors were also more accurate than the ADF standard errors under all di stributional conditions at the sm aller sample sizes (i.e., 150 or 250). Hoogland (1999) investigated the effect of four factors on rela tive bias of standard error estimates: sample size, size of factor loadings, model complexity, and number of indicators per factor. Model complexity was varied as follows: four factors with three items, four factors with four items, and five factors with three items pe r factor, and one hybrid model with five factors and three items per factor. For N = 200, the mean absolute relative bias of the Robust ML standard error estimator was larger for each parameter type when the indicators were poor measures of the latent variables. For N = 400, the mean absolute relative bias of the Robust ML PAGE 37 37 standard errors was also larger for smaller factor loadings. For N 800, the effect of the size of the factor loadings on the mean absolute relative bias of the Robust ML standard error estimator was mostly small. In case of model complexity, th ere was hardly an effect on the mean absolute relative bias of the Robust ML standard errors as the m odel complexity increases for N 200. The mean absolute relative bias of the Robust ML standard errors decreased somewhat as the number of indicators per factor increased when the sample size was small. The difference between the CFA model and the hybrid model was negligible for the mean absolute relative bias of the Robust ML standard errors. According to the previous studies of Robust ML (Chou et al., 1991; Finch et al., 1997; Hoogland et al., 1999), Robust ML generally showed be tter performance on standard errors rather than other estimation methods investig ated over all sample sizes and degrees of nonnormality. For Robust GLS, no Monte Carlo study has been conducted to investigate the effect of sample size on accuracy of estimated standard erro r. Thus, a Monte Carlo study of the effect of sample size on accuracy of estimated standard errors under varying degrees of nonnormality with three CFA models using ML, GLS, Robust ML, and Robust GLS will provide more precise guidelines in regard to sample size in SEM fo r a nonnormally distributed data and a complex model. The scale of measurement for a factor can be changed without affecting the fit of model to the data, which is called scale indeterminacy. The data cannot determine the values of the parameters due to scale indeterminacy and therefore this is an identifi cation problem. Arbitrary restrictions are required to rem ove scale indeterminacy. A parameters standard error is sensitive to how the model is identified (i.e., how scale is set). The different scalesetting methods to PAGE 38 38 identify a model may yield different standard errors and reach differe nt conclusion about a parameters significance although eq uivalent models on the same data are tested (Gonzalez & Griffin, 2001). There are two common methods to remove scale indeterminacy: 1) Factor variances are set equal to one, and 2) Factor lo adings are set equal to one. Ch anging the method for setting the scale of a factor multiplies the factor by a constant. Estimates of parameters related to the factor are affected as one would expect for a scale change. For exampl e if the method of setting the scale multiplies the factor by k, then the factor loading is multiplied by 1. k However, the standard errors of the factor loading are not simply multiplied by 1. k As a result it is possible that the method for setting the scales may affect the accuracy of estimating of standard errors for factor loadings and this questi on has not been investigated in the previous robust studies. Note that the scalesetting method does not affect the si ze of the residual variances at all, whereas the scalesetting method affects the size of the f actor loading, factor variances, and factor covariances. PAGE 39 39 Table 11. Overview of previous robustne ss studies against sample size and nonnormality Paper Number Author(s) Year Sample size Nonnormality 1 Anderson & Gerbin g 1984O 2 Andreassen Lorentzen & Olsson2006 O 3 Babakus Fer g uson & Joresko g 1987O O 4 Baldwin 1986O 5 Beardeu Sharma & Teel 1982O 6 Beauducel & Herzber g 2006O 7 Benson & Fleishman 1994O O 8 Bentler & Yuan 1999O 9 Boomsma 1983O O 10 Brown 1990 O 11 Browne 1984 O 12 Chou Bentler & Satorra 1991O O 13 Curran West & Finch 1996O O 14 Dolan 1994O O 15 Ethin g ton 1987 O 16 Fan Thom p son & Wan g 1999O 17 Finch J. F. 1992O O 18 Finch West & Mackinnon 1997O O 19 Gallini & Mandeville 1984O 20 Gerbin g & Anderson 1985O 21 Harlow 1985O O 22 Hau & Marsh 2004O O 23 Henl y 1993O O 24 Hoo g land 1999O O 25 Hu Bentler & Kano 1992O O 26 Huba & Harlow 1987 O 27 Jaccard & Wan 1995O O 28 Jackson 2001O 29 Ka p lan 1989O 30 Lee Poon & Bentle r 1995O O 31 Lei & Lomax 2005O O 32 Muthen & Ka p lan 1985 O 33 Muthen & Ka p lan 1992O O 34 Nevitt & Hancock 2000O O 35 Nevitt & Hancock 2001O O 36 Nevitt & Hancock 2004O O 37 Olsson Foss Tro y e & Howell 2000O O 38 Pin g 1995O 39 Potthas t 1993O O 40 Redd y 1992O 41 Sharma Durvasula & Dillon 1989O O 42 Sivo Fan Witta & Willse 2006O O 43 Tanaka 1984O O 44 Tanaka 1987O 45 Yan g Jonsson 1997O O 46 Yuan & Bentle r 1998O O 47 Yun g & Bentle r 1994O O PAGE 40 40 Table 12. Estimation methods investig ated in the past robustness studies Pa p er numbe r Estimation methods 1 ML 2 ML GLS ADF 3 ML 4 ML 5 ML 6 ML DWLS* 7 ML ADF 8 ML 9 ML 10 ML 11 ML ADF 12 ML ADF Robust ML 13 ML ADF 14 ML ADF GLS 15 ML ULS 16 ML GLS 17 ML 18 ML ADF Robust ML 19 ML 20 ML 21 ML ADF 22 ML ADF 23 ML GLS ADF 24 ML GLS ADF Robust ML 25 ML GLS ADF 26 ML GLS ADF ULS 27 ML GLS ADF 28 ML 29 ML GLS 30 GLS 31 ML GLS 32 ML GLS ADF 33 GLS ADF 34 ML 35 ML 36 ML ADF 37 ML GLS ADF 38 ML ADF 41 ML GLS 43 ML ADF 44 ML 45 ML ADF 46 ADF 47 ML GLS ADF In this article, DWLS means WLSMV (W LS means and variance adjusted) which is weighted least square parameter estimates us ing a diagonal weight matrix with standard errors and meanand varianceadjusted chisquare test statistic that use a full weight matrix. PAGE 41 41 Table 13. When are ML, GLS, and ADF equivalent? Model/Distribution Normal Nonnormal Correct model ML GLS ADF ML GLS asymptotically equivalent asymptotically equivalent Misspecified model GLS ADF No equivalence asymptotically equivalent PAGE 42 42 Table 14. Degree of nonnormality exam ined in the past robustness studies Paper number Degree of nonnormality (Skewness, Kurtosis) 2 Nonnormal real data 3 Combinations of skewness = 0.0, 0.5, 1.5 7 (1.0, 2.0), (2.0, 6.0) 9 Various combinations of skewness and kurtosis 10 (0.0, 0.0), (0.25, 3.75), (1.0, 3.75), (1.75, 3.75) 11 Multivariate normal distribution vs. rescaled multivariate chisquare distribution with 2 degree of fr eedom and all marginal relative kurtosis coefficients equal to 3.0. 12 Various combinations of skewness and kurtosis 13 (0.0, 0.0), (2.0, 7.0), (3.0, 21.0) 14 (0.0, 3.0), (0.0, 1.0) 15 Combinations of skewness = 0.0, 1.5, 1.5 17 (0.0, 0.0), (2.0, 7.0), (3.0, 21.0) 18 (0.0, 0.0), (2.0, 7.0), (3.0, 21.0) 19 Possible combinations with sk ewness = 0.0, 1.0, 2.0 and kurtosis = 1.0, 0, 1.0, 3.0, 6.0 with the restriction of (skewness)2 < 0.629576(kurtosis) + 0.717247 21 13 combinations of skewness and kurtosis 22 (0.0, 0.0), (0.5, 0.5), (1.0, 1.5), (1 .5, 3.25), Balanced ( 0.5, 0.5) 23 Multivariate normal, elliptical, nonelliptical 24 11 combinations of skewness and kurtosis 25 7 conditions of the range of kurtoses 0.010 ~ 0.010, 0.502 ~ 3.098, 0.502 ~ 3.908, 0.262 ~ 0.989 4.658 ~ 6.827, 4.635 ~ 9.659, 3.930 ~ 20.013 26 Nonnormal real data 30 Continuous and polytomous variable 31 (0.0, 0.0), (0.3~0.4, 1.0), (0.7, >3.5) 33 (0.0, 0.0), (0.74, 0.33), (1.22, 0.85), (2.03, 2.90), (0.0, 2.79), (0.0, 1.3) 34 (0.0, 0.0), (2.0, 7.0), (3.0, 21.0) 35 (0.0, 0.0), (2.0, 7.0), (3.0, 21.0) 36 (0.0, 0.0), (0.0, 6.0), (3.0, 21.0) 37 11 different levels of kurtosis in the latent variables ranging from mild (1.2) to severe peakedness (+25.45) 39 Negative kurtosis(1.9, 1.12), zero skewness and kurtosis(0.0, 0.0), positive kurtosis(0.0, 2.79), high skewness and kurtosis(2.52, 5.8) 41 Combinations of skewness = 1.0, 0.0, 1.0 and kurtosis = 2.0, 4.0, 6.0 43 Zero skewness and various kurtosis 45 After data generation and estima tion procedure, checking the degree of nonnormality (kurtosis and skewness with pvalue) PAGE 43 43 Table 15. Range of sample size exam ined in the past robustness studies Paper number Sample sizes 1 50, 75, 100, 150, 300 2 1042 3 100, 500 4 100, 200 5 25, 50, 75, 100, 500, 1000, 2500, 5000, 10000 6 250, 500, 750, 1000 7 200, 400 8 60, 70, 80, 90, 100, 110, 120 9 25, 50, 100, 200, 400, 800 10 500 11 500 12 200, 400 13 100, 200, 500, 1000 14 200, 300, 400 15 500 16 50, 100, 200, 500, 1000 17 100, 200, 1000 18 150, 250, 500, 1000 19 50, 100, 500 20 50, 75, 100, 150, 300 21 200, 400 22 50, 100, 250, 1000 23 75, 150, 300, 600, 1200, 2400, 4800, 9600 24 200, 400, 800, 1600 25 150, 250, 500, 1000, 2500, 5000 26 257 27 175, 400 28 50, 100, 200, 400, 800 29 100, 500 30 100, 200, 500 31 100, 250, 500, 1000 32 1000 33 500, 1000 34 100, 200, 500, 1000 35 100, 200, 500, 1000 36 n:q = 1:1, 2:1, 5:1, 10:1 (n: sample size, q: number of parameter estimates 37 100, 250, 500, 1000, 2000 38 100, 300 39 500, 1000 40 100, 500 41 150, 300, 500 42 150, 250, 500, 1000, 2500, 5000 43 100, 500, 1500 44 50, 1200 45 100, 200, 400, 800, 1600, 3200 46 150, 200, 300, 500, 1000, 5000 47 250, 500 PAGE 44 44 Table 16. Number of replicatio ns in the past robustness studies Paper number Number of replications per each condition 3, 4, 23, 24 300 8, 31 500 9, 21 100, 300 10, 12, 14, 19, 20, 38, 39 100 11, 43 20 13, 16, 17, 18, 25, 28, 34, 35, 42, 47 200 15, 40 50 18 400 22 100, 400, 1000, 2500 32 25 33 1000 36 2000 37 16500 41 120 45 600 PAGE 45 45 CHAPTER 2 DESIGN OF MONTE CARLO SIMULATION STUDY Monte Carlo sim ulation relates to or involve s the use of random sa mpling techniques and often the use of computer simulation to obtain ap proximate solutions to mathematical or physical problems especially in terms of a range of valu es each of which has a calculated probability of being the solution (Fan et al ., 2002; Paxton et al., 2001). Monte Carlo simulation simulates the sampling process from a defined population repeat edly by using a computer instead of actually drawing multiple samples to estimate the sampling distributions of the events of the interest. Data Generation There are several methods for generating non normal multivariate data for simulation (Mattson, 1997; Reinartz et al., 2002; Vale & Ma urelli, 1983). In some of the past robustness studies of SEM the performance of the Vale a nd Maurelli data generation method was assessed. This method has received much attention due to its flexibility (Harlo w, 1985; Mattson, 1997). The Vale and Maurelli method is an extensi on of Fleischmans power function method for univariate case (Vale & Maurelli, 1983). Fleishmans Power Transformation Method Fleishman (1978) introduced a method to simulate univariate nonnor mal conditions with desired degrees of skewness and kurtosis. This method uses polynomial transformation to transform a normally distributed va riable to a variable with sp ecified degrees of skewness and kurtosis. The polynomial transformation shown in Fleishman (1978) takes the form: 23,YabZcZdZ where Y is the transformed nonnormal variable with specified population sk ewness and kurtosis, Z is a unit normal variate (i.e., a normally distri buted variable with population mean of zero and variance of one), and a, b, c and d are coefficients needed for transforming the unit normal PAGE 46 46 variate to a nonnormal variable with specified degrees of popul ation skewness and kurtosis. Of the four coefficients, ca Vale and Maurelli Data Generation Method Vale and Maurelli (1983) presented a procedure for generating multivariate nonnormal data that is related to Fleishmans power tr ansformation method. By using the Fleishmans power transformation method, two normal variates, 1Z and 2Z can be transformed to two nonnormal variables 1X and2X, each with known skewness and kurtosis: 23 11111111, X abZcZdZ and 23 22222222. X abZcZdZ Once the degrees of skewness and kurtosis are known, the coefficients are available. In addition to these coefficients, the m odeled population correlation between the two nonnormal variables 1X and2X can be specified as12R. Once 12Ris set, and the Fleishman coefficients for the specified skewness and kurtosis co nditions of the two variables (1X and2X) are available, Vale and Maurelli (1983) showed that the following relation exists: 23 1212121212 12 12(339)(2)(6), R bbbddbddccdd where is the correlation between the two normal variates, 1 Z and2 Z and it is called intermediate correlation. All elements of the equation above are known except the bivariate intermediate correlation, This bivariate intermediate correlation coefficient, must be solved for all possible pairs of variable s involved. These intermediate correlation coefficients are then assembled in proper order into an intermediate correlation matrix and multivariate normal data are generated by using PAGE 47 47 y Fz where FF ~, MVNz0Iis 1. k In three previous robustness studies with Robust ML, Chou et al. (1991), Finch et al. (1997), and Hoogland (1999) used the Vale a nd Maurelli method for data generation and problems were not reported. The Vale and Ma urelli method based on Fleishman formulas was used to generate the nonnormal data in the present study. The simulation study had 384 cells (the combination of four sample sizes, four distributional characteristics, th ree models, four estimation met hods, and two methods for setting the scale). Estimation Methods As stated in Chapter 1, four estimation methods were studied: Maximum Likelihood method (ML) Robust Maximum Likelihood method (Robust ML) Generalized Least Squares method (GLS) Robust Generalized Least Squares method (Robust GLS) Sample Sizes As stated in Chapter 1, 200 was chosen as the minimum sample size to avoid problems of nonconvergence and improper solution. The sample sizes that were stud ied are 200, 400, 800, and 1200. The sample size, 200 is considered to be small, 400 and 800 are considered to be medium, and 1200 is judged to be large. The sample sizes 200 and 400 were often investigated in past robustness studies and are al so often in this range in obser vational research according to Table 11, Chou et al. (1991), and Hoogland (1 999). The sample size of 1200 was chosen to investigate whether there will be important improvements from sample size 800 to 1200. PAGE 48 48 Distributional Characteristics The degrees of nonnormality are reflected by various skewnesses and kurtoses of the variables. In previous studies, several classifications for categorizing the degree of nonnormality for simulation were introduced (Harlow, 1985; Curran et al., 1996; Muthen & Kaplan, 1985, 1992; Potthast, 1993; Hoogland, 1999). In the pr esent study, four different levels of nonnormality were selected to reflect distributions thought to be common in the social sciences: Zero skewness and zero kurtosis (normal distribution) Zero skewness and equal negative kurtosis (plat ykurtic ): Each variables kurtosis is 1.15. Zero skewness and equal positive kurtosis (l eptokurtic): Each variab les kurtosis is 3. High equal skewness and high equal kurtosis: Ea ch variables skewness is 2 and kurtosis is 6. The baseline case of zero skewness and zero kurtosis affords opportunity for comparison and contrast with zero skewness but negative kurtosis (1.15) re presenting almost uniform distribution (platykurtic), a peaked (leptokur tic) distribution with zero skewness but positive kurtosis (3), and a distribution with both high skewness (2) and kurtosis (6). The effects of nonnormality on standard errors have not been inve stigated in Robust GLS. Thus, these nonnormal distributional conditions are st udied in the present study. Model Characteristics Two factors with three indicator s per factor, three f actors with three indi cators per factor for hybrid model analysis, four factors with three or four indicators per fa ctor, and five factors with three indicators per factor for CFA and hybrid models we re studied in previous robustness studies of Robust ML (Chou et al., 1991; Finch et al., 1997; Hoogland, 1999).Three CFA models were chosen as population models in the present study. The CFA model 3F3I consists of three factors with three indicators per factor, the CFA model 3F6I consists of three factors with six PAGE 49 49 indicators per factor, and the CFA model 6F3I includes six factors with three indicators per factor. Threefactor and sixfactor CFA models were chosen, because these numbers of factors cover the range typically found in CFA studies. For the models 3F3I and 6F3I, the three factor loadings for each factor were 0.6, 0.7, and 0.8. For the model 3F6I, the six factor loadings for each factor were 0.6, 0.6, 0.7, 0.7, 0.8, and 0. 8. In addition, for each population model the factor correlations were 0.3, because this is most often chosen in the previous studies and also often found in CFA studies. This study focuses on increas ing the number of fact ors and indicators for model characteristic and exclude s the effects of factor loadings and factor correlation. The average of the factor loadings in each CFA model is 0.7. Analytical Model Characteristics The data matrix typically used for computati ons in SEM programs is a covariance matrix, and also the general rule is that a covarian ce matrix should be analyzed. However, in many social/behavioral sciences applications the scales of the observed variable s are often arbitrary or irrelevant. Therefore in many cases a correlation matrix is anal yzed instead of a covariance matrix for convenience and interpretational purpose s. The analysis of a correlation matrix may cause the following problems (Cudeck, 1989; Jreskog & Srbom, 1996): (a) modification of the model being analyzed, (b) incorrec t values of test statistic, and (c) incorrect standard errors. Boomsma (1983) concluded that the analysis of correlation matrices lead to imprecise values for the parameter estimates in a struct ural equation model. She specifically found a problem with the estimation of standard erro rs for the parameter es timates. Browne (1982) pointed out that parameter estimates and chisquare statistic are unaffected when a model is scale invariant under the analysis of a correlation matrix. Suggested corr ections for the standard errors when correlations for standardized coefficien ts are used have been recommended by Browne PAGE 50 50 (1982). For these reasons, a covariance matrix was analyzed and the same unit of observed variables across factors was assumed in this study. The scale of measurement for a factor can be changed without affecting the fit of the model to the data, which is called s cale indeterminacy. The data ca nnot determine the values of the parameters due to scale indeterminacy and therefore this is an identifi cation problem. Arbitrary restrictions are required to remove scale in determinacy. Two methods to solve the scale indeterminacy were investigated: factor variances are set equal to one, and factor loadings are set equal to one. In all conditions th e first observed variable has its loading set equal to one for the scalesetting method: factor loadings equal to one. Number of Replications The number of replications determines the a ccuracy with which sampling distributions of parameters estimates are approximated. The larger number of replications the more closely the sampling distributions will be approximated. The previous robustness studies on SEM have often used 100, 200, and 300 replications, according to Table 16. Fourhundred replications is the largest value in the past robustn ess studies of Robust ML (Finch et al., 1997). In the present study, the number of replications, 1000 was c hosen to obtain precise simulation results. Research Summary Statistics In the present study, the following research summary statistics were used: Empirical standard error The empirical standa rd error is the standard deviation of the parameter estimates. Standard errors LISREL gives estimates of th e asymptotic standard errors of parameter estimates in each sample. These can be compared with the empirical st andard errors to see if standard errors are overe stimated or underestimated. Standard error ratio More detailed informa tion about the standard errors can be obtained by studying standard error ratio, i.e., the ratio s of the standard errors to the empirical standard error (Jonsson, 1997). PAGE 51 51 Standard error ratios were used instead of m ean absolute relative bias for standard error estimation because mean absolute relative bias cannot detect underestimation or overestimation of standard error estimates nor can it detect variation in bias in the standard errors across different parameters and/or parameter values. Thus standard error ratio estimates were compared for all combinations of sample size, degree of nonnormality, model, estimation method, and scalesetting method. In addition, although all methods provide cons istent estimates of model parameters, at some sample sizes estimation bias may vary over estimation methods and there may be an interactive effect of estimation method with one or more of the other factors in the study. For this reason, the bias of the parameter estimation was also investigated. For the reasons cited in regard to standard errors, parameter estimates were compared instead of mean absolute relative bias for parameter estimation. Thus, parameter es timates were compared for all combinations of sample size, degree of nonnormalit y, model, estimation method, a nd scalesetting method. Ways to Summarize and Present Results Because Monte Carlo simulations can produce a great deal of data, data summary and presentation of results are importa nt features in simulation studies. There are three main ways to present results: descriptive, graphical, and inferential (Pax ton et al., 2001), though clearly inferential procedures can be used to supplement descriptive and graphical presentation of results. Descriptive statistics (i.e., m ean, variance, mean relative bi as, correlation, etc.) present information concisely and simply. Graphical tech niques include using figu res such as box plots, scatter plots, or power curves. Inferential t echniques allow for the formal testing of the significance of the design factors as well as the computation of various effect size estimates. The inferential approach ha s the following advantages: PAGE 52 52 The statistical significance and importance of th e main and interaction effects of the factors in the design can be determined, and the presentation can focus on effects that are both statistically significant and important effects. Designation of effects as si gnificant and important is ba sed on objective criteria. Selecting effects that are si gnificant and important for presentation can allow for concise summarization of a large amount of data. In the present study, inferential techniques such as WelchJames test and eta squared statistic were used because of the advantages above. PAGE 53 53 CHAPTER 3 MONTE CARLO STUDY RESULTS A Monte Carlo sim ulation study was conducted to examine the effects of five factors on parameter estimates and standard error esti mates for CFA model. These factors were: 1. CFA model complexity, with th ree levels: a model with three factors and three indicators per factor (Model 3F3I), a model with three factors and six indicators per factor (Model 3F6I), and a model with six factors and three indicators per factor (Model 6F3I). 2. Sample size with four levels: 200, 400, 800, and 1200. 3. Distribution with four levels identified by the skewness 1 and kurtosis 2 of the distribution used for all indicators within a model: 12, (0, 0), (0, 1.15), (0, 3) and (2, 6). The (0, 0) distribution is a standard normal distribution and the multivariate distribution is also normal. The (0, 1.15) di stribution is a shorttailed distribution, the (0, 3) distribution is a longtailed distribution, and the (2, 6) distribution is a skewed and longtailed distribution. 4. Estimation method with two levels (GLS and ML) that could affect estimates and four levels (GLS, Robust GLS, ML, and Robust ML) that could affect standard errors. The robust procedure and a nonrobust procedure (GLS and Robust GLS; ML and Robust ML) within the same estimation method produce exactly the same parameter estimates, whereas GLS, Robust GLS, ML, and Robust ML generate different standard error estimates. 5. Scalesetting method with two le vels: setting factor variances to unity and setting factor loadings to unity. The first three factors are betw eensubjects factors. For each co mbination of levels of the betweensubjects factors 1000 re plications were conducted. Preliminary ANOVA Tests For Model 3F3I, for example, the population valu e for three of the fact or loadings is 0.6. Evidence that the results are similar for the thr ee indicators for which the factor loading is 0.6 would permit simplification of the analyses and re porting, as an analysis of the estimates and standard errors for only one of th e three parameters would be suffici ent. This approach can also be applied to the three indicators for which the fact or loadings are 0.7, to the three indicators for which the factor loadings are 0.8, and also to ot her parameters such as factor correlations and PAGE 54 54 residual variances with the same population value. Due to the likelihood that estimates have a nonzero sampling correlation for a particular pa rameter, repeated measures ANOVA tests were conducted to investigate differences among the m eans of parameter estimates with the same population value for each of the combinations of the factors; ANOVA test s were also conducted for the means of standard error estimates. The results of the ANOVA tests for parameter estimates are presented in Table 31 and 32. The results are the frequency of significance differences among mean parameter estimates per 16 combinations of distribution and sample size within each combination of parameter value, model, and estimation method. In all cases the frequency was three or smaller. Even when the ANOVA test was significant, the means of the parameter estimated were very similar. For example, when the population factor loading was 0.6 in Model 3F3I, GLS was used, and factor variances were set equal to one, differences amon g the three estimates were significant in three of the 16 combinations. The smallest p value [ F (1.99, 1989.81] > 3.51, p = 0.0303] occurred for the sample size of 200 in combina tion with the (0, 1.15) distribu tion. Mean parameter estimates for this condition are presented in Table 33. Inspection of the resu lts in Table 33 indicates that the means of parameter estimates with the same population value of 0.6 are similar although the ANOVA test resulted in significant differences. The results of the ANOVA tests for standard e rror estimates are presented in Table 34 and 35. The number of significant te sts of mean equality of standard error estimates per 16 combinations of distribution and sample size in e ach combination of parameter value, model, and estimation method is two or fewer. The means of st andard error estimates for parameters with the same population value were similar even when the F test was significant. For standard error estimates of factor correlation in Model 6F3I, Robust ML, and setting factor variances to one as PAGE 55 55 an example, the numbers of significance are tw o in Table 36. The combination of a (2, 6) distribution and a 1200 sample size resulted in the smallest p value [F (13.64, 11033.76) > 2.34, p = 0.0034]. Mean parameter estimates for this combination of distribution and sample size are reported in Table 36. Inspection of the results in Table 36 indica tes that the means of standard error estimates with the same population value of 0.3 are quite similar, differing only at the thousandths place even though the ANOVA test resulted in a significant difference. In sum, the results indicate that estimation by a particular estimation method (e.g., ML) of the same parameter type (e.g. a factor loading), with the same value results in mean estimates and standard errors that are very similar. Therefore, in the following section, results are presented for just one of the parameters of the same type and value. WelchJames Tests and Eta Squared Statistics Using results from Johansen (1980), Keselm an et al. (1993) show ed how to apply the WelchJames (WJ) test to repeated measures designs and presented evidence that this method provides better control of Type I error rates than does MANOVA for repeated measures when covariance matrices are unequal ac ross levels of the betweensubjects factors. In the present study covariance matrix heterogeneity for the para meter estimates would certainly occur for the various sample sizes and therefore the WJ test was used to test hypotheses about parameter estimates. In testing hypotheses using the WJ method, a saturate d means model was used and all possible main effects and interactions were tested. Consider the linear model (Lix and Keselman, 1995): Y=X + where Y is an N p matrix of scores on p dependent variables or p repeated measurements, N is the total sample size, X is an N r design matrix consisting entirely of zeros and ones with PAGE 56 56 rank(X) = r is an r p matrix of population means, and is an N p matrix of random error components. Let jY ( j = 1, r ) denote the nj p submatrix of Y containing the scores associated with the nj subjects in the jth group (cell). In the present it is typically assumed that the rows of jY are independently and normally distributed, with mean vector j and variancecovariance matrix j [i.e., N (jj,)], where jj1jp=( ), the jth row of and jj ( j j ). Specific formulas for estimating jand as well as an elaboration of jY are provided in Lix and Keselman (1995). In the present study, there were three betw eensubjects factors: sample size with four levels, distribution with four levels, and mode l with thee levels. Each combination of these factors constitutes a cell. Conse quently there were 48 cells. Each cell was replicated 1000 times. So 48000.N In each WJ analysis, estimation method was the only repeated measure. For analysis of bias of parameter estimates, ML and GLS estimates were analyzed. So 2. p For analysis of standard errors, ML, Robust ML, GLS, and Robust GLS estimates were analyzed. So 4. p In all WJ analyses all possible main effect s and interactions of betweensubjects factors were included in the model. So48. r The general linear hypothesis is :=,0HR 0 where T=,ROU O is a Odfr matrix which controls cont rasts on the independent groups (betweensubjects) effect(s) with rank(O) = Odfr, U is a U p df matrix which controls contrasts on the withinsubj ects effect(s) with rank(U) = Udfp, is the Kronecker or direct product function, and superscript, 'T' is the tr anspose operator. For multivariate independent PAGE 57 57 groups designs, U is frequently an identity matrix of dimension p (i.e., pI). The R contrast matrix has ()()OUdfdf rows and rp columns. In the equati on of the hypothesis above TT 1r=vec()=[] In other words, is the column vector with rp elements obtained by stacking the columns of T. The zero (0) column vector of the equa tion of the hypothesis is of order OUdfdf [See Lix & Keselman (1995) for illustrative examples]. The generalized test statistic given by Johansen (1980) is TT1 WJ =()()(),TR R RR where estimates and 11rr =diag [/n/n], a block matrix with diagonal elements jj /n. This statistic, divided by a constant, o (i.e., WJT/ o ), approximately follows an F distribution with 1=OU dfdf and 211=(+2)/(3A) where 11=+2A(6A)/(+2) o The formula for the statistic A is gi ven by (Lix and Keselman, 1995) r TT12TT12 jj j j=11 A=[tr{()}+{tr(())}]/(n1) 2 RR RRQ RR RRQ, where tr is the trace operator and jQ is a symmetric block matrix of dimension rp associated with jX, such that the (s, t) th diagonal block of jp=QI if s = t = j and is zero matrix (0) otherwise. The scalesetting method is known to affect the si ze of the factor loadi ng, factor variances, and factor covariances. As an example, consider the effect of scalesetting method on the factor loadings. Recall that the factor loadings were 0.6, 0.7, or 0.8 when the scale was set by specifying factor variances equal to unity. Wh en the scale was set by specifying unit factor loadings, the loading for the first factor (i.e., th e factor that had a 0.6 loading when unit factor variances were specified) was set equal to one Thus the other two factor loadings became PAGE 58 58 0.70.6 and 0.80.6, respectively. Including scalesetti ng method as a factor in the analysis would results in artificial effect of scale setting method on the estimates and perhaps artificial interactions of the other factor s with scale setting method. Ther efore the data obtained by setting factor variances equal to one and factor loadings equal to one were analyzed separately. However, the scalesetting method does not affect the size of the residual variances at all and the results obtained by setting factor variance equal to one and fact or loadings equal to one are exactly the same. Thus only the data obtained by setting factor varian ces equal to one was analyzed and presented in the present study. For the data when factor variances were set equal to one, seven WJ analyses were conducted: three were for results estimating the 0. 6, 0.7, and 0.8 factor loadings, respectively; one was for the results estimating the 0.3 factor correlation, and three were for the results estimating the 0.64, 0.51, and 0.36 residual varian ces, respectively. For the data when factor loadings were set equal to one, four WJ analyses were conducted: two were for results estimating the 0.70.6 and 0.80.6 factor loadings, respectively, one was for the results estimating the 0.36 factor variance, and one was for results estimating the 0.108 factor covariances. Residual variances are not affected by the scalesetting method, so analyses of the residual variance estimates were not conducted again. Because of the large sample size, it was e xpected that many effects would be significant even if the effects were quite small. To a ddress this problem an effectsize measure was adopted. An effectsize measure is a standard ized index, estimates a parameter that is independent of sample size, a nd quantifies the magnitude of th e difference between populations or the relationship between explanatory and resp onse variables (Olejnik & Algina, 2003). Two broad categories of effect size ar e standardized mean differences and measures of association or PAGE 59 59 the proportion of variance explaine d (Olejnik & Algina, 2003). In the present study eta squared was used as an effect size. Eta squared is the proportion of the total su ms of squares that is attributed to an effect: Total EffectSS SS2, where EffectSS is the sum of squares for the effect for wh ich the effect size is being estimated and TotalSS is the total sum of squares. These sums of squares were calculated by using an ANOVA for repeated measures. The coefficient 2 was used rather than2 (Omega squared is an estimate of the dependent variable population variability accounted for by the independent variable.) Because the latter is based on the assumption of covariance homogeneity and compound symmetry but the former is not. Compound symmetry is an assumption about the variances and covariance of the repeated measures and al l correlations are equal under compound symmetry. Effects on Parameter Estimates Tables 37 and 38 contain results for estim ation when the scale was set by specifying factor variances equal to one. Table 37 presents the F statistics for the WJ tests and Table 38 presents eta squared statistics for all effects with significant F statistics in Table 37. Inspection of the results in Table 38 indicates that most ef fects were quite small, with only the effect of estimation method on estimates of the residual variance accounting for as much as approximately 5% of the total sum of squares. To select effects for further examination the following rules were applied. For each column, effects which did no t enter into higher orde r interactions and for which eta squared was at least 0.01 were selected. PAGE 60 60 Applying the rule, no factors or interactions had important effects on estimation of the factor correlation and only estimation procedur e had an important effect on estimates of the factor loadings. This effect o ccurred only when the popul ation factor loading was 0.8. Table 39 contains marginal means of factor loading es timates, classified by estimation method, for a population factor loading equal to 0.8. Inspection of Table 39 s uggests the effects of estimation method on estimates of the factor loadings were trivial in size, although GLS estimates have more bias than ML estimates. Several factors had important effects on estim ates of the residual variance. Table 310 contains the means by estimation and sample size for estimates of populat ion residual variances. The results show that ML estimates have mi nimal bias at all sample sizes; whereas GLS estimates have more substantial bias when sample size is 200 or 400 and suggest the GLS estimates are not unbiased even when the sample size is 1200. Table 311 presents the F statistics for the WJ tests and Table 312 presents eta squared statistics for all effects with significant F statistics in Table 311. Both tables contain results for estimation when the scale was set by specifying f actor loadings equal to one. Inspection of the results in Table 312 indicates that most effects on parameter estimates of factor loadings and factor correlations were quite small and none of them did accounted for even 1% of the total sum of squares. Because none of the effect accounted for even 1% of the total sum of squares, Table 313 presents the grand mean estimates of parameter by population parameter value. The results indicate that the grand means are ve ry similar to their population values. Effects on Standard Error Ratio Estimates LISREL provides estimates of the asymptotic st andard errors of parameter estimates in each sample. These asymptotic standard errors are called standard errors in the present study. An PAGE 61 61 empirical standard error is the standard deviatio n of the distribution of the parameter estimates. For each population parameter value, empirical standard errors were calculated for each combination of the five factors. An empirical standard error provides an estimate of the actual standard error for a particular co mbination of population parameter value and the five factors, an estimate that does not rely on large sample theory for its validity. The ratio of the standard errors to the empirical standard error can be used to determine if the standard error overestimates or underestimates the sampling variab ility of the estimates. The WJ tests for the standard error ratios were conducted and eta squa red statistics for the standard error ratios were calculated in the present study. Tables 314 and 315 contain re sults for standard error ratio s when the scale was set by specifying factor variances equal to one. Table 314 presents the F statistics for the WJ tests and Table 315 presents eta squared statis tics for all effects with significant F statistics in Table 314. For standard error ratio estimates, every effect wa s significant in the WJ te sts as shown in Table 314. To select effects for further examination the same rules used for the effects on parameter estimates were applied. Table 315 indicates that when the populati on factor loading was 0.6, sample size accounted for more than 1% of the sums of s quares. Table 316 contains marginal means of standard error ratios for factor loading estimates, classified by sample size, for a population factor loading equal to 0.6. As the sample size in creases the bias of standard error estimates of the factor loadings decreases. Although not included, tables of marginal means when the factor loading was 0.7 or 0.8 show a similar pattern. Inspection of Table 316 suggests the effects of sample size on standard error estimates of the fact or loadings is consistent with theory that PAGE 62 62 standard error estimates will converge to their tr ue values as the sample size increases without bound. As shown in Tables 314 and 315, for each factor loading population value, the interaction of estimation method and distribution is significant and accounts for a substantial portion of the total sums of squares for the standard error ratio. Table 317 contains mean standard error ratios of factor loading estimates by estimation method a nd distribution. The results show that standard errors of ML estimates have minimal bias only when the distribution is normal, whereas Robust ML standard error estimates have a relatively small bias regardless of the distribution. GLS standard error estimates have about the same bias as ML estimates, with slightly more bias for the normal distribution and slightly less for the shorttailed distribution (0, 1.15). Robust GLS estimates of standard errors tend to have sli ghtly more bias than do robust ML estimates. For each estimation method, the standa rd error ratio varies as a fu nction of the distribution. The mean standard error ratio tends to be the highe st for the (0, 1.15) di stribution and indicates overestimation when ML or GLS was used and s light underestimation when the robust variants were used. With the (2, 6) distri bution, the mean standard error ra tios tend to be the smallest. In particular GLS and ML standard e rrors are gross underestimates with this distribution. When the distribution is long tailed (0, 3) estimates also tend to be too small and the ML and GLS estimates strongly underestimate the standard errors. The results of the WJ tests and eta square d statistics indicated a significant threeway interaction among sample size, model, and the di stribution on the standard error ratio for the factor correlation, accounting for 5% of the to tal sum of squares. The means for the standard error ratios of factor correla tion as a function of sample size, model, and distribution are presented in Table 318. The standa rd error ratio varies as a function of the sample size within PAGE 63 63 each model and distribution. The bias of the standard error estimates tends to decrease as the sample size increases for the (2, 6) distribution and the standard error estimates in the (2, 6) distribution tend to be underestim ated regardless of the model. In spection of Table 318 indicates that for the other distributions th e bias tends to decrea se as the sample size increases from 200 to 400, but the effect of further increases of sample size on bias is irregula r, particularly in the normal distribution and (0, 1.15) distribution. Shown in Appe ndix A are empirical standard errors and mean standard errors and mean sta ndard error ratios, by m odel, sample size, and distributions. Results are presented in four tabl es, one for each estimation method. The results show that the empirical standard errors and mean standard errors decreased as the sample size increased from 200 to 1200. However as the ratio of the mean standard error to empirical standard error approached 1.0 as a function of sa mple size, relatively small differences between the mean standard error and empirical standard e rror resulted in ratios th at varied around 1.0. For example, with ML estimation, Model 3F3I and a normal distribution, the mean standard error and empirical standard error were 0.0607274 and 0.0598677 400,N0.0430910 and 0.0448250 800,N and 0.0351752 and 0.0341341 1200,N resulting in standard error ratios of 1.014, 0.961, and 1.031, respectively. T hus, although at first glance the behavior of the standard error ratio does not seem to be consistent with the expectation that the bias of standard errors would decline as the sample size increased, it seems more likely that the results mean that for all distributions except the (2, 6) distribution, the standa rd error became nearly unbiased when the sample size was 400 and the variation of the standard error ratio around 1.0, is due to sampling error. The interaction of distribution and estimati on method accounted for approximately 8% of the total sums of squares in the standard error ratio for the factor correlation. The means for the PAGE 64 64 standard error ratios of factor correlations as a function of di stribution and estimation method are presented in Table 319. When th e distribution was normal, (0, 1. 15) or (0, 3), ML and Robust ML showed better performances than GLS and Robust GLS in the bias of standard error estimates, whereas the robust estimation procedur es showed better performances than the nonrobust estimation procedures for the nonnormal dist ributions, with a strong advantage for the (2, 6) distribution. Tables 314 and 315 indicate a significant main effect of sample size on the standard error ratio that accounts for 1% of the total sums of squares when the residual variance was 0.36. Table 320 contains the marginal means for this main effect. Inspection of Table 320 suggests the effects of sample size on standard error esti mates of the residual variance are consistent with theory that estimates will conve rge to their true va lues as the sample size increases without bound. That is, the bias of standard error estimat es decreases as the sample size increases. Similar results, not tabled in the dissertation, were found when the population residual variance was 0.51 or 0.64. When the residual variance was 0.36, there was a significant Model Distribution interaction that accounted for 1% of the total sums of squares. The mean standard error ratios are presented in Table 321. Inspection of the resu lt in Table 321 indicates that the bias of the standard error estimates increase as the degr ee of nonnormality increases regardless of model type, but the effect of nonnor mality appears to be stronger when there are more indicators. Similar results, not tabled in the dissertation, were found when the population residual variance was 0.51 or 0.64. The results of WJ tests indi cated a significant twoway inte raction between the estimation method and distribution accounting for 20% of th e total sum of squares when the residual PAGE 65 65 variance was 0.64, 14% when the residual variance was 0.51, and 10% when the residual variance was 0.36. The means for the standard erro r ratios of residual vari ances as a function of estimation method and distribution are presented in Table 322. Fo r the normal distribution, ML and Robust ML showed better performances than GLS and Robust GLS in the bias of standard error estimates, whereas the robust estimation pr ocedures showed better performances than the nonrobust estimation procedures for the nonnor mal distributions. Specially, ML has the minimal bias of standard error estimates for th e normal distribution, whereas Robust ML has the smallest bias of standard error estimates for the nonnormal distributions For the distributions with positive kurtosis (0, 3), and high skewness and high kurtosis (2, 6), the bias of standard error estimates tends to decrea se as the population parameter va lues (0.64, 0.51, and 0.36) decrease. Tables 323 and 324 contain re sults for standard error ratio s when the scale was set by specifying factor loadings equal to one. Table 323 presents the F statistics for the WJ tests, and Table 324 presents eta squared statis tics for all effects with significant F statistics in Table 323. Inspection of the results in Table 324 indica tes that most effects on standard error ratio estimates of factor loadings and factor correlations were quite sm all compared with the effects of distribution and the twoway inte raction between estimation and di stribution. To select effects for further examination the rules used when factor variances were set equal to one were applied. For the residual variance, there is no differen ce between both scalesetting methods because the scalesetting methods do not affect the estimates of residual variances. Tables 323 and 324 indicate that the Estimation Distribution interaction was significant for both factor loading population valu es and accounted for 11 % and 9 % of the total sum of squares when the factor loading was 0.70.6 and 0.80.6, respectively. The means for PAGE 66 66 the standard error ratios of factor loadings as a function of es timation and distribut ion, classified by parameter value, are presented in Table 325. Differences between the robust estimation procedures and the nonrobust estimation procedures are very small for the normal distribution. For the nonnormal distribution, the robust estim ation procedures show ed better performance rather than the nonrobust procedures. For the (0 1.15) distribution (n egative kurtosis), the standard error estimates tend to be overesti mated by the nonrobust estimation procedures and underestimated by the robust estimation procedures For the other distri butions, the standard error estimates tend to be underestimated regardless of the estimation procedure. Tables 323 and 324 indicate that the sample size effect was significant for the factor covariance and accounted about 1.5 % of the total sum of squares. Table 326 contains marginal means of standard error ratio estimates for fact or covariance parameter, classified by sample size, for the factor c ovariance. Inspection of Table 326 suggests the effects of sample size on standard error estimates of the factor covariance c onsistent with theory that bias will decrease the sample size increases and 1200 sample size had the minimal bias of standard error estimates. Although means by sample size are not reported for the other pa rameters since eta squared statistics are less than 1% of the total sum of squares, results show the same pattern of results as in Table 326: less bias with increasing sample size accounting for rounding 1% of the total sum of squares. Tables 323 and 324 indicate that the Estimation Distribution interaction was significant for both elements of the factor covariance matrix a nd accounted 14% of the total sum of squares for the factor variance and 5% of the total sum of squares for the factor covariance. The means for the standard error ratios of factor correlations as a function of estimation method and distribution are presented in Table 327. For the normal distribution, ML and Robust ML PAGE 67 67 showed better performances than GLS and Robust GLS in the bias of standard error estimates, whereas the robust estimation procedures showed better perf ormances than the nonrobust estimation procedures for the nonnormal distribu tions. Robust ML has th e smallest bias of standard error estimates for the nonnormal distributions. There are the consistent patterns between factor variances and f actor covariances with different parameter values (0.108 and 0.36), that is, the means of standard error ra tio estimates in factor variances and factor covariances showed similar behaviors. Practical Problems in Estimation As noted in Chapter 1, estimates in SEM are found by minimizing a discrepancy function. Because a direct algebraic solution to finding th e minimizer is not available for all models, estimation in SEM is generally carried out by iteration. As a result, nonconvergence can be encountered when SEM is used. The iterative pr ocess is said to have converged when the criterion for convergence is met. In ML estimation in LISREL, the crite rion for convergence is a change in the likelihood from one iteration to the next that is smaller than 0.000001 (Jreskog and Srbom, 1996). The maximum number of iterati ons is set a priori and nonconvergence in LISREL occurs when the change in the likelihood ne xt to last and last iteration is larger than 0.000001. Another possible problem is a nonpositiv e definite matrix of secondorder partial derivatives of the discrepancy f unction, a matrix called the Hessian matrix. The Hessian matrix is used to diagnose identification pr oblems and to compute standard errors. If the Hessian matrix is not positive definite the solution is classified as nonidentified and, in addition, standard errors cannot be computed. In addition, unless the iterative procedure is constructed to keep variance and covariance estimates within the bounds of proper estimates, estimates outside these bound can occur, for example, negative residual variances. Such estimates are called improper estimates PAGE 68 68 in the present study. For the deta ils on the problems, potential so lutions, and limita tions to these potential solutions, see Chen et al (2001), Bentler and C hou (1987), and Wothke (1993). Previous studies reported nonconvergence ma inly for the ML estimation method. Nonconvergence of ML estimation method decreased wh en the sample size, the number of indicators per factor, or the size of the factor loadings increased (Ande rson & Gerbing, 1984; Boomsma, 1985). Also Hoogland (1999) reported that none of the estimation methods had substantial convergence problems for N 200, and for N = 200, the percentage of improper solutions was considerable. This percentage increased when the size of the factor loadings or the number of indicators per factor decreased. While conducting the simulation the following pr oblems occurred in some replications of some conditions: (a) nonconvergence in 10,000 iterations, (b) improper estimates (e.g., a negative residual variance), and (c) nonpositive definite Hessian matrix (so standard errors could not be computed). The frequency of nonconvergence, improper estimates, and nonpositive definite Hessian matrix for certain combinations of model, distribution, sample size, scalesetting method, and estimation is presented in Table 328. Nonconvergence did not occur for the normal distribution at all and mainly o ccurred for sample sizes such of 200 and 400 for the nonnormal distributions. For the 3F3I model, nonconvergence occurred only for ML and Robust ML, but occurred at a very low freque ncy and never when the sample size was 800 or 1200. For the 3F6I model, nonconvergence, im proper estimates, and nonpositive definite Hessians occurred for all estimation procedures, but only when the scale was set by specifying factor loadings equal to one, and again at a ve ry low frequency and never when the sample size was 800 or 1200. Increasing the number of indicat ors per factor caused more nonconvergence, improper estimates, and nonpositive definite He ssian matrices. However, setting the scale by PAGE 69 69 specifying factor variances equal to one reduced the problems of nonconvergence, improper estimates, and nonpositive definite Hessians. For the model 6F3I, nonconvergence, improper estimates, and nonpositive definite Hessians oc curred only for GLS and Robust GLS and at a very low frequency. There was no nonconvergen ce problem at all when the scale was set by specifying factor variances equal to one. As ha s been reported in previous studies (e.g., Boomsma & Hoogland, 2001) degree of nonnormality the number of indicators per factor, and the sample size played important roles in wh ether nonconvergence, improper estimates, and nonpositive definite Hessians occurred. In addition, the present study su ggests that the scalesetting method has an important effect under nonnormality. PAGE 70 70 Table 31. Freque ncy of significance F tests on parameter estimates per 16 combinations of distribution and sample size: factor variances equal to one. Estimation method Model Parameter value ML GLS (0.6) 2 3 (0.7) 1 1 (0.8) 1 0 (0.3) 1 1 (0.64) 0 0 (0.51) 1 1 3F3I (0.36) 0 0 (0.6) 1 2 (0.7) 0 0 3F6I (0.8) 0 0 (0.3) 1 2 (0.64) 1 1 (0.51) 1 1 (0.36) 0 0 6F3I (0.6) 0 1 (0.7) 1 0 (0.8) 0 0 (0.3) 1 1 (0.64) 1 1 (0.51) 1 0 (0.36) 1 1 Note. indicates factor loading, : factor correlation, and : residual variance. PAGE 71 71 Table 32. Frequency of significant F tests on parameter estimates per 16 combinations of distribution and sample size: f actor loadings equal to one. Estimation method Model Parameter Values ML GLS (0.7/0.6) 0 0 (0.8/0.6) 0 1 (i,i) = (0.36) 1 0 3F3I (i,j) = (0.108) 1 1 (0.7/0.6) 2 2 (0.8/0.6) 2 2 (i,i) = (0.36) 1 2 3F6I (i,j) = (0.108) 1 2 (0.7/0.6) 2 2 (0.8/0.6) 0 0 (i,i) = (0.36) 0 1 6F3I (i,j) = (0.108) 1 1 PAGE 72 72 Table 33. Mean parameter estimate for = 0.6: Model 3F3I, GLS, N = 200, factor variances equal to one, and (0, 1.15). Three factor loadings: (0.6) Means of parameter estimates 1 0.593 2 0.593 3 0.586 Table 34. Frequency of significant F tests on standard error estimates per 16 combinations of distribution and sample size: f actor variances equal to one. Estimation method Model Parameter value ML RML GLS RGLS (0.6) 0 0 0 0 (0.7) 1 0 0 0 (0.8) 1 0 1 0 (0.3) 0 0 0 0 (0.64) 0 0 0 1 (0.51) 1 1 0 1 3F3I (0.36) 2 1 1 1 (0.6) 1 1 1 1 (0.7) 0 0 1 0 (0.8) 1 0 1 0 (0.3) 1 2 1 1 (0.64) 1 2 1 2 (0.51) 1 1 1 1 3F6I (0.36) 0 1 0 0 (0.6) 0 0 1 0 (0.7) 0 0 0 0 (0.8) 2 1 2 1 (0.3) 2 2 1 2 (0.64) 1 2 0 1 (0.51) 0 2 0 2 6F3I (0.36) 2 1 2 1 PAGE 73 73 Table 35. Frequency of significant F tests on standard error estimates per 16 combinations of distribution and sample size: f actor loadings equal to one. Estimation method Model Parameter value ML RML GLS RGLS (0.7/0.6) 1 1 1 2 (0.8/0.6) 1 1 1 2 (i,i) (0.36) 1 2 1 2 3F3I (i,j) (0.108) 0 1 1 2 (0.7/0.6) 1 1 2 2 (0.8/0.6) 2 1 1 2 (i,i) (0.36) 2 2 2 2 3F6I (i,j) (0.108) 1 0 1 2 (0.7/0.6) 1 1 1 1 (0.8/0.6) 1 0 0 0 (i,i) (0.36) 0 0 0 0 6F3I (i,j) (0.108) 1 0 0 0 Table 36. Mean standard error estimates for = 0.3: model 6F3I, Robust ML, N = 1200, factor variances equal to one, and (2, 6). Fifteen factor correlations: (0.3) Means of standard error estimates SE19 0.0400 SE20 0.0399 SE21 0.0398 SE22 0.0396 SE23 0.0396 SE24 0.0399 SE25 0.0397 SE26 0.0397 SE27 0.0395 SE28 0.0396 SE29 0.0395 SE30 0.0397 SE31 0.0397 SE32 0.0397 SE33 0.0396 PAGE 74 74 Table 37. F statistics (degrees of freedom) for WJ te sts for effects on parameter estimates: scale set by specifying factor variances equal to one. Parameter (0.6) (0.7) (0.8) (0.3) (0.64) (0.51) (0.36) Betweensubjects S 46.02 (3, 22506.63) 61.53 (3, 21599.19) 72.73 (3, 21198.98) 1.83 (3, 24825.76) 620.72 (3, 20239.98) 446.12 (3, 21093.75) 312.53 (3, 20896.37) D 1.66 (3, 17775.07) 5.64 (3, 17700.60) 3.12 (3, 17406.09) 0.62 (3, 17283.86) 14.52 (3, 18013.56) 9.82 (3, 17424.44) 15.52 (3, 15742.61) M 23.31 (2, 19266.16) 38.43 (2, 18800.59) 45.71 (2,18258.67) 7.72 (2, 20687.19) 219.81 (2, 17766.82) 194.44 (2, 18747.50) 67.91 (2, 18928.08) S*D 1.0296 (9, 18427.51) 0.82 (9, 18228.70) 0.72 (9, 18073.82) 1.05 (9, 18432.73) 2.32 (9, 18206.46) 1.53 (9, 17812.25) 3.52 (9, 16505.48) S*M 6.12 (6, 18082.12) 7.33 (6, 17408.11) 8.10 (6, 17138.26) 1.78 (6, 20066.18) 26.30 (6, 16326.06) 23.27 (6, 17217.24) 8.05 (6, 17829.92) D*M 0.84 (6, 14311.52) 0.46 (6, 14305.69) 0.57 (6, 14157.82) 0.75 (6, 13965.46) 2.21 (6, 14588.77) 2.70 (6, 14366.58) 0.13 (6, 13566.43) S*D*M 1.16 (18, 16850.26) 0.69 (18, 16726.97) 0.47 (18, 16682.06) 0.99 (18, 16940.70) 0.92 (18, 16705.38) 0.95 (18, 16628.99) 0.96 (18, 16075.00) Withinsubjects E 12178.18 (1, 16540.83) 17764.71 (1, 16636.72) 25686.78 (1, 16649.31) 4520.48 (1, 15325.14) 152399.97 (1, 17241.46) 108604.31 (1, 16748.83) 52581.77 (1, 15320.01) E*S 1524.95 (3, 20451.17) 2281.22 (3, 20465.43) 3231.08 (3, 20501.19) 471.46 (3, 18942.21) 18111.71 (3, 20302.35) 13166.86 (3, 20372.53) 6457.34 (3, 19094.80) E*D 30.04 (3, 9590.41) 45.51 (3, 9692.51) 85.56 (3, 9709.39) 2.09 (3, 8897.09) 297.00 (3, 10375.78) 274.18 (3, 9880.68) 141.91 (3, 9052.65) E*M 1298.57 (2, 11765.34) 1884.21 (2, 11864.30) 2434.13 (2, 11985.07) 242.89 (2, 11306.97) 12160.34 (2, 12197.84) 8728.98 (2, 11987.15) 4810.11 (2, 11892.42) E*S*D 4.93 (9, 15655.19) 5.85 (9, 15702.67) 12.01 (9, 15683.89) 0.62 (9, 14463.13) 29.90 (9, 15870.74) 29.11 (9, 15730.18) 15.06 (9, 14699.42) E*S*M 162.49 (6, 17485.34) 238.26 (6, 17568.41) 308.24 (6, 17691.47) 25.60 (6, 17204.20) 1410.96 (6, 17274.57) 1040.98 (6, 17475.85) 572.56 (6, 17603.68) E*D*M 1.57 (6, 8198.83) 3.84 (6, 8276.55) 6.43 (6, 8424.32) 0.71 (6, 7874.98) 37.14 (6, 8788.30) 35.89 (6, 8603.60) 19.99 (6, 8483.09) E*S*D*M 0.78 (18, 15209.78) 0.79 (18, 15292.40) 1.22 (18, 15427.25) 0.69 (18, 14955.98) 6.36 (18, 15340.25) 5.13 (18, 15485.72) 2.36 (18, 15525.14) Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, : factor correlation, and : residual variance. PAGE 75 75 Table 38. Eta squared for si gnificant effects on parameter estimates: scale set by specifying factor variances equal to one. Parameter (0.6) (0.7) (0.8) (0.3) (0.64) (0.51) (0.36) Betweensubjects S 0.0033 0.0044 0.0055 0.0397 0.0300 0.0221 D 0.0005 0.0003 0.0010 0.0007 0.0011 M 0.0010 0.0015 0.0018 0.0003 0.0079 0.0074 0.0030 S*D 0.0003 0.0010 S*M 0.0009 0.0010 0.0010 0.0039 0.0034 0.0012 D*M 0.0003 0.0004 S*D*M Withinsubjects E 0.0053 0.0073 0.0104 0.0019 0.0693 0.0540 0.0286 E*S 0.0024 0.0033 0.0046 0.0005 0.0293 0.0233 0.0123 E*D 0.0001 0.0001 0.0001 0.0005 0.0005 0.0003 E*M 0.0009 0.0012 0.0015 0.0002 0.0085 0.0067 0.0039 E*S*D 0.0000 0.0000 0.0001 0.0002 0.0002 0.0001 E*S*M 0.0004 0.0006 0.0007 0.0001 0.0034 0.0027 0.0016 E*D*M 0.0000 0.0000 0.0001 0.0001 0.0001 E*S*D*M Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, : factor correlation, and : residual variance. Table 39. Marginal means by estimation for estimates of = 0.8 Estimation Marginal means GLS 0.788 ML 0.800 Table 310. Mean estimates of residual variance parameters by parameter value, estimation method, and sample size. Sample size Parameter value Estimation 200 400 800 1200 0.36 GLS 0.310 0.334 0.347 0.351 ML 0.353 0.356 0.359 0.359 0.51 GLS 0.442 0.475 0.492 0.498 ML 0.506 0.508 0.509 0.509 0.64 GLS 0.554 0.596 0.617 0.625 ML 0.634 0.637 0.638 0.639 PAGE 76 76 Table 311. F statistics (degrees of freedom) for WJ tests for effects on parameter estimates: scale set by specifying factor loadings equal to one. Parameter (0.7/0.6) (0.8/0.6) (i,i) = (0.36) (i,j) = (0.108) Betweensubjects S 30.28 (3, 22125.35) 38.49 (3, 22189.49) 8.57 (3, 22208.53) 19.33 (3, 24267.26) D 4.77 (3, 16883.50) 7.66 (3, 16473.47) 2.40 (3, 17582.66) 0.21 (3, 17633.72) M 2.41 (2, 18254.97) 4.19 (2, 18080.90) 26.54 (2, 18840.57) 7.75 (2, 20592.78) S*D 1.93 (9, 18240.52) 1.41 (9, 17942.50) 1.14 (9, 18356.55) 0.82 (9, 18449.22) S*M 1.69 (6, 17835.21) 0.79 (6, 18077.52) 6.37 (6, 17881.18) 0.95 (6, 19593.93) D*M 1.08 (6, 13645.46) 0.51 (6, 13494.24) 0.86 (6, 14207.07) 0.42 (6, 14255.28) S*D*M 0.63 (18, 16747.68) 1.09 (18, 16671.06) 1.19 (18, 16820.97) 1.17 (18, 16942.48) Withinsubjects E 2.62 (1, 15632.45) 3.20 (1, 14317.90) 11807.82 (1, 16404.87) 1207.51 (1, 15341.86) E*S 1.61 (3, 20172.44) 1.90 (3, 19755.23) 1444.36 (3, 20283.60) 214.33 (3, 19103.04) E*D 0.21 (3, 9092.03) 0.90 (3, 8681.75) 31.49 (3, 9653.42) 9.39 (3, 9082.70) E*M 1.51 (2, 11208.28) 0.44 (2, 10793.27) 1243.90 (2, 11644.45) 425.01 (2, 11327.47) E*S*D 0.89 (9, 15496.59) 0.99 (9, 15387.27) 5.29 (9, 15680.82) 2.45 (9, 14733.99) E*S*M 0.91 (6, 17372.35) 1.10 (6, 17429.38) 151.13 (6, 17338.56) 68.39 (6, 17007.79) E*D*M 0.54 (6, 7845.83) 1.42 (6, 7776.98) 1.55 (6, 8244.30) 3.37 (6, 8050.43) E*S*D*M 0.87 (18, 15166.03) 0.78 (18, 15366.89) 0.76 (18, 15233.44) 1.00 (18, 14969.96) Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, and : factor correlation. PAGE 77 77 Table 312. Eta squared for si gnificant effects on parameter estimates: scale set by specifying factor loadings equal to one. Parameter (0.7/0.6) (0.8/0.6) (i,i) = (0.36) (i,j) = (0.108) Betweensubjects S 0.0024 0.0030 0.0005 0.0018 D 0.0003 0.0005 M 0.0002 0.0011 0.0003 S*D 0.0004 S*M 0.0009 D*M S*D*M Withinsubjects E 0.0050 0.0005 E*S 0.0022 0.0004 E*D 0.0001 0.0000 E*M 0.0008 0.0002 E*S*D 0.0000 0.0000 E*S*M 0.0003 0.0002 E*D*M 0.0000 E*S*D*M Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, and : factor correlation. Table 313. Grand mean estimates of parameter by parameter value Parameter Grand mean (0.7/0.6 = 1.167) 1.176 (0.8/0.6 = 1.333) 1.345 (i,i) = (0.36) 0.358 (i,j) = (0.108) 0.107 PAGE 78 78 Table 314. F statistics (degrees of freedom) for WJ tests for effects on standard error ratio estimates: scale set by specifying f actor variances equal to one. Parameter (0.6) (0.7) (0.8) (0.3) (0.64) (0.51) (0.36) Betweensubjects S 595.90 (3, 15782.04) 467.62 (3, 16318.87) 379.78 (3, 17343.85) 3603.70 (3, 23369.23) 198.16 (3, 19385.97) 242.22 (3, 20429.42) 363.96 (3, 22444.21) D 20946.67 (3, 19027.12) 26581.99 (3, 18729.86) 22200.70 (3, 18881.47) 4595.39 (3, 19551.74) 10741.40 (3, 19649.16) 7329.45 (3, 17648.10) 3889.99 (3, 18945.82) M 225.66 (2, 15794.47) 42.24 (2, 16166.66) 116.43 (2, 17599.89) 273.00 (2, 22385.22) 15.44 (2, 18824.20) 169.66 (2, 19600.56) 218.83 (2, 21405.65) S*D 48.99 (9, 17638.50) 218.89 (9, 17553.80) 101.65 (9, 17538.81) 194.24 (9, 18642.07) 49.07 (9, 17772.69) 48.22 (9, 17301.84) 96.55 (9, 17901.15) S*M 87.91 (6, 12666.42) 198.36 (6, 13141.15) 57.97 (6, 13948.56) 68.55 (6, 13948.56) 34.21 (6, 15528.44) 31.56 (6, 16452.99) 30.47 (6, 18161.94) D*M 135.74 (6, 15268.67) 127.56 (6, 15117.87) 276.67 (6, 15383.29) 263.29 (6, 15693.62) 80.08 (6, 15761.30) 111.48 (6, 14712.34) 234.54 (6, 15264.45) S*D*M 82.00 (18, 16102.75) 107.39 (18, 16146.65) 137.13 (18, 16225.82) 627.60 (18, 17017.36) 77.87 (18, 16322.40) 42.42 (18, 16228.62) 29.03 (18, 16583.77) Withinsubjects E 77257.59 (3, 15303.02) 69198.35 (3, 15692.24) 36958.85 (3, 14287.87) 23486.29 (3, 14896.53) 22848.71 (3, 18411.56) 14151.02 (3, 17556.47) 11625.93 (3, 15108.66) E*S 10195.94 (9, 19079.13) 7328.54 (9, 19516.57) 2784.84 (9, 19235.63) 2703.90 (9, 18792.67) 932.12 (9, 22540.47) 497.97 (9, 22243.10) 620.77 (9, 21133.74) E*D 21183.88 (9, 14286.31) 27486.10 (9, 14395.97) 27151.98 (9, 13363.58) 1147.50 (9, 13708.24) 15919.85 (9, 14747.70) 8033.84 (9, 13891.66) 5491.45 (9, 13233.02) E*M 4459.91 (6, 12911.48) 5099.82 (6, 13239.75) 3952.74 (6, 12553.37) 1887.56 (6, 12425.43) 1215.18 (6, 15405.97) 706.68 (6, 15007.13) 473.90 (6, 13573.18) E*S*D 1497.61 (27, 21971.93) 1621.64 (27, 22161.95) 529.65 (27, 21948.32) 99.75 (27, 21536.83) 980.36 (27, 21907.35) 509.83 (27, 21498.40) 236.39 (27, 21159.85) E*S*M 1247.12 (18, 18044.97) 927.37 (18, 18551.41) 540.04 (18, 18499.14) 404.69 (18, 17733.29) 459.54 (18, 21227.91) 222.73 (18, 21268.16) 272.15 (18, 20822.39) E*D*M 1360.22 (18, 13539.20) 1946.81 (18, 13654.05) 2745.12 (18, 13203.38) 53.13 (18, 13049.66) 1163.97 (18, 14049.75) 389.33 (18, 13741.50) 247.23 (18, 13283.14) E*S*D*M 499.25 (54, 21984.87) 1017.96 (54, 22257.57) 583.71 (54, 22170.23) 157.36 (54, 21639.51) 268.91 (54, 22006.12) 150.17 (54, 22095.98) 124.57 (54, 21887.70) Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, : factor correlation, and : residual variance. PAGE 79 79 Table 315. Eta squared for signi ficant effects on standard erro r ratio estimates: scale set by specifying factor variances equal to one. Parameter (0.6) (0.7) (0.8) (0.3) (0.64) (0.51) (0.36) Betweensubjects S 0.0107 0.0065 0.0067 0.0965 0.0041 0.0070 0.0152 D 0.2918 0.3201 0.2882 0.1195 0.2300 0.1783 0.1165 M 0.0023 0.0004 0.0010 0.0039 0.0002 0.0026 0.0045 S*D 0.0008 0.0046 0.0022 0.0145 0.0019 0.0021 0.0053 S*M 0.0017 0.0051 0.0015 0.0023 0.0014 0.0010 0.0018 D*M 0.0051 0.0027 0.0057 0.0099 0.0030 0.0049 0.0101 S*D*M 0.0037 0.0043 0.0057 0.0526 0.0038 0.0038 0.0054 Withinsubjects E 0.0612 0.0572 0.0602 0.0333 0.1077 0.1120 0.1008 E*S 0.0028 0.0024 0.0020 0.0070 0.0017 0.0018 0.0020 E*D 0.2290 0.2542 0.3009 0.0817 0.1960 0.1371 0.1008 E*M 0.0004 0.0005 0.0004 0.0018 0.0002 0.0010 0.0043 E*S*D 0.0013 0.0011 0.0013 0.0014 0.0012 0.0012 0.0012 E*S*M 0.0003 0.0002 0.0002 0.0007 0.0003 0.0003 0.0002 E*D*M 0.0009 0.0024 0.0064 0.0002 0.0008 0.0017 0.0029 E*S*D*M 0.0002 0.0005 0.0004 0.0007 0.0002 0.0003 0.0004 Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, : factor correlation, and : residual variance. Table 316. Marginal means by sample size for standard error ratio estimates of = 0.6 Sample size Marginal means of standard error ratio 200 0.907 400 0.936 800 0.937 1200 0.949 PAGE 80 80 Table 317. Mean estimates of standard error ratio for factor loading parameters by parameter value, estimation method, and distribution Distribution Parameter value Estimation (0, 0) (0, 1.15) (0, 3) (2, 6) 0.6 GLS 0.971 1.072 0.828 0.681 ML 0.994 1.102 0.838 0.683 Robust GLS 0.965 0.968 0.952 0.950 Robust ML 0.988 0.992 0.967 0.965 0.7 GLS 0.967 1.102 0.800 0.650 ML 0.989 1.138 0.808 0.652 Robust GLS 0.961 0.963 0.955 0.943 Robust ML 0.982 0.992 0.967 0.958 0.8 GLS 0.972 1.109 0.787 0.634 ML 0.991 1.142 0.795 0.633 Robust GLS 0.967 0.941 0.970 0.953 Robust ML 0.985 0.965 0.983 0.965 Table 318. Mean estimates of standard er ror ratio for factor correlation parameters, (0.3) by model, sample size, and distribution. Distribution Model Sample size (0, 0) (0, 1.15) (0, 3) (2, 6) 3F3I 200 0.916 0.948 0.933 0.856 400 1.010 0.992 0.988 0.883 800 0.955 1.009 0.973 0.911 1200 1.027 0.974 0.958 0.970 3F6I 200 0.941 0.932 0.946 0.898 400 1.010 1.012 0.976 0.917 800 0.942 1.007 1.003 0.923 1200 1.004 1.041 1.021 0.920 6F3I 200 0.939 0.928 0.982 0.858 400 0.990 0.986 0.954 0.933 800 1.038 0.967 0.957 0.951 1200 0.989 1.009 0.989 0.975 Table 319. Means estimates of standard er ror ratio for factor correlation parameter, (0.3) by estimation and distribution. Distribution Estimation (0, 0) (0, 1.15) (0, 3) (2, 6) GLS 0.974 0.974 0.968 0.856 ML 0.991 0.992 0.985 0.875 Robust GLS 0.969 0.975 0.962 0.956 Robust ML 0.986 0.993 0.979 0.977 PAGE 81 81 Table 320. Marginal means by sample size for standard error ratio estimates of = 0.36 Sample size Marginal mean s of standard error ratio 200 0.864 400 0.905 800 0.913 1200 0.920 Table 321. Means estimates of standard error ratio for residual variance parameter, (0.36) by model and distribution. Distribution Model (0, 0) (0, 1.15) (0, 3) (2, 6) 3F3I 0.959 0.947 0.934 0.811 3F6I 0.991 0.909 0.867 0.770 6F3I 0.956 0.922 0.914 0.827 Table 322. Means estimates of standard er ror ratio for residual variance parameters by parameter value, estimation method, and distribution. Distribution Parameter value Estimation (0, 0) (0, 1.15) (0, 3) (2, 6) 0.36 GLS 0.962 0.903 0.839 0.664 ML 0.982 0.910 0.840 0.653 Robust GLS 0.956 0.940 0.966 0.946 Robust ML 0.975 0.949 0.975 0.949 0.51 GLS 0.973 0.951 0.766 0.601 ML 0.992 0.972 0.760 0.592 Robust GLS 0.967 0.950 0.955 0.938 Robust ML 0.985 0.970 0.959 0.948 0.64 GLS 0.964 1.027 0.719 0.567 ML 0.986 1.057 0.715 0.556 Robust GLS 0.957 0.960 0.954 0.948 Robust ML 0.978 0.982 0.962 0.953 PAGE 82 82 Table 323. F statistics (degrees of freedom) for WJ tests for effects on standard error ratio estimates: scale set by specifying factor loadings equal to one. Parameter (0.7/0.6) (0.8/0.6) (i,i) = (0.36) (i,j) = (0.108) Betweensubjects S 220.62 (3, 25086.80) 178.34 (3, 24879.45) 231.28 (3, 17868.11) 249.85 (3, 21326.61) D 4013.23 (3, 18064.26) 2570.75 (3, 17484.25) 6248.21 (3, 18068.64) 1829.03 (3, 18537.10) M 34.15 (2, 21571.44) 103.02 (2, 21102.18) 75.84 (2, 16417.49) 68.43 (2, 19090.67) S*D 19.71 (9, 18614.53) 11.82 (9, 18401.97) 9.60 (9, 17774.22) 60.09 (9, 18451.59) S*M 28.90 (6, 20170.81) 13.17 (6, 20142.66) 23.57 (6, 14309.88) 5.54 (6, 17071.21) D*M 42.28 (6, 14530.87) 57.75 (6, 14186.86) 44.35 (6, 14467.44) 13.43 (6, 14855.56) S*D*M 22.67 (18, 17021.57) 18.33 (18, 16949.97) 24.57 (18, 16181.22) 56.38 (18, 16805.40) Withinsubjects E 14355.94 (3, 16937.01) 13654.15 (3, 15881.56) 12849.56 (3, 13520.35) 6768.33 (3, 15521.07) E*S 477.94 (9, 20805.45) 402.60 (9, 20286.70) 1146.01 (9, 18049.14) 551.36 (9, 19104.87) E*D 11663.55 (9, 14485.95) 10329.60 (9, 13932.84) 14093.12 (9, 13417.50) 2848.15 (9, 15184.48) E*M 196.35 (6, 14082.87) 195.18 (6, 13308.35) 546.87 (6, 11479.07) 425.92 (6, 12913.51) E*S*D 133.87 (27, 21842.24) 96.65 (27, 21851.62) 241.73 (27, 21595.30) 29.38 (27, 22533.00) E*S*M 51.77 (18, 19681.63) 46.87 (18, 19066.28) 171.71 (18, 17130.85) 92.77 (18, 17868.57) E*D*M 135.32 (18, 13767.40) 242.49 (18, 13309.31) 250.44 (18, 12822.42) 53.21 (18, 14338.56) E*S*D*M 36.30 (54, 22217.71) 22.02 (54, 22209.29) 71.59 (54, 21709.12) 19.90 (54, 22375.98) Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, and : factor correlation. PAGE 83 83 Table 324. Eta squared for signi ficant effects on standard erro r ratio estimates: scale set by specifying factor loadings equal to one. Parameter (0.7/0.6) (0.8/0.6) (i,i) = (0.36) (i,j) = (0.108) Betweensubjects S 0.0094 0.0086 0.0093 0.0150 D 0.1576 0.1139 0.1789 0.0910 M 0.0009 0.0029 0.0016 0.0020 S*D 0.0012 0.0014 0.0005 0.0049 S*M 0.0020 0.0013 0.0008 0.0004 D*M 0.0036 0.0045 0.0035 0.0013 S*D*M 0.0041 0.0035 0.0031 0.0086 Withinsubjects E 0.0421 0.0364 0.0385 0.0153 E*S 0.0021 0.0025 0.0015 0.0019 E*D 0.1098 0.0898 0.1415 0.0486 E*M 0.0003 0.0005 0.0002 0.0005 E*S*D 0.0012 0.0013 0.0005 0.0005 E*S*M 0.0002 0.0002 0.0002 0.0002 E*D*M 0.0008 0.0019 0.0006 0.0003 E*S*D*M 0.0003 0.0003 0.0002 0.0001 Note. S indicates Sample size, D: distributi on, M: model, and E: estimation method. indicates factor loading, and : factor correlation. Table 325. Mean estimates of standard error ratio for factor loading parameters by parameter value, estimation method, and distribution. Distribution Parameter value Estimation (0, 0) (0, 1.15) (0, 3) (2, 6) 0.7/0.6 GLS 0.964 1.040 0.812 0.670 ML 0.989 1.073 0.824 0.673 Robust GLS 0.959 0.964 0.944 0.937 Robust ML 0.983 0.993 0.960 0.953 0.8/0.6 GLS 0.959 1.019 0.825 0.685 ML 0.987 1.048 0.839 0.692 Robust GLS 0.954 0.953 0.946 0.937 Robust ML 0.981 0.978 0.964 0.957 Table 326. Marginal means by sample size for standard error ratio estimates of (i, j) = 0.108 Sample sizeMarginal means of standard error ratio 2000.921 4000.9618000.96412000.975 PAGE 84 84 Table 327. Mean estimates of standard erro r ratio for factor correlation parameters by parameter value, estimation method, and distribution. Distribution Parameter value Estimation (0, 0) (0, 1.15) (0, 3) (2, 6) (i, j) = 0.108 GLS 0.976 1.022 0.921 0.795 ML 0.994 1.043 0.938 0.807 Robust GLS 0.971 0.982 0.960 0.945 Robust ML 0.988 1.003 0.978 0.964 (i, i) = 0.36 GLS 0.969 1.072 0.821 0.672 ML 0.993 1.100 0.832 0.675 Robust GLS 0.963 0.966 0.950 0.948 Robust ML 0.986 0.990 0.965 0.964 Table 328. Frequency of nonconvergence (N C), improper estimates (IE), and nonpositive definite Hessians (NP) Model Distribution Sample size Scale set Estimation NC IE or NP (0, 3) 200 Loading ML/RML 0.1% 0.0% 200 Variance ML/RML 0.2% 0.0% 3F3I (2, 6) 400 Loading ML/RML 0.1% 0.0% 200 Loading ML/RML 0.2% 0.0% (0, 1.15) 200 Loading GLS/RGLS 0.0% 0.1% 200 Loading ML/RML 0.8% 0.5% 200 Loading GLS/RGLS 0.2% 0.1% (0, 3) 400 Loading ML/RML 0.1% 0.1% 200 Loading ML/RML 0.4% 0.3% 200 Loading GLS/RGLS 0.1% 1.3% 3F6I (2, 6) 400 Loading ML/RML 0.1% 0.1% 200 Variance GLS/RGLS 0.0% 0.5% 6F3I (2, 6) 200 Loading GLS/RGLS 0.1% 0.2% PAGE 85 85 CHAPTER 4 SUMMARY AND CONCLUSIONS For each of four estim ation methods (ML, GLS, Robust ML, and Robust GLS) the behavior of standard error ratio estimates was examined under each combination of four distributions ([0, 0], [0, 1.15], [0, 3], and [2, 6]), four sample sizes (200, 400, 800, and 1200), three CFA models (3F3I, 3F6I, and 6F3I), a nd two scalesetting methods (set by specifying factor variances equal to one a nd factor loadings equal to one) In addition, the bias of the parameter estimation was investigated since esti mation bias might have varied over estimation methods at some sample sizes and there might have be an interactive eff ect of estimation method with other factors. The effects of four factors (estimation method, distribution, model, and sample size) on parameter estimates and standard error estimates were examined within each scalesetting method. Comparison of Findings with Previous Studies Parameter Estimates Recall that ML and Robust ML produce the same parameter estimates as do GLS and Robust GLS, so for bias only a comparison of the ML and GLS estimation procedures was necessary. Important effects of factors were define d as those that were significant by the WJ test and accounted for at least 1% of the total sum of squares in a repeated measures ANOVA of the data. When the scale was set by specifying factor variances equal to one no factor had an important effect on estimation of the factor correlation and only estimation method had an important effect on estimates of the factor lo ading and then only when the population factor loading was 0.8. ML estimates had less bias than GLS estimates, but the effect was very small. The interaction of estimation method and sample si ze had an important effect on estimates of the residual variance. The results showed that ML es timates had minimal bias at all sample sizes. PAGE 86 86 Therefore, although the bias of the ML estimates declined as samp le size increased, the effect of sample size on bias of ML estimates was quite small. The bias of GLS estimates also declined as sample size increases. Compared to the bias of ML estimates, the bias of GLS estimates was larger with larger differences between the me thods when the sample size was 200 or 400 and a trivial difference when the sample size was1200. When the scale was set by specifying factor loadings equal to one, most effects on parameter estimates of factor loading and factor correlation were quite small and none of them accounted for even 1% of the total sum of square s. Recall that estimates of residual variance are the same for both scalesetting methods. Therefore effects of the factors on the residual variance were the same for both methods of setting the scale. Boomsma (1983) indicated that, when the dist ribution is normal and the scale is set by specifying factor loadings equal to one, the bias of ML parameter estimates is small for models with more indicators per factor and higher factor loadings co mpared with the bias of ML estimates for models with fewer indicators per fa ctor and smaller factor loadings. Boomsma only investigated models for which the scale was set by specifying factor loadings equal to one. Muthen and Kaplan (1985) found that there was not much difference between ML and GLS parameter estimates. Henly (1993) also pointed out the similarity between ML and GLS, that is, equivalent conclusion was drawn about the behavi or of the ML and GLS parameter estimates in the distribution and sample size conditions studied: Like the ML parameter estimates, the GLS estimates appeared to produce consistent parameter estimates that are unbiased when N 600 for samples from the multivariate normal and asy mmetric multivariate populations (Henly, 1993). According to Hoogland and Boomsma (1998), the bias of ML parameter estimates increases when the levels of univariate skewness and kurto sis deviate increasingly from those of a normal PAGE 87 87 distribution and a larger samp le size is a remedy to obtain unbiased parameter estimates. Hoogland (1999) reported that ML parameter esti mates were almost unbiased when the sample size was at least 200 and GLS had a much larger bias than ML. For parameter estimates, the overall results of the present study are consistent with the results of the previous studies: ML estimates were almost unbiased at all sample sizes and ML estimation had less bias than GLS estimation, although the differences were trivial for factor loadings. Sample size played more a critical ro le in GLS estimation than in ML estimation of residual variance and, as a result, larger betweenmethod differenc es in bias were observed for estimates of residual variance. Wh en the scale was set by specifying factor loadings equal to one, there were no important effects of the factors on the factor loading, fact or variance, or factor covariance estimates. Standard Error Estimates When the scale was set by specifying factor va riances equal to one, the bias of standard error estimates for the factor loadings decr eased as the sample size increased, showing consistency with theory that standard error esti mates will converge to their true values as the sample size increases without bound. For factor lo adings, the ML standard error estimates had minimal bias for the normal distribution and Robust ML estimates had a relatively small bias regardless of the distribution. Al so, there was relatively little di fference between the biases of ML and GLS estimates of standard errors, and rela tively little difference between the biases of Robust ML and Robust GLS estimates of standard errors regardless of the distribution. ML and GLS estimates of standard errors of factor load ing were overestimated when the distribution was shorttailed (0, 1.15) and strongl y underestimated when the distri butions were longtailed (0, 3) and longtailed and skewed (2, 6). PAGE 88 88 A long tail to the distribution tended to result in underestimation of standard errors of the interfactor correlation and underestim ation was particularly large fo r the (2, 6) distribution. For the (2, 6) distribution, bias of the standard er rors decreased systematically as the sample size increased. For the other distributions, estimates of the standard error ratio did not systematically decline as the sample size increased because the empirical standard error estimates and standard error estimates for the factor correlation decreased as the sample size increased but not the same rate (please see Appendix A). Wh en the distribution was normal, (0, 1.15) and (0, 3), ML and Robust ML estimation of standard errors were less biased than were GLS and Robust GLS estimation of standard errors. The robust esti mation procedures showed substantially better performance than the nonrobust estimation procedures for the (2, 6) distribution. The bias of the standard error estimates for residual variance decreased as the sample size increased regardless of the population value. The bias increased as the degree of nonnormality increased regardless of the model type. For a normal distribution, more indicators per factor resulted in less bias, but resulted in more bi as for the nonnormal distributions. For a normal distribution, ML and Robust ML estimation performed better than GLS and Robust GLS estimation of the standard error estimates of re sidual variances, whereas the robust procedures performed better than the nonrobus t procedures for the nonnormal di stributions. The bias of the standard error estimates for residual variances te nded to decrease as the population value (0.64, 0.51, and 0.36) decreased. When the scale was set by specifying factor lo adings equal to one a nd the distribution was normal, betweenestimation method differences in th e bias of the standard error estimates for the factor loadings were very sma ll. For the nonnormal distributions robust estimation of standard errors of factor loading performed better than did nonrobust estimation. When the distribution PAGE 89 89 was (0, 1.15), the standard error estimates for the factor loadings tended to be overestimated by the nonrobust estimation procedures and undere stimated by the robust estimation procedures. For the other distributions, the standard error estimates of the factor loadings tended to be underestimated regardless of the estimation procedures. The bias of standard error estimates for the factor covariances ( (i, j) = 0.108) tended to decrease as the sample size increased. For the normal distribution, ML and Robust ML estimates of the standard errors for the factor covariance had less bias than did the GLS and Robust GLS estimates. The robust procedures had less bias than the nonrobust procedures for the nonnormal distributions. Robust ML estimates of the standard errors for the factor correlations had the minimal bias for the nonnormal distributions. Chou, Bentler, and Satorra (1991) studied th e performance of Robust ML estimates of standard errors and found that wh en the distribution had excessive kurtosis, the robust estimates of standard errors were superi or to ML estimates. Finch, We st, and MacKinnon (1997) indicated that the Robust ML estimates of standard erro rs provided more accurate estimates of sampling variability than ML and GLS as nonnormality increased, and the standard error estimates generated by ML and GLS were likely to be too small. Hoogland (1999) poi nted out that the ML and GLS estimates of the standard errors were biased when the average kurtosis of the observed variables deviates from zero, th e standard error estimates were underestimated in the case of a positive average kurtosis and overestimated in the case of negative average kurtosis. Also Hoogland (1999) reported that the Robust ML standard error estimat es had a smaller bias than ML and GLS standard error estimates when the av erage kurtosis was at least 2.0 and the sample size was at least 400. PAGE 90 90 The results of the present study generally s howed consistency with the results of the previous studies in that Robust ML estimates we re superior to the non robust estimates in the bias of the standard error estimates for the nonnormal distributions, and the standard error estimates were underestimated for the distributi on with positive kurtosis and overestimated for the distribution with negative kurtosis. However, the present study gives more general results because Robust GLS was added, two scalesetting methods were compared, and results were analyzed statistically. Brief Summary The main purpose of the present study was to answer the question: Which estimation method provides better standard errors? A Monte Carlo simulation study was conducted to analyze and investigate the effect s of four factors (estimation, sample size, distribution, and model) on the bias of standard er ror estimates and the bias of parameter estimates in each scalesetting method. First, from the findings of the study, the followi ng conclusions can be set forth in regard to for the bias of parameter estimates: When the scale is set by specifying factor va riances equal to one, ML estimates of the factor loading have less bias than do the GLS estimates. The bias of ML and GLS estimates of residual variance decreases as the sample size increases and the bias of GLS estimation is mo re affected by sample sizes between 200 and 1200 than is ML estimation. The ML estimates of residual variance have mi nimal bias at all sample sizes whereas the GLS estimates have more substantial bias when sample size is 200 or 400. When the scale is set by specifying factor load ings equal to one, none of the effects on the factor loading or the f actor correlation paramete r estimates account for even 1% of the total sum of squares. PAGE 91 91 Second, the following points can be noted about the bias of standard error estimates: In regard to bias of the standard error estim ators for factor loadings: Regardless of scalesetting method, for a normal distribution, ML es timation is superior to the other estimation procedures, whereas for nonnormal distributi ons, Robust ML is superior to the other estimation procedures. In regard to bias of the standard error estimators for the factor correlation: 1. When the scale is set by specifying factor variances equal to one, the mean standard error estimates and the empiri cal standard error estimates decrease as the sample size increases, regardless of the m odel type, but not the same rate. 2. When the scale is set by specifying factor variances equal to one, ML and Robust ML are superior to GLS and Robust GLS for the (0, 0), (0, 1.15), (0, 3) distributions, whereas Robust ML and Robust GLS are superior to ML and GLS for the (2, 6) distribution. 3. When the scale is set by specifying factor loadings equal to one, ML and Robust ML are superior to GLS and Robust GLS for th e (0, 0) distribution, whereas Robust ML and Robust GLS are superior to ML and GL S for the (0, 1.15), (0, 3), and (2, 6) distributions In regard to the bias of standard error es timates for residual variance: ML and Robust ML are superior to GLS and Robust GLS for a normal distribution, wh ereas Robust ML and Robust GLS are superior to ML and GLS for nonnormal distributions The bias of standard error estimates for re sidual variance increases as the number of indicators per factor increases for nonnorma l distributions, whereas the bias of these estimates decreases as the number of indi cators per factor increases for a normal distribution. Concluding Remarks Based on the findings presented, it can be conc luded that ML estimation method should be adopted for a normal distribution re gardless of sample size, mode l, and scalesetting method to obtain less biased estimates of parameters and standard errors, and Robust ML should be used for nonnormal distributions to improve estimati on of standard errors. However, Robust ML estimation works very well even for normal distribut ions and some cases better than GLS. It has also been found that robust estimation generally worked better than nonrobust estimation for the nonnormal distributions regardless of the sample size and the model type. When the distribution PAGE 92 92 is nonnormal, Robust GLS generally performs well, although Robust ML has less bias than Robust GLS. Problems of nonconvergence, improper estimates, and nonpositive definite Hessian matrices were more common when the scale was set by specifying factor loadings equal to one and the distribution is nonnormal, particularly for the 3F6I model with 200 and 400 sample sizes. However, as a practical matter these prob lems occurred infrequently. Setting the scale by specifying factor variances equal to one s hould be chosen to avoid these problems. Generalization from the findings of Monte Carl o simulation studies is limited by the design of the simulation. Although the present study evaluated the empi rical behavior of parameter estimates and standard error estimates under a ra nge of sample sizes, di stributions, and models which were chosen considering the results of the previous studies, the study of the robust estimation for standard errors st ill needs to be investigated further under the various conditions of models (e.g., misspecified models, or hybrid models), and the other robust estimation procedures (e.g., Robust DWLS, or Robust ULS). PAGE 93 93 APPENDIX STANDARD ERROR RESULTS FOR FACTOR CORRELATIONS: SCALE SET BY SPECIFYING FAC TOR VARIANCES EQUAL TO ONE. Table A1. Mean standard errors, empirical sta ndard errors, and standard error ratios by model, distribution, and sample size for ML estimation. Sample size Model Distribution Statistics 200 400 800 1200 SE 0.08520350.06072740.04309100.0351752 Emp. SE 0.09165270.05986770.04482500.0341341 (0, 0) SE Ratio 0.92963461.01435860.96131801.0305009 SE 0.08534020.06050110.04298520.0351710 Emp. SE 0.08947830.06067890.04257630.0361427 (0, 1.15) SE Ratio 0.95375300.99707031.00960330.9731165 SE 0.08507280.06057910.04291770.0351232 Emp. SE 0.08920470.06091620.04414730.0365462 (0, 3) SE Ratio 0.95368060.99446620.97214770.9610636 SE 0.08438460.06045250.04288930.0351204 Emp. SE 0.10150780.07197240.04996870.0385356 3F3I (2, 6) SE Ratio 0.83131140.83994040.85832330.9113744 SE 0.07477180.05298570.03755880.0306787 Emp. SE 0.07708730.05179380.03959370.0305296 (0, 0) SE Ratio 0.96996371.02301100.94860421.0048813 SE 0.07479550.05291230.03751410.0306708 Emp. SE 0.07845400.05178730.03689420.0293325 (0, 1.15) SE Ratio 0.95336791.02172361.01680201.0456226 SE 0.07483600.05305620.03751880.0306476 Emp. SE 0.07642600.05344270.03722200.0299660 (0, 3) SE Ratio 0.97919600.99276871.00797291.0227563 SE 0.07454670.05286500.03746070.0305994 Emp. SE 0.08447310.05994500.04281350.0352090 3F6I (2, 6) SE Ratio 0.88249070.88189200.87497370.8690774 SE 0.08552090.06065820.04300080.0351380 Emp. SE 0.08872590.06026600.04114500.0353648 (0, 0) SE Ratio 0.96387731.00650751.04510380.9935871 SE 0.08500980.06063010.04300290.0351318 Emp. SE 0.08910510.06088260.04409000.0346944 (0, 1.15) SE Ratio 0.95403980.99585300.97534411.0126064 SE 0.08509000.06060050.04299200.0351811 Emp. SE 0.08430960.06253060.04452360.0356247 (0, 3) SE Ratio 1.00925620.96913200.96559820.9875455 SE 0.08473800.06038360.04295980.0351398 Emp. SE 0.10112570.06728430.04777660.0382489 6F3I (2, 6) SE Ratio 0.83794760.89743970.89918090.9187121 Note SE indicates standard error, and Emp. Indicates empirical. PAGE 94 94 Table A2. Mean standard errors, empirical sta ndard errors, and standard error ratios by model, distribution, and sample size for GLS estimation. Sample size Model Distribution Statistics 200 400 800 1200 SE 0.08654750.06118730.04325700.0352676 Emp. SE 0.09472460.06049010.04541830.0343848 (0, 0) SE Ratio 0.91367541.01152670.95241251.0256748 SE 0.08666790.06096040.04314560.0352652 Emp. SE 0.09168330.06188600.04290620.0362790 (0, 1.15) SE Ratio 0.94529640.98504311.00557980.9720555 SE 0.08632690.06108730.04308760.0352161 Emp. SE 0.09329350.06162070.04441690.0369437 (0, 3) SE Ratio 0.92532580.99134420.97007120.9532371 SE 0.08604430.06106680.04311840.0352416 Emp. SE 0.10668390.07347570.05041000.0387986 3F3I (2, 6) SE Ratio 0.80653520.83111500.85535380.9083230 SE 0.07847970.05426360.03803050.0309424 Emp. SE 0.08520810.05415150.04055040.0308609 (0, 0) SE Ratio 0.92103501.00207090.93785691.0026409 SE 0.07855640.05426920.03798420.0309353 Emp. SE 0.08598180.05416450.03818440.0299032 (0, 1.15) SE Ratio 0.91363991.00193470.99475681.0345156 SE 0.07850630.05438380.03798840.0309098 Emp. SE 0.08408020.05630510.03803010.0303428 (0, 3) SE Ratio 0.93370760.96587630.99890331.0186850 SE 0.07903100.05451710.03805040.0309130 Emp. SE 0.09529440.06383070.04430690.0361298 3F6I (2, 6) SE Ratio 0.82933490.85408900.85879200.8556083 SE 0.08787150.06157710.04333240.0353174 Emp. SE 0.09509060.06288970.04196380.0357739 (0, 0) SE Ratio 0.92408180.97912841.03261450.9872395 SE 0.08745930.06158800.04333570.0353145 Emp. SE 0.09657760.06315810.04527860.0352822 (0, 1.15) SE Ratio 0.90558570.97513980.95709031.0009153 SE 0.08761330.06158880.04334810.0353710 Emp. SE 0.08977610.06540290.04579400.0357338 (0, 3) SE Ratio 0.97590860.94168390.94658950.9898479 SE 0.08803380.06156640.04338580.0353801 Emp. SE 0.10905500.07061360.04862420.0390347 6F3I (2, 6) SE Ratio 0.80724260.87187750.89226790.9063758 PAGE 95 95 Table A3. Mean standard errors, empirical sta ndard errors, and standard error ratios by model, distribution, and sample si ze for Robust ML estimation. Sample size Model Distribution Statistics 200 400 800 1200 SE 0.08415640.06037120.04295780.0350993 Emp. SE 0.09165270.05986770.04482500.0341341 (0, 0) SE Ratio 0.91821001.00840950.95834571.0282767 SE 0.08508320.06057740.04310900.0352981 Emp. SE 0.08947830.06067890.04257630.0361427 (0, 1.15) SE Ratio 0.95088080.99832701.01251150.9766321 SE 0.08381080.05999010.04305420.0351824 Emp. SE 0.08920470.06091620.04414730.0365462 (0, 3) SE Ratio 0.93953300.98479700.97523970.9626822 SE 0.09193800.06730810.04826950.0397790 Emp. SE 0.10150780.07197240.04996870.0385356 3F3I (2, 6) SE Ratio 0.90572360.93519280.96599511.0322664 SE 0.07408260.05268490.03742710.0306684 Emp. SE 0.07708730.05179380.03959370.0305296 (0, 0) SE Ratio 0.96102271.01720360.94527831.0045460 SE 0.07451190.05294340.03759160.0307174 Emp. SE 0.07845400.05178730.03689420.0293325 (0, 1.15) SE Ratio 0.94975281.02232451.01890481.0472134 SE 0.07315630.05262140.03743050.0306659 Emp. SE 0.07642600.05344270.03717850.0299660 (0, 3) SE Ratio 0.95721800.98463241.00677701.0233568 SE 0.08184130.05874170.04229600.0346940 Emp. SE 0.08447310.05994500.04281350.0352090 3F6I (2, 6) SE Ratio 0.96884510.97992700.98791280.9853713 SE 0.08457480.06026010.04289380.0350724 Emp. SE 0.08872590.06026600.04114500.0353648 (0, 0) SE Ratio 0.95321510.99990201.04250490.9917328 SE 0.08468870.06065850.04310390.0352690 Emp. SE 0.08910510.06088260.04409000.0346944 (0, 1.15) SE Ratio 0.95043650.99631870.97763551.0165624 SE 0.08332260.06045040.04312050.0352033 Emp. SE 0.08430960.06253060.04452360.0356247 (0, 3) SE Ratio 0.98829310.96673240.96848460.9881700 SE 0.09221640.06713130.04830980.0399671 Emp. SE 0.10112570.06728430.04777660.0382489 6F3I (2, 6) SE Ratio 0.91189930.99772621.01116121.0449214 PAGE 96 96 Table A4. Mean standard errors, empirical sta ndard errors, and standard error ratios by model, distribution, and sample size for Robust GLS estimation. Sample size Model Distribution Statistics 200 400 800 1200 SE 0.08548790.06083520.04312660.0351923 Emp. SE 0.09472460.06049010.04541830.0343848 (0, 0) SE Ratio 0.90248941.00570610.94954341.0234863 SE 0.08643080.06104560.04326990.0353919 Emp. SE 0.09168330.06188600.04290620.0362790 (0, 1.15) SE Ratio 0.94271070.98642041.00847530.9755476 SE 0.08509320.06053070.04323720.0352852 Emp. SE 0.09329350.06162070.04441690.0369437 (0, 3) SE Ratio 0.91210260.98231050.97343960.9551081 SE 0.09391000.06814300.04859030.0399571 Emp. SE 0.10668390.07347570.05041000.0387986 3F3I (2, 6) SE Ratio 0.88026360.92742100.96390221.0298613 SE 0.07763600.05393500.03788680.0309388 Emp. SE 0.08520810.05415150.04055040.0308609 (0, 0) SE Ratio 0.91113330.99600180.93431221.0025226 SE 0.07826400.05428980.03807180.0309850 Emp. SE 0.08598180.05416450.03818440.0299032 (0, 1.15) SE Ratio 0.91023981.00231410.99705101.0361790 SE 0.07680610.05398440.03792510.0309261 Emp. SE 0.08408020.05630510.03806410.0303428 (0, 3) SE Ratio 0.91348650.95878320.99634991.0192230 SE 0.08673320.06070910.04307800.0350903 Emp. SE 0.09529440.06383070.04430690.0361298 3F6I (2, 6) SE Ratio 0.91016060.95109570.97226290.9712282 SE 0.08699080.06116490.04323450.0352456 Emp. SE 0.09509060.06288970.04196380.0357739 (0, 0) SE Ratio 0.91482020.97257341.03028020.9852329 SE 0.08706430.06163830.04344500.0354532 Emp. SE 0.09657760.06315810.04527860.0352822 (0, 1.15) SE Ratio 0.90149620.97593580.95950311.0048462 SE 0.08580120.06146860.04346880.0354037 Emp. SE 0.08977610.06540290.04579400.0357338 (0, 3) SE Ratio 0.95572390.93984580.94922610.9907638 SE 0.09543630.06820840.04869650.0401757 Emp. SE 0.10905500.07061360.04862420.0390347 6F3I (2, 6) SE Ratio 0.87512120.96593791.00148641.0292304 PAGE 97 97 LIST OF REFERENCES Anderson, J. C., & Gerbing, D. W (1984). Th e effect of sampling error on convergence, improper solutions, and goodnessfit indices fo r maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155173. Andreassen, T. W., Lorentzen, B. G., & Olsson, U. H. (2006). The impact of nonnormality and estimation methods in SEM on satisfaction research in marketing. Quality & Quantity, 40, 3958. Arminger G., & Schoenberg R. J. (1989). Pse udo maximum likelihood estimation and a test for misspecification in mean and covariance structure models. Psychometrika, 54, 409425. Babakus, E., Ferguson, C. E., & Joreskog, K. G. (1987). The sensitivity of confirmatory maximum likelihood factor analysis to violati ons of measurement scale and distributional assumptions. Journal of Marketing Research, 24, 222228. Baldwin, B. O. (1986). The effects of structural model missp ecification and sample size on the robustness LISREL maximum likelihood estimates. Doctoral dissertation, Louisiana State University. Bearden, W. O., Sharma, S., & Teel, J. E. (1982) Sample size effects on chisquare and other statistics used in evaluating causal models. Journal of Marketing Research, 19, 425430. Beauducel, A. B., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186203. Benson, J., & Fleishman, J. A. (1994). The r obustness of maximum likelihood and distributionfree estimators to nonnormality in confirmatory factor analysis. Quality & Quantity, 28, 117136. Bentler, P. M. (1983). Some contributions to efficient statistics in structural models: Specification and estimation of moment structures. Psychometrika, 48, 493517. Bentler, P. M., & Chou, C. (1987). Pract ical issues in structural modeling. Sociological Methods & Research, 16, 78117. Bentler, P. M., & Dudgeon, P. (1996). Covariance st ructure analysis: Statis tical practice, theory, and directions. Annual Review Psychology, 47, 56392. Bentler, P. M., & Yuan, K. (1999). Structural equation modeling with small samples: Test Statistics. Multivariate Behavioral Research, 34, 181197. Boomsma, A. (1983). On the robustness of LISREL (maximum likelihood es timation) against small sample size and nonnormality. Amsterdam: Sociometri c Research Foundation. Doctoral dissertation, Ri jksuniversiteit Groningen. PAGE 98 98 Boomsma, A., & Hoogland, J. J. (2001). The R obustness of LISREL Modeli ng Revisited. In R. Cudeck, S. Du Toit, & D. Srbom (Eds), Structural equation modeling: Present and future. A Festschrift in honor of Karl Jreskog (pp. 139168). Lincolnwood, IL: Scientific Software International. Brown, R. L. (1990). The robustness of 2S LS estimation of nonnormality distributed confirmatory factor analysis model. Multivariate Behavioral Research, 25, 455466. Browne, M. W. (1982). Covariance stru cture. In D. M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72141). Cambridge, UK: Cambridge University. Browne, M. W. (1984). Asymptotically distribu tionfree method for the analysis of covariance structures. British Journal of Mathema tical and Statistical Psychology, 37, 6283. Browne, M. W. (1987). Robustness of statistical inference in factor analysis and related models. Biometrika, 74, 375384. Browne, M. W. & Shapiro A. (1988). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematic al and Statistical Psychology, 41, 193208. Casella, G., & Berger, R. L. (2002). Statistical Inference, (2nd Ed.), Pacific Grove, CA: Duxbury. Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper solutions in structural equation models. Sociological Methods & Research, 29, 468508. Chou, C., Bentler, P. M., & Satorra, A. (1991). S caled test statistics a nd robust standard errors for nonnormal data in covariance struct ure analysis: A Monte Carlo study. British Journal of Mathematical and Statistical Psychology, 44, 347357. Cudeck, R. (1989). Analysis of correlation ma trices using covariance structure models. Psychological Bulletin, 105, 317327. Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in conf irmatory factor analysis. Psychological Methods, 1, 1629. Dolan, C. V. (1994). Factor anal ysis of variables with 2, 3, 5, and 7 response categories: A comparison of categorical variable estimators using simulated data. British Journal of Mathematical and Statistical Psychology, 47, 309326. Ethington, C. A. (1987). The robustness of LISREL estimates in structural equation models with categorical variables. Journal of Experimental Education, 55, 8088. Fan, X., Thompson, B., & Wang, L. (1999). Eff ects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 5683. PAGE 99 99 Finch, J. F. (1992). The robustness of maximum likelihood parameter estimates in structural equation models with nonnormal variables. Doctoral dissertation, Arizona State University. Finch, J. F., West, S. G., & MacKinnon, D. P. (1997). Effects of sample size and nonnormality on the estimation of mediated effects in latent variable models. Structural Equation Modeling, 4, 87107. Fleishman, A. I. (1978). A method fo r simulating nonnormal distributions. Psychometrika, 43, 521532. Gallini, J. K. (1984). An investigation of the effe ct of sample size and specification error on the fit of structural equation models. Journal of Experimental Education, 53, 919. Gerbing, D. W., & Anderson, J. C. (1985) The effects of samp ling error and model characteristics on parameter estimation fo r maximum likelihood confirmatory factor analysis. Multivariate Behavioral Research, 20, 255271. Gonzalez, R., & Griffin D. (2001). Testing parameters in structural equation modeling: Every one matters. Psychological Methods, 6, 258269. Harlow, L. L. (1985). Behavior of some elliptical theory estimators with nonnormal data in a covariance structures framework: A Monte Carlo study. Doctoral dissertation, University of California, Los Angeles. Hau, K., & Marsh, H. W. (2004). The use of item parcels in structural equation modeling: Nonnormal data and small sample sizes. British Journal of Mathematical Statistical Psychology, 57, 327351. Henly, S. J. (1993). Robustness of some estimato rs for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 46, 313338. Hoogland, J. J. (1999). The robustness of estimation methods for covariance structure analysis. Doctoral dissertation, Groningen University. Hoogland, J. J., & Boomsma, A. (1998). Robustne ss studies in covariance structure modeling. Sociological Methods & Research, 26, 329367. Hu, L., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted?. Psychological Bulletin, 112, 351362. Huba, G. J., & Harlow, L. L. (1987). Robust st ructural equation models: Implications for developmental psychology. Child Development, 58, 147166. Jaccard, J., & Wan, C. K. (1995). Measurement e rror in the analysis of interaction effects between continuous predictors using multip le regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348357. PAGE 100 100 Jackson, D. L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: A Monte Carlo investigation. Structural Equation Modeling, 8, 205223. Johansen, S. (1980). The WelchJa mes approximation to the distribut ion of the residual sum of squares in a weighted linear regression. Biometrika, 67, 8592. Jonsson, F. Y. (1997). Nonlinear structural equation models : Simulation Studies of the KennyJudd Model. Doctoral dissertation, Uppsala University. Jreskog, K. G., & Srbom, D., (1996). LISREL 8: Users Reference Guide. Chicago, IL: Scientific software International Inc. Jreskog, K. G., Srbom, D., Toit, S. D., & Toit, M. D. (1996). LISREL 8: New Statistical Features. Chicago, IL: Scientific software International Inc. Kaplan, D. (1989). A study of the sampling variabili ty and zvalues of parameter estimates from misspecified structural equation models. Multivariate Behavioral Research, 24, 4157. Kaplan, D. (1990). Evaluating and modifying covariance structure models: A review and recommendation. Multivariate Behavioral Research, 25, 137155. Kendall M., Stuart, A. & Ord, J. K. (1987). Kendalls Advanced Theo ry of Statistics: Distribution theory, (5th Ed.). Vol. 1. New York, NY: Oxford University Press Keselman, H. J., Carriere, K. C. H., & Lix, L. M. (1993). Testing repe ated measures hypotheses when covariance matrices are heterogeneous. Journal of Educ ation Statistics, 18, 305319. Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling (2nd Ed.). New York, NY: The Guilford Press. Lee, S., Poon, W., & Bentler, P. M. (1995). A tw ostage estimation of structural equation models with continuous and polytomous variables. British Journal of Mathematical and Statistical Psychology, 48, 339358. Lei, M., & Lomax, R. G. (2005). The effect of varying degrees of nonnormality in structural equation modeling. Structural Equation Modeling, 12, 127. Lix, L. M., & Keselman, H. J. (1995). Approxi mate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117, 547560. MacCallum, R. C. (1990). The need for alterna tive measures of fit in covariance structure modeling. Multivariate Behavioral Research, 25, 157162. Mardia, K. V. (1970). Measures of multivaria te skewness and kurtosi s with applications. Biometrika, 57, 519530. PAGE 101 101 Mardia, K. V. (1974). Applications of some measures of multiv ariate skewness and kurtosis in testing normality and robustness studies. Sankhya: The Indian Journal of Statistics, 36, 115128. Marsh, H. W., Hau, K., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The number of indicators per fact or in confirmatory. Multivariate Behavioral Research, 33, 181220. Mattson, S. (1997). How to generate nonnormal data for simulation of structural equation models. Multivariate Behavioral Research, 32, 355373. Muthen, B. & Kaplan, D. (1985). A comparison of some methodologies for th e factor analysis of nonnormal likert variables. British Journal of Mathematic al and Statistical Psychology, 38, 171189. Muthen, B., & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of nonnormal likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45, 1930. Nevitt, J., & Hancock, G. R. (2000). Improving th e root mean square error of approximation for nonnormal conditions in structural equation modeling. The Journal of Experimental Education, 68, 251268. Nevitt, J., & Hancock, G. R. (2001). Performanc e of bootstrapping approaches to model test statistics and parameter standard error estimation in structural equation modeling .Structural Equation Modeling, 8, 353377. Nevitt, J., & Hancock, G. R. (2004). Evaluating sm all sample approaches for model test statistics in structural equation modeling. Multivariate Behavioral Research, 39, 439478. Olejnik, S., & Algina, J. (2003) Generalized Eta and Omega squa red statistics: Measures of effect size for some common research designs. Psychological Methods, 8, 434447. Olsson, U., Foss, T., Troye, S. V., & Howell, R. D. (2000). The performance of ML, GLS, and WLS estimation in structural equation mode ling under conditions of misspecification and nonnormality. Structural Equation Modeling, 7, 557595. Paxton, P., Curran, P. J., Bollen, K. A., Kirby, J., & Chen, F. (2001). Monte Carlo experiments: Design and implementation. Structural Equation Modeling, 8, 287312. Ping, R. A. (1995). A parsimonious estimating te chnique for interaction and quadratic latent variables. Journal of Marketing Research, 32, 336347. Potthast, M. J. (1993). Confirmatory factor analys is of ordered categorical variables with large models. British Journal of Mathematic al and Statistical Psychology, 46, 273286. Reddy, S. K. (1992). Effects of ignoring correlat ed measurement error in structural equation models. Educational and Psychological Measurement, 52, 549570. PAGE 102 102 Reinartz, W. J., Echanmbadi, R., & Chin, W. W. (2002). Generating nonnormal data for simulation of structural equation models using Mattsons method. Multivariate Behavioral Research, 37, 227244. Satorra, A. (1990). Robustness issu es in structural equation m odeling: A review of recent developments. Quality & Quantity, 24, 367386. Sharma, S., Durvasula, S., & Dillon, W. R. (1989). Some results on the behavior of alternate covariance structure estimation procedures in the presence of nonnormal data. Journal of Marketing Research, 26, 214221. Sivo, S. A., Fan, X., Witta, E. L., & Willse, J. T. (2006). The search for Optimal cutoff properties: Fit index criteria in structural equation modeling. The Journal of Experimental Education, 74, 267288. Tanaka, J. S. (1984). Some results on the estimation of covariance structure models. Doctoral dissertation, University of California, Los Angeles. Tanaka, J. S. (1987). How big is big enough?: Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58, 134146. Ullman, J. B. (2006). Structural equation modeling: Reviewing the basics and moving forward. Journal of personality assessment, 87, 3550. Vale, C. D., & Maurelli, V. A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465471. West, S. G., Finch, J. F., & Curran, P. J. (1995) Structural equation modeling: concepts, issues and applications. In R. Hoyle (Eds.), Structural equation models with nonnormal variables (pp. 5675). Newbury Park, CA: Sage. Wothke, W. (1993). Nonpositive definite matrices in structural modeling. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 256293). Newbury Park, CA: Sage. Yuan, K., & Bentler, P. M. (1998). Normal theory based test statistics in structural equation modeling. British Journal of Mathema tical and Statistical Psychology, 51, 289309. Yung, Y., & Bentler, P. M. (1994). Bootsrtapcorrected ADF test statistics in covariance structure analysis. British Journal of Mathemat ical and Statistical Psychology, 47, 6384. PAGE 103 103 BIOGRAPHICAL SKETCH Youngkyoung Min was born in Seoul, South Kor ea. She received her bachelors and m asters degrees in polymer science and engi neering from Chungnam National University, South Korea. She was an engineering researcher of the research and devel opment center in Kumho Chemical Inc. In August 2001, she completed her M. S. in Management at the University of Florida, and she also achieved he r M. S. in Statistics at the Univ ersity of Florida in August 2004. In the fall of 2004, she enrolled for graduate studies in research and evaluation methodology program of Educational Psychology Department at the University of Florida and completed her Ph.D. in research and evaluation me thodology in August 2008. Youngkyoung Min conducted researches as a research assistant at the Depart ment of Health and Clinical Psychology at the University of Florida from August 2004 to August 2005, and in Engineering Education Center of College of Engineering at the University of Florida from August 2005 to June 2008. 