<%BANNER%>

Navigating the Complexities of Legislation: How Elementary School Principals Interpret and Implement Florida's Third-Gra...

xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20110318_AAAALA INGEST_TIME 2011-03-18T18:04:00Z PACKAGE UFE0012802_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 81506 DFID F20110318_AABQEU ORIGIN DEPOSITOR PATH zmach_c_Page_129.jpg GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
c3baf795b76ce4b97a88c9add2d70ddd
SHA-1
30b8352dfc3aab5c0609c938ed3c1017719f744b
52501 F20110318_AABPZO zmach_c_Page_069.pro
146c2aaf3c3286dafadfd7dd623e4f7d
db6c018badd1901475b5aad70f827f17d415584a
48750 F20110318_AABPYZ zmach_c_Page_046.pro
14620d9e87679ad07677432ba3b8d9de
7b27c0ae88e6a68311dce18c1214b74c3f466f3a
29211 F20110318_AABQFI zmach_c_Page_001.jp2
e9b78464049dbc9128e2b5c0081fc445
d37286baf98d333e6c923139cb16081e5df529c0
93163 F20110318_AABQEV zmach_c_Page_131.jpg
d26a2cfa66cc0a517c32cdeae98c7723
d84c0afa378ac3a529720aaab7e242cb3b4ed1dd
49306 F20110318_AABPZP zmach_c_Page_070.pro
3d0751f14ae74a2eb0dafd77821dbb14
b3ef956a1f706e5da345abce643f3b9dbd47c359
96560 F20110318_AABQFJ zmach_c_Page_003.jp2
a78f50d57f71f440f225a91f15feb654
079f4ccce74849d671c6c4e3824ff8bc91f00032
10830 F20110318_AABQEW zmach_c_Page_132.jpg
62e5b09897f0cb3172586ad4beccf831
8afd76fc06facf0ae3ebd9562d4176fc92a797e6
30831 F20110318_AABPZQ zmach_c_Page_073.pro
94aae40a1845996f6a11d5a47083a433
10f509169c04bc86acc360fe91c925bf1e4bd521
20502 F20110318_AABQFK zmach_c_Page_004.jp2
054960efe637e35762dc08ae67b9f146
77898da24aed083533387ee3b5c52ba58c8e5b54
55183 F20110318_AABQEX zmach_c_Page_133.jpg
65cf59d829842e10a6ce46e79bd81ac5
0d54925e72e71221241c31b2416c19d21b058f07
42890 F20110318_AABPZR zmach_c_Page_074.pro
8b05c4f32529409b69ccf518fa4ee940
a72be6e1926c34b29803a43281521ed329b37e38
100157 F20110318_AABQGA zmach_c_Page_024.jp2
678d6a8fe2d491f24485e403fa57cc42
78cd794ab88ad600c995aebe256f6c9e02fd20dd
1051986 F20110318_AABQFL zmach_c_Page_006.jp2
781e2cb0b0a640ad7c4a15078be00bc0
cfacd94f4fcddc36ff0307e12aac4c4ecfd83b26
5675 F20110318_AABQEY zmach_c_Page_135.jpg
a4bdcb6c9d21da1c5a09e3a053f4187a
a7018010b46ab77feb39a2db855844b2901c8474
50406 F20110318_AABPZS zmach_c_Page_075.pro
b3b160bf0653d418a2797db25b971f22
39605d39afd554acf23e62eea72572477bc9c071
110762 F20110318_AABQGB zmach_c_Page_028.jp2
40ec913c5777a3987859fcd760110e4c
67923e25b6c736435f41b758764778943f8feefb
1051915 F20110318_AABQFM zmach_c_Page_007.jp2
672dd6094c9f6cba5262f04a7d787bf8
ac27dad7b90159bc8f96ed9935ac31110e93835c
120272 F20110318_AABQEZ zmach_c_Page_138.jpg
bcf1265250ca071ba956ac90af64df72
8e1356c40ed1ba59e0d0b40d7bd90a1be9f7cbd1
34033 F20110318_AABPZT zmach_c_Page_076.pro
2caff568c7843f295c801ae60863549f
23914a8fade0fcdfa66d1e1443713e95e2eec07e
77579 F20110318_AABQGC zmach_c_Page_029.jp2
aa5391c90407e3625525773153d37dba
25ab81aea73a4e6151b725344eb300f758008ffa
664136 F20110318_AABQFN zmach_c_Page_009.jp2
f9ee58e279bdf4935db3d877964607f6
3e59949f90b359e31d7c5ada17835d07e19fe087
35760 F20110318_AABPZU zmach_c_Page_078.pro
2a9774f738577ad6b8ca3a58ee60750f
77ebb5481b1656f71cec80db7ed0358223cdbba0
113767 F20110318_AABQGD zmach_c_Page_030.jp2
f831ddf004e13e387c2b726d7904f243
1e915269d611c98a7068f40b83a5e99d5eeeb5ed
85736 F20110318_AABQFO zmach_c_Page_010.jp2
9356dacd623c5b427bf91d6907704186
020f7b9e3c2eaf3e87831686a52ba61c56193054
41745 F20110318_AABPZV zmach_c_Page_082.pro
c9fe9e7917f40a7d90c5abf171cb2532
2095ebca2ec8666d11cf5f34209ba358f7d9d02c
110273 F20110318_AABQGE zmach_c_Page_032.jp2
ff3fff4dd9ca476e8220a053f7bfe011
730f755409ff217dca4c83645bda093d3682266e
70386 F20110318_AABQFP zmach_c_Page_011.jp2
c1c408b2e0b111edf75c743bab9e9ccb
023802e04452e54c4a1e174052e86b96422d48f2
47198 F20110318_AABPZW zmach_c_Page_083.pro
4c7a55272b4fa35b26a325b86fb1d526
b1d1fb3f00ada778a57b55dfb085be930618b011
111069 F20110318_AABQGF zmach_c_Page_034.jp2
7168ac2805aa9a1479ad8cff90466a93
afa366ef8a7a72b7e0fde04500399524379daa34
92394 F20110318_AABQFQ zmach_c_Page_012.jp2
556764b966fb684fc7037ec6292fb8a6
d0e76d16b831648e20c12dc2304380e797ce805e
22732 F20110318_AABPZX zmach_c_Page_084.pro
cdff9a2e8f8b274046b9a91e41aa5184
b11997f343075d318e53d089f27fd4cb8d9db4c0
112299 F20110318_AABQFR zmach_c_Page_014.jp2
235675521063f7102ad6f28b194222ab
75d0966dc673976803d2c86ec434a14cb3c1fa69
32997 F20110318_AABPZY zmach_c_Page_085.pro
fac9f7b3ae466d529ada423c0a9b67b6
85193e5ead310bf1c30fbac1c74648ce48c0e367
112105 F20110318_AABQGG zmach_c_Page_035.jp2
4a3013f4d468d20c3bcfebb9b331c5fa
3f1b293e80cc97b0eefa0831520ea61dee96b493
99138 F20110318_AABQFS zmach_c_Page_015.jp2
d5642285d86fb578e252eecb85901dbb
40abf81cbb3d9fb8f6047c2255a77bc508639c03
46659 F20110318_AABPZZ zmach_c_Page_088.pro
fd7d51119f50a6704995c9d010c33b69
278cf59730587228696e24d1d866d071c713e9f1
107601 F20110318_AABQGH zmach_c_Page_037.jp2
716395f78fa142605da0c973941488d7
100e78b471e491e48098a8e58776e909a2000308
90397 F20110318_AABQFT zmach_c_Page_016.jp2
159f5a67895d2c19440ea6dfeee3ff22
1d5be7fd833a0ec57ac883152b5e0dad9d6c86c9
104835 F20110318_AABQGI zmach_c_Page_038.jp2
85c74d9ed52a6c099aa776c46dd8d6cb
46bca054b8d3f2894e3dd9cc554344d353255705
108984 F20110318_AABQFU zmach_c_Page_018.jp2
6c3ba32d349ed87b132a34911f624f88
ab857295791181231aa2011eec7e220d2545be8c
109370 F20110318_AABQGJ zmach_c_Page_039.jp2
2ceb0dc9f5d0d7e3c7a588becef05112
b3700b4128a7eb58a85825290f2d6a1191122c7d
109955 F20110318_AABQFV zmach_c_Page_019.jp2
615e92e4334cc875c8e729538bf7bc3c
4fc8e0b392ffddb5736b411db109bcf24036f868
110711 F20110318_AABQGK zmach_c_Page_041.jp2
5f6caedc4e8179e6dc0efc5508ea129e
cfee6d416fe03c122a5a01376358e14cae70417e
102737 F20110318_AABQFW zmach_c_Page_020.jp2
2b2cdae45f2afaae511d616c74d83d8d
596a1c5f758ac732a0a42bfca0c92e92f12297da
77023 F20110318_AABQHA zmach_c_Page_060.jp2
14c4c9bf951151b1feaa32fe571106dc
8a24baee1fa705eefc95dd553a6996c27295ae53
101602 F20110318_AABQGL zmach_c_Page_043.jp2
42d18be52f214982c347ebe523b411e9
34b95caea7e8721857a5f1819f846cfe3d77eda9
110978 F20110318_AABQFX zmach_c_Page_021.jp2
e4647026084d8950529bfe406ba0530c
d9d189f65c09f67b45166b861bf2b82235d55abd
99277 F20110318_AABQHB zmach_c_Page_061.jp2
55ce49e3fa623624c75ac2d5723dd53c
77a0281702c1c0a153548a403306de9f813ef90f
84601 F20110318_AABPDK zmach_c_Page_063.jpg
0d979aa972d29140dad6bcdbc576b86e
fcd67d3a8ca18bc565de897842b31f06eea491b2
105444 F20110318_AABQGM zmach_c_Page_044.jp2
2b182adfec93adb9bd4a8f1376e1777f
9d123e9bc343e1fc439182a09dca47ca0b2f6806
110277 F20110318_AABQFY zmach_c_Page_022.jp2
93d6eb6d7e575f99ca893839bdab078c
9111c2d06302383a321abdad8295f7bcc0e79628
95893 F20110318_AABQHC zmach_c_Page_062.jp2
8869a33ec21556410105e52e94431935
ef6155032c5b046e0facae40e87989c5dea3d41d
6757 F20110318_AABPDL zmach_c_Page_029thm.jpg
8103c4a433e47fcb37309e4b4c50a004
5206c1c5ccc4c31afc4ba706bbb260683b19ac0a
F20110318_AABQGN zmach_c_Page_045.jp2
457c50e507579071fe6fdf07c442909a
0de01b80ea538028ea5a7d8bc00ac27823086af1
110048 F20110318_AABQFZ zmach_c_Page_023.jp2
47e294b3c75afdede63fb7a6825c76ab
b19262195014ea523d8128cfef74a66df63fef65
77727 F20110318_AABPEA zmach_c_Page_068.jp2
73b47d615ea6bf7cc17cf0f9d54f3705
2d061fb8c5051d430b20d951170dd39a5da92c6f
88659 F20110318_AABQHD zmach_c_Page_063.jp2
2cfd5cc12251bce75fb12699a3eb387d
1ad4dfacb2b5d1916a34f61a4a8263e011a80c03
1053954 F20110318_AABPDM zmach_c_Page_086.tif
2d70f8f7331aa1b2e4d5550954cdf829
36fd0041c2b68229e8b3547d094a3be18dba8d40
97648 F20110318_AABQGO zmach_c_Page_047.jp2
ca1d0aa324b1e2cf70bf6fce017e169a
20b21044ae1a82279a59f76bdf2ebe03dd674a90
F20110318_AABPEB zmach_c_Page_038.tif
b5dfb6c3f96e1a63570526e114ea141c
0a9911f4a97fea5fd19c418d2b84a399e1e39787
100448 F20110318_AABQHE zmach_c_Page_064.jp2
6e809879428db8c1519138da37cda50c
f4e2ab6e7e806057ff658e86866198c62d54dedf
22093 F20110318_AABPDN zmach_c_Page_072.QC.jpg
29b817f1be697ec4fa1c92965d0d9836
c67ad1ad022316a54ab8b18125e2645c2b35c687
123289 F20110318_AABQGP zmach_c_Page_048.jp2
93d048f3f17c9207e8137fd34358bb0e
76b0a1266ebf819d8fbf76f56cca901f5792936c
1623 F20110318_AABPEC zmach_c_Page_107.txt
d22be25ec61de735872c92ab962d37e8
d5ef85653191e4da0742fdbe318358ddec4ec424
839884 F20110318_AABQHF zmach_c_Page_065.jp2
2b6d642b56210937b2825da081f8b7fe
8c355fd4e893da2d2464ca9656d2b3d78eb33371
6621 F20110318_AABPDO zmach_c_Page_098thm.jpg
9a59a8cd1c8ed1e1d54a3bf78188c60d
2c88907ad053b457864a58560a5f7b9d31fbe252
110336 F20110318_AABQGQ zmach_c_Page_049.jp2
eec434b9268773973779436db91830ba
9248a0011c57dcdc2f72a1996759ada3aaa0e686
2070 F20110318_AABPED zmach_c_Page_113.txt
2b30878ee9e7b47826a4360d92837c19
eba4c3aff15d48e908a64a799ceb8806d308dba5
108713 F20110318_AABQHG zmach_c_Page_066.jp2
bc2455d9a37ecf9fb032336023388577
af13ae22b0e17eef4236a5f14931b84a5c0d2a66
110020 F20110318_AABPDP zmach_c_Page_027.jp2
d68a2eec13b0707757b58ba40e604b0d
a56f01207e76d19b30ac8c1fa9215c14ddd69ea0
109841 F20110318_AABQGR zmach_c_Page_050.jp2
4149c9793517032635c9933c51722940
b77cacdce4ef4847e7ddf45eb98acb6ee67acae6
1936 F20110318_AABPEE zmach_c_Page_053.txt
e14dd05f04c7b71baa38222d768f87d9
b5ca5b701d0ee196e343fdf5bb7802018df2be06
36649 F20110318_AABPDQ zmach_c_Page_065.pro
679cb4ca38209eeb8955a91bd1836e3f
81c78ac46f3e56705db087ef7add49404b1c95c0
105794 F20110318_AABQGS zmach_c_Page_051.jp2
350316c6b74e96137c22cbe6352c91e8
904680c43fcf8788847684dfd1cd07c63e18ee7d
F20110318_AABPDR zmach_c_Page_124.tif
c552a5f57daa8a999af4f9825030d3da
4fd314442a2b041bbdf71f6fcf9ac7d7788f4c58
95104 F20110318_AABQGT zmach_c_Page_052.jp2
387381d1b375525c48f94db96b3c2d3f
5f832ed2b7c09ced82b83ccbaae0f9f7b2b6a2be
35274 F20110318_AABPEF zmach_c_Page_140.QC.jpg
6f696d8ccc7e9aa8080a09431fe9c8ea
e0eef5bcc1bc03748bb1631aab162748e0350dac
112602 F20110318_AABQHH zmach_c_Page_069.jp2
66ca68f04fb38c0aa0de04d4596b90d3
339fc214566e31de039974eec6498e21a3ec0051
31303 F20110318_AABPDS zmach_c_Page_108.QC.jpg
783407510ecf4dd2a9a9f028871c4ef0
9eea4f736d75c10760d8cfd8a8a9ef9ef66406fb
105671 F20110318_AABQGU zmach_c_Page_053.jp2
b213856a9dbc04ad795d70fb99eb15ab
7b16c3be2d01efce7c770ba7536d7163cc77a71c
F20110318_AABPEG zmach_c_Page_097.tif
d42c0d85caa9a7505d609356a946ad45
75298afec14dba59d12bb0dca2bc8ab8554a61be
105020 F20110318_AABQHI zmach_c_Page_070.jp2
86e73fda0357bbdafe1785c4b46da12d
7806ac9004cd4861f1c9af5ba9a235553957b2af
2023 F20110318_AABPDT zmach_c_Page_116.txt
03476b1661578cb3e013ebac08878f50
12c23d823bd6849bf31f0ce55f336a053166a845
111599 F20110318_AABQGV zmach_c_Page_054.jp2
728b9c15c145e4a68d28764389770388
1ee981098ad99b1b5f24a3a4a141808ac7f6dbf5
51096 F20110318_AABPEH zmach_c_Page_027.pro
4fc6c09a4c18ec39251fc5db8b6281a6
4565d0ac853e9972dda9c4d464694072b98d6eb8
103675 F20110318_AABQHJ zmach_c_Page_071.jp2
5d6015fa5f7b940210363969d4c95410
6a45b0e36ad6e37af9ac238b3d8c027b588233bb
15290 F20110318_AABPDU zmach_c_Page_081.pro
c85464d0963075b96a0d20d0960dbfbb
3ec719850aa984b8eb54472c104c3aba30338cba
111311 F20110318_AABQGW zmach_c_Page_055.jp2
69ab5e428be178bf13dc7411a73b49a3
d714511d290e7cbb34001e16c866961bd1dec995
52066 F20110318_AABPEI zmach_c_Page_049.pro
4c162d276550a481652e686e9a078b6d
f106eb89013a17c715882a9c234ce04380a5b886
94706 F20110318_AABQHK zmach_c_Page_074.jp2
6c572b0538e1054b109813aa0b3b5afb
27dd009b2cfde9f2e669405965c7b5c4ff028b9a
60321 F20110318_AABQIA zmach_c_Page_097.jp2
75801090209ccb5185502a98846ab967
595f058fcf17ac6bb58602dbd65d95d5938d7c57
2454 F20110318_AABPDV zmach_c_Page_146.txt
dd19ee7781cd015ebe9b3df14c4566fa
c845b7b4f41350677f4836cb7a7c3ba76a9a8e02
137608 F20110318_AABQGX zmach_c_Page_056.jp2
659e8199ab89b85c02d4092db013c803
d3431ad86204ea2e27b81374bda8a22c098b57d2
31177 F20110318_AABPEJ zmach_c_Page_001.jpg
a55e7f3cc6e545b29f8201b68220d737
ed8649172bd59b5067ddf208813848d5082ae473
746461 F20110318_AABQHL zmach_c_Page_076.jp2
0de641f1bf35613c67e1dc6dd38fac77
2d4fa09aea012e036e93b817f2635cc7a27f1724
80901 F20110318_AABQIB zmach_c_Page_098.jp2
0c4f843c60ca1047e679ee6b063ffab0
237a46d91c74553d4678d9164e1aef4b09303dec
25271604 F20110318_AABPDW zmach_c_Page_045.tif
79b8be4575f7ccadd6a82f3433f306af
dc4e115a2c19913940c88ed47e29cbc2ddabe2ab
107849 F20110318_AABQGY zmach_c_Page_057.jp2
22b3b0c3f72ed1ed3cbfe87f10929d3b
7717604a4d736091c1096dc45e6cb035cbc01631
75615 F20110318_AABPEK zmach_c_Page_029.jpg
e8e3a4953245e121e9fd27881d5e21fe
355fb46da09d6aef3de551ce13573210cb6b5e5b
695461 F20110318_AABQHM zmach_c_Page_077.jp2
46660f29306125e541159c4bdf9f6abe
e37b70d96036571664e48025a811eb4df1156433
774642 F20110318_AABQIC zmach_c_Page_099.jp2
02bcd3954146a75fae10972984a85f4b
6d5dc83f0f10b9ad80870ea5b77d185789622c1d
38720 F20110318_AABPDX zmach_c_Page_010.pro
e106dc14eced0a9caadbff08cceee011
c86acda7d949e6ec9ff7c819d77f1fa5b636ce2f
37858 F20110318_AABQGZ zmach_c_Page_058.jp2
31224265a383d11365789ad30e20f84c
31f037251250e012e640c9f719cc14f5ab38a58d
F20110318_AABPFA zmach_c_Page_018.tif
a2379f4cd701e5e35d0a39b96cab898b
c0885147e11780f8049f7abded6e7e1b3e117883
F20110318_AABPEL zmach_c_Page_041.tif
66366f5d494ea4705f58c6620a1b7c9a
350bdeb6a8e88be001bde3299b8a79cd87d3bf03
983193 F20110318_AABQHN zmach_c_Page_078.jp2
a16ea3a13c55eb03cb580021b6f6c355
a2c1416a101523685b86f4807f3de65df3fd0ad1
465573 F20110318_AABQID zmach_c_Page_101.jp2
94a1ac3ae9d8e34f71c4d618d0f07610
2476583974e944b4c2ef6e82b6f8a07be00f8559
18261 F20110318_AABPDY zmach_c_Page_130.QC.jpg
bdf55bc10a0b8c9bea73fa6fea0979de
6ece8e6b067a96f7ddae770f243e92d41c89b5bf
35563 F20110318_AABPFB zmach_c_Page_144.QC.jpg
7fdc6d62d13dac4f900ddc1c1ba4fc06
8b2b5be8545b31a15cf6a9679e49e8d2b1814cbc
F20110318_AABPEM zmach_c_Page_084.tif
1534caec7aa4058f1c49879a94660774
d2cca4f331fe19d72ea6ff34ccec59536fb8695d
108144 F20110318_AABQHO zmach_c_Page_079.jp2
9af0370ae5d06e0a0bb45f10019f6f0c
ba122d9d1ec6bbe922b39c37c35da2029d137809
110223 F20110318_AABQIE zmach_c_Page_102.jp2
b6a7e7231d8872783db24ff917ea565a
64a88981e2cc08bab27244eb720bfc665a2ae44c
F20110318_AABPDZ zmach_c_Page_015.tif
0de57bfc83c27e11cd87e6be82c8d2f7
066cdc1ec5f3afd35a6e54ce0893916e5b007576
6623 F20110318_AABPFC zmach_c_Page_010thm.jpg
9b145944f7907aed4fff4a17a572493e
660a5fd2faede533f0dd7cbdf5af9a8e733932e6
58580 F20110318_AABPEN zmach_c_Page_138.pro
3dd7309f14aed5167016776b3e623d9b
8d59db1ee9cbf90c9d6789903be694dc9f933ecd
336293 F20110318_AABQHP zmach_c_Page_081.jp2
69118f8e00390135bf1e97594e23d128
ca1ba9cf7ee9d1fb4da89c593e9384584a9cf82a
62941 F20110318_AABQIF zmach_c_Page_103.jp2
974eabeb753c62f36a4403767199e051
d53ee5bf059cf45c92266062d19529e254958b10
819804 F20110318_AABQHQ zmach_c_Page_082.jp2
4beedd0dc7ea705b1e0362781ed45b04
394f34298fbd92c491ba9af774219e0be2e2842e
2395 F20110318_AABPFD zmach_c_Page_048.txt
5aa4d90d7e81a73189c5d1a9b36a409c
56024820d9865007377533a8e24c274ee6cf153e
79325 F20110318_AABPEO zmach_c_Page_148.jpg
b07e4b6a816201d9687437154e289b37
e6d51f00c386773b9f64f814a1f2918403f33c93
108617 F20110318_AABQIG zmach_c_Page_104.jp2
b59f13826a871ebba8bd9c1c90f1b637
fd9f038d6fc9e51e5dbafb7a14cc37beada1354d
64744 F20110318_AABQHR zmach_c_Page_084.jp2
41c5c05766098e7c837b531c9b6d75cb
93118b2d908304a1ce7c50f3f0f0eda09803a2b5
27653 F20110318_AABPFE zmach_c_Page_012.QC.jpg
ef463a54cfc7b3db4603076615856af0
d3fa1d74bd5fb25fe30b035fa672884f17296684
36261 F20110318_AABPEP zmach_c_Page_098.pro
8aa24b03771ce353aff4ed773e9804ff
57c08d3081e71fe44c07285530a4cb149fb5f9e1
57144 F20110318_AABQIH zmach_c_Page_106.jp2
edc9a9a5555d7ccdf0b81e8e315931c8
565ff0dca279f807157c983696d1e99f18ea8880
728121 F20110318_AABQHS zmach_c_Page_085.jp2
80d05c8afb603d12c40caa5795ee378b
edbe0c2e47470464a73451f43a3e9c8346507f1b
32657 F20110318_AABPFF zmach_c_Page_036.QC.jpg
976eb5e193a93dc640ec9f88fe2e0e8a
775b9c26b827b89d479ce7811fa704e48461e0d9
47368 F20110318_AABPEQ zmach_c_Page_038.pro
cdca3a084530b4adf022c2af5a308eb6
d7f49d0f99a0b6e3ee3c1b879e58be8e78d7b171
755340 F20110318_AABQHT zmach_c_Page_089.jp2
dc424d29a846bbda6f61182bac7b7649
67cde440d34e8db076d9707c7b1ca47cb9d1999f
126 F20110318_AABPER zmach_c_Page_002.txt
362067f5cc986471a6c9ba6db5d59615
15da41aa08c119466da974f6ddbfe3ef335149df
74247 F20110318_AABQII zmach_c_Page_107.jp2
c56d146f3092f13474fde29104b0a12d
338a471fc7539058745f5b6c4da4dbe133e170e2
92558 F20110318_AABQHU zmach_c_Page_090.jp2
e0d41910879c561d0192110c22f55a37
f4de54e0b6540b15d1b05955c00a4a7a89b1a73d
28107 F20110318_AABPFG zmach_c_Page_134.pro
dc5805601f08cd37c795a2ca544c36d0
baa24d1547afed32b73692618c002d5172cef5a6
56304 F20110318_AABPES zmach_c_Page_130.jpg
dc42b189b5e846f166f80b103ad9b816
ebed65a740a12a6d941989c0a4526f7b96180a4e
52521 F20110318_AABQIJ zmach_c_Page_109.jp2
edf4cce20af0bf25d9004a11af5b35b3
a41a96f4adcdc71bac346d32e44aa9923041fcd4
914887 F20110318_AABQHV zmach_c_Page_091.jp2
c819a42e2a3dc244a018d19b0e6e150c
304e9d50f61c15a86bce28db8028aa9724a28e57
F20110318_AABPFH zmach_c_Page_023.tif
072386ce2c8dbeddd1c53461ba8c684a
198766e917a39d2c25bc754b7952e9edff0492c4
1051985 F20110318_AABPET zmach_c_Page_008.jp2
b52b5bfd52f00ea65374a7842e1b2705
b9a6ec917f4063f78f738ac8288b8d4d1801229b
1038356 F20110318_AABQIK zmach_c_Page_110.jp2
8e014aedc40e430fdb9e8103e9a2c5e3
73400e372784ac6d1da11dd3eece79909b369730
93281 F20110318_AABQHW zmach_c_Page_092.jp2
ca021548f13dc66b3e921a05f581b427
045ccf4cec8575bf6936272c30d8a28223c9b9b1
96019 F20110318_AABPFI zmach_c_Page_017.jp2
3f3839215b2edeff05e169bd668acc61
3a2290d368753069b461ebb959832c54770d61bc
50115 F20110318_AABPEU zmach_c_Page_079.pro
c3e8cab7d3599d61c507ce47f8febd1e
9d788778065946e943a89d08a8ba4e03c2a92160
82913 F20110318_AABQJA zmach_c_Page_129.jp2
9ec7a223051c082a591ffb67f2daea1e
c0f07159366f7a47f737e0b68b9b8e34005259e5
108049 F20110318_AABQIL zmach_c_Page_111.jp2
72172beec96ecc3364aca6a62871ec6b
f242dc48c83c4b35926fd7479caffad64eef26e6
94886 F20110318_AABQHX zmach_c_Page_093.jp2
bc50d9159275189205e06cad4b87ad4f
2694036a200c48769e9e5185ef0bb7f1c73d6931
58226 F20110318_AABPFJ zmach_c_Page_133.jp2
cc1d9583d8ca5fa9b4e48c516fea7f97
c02b72b8b3da02a5dabc7856da85bce677c663aa
8553 F20110318_AABPEV zmach_c_Page_054thm.jpg
7e0b67f7f3ce1fe26883aea4304c68a9
e699955f1fb9bba2a07e4fe4d8e0ea7ac122920a
58338 F20110318_AABQJB zmach_c_Page_130.jp2
b8e81fd9432183c5f966ee83133e83b9
0e6fd5b9537f9f60d63a21e3118354df9495ba26
111005 F20110318_AABQIM zmach_c_Page_112.jp2
e4a02883595f19be44868a8ce425e11d
1567d0f8d064153f9657508942f278ecca1d6de1
93630 F20110318_AABQHY zmach_c_Page_095.jp2
33560cccbeeebb0be7fa4c8d9f8b30e7
76f11f93ec370f497c331b592216d3e5f8e15611
5810 F20110318_AABPFK zmach_c_Page_109thm.jpg
6f98ff8a66835aad83d3920bb86027e7
9d4b2b2224087ea60c122925db9e48d2a35ea1f4
100906 F20110318_AABPEW zmach_c_Page_013.jpg
c0c2b6a78502aeae16ced6d4ec1d19ad
66c9ed22ca86a862afac681151115e36dff4c64f
98621 F20110318_AABQJC zmach_c_Page_131.jp2
548b1d4d092b481ab5726d1dd23e0ee5
bfb5b6b5cddc88c4aa42e12cabf76647610eea48
106165 F20110318_AABQIN zmach_c_Page_114.jp2
cb37e392350e64a189650f673fa8552a
3362a30c9fda128ba42956a7391e840b078f56b0
110609 F20110318_AABQHZ zmach_c_Page_096.jp2
71093dfa3eada48bdf08520d04af13c5
623f80d2fa935651e955dc092ccf2766f6b2fbae
17301 F20110318_AABPGA zmach_c_Page_134.QC.jpg
b9581b07a2e9283055e69c24576bd28e
99a9af3629d16823a4fd4031bdeace787a08e8c4
50587 F20110318_AABPFL zmach_c_Page_039.pro
405ce32c01e24481524621c051e6ecce
ad95878f02321c3e0157a54b846a10771f727584
F20110318_AABPEX zmach_c_Page_115.tif
6141f73417ac8d685834913072efa0da
70996588c23522709eba977545aa4cf3194daacb
1051970 F20110318_AABQJD zmach_c_Page_138.jp2
5e838893e8060d3d8ab3a29a649b29f8
7ed325693553f33522b6bf13982915e9abff8067
112070 F20110318_AABQIO zmach_c_Page_115.jp2
14b153ca737a87eea68e546093dd798d
e5d48f7486474a83cfffe11328da61dc05b296e6
1892 F20110318_AABPGB zmach_c_Page_025.txt
d43a3511d101162839fd5e22be798e64
67724d786c92fffb446ba8275a0f36d9d34e3d41
127816 F20110318_AABPFM zmach_c_Page_056.jpg
196c331c98327aa1022bb9028f17e403
53738e0bdbc2f83a528bd50639e0c8a55ec3515f
102526 F20110318_AABPEY zmach_c_Page_019.jpg
535acf53670de176e4998fea60cbacd4
d32b414e25ee85119ddd8ca64c3dddfe429a7b51
1051966 F20110318_AABQJE zmach_c_Page_141.jp2
e5b330048c2f9557cd3bdef790cc4bfb
74c4511a10278d22de13370bbceb664aa16a3c22
112009 F20110318_AABQIP zmach_c_Page_116.jp2
ecfb5babd12a9ae642b79f26020d9696
519fc84ab9c853f2fed88d0648eff536ddb359ff
F20110318_AABPGC zmach_c_Page_079.tif
0104d2571620339dea3850b28ebe0bec
20152119ab22ec1e1602145bd332c57c20b1820b
11743 F20110318_AABPFN zmach_c_Page_132.jp2
127e47c3700bdf615d374d8adcfeacc0
b6994d44cf95af551854a092e72aee89538ab3f0
106810 F20110318_AABPEZ zmach_c_Page_040.jp2
a0958259fde1dbb850a2b027bddc9da4
40f0dee3d81585ea52c06893dee85d0df6a4fda7
1051982 F20110318_AABQJF zmach_c_Page_143.jp2
b238629b82427fecf6be38b01bcbbd19
a1c5cbb255525912ff8d8f2065d796efbd2f4a31
110224 F20110318_AABQIQ zmach_c_Page_117.jp2
ccadd8c79901f3a9026081163503cf15
89bfe8a31424d3c93a8abb47a46d8c0db91ef434
112081 F20110318_AABPGD zmach_c_Page_048.jpg
fe86c3c791296a8dd8981516cc43463f
81ff593ad8760af6884a2cebe68c8dc202274edf
1893 F20110318_AABPFO zmach_c_Page_033.txt
e55107ad625e3c856896b5abd3d06896
eb468ea5ae3cb4465d5664b2dad9708097fad089
1051983 F20110318_AABQJG zmach_c_Page_144.jp2
fc4357ce0424ee6145f2d3754e55d142
0af72196b93d0cb07b88359b20d7867004e84d35
99003 F20110318_AABQIR zmach_c_Page_119.jp2
f999ef7d0a168f8bcdab468b02381b39
2ef9aedad8fadb3fac11f16da2b2c26ca563da8e
F20110318_AABPGE zmach_c_Page_142.tif
2c154176913bf643900d2967c4b358ce
1beccfa8e1790a4efea2a39022d295ecc25986d0
F20110318_AABPFP zmach_c_Page_142.jp2
609adcb6975e1f1b335638de1fa07383
844c350d54942523db54de829ae3b467c984c2fc
132778 F20110318_AABQJH zmach_c_Page_145.jp2
c2c40c26ffe6170aa4e3b7d1334c0952
116d7d49c4bfd5b491ec1e7a8bffa432bdbee9a4
108534 F20110318_AABQIS zmach_c_Page_120.jp2
80e68f520282dbba547e5b5c6263c131
34017271b6158b78a422553605009f98850af422
115612 F20110318_AABPGF zmach_c_Page_100.jp2
529c6a8dddbc8e1f08f528999234821b
9892deb488d97a3cac5df9d4eb54158dc15377d5
1949 F20110318_AABPFQ zmach_c_Page_070.txt
2406b540be6eafc7fc9978e4958173cc
37a6d87e8e8e2929f2ce992c7a2b432b58b91bc2
1051950 F20110318_AABQJI zmach_c_Page_146.jp2
6ddeeae209ea4bb25e512aad9368e7df
d4ce31f71f55a28fffe9e49c6da731b3b94813ed
63327 F20110318_AABQIT zmach_c_Page_121.jp2
1c098b69c0a8e9ed35b320c7f45ebf47
d952bf60b59eb80dd2035c736cc6ae76d7eff5c2
8176 F20110318_AABPGG zmach_c_Page_120thm.jpg
5a3274beaf0d266ca0d55e8e6d8ffbd2
56e688077d8e3aed66d97450ab63b3c2a8915a3f
91232 F20110318_AABPFR zmach_c_Page_059.jp2
1f4fe10906efd552cc3dac64b641a153
5b14c7d1d01403d8dc87b3e853919d64f301ddf0
6169 F20110318_AABQIU zmach_c_Page_122.jp2
170aaa3c323b7dc2db558106357b7095
c46d1ddda482f1cfc5e01ed7282eb00fd547100e
6236 F20110318_AABPFS zmach_c_Page_002.jp2
04fe08c47047498c29d2c9b7f15aba94
9babca9585a77ecccd4bee482ac2bba183239c55
1051979 F20110318_AABQJJ zmach_c_Page_147.jp2
6b64fff2bf064f79be1ad1bd09f6b21c
8633f226c082774f517bdb382fa59b7f6260452d
813942 F20110318_AABQIV zmach_c_Page_123.jp2
13d9d1c05faa1b8f2bfda47b9db6321b
e2250ee10a358451cc22dc8ffad695ef04f5f6ce
F20110318_AABPGH zmach_c_Page_140.tif
bd225edc05bef2fc0c10c19c68aa50f3
f34fd0f5f0eb126be92a8d3e93dc27d98afce020
21323 F20110318_AABPFT zmach_c_Page_091.QC.jpg
dac14ee2c289141b7d9276c578914ecb
95d082a27181ae3f13ce36be79096c651ecdb784
85062 F20110318_AABQJK zmach_c_Page_148.jp2
66d555933699de636f80e5b7f3b5c95e
f7af2d348950c0a905b4b220c9d844b7aa1c64b8
72957 F20110318_AABQIW zmach_c_Page_124.jp2
718499e6446fce49dd2ea7f7c5d0c931
2520a0f2337a2664ee52a68871cbce48b9d4cc71
50671 F20110318_AABPGI zmach_c_Page_120.pro
ecee966993d14ef744bb723f994077c7
221055313318a0c1e37e288153da4c9469477cb7
52618 F20110318_AABPFU zmach_c_Page_035.pro
31ed2986e067a1bf8eb09b4e6ee8cfdb
1f0699c4919c456ddc3e57071c9c7eb7577a16e5
18625 F20110318_AABQKA zmach_c_Page_005.QC.jpg
8ddcad9ed4861c450589289422a897b4
889d14c8b2c91e7964a7665551766186f8518332
8307 F20110318_AABQJL zmach_c_Page_056thm.jpg
34ceb1651d19712e01dab2f5e70db302
8ed4d50c1ebb9093d2f0821d4948260de6bb40ac
89663 F20110318_AABQIX zmach_c_Page_125.jp2
48e82a0459dc91b8db2822869cab5e60
01cf39661f79eb193155600f66f236e7b1e34549
1746 F20110318_AABPGJ zmach_c_Page_094.txt
455711c06a0fe5255df69ed86efb64be
d28370101b08bf2c9297cb77e0dae7096850aa7f
56477 F20110318_AABPFV zmach_c_Page_008.pro
e099e4a4e1ed9eb0ab3b32cbeb6d9f20
5547b2a0c5f608b45e31e2d58e86b5b2b7321353
19632 F20110318_AABQKB zmach_c_Page_007.QC.jpg
5a711351eaa5962485c1ea702e6e611c
e5956526aeedaa4547c4a6f1c88beb6550e28d33
8237 F20110318_AABQJM zmach_c_Page_023thm.jpg
f597f013eda693832b7c6451fb456a36
d39e416a612fab4c5487bed855dcec1083b9e52f
81081 F20110318_AABQIY zmach_c_Page_126.jp2
e3ff911f247691fb0c37b64a8ea81188
c4d623cd93a39eae5ef147ebf70af7a922b4bff9
29507 F20110318_AABPGK zmach_c_Page_077.pro
a562bb4455aa4e2f0079bc5a0b28ae9e
17f9ded67976e87cfe5bb856caeb110efda9dca6
15549 F20110318_AABPFW zmach_c_Page_006.QC.jpg
2c3f509502a57f17faffdbc0a4829fac
971f3c28e675ff1c8c7f2ebb243032563d66af97
21899 F20110318_AABQKC zmach_c_Page_011.QC.jpg
a79aeb137d27472c9ebb454e166ea4d1
4238c77f6484243d1c556d63ec5624c308f3d5ea
8346 F20110318_AABQJN zmach_c_Page_034thm.jpg
9b2342c8e8123d831ce6cc3b54faca80
eac49ff93a6c47a7882bdfa152a49657b96b5f6b
80277 F20110318_AABQIZ zmach_c_Page_127.jp2
c2c9bf918186b6ade7297cb389ae9302
a004ee784f4d1b167f19f13566864afde6554a5a
104755 F20110318_AABPGL zmach_c_Page_031.jp2
7be1c5e7658a78f19be29418a706b844
7262a2f49fd2d24d01b8e3cb66c5124c539dcf9d
34434 F20110318_AABPFX zmach_c_Page_035.QC.jpg
8a3edd62ae6228ec22939ab2a0c67b6a
6cc6f94727534c80d7880fd907bfd432e0ecff76
103753 F20110318_AABPHA zmach_c_Page_049.jpg
9180f239eba61d92ec26300750063b64
e4fba9fdecc5746c0f7cf2e6f85ddc4a9c68076a
33046 F20110318_AABQKD zmach_c_Page_013.QC.jpg
aef37bc70bce573b445cf55e8051ced6
6bddbede05ddd6f481d43ec8d37ec13b1921fd4e
31484 F20110318_AABQJO zmach_c_Page_051.QC.jpg
867c492593fac7b0569e52fb401e68d9
43d66cc906119b4cd7360bb4227f0f4805130d63
103661 F20110318_AABPGM zmach_c_Page_112.jpg
866dd4fcbeb7e2f5bdeae27f82699898
5a78f04d79a9fd3ff700a93d981e0dce5b325385
1789 F20110318_AABPFY zmach_c_Page_091.txt
7524583a0d0d136b41cf7205d709f5d9
0b76a58293577f1ec3f06c560ba95ca20f7f5f62
1437 F20110318_AABPHB zmach_c_Page_128.txt
64844fc6251e95151031710af1d66236
0d78959dd8e31fb20b14324a6ddd09f77706a003
33896 F20110318_AABQKE zmach_c_Page_014.QC.jpg
54dad8fbf7589a023dd37db8a3452290
70326ba470df0db27d7cd1e3bcdb70e68c34d4fb
7490 F20110318_AABQJP zmach_c_Page_131thm.jpg
5ac195ecadc046c8280ba6378b8ea5d8
604934a112add7995a8c5cd2020658f598d0bbab
75684 F20110318_AABPGN zmach_c_Page_128.jp2
0efbd1fb66829f01f23702b799b2e2d6
3a7bc387327134bf7758143658ac69144bdb478c
6250 F20110318_AABPFZ zmach_c_Page_135.jp2
c9a4b77b40466b033e6a5165114aedcf
0a9fb7a3cb54c931c9ad2dac5bce7351aeb45db0
99709 F20110318_AABPHC zmach_c_Page_114.jpg
b66178cb292c90674448e5ec1831cd57
677fa5a49d296e0ceafba2bea662ccbc9ade43eb
29732 F20110318_AABQKF zmach_c_Page_015.QC.jpg
d8dde4368bd7859fb0e52235016de7f2
113edd0f8249561d23ab1fed2803ff55cd4df530
8061 F20110318_AABQJQ zmach_c_Page_083thm.jpg
4eb9b65d9dcc6f8a7c86feebf8de2c3d
29073751c7fa6049be015c8bff25cca191584735
86263 F20110318_AABPGO zmach_c_Page_090.jpg
18d10fba2f82eb92c1f190a1c266e167
72c7fc04791cc61594d04b51301d15b88953407b
8287 F20110318_AABPHD zmach_c_Page_104thm.jpg
4e76bc60536b0d00f6219dba8c2c0ede
a906a6bb1b86ccda8728633e523c5f1883454bd2
27187 F20110318_AABQKG zmach_c_Page_016.QC.jpg
a12c6bed6ea26f96fddc7a4f5ccb2d15
5c68908b9b395d3b6cdddb245e39edde1fe96770
8193 F20110318_AABQJR zmach_c_Page_013thm.jpg
4e99dbaa43bd416cb89c2a76d0859f65
45bdf35a59489949a1275cf89334cffa42b1354d
12045 F20110318_AABPGP zmach_c_Page_058.QC.jpg
08c33f148f71456beaa184c57f193192
a5fd7a064c0d69d451d4e9451db435b0829cf849
1799 F20110318_AABPHE zmach_c_Page_059.txt
c5f1c0deb37437e810a24d7c2d78d76c
e14821a95a310f18d0410524b123a44392b265d6
28760 F20110318_AABQKH zmach_c_Page_017.QC.jpg
1bfbe84b1f5e0eee75ac1809f9416904
40f390b420cb90e8928e141f6636c4672f420455
1066 F20110318_AABQJS zmach_c_Page_132thm.jpg
7f1111a15ef5c8e5d390a1ddba8622ea
11c597d9ecefc51d05d49d7a8c11ad3104dd4056
6863 F20110318_AABPGQ zmach_c_Page_016thm.jpg
d85415415426bf5c8ed597cc5ca19e3f
f683bbf980959aee6b04c223aa3f35cb7a5416ef
48986 F20110318_AABPHF zmach_c_Page_023.pro
b3c49ddaec2645c45b2e9c6e05d9ccc0
9313bc95cf45345962c5a3495772ad913f0eabe4
33195 F20110318_AABQKI zmach_c_Page_018.QC.jpg
14d1ed05eb8f5361268dcd668ba6054e
b4f9930e4e7cdd5fc3f75a4d28682506977fffea
22827 F20110318_AABQJT zmach_c_Page_089.QC.jpg
9b71037e2a231bf2c8f397f974c39d29
3d2f7562bde736c253ef6aeeb2187e12e1190649
6310 F20110318_AABPGR zmach_c_Page_077thm.jpg
3d96a634c12486cba018d3bd858dda3e
82be02bd1e0c9cb7cc3e6de7f9fb78cef5f2ad01
97723 F20110318_AABPHG zmach_c_Page_071.jpg
6abfcec6e0c0c8af23e0ad9ebf184b1a
69bac0540061ef64c9617f64120acffa4d3cbebf
33152 F20110318_AABQKJ zmach_c_Page_019.QC.jpg
1b5f0df1531315a42b2ecac6f06cdc74
c6878b7099a92ee47a7c5170759099e4a375b2db
8137 F20110318_AABQJU zmach_c_Page_036thm.jpg
e9ea3748f4b5c2346f47e0e290ec70fd
383de1d8ce7fd634426ffaf9e59ddbc39c4896ee
652 F20110318_AABPGS zmach_c_Page_002thm.jpg
59c228604d1205725fd085536ee2b010
2300fa15cf612a10f9a9d1781076d577b623dc2c
27179 F20110318_AABPHH zmach_c_Page_088.QC.jpg
13ca675567eccced9957f498489176ca
5b6a1b5ed04ba11ff118bb42534f22a159a8bb1e
2933 F20110318_AABQJV zmach_c_Page_058thm.jpg
02d3b80cbe8327d3094239bf3c1c1b51
f6b217f1257c3223a130bd36eab439d1fc61e34b
34758 F20110318_AABPGT zmach_c_Page_113.QC.jpg
8c0327e563c97f06dcaf4fb7a9b5cf03
e5c217e2434f89dd85c7cf2ec1bad8efd5fe61e9
31388 F20110318_AABQKK zmach_c_Page_020.QC.jpg
8f6dda6143caf5da64b80560511ec49e
2a94d9fdff42cd119427c2a54ba248173aabf18d
237928 F20110318_AABQJW UFE0012802_00001.xml FULL
0ff61a58f82f0d649ae2d76def73fb37
56e17b14e57b65762c833317ba2e94e46a3767ea
BROKEN_LINK
zmach_c_Page_009.tif
5777 F20110318_AABPGU zmach_c_Page_084thm.jpg
d3fd75f8f5b716fa0798514e1c6ba342
89e50ef53f5e9387feccfc6968d174ac7fd4137b
20967 F20110318_AABPHI zmach_c_Page_086.jp2
acfed0f174868109143078b4e1011471
0b4445b41b8418debdc32aa03fcfda73e9a11fe2
33912 F20110318_AABQLA zmach_c_Page_042.QC.jpg
9691bce482127f6e5ebddbbdf790673a
fc6de4313f02dc9d8725699c27a35169644a4422
33808 F20110318_AABQKL zmach_c_Page_021.QC.jpg
85cb3dbda6f6dcb8c1bce93689ec5039
3ad8c586a383478e7b13b4abdc53b4a0cc61e874
9694 F20110318_AABQJX zmach_c_Page_001.QC.jpg
559dc8aa0ed4e340fbf92e4b706aa676
f77caca13ad312e6d84ae60311df912103186d78
100178 F20110318_AABPHJ zmach_c_Page_036.jpg
482ed47116aa4916dcfa964a2ae1056c
9ff845730aef6a6469017ee65f4523f87f5765d3
34814 F20110318_AABPGV zmach_c_Page_048.QC.jpg
3120f90bd49760f386aac8214511f262
9c8768a77afd84bba7a0b5dd0bdf98c351904a4d
31058 F20110318_AABQLB zmach_c_Page_044.QC.jpg
0561e964dbca551ecd48e0fe07e251db
ef33c89d6171f610a3c74b01312f8fdefee2993a
33547 F20110318_AABQKM zmach_c_Page_022.QC.jpg
bb883cf053b277a41d679737d78a8b4c
04d2f5c4bfb3b13aa29ee4c9eb261c182faf5e26
1823 F20110318_AABQJY zmach_c_Page_002.QC.jpg
4118e891e4c1819a0c15ab3b2615be4a
25a7fd46d1be41aada22e6649cde6885c44b9770
6295 F20110318_AABPHK zmach_c_Page_107thm.jpg
c2b2fc3fe454c7a2c5f4f6fee4607b70
830653e532bb7a95244e2b39a3537bec5bb89518
F20110318_AABPGW zmach_c_Page_132.tif
64afc9bf26dc0b9fb8e16e36b2cffdb5
f7e241d1269101588b67651ebb7bcd784cfbae87
31870 F20110318_AABQLC zmach_c_Page_046.QC.jpg
5e11afbe871994d11f30e874e20971d6
db8dfb5bab6ff1dc1beb5a5b38671d4e02dcd0aa
32666 F20110318_AABQKN zmach_c_Page_023.QC.jpg
9a889bb87676f28e59e59051c8709ecb
3537d820efb5acac91c1d55bffcc0074b940ebd6
30134 F20110318_AABQJZ zmach_c_Page_003.QC.jpg
59b5f3d3981c4c2af9a2c90524038fa5
8ceb382019fbe10289c070e41e0dd1930e883829
7933 F20110318_AABPIA zmach_c_Page_070thm.jpg
12a32d9b17f72e5bce2ba3cf5f34427e
02034cbdfd8879ad62b24925108f0b8f1a315baa
29827 F20110318_AABPHL zmach_c_Page_043.QC.jpg
f83990dae97fe68b5031f3d894b35571
85352fb0f870e8fc1047d24f59c4ddffd4c533da
F20110318_AABPGX zmach_c_Page_041thm.jpg
551801506f0ebf282e9ec0cd97d0dfa1
7bd1953a63d4e33dd9f8f05b2d0f6a6068e71686
32897 F20110318_AABQLD zmach_c_Page_049.QC.jpg
2af0b066db186baa7aa7c0af24871b6f
13d9440469f08102f7b48d49f486980346cefa6b
29498 F20110318_AABQKO zmach_c_Page_024.QC.jpg
765ce0a0196b07652cb90f2982879a21
8fde0a8c22bbd47f6be1e27780639543fa56d7ed
8138 F20110318_AABPIB zmach_c_Page_086.pro
ea334f321f72e25f581b891f7aeb8e01
7ddc2c6a819c9a8a4b746cace94bf67fbd59eeee
1868 F20110318_AABPHM zmach_c_Page_038.txt
068810ec9af9bde5ceb88644593fb6ae
7e31f0481c249564f9875659710072c70c1fec30
48049 F20110318_AABPGY zmach_c_Page_036.pro
22244a8993a24a0c0eab1d0cccda9463
fb5af3d550a1514ee69af2d830ad88b89ffe2556
33956 F20110318_AABQLE zmach_c_Page_050.QC.jpg
03e1809cdaf0a49dc93ab4cdd069f3f9
72182b2ad38492dc61c6f66958bd7ef58e41cd37
31419 F20110318_AABQKP zmach_c_Page_025.QC.jpg
dd30edcab4e0631e738ca67ea930df87
6240b9b7f602034ed5035719c8043314c2a7d69b
8279 F20110318_AABPIC zmach_c_Page_022thm.jpg
d5c1df584d657e4b35faefc27d13644d
744872a92c8b330a2e505c37f485e9ef00dad2eb
10593 F20110318_AABPHN zmach_c_Page_009.QC.jpg
d5ab2ba379d3fdb7f21d28eaf121021c
83c9392b745bab00c95774441b51ea6276ce49b0
58090 F20110318_AABPGZ zmach_c_Page_121.jpg
f9339efa3310bd13d3e4ab0f74aaa38b
095e1991ccb3fb69569067be3a0728bd1bb1c139
30590 F20110318_AABQLF zmach_c_Page_052.QC.jpg
1588f44551f88c6899319512a450d1dc
8459c0f7298b79707ea2ad6b211d92347b7919cd
31662 F20110318_AABQKQ zmach_c_Page_026.QC.jpg
0aca371bfda98aebeb448a21510be1e1
39241be72ff9a1b486da3a415bc379e26bc99964
107656 F20110318_AABPID zmach_c_Page_013.jp2
15d8f900f87bf3b901bb67ba34168876
618930d4346326eca7a12220e8e555b00598f8f3
1917 F20110318_AABPHO zmach_c_Page_126.txt
0fedbf2ad55e65d778ca83e18a50b606
e83d0d1c900f3c0c424475706b6441b33ba993f6
32506 F20110318_AABQLG zmach_c_Page_053.QC.jpg
8a08a067904c23b79232275bda8ac63b
8279d52cef0fd5fca5efd6c121a5193bb74575ce
33456 F20110318_AABQKR zmach_c_Page_027.QC.jpg
26ee414a3baffe4b413d18b61a057137
4d1ab7b7fb39fcf2d5c6465915756d5fae203c3e
2044 F20110318_AABPIE zmach_c_Page_136.txt
b9f92006c57453e6b9dc20b983650a8b
37afb0c0be5cec3554c5a9760ba8635d62808eb7
F20110318_AABPHP zmach_c_Page_093.tif
6db776069f0599993f269e4ffac0b5f0
974e03923723871e5781bbf555602e4611a36370
33296 F20110318_AABQLH zmach_c_Page_054.QC.jpg
c6cb80929127d64eebbf0aa6cccfa37f
352670a147a91390add93d9099498383bcf3c39e
24999 F20110318_AABQKS zmach_c_Page_029.QC.jpg
b7944ca5ab171bf5927664ed845ca250
b73364ae6582c9a24e6053763b54a00fb2dc5b67
63576 F20110318_AABPIF zmach_c_Page_134.jp2
d7fdc9c99eeed8443f3178dab45d89e9
2a310bd4bbe4b07b0bb507fa00d8195f083372ee
116824 F20110318_AABPHQ zmach_c_Page_137.jpg
2f00774cfe908b8f457c5556784d81e7
4441032ebea95b1fc7ceec224a890b096c206d8f
33742 F20110318_AABQLI zmach_c_Page_055.QC.jpg
74a59bbbafd71d368c5f7fa8266188e6
5db57c5486e17e1961ceed013482a1b13b475e72
34435 F20110318_AABQKT zmach_c_Page_030.QC.jpg
b7f0079bb300ff0b1e3a63684b44a55c
30ff99d2f99aa2d0f77efcb467d67b928826b9dd
114587 F20110318_AABPIG zmach_c_Page_113.jp2
a1ee3b794a1a19c1d9038e4e005292d3
af2cb3ba8c47903b8c0a6cc68d840ff952ec8d11
533 F20110318_AABPHR zmach_c_Page_001.txt
47221b204ee5a7ba356e4ce7ffbf8d84
5742d20cf7b48c95ad947020d3da51637c8c0b60
35545 F20110318_AABQLJ zmach_c_Page_056.QC.jpg
c8e67fc140eaa9690a4fcab1c7060332
35f9950ba2f97a4ba04db413a37befcd477a0634
31385 F20110318_AABQKU zmach_c_Page_031.QC.jpg
7f44cbc76eea3ea6d3dd73b35badc0c6
7cf55903cf5d7b5c2c6ec1c9b9eeae0bd24f3aa5
1596 F20110318_AABPIH zmach_c_Page_098.txt
42f80cc620f0203a0fc93c76a25c794c
9ecb1a2cdf6a3f55f8895640e51885393f1f5925
31741 F20110318_AABPHS zmach_c_Page_124.pro
da364484e8cf106192511d9d2c059bd9
4931ab7b1cd6578019f03223618ff4a00a321af6
27651 F20110318_AABQLK zmach_c_Page_059.QC.jpg
857fbb74aa8341ebb61e3b4b39c8fdd7
02ccb64b32922f21152cbd5874eeebb78e6c1475
33088 F20110318_AABQKV zmach_c_Page_032.QC.jpg
9ee52eb74d1829ed59c11812a6b5f49d
a75f4bcfd6acd9db86c23e3b3570e0ecd8b06e78
8488 F20110318_AABPII zmach_c_Page_145thm.jpg
b39396732ceea4ce6bf01fe56e13b2fb
055c322fe7a7b4a44c468d58c2844a63441371bb
1869 F20110318_AABPHT zmach_c_Page_017.txt
c87f1464c9685bd6260dd0cd930f3744
802bcd47a8b307fe1131b46df91eaca37bb00e4a
32218 F20110318_AABQKW zmach_c_Page_033.QC.jpg
dc6f68d16229308c6dac327916b776ea
01cab18c42b1a467d341b480066385cd4583943c
34375 F20110318_AABPHU zmach_c_Page_099.pro
39a6f4411e3e4ccdf4c5cf2e5ed4fd0c
1a1277ee4f452736f2ea6102c3ab0afdb8e4afcd
32424 F20110318_AABQMA zmach_c_Page_079.QC.jpg
c921d11edc3bfef312942274002ba62b
4fa3bcee83fe5ae5787bcda17e414b90fe686b86
30644 F20110318_AABQLL zmach_c_Page_061.QC.jpg
97b025c574dce7b88de40cedfeb73111
4eb5e7f999bbc594503f487644b76c2cd3deb31c
32280 F20110318_AABQKX zmach_c_Page_037.QC.jpg
4e0016570c4e801bf8acd0cc765c49be
ad42562542a5775a71ea6875098a9b71c1fe68cc
74097 F20110318_AABPIJ zmach_c_Page_005.pro
adf95fbf3483530d4ff30f30a35e9f48
496efa49650e75bb71af16892865dcf25c01df68
F20110318_AABPHV zmach_c_Page_091.tif
af443b00eaf0efa23101c8a8277a6484
04ad2c6df8a228e3d7c1f55f07b716500d4241e3
32903 F20110318_AABQMB zmach_c_Page_080.QC.jpg
6dc80e893d05b90f374c13e35755b558
d39e71d24f4f8248a857760377b3f2da1ed9eedb
29324 F20110318_AABQLM zmach_c_Page_062.QC.jpg
7283c83d7ea69a6ab653ed1c4cb98997
73bfb560be7b6e8a59e8b8c181103126da475d11
33424 F20110318_AABQKY zmach_c_Page_039.QC.jpg
229415fb23141df999019f45b6e66f31
bd75c3d624e1595c9527a2891b6dc87c2a35f599
110851 F20110318_AABPIK zmach_c_Page_075.jp2
7c15f624740cbd3bb50d4bcdd9acf045
b52e84c80d20e84d98d30c88a0a15247ee4a9d4e
103183 F20110318_AABPHW zmach_c_Page_041.jpg
173b2d0486e798b197fd93b3e5b2a273
1ca0ca5e1da27e8298c0659fe4701a3afcb6ec85
12780 F20110318_AABQMC zmach_c_Page_081.QC.jpg
d69c695279f99c58d615c1b1a1b1c5b5
e6ea0e478346c342ec15f01653690b82cc86eeb0
27390 F20110318_AABQLN zmach_c_Page_063.QC.jpg
087219110bc19ecc702d07350cdbf0f5
8a0b69b90c5cec46966c4c6158ba0d827522faa7
33214 F20110318_AABQKZ zmach_c_Page_041.QC.jpg
259b1f701470e89bbe6f94eeaddfb8bb
127d9bd139d3e1954f38a2d960f1d6fa24335b8a
81159 F20110318_AABPIL zmach_c_Page_087.jp2
bb643a0a0935eb43158f61a017027270
3548c1ec3aac5a3c7f6ce8cb5f8ec717beda4a45
60604 F20110318_AABPHX zmach_c_Page_136.jpg
706c9c0a044790f6675b3d766257af14
0c3c6eb81a036c9b47d4763d4eb57fd42d877b85
33689 F20110318_AABPJA zmach_c_Page_075.QC.jpg
fe84b9f337ebe1f2345e46ea237c2176
68f2fa8dc27525f203d0aaa9c5d5716cefdd373e
25897 F20110318_AABQMD zmach_c_Page_082.QC.jpg
c3a9d4ee0716e8b7118850762aa4b94e
99e7224993be859bb162af643c235d73f0e3e9dd
31572 F20110318_AABQLO zmach_c_Page_064.QC.jpg
e55d02919aa6d05dde6a832fef2e848c
d52c707e3b46e3bb4caea6f8d5a18367963e698c
31124 F20110318_AABPIM zmach_c_Page_011.pro
f2a3bcac477d53d879f93e7b140b6c77
b72b1a143816abe174bb5b7ea6b4b7629f96f10b
103014 F20110318_AABPHY zmach_c_Page_102.jpg
7d08cb29068f0940f737058930a30cda
ccc6a7cf6c6dc9728bb508718c30fbb7ca2e76e9
1243 F20110318_AABPJB zmach_c_Page_011.txt
921f1256cc0b03923c9a6dc0a4671d8d
40ed686685895eb52793a352c68f475f1d066329
31761 F20110318_AABQME zmach_c_Page_083.QC.jpg
3e1514e3c631de5174fd8c6e7e87553b
588cda55f21ff8f296ded651fa971f7e1339df2c
26464 F20110318_AABQLP zmach_c_Page_065.QC.jpg
ad1ae99769c645e928ab2a5169730444
74816c3833136a5dfd3b424f996dcfd8d1114c96
85932 F20110318_AABPIN zmach_c_Page_126.jpg
ea53354749f6ebc7201038892e2bce37
8fb48f0c3e48a79990819d4c9f7e36c770e8c61f
30741 F20110318_AABPHZ zmach_c_Page_110.QC.jpg
08c28b90699e069dc81646829ccbbdea
a62b6e1a9689084abc774be2f9064eac7462eabd
F20110318_AABPJC zmach_c_Page_025.tif
45722e1c93cc3d67340511739cfbdba6
fda17b875542eaa78f4983dbcc08f23e24797c8d
21944 F20110318_AABQMF zmach_c_Page_084.QC.jpg
96d6014689a93abe48b81ab83c4449f4
f2714dbf07c4248088688730cef96811c0c0679a
32966 F20110318_AABQLQ zmach_c_Page_066.QC.jpg
b79e39b8ef0253b31a2b29487cfdb8be
e3bc8f4786f012bb9c0fc6420dee88cba148059d
F20110318_AABPIO zmach_c_Page_088.tif
59cdecae19f674473a564e34a48dd3b0
c41de70fead3dee2ca4f980df75e821d4c87e6fb
F20110318_AABPJD zmach_c_Page_061.tif
f2d3fd795222a4aefdff509aa56ad426
252e1d0f545df4df2ffbd0db49b835618b15528d
28845 F20110318_AABQMG zmach_c_Page_092.QC.jpg
8850a3822a61d04124bea83e50de42bc
43e641772f63782fe647cf49573f1b93541e6215
32942 F20110318_AABQLR zmach_c_Page_067.QC.jpg
eb3f765ce0fc2413799f95d64e2b66ee
2d521a3d2f1886ae7b9d19f2b0f00d920c45493e
5611 F20110318_AABPIP zmach_c_Page_103thm.jpg
3264421a10689486c616b663bc39b7f6
12bc8c49280cf156245fbba7917c6bed57f58901
F20110318_AABPJE zmach_c_Page_144.tif
ccc398f33de4d67518b8acd3705e5671
bed367022f4029c3a0dfc72afd1caa28fa77eb3d
29614 F20110318_AABQMH zmach_c_Page_093.QC.jpg
f4413caa44caa2b30574fcc6898ba60f
b6516fc90aaf9800046466ce921ef0ec570e5bef
34243 F20110318_AABQLS zmach_c_Page_069.QC.jpg
bae5fc706f03bf378519e84281a9fae0
fda18eac8fba5d15cfba4de752064b8f55393923
8843 F20110318_AABPIQ zmach_c_Page_139thm.jpg
f642aa880ed228a7cbdd530eb5f4eb9b
694dba42f77a7196e874d9aef5192a96c7923559
102591 F20110318_AABPJF zmach_c_Page_096.jpg
caea4ef5abc2dca0b5b34cde06037508
efbc008c7aeb78162b246b1c1c5d67ac94e04c40
30642 F20110318_AABQMI zmach_c_Page_094.QC.jpg
cbe884b69c4840a36d3843231d0834b8
01545ad4204fcd2038ed57b8c1194f9845f412aa
31813 F20110318_AABQLT zmach_c_Page_070.QC.jpg
8e6af09495bbf51754137380f107d2f1
3d1835a6aa322d1a04eda2f9c702d5193bb17bcc
2015 F20110318_AABPIR zmach_c_Page_030.txt
3394b51372887b192d782eed768438de
1dbd9c2f3a2eb8d1137784de610435e47f47a2aa
1176 F20110318_AABPJG zmach_c_Page_134.txt
f1cb98e90847aa693192ea57088bf0c8
ad0e35fc819719010356aa361b060984969016dd
29074 F20110318_AABQMJ zmach_c_Page_095.QC.jpg
ad102cb014c2b23a0afadb2e22856b8b
a6d178300bc9e8e0ea46583dfdec4543cdffc9d3
31578 F20110318_AABQLU zmach_c_Page_071.QC.jpg
c82ac9ff255ebb87def3494cd7adc7ee
9b922d56b642f1290263e6ae8b0a9c224368e420
98000 F20110318_AABPIS zmach_c_Page_105.jpg
08b47c6ce5cdc2a5858c8838e1248525
8dc15c1d7ac62481a642601c10ef9b8a2dd11ef5
34451 F20110318_AABPJH zmach_c_Page_045.QC.jpg
8993823c276811214427d7be068ce153
6e2b286a2e90aa5f484d3d83490385f7760c46d8
18771 F20110318_AABQMK zmach_c_Page_097.QC.jpg
b06f1a01cca202e7024b7f3d08fec9dd
a5b7183572b31e46ff65fde0e2c91e07cd2cd6ad
23871 F20110318_AABQLV zmach_c_Page_073.QC.jpg
be9b8d14bb825a134f3fde6372a46f0d
718c9ae9477e64fe8c94a69d610300627b63c65a
1524 F20110318_AABPIT zmach_c_Page_065.txt
4f64a028d2a66a793cb0d1991433fb11
5ac5fbf5a779025326330a3d39263b1e059251f5
15775 F20110318_AABPJI zmach_c_Page_058.pro
73baaafb9b5e103d1b4eb9b2bedaab2f
02751e0595ac9206a4da50a09194266a2f25d865
24668 F20110318_AABQML zmach_c_Page_099.QC.jpg
59973735021b06dd2b0b4477e0a6cffe
2847c2299bc25cb803b174546934eada6e1d789e
29275 F20110318_AABQLW zmach_c_Page_074.QC.jpg
20bc8f51f254e2e31f97a53aa1b36ca4
a75e67513e8114a0dedc4f67c61f568e6bf4b744
48879 F20110318_AABPIU zmach_c_Page_013.pro
cebee21186f522b557e917ab7b5eee1e
7981599a2f64055778db6658ee82fab40981cbb0
105396 F20110318_AABPJJ zmach_c_Page_026.jp2
1770f2d1a5c6714e01fcf05ffe03f1fe
7b81633c0c042114a8f8e295deef1d2db8d8e9f8
33274 F20110318_AABQNA zmach_c_Page_120.QC.jpg
ce40b4d9b718ec49ea56b1d41e305bdb
b0eaeccd0ce4b791ecb858d2b3d4e33275adffce
22744 F20110318_AABQLX zmach_c_Page_076.QC.jpg
074866f142c656d4c4fd77c122a729cf
e4a2dc553668c91a6240371e86a1ed6aa197b7c9
48562 F20110318_AABPIV zmach_c_Page_105.pro
90d6d38d580147b75de5b36de497dcc0
2b12f8a24434c8685abc1162083da3cf3ed3d2bf
18157 F20110318_AABQNB zmach_c_Page_121.QC.jpg
4e4e454efb4b4a86eae65043e0f63a45
d753c96da471fad969d4c1022e1a7c823d5ab808
36054 F20110318_AABQMM zmach_c_Page_100.QC.jpg
d209e8537650020080e9a955ba5f3e35
bdd52e1f6bdfb73eddfe6c50a1d7b9059ee8c1f0
24252 F20110318_AABQLY zmach_c_Page_077.QC.jpg
e70579a25d62a593c6091c1e59636afb
a411540e81783307103e35b37d6776409e306b6a
F20110318_AABPIW zmach_c_Page_005.tif
e92d877a5f941c12b7546f15f7c376d5
c769ab7b830fe94d743831b30163eedbcc0c9db0
F20110318_AABPJK zmach_c_Page_094.tif
9a2d2adfe860d62f719ae5747d0464e0
5c218a7a65ec0f35a7a9e2fc7cbee1a021f5ca51
1805 F20110318_AABQNC zmach_c_Page_122.QC.jpg
525e1d13ea4584f4b6d2e444a937c192
c3bc8ab8d6ee284e388ef978d60e1665d4c50418
34316 F20110318_AABQMN zmach_c_Page_102.QC.jpg
2a4ca570794f225f075218299aa4ff09
47f8e5805060abc98b8c9ac2e587f46149439d93
24996 F20110318_AABQLZ zmach_c_Page_078.QC.jpg
a5cd761cb5648ba49430027b2a6c608c
e1ce1cee2dceb00b6d2ae3587674a78768c165e1
8505 F20110318_AABPIX zmach_c_Page_116thm.jpg
928d991fd2ce00b78b0148929a84edca
94bc12e933f9e8f717d21d18f1b5dbce6a4bffb9
F20110318_AABPKA zmach_c_Page_092.tif
5576d2c476d87a5ef9cbd5b6b56c1af1
b2306f58b2299a500d9c8b491296972f4b2d2d02
F20110318_AABPJL zmach_c_Page_105.tif
618f61142eea8702c83956b1ef8002e2
8dd2e4d63b5f18955773aec2f257d4b7c7dc1740
24432 F20110318_AABQND zmach_c_Page_124.QC.jpg
ff24103a3a357b1bad7992fbd2a40be5
c3a31eef42e41d69ea86c8b456730d018f712c31
19656 F20110318_AABQMO zmach_c_Page_103.QC.jpg
0de8468a8a333eef32ebbfa50f6f8aba
71f69d735fb54300e06a52eb7964f01b33aefa23
17738 F20110318_AABPIY zmach_c_Page_101.QC.jpg
ecb0cbd5e242bce9b8cfc475f4ae2873
0c6ab605f14fe6ffe9712a543e141a8325c93b48
106268 F20110318_AABPKB zmach_c_Page_014.jpg
70add823523cbc4257e857006cbe97b8
145ee15610b5d2358652fad53aa5ed389fb8626e
F20110318_AABPJM zmach_c_Page_099.tif
81bc75f0502975a29a3316e175d9460c
7a4dfdca47d31aef799aa0c279c8ad44c7dcda21
28325 F20110318_AABQNE zmach_c_Page_125.QC.jpg
746e440093ec9b571612cab607bfd71b
a7c67add57eeb2822aad6b7456048dff8b5574fa
33881 F20110318_AABQMP zmach_c_Page_104.QC.jpg
a81558016081656f634a0d79a25cfce6
902d4a0a1cccdf4b91f00610b8f42fdfa8b7f067
8570 F20110318_AABPIZ zmach_c_Page_019thm.jpg
58c2a1a6b001000bd840b46b59ac1d6e
76acf8d18529bf07e8a26d00abd146f7b93549fd
F20110318_AABPKC zmach_c_Page_024.tif
0157d94de2f358350c665ddf5544f885
238815ca79d4bc4192e84dab4ee24cb43887f3a9
97565 F20110318_AABPJN zmach_c_Page_094.jp2
884d6d38cbbfdae7698fa5d2d7063074
7cdd9b7ed572b75df76f214384eaebe7db142cc8
25499 F20110318_AABQNF zmach_c_Page_126.QC.jpg
3ba45dc1de0e633ba9847c05cf11acb1
0eb0eebcaf5bacd70f9569f7f72e84b16011c3c6
31606 F20110318_AABQMQ zmach_c_Page_105.QC.jpg
f20e1016eb833376a721b792f0d248f5
80a59957115db90648b71c0dd78b594e1ecc4fe7
61409 F20110318_AABPKD zmach_c_Page_134.jpg
476d25c73e61f673ad0a395671855e5c
669f43ca5dfd91997719b1060b7ed554dd664b54
6676 F20110318_AABPJO zmach_c_Page_008thm.jpg
934f9ba9063034d19ca32e79883e458a
ee8284a8eb38c08b1d4719c976793bafd22a75ff
23477 F20110318_AABQNG zmach_c_Page_128.QC.jpg
38847366a080d5a6bd752b5d8399892a
47e11fa541ebbc1fd5b1a0d6f9dc40bd7ef99371
19107 F20110318_AABQMR zmach_c_Page_106.QC.jpg
3c95f7499f37236efb13f7a57adbff2c
72e637ab477ffec06bd1c09a48faee185bed43f9
1756 F20110318_AABPKE zmach_c_Page_074.txt
28908395f343d3af8f8e934453c522a4
2d0869a3eaa4c5811db91700e5e5fb2c45d601e3
24487 F20110318_AABPJP zmach_c_Page_068.QC.jpg
a8fa7f523d3c05a927d9f0a5e2650217
8f29846d0f81bd18b56aa84b9dca49687fc7a4aa
30075 F20110318_AABQNH zmach_c_Page_131.QC.jpg
71f39a2f83011a810268d901258e0f80
faa54ecb26a567651f0c5c074a0a0660ad463d03
23552 F20110318_AABQMS zmach_c_Page_107.QC.jpg
7d8bb9c0935ac55f470f1915419ad3c3
b3028fb3bdfed38eace1a99834fdacdb62eedcb3
98028 F20110318_AABPKF zmach_c_Page_051.jpg
d5630a037005f96445207bcaa3d0db72
3bfc64f7b4efdd6cc39fd1fceffb301d0e0613c4
24884 F20110318_AABPJQ zmach_c_Page_127.QC.jpg
b5ae0dc8c7b64e608dde8140df96fbc9
e051c26feb8967c8d694da8c339d7d8a24350a7b
4040 F20110318_AABQNI zmach_c_Page_132.QC.jpg
a6cc12711f37408a406dd3cae579d704
fb28cb0bce74a8fb83adfc4e0f948241dcabbe7b
17895 F20110318_AABQMT zmach_c_Page_109.QC.jpg
93c925455eef57cde24622713cd27d9a
2b8b5b7a762e160df96ba8bd35c3867d3e3f66a7
F20110318_AABPKG zmach_c_Page_109.tif
471a942bdafe9feb7b4d0e16ecc8ef5b
f17541cd5a47c2e69c2e30b68117c244c83be702
49930 F20110318_AABPJR zmach_c_Page_111.pro
190a35f7b97262f3a0c75257789cbba1
38c608e9b0e153c994e87d66b3059afcbf400f45
17066 F20110318_AABQNJ zmach_c_Page_133.QC.jpg
96992da7a5075e49d3ab91b71016f9d3
d7c92e1cb754f17fe663972859ab91c471e3d997
32556 F20110318_AABQMU zmach_c_Page_111.QC.jpg
fb3ba90883c57cd3913b4f6f178d6206
abaa397733a057025700da785ed1e8a2085d0c26
2859 F20110318_AABPKH zmach_c_Page_009thm.jpg
86b5a79e86809cc98122dd2a8a9b5d73
9b6bcde8d4b56cdd1529080e9ffc3e47134a3fc0
40418 F20110318_AABPJS zmach_c_Page_125.pro
16dee573b3e4b0632f49989524ef4f92
cbb910f8893dabb33d7804e8ea32b04ac0856e23
19161 F20110318_AABQNK zmach_c_Page_136.QC.jpg
26948e8cc308ac0bdf5ca3a908813994
e12968a041c3d305a1deab18e473f8eef681d194
33467 F20110318_AABQMV zmach_c_Page_112.QC.jpg
0dc66b129610a37bb39cd9c4ddd51f21
dbeb6042b37d487a976e8df0bb62d2b955e51fed
1866 F20110318_AABPKI zmach_c_Page_020.txt
01d2015db3990c9afd6ea14ffb13f46b
7eca7112c3d77768baf08931b3d500be83886dcf
37018 F20110318_AABPJT zmach_c_Page_068.pro
122b9124fd3c460e34000b1d3d01d220
a49b730e37e486aeb4eaa18cec7d06a37d3385db
32262 F20110318_AABQNL zmach_c_Page_137.QC.jpg
e4c4404d1db0b0203cdea26ece27403e
f7da32b80cf01a24dc6b50ee243a8bae02ece40f
34170 F20110318_AABQMW zmach_c_Page_115.QC.jpg
e0cba2d70a2a405d650edf90d2373718
89e3371d5f6cd157ff300b10511aaab7116592c3
23274 F20110318_AABPKJ zmach_c_Page_123.QC.jpg
8dd34bd5810302c44d8b20f4bb134b1f
1a9359a387cb2b6cb1181ed19161b8559dd987e4
50509 F20110318_AABPJU zmach_c_Page_018.pro
368f03257ee18c7b074a678b3e090e76
cc9c320157fd3e586626e082764ce57c6d2dc057
7067 F20110318_AABQOA zmach_c_Page_012thm.jpg
ed65970758f9a3c683b95f66a2979b74
33ed76c13067fc6be048e77368c281a9e0cc4a8e
34327 F20110318_AABQNM zmach_c_Page_138.QC.jpg
c5424154492a733792509d9eca73a650
ff3f7a296d2130de155a69feb4e5d5ec84d7abc3
33264 F20110318_AABQMX zmach_c_Page_117.QC.jpg
bbc1f2648354da641012769871735263
9160b1fa6c2722be6c0c7ca9133926ca495ccf5c
27217 F20110318_AABPKK zmach_c_Page_008.QC.jpg
821e32a75bd4204cc9aec4f9150aa33f
371b273566dd91a0bddec187dfaf43e47e8dfe30
F20110318_AABPJV zmach_c_Page_051.tif
1b00f008160dedb41a51bff3c0f58427
e75017bc74a80cd523d3a772a54bee2858918cff
7145 F20110318_AABQOB zmach_c_Page_015thm.jpg
c0eb38c9568e0e41c1979e9fb185db22
25022e52a85b8d3fa17f20e61cb83a32dea6ebff
31223 F20110318_AABQMY zmach_c_Page_118.QC.jpg
8f817550cbe7d49fe7ecd71805d67e77
a33e38a9de7f0dc38407b577c7d7d7fb59617442
F20110318_AABPJW zmach_c_Page_131.tif
d9d8ce43ae6fcbfde5525794d76ba7d1
0f800d9382beea32f2b3fe652783038632c5fc4b
7229 F20110318_AABQOC zmach_c_Page_017thm.jpg
762abe06675c0c695433647ba2d3251f
b467bdb2d9fea3630822535cab1f21c836434b3c
35907 F20110318_AABQNN zmach_c_Page_139.QC.jpg
df42c04a97a389f83e22c523f1202d7b
9d698710c1f0bc45ff1da3d2b8a10aec41844c0c
30319 F20110318_AABQMZ zmach_c_Page_119.QC.jpg
f5a568eadf2aa9b9a4f013b881eb4841
b65636090ed5794282403347641a975c4216e6c1
24717 F20110318_AABPLA zmach_c_Page_130.pro
ad34e40f1d7732e9e1e97fd474820ee0
c69c0ba7ececa59a442b2aba1f4348c507d98d65
101413 F20110318_AABPKL zmach_c_Page_066.jpg
6c021c1b38feb7d1ed6e8f5b979e0e34
b7dd69408828fd9ce096888db28d99f79987054a
F20110318_AABPJX zmach_c_Page_137.tif
0fc852af847a9949cdd4909611c90c42
856e4a7f1ba0344cb0d11e08ae3d87f249a27b6f
8023 F20110318_AABQOD zmach_c_Page_018thm.jpg
0d7054a3d8f61cbff2532757a032cff3
117de34739933f4fd6813c6516337623ee1414ed
34454 F20110318_AABQNO zmach_c_Page_141.QC.jpg
57953e837f9b02e551ca7dedce12a055
f18bbb8876f74536a9faad298407acf1bfaf2691
93406 F20110318_AABPLB zmach_c_Page_088.jp2
1505a70e827e8a3e6b0b924defe3d44c
46970f8963611174d6f26130db7377a87fcca019
56824 F20110318_AABPKM zmach_c_Page_045.pro
28cd898ab6415da163f71abbb016c356
3a8cc3d0ca97091bc71a28407c095e706d7f579c
57002 F20110318_AABPJY zmach_c_Page_147.pro
f9539baf1159c9aa2902d3d35130f3fa
1fd70bb3d430296048e169f019381ac25772d518
7937 F20110318_AABQOE zmach_c_Page_020thm.jpg
f004d4ab0a50192f43405e1273ecdf07
6bd151ce5c38cd7655a3ff96f2b903614c42b180
33907 F20110318_AABQNP zmach_c_Page_142.QC.jpg
8ecf3eb9ac4d8e7e4c2156dba333cb91
f5b68f04a56b3a19f300e0c2f0cdca7457c116a6
44746 F20110318_AABPLC zmach_c_Page_017.pro
85b8fc646a1189c183067a1f113cf943
3ca188c1df1d6603a28b32f8a5881c4c0b0a5fc3
2007 F20110318_AABPKN zmach_c_Page_096.txt
547b5ff4748d1ba63ac188e6f630c9a1
3d03d962503f383b9d2ee01ff88c29cd879729e4
6271 F20110318_AABPJZ zmach_c_Page_072thm.jpg
854751c296cfcb78ad470193f9ebb58c
927eb2a967b03bfe3892db9490e120451cc973bc
8339 F20110318_AABQOF zmach_c_Page_021thm.jpg
be98737e4dee7a7455ffca596a73d0fe
d7efa75e027af8ff2448dc72c71ff6d394a8a3d2
31988 F20110318_AABQNQ zmach_c_Page_143.QC.jpg
697e0e291830cfc6c0fa8f85800421c0
635076ec0936c3afcd5b5a27fb9c20db8b3b4ae0
1491 F20110318_AABPLD zmach_c_Page_073.txt
7e31261c485521fb59720a677b9a20ab
c3e1ef26ce10b26f116ef8de03548a9b136a97fd
F20110318_AABPKO zmach_c_Page_040.tif
a414d12b25649504be46d911191a7853
c18525fce16602d1dad6791718a36fc8027c651b
7810 F20110318_AABQOG zmach_c_Page_026thm.jpg
78e4785a6c4d386fb8439ee75fd7e989
4de600b562919c08ed697519f688d1b39a4d9e8c
33224 F20110318_AABQNR zmach_c_Page_145.QC.jpg
c0a2fa7b35be87c20bc8eb5e86031b65
318f4fb23fb4b340741c31a340c43d739de44958
2194 F20110318_AABPLE zmach_c_Page_014.txt
589b7333620e6dd1c2391d47d0937004
0d07b64fd1e24306cbe113d7e853d935bbaeb958
35034 F20110318_AABPKP zmach_c_Page_127.pro
6e37b947cf36599018eb14d6c4c47af4
0e9d8c64aba4dbc694cd5ccbc110007eb7bb7408
8234 F20110318_AABQOH zmach_c_Page_027thm.jpg
a0c19b62801c83217af1e329895453f7
ec10fbb9ddd0168bef9a8d1bafe4d4eb820701c1
35151 F20110318_AABQNS zmach_c_Page_146.QC.jpg
a135095bb00d6ce95626c2be5c83a5ca
4fbdf4ddf3325638d57203aae890180e2d341007
41180 F20110318_AABPLF zmach_c_Page_016.pro
0233fd496278059f44bc280d9bf1127f
8fe5abf57615a24ddb319e35de27421964446b11
1788 F20110318_AABPKQ zmach_c_Page_086thm.jpg
731ff14662cc0bbf4ccf6ec32e08f495
26a79e9636208ee1cea0dd3af5c63a150cb0b102
8487 F20110318_AABQOI zmach_c_Page_028thm.jpg
bd336794232be6a771291589aef22418
74a02a3f19bc4fcd7adc156d552fefcff8c50547
33612 F20110318_AABQNT zmach_c_Page_147.QC.jpg
6b5ccf527586c8506993b772f7d6d228
9d681e4cf2013e605a3c12d319c19a387cb79f3a
1994 F20110318_AABPLG zmach_c_Page_102.txt
a1f84cc6da5d6d6e8a1136604e60c273
fb514cb663d322f4221c7539aa41c96ea8c6d881
6378 F20110318_AABPKR zmach_c_Page_124thm.jpg
3cae6d678023fdba7680784fd9313641
b9a85a76c6d1becadbc5c82de1382b6421fa635e
8303 F20110318_AABQOJ zmach_c_Page_030thm.jpg
834e322989fb28741543929da1d41e89
3178cccb776ada870e75f33c3f579a3ab03d288e
7613 F20110318_AABQNU zmach_c_Page_003thm.jpg
e4cb27234a07fadf171deeb7a884a5fc
5f21c405436155709b98281051b4364645073d08
91197 F20110318_AABPLH zmach_c_Page_052.jpg
affad00f172d74ea398905a881a157f2
0d83b5c150e70c0ac2bc9ee91ef94c441adb2f4f
81347 F20110318_AABPKS zmach_c_Page_065.jpg
39ffc986967c82d4fcf7c664c66a38c5
1183fcbc07e83e493054658d309ad0f7f2429c94
8048 F20110318_AABQOK zmach_c_Page_031thm.jpg
ff5c9d8b7b4970dd3e67faa3d2d00148
5aa625e73f4064f9192b37737140d35f04f84855
1859 F20110318_AABQNV zmach_c_Page_004thm.jpg
d0df39c36b742df0773e998b35cf0c09
09845c1d7b9833bfe3adac80fa9c9128b21ddbce
2042 F20110318_AABPLI zmach_c_Page_049.txt
11c3f9a3a29488e941c16d0bdba861f5
9225e5b4d8d396e75d2329f8b9767f7ddf41352b
103074 F20110318_AABPKT zmach_c_Page_022.jpg
ab1556b3444a26f42c70493eef59aaff
70eb9c9be4c4b423a351b9d568911c43b04c7f4c
8215 F20110318_AABQOL zmach_c_Page_032thm.jpg
c6f2d6ccf7a4f318f4d0a324093ad1f2
b80f80a8ed2c6371ed3a250f37511036ee4f8c1f
4567 F20110318_AABQNW zmach_c_Page_005thm.jpg
4ac67d36630481bb2ee5640cd7cc6f4e
c3622cdb7c24396530efc3224b34bf6bfd3d0ea3
2391 F20110318_AABPLJ zmach_c_Page_147.txt
2264f3bad2b57b983942d69ffbec149c
406d96c6e293eb99edc7bb13f056a8190f3604e2
7993 F20110318_AABPKU zmach_c_Page_064thm.jpg
5310946b65da74b565b6aa735cc45e65
5767329a931cc884d7718ccea11fb41b5b30cad1
7526 F20110318_AABQPA zmach_c_Page_052thm.jpg
ab823468c6e7de47f456777bcf59560c
ff3f724b046053e99cf61921fe0eb0cfbb5f4b94
8291 F20110318_AABQOM zmach_c_Page_033thm.jpg
33c63be23969103f0f07a2552ba37b9b
b705edee64db42759e1bcbe1ae85157ce999b94e
3913 F20110318_AABQNX zmach_c_Page_006thm.jpg
59ee55aa24346779ed42282a528fbba6
45aad06bcf6cecc66505311fb0c116362f11d7e1
99540 F20110318_AABPLK zmach_c_Page_064.jpg
80655b6812c16133b46ff982e6aae90b
d4984201d66f6d13d32ebaf655bc09a37d409e66
51654 F20110318_AABPKV zmach_c_Page_071.pro
972ce12e33c9dfba813fb30928452b5b
89762d5e44844d2a0b98bf5108ebb11866c9ab12
8242 F20110318_AABQPB zmach_c_Page_055thm.jpg
33c5a96bb90f3fe7fe6c767f15ce0278
5daa11965828aaf2f2785479852a0663c99ac240
8516 F20110318_AABQON zmach_c_Page_035thm.jpg
39dc97ac7b4b393d0d382bcada258f95
60b3d38ba3e3a42e8a009ca4e39777b52062d513
4853 F20110318_AABQNY zmach_c_Page_007thm.jpg
d85aa07b783c12c57d107ac20130b389
e9fe489b0c335b4f079425d7fae360967435e1e1
64795 F20110318_AABPLL zmach_c_Page_136.jp2
2e97b66df09e164aa406af862323f068
ae19f54d316de8b909c15bdb474bee11cc5f3550
100260 F20110318_AABPKW zmach_c_Page_046.jpg
6cfa666115bedeff10b653fa6256728b
fd40fe1458907d5dafebb388d18abd33f353ae50
6814 F20110318_AABQPC zmach_c_Page_059thm.jpg
d00895877de75005553b3988f3628041
e1606b6332cec9f95374d9872b2a4de35c5ab819
5326 F20110318_AABQNZ zmach_c_Page_011thm.jpg
4bf43348811d8ea465cd128e2473aea4
5b783870ad053a7c30919b69fb813bc61cbd9d9f
7819 F20110318_AABPKX zmach_c_Page_093thm.jpg
fb091eb164c6d21fe8e153e589f94ce5
003ea52f8b4debf050b53f70216ec10b2396237a
F20110318_AABPMA zmach_c_Page_139.jp2
5f132c12f5137307b2530212b080d3e5
00f450a33c23d7c9f8d12240e53097eb874f87c4
6834 F20110318_AABQPD zmach_c_Page_060thm.jpg
97dca93001d081bc72830b3dd1ec79fc
8349d9f8e5f19b2a9e92ad619e077ff3cb28ca14
8205 F20110318_AABQOO zmach_c_Page_037thm.jpg
92e207249f75679188e306655944a228
05734b21c737cd3551d8075b2740d42ed3d7f845
31621 F20110318_AABPLM zmach_c_Page_040.QC.jpg
a7e7336a1f0fbc05a7ab809f0864c65b
59e3ff6b7cd770bbed42c7cbfa030a78910011e7
2026 F20110318_AABPKY zmach_c_Page_028.txt
7d9371aaf0354a998bb841ef65469721
fe312264d1df47da8b916efe92153b0bdc6b6e04
106702 F20110318_AABPMB zmach_c_Page_046.jp2
b57efb1705acc0d2037826b007da0ab5
cde5a26d284e4ed461faf63f015a888cee4613d1
7331 F20110318_AABQPE zmach_c_Page_061thm.jpg
f585222855dfca7585b97cb34064e657
76b26dac35f300541b5440930f73392bc5f555d0
7872 F20110318_AABQOP zmach_c_Page_038thm.jpg
531606d20bf9ba53a4a8b02a659eae37
6d25077779208e3a7dd4d563f868e52ca22b3294
F20110318_AABPLN zmach_c_Page_019.tif
38040f3131996a613907baeebb2cadec
b811e6227051328db964b846084d32b0855a4089
76055 F20110318_AABPKZ zmach_c_Page_128.jpg
1f01e86463b898c5761c304ba071b2fc
8cd2092aaea441dc3143907969843b25b12a86fc
F20110318_AABPMC zmach_c_Page_135.tif
8e976ed5c417c1352b48b8088737911b
8f1f38ed0d189044066e38a8fade15c5c6f12fec
7429 F20110318_AABQPF zmach_c_Page_062thm.jpg
493cf4527ba8ae6dece9ae14c91fe556
7a93211ad6cc166657d90d1a6bed791a6b8b4ead
8343 F20110318_AABQOQ zmach_c_Page_039thm.jpg
11c4d00dd96503d4f3e421e5c3a1e091
fa08e7547ee56cae1a1fbecd9098ce0d2225c045
6533 F20110318_AABPLO zmach_c_Page_004.QC.jpg
9ee0dd5e43db5e70edd8f6404d54cbe0
2b6b6a8043516dfbfc40b25729fbffa62391c17b
52458 F20110318_AABPMD zmach_c_Page_101.jpg
9ebaaef52fd7bb5cb0fae73d8b48f90b
275a5609d306247368992f272594583357b76f13
7091 F20110318_AABQPG zmach_c_Page_063thm.jpg
b8c42fd9c35ddf1b32e4e1e016828764
d7015b746d9b8a0aeac74a1ce71dd4b7f3373d76
8196 F20110318_AABQOR zmach_c_Page_040thm.jpg
b7705901a132d3b1661c97f35b326400
47845f210d31d8326b5158712f0ecf2a4690fe5f
27842 F20110318_AABPLP zmach_c_Page_103.pro
76336d518ea78cc02cdde12961dcc45b
fd1f18bff409c52cd66f7d045b21890f9ddc7083
107148 F20110318_AABPME zmach_c_Page_067.jp2
74f991d092e81953de0551b70f0d5856
0c3fd4b7de33e0ff6e2588a64b0a3b5779f144b2
7005 F20110318_AABQPH zmach_c_Page_065thm.jpg
333bbe9d6438446476fd7a52763ba63f
f8273277723f6760f91970d591b9f82784d17fd8
8060 F20110318_AABQOS zmach_c_Page_042thm.jpg
1f25e2be4f877fb8232070ea410566d6
ef1801ba0f2869b27813e0a8962a8604f1e60b03
1550 F20110318_AABPLQ zmach_c_Page_148.txt
9f6a421b29a0da8b4a0912880e99f902
5759e2b7c1c3116cf84a2436c5ff8e0197667634
24327 F20110318_AABPMF zmach_c_Page_098.QC.jpg
2ef3d4b0c7f768e11f55b7ea24eed03a
0e81b3584ad1abef5a2c78eb881e094f2fee68ae
8149 F20110318_AABQPI zmach_c_Page_066thm.jpg
f65ede55eed0ade5ec273540fb1e7b05
7a00f89423f2a87546825f9db53d0a39860ee781
7574 F20110318_AABQOT zmach_c_Page_043thm.jpg
2e60dcee524ee7f776ac20540f05c092
3ed70d6a9ed4885449e615c6848620de830a9173
8209 F20110318_AABPLR zmach_c_Page_057thm.jpg
aa76c917ab371c1e27fe183ee829711d
288ad55b88d9ff221607d7f73df175d03cfef12c
35288 F20110318_AABPMG zmach_c_Page_009.jpg
ac22b8c476847e0d24f01b2c657f2e1f
8e3746cc5520f44a5b962a42206784d1b40ca577
8426 F20110318_AABQPJ zmach_c_Page_067thm.jpg
8c0cf17175aa1f571d1dab82470504a7
7036937dd3066097869bfc62511e893358472486
7713 F20110318_AABQOU zmach_c_Page_044thm.jpg
18c70ed396ce0f61ec23bf5531d2e144
38902936cbcf8ebe3b546d0e390c3aa35ea4ef7d
2001 F20110318_AABPMH zmach_c_Page_066.txt
f864592ed0941338022af90939c1c4d6
cc8fc485c8d043dbac2cb274a2d1db0b134423e2
F20110318_AABPLS zmach_c_Page_047.tif
4878d07bcefa00a5353ec3a570a5865c
a7fbd5a1716eb49417fd643c84fa47871edaa01d
6251 F20110318_AABQPK zmach_c_Page_068thm.jpg
a0b54878263a46dd70c12a39c4a20439
3ef83f849850bbec926088f98fa21109aee0a168
8453 F20110318_AABQOV zmach_c_Page_045thm.jpg
6f616f4fee9eeebe045d1421db6c91a6
2dff6a219d949056dc8eecc1b1d1210c0f700e24
F20110318_AABPMI zmach_c_Page_102.tif
f66ce6425244946a941be6874502915e
5942848c40986746cb77fc39f0866ddc3285d1d1
F20110318_AABPLT zmach_c_Page_090.txt
b192d22aed95064dde9c322c1a2ba598
c22bcc02ef5528cdc73c3ea0017d280b4b34f3df
8454 F20110318_AABQPL zmach_c_Page_069thm.jpg
7f43581235865e712116dc64b0d80e6a
1517aa981d222c518935ccc5eabdd5c689150523
8030 F20110318_AABQOW zmach_c_Page_046thm.jpg
ac2ddf3913dc410b744118c998687549
db405084e0c07ede8ffbdd0923a98277ade62002
5168 F20110318_AABPMJ zmach_c_Page_097thm.jpg
f28a7204e8adacb9157ff3238df8ac4f
82844b517837a709407429047705c3a11d7699ea
95138 F20110318_AABPLU zmach_c_Page_043.jpg
6141f1cab4e93a2c48bde967d83afc41
1168168ff6b937d826765efd53c99d11d1e9b560
8452 F20110318_AABQQA zmach_c_Page_096thm.jpg
474a16314ea7c4002365ff3345f3fc60
ac123919c768428ac1fa83445fc3c66c57689d05
6835 F20110318_AABQPM zmach_c_Page_073thm.jpg
76e2f3ff1d3ee5c2cea7abcc13c4405d
327aacad8c6fc9a1dfeba2927c974474fac0767c
7784 F20110318_AABQOX zmach_c_Page_047thm.jpg
70f920708bb8b625a8c2890f1fb81ba2
f71bd98a2ebe4198a13e224de57240ee32e54e3a
33766 F20110318_AABPMK zmach_c_Page_057.QC.jpg
f93f151f21d5d393de0004836fa2e5a5
b0ccf4816d1e5503f0856d6897552c21edfdfb47
29488 F20110318_AABPLV zmach_c_Page_047.QC.jpg
de44776eeb9b9b4443d79e927f057ad1
e2d33c07cf9d38551961fd6016b9042c1493ce11
6350 F20110318_AABQQB zmach_c_Page_099thm.jpg
0d41572462bc3a020fce9a576f0524d6
6ac1a8b0e2f3f17938917312c00474486f95f025
7693 F20110318_AABQPN zmach_c_Page_074thm.jpg
432753903dd9a5fc6745ae1783a3c032
e38545be6e971142782ec41fc81bf8953a4fe575
F20110318_AABQOY zmach_c_Page_049thm.jpg
1693c58c84eb1981132e81f1baeb39a5
cfd001635f3f2fa81a3840665fab15a46ed44472
7316 F20110318_AABPML zmach_c_Page_090thm.jpg
56ffc0086f15c1e6676651412b19ba26
41fa7991ce73ae180cec710d5ece02bb6a90e0fc
98625 F20110318_AABPLW zmach_c_Page_033.jpg
83ce873687f548786a9355c43b3b64e5
858c7d64a55424f4dafbe4bcc83757cae22b0f5c
5182 F20110318_AABQQC zmach_c_Page_101thm.jpg
123710c95ff151c4b40ecc8de7855c99
881152071a1a61ec1dba0d5320b794fc9b504f57
6344 F20110318_AABQPO zmach_c_Page_076thm.jpg
0daffe3e22ee8ec97b29bca521c0adb8
3df8f600adf29b31f12be298ad769a4f15a48f64
8439 F20110318_AABQOZ zmach_c_Page_050thm.jpg
223e7b2203b95f8aca8b30ecf718c6b9
a2341a61c339bea2c375811936fa79a77cfb0af7
48999 F20110318_AABPNA zmach_c_Page_053.pro
ca5fe3fff5b15ec46639abeb2f593b80
35eb58fec35403810b363b90be457cf8008df4fd
24284 F20110318_AABPMM zmach_c_Page_129.QC.jpg
52f2b3aad31b52ae613e108ca2aefdae
3800596ada83119ee2d89f78093a1e42963c032c
51396 F20110318_AABPLX zmach_c_Page_116.pro
6b320fecf8c1034cad2a70303cd51ec3
a8a5fc00d8fddcbe89be91bdbb77f5454a6a1ec2
8517 F20110318_AABQQD zmach_c_Page_102thm.jpg
2016432dabae716613211e51f1e688e0
7b076298970124878e036df1dae5e4639319af4f
8318 F20110318_AABPNB zmach_c_Page_053thm.jpg
8393fdb8e48f483bd8c3f3ab849d4027
c7e866a59b8c41c5003f9e43ea80746ba4406f4e
F20110318_AABPLY zmach_c_Page_072.pro
a40d99fcf5d6d2cf470bbdbddc7f72e3
ba1f74688acc657b6251e4f2afecf46cf7c54257
7910 F20110318_AABQQE zmach_c_Page_105thm.jpg
cf0085da0c7e492431eda540fb259724
638746cd1cbf3a54d4c982a8860ba7d3d9ea42fb
6577 F20110318_AABQPP zmach_c_Page_078thm.jpg
bc9835943ad7a561fd5837515e0c89a2
3d393c5d1d7ee1271bdfd1dac9cd7f9bdd13475f
F20110318_AABPNC zmach_c_Page_129.txt
520496c5ddf2505b7d5f93716570b50a
d8614e920ae5f3e715a5e0691d1f1f66ba030d0c
54111 F20110318_AABPMN zmach_c_Page_100.pro
94af5552a70ad42e9d743afb39cc4234
802c9c0365219a7ed706621780dcc37599d54dbd
27483 F20110318_AABPLZ zmach_c_Page_090.QC.jpg
7f5b35f7fac9c9172502249028466a95
e7f6df586efe191744df58f64968aac06b872d3f
5609 F20110318_AABQQF zmach_c_Page_106thm.jpg
f326aa25f60e0c81326cb5a94da03b7a
04e19ef8fe40841910f5cf86b42e3d6cb8953942
8286 F20110318_AABQPQ zmach_c_Page_079thm.jpg
d250d4644f36814022e2daaf50ed8992
b8a0df64ec2a0fc7b13c298b64f8443f76ddd44e
1051978 F20110318_AABPND zmach_c_Page_140.jp2
3925b6dd4e797e5b669752e163a0d1de
aee685e28b95836eb62f2bcfd26787041da463f4
6619 F20110318_AABPMO zmach_c_Page_087thm.jpg
46ebccfcf51fc073ad9538fef4b0c2c7
035b6f1660ea8706b4edd581a55a23a1cf7366f5
8301 F20110318_AABQQG zmach_c_Page_111thm.jpg
25688ec5b0bf4148d6ce61eff7006401
16dfe9ef9f353061a0544d4732b6f84f83cd4f8d
8258 F20110318_AABQPR zmach_c_Page_080thm.jpg
fab3580f99836a56328463f251126ee4
0a43150fbc18e934454809905084f2ea6f0e9354
54254 F20110318_AABPNE zmach_c_Page_143.pro
55965049ae79dadfb255b69f22d2f28d
7a8d285b0b6c706888790eb8781a4dba2cab601f
F20110318_AABPMP zmach_c_Page_080.tif
437a1b64749510f63db35d0741bdb519
55ff8a552a4a19aa2994aaeff04c127ec825343b
8462 F20110318_AABQQH zmach_c_Page_112thm.jpg
bcbc978d6c144a45da61ec3f4030a585
f95cb728637959a10f8752d8945a68e6e61bf3e0
3941 F20110318_AABQPS zmach_c_Page_081thm.jpg
a8185646a0b998b2c4e8c5cc9c341555
2b4da6d20c7aeef86b3316f1188f1eea7f82b6e1
8541 F20110318_AABPNF zmach_c_Page_100thm.jpg
381b8d247307cb1028c57dc4e14f69d4
7fdc77428f2e4b50fd3c87a09165499b7adbbb3c
31678 F20110318_AABPMQ zmach_c_Page_114.QC.jpg
dbcd70bcd8cbf9ae0f3136de0da30681
66996a4d2657d1336d299d3e08f9621a1a1e03c7
8620 F20110318_AABQQI zmach_c_Page_113thm.jpg
d04b03862288e0b3557d5fe9fc707a8a
f14f8064af7daed1f33db6e5a28c58f65ec35530
7367 F20110318_AABQPT zmach_c_Page_082thm.jpg
77068b1f51f5fa57f1cf768661c0b049
8877056cffd2e25a1f2f93494f83fa4272f7a949
F20110318_AABPNG zmach_c_Page_020.tif
d3c236c05f62ebd2574638bff1da55b7
c4f771e0800bb478a59534bb2371c27d535abe57
1802 F20110318_AABPMR zmach_c_Page_052.txt
a366f6471e2d9b2a8e93ff92fc09b798
6553d1a2b6215a7a374cc2621b1608f8e2080a8e
7999 F20110318_AABQQJ zmach_c_Page_114thm.jpg
8d47e75a453faca9f0e46c96cb378292
fedd74d75d09871a214418b80b32ea6bff0477ef
6188 F20110318_AABQPU zmach_c_Page_085thm.jpg
9b2986cdad13cf660fb8a65e89f5a235
7a74498edc1f9acde0b570671ad8073d4c59b429
6547 F20110318_AABPNH zmach_c_Page_129thm.jpg
5e7961e609bc9d4dfdd3ef62f5bbb9bc
1e7ad6448927c967d3797c2bd668f0a8f98c22ea
7859 F20110318_AABPMS zmach_c_Page_025thm.jpg
3162aa3ff95752550196fda9f94b5e7a
a817b2bbfdcda7a3c256266cf2a5bcddd13e11ed
8212 F20110318_AABQQK zmach_c_Page_115thm.jpg
9e988904e0b0046df47e6a15d2242afd
4b6107066f9faa8739714788c3828d9d20fe79cc
7031 F20110318_AABQPV zmach_c_Page_088thm.jpg
c20f5e02dd6dc0dd7b0f9e7b88432c04
b8f6609835fb6e92a96aefc9eb1d824881a14172
1186 F20110318_AABPNI zmach_c_Page_121.txt
18867eba2814b6461c5f3d36c0195d2d
7e73967fc16b801b36281ee78398190b6e1aded5
107772 F20110318_AABPMT zmach_c_Page_042.jp2
42db227478d7ff03992fda0e913eed87
e5f1100bdc27f929f63c0b5bf7389ea5cbee6f2a
8368 F20110318_AABQQL zmach_c_Page_117thm.jpg
7fefee6748ab5f7d9e1a73360bfe15c3
cda5ad1c248ce6567d7f586ad67df8caa20c0228
5835 F20110318_AABQPW zmach_c_Page_089thm.jpg
67c9de80298364bde0feb895d149f773
0687b76148fbcdcaa816baea9e8ee05df4583dcb
F20110318_AABPNJ zmach_c_Page_073.tif
66a4ec85f69858b3cdf201517c046e47
2d3a950307e649ce79049c6e27585ccad1660608
2000 F20110318_AABPMU zmach_c_Page_135.QC.jpg
222a74014b344414cb42d1647c45b48f
a85ffa2fa63af8d4d5dfc77d3d15e11cd8a612fa
F20110318_AABQRA zmach_c_Page_138thm.jpg
ea8a1ef777f2b7bac880a386ba943079
e8c7f32c59b8729249d224f18b26525bd87de899
7824 F20110318_AABQQM zmach_c_Page_118thm.jpg
6a4052e4208fff0609507010c99227ba
440d0b488aa7a85c345a7fcafd2ffcd815115bff
6644 F20110318_AABQPX zmach_c_Page_091thm.jpg
084540c6fe517d348fdf6405caded98a
ad613bf66dd89a45ee79151ed200019bb693a3bc
7878 F20110318_AABPNK zmach_c_Page_071thm.jpg
cd031cefc4c3071d7ac57fa37ab5f1b3
4999e7936e3c87e11c47a75b88c5dcaa01bb58fe
1964 F20110318_AABPMV zmach_c_Page_110.txt
e4b7414ab1a6b66ab0500e9eff43dae1
559e2c1319a223713429feeafe96f265b1d563c3
8724 F20110318_AABQRB zmach_c_Page_140thm.jpg
d54651cc6ca33c4bbae40db76ec5e92d
8f83ec226e43c61e9074b3ce711a15a2509541f0
7327 F20110318_AABQQN zmach_c_Page_119thm.jpg
393ef59b121821ab8ae02164ae688e8a
26f76da185749be01bf33a1d8b35cf2a1791d311
7453 F20110318_AABQPY zmach_c_Page_094thm.jpg
2360bb4792f9b20948fcd6c5c23f3031
64ebce1c7d701619a69da7e1bb7053d44c716192
1924 F20110318_AABPNL zmach_c_Page_118.txt
102679254807ffa4bfa3beb645943707
43488b932a518e95c93d24891124a0f558d3a88f
5280 F20110318_AABPMW zmach_c_Page_122.jpg
000cd8869eafc8b268304976838e3633
a672bcc6e5c63ea715bea0fbdf60f0786584c038
8752 F20110318_AABQRC zmach_c_Page_141thm.jpg
634ef04b0f96cf44fda3615236f3ddc0
905ee0391b59e5d2bdf555dfb71a356e1c7b0d25
4429 F20110318_AABQQO zmach_c_Page_121thm.jpg
82c6b9f41b3c49921e4d0f21bb562f49
dcb4f6f042085889e5df2fab6161c491076e7a7a
7402 F20110318_AABQPZ zmach_c_Page_095thm.jpg
9d7ab93e3eaa69e860bec44e6818fa1c
ccd363deb6886d13a70e5afabf13db4e183f32b2
1926 F20110318_AABPNM zmach_c_Page_046.txt
aa80ac6da84fc8fac9d8865548ef9e96
6cd822655b02f2ebf116e96ec780278e5c19116b
32395 F20110318_AABPMX zmach_c_Page_107.pro
24b8ad09f367ff21cbfec38539182440
c3860729d3febfe498959e234df40e92e7622f7b
47919 F20110318_AABPOA zmach_c_Page_026.pro
787fc631744b2d6170ada68477ab81a6
dc95d5446f1d569a555d3b235a2a667ab9ef0581
8834 F20110318_AABQRD zmach_c_Page_142thm.jpg
69ab5b3fc1fda60e3ffc767a24bf26b0
b2820b03a8f518e54c8369e0c89bfa35e8d1a6ef
536 F20110318_AABQQP zmach_c_Page_122thm.jpg
7fc810fc147d9ecf53aad49bd0313e2c
8b1ccc96b4082d94142c63273ccf99a41a0850e6
25164 F20110318_AABPNN zmach_c_Page_087.QC.jpg
bb002122a4545182f1780acfed9a877e
2cb30a0e52bd715abc0dfdba927366bbe91d98bf
2028 F20110318_AABPMY zmach_c_Page_104.txt
cb351c1896175b8d95e27897ab8d8d57
d6a0769421da1ac6d4a0477fe5460fbeca6690c7
1314 F20110318_AABPOB zmach_c_Page_135.pro
fda30b28b39c9825ee73bc5820fd7989
00192649e9beb386f01d50e3956d3f0583190d55
8400 F20110318_AABQRE zmach_c_Page_143thm.jpg
c61753d4aacefdbea0fb5bf33a61a22f
726f39a5e0d9d8b52e08ecefd5ef623a9ca1c979
69161 F20110318_AABPMZ zmach_c_Page_076.jpg
0f8e2cf9046eec6b9995ea60bb40e165
1096fe78c08b74a29411a6a8c5a3b0fd5d7d1016
44928 F20110318_AABPOC zmach_c_Page_062.pro
6226d42019b800023f4a69af3517ec63
dd086bf2f2f6b98bbe2b52b5ad84fc54f1d32a2a
8626 F20110318_AABQRF zmach_c_Page_144thm.jpg
0c42f80676e03403c44267cc6079006f
174f14e0b30cf0b0370cbb5b5270d945a3ae7e08
6140 F20110318_AABQQQ zmach_c_Page_123thm.jpg
6e73020a315b57d9aa2ae880ab865a56
25af5b2beb62d52e2a0079ef9ec6973aafcd4aa1
105126 F20110318_AABPNO zmach_c_Page_034.jpg
500c6da99b2a44aad56ab213076d07d2
ab70fc17b27b49ec31481fb50ac595152efda4dc
119126 F20110318_AABPOD zmach_c_Page_145.jpg
dfd045ac81d6fed90edc9173b306f952
51dbf45c2c200e3580df8999d938073d9c496f47
8609 F20110318_AABQRG zmach_c_Page_146thm.jpg
a46f26605da65facf9215f60f9e5d740
e469e321d74e5a9dabe012957f626d1752691aa6
F20110318_AABQQR zmach_c_Page_125thm.jpg
09f14cf12ab1ae5fb8c86d1b6e0a9962
5568b890a9afc6f4581daf4c56ced8f88cb9cbfe
1921 F20110318_AABPNP zmach_c_Page_105.txt
96fb7551122497d9773e480a53666a29
c58b4d29902f01ade607b00c43e0063988b85de3
F20110318_AABPOE zmach_c_Page_133.tif
ff62ca43bf35616593028b875cc0e401
ef890ccb152525928ab4f48fbfcfbcde3fc0d72e
8202 F20110318_AABQRH zmach_c_Page_147thm.jpg
0cbea3d0bdd3f511d46bc578ffea7479
110bee199c2011a8b60777dad524835acfbeffb7
6736 F20110318_AABQQS zmach_c_Page_126thm.jpg
294b67d45f5c678279e9ed1c843ae5d4
f887e4b881d0116e57b9d85dd62f17db24b64e54
1969 F20110318_AABPNQ zmach_c_Page_109.txt
39726614310bfd441e172e7455a3b9d2
7a4ec285f167746cd96dbfdcad89eb02c5c0f0f8
98559 F20110318_AABPOF zmach_c_Page_026.jpg
356d943864b9c0de7b7c654f545030f6
196fbfb2e0d6bfc82f664f07a5da2246ba4201dc
F20110318_AABQRI zmach_c_Page_148thm.jpg
4d0df12052a9a57af6340bd6170b4123
f3de494a117692199dd2d0fb734476f6813ad0ff
6688 F20110318_AABQQT zmach_c_Page_127thm.jpg
7c7b46619b811afc0e9db3775335fe53
c1343d5751197e20201bf2cd321180b7af5667a5
106086 F20110318_AABPNR zmach_c_Page_036.jp2
ba37bcc4fadecee42d9f8e5bcd201310
4e77525d83d60662e041b29ac965806d797b096c
24454 F20110318_AABPOG zmach_c_Page_085.QC.jpg
1c321fb5088e55600ed3ae7a584514c4
482ec34c754c39372c2a5aea37169f871c6d18ab
1058567 F20110318_AABQRJ zmach_c.pdf
e1df642c27ea0e84e666577182a45736
5dc5ec8d68596e90a393bc80375e6d64be2112a4
6389 F20110318_AABQQU zmach_c_Page_128thm.jpg
105601e79769f05f2fa81f49385433be
b7ed0c4c5c4232bdadb49ab09309a4b889c2f48a
8220 F20110318_AABPNS zmach_c_Page_051thm.jpg
3034ca8355389942b292221771bed545
128d2868a58a2ccae5ed7929939f5ec3adf47379
106506 F20110318_AABPOH zmach_c_Page_080.jp2
bf6c9686ca0bb5bd5dc2130deae22861
43182269c023294583b9df76a8e606e590ae5798
5246 F20110318_AABQQV zmach_c_Page_130thm.jpg
ba50edb1243334fa0f85c932d159e9b4
a7926ee8a506c96dcc9430c2052fffe74c310c50
98778 F20110318_AABPNT zmach_c_Page_080.jpg
62f4d3f317de065cf8608c365812caac
b04505f313f48a384bb7da49ee42970d8d758163
36383 F20110318_AABPOI zmach_c_Page_087.pro
ac8240ac974711992699a291609f5721
a3e02188f2be790405c88c08c9dfad2765e85c1a
4862 F20110318_AABQQW zmach_c_Page_133thm.jpg
f44fcbe970c1cd4fef4052b12efbdc0a
c770f3532f71420bd4b99f399fa510392f13cb9d
24966 F20110318_AABPNU zmach_c_Page_010.QC.jpg
2fcc142df210bff03ea036f0a88adc2d
2cb2159c3380f3d9e09d30607b81fd75a7e7213c
60470 F20110318_AABPOJ zmach_c_Page_144.pro
2d3e06029765cafc47b03c4de8c2c9ab
d5d93c6154c99fc299df478457049eb069276a68
4321 F20110318_AABQQX zmach_c_Page_134thm.jpg
4819237d2bde6f5c6f462b99682329f0
053a93eca6ec25f64bac9c3698fd13579e3c265c
64706 F20110318_AABPNV zmach_c_Page_072.jp2
daef7c4c367fa7da13f88c8a6cf78e57
73783382803588ecc5c6e82f0327f5090eb7c4d6
51888 F20110318_AABPOK zmach_c_Page_115.pro
0fef1ac587feaf3e75c91d2cb1b820b9
06c2659ebb42fc421c3817ee055347487968fe55
565 F20110318_AABQQY zmach_c_Page_135thm.jpg
bde67ca08cda247a2d50a878eee73f06
e574591138c379806b5004b166ebea839496f72b
104593 F20110318_AABPNW zmach_c_Page_105.jp2
f393ebe18b7903c9dd1d58f001ebac86
1a1f3dbbf7497db2653bdda59975b183a9f7a8b9
8499 F20110318_AABPOL zmach_c_Page_048thm.jpg
402b8dcad2a612fb766a3d0e855729d6
8bc8d416434a3ef6d8e3512a5f7b42ad059be7b6
8100 F20110318_AABQQZ zmach_c_Page_137thm.jpg
41c8bd21c00bb69d742b0e73f95816b9
9b92abdfa9fbbd840c0dba11a217a3f40d322680
F20110318_AABPNX zmach_c_Page_127.tif
288c90d0845c1e0230f47bbec24deae7
24be6db43b5622f644ae654bb9b9075c63f9564b
F20110318_AABPPA zmach_c_Page_066.tif
52cbc1d03174e0b066c41260753b4c01
9d1fcec957b665f2611b62f874efcb4f3f47b104
44370 F20110318_AABPOM zmach_c_Page_047.pro
98877dc333868451574d4265a605d651
c277a2721f932f1f0a726b9df123f6dd65a32b8b
6367 F20110318_AABPNY zmach_c_Page_086.QC.jpg
209ef4f364ef67a9f30189bcd7ec233f
bc54b9ba1e43d3ba85799dc5a46227b69d54a3bf
46197 F20110318_AABPPB zmach_c_Page_015.pro
187f85c8a983520fe40df74e72b7bfd3
462d691c4c8274d87f08d092c8fd1afcb4bfd0e3
104531 F20110318_AABPON zmach_c_Page_025.jp2
9d171319b5d7e58578fea330c709a6b9
461df82a177f113bc024729a21ab1da2206e410b
8076 F20110318_AABPNZ zmach_c_Page_110thm.jpg
15118d4df20d8233bdae81a908e245d0
1cf493ac915afa1d333d3530a3429594bf3f5116
102604 F20110318_AABPPC zmach_c_Page_083.jp2
4133d4fb50474af34cd4eec211440876
d448cdd4feb5065d25235ccc12401b21c04e2a99
51567 F20110318_AABPOO zmach_c_Page_104.pro
58bf7f142400e800968928737801c96b
22cd0a5f497651bc7b59f693e478215142c25dcd
F20110318_AABPPD zmach_c_Page_069.tif
ec598a45c0cccfc21a7721b657a83628
5ad70527c930bf3fab23b9e94ae1924e5246cd16
34651 F20110318_AABPPE zmach_c_Page_034.QC.jpg
af7e254d8de4505739b7a1a24e1874ec
36f02dbbb46db5c029901a9340a3eecb70fdc4f2
2538 F20110318_AABPOP zmach_c_Page_001thm.jpg
ef69ce723de27862c6064df8a7783913
22d3d5555cb8a162db352842801cba41f7ddc161
7749 F20110318_AABPPF zmach_c_Page_024thm.jpg
a5aab5864f8552da69b19ca28a5e7de5
24955f20afb65c48f94ec0c80989965c454bb2cb
3002 F20110318_AABPOQ zmach_c_Page_005.txt
7dc5473db563efb31ac433c69d6195cf
0e3c78562dd0cce970135ce3ab2b531f0531f14d
59620 F20110318_AABPPG zmach_c_Page_103.jpg
ed4a1abc4d5d2f9d7ce188818e47080e
15c1432ec238258f2504d22f6d88c79f741855d3
8019 F20110318_AABPOR zmach_c_Page_014thm.jpg
24be8ae90276d61d1b30f06e4023c2f1
170757e6c1bab9b52c4b7394a3c8316593d2599d
707378 F20110318_AABPPH zmach_c_Page_073.jp2
c6cc20608b12d6341b1021c3200809f8
e82286c99bb1ccab0233997fc2a93db20f85f177
33947 F20110318_AABPOS zmach_c_Page_116.QC.jpg
23a3b7fa0a9215f56720e3b019c930c2
e9e2a68e976e3715f542da56a1c90d003582045d
49186 F20110318_AABPPI zmach_c_Page_080.pro
2c09707ff048bc8e83bf56bce5a8bec3
4561421ba55604b92694b671a1b9814e66a9bcf8
1051980 F20110318_AABPOT zmach_c_Page_005.jp2
82a07a2c0e53534c909dd4732a59ba72
ebcde7fef8a61f242ce86a5aaf9a477885cd1a36
5550 F20110318_AABPPJ zmach_c_Page_136thm.jpg
7e333f02c0c66c0907901452b64f76f7
ef896953a5aa8078cba29c891b2781dcd3198fdc
8049 F20110318_AABPOU zmach_c_Page_075thm.jpg
86105195e6ceffb1297fb30f49ade211
d960d675a83e0307f58b19603e510a5f38424508
2011 F20110318_AABPPK zmach_c_Page_082.txt
8a4f007f30456efef0aed2653d7e34f1
f41f61a117788e6f50a82c9615eaf0ced784a153
2481 F20110318_AABPOV zmach_c_Page_145.txt
32170f2edf7e16e94853f78519c741f2
99915c53c3c908399b274292c8a23c2f4f91fea5
104733 F20110318_AABPPL zmach_c_Page_033.jp2
71c27c08147422b12776187c6ca053ef
e1234f491d1fc18f649ddd9ac7949ded75d180e4
33613 F20110318_AABPOW zmach_c_Page_028.QC.jpg
19474432fc1e55d0af5d984bf78f89c3
a8ba3398f8743c6f80b525ed943e9ac1017994aa
82449 F20110318_AABPQA zmach_c_Page_005.jpg
58cb967dab9a65dc55c3af917acf9603
b4422968ea6e7d61f0b4f9dfd6d3ac644f0cc329
24595 F20110318_AABPPM zmach_c_Page_060.QC.jpg
f1764de78efb45e5d9ec6dab9c00ccc0
faeba29c6182351dc89811b288f18aa10702e212
106186 F20110318_AABPOX zmach_c_Page_108.jp2
0dd0bdd3be17cb770cde8ba4fa98bac0
b7481a41298d25ff85fc9d69b38bd926f3763798
1051969 F20110318_AABPQB zmach_c_Page_137.jp2
a1a45b11489f6beee5d8d3c08a6d37c4
1e7186a5326c1118ee743756561627db61d62aa0
F20110318_AABPPN zmach_c_Page_002.tif
e9dded0b03b1efff81df65a53195fd84
2e6cd2602cff743a255698358a9b9067433c666d
103785 F20110318_AABPOY zmach_c_Page_118.jp2
57794681d66f4b1d4ecb4a9ecd3a25f7
447d9cfa1ae1212f7c7f90f0046a2f5b783171e1
2503 F20110318_AABPQC zmach_c_Page_144.txt
37ce7da0d57332be4d4df7235055be90
f0c81e851a97c4aca4f3defbb5af19e0115669cd
51440 F20110318_AABPPO zmach_c_Page_050.pro
a903b893404e87ff8fc61d4566480e75
66ed95ff7d8ba2b923abcada057795cee2c431f6
31237 F20110318_AABPOZ zmach_c_Page_038.QC.jpg
91ac662bf134a966d59ac4f959c6abe4
367ae598ad5e338a647da455f6054b5535dcb456
F20110318_AABPQD zmach_c_Page_050.tif
88162d7e4eb69138935577f09a773f61
f2d028e016b6c3ae5c141524a2f6d8b9c9b125d8
F20110318_AABPPP zmach_c_Page_098.tif
f26804816ca160f69ecf69bd6fe20a7c
24a0bacfc3bc23615a4a578090c73d42f0f1d0ec
F20110318_AABPQE zmach_c_Page_104.tif
fb68cd5f70178f8159683d8ad2e01388
cffe48a086cd2b8b1b3d3d1a0f7b9c76ea04491a
63346 F20110318_AABPQF zmach_c_Page_072.jpg
e77fcfda9fd0fce6c3b890f6540f15a7
e0d67a3a37437e18a3486b4af4cd8805ef6922d1
2136 F20110318_AABPPQ zmach_c_Page_100.txt
34971b46755b104e75edd95c17ce438f
6797aa0e45dd90df8439e6f4b66c9eac269c5ce3
90689 F20110318_AABPQG zmach_c_Page_062.jpg
d3be9c2e5a43995abe6ad90f7cdc0136
dc2626132c217d13b98f5c411a7b03872f6ba46d
8253 F20110318_AABPPR zmach_c_Page_108thm.jpg
b05ca64bec4f324e0d1794b23a73a420
b530def1dbaacb63927ed5ebe96082ed9c00d418
25502 F20110318_AABPQH zmach_c_Page_148.QC.jpg
9942ffd307754c7978d3832f2fa1af43
65c6ebf84ccb7604544e65b94ac2d469ab3fdabb
1935 F20110318_AABPPS zmach_c_Page_040.txt
938953e6300bdedcad8fa1f437050099
4ec422e41d08e5504c022454bee0b88e26085b5a
50318 F20110318_AABPQI zmach_c_Page_102.pro
3f99b2f5573134f53bd3fc0eb869dd72
a49dda17e375cd2e648e3d26a2bfc7e26b96bbea
33672 F20110318_AABPPT zmach_c_Page_096.QC.jpg
28f439269a396e7001237b2eaa4ab27c
64ef44c75174dfe8bdb1c32eb46dbf36d4beb163
172962 F20110318_AABPQJ UFE0012802_00001.mets
aa94801a0a8296dd8e9dc651e8db715e
3f08e9464dd3432d3384feb66bfa70239c7bf31d
zmach_c_Page_009.tif
8423998 F20110318_AABPPU zmach_c_Page_076.tif
3defc1f79175006b5d97993dc49f067d
647d4f76fdb14d1c05e4640b4330821be67768c7
F20110318_AABPPV zmach_c_Page_004.tif
2d4fe682ddfaf2670d1d06ecc0da98bc
b6745690e13bb2d7c6a438d2377a4a8aaba317d2
88590 F20110318_AABPPW zmach_c_Page_074.jpg
f41c0ec01a0cb444d521a6e59811451f
e0600852e1858eb15a562c8f3b8e1bfb62934c56
F20110318_AABPQM zmach_c_Page_001.tif
89fdc9df706f5e4558cea21f964d8ad7
a4b1518e2e72ad806c940f0d1b62e38116cbb9f9
1930 F20110318_AABPPX zmach_c_Page_023.txt
ff7658523159335489da7aad1a1a017f
9fd4401d5d38a11f6a00b6c672317508c2216862
F20110318_AABPRA zmach_c_Page_026.tif
ff1fb6c9e2819c3eaedec1dfd0deba19
edc5f767f552d69c84df0d0e3b408cc19d29aabf
F20110318_AABPQN zmach_c_Page_003.tif
0edc05df9e8b3651b38e5719d5caa7f0
957a68ac72c1ce71ed6a870217c618fab031aa5c
F20110318_AABPPY zmach_c_Page_047.txt
f00697ae3f94baa4e1eecdf4b8327618
5c8a25c7fa9bac786a1bb3e83b88c74680fc6ae9
F20110318_AABPRB zmach_c_Page_027.tif
1721e891f7cb1bdbd264bf03c33463b1
c8b68284f12fc7a1e04c6284f0443f88f3a936d0
F20110318_AABPQO zmach_c_Page_006.tif
f66612320beb2c3f9016c7469fdd3109
03d8e62d3cc396b9a33d4331c6720f4a64ab63c6
7249 F20110318_AABPPZ zmach_c_Page_092thm.jpg
500efc51f3af9752170070746316519f
b29ecf7649965ab4aae48e8c3969dfa01306496b
F20110318_AABPRC zmach_c_Page_028.tif
6a3bb7b2022842e82f58dd20e9f29e3c
8c33561fa9677cc0cb0eba500bae63186a1e283f
F20110318_AABPQP zmach_c_Page_007.tif
9f14647d1bbb1a04f02ffb77e6fc1d2b
a46f18beac1f8287c90c4a487645af55db3492ff
F20110318_AABPRD zmach_c_Page_029.tif
5c0d72d9ba5b6d744dd62a4b39f9f43a
41244afb10dcc247c842dc262bf1c4e1617d8ed0
F20110318_AABPRE zmach_c_Page_030.tif
23defd35b93c881fc3831007f9ee361b
7d5029b1fc007fcde8e746c973f276ff70274757
F20110318_AABPQQ zmach_c_Page_008.tif
d6c3dc60f1ba3505896082c82a7ce85c
371540a9c3db1823c8ca4c5014c3e569ea983ca9
F20110318_AABPRF zmach_c_Page_031.tif
a997545a748a431eb739001d6a6c1098
95463547f720d352ad72fd732dc72273a3a3785e
F20110318_AABPRG zmach_c_Page_032.tif
e4af9f7bb0c5facdbff42d09097ffc80
d28a2b202ec4e13e3247ed8a1ee7507da692bbb1
F20110318_AABPQR zmach_c_Page_010.tif
fe6bf26def7752f29df141e845a0819f
fc22ada2e9cc8deb5ca836e927304b8b572927c4
F20110318_AABPRH zmach_c_Page_033.tif
219a3d148f798e5774280ed8939704a9
053cb40b321ab3fc3420c274783a0c30501ffbfd
F20110318_AABPQS zmach_c_Page_011.tif
515aad960492f491b585f92c7fe3d90d
2c23b41f128dffc117f8a03facbc309180d3d608
F20110318_AABPRI zmach_c_Page_034.tif
89fe3cf5f92acf9de7fde70e518acedd
3a9071e9c6985d9c5cfc533650de0966df858034
F20110318_AABPQT zmach_c_Page_012.tif
df76745587f68859c52e3d5d3678ea16
ac42d7fda42607ea400511d0179fff380f8061f5
F20110318_AABPRJ zmach_c_Page_035.tif
ed8654f2fa9504f171a675696e0c1a03
2b75830c1e33438a22e47ca8e9aa28c805116b46
F20110318_AABPQU zmach_c_Page_013.tif
2e174ad301930cdbe47826f2b6ae5063
94abbf64c338fc04c74af01224130e1fd27d4921
F20110318_AABPRK zmach_c_Page_036.tif
3a56a14847b86098555ef00adcf9ebb1
bb97b586bcb7a8b27a434182be3b0984751680b7
F20110318_AABPQV zmach_c_Page_014.tif
46151d1c2789a9ea3687e7ac8062e664
f04f66b6d292902bcd4dd6d2290c2867776180a8
F20110318_AABPRL zmach_c_Page_037.tif
367312cc2f03150d7de6f87c2b2ef8b6
e935b790ea0e263f45dd591357ebf9f1feeed5f1
F20110318_AABPQW zmach_c_Page_016.tif
d28ff302d613fa79382ec1306e654e3f
45c626136dc16d7eff60959311b1a37ac1ff3ed3
F20110318_AABPSA zmach_c_Page_059.tif
46dc8d7629ce910e56f92088d6adbd4f
a32aee1de3e473050b6959f715f100b69dad6b79
F20110318_AABPRM zmach_c_Page_039.tif
2d050b6abf4d4612dfbb25c764851db2
160d7b1a70732b56d173bc16c44213ff75c907dc
F20110318_AABPQX zmach_c_Page_017.tif
54a18a6a0a6157212fb0504933582b3a
4c0a2216161a9c08e66e83cc462c61769ddb0328
F20110318_AABPSB zmach_c_Page_060.tif
b4b1d40ded635a93eb7b1b55de0a9892
af4c2d373fb8bdf8f9810bfe226a5d6a7153fb9c
F20110318_AABPRN zmach_c_Page_042.tif
1ebb5b253ac23e7e8d4b380b1e48e604
41bc326b3a979706fcfe9ba9734c6ec7fef3784d
F20110318_AABPQY zmach_c_Page_021.tif
e2acc0df466aa490189e3b92b86ea710
28e83ea3209bbaeba9747b0ae504f2fd53d4e225
F20110318_AABPSC zmach_c_Page_062.tif
786061b36cc1aef3e5fc79f232e5df91
a1a3f4b07b9aa9f5563871920fd85a300d948dc8
F20110318_AABPRO zmach_c_Page_043.tif
da7d1559ff7c68c21e36da8b8cff4463
d56e655350d6aa74a1f8bcee7de86358dd6a6679
F20110318_AABPQZ zmach_c_Page_022.tif
811bffc928b8c4be7e8e58493a436ffe
1d4e037cb889b42b0cb377842804d339cb805a79
F20110318_AABPSD zmach_c_Page_063.tif
820918a7c54d10a28d71ea8948b80c57
3dfd3df8664f9a2ab5b4253aba593769547b6d4f
F20110318_AABPRP zmach_c_Page_044.tif
6cd3d2822d5a5e1c2b7484521bc8ccd4
afe275e300e580378a698dcc7b42c6da9c1ff52c
F20110318_AABPSE zmach_c_Page_064.tif
3d3d0a61852ffddbff57ccb34e98a2ae
5d7b2089ced2495aecd1afaa9f9588912e341249
F20110318_AABPRQ zmach_c_Page_046.tif
4acda1670f5375729c0d7936377afc30
148349914c56a5080e2163b6a1869a6f3b71f99c
F20110318_AABPSF zmach_c_Page_065.tif
e8324c2e3aed079e1058a02104d59182
e7cf3b73678e4948ec2ff982f72e6168f2f16832
F20110318_AABPRR zmach_c_Page_048.tif
350f84114c278706c9c2d7372599c2e1
891e0a5a6193024b26150be63b88600628b35ef8
F20110318_AABPSG zmach_c_Page_067.tif
7c30959563611f97c5de57fb0de10c9a
a4d9fbcf403ad307f40664ed428cddeec86b517d
F20110318_AABPSH zmach_c_Page_068.tif
46ae34385c27913690f0f0113bb51332
76bc534d0c4665387be8c5b5aa70116d8e32a8d6
F20110318_AABPRS zmach_c_Page_049.tif
3404c839ba41b2d50b16b65e6a3b5387
1681061e11ded6717cfb76d4ac4163998833629e
F20110318_AABPSI zmach_c_Page_070.tif
a13765f0af170811a9be158dd9b3d67c
327ae069bb5ed86b8c4c3d446a310ec3464d3606
F20110318_AABPRT zmach_c_Page_052.tif
3de6da919d578ecc7559fad1d22b0fe5
b0493b77a16ba183d2ff2852d802769285f0c6df
F20110318_AABPSJ zmach_c_Page_071.tif
f80a004d6b54f29c7018318c9cc806f0
a302dbb62742291613b7a9e1b21f880cc042b62a
F20110318_AABPRU zmach_c_Page_053.tif
395ed37c560711c87eef6ac1e3e46892
82495fd0541664765b8c9ab50ff47d6c0e44c6b6
F20110318_AABPSK zmach_c_Page_072.tif
997720683e5d73f9349a412c6553adbc
24718307027149e69e53e6baf8628c6f3d37712a
F20110318_AABPRV zmach_c_Page_054.tif
ee4668f58cb6e5b658aa0c4b999777f1
0e049ad380f2f09c880c133344e04339e70f725a
F20110318_AABPSL zmach_c_Page_074.tif
6775f2611ac4c665517d735a0e28b0ef
43c3c9b9d5a31cb13101a4b44aee74d902191edf
F20110318_AABPRW zmach_c_Page_055.tif
1ce1365be6b23ad45459694c017cfd07
acc927bd1e1b14f79384fabfe3ff1d23c6b0eb4e
F20110318_AABPSM zmach_c_Page_075.tif
81d998a6cae7d4891e349c88d8883f1d
037ee4f3ccb5445bcb8e7b970023903d8c534e12
F20110318_AABPRX zmach_c_Page_056.tif
0265a78b6b605d9f1e085ab2de28c619
cd0d0bb171b0a29a16c8917d00427aa5a2734dbe
F20110318_AABPTA zmach_c_Page_103.tif
02cac35041138d52b020d6401bc3b0b6
e50b7c52053c84d2c8c09f6de8148cf903ece4c5
F20110318_AABPSN zmach_c_Page_077.tif
0e350b6cf5ee25e198896fca7dedd19f
66bcbab93eb1155aa123b9fd86e0c7103a9c78f8
F20110318_AABPRY zmach_c_Page_057.tif
27b2d8f958af0d43b24b7f4c23b9abb6
0fbd73d9b9e8c72f3dad414a47da6e2f85740823
F20110318_AABPTB zmach_c_Page_106.tif
f4073e22eeb028b030351270f65daa3a
efa71bece8a11fefdd21d71ddb8e421c8b5601eb
F20110318_AABPSO zmach_c_Page_078.tif
6e9873cf40b782e65f45ac2865e5226d
ea6e0a61bdb575ba6b6a0489935b69dfa3a759c8
F20110318_AABPRZ zmach_c_Page_058.tif
22cb2f0fabe97d54f11bf1634fd0a600
525aff65ef74e2b1f68b78ac3514126807f94690
F20110318_AABPTC zmach_c_Page_107.tif
19c431292258ced447b047a4289c36b4
ebd81bdaa337fa1ca221c1a4fdbbe0f382db4227
F20110318_AABPSP zmach_c_Page_081.tif
2a4936619b699dbdf9f02bfd80287d8d
9c0b0f807a9939d0c779e12272a2fa3e3df4a989
F20110318_AABPTD zmach_c_Page_108.tif
aea1176580e8f2bbfe4ccaccb329076f
752b77d6022db22a7e386d1bc6ed6615ec480666
F20110318_AABPSQ zmach_c_Page_082.tif
c3afdcd08f012a5e98110261ed7eb209
c1d910a08b393d806e854b22dd33f59aae3cc4c8
F20110318_AABPTE zmach_c_Page_110.tif
82ff770de491514678ed6c3b99b87f02
6bc48ae99d0ef51c236ea91eaeb777073a480eed
F20110318_AABPSR zmach_c_Page_083.tif
8d670e153887faf5559487a1b5eca84f
a71956f80b0686249ed98300ab40ea08c37056b9
F20110318_AABPTF zmach_c_Page_111.tif
26f851946b9b75c4c5cab06f85b15eb8
020b803f9b702ba0f88bc27f57155d8d5ae7ce52
F20110318_AABPSS zmach_c_Page_085.tif
b26f98fb1a821680e1e024648bc9f6e9
ae718fe8b6976c83837b7bdd8c8e6c8818bd92e5
F20110318_AABPTG zmach_c_Page_112.tif
ed0f83c319ebe88e57d467c977f0b723
bddda515cbf29acd3368d18a7ae7126f40862e59
F20110318_AABPTH zmach_c_Page_113.tif
6f2a3f97326e1ad0b547880353adc9ee
d8e24d32837a42b08f3d306cf79e18d9f1105800
F20110318_AABPST zmach_c_Page_087.tif
44aa0ae6339f201de8545324d0928387
0eb2e28dd97e2a32617a5e45cd9a05fb77fb526c
F20110318_AABPTI zmach_c_Page_114.tif
c9b094985ce2743919881576bfdded28
059299adccfeef020a9bfa950ab0f5074c700e60
F20110318_AABPSU zmach_c_Page_089.tif
bb92f200b90656aa9de8a3990dff32a4
3a50c27036797b3a8e4bf3bffa9d80aaac5dfcf5
F20110318_AABPTJ zmach_c_Page_116.tif
de27ab57a6247dfeb7ba29bddf27ec6e
0ba8dc868d51a9d4e9a06a7e0048749f92e09537
F20110318_AABPSV zmach_c_Page_090.tif
100466baad3e6a0213224c62a4ee0e05
5cef06768dddb3ca558ca0169203a45ed50e96bc
F20110318_AABPTK zmach_c_Page_117.tif
1ffbc2066dd1a5373f86d84d7c800f44
08feb74ba181fb55fc6d28e93e950d429737f25a
F20110318_AABPSW zmach_c_Page_095.tif
d6f7b3f1e85c8186cf6896f6fa4a0b16
aa7c6f2a0c00f647a3b6b44a85d7ca32a7bbc332
F20110318_AABPTL zmach_c_Page_118.tif
9267d15e8478f327c3ac555e5a7e5d5e
72b4e204330d73c1094cf26e6a3376c93c2513c1
F20110318_AABPSX zmach_c_Page_096.tif
13b0253b7fb09031eceb5a79297171e3
7c203b80479cf3a37917ddd6564a0a2a2f0f86c3
F20110318_AABPUA zmach_c_Page_141.tif
a7bee159f299388c4797d7d004c7a8eb
97d4d839f7160a68c8ebdb133ea6f324aa8f5ac4
F20110318_AABPTM zmach_c_Page_119.tif
c64f44f7fd4401138ec964373bb3c490
d3c0d1ce8ceced3331f9a16595e264a33d24601b
F20110318_AABPSY zmach_c_Page_100.tif
528da009bb185260a2c97cbdf325ce35
1d15189a29deaaeb3d62a7076bbb54f9d0e5ef82
F20110318_AABPUB zmach_c_Page_143.tif
c8c6c8cfbbc3161050e989eeba57f6d1
d1767e8ba117cd7e285885781b1bb2c48b055e91
F20110318_AABPTN zmach_c_Page_120.tif
8106fc990068fa9af2b4037a19c52073
a1253048a325821c6771838c34cd833b2ccbe930
F20110318_AABPSZ zmach_c_Page_101.tif
1618b504ff6275da1e3600d7ad1211f4
e3117b705a7bd5eb10fdc96615a21022224d3915
F20110318_AABPUC zmach_c_Page_145.tif
10188f884e64caa34ad092e537615ae6
16018f15e2dff5f569c05676be77391dc51b0623
F20110318_AABPTO zmach_c_Page_121.tif
919d575e1ee350a717758ecae6adf106
b0d1266405eb68104b859ec21fc8a00c2ea6f272
F20110318_AABPUD zmach_c_Page_146.tif
de6424b989457077e7c4c4c44c7ba482
2750f8ba1cfadd564a118a35818bed0edeef01a7
F20110318_AABPTP zmach_c_Page_122.tif
8d7e22e3dfc8df130c22437544637c4f
c89f10138d31eda81bb1b6265a974161e7fdba1d
F20110318_AABPUE zmach_c_Page_147.tif
9f4735183e4b6236fe171f9057a17b58
bac2df5b6f33e9da3e674a9598efc7f66a7fe4e4
F20110318_AABPTQ zmach_c_Page_123.tif
8049ce40f85b47e62d30674a819e09e2
51ce3e0215263f1f5568683023f26a1f65fc4727
F20110318_AABPUF zmach_c_Page_148.tif
1733ec916f1e24271b29adc635ae836b
df76c3cbdd7cd1ed357823a592f727fe9b4d39d4
F20110318_AABPTR zmach_c_Page_125.tif
b077a81d820f41e04d3b8314bc227cfe
842236600c5174275b38ed792bb44a77b1575d98
F20110318_AABPUG zmach_c_Page_003.txt
c25d2b5a468c9d0a809aae5f66064fc2
dbac4f59541490096d5275a66880425f1ba69ad2
F20110318_AABPTS zmach_c_Page_126.tif
643ef355dfcd84b53dc755ce68f88488
44230c2b163fbb7b3fba1c38efaca6b23f9ed3b9
30347 F20110318_AABQAA zmach_c_Page_089.pro
53d187edf91d7708670b7f3894f9f33e
70ebd6e0d5da0ff401116753a9e58e64339b404e
321 F20110318_AABPUH zmach_c_Page_004.txt
9303ed07ea77c6b8dc419d0d52968196
4262654cf410d4b4dda7dc3ecf4cdcac3601d2a8
F20110318_AABPTT zmach_c_Page_128.tif
1b2609cf52509f64fa7b2ec217afcf96
e9f507d38aeb27f0e0955d48ebb4ef70443525ad
43261 F20110318_AABQAB zmach_c_Page_090.pro
c74d4108a97766d1f9dd69ca459066fd
52052bb1a5957128ad1e5c2abcc55ba66e1070f9
1849 F20110318_AABPUI zmach_c_Page_006.txt
a7af24d970fdd2af5d36b34e6f69ded6
5e8d037c2f31a28459db10ca76f5252096b39ad3
29279 F20110318_AABQAC zmach_c_Page_091.pro
82d06b264b77b65c1084dc15148ae1f5
eb62dfdccf228d29dabf85ab9a55c33048daaf47
1574 F20110318_AABPUJ zmach_c_Page_007.txt
0d2f5dc274e8ab3d6e093bfc823bbb66
7d77feb85b5eb8e191d32f490bc9c45fcc489846
F20110318_AABPTU zmach_c_Page_129.tif
8b4af1db2ca6067fe034cb1b2ae2ed8d
c23e3ea2da8d87d42fd46d6c75f770b1efe4b8fe
44545 F20110318_AABQAD zmach_c_Page_092.pro
1162afa66b47c03bc5df6936552bacd2
e282ba106723edbc4d4e8338e300c777d21fc657
2263 F20110318_AABPUK zmach_c_Page_008.txt
4f39f755e649a423a10dcb3abec7ab44
401c706043bd4cbbe58dda232f355e870448d7ef
F20110318_AABPTV zmach_c_Page_130.tif
579490d0dd9a8939ab84fc402b780ee8
4a049b64e51bf1371dd9deb174fc98d14f24c121
43654 F20110318_AABQAE zmach_c_Page_093.pro
db2734b989dcb55e25a0703a172bb287
2aedb0ddfec05a01657a0427779e75acca652546
713 F20110318_AABPUL zmach_c_Page_009.txt
076988f900fe7383cfc14fac77ffe924
052cdaaa86daac3059bcbc103739c638bd5a0fd7
F20110318_AABPTW zmach_c_Page_134.tif
1cb26d6ad1f8732a7d2dbd04e7c889e9
2f35c77e3ce4a127ecd4d30aa4195ae1df988b22
44060 F20110318_AABQAF zmach_c_Page_094.pro
543bcc2ff8f401f93cf4a5dde63519b7
e9a18a34732868e654fae8a9d20e437f298be3e5
1975 F20110318_AABPVA zmach_c_Page_032.txt
6fe08e14345202697fee6de35d843319
0f2f69f2d16688ba70a59519db9820a77f5eeff7
1709 F20110318_AABPUM zmach_c_Page_010.txt
5d660782928dca2fd5c4eba6dc65ab19
fc1c893b1494b54959558990f7b623a6776a104e
F20110318_AABPTX zmach_c_Page_136.tif
f83289a0b49a716aa706566e76a630a7
e44e00e9985b7ab1049f385fb9fdc8342cd4d251
43785 F20110318_AABQAG zmach_c_Page_095.pro
3f08600fe792d5b0c2b9db0bed6f3c1d
2f1327d83e55139e220d1f0ddc6171e70ee070b2
2009 F20110318_AABPVB zmach_c_Page_034.txt
5b5ff44aa15b229eabb523319fa7f7b5
6c26879cbddf4a01c8ebd44452bf6f3af39a4a47
1777 F20110318_AABPUN zmach_c_Page_012.txt
44964e723bc17652bf6330f045b36869
b5c6510fc92321ed1faafefc93bf56d37f7cbf10
F20110318_AABPTY zmach_c_Page_138.tif
709858ecd27711adc9505d8fe92b0f86
c30ac340b88c25f2a207b2afbbfc8ac5f5d4725e
50062 F20110318_AABQAH zmach_c_Page_096.pro
9cfc8e7f9c2a0d1e26f9d78d30a96ba1
690dc9f4ea2b857b7df958d0ac2e0b8f6829fb58
2066 F20110318_AABPVC zmach_c_Page_035.txt
1a772f4b87027e324e41b6fd50488c74
d0a4447e96a9a037abdc37ea6f8506cbdc14c48a
2040 F20110318_AABPUO zmach_c_Page_013.txt
aa32f984100be6e5ad01646f84d4a9fb
01e82c99dfd257b645ce02d5f3f58ba16edcb909
F20110318_AABPTZ zmach_c_Page_139.tif
7517677209d8c345109f83af723ee0a9
dc7a6ad2a8c437390a1fd9ffc1914963d6e26040
34145 F20110318_AABQAI zmach_c_Page_097.pro
bfd2998c0cccfbaaf328031ebb261997
b06a00afda28ec2f22f360f33350bb4235a113c1
1901 F20110318_AABPVD zmach_c_Page_036.txt
46c2030bf777ba37936d4dfd0122ba32
d3f9fbc4d7222d774bd678415cbc3c2d8ea81452
1880 F20110318_AABPUP zmach_c_Page_015.txt
ed3a2f095813453f80ae7d3add34f756
dbb2b7b0d4a81466443884c0dac05fd99b045d37
16135 F20110318_AABQAJ zmach_c_Page_101.pro
25915c8acaafd5db8f4dcab29b4d8bbb
fd3e3d0369257efb819ef730b1e2d3178576fe74
1937 F20110318_AABPVE zmach_c_Page_037.txt
09beedf3a2d33bd25e55187e29966800
7b6066bf2acc09a64f0cffdf2499c07cc4d33dd4
1676 F20110318_AABPUQ zmach_c_Page_016.txt
6b18dd7d90aa617d4661e730de08909f
75d69250b4aeb7308acb59caad09b0bd661d82a2
25235 F20110318_AABQAK zmach_c_Page_106.pro
c9d0b5c3feb0034d73608e36e4d3b83c
87dfcb481435ecfee974599db9dc21deb24b2ad1
1991 F20110318_AABPVF zmach_c_Page_039.txt
a93d04d0f4a1ebd4c4c3a45e204630a2
708086591ec0017c9792660c13f4be68af30b075
2022 F20110318_AABPUR zmach_c_Page_018.txt
fc56bb25957f0892b53ca082761799c6
ed81047462b25bdd59189ccb0613e1333a440339
45491 F20110318_AABQBA zmach_c_Page_131.pro
2e08e7ca906de25f436738e1d7c665cc
dd00a96bac9dc9d9e113117f0bc9a759c6d63951
49068 F20110318_AABQAL zmach_c_Page_108.pro
d3658ce78f5af2fbcfdb029f46e96bfa
e1a9501a244ab2c127a1bbaff4d9410292c86e11
F20110318_AABPVG zmach_c_Page_041.txt
80cd8b6043b9573b219fd7d34447950e
26b5b72d2b1175272066eb0dfc85d4905f980041
1985 F20110318_AABPUS zmach_c_Page_019.txt
8ea8ba47b9f6d4cdeee762f013f37996
862906f1be41284ed80d0b4723bebb1b9dc14c55
33998 F20110318_AABQAM zmach_c_Page_109.pro
3e667f38bf2e682c943033db1c101903
61ba88a6a375595a6f202e5e7f7e1b23f187b8dc
F20110318_AABPVH zmach_c_Page_042.txt
1eecdf491b5c5da3544b91201110bcd6
cda69a91f75c023cb84ddd89d198fa453ed1ffd4
2019 F20110318_AABPUT zmach_c_Page_021.txt
c3b1fca69580214f89671fe5ef42c795
655806916982e34beca336c106e0626bdb2bc158
3946 F20110318_AABQBB zmach_c_Page_132.pro
033cc8d64c1ff8d95cc054d4128e45c3
d7f76314b78f44cc265faac8915c71f126711e9c
46621 F20110318_AABQAN zmach_c_Page_110.pro
4dacc22a927e95f58884940a6240768b
257a111e85753c430aaa1eab3ac9e303ea5efe81
1838 F20110318_AABPVI zmach_c_Page_043.txt
1b8032a2c3b631cf4841d59ca20f8619
c32459610c6092d7eba161c40f9e7fd7f7729ee0
1998 F20110318_AABPUU zmach_c_Page_022.txt
5258f778920828925c4fded2ba6a82e9
fed8340685c9ad48f64034b3a4540fae235736fb
27432 F20110318_AABQBC zmach_c_Page_133.pro
0dd477b4f6ecbc376ce4f821ff45458c
26218f818aed2fda5597f1842d40e7331451fb46
51710 F20110318_AABQAO zmach_c_Page_112.pro
bc54a34aa9e686434f1b2e8d63816993
6921acc329c62e42991d9890781ad9eeb425450d
2006 F20110318_AABPVJ zmach_c_Page_044.txt
2f20936f0b101253c6f401c7dcd362b0
796824b1a9d953d837857830ba4ca85e99941ebb
41286 F20110318_AABQBD zmach_c_Page_136.pro
23074dae9d79cb1eee056903f4dd8f5d
d191b8007a193c4d718cd82ad53a1efdb9775e37
52611 F20110318_AABQAP zmach_c_Page_113.pro
5d3f060a8504816d065b743651de9b85
c6d9616feefc3c165281bcbe1d82dd393b4a2138
2226 F20110318_AABPVK zmach_c_Page_045.txt
d3ea697251d008230acbfb3288f3a617
ef68ab4ecdad3e7892586d07fd98cad555dc5a4e
1824 F20110318_AABPUV zmach_c_Page_024.txt
57115ecfeea047001cb9a9afd9a6ce1f
41b0516309a53e7561ebea01b76c8aba2dd251f0
56647 F20110318_AABQBE zmach_c_Page_137.pro
ffca7cb55836ae98f5321ee185bc4bc0
d00335965a3c11e60e33169c8fff42aeb67f5191
49541 F20110318_AABQAQ zmach_c_Page_114.pro
b96802f1964dcd7dc722c0445d64b3a3
692b272eecbc9713446f228d623d095052439193
F20110318_AABPVL zmach_c_Page_050.txt
1cc4a7eaa51fc582ef07c8f778f35706
23074b4bba6fc768e9cc11207848681ace9d3718
1897 F20110318_AABPUW zmach_c_Page_026.txt
36d3a05de4bb4a439818436cecd8a556
416109c4a77048a5dc39e0cc6c9fe6e8535f7d62
59718 F20110318_AABQBF zmach_c_Page_139.pro
56c6f226ef44eeb8031df0919bd1ddf1
b90fe7b377b6391e4fa230dded125f7aeb14d2dc
49615 F20110318_AABQAR zmach_c_Page_117.pro
6df5585c0614e952340c2879a0e7fef3
9d3e502be5d9a7ecb6f5d74c89ffbd9ed15e5a09
1911 F20110318_AABPVM zmach_c_Page_051.txt
bd9781468e763dce571d3853fc68ac88
ec480a445a1cbbddd87c52852b87499150c9be84
F20110318_AABPUX zmach_c_Page_027.txt
5bd5d60029cf185b14271e0239f10c63
45c9742acec14f09c9e05cb2d2d09a4a44d85d7c
59977 F20110318_AABQBG zmach_c_Page_140.pro
71e9b6bbdfecb0759ced0a9f09309aff
f198ed593f4985bc0271c6bc766e865061363d7f
2171 F20110318_AABPWA zmach_c_Page_071.txt
7d8ba192f83d1a4caf12b29ec5be292b
8ff298a227520ae32418d3885d6f44dee8ca9bbc
47867 F20110318_AABQAS zmach_c_Page_118.pro
d37ee3f7be878ae230f2f8ef159c75e8
050d2afeef279b05ff68528b266c1ae622aee92c
1194 F20110318_AABPUY zmach_c_Page_029.txt
06f346d6e8e671b6b89e5d548166d30e
ef3f44ff07ac73b2512a89e17c074d88f4fa28f4
56955 F20110318_AABQBH zmach_c_Page_141.pro
4b14032dacad715d9b1fa2ab9963e568
09788aaae74d53037b6a5509f3680dfbec885b9a
1495 F20110318_AABPWB zmach_c_Page_072.txt
bbefb499ac0189344076085e935b0e18
656a6e7ffce2a91842b5a12bb52d2f779bc9a8ee
45170 F20110318_AABQAT zmach_c_Page_119.pro
4ecd08e3a7b49c3cf047a7b6227aac0d
7016fffbcb56cc0dc5f22a9424611f9c14ae29e8
2144 F20110318_AABPVN zmach_c_Page_054.txt
5fd5cd38d8743eb5067ac7601db78413
7a33afdda0cc35ad903d94819ce8083a8abddd7b
1855 F20110318_AABPUZ zmach_c_Page_031.txt
dfc8a186a9e10067764f6412e6f5c01c
c6a39800870953bf6a6b61da52f4f2d8cbad97a9
58824 F20110318_AABQBI zmach_c_Page_142.pro
d2c8c30060ae1df0d049af83a2ac9ea6
d4078d7bcec2ce7920867296469895ec6933a10b
F20110318_AABPWC zmach_c_Page_075.txt
0d3504bfbce0ffef97f10873226c3938
e48e8867a748eea1007ad6fc6d705c413bf5cde4
29147 F20110318_AABQAU zmach_c_Page_121.pro
dabe55867f63a7a6233e70a3a222e641
e608d3661585094dc50b3b2d10f89c435b26afee
1999 F20110318_AABPVO zmach_c_Page_055.txt
7324c0db4fdb5e6424997ba9881f8e31
45e28dff799f869101423e4af97919031f3a5133
60399 F20110318_AABQBJ zmach_c_Page_145.pro
1e0d6c3a74bbce94aa15423cbb9fb770
fa840105417b7a741a2e1cf6f4822b18a5461031
1610 F20110318_AABPWD zmach_c_Page_076.txt
b6cda9dbc299afee33f8758617a2ca4b
2da5629e6886e02635ed79ada07881e43802318f
1210 F20110318_AABQAV zmach_c_Page_122.pro
c4a56bdd440e31866bbb8e8db19e2f27
2b155557ce69b272d301e850363963c674fd9273
2783 F20110318_AABPVP zmach_c_Page_056.txt
a3460611aad4ad4e426e571d2305c8e0
b1e25b3fc4a86ac12bb0d630b028c9e088028bfb
59798 F20110318_AABQBK zmach_c_Page_146.pro
5e47ebb97536566680575844fefd605f
604169d3d0d4c162530b28712ba6e079a5e05437
1465 F20110318_AABPWE zmach_c_Page_077.txt
839038365ae6e1d3e91ac5a994f8ce9a
d226600636b49f27d374e6ddccc55f0b158775fa
36534 F20110318_AABQAW zmach_c_Page_123.pro
2652da835d7a7dcd85a8d4d307b5b044
a1ae32cf0bd427a36b0a03fb131771bcc2546ae8
2014 F20110318_AABPVQ zmach_c_Page_057.txt
810342eda0834ae7ec99aefb69441d58
220b63348cc6b54e52ea5742d68ca8ae43869519
103394 F20110318_AABQCA zmach_c_Page_021.jpg
1e0f30de6af577219b68c0203c3ec9b4
a346a1c34586498d8e9089f06f3245847c5b14f6
37953 F20110318_AABQBL zmach_c_Page_148.pro
384d92090fcc0c4d2ea8a76d5a3b5ad8
4e7af77500d18458a73cf71ec149a1d501a5be5b
1598 F20110318_AABPWF zmach_c_Page_078.txt
eff7476294c08ce204c6a6bd4684d850
761b3a85d1b51fe2b3d4cea6a35660f656c5f9f5
40024 F20110318_AABQAX zmach_c_Page_126.pro
6cec9205f942038e80004260e43b7aec
5b4fa2de0eb5a8c8c440dcbc9f0467e8f1ae4fe4
707 F20110318_AABPVR zmach_c_Page_058.txt
b1c86ba80703e53a96c48580df80b0c3
b5dc797e906c90872c29c9d74f0f427e4832e5e5
102043 F20110318_AABQCB zmach_c_Page_023.jpg
1c210ec8d750e832a0d5162d0a6c9207
223677285b29251cdb1bd3b3f29273140736ff5a
5310 F20110318_AABQBM zmach_c_Page_002.jpg
f62e662b4f6f0c885c4c1592fa98a67d
50704938de8f767630c1241b4a8926a2629dfa4b
1974 F20110318_AABPWG zmach_c_Page_079.txt
5a349262332da9d0c0187387f9963562
be7a92429f4751d83f0f23a7659e49a2045f7cef
33068 F20110318_AABQAY zmach_c_Page_128.pro
f4300dc804014bbf3298771488983f70
5a478aa9a6b58d067e29ba9aa00c2598eccdcdbc
1577 F20110318_AABPVS zmach_c_Page_060.txt
b89b68fae57812c6833fca27ab4cfd53
3abb0376c4b0c1b0ac7af462567aad4fec52a1eb
91607 F20110318_AABQBN zmach_c_Page_003.jpg
adc2d64915b647de64e874c6ecc2c7a1
f6234184523b263b192d5dbdccc7e2308a9c650b
1938 F20110318_AABPWH zmach_c_Page_080.txt
41e95fb59c520adbd71611a6cca05bd8
d81ea70b3d682c3d482806192114d7f45b41ef4e
36654 F20110318_AABQAZ zmach_c_Page_129.pro
a9ee18153235ad4d0717b79ba6c4f09c
b8831e9db506ff106f4e92d087bdcb6be1240899
1815 F20110318_AABPVT zmach_c_Page_061.txt
38352a4afefc339e23ab831b916bba5f
220810dc24b53f0e1a24bd7629ca3827035ad2f8
93216 F20110318_AABQCC zmach_c_Page_024.jpg
c1bafa88e105a040dca7c0a85ee75da2
d3ef11b3ed55a120f1397a11d4606ace505eac21
18763 F20110318_AABQBO zmach_c_Page_004.jpg
e58bdcb653e1fea41b70aa69d4c34a92
e7d76ee32d423a5d26b57fabd0897ad12892c595
1354 F20110318_AABPWI zmach_c_Page_081.txt
3b772123eaad88607f5b3bd312436558
42176129fadd71fee739f07b970f05919c00b506
F20110318_AABPVU zmach_c_Page_062.txt
a84bb5f77d510fcba3e0eb36025faaf9
b261da24bb4b5ab9c076ed286833422599a581d7
98186 F20110318_AABQCD zmach_c_Page_025.jpg
44c08fc6c98cd47e24e60e64df0e9318
de8c12a1b56f0949137ca1762e0a0be012d443cf
65347 F20110318_AABQBP zmach_c_Page_006.jpg
d36ab56fa8020a3eb72d6e404775ff63
18fc5c3edf7c2ea3943e79857bb9c905c78ce1a9
1864 F20110318_AABPWJ zmach_c_Page_083.txt
d4c244a9207303f119d92217a7f2d843
67c6f6c2c8c65858df0a5bcca3ab42745f51ca7c
1669 F20110318_AABPVV zmach_c_Page_063.txt
391b4e2de978dccf2d5fb3226aa33dc7
bcc3e3d58c33b3ef7326ace91470a5b536ef7b7d
102165 F20110318_AABQCE zmach_c_Page_027.jpg
eb77272f62917af81917b0cf9a4fd144
dc444f02b20a4a384b05080daec5e6033c6ca2a2
69552 F20110318_AABQBQ zmach_c_Page_007.jpg
67afad12ee901471d9fd8c48b768e7c8
62d2700e1c7550db426258f3d96c2c8753ab5d14
941 F20110318_AABPWK zmach_c_Page_084.txt
39144ea4c2d1170947662bb2d87544fb
26e704d5e9872ac49023132b02e246e6859aa381
103598 F20110318_AABQCF zmach_c_Page_028.jpg
a70dbb9b275afd86b5bbc76880cff64b
9941f8751b4dd7ac82d67f3c84b89ff3c89d3039
94680 F20110318_AABQBR zmach_c_Page_008.jpg
ef587366b04749f81fd7daab9b0cc21a
f47cc751eb790ca7cb9ce9dc8b8bd00d72c1c97d
1397 F20110318_AABPWL zmach_c_Page_085.txt
1b433be2d558d71b59abf909c04aeec0
5b1a60f816c2c2011e7808553cbc52c9472cdfd0
2016 F20110318_AABPVW zmach_c_Page_064.txt
1d3c8001b5c9bd70797da99bd4822d01
0de63e9c7d2c3a8593d6e726b0839a4a24acb259
105733 F20110318_AABQCG zmach_c_Page_030.jpg
2523b9fed8bdaeb858ea03600af3b96f
c93b6d91f43a9dc6bfef0b8c522c6069dafdc371
2034 F20110318_AABPXA zmach_c_Page_112.txt
e705f09994ee56815cfd8722bbcb6157
d33982580eaaea1bd994278bc25333dc551c27d5
80790 F20110318_AABQBS zmach_c_Page_010.jpg
98e4fffec586df724b0882396b40395f
ddddc4f9de0edce6cc0ef4d93479ee1ef34be6bd
408 F20110318_AABPWM zmach_c_Page_086.txt
4bd251e4ae577cad7b49d11fdf569578
8b9efa1b1f31e4c44293bcded59d293a10070ed1
F20110318_AABPVX zmach_c_Page_067.txt
f38752cfc5cb58e5aaf9d17f3a3c1f90
1f3e953525e3dd9a1372dd471155a064d3e7e50d
97082 F20110318_AABQCH zmach_c_Page_031.jpg
172f775fa06df34920c900a6cd528a4f
7ec5d061bb24eb6e911a369710e2bde68f85d464
1952 F20110318_AABPXB zmach_c_Page_114.txt
ba514aa56049baf875b1b27cbfab97ca
022b7467e95be350dd20d6655c5091b24d613a5e
65274 F20110318_AABQBT zmach_c_Page_011.jpg
33f1071d17ee42cfdf3701cc4803c4ba
14e80fba5ab654c06a9b50ff350e3c714733dbbb
F20110318_AABPWN zmach_c_Page_087.txt
f44560afe8dccfd551ebe33d15fc8c59
6d4a37d8ddbb45157753d1e1ff5d5284830bf5bb
1724 F20110318_AABPVY zmach_c_Page_068.txt
9780374912da841056b35bc9eb964dd1
32855bd868ca98e0f092d2cecd58fc3c5499a005
102130 F20110318_AABQCI zmach_c_Page_032.jpg
52df0b077facb3a8b2c7800d454829b3
48ae231519ba0dda334e139418cca501fcaa968e
2077 F20110318_AABPXC zmach_c_Page_115.txt
7eb088b4089940800ac7eb86a936e319
e3ab1d84cb06956362588b9ffae38c26f0977b5f
87355 F20110318_AABQBU zmach_c_Page_012.jpg
808d280851eb7ad168957450df0a8fa6
2d673414ad49375b18958d97a502c7a65a9d33b1
1986 F20110318_AABPWO zmach_c_Page_088.txt
c29ec041896cfb1c1e526a4f4e700433
197fdd06b69e03cb3e18c95bd89b28594304d0ea
2072 F20110318_AABPVZ zmach_c_Page_069.txt
360f4a5b13cd3a09b136d4a8e2f6730f
2c0e3e5a152e7f242597e193e89a89c950c4aa62
104561 F20110318_AABQCJ zmach_c_Page_035.jpg
3d1c808f426b17ac1ca8d52fb00655f0
f66b75632c48856498f435a262a954506d616c51
1953 F20110318_AABPXD zmach_c_Page_117.txt
8ffff853b1a040bac4d554c8932538a3
387914d597ef3f83428cf2cc4a4b0b6e59ae302c
93018 F20110318_AABQBV zmach_c_Page_015.jpg
8606060e433e93b183df0dda4d3b2bc1
0973be5e3166151a854fafc2325e7879eec955e4
1470 F20110318_AABPWP zmach_c_Page_089.txt
98fd1cfb6b30091c8e5dd3a92eba6e46
41df8c1b4106eb498652828c2de8f1761d5140a1
F20110318_AABPXE zmach_c_Page_119.txt
2fb5bd6314166bfda3018f93ab56794a
a059ebc1fa10cbca40a7738704180dc1a5db1fb0
84963 F20110318_AABQBW zmach_c_Page_016.jpg
a7b077f98d7b4c01e49aa835f7737bb9
f644a2421d84c8876b9b5dc4825134e24a7f2ea2
1841 F20110318_AABPWQ zmach_c_Page_092.txt
006f8c808a069913832b3a0918604cb1
05c90050cb5523d0e640bb38ef62de5e93aeda1e
99992 F20110318_AABQCK zmach_c_Page_037.jpg
3d8c8afac57f32632eaf400b93806209
c516933cb5cb5f57e5bb74e449524974b50f882d
2033 F20110318_AABPXF zmach_c_Page_120.txt
5e7f836571a1da26661fc10576d7676f
9d65a85b6b0dc0320a8ee4c982a5b793bf171a22
91052 F20110318_AABQBX zmach_c_Page_017.jpg
366e3d0f39f2c7589695e8013769a80c
6a0d2d92a7c4ef3d6f530c4e591d776e57d4c076
F20110318_AABPWR zmach_c_Page_093.txt
3022c9d1bc3ac439b5a2a4f8f3d05707
f157410df05b67cb6aea2e14cd45d63640245c0a
91770 F20110318_AABQDA zmach_c_Page_061.jpg
4093d819d6347db52d15ef4b8878f70c
0be6facf6a07a2166eaa9384ae3fdf340c991996
96239 F20110318_AABQCL zmach_c_Page_038.jpg
b20303812bc1c6cf8c2d2b6b8121e7fb
3b3c56c9da2dd648df84ca4b760a1a90fa15494c
72 F20110318_AABPXG zmach_c_Page_122.txt
1246a625b8cfca1bd9a85aee52caed02
ff4187e7f5c441bbe1f051daa20d14f87ce4109d
101333 F20110318_AABQBY zmach_c_Page_018.jpg
4b47a488607d751c44cd7bf4efbc0d3e
a7b418ee80c457027810057888b8db453b3e8a38
1825 F20110318_AABPWS zmach_c_Page_095.txt
d7ca6e750b71418e93075e839fcde5a7
5794d5ebf3940e392700fef5a9edd389f3755095
99530 F20110318_AABQDB zmach_c_Page_067.jpg
e1a2bbbc1f92be356cc231bca9e8665d
7eeeb4d28de5741f43deeb0dffb1b60b6351b920
103739 F20110318_AABQCM zmach_c_Page_039.jpg
de26bb072f9e9cb3739f6e325eb9c8e0
6dd2815f89af0bc47ffc6ce5d41d0058195f7eaa
1537 F20110318_AABPXH zmach_c_Page_123.txt
da01a3d1e67b9c7a4549d82339f77e0c
fd910284cf31359a77fd793f9b2db85c35f6882b
96269 F20110318_AABQBZ zmach_c_Page_020.jpg
a16be732f11dbd9992eb5568fb5ff810
8f3257af91b0c84d512b01d2e0b89d6d7cadc6e1
1715 F20110318_AABPWT zmach_c_Page_097.txt
bf33017258da39d2d0a60ca94bbf8cd4
4760afcae23580d01711a7df14749fd3a1482630
74394 F20110318_AABQDC zmach_c_Page_068.jpg
8bc2587bed7646cb5a9d2a070e7505e1
13501ad29e738305edda0f51fbba34008015a2f0
98868 F20110318_AABQCN zmach_c_Page_040.jpg
21c2cb807fdf72c1298c07e29c2e90e1
7f1ce6e127dd8ec971bf361198a1fb35d2040d6f
1370 F20110318_AABPXI zmach_c_Page_124.txt
94300f4aea6cf918a0a582f9e5c6a7af
a486164adce04e5bddfb312c33ec6daf63ddd622
F20110318_AABPWU zmach_c_Page_099.txt
731fe95070cbf6eda52a8485e87bf44b
6626134bb4e0c66db26ffc21d7e9ee8bd319c09b
101795 F20110318_AABQCO zmach_c_Page_042.jpg
56cda93b02c7a7d355967921146ba7d3
b3a901afd9e7dfbf85d237e5357b133277576b32
1694 F20110318_AABPXJ zmach_c_Page_125.txt
28fe2b6dd65239959ae528d62718831b
3487c4f8451224bfbb43e95651168292f7825fa5
795 F20110318_AABPWV zmach_c_Page_101.txt
fb5a26680561351972ccff71a8c93466
458903a1783a958af793f5feb642c9241696d215
105790 F20110318_AABQDD zmach_c_Page_069.jpg
7a852abbd8d4ddf0c8c0abca00a52850
618811531dc4bf7a1420a7014314babbc7b1a07a
100781 F20110318_AABQCP zmach_c_Page_044.jpg
c02882b41bafb2593a4158cd3aeba9ba
e1d00e5a114039987867817c5782b4e69d35f629
1497 F20110318_AABPXK zmach_c_Page_127.txt
d06b4ded5cd825a85d2ef9d38a628f14
611441cb22abd5fc0e417f07633b4d656a82c30a
1444 F20110318_AABPWW zmach_c_Page_103.txt
c91572d25490c4f1acacdea99fc84426
b5fc7235bfa1f46fd39ff491fb6ee1dc97d088e0
98402 F20110318_AABQDE zmach_c_Page_070.jpg
60c49154b4b7059c1f7a20d99721eecc
a2c114ea1068947796b680e103a4d413bd9d44fb
107933 F20110318_AABQCQ zmach_c_Page_045.jpg
5d88d9cef3c79133e5fc1bfc1e3e3832
f388dcd4b94d4d30e3c2c565d883b495b360e881
1003 F20110318_AABPXL zmach_c_Page_130.txt
efe394f94cbf29da00f1b8f737e72aea
8d47780a4fe2aac5581a459ab109928cd432fe8c
72332 F20110318_AABQDF zmach_c_Page_073.jpg
bac68e5c5517b63d1f94ec24bce5558f
c254a9f44bc4d9ad7e9cb72f1b79460f8cb793aa
93734 F20110318_AABQCR zmach_c_Page_047.jpg
a93a75350538e7927354e82d8b1fcc8f
4f0e86713d123c97190dfbf77e206853c61e9f5d
1856 F20110318_AABPXM zmach_c_Page_131.txt
643f8faeddb0ddf50468874bd349124c
9814bbc0d869af0534abba57551afd43d9d95c76
1076 F20110318_AABPWX zmach_c_Page_106.txt
0e471f01b91706a330aff3416ccb1c48
2a5e6440c76a2d9ebe99dac75b79a26b76bafe32
104313 F20110318_AABQDG zmach_c_Page_075.jpg
47b9240d10224d3cc7d508ac6b409a57
38514805b59e3e18b34d157ba16d0c3aa478f88e
7844 F20110318_AABPYA zmach_c_Page_004.pro
920eefe7f5e7bf37d086961d3d775646
9f4824a940183d7e6d4e21cc4697731772a4ce41
102567 F20110318_AABQCS zmach_c_Page_050.jpg
ac0e42ec6187c3ca56427e644d06a2d2
cb93ac47c4915864e6265380d240b4a325962fe9
239 F20110318_AABPXN zmach_c_Page_132.txt
a883e491a7f006e3b79a27479d64637c
55742c4385a79da45c1cce46f9b2dfcf4b98979b
1965 F20110318_AABPWY zmach_c_Page_108.txt
7790cf820b36649e4c1235914f56eb7c
7dfde32501a741eedc346fb25ad490feb56b7299
69507 F20110318_AABQDH zmach_c_Page_077.jpg
ddad814799d544714663b2e09f072db8
b29387fa65fb9fed448b8ea618ead1ef50a6df6e
47235 F20110318_AABPYB zmach_c_Page_006.pro
9eb4b69830292199adc7f27c894f17e0
f7255c43b7e0ce62170ce29397e3df11e5e09028
100158 F20110318_AABQCT zmach_c_Page_053.jpg
c68e53e88db13df94e00081434d9668a
76c5cc3d5dfc5e7408ae05a2ef709dd0a273b1d6
1388 F20110318_AABPXO zmach_c_Page_133.txt
982f488f22ce8a7a841f747ec05883d4
5597d9a4785b7ce0a285c562d2f44c82118bfd2c
F20110318_AABPWZ zmach_c_Page_111.txt
a1e965980cd34df710f77178635e0bd9
e374cc99bc42f993a3df35b68db5e3511b9e08b4
79734 F20110318_AABQDI zmach_c_Page_078.jpg
042731989a42a9169a1c3994b264e7d5
9da3af0cb4987479213d0636c4c1adc92ab6f205
37822 F20110318_AABPYC zmach_c_Page_007.pro
f9753b82faf14af292e5d2a472c60aba
2d8b944113db8ab519b00b28761cd39fbac2cb7d
102414 F20110318_AABQCU zmach_c_Page_054.jpg
04644f8fa97f9afe9e00e5c778aafbe9
91527bd4b82cdd9c4d8d11fee643f621e65158b3
77 F20110318_AABPXP zmach_c_Page_135.txt
c38d8996aa5b5380484f210b4aa3ef10
b66c85294ed4dc9a9d644a1b3b1829d93a3f43f6
100961 F20110318_AABQDJ zmach_c_Page_079.jpg
39c87d8bb9d68850a64fd982a0417755
2758c78c467d1e7f2bb43cdefd18bb6d75e57af9
17665 F20110318_AABPYD zmach_c_Page_009.pro
0b473a9e8d795941caae587b881e89bb
28062c66e605370736a1e69143ded70a012cb184
104400 F20110318_AABQCV zmach_c_Page_055.jpg
1c3da37bab8ab3558c5ff944a0919aa9
c61f1ca5ce7b022f74147d18c1016e6773dbad2a
2317 F20110318_AABPXQ zmach_c_Page_137.txt
19f2474e9605da0f21b2acd831e6ea9c
baccfcbce49deb96955186faff1a11cdef1e2b37
36242 F20110318_AABQDK zmach_c_Page_081.jpg
efabe07afd469509e43131ab773c7bbc
1c83b3b543a8a4ee12aa251cfa85b59730dc3039
42200 F20110318_AABPYE zmach_c_Page_012.pro
9a057cc06742b1bb9f7b059a5ed706ca
609602075e4c4300365ba8c54d2cf1d7a0204f85
101307 F20110318_AABQCW zmach_c_Page_057.jpg
30ed3f9ab3a477dfe75ee395ce80184f
a88dc8bd5bd734de30b0ee9f97809a26e7029c0a
2386 F20110318_AABPXR zmach_c_Page_138.txt
ac9a49f50a59c236979cc4c4c6c204ad
4a69424fd0131418fedbc2908616e6acfe750968
75498 F20110318_AABQEA zmach_c_Page_099.jpg
f179aee32d1bdc7da10010b0b007873f
61047bba6feea820a94db56a0127d66530c8d4c9
78271 F20110318_AABQDL zmach_c_Page_082.jpg
6bf6213321c526c5395521d90a44bd3d
41d5c77801d370e3e0c2c5979ab17a5a0ad6f67b
52912 F20110318_AABPYF zmach_c_Page_014.pro
8bea7020f308d4544f68b0565c3e10e9
f06b5b9383c6a5e2bbbac1dcb6eda3e9d70ca72d
34844 F20110318_AABQCX zmach_c_Page_058.jpg
4dd4bd8a62cb06faa9496d09b81002fb
ff76a41809d80026a00990958967132d93c20de9
2458 F20110318_AABPXS zmach_c_Page_139.txt
8ee7377b61e093a86b269d2cb565010e
5a9582025070eed474951f37b2baf684077e9473
109234 F20110318_AABQEB zmach_c_Page_100.jpg
c6e00e0a787bec231364a3a0b2099089
877cfac4caecdc7899b91198fde46ec9dfcf0c2f
97477 F20110318_AABQDM zmach_c_Page_083.jpg
c386e3451e500ee107284af5c70a7b07
2df122dbd9b59fa7f47de3a58c27ea513b887f94
50246 F20110318_AABPYG zmach_c_Page_019.pro
b047bdbcbcfa986b6becb97fc0b0c554
bb1e380b7636379c9c6b7e21978cade1d6bccbac
86675 F20110318_AABQCY zmach_c_Page_059.jpg
a9474a249d9250666fd046ba4e77f942
50fc4bd2a4f9f8c4c72364b3ba7ca8dda8bd6a5e
2485 F20110318_AABPXT zmach_c_Page_140.txt
89928fbf55474dc40ca0374a07413c8b
07b3d101d2f6122862495b7552a1cad81f38ca48
102090 F20110318_AABQEC zmach_c_Page_104.jpg
b9fb723cb9f3085a7fc82aaad302b904
47787e9925b4d25cfd6908b06ea34c535cd399c3
65226 F20110318_AABQDN zmach_c_Page_084.jpg
2b99ffec4df40d7f4e8f9cbb9233da85
3749bb1079835ab099c0fd352a355cf4505f75db
47136 F20110318_AABPYH zmach_c_Page_020.pro
079c7d8ceed5ec0f628d76e73f4885c0
f73ed18d2aad2a52bf6ac547969ef672e02db800
73377 F20110318_AABQCZ zmach_c_Page_060.jpg
c11b682f9eb7c6d661c4e960734420ec
e94f3a61e02843bd5b8423ca1efa9f38dce98021
2349 F20110318_AABPXU zmach_c_Page_141.txt
8a8c5fc618bf9fb43b0ade006eb6ebb1
e599483c29dcdd8e680221392a4c153fd79c9f6c
57023 F20110318_AABQED zmach_c_Page_106.jpg
22dfc1a9dc66b65155f821e5c6493431
8474e2ed845e8806a524782b57ab5e87ed47dd7d
70840 F20110318_AABQDO zmach_c_Page_085.jpg
62203412603a095f9d0be4e86ecf5d23
2a877b12a1e5e1b86c876e1d01f34fbd50e01e9a
51065 F20110318_AABPYI zmach_c_Page_021.pro
586f93bf302a6a7ada447713b20521a9
0a47a48dcf233fa0470d26e4516bd4c0215532d1
2427 F20110318_AABPXV zmach_c_Page_142.txt
c9bd30917b4e1fb1696f6153075c4914
32d35bcc2dec4c3a6b34fe2b7fe2719c750938d4
19174 F20110318_AABQDP zmach_c_Page_086.jpg
2f4a32055b9e78eed69000901940f949
7a2e623e6f1d5f4d7e6691736c1f7187b84143da
50792 F20110318_AABPYJ zmach_c_Page_022.pro
a7fe40349cd15b108fe2325c50e6d13f
59ff16573e3890741a832be7dd5c906079804380
2249 F20110318_AABPXW zmach_c_Page_143.txt
9946c5474c44e96c6087f471ed1ef5df
6f97b942665024626bee0f7270cc2730bd80cc55
71153 F20110318_AABQEE zmach_c_Page_107.jpg
0de121458b8d3a20450ddee35b95ec2d
74f2899bc17cd133dfdbcb683631f49cb19accbd
77597 F20110318_AABQDQ zmach_c_Page_087.jpg
ebba919386cc3ce54b38c3ea4c2bccd7
99adddea171ff9397f8e2d7b4a7f98166dd2102d
45970 F20110318_AABPYK zmach_c_Page_024.pro
404bfd6bba21082d9d38fb47424ddb52
8bf188469725d9374f63ccc40afbaf59fc304ee4
10089 F20110318_AABPXX zmach_c_Page_001.pro
6a19637334d985e81da26781763c962b
cfc5b75c0eaef3e91093afaa31112327b1b1ab1a
98007 F20110318_AABQEF zmach_c_Page_108.jpg
1b86bb8d7ce62d47819752dc157e0730
cff7e020491ddf7823483225871d844a04e97d17
87033 F20110318_AABQDR zmach_c_Page_088.jpg
40355e88a429dee2f86ac6af71efb0b4
9579f3216d641a38ebb2d9cda4f90042539a2819
47816 F20110318_AABPYL zmach_c_Page_025.pro
be4a1e29407abdd9a0c61ef367b2b96f
5b2442537e23065e1040f1aa94e79aac6b615a6b
55249 F20110318_AABQEG zmach_c_Page_109.jpg
739b2f7fe2c6827076edb07b5789030b
1d29b73a9b03d8032ecd684b195f1976e01b0bdf
61364 F20110318_AABPZA zmach_c_Page_048.pro
8611276ea9719d798a548a2e973406d2
aa0b51135fb015920182a2fd6ce29600629ea2d0
69824 F20110318_AABQDS zmach_c_Page_089.jpg
f12fee6d580b73fb9689711f88a6113f
6bee0401166be25a4dce382dbc3ac322b7388b2c
51591 F20110318_AABPYM zmach_c_Page_028.pro
ef47673059626c5ba600448559b70e71
d77cc45c9ef816979351cce26c3dbfdb47d69488
1381 F20110318_AABPXY zmach_c_Page_002.pro
c03fb23e11dda4442b382fad7453b394
01ca187787e3647c824576f19d4b39a611c63dcd
96028 F20110318_AABQEH zmach_c_Page_110.jpg
ed3722189437fa690101faf8f754b7fe
7929fdc5bd8ee83cb7fa256ec0ba56fe61c59d51
48616 F20110318_AABPZB zmach_c_Page_051.pro
63100438cebae62d8122531b8a83ab56
cda2e141a1fa2bfa649ab9ef729b2e18e5724f7d
69045 F20110318_AABQDT zmach_c_Page_091.jpg
f47db76547d3b91e5d408ab845599326
acfdf7d04e6feea5f4a7e1c422ece94e4216396e
29177 F20110318_AABPYN zmach_c_Page_029.pro
71da06b4952c3a76d7eee833987dca43
86fa06ca1e3dc3f41011ce0ce019e8e25c7f47b2
44388 F20110318_AABPXZ zmach_c_Page_003.pro
c2adc361a52d4f415bf0240fe77f97b4
a8990b862145c69c900216829cc4995cd3c45281
100388 F20110318_AABQEI zmach_c_Page_111.jpg
c50273275ebed2a02ece17c13d4e5eb9
7398a63f08333a34602c8157c82abf12ee33444a
44407 F20110318_AABPZC zmach_c_Page_052.pro
ec509df09864ac9027600da7ea0b64bc
696ed63726ced2407e4f059255bfd99b2b799478
88706 F20110318_AABQDU zmach_c_Page_092.jpg
485c8af206c72ec85e0dded29234df73
7583e5823622ca06a526985a6d22af8c12fa97fd
51281 F20110318_AABPYO zmach_c_Page_030.pro
bac5473da85ba3562cca65c7bccaee6f
a6bb01cca7e7be77c43ced4e47a2020c576c5e09
107294 F20110318_AABQEJ zmach_c_Page_113.jpg
2cdeebe5765cb0ad42747141729b7637
6251f7fe97dc2963b8456c0b1da6aa8d72fd4e74
54330 F20110318_AABPZD zmach_c_Page_054.pro
46a0871efd5dd574780b080b46672f6a
0808dbbadc27ef60cddbf80fbace24b07d6a4033
92837 F20110318_AABQDV zmach_c_Page_093.jpg
677306298aa76f3d5f58203f9492bae7
1bbe23fa10c1e9f5e4be0d286a2af9ecb7ae1c3e
46928 F20110318_AABPYP zmach_c_Page_031.pro
bd9359e4b63728dad462aa0af0a968f1
a27762afd247caeb409b9cd2e5b911f6c624c947
106324 F20110318_AABQEK zmach_c_Page_115.jpg
2fec8146e9565e845f674f9943ce8863
0a89e3f2338a3fe8b0a8b5e9970b74984471a8e0
50783 F20110318_AABPZE zmach_c_Page_055.pro
99747fecd2a3f8f22b0beffbb88c17a3
51e2872f13f798e10cff633f8484a8f821b7ce67
92083 F20110318_AABQDW zmach_c_Page_094.jpg
9775415e5e7ffcc903fe04d75b3ed6fd
89b9ad6ee0790c9dd697d78dd2071c13c6234f5a
50064 F20110318_AABPYQ zmach_c_Page_032.pro
f83af6fa6463a6374a61f7dc40c5bff9
65e60f54eeb4eed71a7a2838bec017061844eedc
104199 F20110318_AABQEL zmach_c_Page_116.jpg
f04825c87f3b05ce2abe0b67af555511
e88fa231b29343e467d21d169d40bdaa1d4059fc
68969 F20110318_AABPZF zmach_c_Page_056.pro
080bb707217fb854f9059217b68ea80c
0abd68812fea7b5425e2cf6f75245c54dea668e1
87610 F20110318_AABQDX zmach_c_Page_095.jpg
1882fdfddacf94af232f16304c2b6623
6ed12066f785b42663b7cc69b4c62f66f2d51b88
47968 F20110318_AABPYR zmach_c_Page_033.pro
2cabb184553fc666a41a2a5c751012e2
92d5ab9c872386f9b135c8f70008a5985f7585a2
126125 F20110318_AABQFA zmach_c_Page_139.jpg
673101c18ad4f0e08ecc6c2ac3acd447
b6ddbdcaf7f284a0ad46054f6e1293fbbad8327f
103572 F20110318_AABQEM zmach_c_Page_117.jpg
a3191e3a02f63d73a1798980eca8b86c
ea2975b5e5bc0285e9057d4f7e6b07483e4e825a
50016 F20110318_AABPZG zmach_c_Page_057.pro
da214cad3ce5e62792ef741697a1467e
9a8eb945195192611dfd3eb9ed02d10e148c7f2f
57939 F20110318_AABQDY zmach_c_Page_097.jpg
7b9efdf519541cf4b7c51c9e35e6ba55
dd402f7eb6b2ec1541768613ba14ed3d5a226522
51139 F20110318_AABPYS zmach_c_Page_034.pro
5afadb9c53814d2af77db25fd30cbbe9
85a862ab1f458411a0ed9b0a7c38fd97129fbf04
131264 F20110318_AABQFB zmach_c_Page_140.jpg
a0c4522aa4cf2e39a98f875fabd68b68
242986a73899162b1a8ab9e44065640f5ea19cc6
96923 F20110318_AABQEN zmach_c_Page_118.jpg
61417adf6bfb1500577e16bd4f2a30dc
6ea3ae55d0fbbf5f54c649e79ac2d8bc22268aec
42527 F20110318_AABPZH zmach_c_Page_059.pro
fccf7afe5baac091f60bedebcfb1348d
68eb667ddf2c6095c16b1eb8f64e96c6acdf0969
76344 F20110318_AABQDZ zmach_c_Page_098.jpg
897e52f31241be98e82180ee1f04b113
137a5308ae5b2898162eae9a5d71fb648b235afe
49165 F20110318_AABPYT zmach_c_Page_037.pro
251a3b66eccfd447ba310b18333918fe
30f301aa19914bb0f6353072cb4d1adf791a1cb0
120100 F20110318_AABQFC zmach_c_Page_141.jpg
442f982bb6b205088e707c6ff9273812
9f178dcf94ce23708755abc0fc48d951a4bec87e
93081 F20110318_AABQEO zmach_c_Page_119.jpg
4374f0fa0fe62f73b6cb1e7f894ba604
d77bc0e0ab6b1848fad79284279022623de1fa8e
35610 F20110318_AABPZI zmach_c_Page_060.pro
65b897604da4752646780043dbe30a61
b3698aab973084c655d4fa1ca1bf8d31a9af3213
49137 F20110318_AABPYU zmach_c_Page_040.pro
3d24c78f83900b2056c99c53918db738
f521f72a4efbcece4ffde55da347310598317877
118408 F20110318_AABQFD zmach_c_Page_142.jpg
a4a986281203c30759664f2eae28d007
b937dec3b9628645f5dda2fd95b077d11744a148
102151 F20110318_AABQEP zmach_c_Page_120.jpg
039966b52dbd9eb00609f5eb61336664
38cc13b5675aed6895b0a86864734ba58575128f
45818 F20110318_AABPZJ zmach_c_Page_061.pro
60ae1ad2d4a8a470c53485831c3ba616
3670fd7e34c6982d8a02e614532cac1fd097c870
50829 F20110318_AABPYV zmach_c_Page_041.pro
bb7f4b2ab7c039b53be655bc937dcd9f
7b7eaab32051cde02cd75fbab24997648ca18b6b
112910 F20110318_AABQFE zmach_c_Page_143.jpg
2b6731ee624573bfa32583c4274b0595
a0878e53d25b616e4aa31d8f5148a99e5be6ee49
79586 F20110318_AABQEQ zmach_c_Page_123.jpg
001735cd8ec93454cdfdb69520940618
42c3ba85a1cf008f18dfd061f6754116f7fde10d
39677 F20110318_AABPZK zmach_c_Page_063.pro
78ba96379546ea1d8fee1e47cb11b6a8
6088d8d3a0dfdf7b4f9c90d0be303aceb5560d56
50367 F20110318_AABPYW zmach_c_Page_042.pro
8da60536bdfae898502fcfe4388f8301
c9e76d8b5bd23acc0387686a72272ff84f9f1c71
73683 F20110318_AABQER zmach_c_Page_124.jpg
37098ec632240c956b6a255a737804c1
ecc200771f5eef6a05aceb40467296f1c7a87ebf
46133 F20110318_AABPZL zmach_c_Page_064.pro
5f1a0b825f43d2e0802e5418c415de8c
885245c35e2b556181d4a7782482b24f6638ab44
46430 F20110318_AABPYX zmach_c_Page_043.pro
864eb09a7ca186074086374251d74fd5
3229b95bb8e663c31cee2024f7390c11ed22e4e0
129099 F20110318_AABQFF zmach_c_Page_144.jpg
c00ed18452ac106d08b84f9d7bbeecb2
46cefc4705dddb7a7e0424aeb6a749d0e2b51184
93330 F20110318_AABQES zmach_c_Page_125.jpg
e40a376e2ebf7878603a72a88fe7dcc4
634c0eea556bbde68c2197ef2443f0c71970abc2
50571 F20110318_AABPZM zmach_c_Page_066.pro
ece3abc829c3ec35a0e1abe6160087b8
727866921c2fa8fe60925694b6cb851bd5e51564
48681 F20110318_AABPYY zmach_c_Page_044.pro
4cc51f4063735bc4b5889f8df6616e22
8b91adb1a165a56d8e2e434236123286abc11ec3
127195 F20110318_AABQFG zmach_c_Page_146.jpg
f742e1fca3433635c9712dde5678599c
1cf6ed013820a2bfd36a6e88bd31cbdf5b171554
83677 F20110318_AABQET zmach_c_Page_127.jpg
59d6125ad263be03c55243f759089985
3d224030168e4746dd8bbc6fe194e2ec8c2c6a53
49458 F20110318_AABPZN zmach_c_Page_067.pro
2261d4dfbfe547b36381f32f20154b9c
e73ece23c35302be748b7a6a5ebb9f4fd513353c
123602 F20110318_AABQFH zmach_c_Page_147.jpg
17f8edccde4eebb9b624ace794aa7d7b
34e9e6970c0b9b2e53001877c0b72d0f88a3c16c



PAGE 1

NAVIGATING THE COMPLEXITIES OF LE GISLATION: HOW ELEMENTARY SCHOOL PRINCIPALS INTERPRET AND IMPLEMENT FLORIDAS THIRD-GRADE RETENTION POLICY By COURTNEY CALDWELL ZMACH A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2006

PAGE 2

Copyright 2006 by Courtney Caldwell Zmach

PAGE 3

ACKNOWLEDGMENTS Many people contributed to my educational background, to shape the educator and researcher that I have become. It is difficult to know where to begin, because each is important. I thank my steadfast supervisory committee chair (Dr. Richard Allington), cochair (Dr. Anne McGill-Franzen), and members (Dr. Gregory Camilli, Dr. Thomas Dana, and Dr. Sandra Russo). Each of them supported my scholarship and development in becoming a reading researcher. Dissertations can be a lonely journey and I thank those who spent time listening to me. At different stages and in different ways, Dick Allington, Anne McGill-Franzen, Greg Camilli, Tom Dana, Sandra Russo, Jenn Graff, Evan Lefsky, Jeff Miller, Katie Solic, and Lunetta Williams provided listening ears. I value their time, suggestions, and unwavering support. My thanks go to the many members of the Just Read, Florida! Office and the Florida Department of Education for taking time to answer questions and for gathering data files for my study. I am indebted to Hajnalka Peto and Jenny Williams for the time they spent transforming state data files, so I could create the databases used in my study. I thank my parents and family, who supported my decision to uproot my life as a classroom teacher, to return to school. From as early as I can remember, my parents shared with me the value of education. Their support and encouragement led me this far. My former colleagues and students deserve thanks for inspiring me to reach for higher educational goals. I also express gratitude to my new colleagues at the American Institutes for Research (AIR), for supporting my continued scholarship. iii

PAGE 4

Of course, this research would not have been possible without the cooperation of the twelve Florida school districts. Hurricane Season 2004 was an especially trying time for Floridians. Special thanks go to the elementary school principals who gave their time to participate in this study. iv

PAGE 5

TABLE OF CONTENTS page ACKNOWLEDGMENTS .................................................................................................iii LIST OF TABLES ............................................................................................................vii LIST OF FIGURES .........................................................................................................viii ABSTRACT .........................................................................................................................x CHAPTER 1 INTRODUCTION TO STUDY...................................................................................1 Introduction...................................................................................................................1 Policy Problem..............................................................................................................2 Study Approach............................................................................................................3 Perspectives..................................................................................................................4 Purposes of this Study..................................................................................................5 2 REVIEW OF THE LITERATURE..............................................................................6 Introduction...................................................................................................................6 Research on Retention..................................................................................................7 High-Stakes Framework.............................................................................................18 Florida Policy Context................................................................................................31 Summary.....................................................................................................................46 3 METHODS.................................................................................................................48 Introduction.................................................................................................................48 Participants.................................................................................................................49 Data Collection...........................................................................................................51 Procedures...................................................................................................................54 Design.........................................................................................................................63 4 RESULTS...................................................................................................................76 Introduction.................................................................................................................76 Multilevel Models.......................................................................................................76 v

PAGE 6

Summary.....................................................................................................................85 5 DISCUSSION.............................................................................................................87 Introduction.................................................................................................................87 Research Question 1...................................................................................................87 Research Question 2...................................................................................................89 Research Question 3...................................................................................................91 Policy Considerations.................................................................................................97 Policy Recommendation...........................................................................................104 Future Research........................................................................................................107 Conclusion................................................................................................................109 APPENDIX A FLORIDA PRINCIPAL SURVEY..........................................................................111 B REVIEW OF SURVEY PILOT STUDY.................................................................120 C DATA SOURCES....................................................................................................122 D SAMPLE FROM STATISTICAL SYNTAX...........................................................123 E RESULTS TABLE FOR ALL MODELS................................................................124 LIST OF REFERENCES.................................................................................................126 BIOGRAPHICAL SKETCH...........................................................................................137 vi

PAGE 7

LIST OF TABLES Table page 2-1 Florida retention policy good cause statewide exemptions......................................36 2-2 Supports identified in Floridas legislative intent....................................................41 3-1 Time in years: Pre-policy versus post-policy...........................................................49 3-2 Degree of support category, Florida Principal Survey............................................54 3-3 Survey response rates by district size.......................................................................57 3-4 Usable sample versus returned responses by district size........................................60 3-5 Third-grade percent retained usable sample principals versus the total population.................................................................................................................61 4-1 Results of fitting a taxonomy of multilevel models for change to the log percent retained data, Models A through E (n = 102)...........................................................86 5-1 Third-grade FCAT reading achievement Level 1....................................................99 C-1 Sources of data.......................................................................................................122 E-1 Results of fitting a taxonomy of multilevel models for change to the log percent retained data, Models A through F2 (n = 102).......................................................125 vii

PAGE 8

LIST OF FIGURES Figure page 2-1 High-stakes framework............................................................................................18 3-1 Item 9, Florida Principal Survey.............................................................................52 3-2 Item 14, Florida Principal Survey...........................................................................53 3-3 Item 19, Florida Principal Survey...........................................................................53 3-4 Employment history, Florida Principal Survey.......................................................54 3-5 Population versus usable sample mean percent retained by year.............................61 3-6 Mean SES values over 5-year period for usable sample schools.............................62 3-7 Mean retention rates by year....................................................................................65 3-8 Examples of principals third-grade retention rates over time.................................66 3-9 Third-grade retention over time...............................................................................67 3-10 Histogram of percent retained before the transformation........................................70 3-11 Histogram of percent retained after the transformation...........................................70 3-12 Examining the homoscedasticity assumption..........................................................71 3-13 Multilevel model for change taxonomy...................................................................73 3-14 Number of zero percent retained values removed by year.......................................74 4-1 Scatterplot of log percent retained plus one.............................................................78 5-1 Examples of mean predicted retention trends over time..........................................88 5-2 Mean predicted third-grade retention rates by poverty level over time...................90 5-3 Example plotting retention rates using principals mean SES values before and after policy implementation.....................................................................................90 viii

PAGE 9

5-4 Mean predicted third-grade retention rates by principals current retention belief over time...................................................................................................................92 5-5 Principals retention belief by the mean predicted third-grade retention rates over time disaggregated by poverty group...............................................................95 5-6 Mean predicted third-grade retention rates by degree of support over time............96 5-7 Degree of support by mean predicted third-grade retention rates over time disaggregated by poverty group...............................................................................98 ix

PAGE 10

Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy NAVIGATING THE COMPLEXITIES OF LEGISLATION: HOW ELEMENTARY SCHOOL PRINCIPALS INTERPRET AND IMPLEMENT FLORIDAS THIRD-GRADE RETENTION POLICY By Courtney Caldwell Zmach May 2006 Chair: Richard L. Allington Cochair: Anne McGill-Franzen Major Department: Teaching and Learning Retention policies affecting third graders have become increasingly common as schools, districts, and states across the country work to comply with the federal mandate for all third-grade students to read at or above grade level by 2014. Florida is one such state to enact a high-stakes retention policy as a way to meet accountability challenges. Third graders in Florida who fail the state assessment, the Florida Comprehensive Assessment Test (FCAT), are held back in grade. This exploratory study gained understanding of Floridas third-grade retention policy by focusing on the elementary school principal (n=102) as the unit of analysis. To examine the Florida policy, it seemed useful to better understand patterns of third-grade student retention over a 5-year period, retention beliefs of principals, and degree of support given to students affected by the policy. The overarching research question of this study was What is the trend of third-grade retention practices before and after x

PAGE 11

implementation of Floridas third-grade retention policy and how is this trend impacted by other variables? Using a combination of items from a researcher-designed survey of elementary-school principals across twelve districts, and data provided by the Florida Department of Education, I used a multilevel statistical model for change examining within-person change and between-person differences in change, beginning with the 1999-2000 school year third-grade retention rates. Study variables also included measures for preand post-policy, poverty, trend within time, retention belief, and degree of support. Results show that the trend within time and poverty at the school level dramatically affected retention rates across this sample. Students in higher-poverty schools were impacted most greatly by the policy. In other words, lower achieving students in higher-poverty schools were more likely to be promoted pre-policy and because of this, the mandate had more impact on students in higher-poverty schools than in lower-poverty schools. Regardless of the actual impact on reading achievement, Floridas retention policy has eliminated social promotion in third grade. xi

PAGE 12

CHAPTER 1 INTRODUCTION TO STUDY Introduction Even with the national agenda to leave no child behind, substantive segments of the student population are held back in grade because of mandatory retention policies. Today, thousands of students in Chicago, New York City, and across Texas and Florida (and elsewhere) have been affected by the use of high-stakes assessments to make promotion and retention decisions. Retention in grade (sometimes called nonpromotion, being held back, flunking, or failing) is a policy that retains a student in the same grade until the student meets requirements to move to the next grade (Jackson, 1975; Jimerson, 2001; Shepard & Smith, 1989). Retention is not a new idea: it is an old and highly contested practice that has fluctuated across history. Earlier research indicated reasons for retaining students, including low performance on standardized assessments, youngness for the grade, physical size, immaturity, and even poor attendance. Today, some believe the conventional wisdom that if a child does not fare well on a standardized assessment, then that child is ill prepared for the next grade level. Once again, retention has resurged because of mandates tied to meeting accountability criteria. The antithesis of retention is social promotion, where students are promoted from grade to grade, with same-age peers. Social promotion is something of an enigma, as many state departments of education have not collected information determining how widely it has infused the system. Equally troubling, limited information is available 1

PAGE 13

2 regarding how retention decisions are made, or whether standardized assessments are a determinant for retaining students (U.S. Department of Education, 1999). In the Council of Chief State School Officers (CCSSO) 2001-2002 Survey of Student Assessment Programs (SSAP), 10 states reported that standardized assessments played a role in retention decisions (personal communication, September 14, 2004). However, we know that this number has grown, as Florida (in 2003) joined the ranks using reading achievement scores from the Florida Comprehensive Assessment Test (FCAT) for promotion and retention decisions. Policy Problem What truly is the crux of the problem at hand? Not performing on grade level is seen by some as a social problem (Oakes, 1999). To fix this perceived problem, lawmakers instituted what they believed to be a solutionretention in grade. If the children did not learn to read at a proficient level the first time in third grade, will another year in third-grade be the solution? It would appear that the policy is working when more students achieve a passing score. Retention in grade has been researched for decades and proven ineffective (Allington & Walmsley, 1995; Shepard & Smith, 1989); yet Florida persisted in enacting a retention policy to remedy the perceived social problem. The challenge we face is more than whether failing a student is justified. It comes down to how we, as educators and researchers, can most effectively help students who struggle with reading, especially those who fail to make a passing score on high-stakes assessments. Nevertheless, retention policies tied to reading achievement are becoming increasingly common as schools, districts, and states across the country contend with the No Child Left Behind of 2001 mandate for all third-grade students to read at or above

PAGE 14

3 grade level by 2014 (US Department of Education, 2001). Florida is one such state to enact a high-stakes retention policy to meet the challenges of accountability. One wonders how this retention policy benefits Floridas students. As reported in a dataset provided by the Florida Department of Education (FLDOE), over 27,000 of Floridas third-grade students were impacted by the retention policy after the 2002-03 school year. Of this number, after 1 year of retention as part of the States chosen intervention method to help increase student achievement, only 59% passed the FCAT after their 2 nd year in third-grade (Florida Department of Education, 2004c). There are a great number of areas awry in education today, but retention used as a reading intervention is questionable. State-level policymakers believed that Floridas retention policy (F.S. 1008.25) would help mend the perceived problem of third graders not reading on grade level, but the research base does not support that belief. Study Approach In the early stages of [policy] implementation, summative measures usually are inappropriate. More appropriate questions for analysis involve the extent to which necessary resources are available to support implementation, whether there is evidence of good-faith efforts to learn new routines, or indication of commitment and support within the implementing system for policy strategies and goals. Milbrey McLaughlin (1987, p. 176) On considering how best to capture the spirit of my study, I decided to collect data using a survey questionnaire, and to use historical retention data and poverty indicators provided by the FLDOE. My exploratory study aimed to gain understanding of the interpretation and implementation of Floridas retention policy by focusing on the elementary school principal as the unit of analysis. To illuminate the current situation in Florida, it seemed useful to better understand Patterns of third-grade student retention over a 5-year period

PAGE 15

4 Impact of school-level poverty Other variables such as a principals retention belief and the degree of support provided for students affected by the new state policy The overarching research question of my study was What is the trend of third-grade retention practices before and after implementation of Floridas third-grade retention policy and how is this trend impacted by other variables? To answer this question, I designed a multilevel model considering factors such as the schools level of poverty, while determining whether past retention practices appear to influence the level of current support offered to students at risk for retention or currently retained. Perspectives The worldview of the researcher bases socioeconomic status (SES) as a predictor for students educational successes. [Poverty] is a condition, like gravity, that affects virtually everything (Bracey, 2003, p. 46). Foci throughout my study presuppose that economic disadvantage places many schools and their students at risk for failure. The correlation between student achievement and poverty level is widely accepted (Linn, 2004). Florida elementary school principals were selected to gather information first-hand, from a range of schools across the state. Principals are vital stakeholders in making decisions at the school level, and being informed about the happenings in their schools. Principals historical retention practices were built into my study design to examine retention over time in relation to retention belief. Selection of principals in no way suggests that principals deserve sole responsibility for matters concerning retention; however, as the figurehead of leadership in a school, they serve an important role.

PAGE 16

5 Purposes of this Study Results from my study serve multiple purposes. As a result of the third-grade retention policy, thousands of third-grade students have been retained or re-retained (Florida Department of Education, 2004c); however, this policy has brought forward several mandates and initiatives geared to support students who struggle with reading. Because of the mandate to use particular supports, examining the relationship between degree of support available and retention rates may help current principals assess (or reassess) their practices to provide the interventions proven to benefit students at risk for failure. Stakeholders from school, district, and state levels may have a clearer vision of how this retention policy has been enacted. We know that beliefs held by those who implement policy play an important role (McGill-Franzen, 2000; Spillane, 1998); therefore, we must understand how these beliefs affect the retention policy. Regardless of whether one concurs with or opposes this policy, it is important to understand whether retention beliefs held by administrators across the State impact the proportion of students retained, or even how supports are distributed to help students. This research may help administrative leaders (and district and state leaders) consider patterns of retention in a different way. Results are unique to Florida and may not be generalizable to other states; however, other states practicing retention or considering implementing a retention policy might gain insights that may benefit their locale.

PAGE 17

CHAPTER 2 REVIEW OF THE LITERATURE Introduction Today in Florida, a retention policy prevents third-grade students from progressing to the fourth grade when they fail to master grade-level standards (or expected learning) on the state assessment, the FCAT. Given this policy impact on students, why is Floridas retention policy (Section 1008.25, Florida Statutes) worth study from the vantage point of an elementary school principal? As the primary instructional leader of a school, the principal has the extraordinary task of remaining abreast of policy changes, while communicating vital information to staff members and parents. The school principal serves as a gatekeeper of information for the instructional staff, and for the students they serve and protect. The principal is largely accountable for all the happenings in his or her school. In the hierarchical system instituted in districts across Florida, it is important to understand how administrators (in this case, principals) support those students at risk for retention or already retained in grade. Over the past decades, other investigations studied the effects of retention policies, including student achievement, dropout rates of students affected by retention decisions, and beliefs about retention (Shepard & Smith, 1989). Since Florida implemented the retention policy in 2003, ending social promotion for third graders who fail to read on grade level, many facets are worth studying that can add to and extend the existing body of knowledge. While one can choose to focus on retention at the classroom level (examining issues related to students, teachers, or classroom instructional practice), I 6

PAGE 18

7 believe the principal holds direct influence on the aforementioned. Few studies have examined the role of principals and how they enact retention policies. Since the principal acts as an intermediary or messenger disseminating information from the district (or state) to the classroom level, how a principal interprets and implements this retention policy is vital to our understanding and evaluation of what makes a difference for those at risk for retention. Research on Retention First, we must consider why retention in grade has resurfaced as a commonly accepted practice, or intervention of choice, in states and districts across our country. To understand where we are today, let us go back and review a brief history of how educators, in the United States, came to practice retention. In the 19th century, as America developed and expanded its public school system, formalized instruction typically occurred in ungraded one-room schoolhouses with students ranging in age (Ruhl, 1984), where students progressed through the education system learning content. By the mid-nineteenth century, the European education system began to influence multiple aspects of the United States education structure (Balow & Schwager, 1992; Ruhl, 1984). Soon, many facets of the United States system became gradedeverything from schools, students, curricula, teachers (Balow & Schwager, 1992). A graded, or sequential system, turned learning into discrete parts, as students were held accountable to learn specific information that built upon one year to the next. As student learning became linked to yearly increments, the introduction of textbooks helped standardize the curriculum (Jackson, 1975). With the use of textbooks, it became clear that students did not learn at the same rate. Hence, originated one consequence when textbook content was not masteredretention in grade. For students

PAGE 19

8 who did not master the material, this solution required them to repeat the same grade level to continue learning that material (Holmes & Matthews, 1984; Jackson, 1975). Decades later, Leonard Ayres described in his book Laggards in our Schools (1909) a trend that Dr. William Maxwell, Superintendent of the New York City schools, reported in 1904. Maxwell observed discrepancies in the normal ages of students within the New York City school system. He calculated a staggering 39% of children in the elementary grades over the standard age for a given grade. This inconsistency between age and grade when students fall backward in grade was termed retardation (Ayers, 1909). This fluctuation was the result of retention in grade and research ensued to learn more about this phenomenon. One of the first retention studies, The Backward Children Investigation, led by Ayers in 1907 studied children who were not progressing through school at typical ratesthose who were over-age for the grade. Ironically, Ayers areas of inquiry are quite similar to present-day wonderingslearning more about differences in student characteristics, the age of students when starting school, teacher effectiveness, and curriculum differences. His research questions included, How many of the children in our schools fail to make normal progress from grade to grade and why do they fail? How many of the children drop out of school before finishing the elementary course and why do they drop out of school before finishing the elementary course and why do they drop out? What are the facts and what are the remedies? (Ayers, 1909, p. 2). Ayers calculated the survival and mortality rates of students by establishing the proportion of students who remained in the school system without dropping out. Based on his calculations, most students remained in the system through the end of grade school

PAGE 20

9 with a severe drop off in attendance during the junior high years and few retainees remained to attend high school. The retained were more likely to leave school before their promoted same age peers. His findings established that students, who do not succeed, simply drop out (Ayers, 1909). Evidence has mounted for nearly a century that retention does not benefit students. In the 1930s and 1940s, researchers conducted studies dispelling the notion that retention helps children. In fact, Scott and Ames (1969) reported that the negative findings on the effects of non-promotion were so uniform in the 1930s and 1940s that many investigators considered the question closed (p. 433). However, Goodlad (1952) revisited retention when it surfaced as a hot topic of the day with educators asking the question What is best for the development of this child? (p. 150). Over fifty years ago, Goodlad recognized that retention was not a one-size-fits-all solution asserting that promotion on the basis of fixed minimum standards is not adequate (Goodlad, 1952, p. 150). He tested hypotheses dealing with differences in social and personal adjustment between children who repeated a grade and those who did not repeat. Based upon his findings, he found promotion to be a more justifiable educational practice, taking into account the best interests of the child. Goodlad recommended that 1) children be treated as individuals, not subjected to system-wide policy, 2) teachers use facts related to achievement, intelligence as well as human growth and development in decision making, and 3) [the] instructional needs of the pupil should take precedence over matters of administrative expediency in dealing with questions involving promotion and nonpromotion (p. 154).

PAGE 21

10 Using Goodlads (1952) recommendations, Lobdell (1954) conducted a study where one school district adhered to criteria guiding both teachers and principals in making retention decisions for low achieving students. Criteria for these decisions were both general and specific. Briefly, general criteria included what would be best long-term for the child, including 1) holding a child back only one time before sixth grade, 2) avoiding repetition of sixth grade and finally, 3) children should not spend the repeated year with the same teacher as the previous year. Specific criteria suggested that many aspects be taken into account beyond the score on a standardized assessment, including current grades, intelligence scores, age, size, social characteristics and the attitude of the parents toward the childs progress (Lobdell, 1954). Essentially, all children were to be treated individually when making retention decisions. Lobdell (1954) asserted that holding back a child for two or three or more years beyond his normal graduation age, as practiced in past generations, can in no way be defended (pp. 335-336). Both the teacher and the principal were involved in making these decisions; however, the role of the principal was more of a judge. Principals insured that the facts were present and that the best interests of the child were taken into account (Lobdell, 1954). Retained students who were subjected to these criteria showed short-term success, however the long-term effects were not discernable from Lobdells available data. Like Goodlad, Lobdell (1954) concluded that broad policies for retention needed to be replaced with criteria to guide the decision making process, as this was believed to be in the best interests of the child. Changes in promotional policies was the subject of the Hall and Demarest (1958) piece in The Elementary School Journal. They identified three levels of promotional

PAGE 22

11 policy: 1) Grade-standard: students learned a predetermined amount to move to the next grade, 2) Continuous-promotion: as in social promotion, where students move through the grades with same-age peers, and 3) Continuous-progress: students are retained based upon individual retention decisions (Hall & Demarest, 1958). This study examined the policy shift in Phoenix, Arizona treating children as individuals to make retention decisions. The move to continuous-progress was seen as a positive shift for children (Hall & Demarest, 1958). Research on retention was sparse in the 1960s. However, as the decade neared an end, Chase (1968) and Scott and Ames (1969) published their work on the subject. Both studies omitted students retained for reasons such as low intelligence, brain damage, or emotional disturbance as these were not viewed as necessarily correctable via retention and included only those who were immature because of youngness or behavior. Findings from these studies supported retention as a means to increase maturity. Nonetheless, over the years, research has consistently shown that retention causes negative long-term consequences. Efforts to increase achievement by means of retention simply present more hurdles for the student to overcome. Grissom and Shepard (1989) created a structural equation model, or causal model, in hopes of explaining the causes for dropping out of school. Although they found that substantially more students who were retained drop out of school early compared to their promoted peers, other factors, in addition to flunking, contributed to the dropping out, such as the need for the student to work or attend to other family matters. Research on dropping out of school has been studied in more recent times with similar results to Ayers (Roderick, 1994). Roderick (1994) also found that the retained were more likely to dropout leaving school early.

PAGE 23

12 There are other long-term educational outcomes worth consideration. Retained students are more likely to be placed in special education programs (Barnett, Clarizio, & Payette, 1996; Guthrie, 2002; McGill-Franzen & Allington, 1993), and students placed in special education or other remedial programs rarely escape their special education label (McGill-Franzen, 1987; McGill-Franzen & Allington, 1993). Retention Research in Chicago The Consortium on Chicago Schools Research at the University of Chicago studying the implementation of Chicago Public Schools 1996 retention policy provides a wealth of insight into the effectiveness of that policy (e.g., Roderick, Bryk, Jacob, Easton, & Allensworth, 1999; Roderick & Engel, 2001; Roderick, Nagaoka, Bacon, & Easton, 2000). When students in Chicago did not meet promotional gate cut-off scores on the standardized assessment, Iowa Tests of Basic Skills (ITBS), they were held back. Promotional gates in Chicago occur at third, sixth and eighth grades. After analyzing 2 years of retention records, these researchers determined that many children were being placed in special education programs after the retention decision was reached. In fact, they calculated that between 17 and 20% of retainees were recommended for special education placements (Nagaoka & Roderick, 2004). One caveat Nagaoka and Roderick (2004) acknowledged was the difficulty discerning whether these students had unidentified difficulties prior to the policy or whether these students were being pushed into special education programs to improve accountability reports. Both explanations are plausible (Allington & McGill-Franzen, 1992; McGill-Franzen & Allington, 1993). In addition, one must strongly consider that lack of teacher expertise is another possible explanation for these referrals. When teachers use

PAGE 24

13 all they have in their bag of tricks, they commonly turn elsewhere for guidancespecial education is an intuitive choice. Some retained students did not meet the criteria to move through the promotional gates after their second attempt. These students remained in the same grade for a third year in the same grade; these students are called the double retainees. The Chicago team worked to determine whether the double retainees were better off after these retentions (Roderick et al., 2000). After equating the scores of the third-grade ITBS to the fourthand fifth-grade tests, the researchers found little conclusive evidence to suggest that double retentions benefited Chicagos third graders. In fact, after three years in third grade, about 80% were promoted to fourth grade, while about 10% of students were placed in special education programs. Thus far, results from the study in Chicago confirm past research efforts that retention does not work. In fact, they have found little evidence that students who were retained did better than their low-achieving counterparts who were promoted (Nagaoka & Roderick, 2004, p. 45). Retained students who were offered additional chances to meet the promotional criteria struggled to reach the cut point. Consortium researchers support early intervention; however, they caution that this does not mean high-stakes assessments need to be moved to earlier grades (Roderick et al., 1999). Reviews of Retention Research Jackson (1975) presented the first review of research on retention by dividing thirty studies published between 1911 and 1973 into three groups according to the research design. Based on his analyses of naturalistic, pre-post and experimental design, he found

PAGE 25

14 that retention may benefit some students, although more students seem to benefit from promotion. Jackson also suggested that the retention is unfounded as an intervention. Next, we examined evidence from using meta-analysis, a research method that takes into account the quantitative outcomes of multiple research studies (Cooper & Harris, 1994; Light & Pillemer, 1984). The steps of meta-analysis are important to evaluate highly contested issues, as it helps compensate for biases in the sample and findings (Camilli, Wolfe, & Smith, in press; Holmes & Matthews, 1984). One benefit of meta-analysis, and the reason why these studies are presented here, is that this technique is much more comprehensible to the reader than lengthy recounting of each individual studys methods and results (Shepard & Smith, 1989, p. 16). Two noteworthy meta-analyses conducted over a decade ago aimed to learn more about the effects of retention on students (Holmes, 1989; Holmes & Matthews, 1984). The 1984 meta-analysis systematically reviewed 44 studies that fit the selection criteria. Essentially, research studies needed to have specific features for the meta-analytic calculations. Holmes and Matthews (1984) required original studies to contain these criteria: 1) report the effects of retaining students in grade school or junior high, 2) provide enough data to calculate effect sizes, and 3) compare retained students to promoted students. After statistically combining the results of these studies, overwhelmingly the evidence confirmed that retention does not work (ES = -.37). The sign of the effect size indicates whether combined outcomes of the studies had a positive or negative effect on students. An effect size of -.37 suggests that the negative effects of retention far outweigh the positive.

PAGE 26

15 Only 5 years after Holmes and Matthews published these results, Holmes (1989) performed another meta-analysis using the original forty-four studies along with an additional nineteen studies conducted since that review. In the 1989 analysis, 9 studies showed positive effects, benefiting the retained. Although the overall effect size was slightly smaller in the 1989 analysis (ES = -.15 vs. ES = -.37), the overall direction of the results confirm that retention does not support the development of students academically or personally (Holmes, 1989). Additionally, Holmes (1989) conducted a secondary analysis to ascertain why there was a difference between these effect sizes merely 5 years later. He found that studies fell into two types, positive and negative. Most of the positive studies were conducted in suburban schools with few minority students, contained remediation plus intervention, and compared grade level peers as opposed to same age peers. When studies were matched on IQ, achievement tests, socioeconomic status, gender, and grade, the effect size was -.30, after controlling for possible differences in the samples. More recently, Jimerson (2001) conducted a meta-analysis examining the results of studies dating from 1990 to 1999. Compared to the meta-analyses cited above, selection criteria for this analysis included accepting only studies with matched comparison groups. With an effect size equaling -.31, meta-analytic methods repeatedly substantiate that retention is not an effective method to increase student achievement (Jimerson, 2001). In his conclusion, Jimerson (2001) recommended moving beyond questioning whether to retain students and to move forward considering which interventions are effective to remediate students who struggle academically. Retention Trends

PAGE 27

16 Studies exist documenting trends of retention practices over time. Of particular interest here is Allington and McGill-Franzens study (1992) investigating school-extending practices in the New York State. These researchers wondered whether there was a shift in retention to Grades K to 2 in an effort to delay the predicted third-grade retention. This clever move allowed a delay in public accountability reports (Allington & McGill-Franzen, 1992). Early identification of students in Grades K to 2 who would likely be retained in third grade were held back in earlier grades in efforts to remediate them prior to the mandated flunking in third-grade, if they failed the state assessment. Retention Beliefs The contradiction between research and actual classroom practice is deep-seated in a belief system that has been delivered over the past 90 years (Reitz, 1992), or over the past 100 years now. Two studies included here, focus on educators beliefs about retention, were conducted during an era when social promotion was a more acceptable option. Byrnes (1989) conducted a study on the attitudes of students, parents and educators on the practice of retention through two means, survey and interviews, in a large city in the southwestern United States. We concentrate here on the questionnaire sent to forty-five principals and assistant principals with a 78% return rate. Findings from the principal questionnaire determined that most principals (74%) favored retention for students who were not meeting the grade level standard. Principals also favored immaturity (54%) as a cause for holding students back in grade. Contrary to popular belief, in that study retention was not seen in a negative light. Significant differences were detected when respondents were asked who should have the final judgment in retention decisions. Teachers felt this was their responsibility; whereas principals felt

PAGE 28

17 this fell in their domain. Alternatives to retention were explored with teachers and principals choosing options such as smaller class sizes and more individualized instruction compared to other ideas that they perceived as changing the current school practices. For instance, options not selected included flexible entry age, transitional maturity classes, and multiand non-graded school structures (Byrnes, 1989, p. 114). Conclusions from this study showed that teachers, principals and parents believe retention benefits those students not performing on standard or who were thought to lack maturity. Byrnes (1989) described retention as an intuitive choice, even though research still did not conclusively suggest a benefit for students. Another study using a multi-method approach focused on the attitudes of teachers of grades K to 7 (Tomchin & Impara, 1992). Using a questionnaire and interviews, Tomchin and Impara found that primary grade teachers (grades K to 3) are more likely to retain students, compared to teachers from middle grades (grades 4 to 7). Primary grade teachers do not believe harm will follow a retention decision, whereas teachers in older grades were more skeptical. On the questionnaire, almost 98% of teachers disagreed with the statement that Children should never be retained (Tomchin & Impara, 1992). As principals are typically former classroom teachers, it is of interest in the present study to determine the retention beliefs held by principals, especially now that social promotion is constrained by policy. "Parents, teachers, and principals seem to play a crucial role in the decision-making process, and generally a veto from any of these consultants can result in promotion instead of retention, regardless of performance on competency tests" (Nikalson, 1984 and Rose et al., 1983 as cited in Jimerson, Carlson, Rotert, Egeland, & Sroufe, 1997, p. 4). In Florida, a veto is not possible.

PAGE 29

18 High-Stakes Framework To understand my policy study, one must be familiar with the high-stakes framework (Figure 2-1) within which retention is situated. Also, as students reading success is pivotal to the national goal for students to read on grade level by 2014, we see how retention interconnects with the literature of educational reform, accountability, and high-stakes standardized assessments. We begin with a review of the essentials to understand this high-stakes framework beginning with an overview of national educational reform exploring past and present movements focusing on standards-based reform, seeing how reform entwines with accountability and standardized assessments. Next, we examine the uses and possible misuses of standardized assessments and the relationship to retention. Finally, by focusing on how policy is interpreted and implemented, we work toward a common understanding. Adding to the background of my study, efforts were made to focus on elementary school principals and the State of Florida within each area of the framework. Policy Interpretation Policy Implementation High-Stakes Retention Policy Standardized Assessments Reform Accountability Figure 2-1. High-stakes framework

PAGE 30

19 Reform Movements In considering where we are today, let us remember past policy, or first generation policy (McLaughlin, 1992 as cited in McGill-Franzen, 2000, p. 800). The current talk about educational equity, and raising achievement are not new ideas. These have been policy goals for at least 40 years, when the federal government passed the Elementary and Secondary Education Act (ESEA) of 1965, in particular Title I, aimed to improve the education for students from impoverished backgrounds (McGill-Franzen, 2000). For 40 years, high-need schools enrolling at least 40% of families from economically disadvantaged backgrounds have received funds from the largest federal funding program, Title I. Economic disadvantage is determined by the percentage of students who are eligible for a free or reduced price lunch. One goal of Title I funding has been to close the achievement gap by providing all students with the opportunity to attain a high-quality education, regardless of their economic disadvantage or advantage. Educational attainment for all students was on the national educational agenda. With reform movements continuing to permeate the United States educational system and gain strength, there is impetus for broad, sweeping transformations and having them happen instantaneously. Reforms are one method of instituting change. A Nation at Risk: The Imperative for Educational Reform is widely regarded as the origin of modern American educational reform (National Commission of Excellence in Education, 1983). However, such reports moved us into the next generation of education policy (McGill-Franzen, 2000). It called for the American public to restructure the educational system and raise standards (National Commission of Excellence in Education, 1983). Specifically, one recommendation applied to retention in grade. Recommendation C, Number 8 suggested redesigning the grade leveling system

PAGE 31

20 lessening importance on age restrictions for placing and promoting students, instead it emphasized that academic progress and students needs should serve more to guide promotion and graduation decisions (National Commission of Excellence in Education, 1983). Politically, support for retention appears to be bipartisan. By the late 1990s, many lawmakers took note of the commissions advice and scrutinized students academic progress. In his 1998 State of the Union Address, then-President Clinton joined a host of other political leaders, from the Democratic mayor of Chicago to the Republican governor of Texas, all calling for an end to the promotion of students whose achievement does not meet the expectations for that grade (National Research Council, 1999, p. 41). We note here that when President Clinton publicly denounced social promotion again in his 1999 State of the Union Address, he also praised the efforts of Chicago Public Schools for raising student achievement when it ended social promotion (Clinton, 1999). Although five years later, it remains to be seen whether the Chicago policy has helped the students held back in grade (Nagaoka & Roderick, 2004). The United States Department of Education (hereafter, USDOE) later released the document, Taking Responsibility for Ending Social Promotion: A Guide for Educators and State and Local Leaders (1999), where President Clinton directed states and localities to practice retention as a way to increase student achievement. Here, he outlined the steps and rationale for ending social promotion, or the practice of simply moving children from grade-to-grade with their age cohort. To accomplish this goal, President Clinton described a plan that included rigorous standards aligned to the

PAGE 32

21 curriculum, reduced class sizes, well-prepared teachers, and extra support for students through after-school or summer programs (U.S. Department of Education, 1999). National policy spotlights how children learn to read, the best pedagogy for teachers, and measuring reading growth as children learn how to read. Policymakers have been led to believe that there is one way, or a best way, for teaching students how to read as evidenced by a review of research that validates only experimental or quasi-experimental research (Allington, 2002b). As an example, the Congressionally requested report of the National Reading Panel (National Institute of Child Health and Human Development, 2000) focused their review of existing research using a specific set of methodological guidelines. In conducting their analyses, they concentrated on five components of literacy, including 1) phonemic awareness, 2) phonics, 3) fluency, 4) vocabulary, and 5) comprehension, using studies that met stringent guidelines. The emphasis on systematic and explicit scientifically-based reading research has some policymakers believing that only particular instructional methods work best for teaching children to read. While the National Reading Panel (NRP) provided many interesting and worthwhile recommendations, the report restricted review of other components known to be essential to teaching children to read such as writing instruction and independent reading. This review of research was central to recommendations set forth in the reauthorization of the Elementary and Secondary Education Act, the No Child Left Behind Act of 2001 (Bush, 2001). The resounding theme of every child is clearly seen in the No Child Left Behind (NCLB) framework (Bush, 2001), the most recent wide-sweeping educational reform. Four pillars, as described on the NCLB website, are the cornerstones of this reform,

PAGE 33

22 including 1) stronger accountability, 2) more local freedom, 3) proven methods, and 4) choices for parents (U.S. Department of Education, 2002). Each of these, when joined together, stimulate change as efforts to reform the American educational system take place. Central to implementing NCLB is understanding the funding structure of this mandate. The two main sources of federal funding are Title I and Reading First grants. Title I funds were always designated for use in schools with higher levels of impoverished students; however, NCLB also provided a host of new stringent guidelines regulating how these funds may be spent. For our purposes here, it is fundamental to know that these funds must be spent upon scientifically-based methods and strategies, as defined by NCLB (and the NRP). In fiscal year 2004, the federal government appropriated $18.5 billion dollars for Title I. With the advent of NCLB came another federal grant called Reading First. As the name implies, this grant is geared specifically at boosting literacy achievement for students in the primary grades. Beginning with fiscal year 2002, $900 million dollars were appropriated for this grant with similar funds available for the next five fiscal years (U.S. Department of Education, n.d.). Florida was one of the first states awarded a Reading First grant and as of May 2005, Floridas Department of Education has received an additional $100 million of federal support for districts (Florida Department of Education, 2005b). However, to receive monies districts (or consortiums of districts) must win a sub-grant from the states Reading First funding. To obtain these highly competitive grants, applicants must document how the monies will be spent while adhering to the State grant approved by the federal government.

PAGE 34

23 In Florida, Reading First grants are reviewed by the FLDOE, as well as faculty affiliated the Florida Center for Reading Research (FCRR) at Florida State University (Florida Department of Education, 2005b). Eligibility for Reading First funds differ from Title I. Title I focuses on improving the educational achievement for students from high poverty backgrounds; Reading First focuses efforts on educational achievement for students who struggle with learning how to read in a way that prohibits reaching accountability. Reading First schools must come from districts with 15% or more of enrolled students from high-poverty backgrounds, and schools must have 10% or more of economically disadvantaged students, as opposed to the 40% level of economic disadvantage to receive Title I. As of the 2004-05 school year, Florida has over 400 Reading First schools. Link to Accountability Accountability has been part of policy discussion for years, however the signing of the NCLB Act of 2001 (Bush, 2001) pushed it into prominence. One key element of this reform is a rigid accountability requirementschools and districts across the nation must meet what is called Adequate Yearly Progress (AYP) by 2014. Federal policy provides a single definition of adequate yearly progress, the amount by which schools must increase their test scores to avoid some sort of sanctionan issue that in the past has been decided jointly by states and the federal government. And the federal government has set a single target date by which all students must exceed a state-defined proficiency levelan issue that in the past has been left almost entirely to states and localities (Elmore, 2002, p. 31). State departments of education have the enormous task of developing accountability plans to comply with the federal mandate (Erpenbach, Forte-Fast, & Potts, 2003). Hence, the era of accountability is upon us and there is much

PAGE 35

24 information to disseminate from federal, state and district policy initiativesthis concerns every elementary school principal across our nation. Accountability legislation can include both rewards and sanctions (Massell, 2001). NCLB provides serious sanctions for schools and districts not meeting AYP that increase in severity with each year a school or district does not meet the standard. For some, the final penalty will result in a full restructuring, meaning that school personnel may be fired, schools could be converted into a charter school, or even turned over to a private enterprise. NCLB holds the system responsible for educating children. However, in the case of retention in grade as a measure of success, the onus of accountability falls on the student (Massell, 2001). Similarly, McGill-Franzen (1987) asserted in her case about students placement in compensatory and special education programs, that we place the burden of the problem on the student (p. 488). I suggest the same is true of retention. Many problems that students face are out of their control and as a result, they suffer. Although the system pays, in a sense when accountability is publicized, it is the student who may carry the life-long consequence of failing a grade. Standardized Assessments In this section, we see how standardized assessments are inextricably tied to both reform and accountability. To add to the context of my study, the reading portion of Floridas standardized test, the FCAT, will be the focus of this section. The FCAT was first administered in spring 1999 as part of the states response to national education reform. In 1999, Governor Jeb Bush brought high-stakes to Florida using the criterion-referenced portion of the FCAT as part of the state reform plan, the A+ Plan for Education, to boost student achievement. This plan assigned letter grades to successful schools providing monetary rewards, while sanctioning low performing schools (Dorn,

PAGE 36

25 2004). In 2003, Florida attached another high-stake consequenceretention in gradefor third graders failing the FCAT-SSS, or FCAT. The FCAT is given in grades 3 to 10, with two parts making up the reading portion of the FCAT (Florida Department of Education, 2003c). One part is criterion-referenced, assessing the state mandated standards-based curriculum, the Sunshine State Standards (SSS). Officially called the FCAT-SSS, most often it is simply called the FCAT. Scores are reported as Achievement Levels (Level 1 (low) to Level 5 (high)), and Developmental Scale Scores (DSS), which accumulate over time beginning in third grade. According to Florida state law, students who score Level 1 fail the assessment. Interestingly, the FCAT-SSS is said to be one of the most challenging state assessments currently in use, especially in terms of passage length and text difficulty of the passages (Hiebert, 2002). The other portion of the FCAT is norm-referenced (FCAT-NRT), comparing Florida students to a national sample. Floridas norm-referenced test is a modified version of the Standard Achievement Test (SAT). Classroom teachers typically administer both parts of the FCAT with scoring conducted by contractors hired by the state. Floridas assessments are secure, meaning the tests are highly protected before, during and after administration (Florida Department of Education, 2000b). It is widely endorsed that one assessment should not be the sole determinant in a high-stakes decision, such as retention in grade (American Educational Research Association, 2000; International Reading Association, 1999). Although Florida, too, claims to support this, children receive mandatory retention upon achieving Level 1 on the FCAT (Warford & Openshaw, 2004b). Technically, the FCAT is not the sole

PAGE 37

26 determinant for retention, however other options would not be considered if it were not for a student failing the FCAT. It is the reverse of the adageinnocent until proven guilty. Instead, Florida third graders are guilty of failure, unless they qualify for a good cause exemption, described in more detail later. Researchers have been studying retention for decades using standardized assessment data. With many retention decisions tied to high-stakes assessments, it is important to consider the legitimacy of these decisions and the relationship to education placements. It has been argued that some placement practices, such as retention and inclusion in special education programs, are the result of standardized assessments and the stakes attached to them (McGill-Franzen & Allington, 1993). After studying low-achieving children in multiple contexts, concern surfaced that some schools retained students before third grade to delay accountability penalties by classifying more students in special education programs. These efforts appeared to increase the reported achievements on a high-stakes assessment in these schools (McGill-Franzen & Allington, 1993). Lastly, the State of Florida has relied on core reading programs as the panacea to help students read more proficiently, with a goal of increased performance on FCAT to meet accountability measures. In 2002, the State Textbook Adoption Committee found six core reading programs meeting the adoption criteria, meaning these programs are research-based and aligned to the states reading curricula (Florida Department of Education, 2003b). In a study of third grade core reading programs used in Florida, two programs were selected for a content analysis focusing on the instruction provided to students (McGill-Franzen, Zmach, Solic, & Love Zeig, in press). Using school-level

PAGE 38

27 data, these researchers worked to understand the relation between the percentage of third-grade students scoring Level 1 on the FCAT to the core program used, determining if schools using a particular program had an academic advantage. Using each schools percentage of students eligible to receive free or reduced price lunch as a proxy for poverty, McGill-Franzen et al. (in press) found that school level of poverty was a better predictor for academic achievement than the core program used. High-stakes standardized assessments have many unintended consequences, particularly for those most at risk for retentionstudents attending schools with higher rates of poverty. High-Stakes Retention Policy Built on the idea of setting high expectations for all students, standards-based reform resulted in standardization of the curriculum. It seems inherently logical that we want to test this knowledge to determine the effectiveness of instruction on students learning as measured by standardized evaluations. However, assessments became high-stakes after attaching consequences and rewards. When then-President Clinton publicly called for the end of social promotion in his 1999 State of the Union address, it was clear that federal policy makers felt a decisive measure was needed to establish accountability (National Research Council, 1999). Thus, the emphasis on standardized assessments given in schools across the nation changed to hold students and schools accountable to achieve certain academic standards at specific points in a school career. Assessments moved from lowto high-stakes as they became used for retention and promotion decisions, or in grading schools. Retention policies based on students performance on high-stakes assessments are becoming more common.

PAGE 39

28 Policy Interpretation and Implementation The last stage of the high-stakes framework described here involves how policy is interpreted and implemented. Policy logic, as described by Allington (2001), proceeds on the assumption that implementing particular policies will have some intended effect (p. 275)there are two essential facets to this logic. The first presumes that intended policy will shape (or reshape) instruction. The subsequent presumption is that this reshaping will generate the desired results via changesometimes in instructional practices (Allington, 2001). Those who study educational change know the complexities of this process (Allen, Cary, & Delgado, 1995; Flinders & Thornton, 1997; Fullan, 2001; Tye, 2000). Principals who handle change the best are more successful in implementing new programs and policies (Fullan, 2002). Vast amounts of research have occurred on the interpretation and implementation of educational policy with teachers. Thinking about the school-level context, research has shown that the policies introduced by state lawmakers often have few similarities to actual practice (McGill-Franzen, 2000). Researchers have encountered teachers who were attempting to implement policy, but in actuality were creating their own policies based upon the intended legislation (Spillane & Jennings, 1997, cited in McGill-Franzen, 2000, p. 901). One would suspect that the same is true of school administrators. As policies are transformed from legislative intent to practice, there can be some loss in the translation. What actually is delivered or provided under the aegis of a policy depends finally on the individual at the end of the line (McLaughlin, 1987, p. 174). If we characterize the typical elementary school teacher as the end of the line, then the school principal acts as the intermediary between the district and the teacher. This junction

PAGE 40

29 serves an important purpose in understanding what ultimately happens at the end of the line. Resistance to policy is one reason for a policys demise, when in fact, for many, it is a lack of understanding on the part of those charged with the implementation (Darling-Hammond, 1990). Policy analysts call for awareness on how policy is enacted locally. This insight can help guide the successful implementation of the said policy. The institutionalization of a policy depends on this understanding. Other problems in policy implementation exist. Lack of a core understanding is illustrated in the case study example of a teacher named Carol Turner, who in her mind, perceived that she was successfully implementing the new California Mathematics Curriculum Framework (Ball, 1990). With a lack of understanding of the new mathematics framework and a new textbook in hand, Carol was under the misguided impression that following the new textbook constituted enacting the new curriculum. Although it is unclear how a textbook can be the impetus for change, nonetheless in this case, it was viewed as the messenger of change (Ball, 1990, p. 257). The notion of a messenger is vital. Much of the literature overlooks the role of the principal, who can be seen more in the role of policy messenger, than anything else. Nevertheless, even as a messenger, who may be deemed powerless by an instructional staff, the principal is the gatekeeper of knowledge through which the details of the policy permeate to the classroom teacher (Ball, 1990). As the instructional leader of a school, how a teacher learns the essentials of a policy is the responsibility of the principal or their designee. Valencia and Wixson (2004) explain a model illustrating a multi-level system through which policies are shaped as they move toward the core of educationthe

PAGE 41

30 classroom. As policy trickles through the system, there are many possible outcomes. Depending upon factors such as opinion or perception of a policy, there can be positive or negative conclusions to a particular policy (Valencia & Wixson, 2004). We know implementation of policy is not simpleit is a complexly interactive process without beginning or end (Lindblom & Woodhouse, 1993, p. 11). It takes direction to achieve the desired results. With that said, past research has found that when it comes to policy implementation there are many school principals who feel less than adequate when enacting new policy (Musella, 1989). Senior administrators in a large school district requested Musella observe an in-service session as principals learned about a new governmental policy. As Musella witnessed this training, he noticed concerns regarding ambiguity of the policy goals, nonspecific policy guidelines, frustration about the lack of input in the policymaking process, inadequate funding resources, and insufficient time to enact changes. In an effort to share his observations, without pointing out the obvious faults in their process, Musella shared research on typical reasons for opposition to change (Zander, 1962, cited in Musella, 1989). A strikingly obvious, but often overlooked reason for policy failure is that those responsible for changing are not involved in the planning (Musella, 1989, p. 95). The principals observed felt powerless; they were not part of the change process; however, they were expected to enact a major governmental policy. How administrators implement and understand new policy is important (Allington, McGill-Franzen, & Schick, 1997). Simply because a policy exists, does not ensure that it is acted upon, or adequately understood. Allington et al. (1997) described many important characteristics in their qualitative study of administrators understanding of

PAGE 42

31 learning disabilities. Although many of the administrators interviewed supported state and federal programs, one essential finding of this study was that administrators perceived creating and funding almost any intervention was someone elses responsibility (Allington et al., 1997, p. 231). These findings are fundamental to the third-grade retention policy in Florida. How principals implement the retention policy may rest more upon their beliefs, as policy implementers do not always do as told (McLaughlin, 1987, p. 172). Florida Policy Context Although the federal government endorses the use of retention in grade, this is a policy matter for local and state policymakers, such as a state department of education or a school district. The Florida policy mentioned thus far in my study was not the first attempt to enact a statewide retention policy. In 1999, the Florida Legislature did not pass a law for retention decisions based solely on a students FCAT score for fourth-grade students (Florida Department of Education, 2003a). Instead, student performance throughout the school year was deemed appropriate to make promotion and retention decisions, not simply a score from the FCAT. At that time, the Legislature decided that retention decisions should be left to the discretion of individual school districts. In August 1999, in lieu of a formal policy, the FLDOE recommended three options to school districts for students who were not meeting their districts progression plan. Options included to, (1) remediate students before the beginning of the next school year and promote, (2) promote and remediate in the following year with intensive remediation, and (3) retain and remediate (Florida Department of Education, 2003a, p. 4). Even after making these recommendations to districts, the State tried again to enact a retention policy and this time met success.

PAGE 43

32 Policy Enactment The 2002 Legislature passed a rewrite of the Florida School Code (F.S. 1008.25) instituting a mandatory retention policy into effect on January 7, 2003, immediately ending social promotion in Florida schools (Florida Statutes, 2002). Of interest to this study is the mandatory third-grade retention policy enacted during the 2002-03 school year. The new law set the Grade 3 reading FCAT as the critical gateway to identify students who, after remediation, are still unable to demonstrate reading proficiency and clearly need more time to learn the basic skill of reading (Florida Department of Education, 2003a, p. 2). Here is what third-grade students encounter: The law asserts that third-grade students, who do not achieve Level 1 on the reading portion of the FCAT, fail. Students performing at Level 1, as defined by the State, are said to have experienced little success with the standards-based curriculum, the Sunshine State Standards (Florida Department of Education, 2000a). This policy focuses on identifying students who need a stronger literacy foundation regardless of the reason that is causing iteven a learning disability, limited English proficiency, or a disadvantaged backgroundneeds to be addressed and corrected before the student can be expected to move successfully on to the more difficult work of the higher grades (Tremor & Butler, 2004, p.1). The Legislation has mandated school districts to allocate remedial and supplemental instruction resources to students in the following priority: (1) firststudents who are deficient in reading at the end of grade 3, and (2) nextstudents who fail to meet performance levels required for promotion (Florida Department of Education, 2000a, p. 2).

PAGE 44

33 Retention loopholes As mentioned earlier, loopholes (or narrow exceptions) written into this law, called the good cause exemptions, allow some children to forego retention (Florida Department of Education, 2004d). Currently, six good cause exemptions allow promotion to fourth grade, if documented. Third-grade students meeting one of the following good cause options are eligible for promotion to fourth grade, even after scoring Level 1 on the FCAT 1. Limited English proficient (LEP) students having two years or less of English instruction 2. Students with disabilities (SWD) not participating in the state assessment because their individualized education plan (IEP) or Section 504 plan indicates that FCAT is not an appropriate assessment 3. Students scoring at the appropriate level on an alternative assessment 4. Students meeting district determined criteria to score the equivalent to Level 2 on FCAT via a good cause portfolio 5. Students with disabilities (SWD) previously retained in grade receiving two or more years of remediation 6. Students who have been retained two years in any grade, kindergarten to grade three, and have received two or more years of remediation (Florida Department of Education, 2004d) Here is further explanation for Options 3 through 6. Option 3, the FLDOE recommends districts use the Stanford Achievement Test (SAT) as the alternative assessment for students who fail the FCAT-SSS. Because of the challenging nature of the FCAT, this is one of the reasons why, in third-grade, a student must achieve at the 51 st percentile on the nationally normed SAT9 test of reading in order to insure that they have sufficient reading ability to achieved above Level 1 on the FCAT (Torgesen, 2004, p. 2). On May 17, 2005, the FLDOE provided a new cut score for the FCAT-NRT, now

PAGE 45

34 version 10 of the SAT, to at or above the 45 th percentile as announced in a memorandum for District School Superintendents (Warford & Openshaw, 2005). Districts electing to use the SAT-10 as their alternative assessment will adhere to the 45 th percentile or above, while districts still using the SAT-9, described by Torgesen (2004), must continue using the 51 st percentile or above as the criteria. Depending on the version, SAT-9 or SAT-10, students must meet the designated cut-point to be eligible for a promotion with good cause. 1 Option 4 allows a good cause portfolio designed by school districts. Each district had the opportunity to create a portfolio for classroom teachers to compile on behalf of students identified at risk to fail the FCAT. The FLDOE notes that only teachers may initiate and compile these portfolios. The state recommended four districts, Citrus, Clay, Orange and Pasco, as examples for other districts to use as models to establish their portfolio criteria (Florida Department of Education, 2005). Good cause portfolios are designed to be an on-going, collection documenting students classroom performance equivalent to Level 2 or higher on the FCAT. If a student fails the FCAT, the teacher submits a portfolio to the school principal to determine if the portfolio meets the criteria to promote the student for good cause. 2 If the school principal determines that the students should be promoted, the principal must recommend it in writing to the district superintendent. The district superintendent must accept or reject the school principals recommendation in writing (Florida Department of Education, 2004a, p. 8). 1 It is unclear how the FLDOE determined the new cut-point for the SAT-10. Based on information provided by the test publisher, Harcourt Assessment, the SAT-10 cut-point mandated by the FLDOE is lower than the equivalent calculated by the publisher. Harcourt equates a 51 st percentile rank on the SAT-9 to a 46 th percentile rank for the total reading score, not the 45 th percentile. 2 For more about good cause portfolios, please visit www.firn.edu/doe/commhome/progress/proghome.htm

PAGE 46

35 The latter two good cause promotion exemptions are complex with numerous scenarios by which a student could be promoted. Complexities arise when understanding the options available to those students previously retained in K to 3. Option 5 relates to students with disabilities (SWD) retained in any grade, kindergarten to grade three. For these students, they may be promoted to fourth-grade after being retained one time; however, they must have received at least two years of reading remediation. Hypothetically, a SWD who was retained in first grade and received extra remediation during the year of retention and in second grade could be promoted to fourth grade after failing the third-grade FCAT. Another hypothetical student may have been identified in second grade as a struggling reader. However, this SWD student received remediation in second grade and was promoted to third grade. After a second year of remediation, this student would not be eligible for promotion because the child was never retained. After a second year in third grade, this child would be eligible for a promotion with good cause, if they still do not pass the third-grade FCAT. The final good cause exemption, option 6, allows promotion for good cause to any students who have received the intensive remediation in reading for two or more years, but still demonstrate a deficiency in reading and who were previously retained in K-3 for a total of two years (Florida Department of Education, 2004a, p. 7). This good cause promotion set a limit on the number of times a student may be retained in K to 3, provided they received the appropriate remediation (Warford & Openshaw, 2004b), although Florida does not currently limit the number of retentions a student may experience in elementary school. As can be seen, good cause exemptions are stringent

PAGE 47

36 and most complex. In a sense, upon failing the FCAT, students promotional decisions are treated on a case-by-case basis. Lastly, state law requires districts to report the specific type of Good Cause Exemption used to promote a student each year, beginning with the 2002-03 retentions (Warford, 2004b). Using data provided by the FLDOE, Table 2-1 disaggregates the number of Florida students promoted with good cause after the first and second years of the policy. Table 2-1. Florida retention policy good cause statewide exemptions 2002-03 2003-04 # % # % 1) Limited English proficient students with fewer than 2 years in English as a second language program 2,974 22.80 2,511 11.79 2) Students with disabilities (SWD) not participating in statewide assessment as per individualized education plan (IEP) 1,016 7.79 1,647 7.73 3) Students who demonstrate proficiency on an alternate assessment 3,307 25.35 3,845 18.05 4) Students who demonstrate proficiency through a portfolio 1,514 11.61 3,468 16.28 5) Students with disabilities (SWD) retained once with 2 or more years of remediation 3,637 27.88 7,906 37.12 6) Students retained twice with 2 or more years of remediation 598 4.58 1,924 9.03 TOTAL PROMOTED WITH GOOD CAUSE 13,046 21,301 Note. Data provided courtesy of Florida Department of Education (L. Fleming, personal communication, June 6, 2005). Let us compare the numbers of students scoring Level 1 on the FCAT to the numbers of students promoted for good cause to understand the impact of this policy. Over 43,000 students scored Level 1 on the FCAT in 2002-03, and the FLDOE reported,

PAGE 48

37 as shown in Table 2-1, that after the 2002-03 school year over 13,000 students were promoted with good cause to fourth grade. Based on this information, it would appear that nearly 30,000 students were retained after the first year of policy implementation. However, according to data also collected by the FLDOE, state files document that over 27,000 third-grade students were retained after policy implementation and after application of good cause exemptions and we will rely upon this number. 3 Policy Guidelines Much of the retention policy echoes the mandates set forth in NCLB. Identifying students who struggle with reading and evidencing their reading achievement are central to both national and statewide reform efforts. However, like retention, determining the best technique to help students who struggle with reading has a long and contentious past (Allington & McGill-Franzen, 2000). Rather than debate here which techniques are best, I present the mandates and suggestions endorsed by the FLDOE in the 2004 Legislative Intent and other technical reports disseminated to Florida schools via the state paperless communication system (Florida Department of Education, 2004a; Tremor & Butler, 2004; Warford & Openshaw, 2004a, 2004b, 2004c, 2005). The Legislative Intent (hereafter, Intent) for the state third grade policy (F.S. 1008.25, Section 6) has three main areas of focus, including: 1) determining reading proficiency, 2) parental notification, and 3) developing comprehensive plans to address the Intent goal for all students to read at or above grade level. Two new initiatives were also announced by FLDOE officials in the 2004 Intent (Florida Department of Education, 3 School districts reported data to the State during the same collection period in August 2003. The reason for these discrepant numbers is unclear; however, for the purposes of my study we will rely on the numbers disaggregated by school (i.e., approximately 27,000 third grade students retained in 2002-03), not the total extrapolated by the researcher, even though these should have been nearly the same. One FLDOE official suggested the amount may be discrepant due to students moving. Although this suggestion would only apply to state numbers if students were moving out-of-state, not within the state.

PAGE 49

38 2004a; Warford & Openshaw, 2004a). One, called Reading Enhancement and Acceleration Development (READ) Initiative, is a retention prevention program aimed at students in grades kindergarten to three, including newly retained third graders. Multiple lines of legislation outline several steps and sub-steps that comprise this initiative (s. 1008.25(7) (b) 7, F.S.; Bill page 17, line 1 page 19, line 23) (Florida Statutes, 2002). Another initiative calls for establishing classrooms for retained third grade students who subsequently score at Level 1 on the reading portion of FCAT (F.S. 1008.25(7) (b) 8; Bill page 18, lines 1 23) (Florida Statutes, 2002). These classrooms, called an Intensive Acceleration Class, have specifications outlined in the legislation. A brief overview of how at risk students are identified, how parents are notified and how students are supported follows. 4 Identification First, all students in Kindergarten through Grade Three (hereafter, K to 3) have their reading proficiency determined and monitored. The FLDOE is focused on identifying students in K to 3 who lack the required proficiency to pass the third grade FCAT, defining these students as exhibiting a substantial deficiency in reading and must be given intensive reading instruction following the identification. To make diagnoses, elementary schools are required to assess their students regularly to identify the exact nature of the students difficulty in learning to read. Reading First school students must be assessed using the statewide assessment system, the Dynamic Indicators of Basic Early Literacy (DIBELS; Good & Kaminski, 2002). Non-Reading First schools must also identify their students, however in addition to the DIBELS, these schools may 4 To provide the reader the opportunity to hear the tone of the 2004 Legislative Intent, specific terms or wording used in the Legislative Intent were included here using quotation marks.

PAGE 50

39 select from other assessments. Newly identified students are then subject to a remediation plan that must be provided during regular school hours. All of this can be likened to a critically-ill person being treated as an out-patient. Notification Upon labeling a student as reading deficient, parents must be notified as soon as possible with a description and explanation, in terms understandable to the parent. It is expected that parents be consulted in developing an academic improvement plan (AIP) and that parents are provided a list of the supplemental instructional services and supports to be provided until the deficiency is corrected. Schools are to keep parents apprized of their childs progress. It is also required that parents be provided strategies to help their child with reading. At the end of third grade, parents are also alerted that if the services and supports do not correct the deficiency, as demonstrated by scoring at Level 2 or higher on the statewide assessment test in reading for grade 3, then the student must be retained, unless they qualify for promotion with good cause. Retention plus remediation In addition to diagnosing students and sharing this news with parents, the Intent specifies how to remediate students. Schools and districts must follow the regulations of the READ Initiative for those students who qualify. This Initiative aims to prevent retention for students in K to 3, and for the newly retained third graders, it offers an accelerated, or intensive approach, to help prepare students for promotion to fourth grade. Each student must receive extra support during the regular school day, which is in addition to the mandatory 90-minutes of daily, uninterrupted reading instruction utilizing a scientifically-based program. These are the state approved and Florida Center for Reading Research (FCRR) reviewed core reading programs focusing on the five areas of

PAGE 51

40 reading identified by the NRP: 1) phonemic awareness, 2) phonics, 3) fluency, 4) vocabulary, and 5) reading comprehension (National Reading Panel, 1999). Ongoing progress monitoring must also be provided. All core subject areas, such as science, mathematics, social studies, are to be incorporated into the school day, in addition to the above. Double retention plus remediation In 2004-05 a new initiative began for students who were already retained in third grade, but did not pass the FCAT again. Each district must establish an Intensive Acceleration Class and qualified students must participate. The goal of this program is to accelerate learning for students to gain two years worth of material in one year. Theoretically, the FLDOE suggests that successfully remediated students could advance to fifth grade and by-pass fourth grade. This child could potentially rejoin his or her same age cohort. The Intent does not suggest that students would advance to the fourth grade via this program. To accomplish this extraordinary feat, the Intent explicitly describes several rules for districts (Florida Department of Education, 2004a, pp. 11). It recommends reduced teacher-student ratios, however ratios are not defined. The FLDOE notes that class sizes are expected to be smaller than other third-grade classes. Teacher-student contact time is expected for most of the day. In other content area subjects, such as Mathematics or Science, students are expected to be taught from the fourth-grade Sunshine State Standards. As well, the fourth-grade Language Arts strand is used. This provision helps ensure students will have the necessary background knowledge to cope if promoted directly to fifth grade. Students in an Intensive Acceleration class also have services of a speech-language pathologist, if needed.

PAGE 52

41 Students are to be assessed weekly. Districts must monitor all students taking part in this Initiative. Central supports Several supports are embedded within the Intent for all identified students with reading difficulties. State law requires districts to offer certain supports, while others are suggested, but not mandated. Six supports (Table 2-2), central to my research, and their purpose within the Florida retention policy context, are described here (Florida Department of Education, 2004a). Table 2-2. Supports identified in Floridas legislative intent Required Recommended Focus on early intervention Extended learning opportunities Current academic improvement plan (AIP) Transitional class available for retained students On-going portfolios meeting state requirements Mentor or tutor with specialized training Focus on early intervention. Students who are identified as early as kindergarten are provided extra supports. As part of the establishment of the READ initiative described earlier, third grade students are provided an accelerated curriculum and students in grades K to 2 are provided intervention to foster their reading development. The central tenet here is to help prevent third-grade retention. No laws exist to retain children before third grade. Prevention can benefit students experiencing difficulty (Allington, 2002a; Allington & Walmsley, 1995). Current academic improvement plan (AIP). An Academic Improvement Plan (AIP) formally documents the remedial strategies provided to the student over a period. The FLDOE requires schools to collaborate with parents when making or revising an

PAGE 53

42 AIP. The AIP must name which of the five areas of reading identified by the NRP (1999) is lacking. On-going portfolios meeting state requirements. All retained third graders must have an active, ongoing portfolio that may be used as part of the good cause promotion options. As mentioned earlier, these portfolios are district-created, so there is expected variation across districts. Essentially, these teacher initiated and compiled good cause portfolios must contain evidence that accurately reflects whether the student can achieve comparable to Level 2 on the third-grade FCAT. Both the school principal and district superintendent will review and review these portfolios. Extended learning opportunities. All students who have trouble with reading must be provided extra intensive support beyond the 90-minute reading block. The FLDOE advocates the use of extended learning outside the regular school day, although by definition, this would be in addition to the extra support that must be provided during the regular day beyond the 90-minutes of instruction. Extended learning may occur beforeor after-school, or on the weekend, such as a Saturday School. Another option includes the use of an extended school year beyond the minimum 180-days. Extended learning, as suggested by the FLDOE, is different from summer school. All students who fail the third grade FCAT must be provided the opportunity to attend summer school, or as they are called in Florida, Summer Reading Camp. Transitional class available for retained students. The Intent mandates that retained students must be offered the choice of a transitional type setting with the purpose of producing learning gains. The goal of such classes, as noted by the FLDOE, is what is being provided to help the student catch up, not where it is being provided (Florida

PAGE 54

43 Department of Education, 2004a, p. 13) [emphasis in original]. Configurations may vary, and schools may choose transitional classes that contain thirdand fourth-grade students or re-retained third graders only. Districts may elect to offer the transitional classroom at a central location. This type of class is akin to transitional classes preparing students for first grade (e.g., pre-first grade, junior first grade, readiness room) (Shepard, 1989). Mentor or tutor with specialized training. All parents of retained students must be provided with either supplemental tutoring in scientifically research-based reading services in addition to the reading block (Florida Department of Education, 2004a, p. 10). This tutoring may happen before and/or after school. Alternatively, students may be provided with a mentor or tutor with specialized reading training, as opposed to using a scientifically research-based program. 5 The FLDOE provided many alternatives for districts to select with their allocated funds. Other options recommended in the Intent include: reduction in teacher-pupil ratio, although the FLDOE does not endorse a capacity or class size. The Intent also recommends more frequent progress monitoring, which may include the assessments found within the scientifically research-based programs. A frequency is not prescribed. Implementing the policy Policymakers acknowledge the complexities of the process (Hart, 1996). To cope with the policy fluctuations, local school districts are provided with communications and meetings as a means to guide changes and aid understanding in a timely manner. They serve as a way to distribute the recipe with state policymakers telling districts what needs 5 In September 2004, the FLDOE surveyed districts to learn which of the three options they selected to provide parents of retained students. Districts are required to provide at least one of the following: 1) tutoring with a research-based program, 2) a tutor or mentor trained in reading, or 3) a Read at Home contract.

PAGE 55

44 attention and how to do it. To assist struggling readers, Florida mandates schools and districts use special classes, use certain types of curriculum materials, as well as provide summer school and make certain services available. Timely explanations are needed to assist policy implementers to understand mandates as this law continues to evolve. The FLDOE is committed to providing updates and technical assistance papers concerning this policy via an open-access paperless communication system. Although many documents are intended for school district superintendents or other district leaders, interested members of the public are free to register for an e-mail service that delivers documents upon release. As of June 2005, twenty-four notices dating back to August 26, 2002 were posted on the FLDOE website regarding this policy, plus related attachments (Florida Department of Education, 2005c), although many other documents have been distributed since the last entry on January 11, 2005. Recent documents are presumably available elsewhere on the FLDOE website. Following the first year of policy implementation (2003-04), the FLDOE sent an electronic memorandum to Floridas district superintendents outlining amendments of two Florida state educational laws (F.S. 1002.20 and 1008.25) (Warford & Openshaw, 2004a). As part of this paperless communication system, a carbon-copy (cc:) of this memorandum was also sent electronically to the Assistant Superintendents for Curriculum and Instruction, Directors of Student Services, Directors of Elementary Education, Directors of Exceptional Student Education, Elementary School Principals, and Elementary Guidance Supervisors of each district. Revisions to 2004-05 student progression plans require changes, many of which relate or interrelate to the third-grade retention policy. Essentially, the memorandum provides a lengthy list of requirements

PAGE 56

45 for each elementary school with a copy of the bill attached. Specifically, the FLDOE requires each school to assess the reading ability of each K-3 student, provide parents with notification of any reading deficiency, implement a detailed academic improvement plan (AIP), and provide intensive reading instruction; revise the required notice to parent of third grade students with substantial reading deficiencies to include information about additional evaluations, portfolio review and assessment to determine whether the student is ready for promotion, and information on the districts specific criteria and policies for mid-year promotion; define mid-year promotion; make a technical correction related to students who were previously retained in grade three; for third graders who are retained, require appropriate intensive interventions, including the provision of summer reading camps; specify the activities and supports to be provided to retained third graders, including the use of a state-identified reading curriculum that meets certain specifications; provide for intensive acceleration for students currently retained who score Level 1 in reading; require a report to the State Board of Education on the interventions provided; and require the option of placement in a transitional setting for retained third graders (Warford & Openshaw, 2004a, p. 1). After school districts had opportunity to review the wide-ranging edicts from the memorandum outlining the legislative changes, the FLDOE issued another memorandum with a chart clarifying the old and new legislative intents (Warford & Openshaw, 2004b). This chart, covering 14 pages, provides comparison of the original intent to the current modifications (Florida Department of Education, 2004a; Warford & Openshaw, 2004b). This serves as evidence that the FLDOE does provide support to districts in their efforts to decipher and enact changes in a timely manner. Even with these technical assistance reports, questions and confusion exist as implementing this policy continues. In October 2004, members of the FLDOE Just Read! Florida 6 office traveled to multiple sites across the state holding meetings, offering assistance to principals and district-level administrators. I had the opportunity to attend 6 Members of the Florida Department of Education Just Read! Florida office participated in writing the Legislative Intent for the retention policy. This office oversees the technical assistance of this policy. Also, these October meetings were originally slated for September 2004, but were rescheduled due to hurricanes.

PAGE 57

46 the October 7 th meeting held in Tampa, Florida. The purpose of the meeting was for attendees to receive clarifications to their implementation questions, and for attendees to provide input prior to the impending passage of the mid-year promotion rule. Several matters were discussed at length; however, I will provide just two examples. First, the FLDOE, as described earlier, introduced two new initiatives, READ and Intensive Acceleration, for the 2004-05 school year. When these initiatives were introduced and as they were written in the Legislative Intent, they were referred to by these titles. However, the FLDOE released another document that described these initiatives as a tiered system. At the meeting, members of the FLDOE used the titles interchangeably, causing confusion. After explaining that Tier 1 (identified as at risk for retention) and Tier 2 (retained once) students participate in the READ Initiative, while Tier 3 (retained two times in the same grade) students participate in Intensive Acceleration Classes, confusion appeared to ease. Another area of questioning related to the number of times students may be retained. Participants received advice regarding how to handle specific cases for students with multiple retentions explaining how students who received two years of intensive intervention may not be retained in a grade for more than two years. After two years of intensive intervention, students would be eligible for a good cause exemption. The observed scenario was reminiscent of what Musella (1989) described in his work observing an in-service training. Similarly, I heard concerns regarding ambiguity of the interventions and unclear policy guidelines. Summary Themes resonating from the past research on retention indicate that 1) students need to be treated as individuals when making a life-altering retention decision and 2)

PAGE 58

47 students who are at risk for retention or retained need adequate support to help them achieve their highest potential. As we can see, retention is deeply rooted within the high-stakes framework presented and examined within this chapter. In Florida, simply because FCAT achievement data appears to point toward success, how has the state managed to raise achievement while drastically increasing the number of students retained in grade? In order gain more insight into this policy, we work to understand how Florida elementary school principals provide support within the given framework.

PAGE 59

CHAPTER 3 METHODS Introduction Historically, retention has been a dichotomous debate with little middle ground; however, in the previous decade, it has gained acceptance with more states using it as a component of a state accountability scheme. Florida is one such state to enact a policy preventing promotion in grade when students do not meet set criteria. The impetus of this policy is to help the state meet the national challenge to have all third-grade students reading on grade-level by 2014. Florida is determined to beat the national goal, calling for districts to have all third-graders reading on level by 2012. This retention policy is relatively new with little known about school-level implementation. The foundation of my study was rooted in a multilevel model considering factors, such as level of poverty in their schools, while determining whether the past retention practices of a principal influenced the level of current support offered to students at risk for retention or currently retained. This multilevel model allows the reader to look broadlydiscerning whether retention practices have changed since the start of this policy. Using 5 years of percent retained data enabled examination of differences in retention rates before and after policy implementation. As recommended by Allington (2001) for the study of policy, data from pre-policy and post-policy provide opportunity for comparison. To better understand the school years discussed in the balance of this study, Table 3-1 displays the school years and their relation to the start of the retention policy. 48

PAGE 60

49 Table 3-1. Time in years: Pre-policy versus post-policy Year School Year Retention Policy 1 1999-2000 2 2000-2001 3 2001-2002 Pre-Policy 4 2002-2003 5 2003-2004 Post-Policy Participants Population To study the Florida third grade retention policy, Florida principals were selected. This State has 67 traditional public school districts encompassing over 1,800 elementary schools. Each school district covers an entire county. Sample For this study, districts were selected both purposefully and randomly for a total of 920 possible public elementary schools. 1 The six districts selected with purpose are among the largest of Floridas school districts, while the other six districts were selected at random from the remaining available districts. This sample included districts from each geographic region of the State the panhandle, the north, the south, landlocked areas, the Gulf coast, and Atlantic coastal regions. Within and across all districts, combinations of urban, suburban and rural schools exist (National Center for Education Statistics, 2002). Prior to submitting proposals to conduct research in these districts, ten of the twelve districts expressed interest in participation, responding positively with a letter of 1 Charter schools were not included in this study.

PAGE 61

50 interest to the researcher. The other two districts welcomed submission of the research request. Of these, all districts willing agreed to partake in the study after independent review by district representatives. Across all districts, I sought formal school district approval to request principals voluntary participation in the Florida Principal Survey. One large district restricted access to a cluster of thirty elementary schools participating in a district research evaluation, reducing the available sample from 920 elementary school principals to a maximum of 890 possible elementary school principals. Confidentiality Protection In the district approvals, it was agreed that I may refer to districts by enrollment group, as opposed to specific district names. Based on their PK to 12 district enrollments gathered using the Common Core of Data Build a Table tool (National Center for Education Statistics, 2004), I classified these 12 districts into three groups: 1) small, 2) mid-size, and 3) large. Districts with fewer than 30,000 students enrolled were classified as small; districts with greater than 30,001 students enrolled were deemed mid-sized. The other six districts, with over 100,000 students enrolled, were classified as large. This classification system may be unique to Florida as its districts are countywide. To protect the identities of the participating districts, specific enrollment figures are not displayed. Also, specific school and principal names will not be revealed, nor released publicly. Each principal was assigned a unique identification code as an additional safeguard to protect the identities of the respondents. Appendix A shows the Florida Principal Survey that participants received.

PAGE 62

51 Data Collection Survey Instrument As this policy stretches across Florida, survey design was selected as a data collection method because it is representative, objective, quantifiable and systematic (Isaac & Michaels, 1995). The survey questionnaire, researcher created, referred to as the Florida Principal Survey contains three distinct sections. Part 1 has four items requesting respondents to identify their name, school and district, as well as the unique identification number provided in their consent letter. Part 2 contains items for principals to share their attitudes and practices related to the retention policy. Most topics for the survey were derived after reviewing the Intent memo the FLDOE sent to schools and districts outlining the legislative changes (Warford & Openshaw, 2004b). For my study, the focus of item construction was the support available for at risk students, including those at risk for retention, those currently retained, or those retained again in third grade. 2 The closed-ended items in this study were fixed-alternatives. One item offered two response options (Figure 3-2) and six items were categorical by design with response choices, yes, no, or not sure (Figures 3-3 and 3-4). However, the not sure option was not selected by this sample. For the purposes of this study, these items were treated as dichotomous, yes, no. Part 3 of the survey requested the principal to enter the years of career experience as an educator, the total years as a principal, and the length of time spent as principal at their current school. A summary of the pilot survey review follows in Appendix B. 2 Other items included in the survey are for a separate report.

PAGE 63

52 Retention belief Item 9 served to identify the retention belief of principals as related to the use of one standardized assessment result to determine a retention decision. As current policy bases the decision on FCAT performance, it was salient to this study to understand principals responses. By design, principals were asked to select between the two options. Principals were provided the response choices shown in figure 3-2. They could click or check the option best matching their current belief. Figure 3-1. Item 9, Florida Principal Survey Directions: Knowing that individual circumstances can exist, what is your response to the following statement? "On the whole, retention in grade can benefit 3rd grade students who score Level 1 on the FCAT" Item Number Response Choices Q9 Yes, retention is a beneficial option for students who fail the FCAT. No, retention is not a beneficial option for students who fail the FCAT. Resources for at risk students The next two groupings of items provided an opportunity for principals to report on the presence of different types of supports or monitoring systems currently used in their schools. As legislation has strongly suggested that schools and districts use particular supports to remediate their students, I wanted to learn which supports were being used by schools. Survey items are presented in figures 3-3 and 3-4. Degree of support Based on the data collected, it would not be possible to make determinations about

PAGE 64

53 Figure 3-2. Item 14, Florida Principal Survey Directions: Think about programs or initiatives currently in place at your school designed to help at-risk students. Item Number Survey Item Q14b Does your school emphasize early intervention programs in K 2 to prevent retention in 3 rd grade more now than in the past? Q14c Do ALL retained 3rd grade students in your school have a current AIP? Q14e Is a transitional class (a class designed for the retained and/or re-retained) available for retained students? Figure 3-3. Item 19, Florida Principal Survey Directions: Which, if any, of the following are used as strategies with retained third grade students? Item Number Survey Item Q19a A mentor or tutor with specialized reading training Q19b Extended school day (such as After School Program, Saturday School, or Extended School Year) as defined in the NCLB legislation Q19c Ongoing portfolios that meet state portfolio requirements the quality of these supports as implemented in each school; however, I decided to examine the number of supports reported as implemented to determine if any patterns exist. A new variable created for the planned analyses lends itself to help explain retention patterns for schools using fewer or more supports, that is the degree of support provided in each school. A categorical variable called suppsys3grp was created based on principals responses to the item numbers in Table 3-2. Schools reporting use of any 1 or 2 supports provided a lower degree of support options; schools using any combination of 3 or 4 supports provided a medium degree of support options; and schools using 5 or 6 supports provided a higher degree of support options.

PAGE 65

54 Table 3-2. Degree of support category, Florida Principal Survey Item number Support type Q14b Focus on Early Intervention Q14c Current Academic Improvement Plan (AIP) Q14e Transitional Class available for retained students Q19a Mentor or Tutor with specialized training Q19b Extended Learning Opportunities Q19c On-going portfolios meeting state requirements Employment history Between these items (Figure 3-4), there was enough information, for the purposes of this study, to determine how long the respondent held a principalship at their current school. The purpose of the length of the principalship at the same school will be described in more detail later. Item Number 28. How many years have you been a principal? (Including 2004-2005 school year) ___________________ Item Number 29. How many years have you been principal at your current school? (Including 2004-2005 school year) ___________________ Figure 3-4. Employment history, Florida Principal Survey Procedures The first phase of the data collection began after the start of the 2004-05 school year with an on-line survey sent electronically to the elementary school principals of the twelve participating districts. Approximately 50% of the districts volunteered to have a district official contact their principals to acknowledge their approval of the study. All participating principals had Internet access.

PAGE 66

55 Principals e-mail addresses were determined using multiple sources. Most districts do not maintain databases to share e-mail addresses outside their school district Intranet. Small districts were willing to provide or check e-mail addresses gathered. Some e-mail addresses were gathered from district and school websites, while hundreds were created using a formulaic method (e.g., a seven character name followed by the district URL or FirstName.LastName@district.k12.fl.us) suggested by district officials. Nearly two hundred schools were telephoned to request the e-mail address of their principal. Surprisingly, the least effective method of obtaining accurate an e-mail address was calling, as many school employees did not readily know this information. Every reasonable effort was made to ensure the survey reached the intended sample. A three-stage follow-up sequence to collect data was planned (Dillman, 2000). Research decisions during the collection period During the data collection period, four hurricanes inundated Florida. Every school district in the state was affected by at least one hurricane resulting in school closures. In the 12 districts from this study, the cumulative number of district closures due to the four hurricanes was evenly distributed by district size with an average of 24 days lost. 3 Two areas of data collection were potentially problematic. First, because of the impact of the hurricanes, a decision was needed whether to continue data collection in these districts. On September 29, 2004, the FLDOE announced that the FCAT would proceed as planned with some provisions for those districts most greatly affected to have more time to administer the assessment (Warford, 2004a). The State Education Commissioner John Winn encouraged districts to restore a sense of normalcy by 3 To protect the identities of the districts, specific numbers are not given.

PAGE 67

56 continuing to focus on student achievement (Florida Department of Education, 2004b, 4). Because of these announcements, data collection proceeded, as there was no hold on the policy under study. The second problematic area was the low response rate from the on-line survey. After 5 weeks of data collection, a 9% response rate was achieved. A decision was reached to offer an alternate paper-pencil survey after learning that districts would permit a paper version of the survey to be sent to the nearly 800 non-responding principals under current district approvals. All non-responding principals received a copy of the paper survey along with a postage-paid envelope to return the survey directly to the researcher. The on-line format continued to be available for those who chose that format. The paper survey offered in October was nearly identical to the on-line format. Questions used the same wording and were asked in the same order. Minor modifications, were made where the on-line survey instructed respondents to click the appropriate response. For example, the paper survey was changed to check the response. Non-responding principals were contacted using a two-stage follow-up (Dillman, 2000). I e-mailed principals two times after the paper surveys were mailed requesting their participation using the survey format of their choice. These revisions resulted in an overall 29% return rate. Of the 255 principals who returned the survey, 137 principals completed the on-line format, while 118 principals completed the paper-pencil format. An additional 2% of the principals declined participation stating lack of time as a reason. Table 3-3 shows the response rates achieved using the district size classification. All returned surveys, including those returned without names, were included in these calculations. Compared to the large

PAGE 68

57 districts, a higher percentage of principals from small and mid-sized districts responded favorably to this survey. Table 3-3. Survey response rates by district size Small N % Responded 18 41.9 Declined 2 4.7 Non-response 23 53.4 Total 43 Mid-Size N % Responded 46 40.0 Declined 4 3.5 Non-response 65 56.5 Total 115 Large N % Responded 191 26.1 Declined 15 2.0 Non-response 526 71.9 Total 732 Usable Sample Procedure While awaiting the close of the survey data collection, several databases of information needed for this study were gathered, including the third grade percent retained calculations and the school-wide percent of students eligible for a free or reduced price lunch. 4 Of the 255 returned surveys, each district was identified; however, seven surveys were not linked to a principal and were therefore not included, bringing the maximum available to 248 surveys. Every reasonable effort was exhausted to locate missing data. All missing pieces of data were hand-searched to crosscheck for accuracy. Prior to determining the final sample, a 10% random sample was selected to ensure the accuracy of the databases compiled for the study and detected no errors in data entry or data calculation. 4 See Appendix C for a list of the data, including the source and year.

PAGE 69

58 For each type of data collected, a systematic procedure determined the usability of a principal in this study. Principals were evaluated to determine how they met the screening criteria. Essentially, each case, or principal, was examined to determine the level of missing data. The criteria included 1) holding a principalship at their present location for 5 or 6 continuous years, 2) assessing the completeness of third grade retention rates, 3) assessing the completeness of school-wide free or reduced price lunch eligibility rates, and 4) assessing the completeness of survey items used in these analyses. Each piece of the screening criteria is described below, as well the how I arrived at the total usable sample. At the end of each procedure a sub-total is provided. First, as surveys were returned to the researcher, they were entered into a database structured at the person-level where each principal represented a row (or case) (Kreft & de Leeuw, 1998; Singer & Willett, 2003). Then, using the names of the responding principals, I created a matrix of each principals employment history for school years 1999-2000 through 2004-2005. Three components constituted a principals employment history which included: 1) the number of years of principalship total as reported on item 28; 2) the number of year of principalship held at their current school, as of 2004-2005, as reported on survey item 29; and 3) the principal needed to still work at the same school, as revealed by the name of the school they provided on the survey questionnaire. Of those who responded to the survey, 121 principals held a principalship for at least 5 or 6 continuous years at the same school and thereby met the time requirement. The second factor considered third grade retention percentages for the 5-year period examined in this study. Using data files provided by the Florida Department of Education, I created two sets of spreadsheet files. One file contained raw numbers of

PAGE 70

59 students retained in third grade from each of the 5 years used in this study, and the other file contained the numbers of third graders enrolled each year. I merged these files to calculate the percent of third grade students retained in grade for each year examined in this study. There were cases when one or both of the state provided data files did not contain either the raw number of students enrolled or the enrollment needed to determine the percentage. There were 6 principals removed due to missing data from one or more time point. This reduced the usable sample to 115 principals. The third factor considered the level of missing free or reduced price lunch eligibility percentages for each school. Following the same procedure as determining the percent of students retained in third grade, free and reduced price lunch eligibility databases were created for each of the 5 years from files provided by the FLDOE. Of the 114 principals, there was one principal with many years of free or reduced price lunch eligibility missing; therefore, to avoid erroneous conclusions, this principal was removed. The total usable was reduced to 114 principals. The survey items used in this study underwent an inspection to assess the completeness of the closed-item responses. The four principals who did not respond to item 9 were removed, as were two others who did not respond to at least half of the items used to determine the degree of support groups. The total usable sample was reduced to 108 principals. Finally, after one final inspection of the 108 principals, I reexamined all cases and removed two outliers. Since only two schools that primarily serve students with disabilities responded to the survey, these outliers were removed due to extremely small third grade student populations. An additional inspection of the data revealed 4

PAGE 71

60 principals with extreme values. These schools contained percent retained values that appeared to be anomalous. Without revealing too much identifying information, an example is provided. One school contained near zero percent retained for two years, while in the next year nearly a quarter of third grade students were retained. To avoid erroneous conclusions, these principals were removed from the final analysis. In all, there is a usable sample of 102 principals in these analyses. The final distribution of the usable sample by district size is provided in Table 3-4. Table 3-4. Usable sample versus returned responses by district size District Size Total Sent Returne d Usable Percent Usable Small 43 18 11 61.1 Mid-Size 115 46 21 45.7 Large 732 191 70 36.6 Total 890 255 102 40.0 Usable Sample Representativeness Population and sample data were compared to determine the probability that the sample represents the percent retained for the population of all state elementary schools. Since total population data was readily available in files provided by the FLDOE for each of the 5 years of retention data used in this study, I compared the usable sample principals (n = 102 school principals) to the state-level totals 5 using non-directional z-tests. Table 3-5 displays means, standard deviations, standard errors, z-scores and p-values demonstrating that there was not enough evidence to conclude that the usable sample mean percent retained was unequal to the total population mean percent retained. 6 For example, as displayed in Table 3-5, for Year 5 (School Year 2003-04) the population 5 The number of schools in the Florida system varies by year. See Table 3-9 for the number of schools statewide for years 1999-00 to 2003-04 as reported on the Florida School Indicators website. 6 In other words, the means were statistically equal.

PAGE 72

61 ( = 10.722, = 3.514) was not significantly different from the usable sample (M = 9.127, s = 5.165), z = -.454, p = .650. Figure 3-5 visually displays the pattern between mean percent retained of third graders over the 5-year period for each principal comparing the population of principals in the state and the sample of principals usable in this study. Table 3-5. Third-grade percent retained usable sample principals versus the total population Year M s S.E. of M n (pop S.E.) N z-score p-value 1 3.077 4.036 .043 72 3.329 1.781 .004 1665 -.141 .888 2 3.118 3.394 .030 100 3.039 1.229 .003 1714 .064 .949 3 3.359 3.974 .035 102 3.338 1.552 .004 1776 .014 .989 4 11.855 7.124 .064 102 14.377 5.250 .012 1829 -.480 .631 5 9.127 5.165 .045 102 10.722 3.514 .008 1848 -.454 .650 Note. Weighted by third grade enrollment at each year 1999-002000-012001-022002-032003-04Time in Years 246810121416Mean Percent Retained SamplePopulation Figure 3-5. Population versus usable sample mean percent retained by year

PAGE 73

62 Additional Variables Four other variables were designed for the analyses that follow. Two are measures of school-wide level of poverty; while two others consider effects of policy and time. Socioeconomic status. For this study, the percent of students eligible for free/reduced priced lunch at the school-level was used as a proxy for the socioeconomic status (SES) of the school population. Prior to analysis, I examined the data to determine the variability in school SES for each principal over the 5 years. Since SES lacked variability over years, I created a new variable called SESMEAN (Figure 3-6). 1999-002000-012001-022002-032003-04Time in Years 0.0000.2000.4000.6000.8001.000Mean SES Cases weighted by lgenrol Figure 3-6. Mean SES values over 5-year period for usable sample schools Mean Centered SES. An additional proxy for poverty used in the analyses that follow is a variable called SESDIFF. This variable subtracts the raw percentage of students eligible for a free or reduced price lunch in a school for each year from the mean percentage of students eligible for free or reduced price lunch for that school over years.

PAGE 74

63 Pre/Post Policy. This is a dummy coded variable used to make a distinction between the preand post-policy periods. The years before the start of the retention policy (i.e., years 1, 2, and 3) have been dummy coded as 0; while the final two years (i.e., years 4 and 5) have been coded as 1 (Table 3-1). This Pre/Post Policy variable estimates the effects of the policy. Trend within Policy Period. A final coding scheme was developed to separately analyze trends over years for preand post-policy periods. The variable twt1 was coded to analyze the differences during the pre-policy years; while the variable twt2 was coded to analyze the post-policy differences. Each time point contains a 1-point difference and each trend is centered on zero (Appendix E). Design Research Questions Here, I investigate the possible explanations to the overarching research question What is the trend of third-grade retention practices before and after implementation of Floridas third-grade retention policy and how is this trend impacted by other variables? To answer this question, I use a multilevel statistical model for change (viz., hierarchical linear model, growth curve model, or mixed models) examining within-person change and between-person differences in change beginning with the 1999-2000 school year (Bryk & Raudenbush, 1987; Singer & Willett, 2003). Using longitudinal data, I ask How do principals retention practices change between policy periods? How is this change impacted by poverty level? What variables reduce this variation? The next sections explain how I answered these questions.

PAGE 75

64 Multilevel Framework A repeated measures analysis of variance (ANOVA) was conducted as a preliminary analysis. Here, the unit of analysis is time where the repeated measure ANOVA treats time as levels of a factor. After inspecting these data structured at the person-level, it was decided that multilevel modeling might lead to a more valid interpretation of the data. In making this decision, I consulted the literature on multilevel modeling (Bryk & Raudenbush, 1987; Hox, 2002; Kreft & de Leeuw, 1998; Singer & Willett, 2003). Kreft and de Leeuw (1998) explain several reasons why one would choose a multilevel model over a typical ANOVA or linear regression. Figure 3-7 shows one limit of examining person-level data. Examining the mean rates of retention may produce misleading results, as it does not consider each individual. Technically, there is variation both between years for a principal and variation between each principal. This can lead to inflated Type I error rates and significant findings that are spurious. Multilevel modeling easily manages unbalanced cases with missing values (Kreft & de Leeuw, 1998; Singer & Willett, 2003). In fact, unbalanced data is thought to be interpreted more readily using multilevel modeling compared to a technique such as repeated measure analysis of variance (Singer & Willett, 2003). Additionally, in a technique such as an ANOVA, typically cases with missing data are deleted (i.e., listwise or pairwise), whereas they are not in a multilevel model (Hox, 2000; Moskowitz & Hershberger, 2002). The techniques of Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence (Singer & Willett, 2003) informed the design of this study. According to Singer and Willett (2003), the first step in building a multilevel model is to determine if it is an appropriate technique. To make this determination, they recommend creating

PAGE 76

65 empirical growth plots to inspect the trajectories using a small sample. To create these growth plots, the data was restructured in a person-period data set where multiple records exist for each individual. If there is sufficient variability between these growth plots, then multilevel modeling for change may be appropriate. 12345Time in Years 2.04.06.08.010.012.014.0Mean Percent Retained Figure 3-7. Mean retention rates by year. Results of repeated measures analysis of variance of principals year-by-year retention rates The figures shown were instrumental in this decision making process. First, the records shown in figure 3-8 display the range of possibilities, illustrating that there is sufficient evidence showing variability amongst principals retention rates for both the intercepts and the slopes. Figure 3-9 displays the variability amongst all the principals of this study. Most principals retention rates begin near 0% and rise sharply over the 5-years displayed. Some principals had seemingly higher rates of retention before the state policy, which then declined over time, while a few remained more constant over time. Nevertheless, there is sufficient variation over years to warrant further inspection.

PAGE 77

66 Singer and Willett (2003) further explained three methodological characteristics complementary to multilevel modeling in the context of a longitudinal analysis examining change over time. Here, I state their recommendations and share how I considered their advice. First, they recommended three or more waves of data to detect change. For these analyses, I selected 5 years of data to model change over time. Three years were pre-policy and the latter two were the available data since Florida implemented the retention policy. For my study, I planned to inspect time within each policy period, as well as time before and after policy implementation. Second, they suggested that data should change systematically over time. With the introduction of the retention policy in 2002-03, the systemization of change over time was exemplified by 010203040Percent Retained 010203040Percent Retained 12345Time in Years 010203040Percent Retained 12345Time in Years 12345Time in Years Figure 3-8. Examples of principals third-grade retention rates over time

PAGE 78

67 the introduction of this policy. Post-policy retention rates changed based upon FCAT results. Finally, they advised researchers to use a noteworthy metric to measure time. For this study, the third grade retention outcome at the end of the school year was used to measure policy years. Since promotional status changes on a year-by-year basis, the school year is the metric for time. Study years were coded as either preor post-policy. Linear Regression 12345Time in Years -10.0-5.00.05.010.015.020.025.0Percent Retained Figure 3-9. Third-grade retention over time. Observed variation in fitted trajectories for all principals (n = 102) Model Assumptions All statistical analyses are based on underlying assumptions. Because a multilevel model is more complex, so are the assumptions on which this model was based. Singer and Willet (2003) identified three key assumptions for researchers to consider when determining the tenability for fitting a model to data. These assumptions are the shape, normality, and homoscedasticity. 7 First, the shape, or linearity, of the dependent variable was inspected to determine whether the means fall on a straight line as a function of the independent variable. That 7 All analyses were weighted by the log of school-level third-grade enrollment at each year.

PAGE 79

68 is, does the mean of percent retained for preand post-policy connect with a straight line? For variables such as Pre-Post Policy, there [was] nothing to assess because a linear model is de facto acceptable for dichotomous predictors (Singer & Willett, 2003, p. 128). Simply put, any two points can be connected by a line. Second, a key assumption was that these data were distributed normally. That is, the distribution of the dependent variable is normal for both preand post-policy. Violations of normality are usually robust; however, for these data, the violation was severe for the dependent variable, percent retained. Figure 3-10 shows the extreme positive skew of the data distribution. The minimum value was 0, and the maximum value was 34.65, suggesting a normal median at approximately 17. However, the clustering of data points near zero resulted in a median of 4.41. The mean of this skewed distribution was 6.29. Using the standard deviation of 6.19 for interpretation would lead to the impossible statement that approximately 95% of retention rates in this sample are between -6.35% and 19.73%. We can refer back to figure 3-9 to see predicted percent retained with negative values. The chosen solution to this problem was a mathematical transformation of the dependent variable into natural log units (McElroy, 2001; Singer & Willett, 2003). Since it is mathematically impossible to take the natural logarithm of a zero value, an alternate metric was needed since this sample contained forty-nine values associated with a zero value. Rather than completely remove these cases using listwise deletion (Wothke, 2000), a decision was reached to add 1 to the percent retained value before taking the logarithmic transformation. The natural logarithm of percent retained plus one normalized the distribution. This transformation of the data permitted zero-values

PAGE 80

69 (Figure 3-11). The resulting distribution had a mean of 1.65, a median of 1.74, and a standard deviation of .917. To assist interpretation, log values can be converted back to the original scale of measurement by taking the antilog and subtracting one (Singer & Willett, 2003). Now, it is more feasible to argue that 95% of the population log percent retained plus one are between (exp(.00) -1) = .00 and (exp(3.57) -1) = 34.52. Finally, Singer and Willett (2003) also recommended examining the homoscedasticity, the assumption of equal conditional variances. In other words, we assume that the variance of the dependent variable is the same at all values of predictor, which is to say that the spread of values around the mean dependent variable is the same for all values of the predictor. So, for these data, there are two mean log percent retained values, one pre-policy and the other post-policy. The assumption is that the variation around these means would be the same. Boxplots were examined to determine whether this assumption was met. For the assumption to be met, the interquartile ranges should not be considerably wider or narrower than the other. Figure 3-12 shows the interquartile ranges for two predictors. Visually, we see near equal variances; therefore, homoscedasticity appears to have been met. Model Building Process The rationale for building a multilevel model was twofold: 1) to explain variation in principal retention rates over 5-years while acknowledging the violation of independence since 5-years worth of data are nested within each principal and 2) to add predictors sequentially to test the significance of that predictor, as well as changes in model fit and variance reduction. Many procedures assume statistical independence make some sort of independence assumption. For example, when comparing two groups,

PAGE 81

0.0010.0020.0030.0040.00Percent Retained 0100200300400Frequency Mean = 6.6008Std. Dev. = 6.44589N = 2,281Cases weighted by lgenrol 70 Figure 3-10. Histogram of percent retained before the transformation 0.001.002.003.004.00Log Percent Retained Plus 1 050100150200250Frequency Mean = 1.6533Std. Dev. = 0.9174N = 2,281Cases weighted by lgenrol Figure 3-11. Histogram of percent retained after the transformation

PAGE 82

71 differences between groups should be due to the variable of interest and not some shared feature of group membership (unless those groups are explicitly included in the model). Violations of this assumption can lead to incorrect standard error estimates resulting in a Type I error rate unequal to the conventional nominal =.05 (i.e., the potential for spurious statistical significance) (Singer & Willett, 2003). Pre-PolicyPost-PolicyPre-Post Policy -2.00-1.000.001.002.00Residuals 350197 NoYesRetention Belief -2.00-1.000.001.002.00Residuals 219 Figure 3-12. Examining the homoscedasticity assumption. Left panel represents the residual by Pre-Post Policy. Right panel represents the residual by Retention Belief. Within these models, I used the following terms to represent the variables and predictors. In these analyses, log(Y ij +1) was the natural log of the percent retained for time i in school j with an addition of 1 to permit log transformations of zeros. 8 Singer and Willett (2003) recommended using a scale to help make the intercept more interpretable. In all models, the intercept parameter 0j indicated the average percent retained for principal where 0 corresponds to Pre-Policy. 1j was the regression coefficient relating time to log percent retained for principal j. 00 was the grand mean log percent retained at 8 The addition of 1 to the log is implied unless stated otherwise.

PAGE 83

72 Pre/Post_Policy=0. 10 was the grand mean slope over principals. e ij was the residual for principal j at time i. There are seven main models in this taxonomy labeled Model A through Model F2. Figure 3-13 provides an overview of the model taxonomy included in this study. To make meaning from this model building process different statistics were examined. Results from these models yield intra-class correlations (ICC) and proportionate reductions in error (PRE). ICC values provided opportunity to compare values associated within a particular model. These values are not compared to other models, whereas PRE values provided context for model comparison. An ICC represents the proportion of total variance that is between-principals for a particular model. A PRE represents the proportionate reduction in variance for a particular variance estimate (e.g., the residual variance) relative to another model. For consistency, all models and sub-models were compared back to Model B, which is the time model called Pre/Post Policy. Finally, there are many ways to ascertain the fit of the model relative to the fit of other models, including Akaikes Information Criteria (AIC), Bayesian Information Criteria (BIC), and log-likelihood (LL) statistic (Singer & Willett, 2003). This research used Akaikes Information Criteria (AIC) to provide an overall comparison of the model fit. The AIC has the disadvantage of a more subjective interpretation (i.e., no corresponding p-value) and the advantage of being able to compare non-nested models (e.g., one model with a predictor X compared with another model with predictor X and not Y) (Singer & Willett, 2003). Using a smaller-is-better approach, these goodness of fit values allow comparison to Model A, the baseline model.

PAGE 84

73 Model A: Baseline Model B: Pre/Post Policy Model C: Level of Poverty Model D: Mean Centered SES Model E: Policy Trend Model F1: Retention Belief Model F2: Degree of Support Figure 3-13. Multilevel model for change taxonomy Study Limitations Limitations may hinder valid interpretation of these results. First, one study assumption needed to be addressed prior to these analyses. Given the violation of normality, I had to make a decision regarding whether to select a different model to accommodate the data (e.g., specify a binomial distribution with a logit link) or to transform the data to accommodate the model. The latter was selected. Initially, these data were normalized taking the log of the dependent variable, percent retained. This would have resulted in dropping fifty-seven 0% retained values from this study, simply because it is mathematically impossible to take the natural logarithm of zero. Figure 3-14 displays the frequencies of values that would have been removed due to 0% retained by

PAGE 85

74 year. Interestingly, here we see that 0% retained is largely a product of pre-policy retention practices, not post-policy. 12345Time in Years 05101520Frequency Figure 3-14. Number of zero percent retained values removed by year This decision could have been justified because of the positive skew of the distribution resulting in the majority of values being close in proximity to the scores being removed. In other words, this was not a case of removing outliers; instead, it was a more sensible technique to handling these data than either permitting the skew or proposing an additional transformation of the data to permit zero-values. However, this technique was not selected because of the richness of the data potentially lost. Because of the pre-post policy focus of my study, it seemed prudent to select a metric that retained these values. Therefore, prior to transforming the percent retained, 1 unit was added to all the values. This adjustment permitted the 0% retained values, and normalized the percent retained distributions. Finally, after the data collection began, four hurricanes inundated Florida, which may account for the lower than anticipated return on the survey. To minimize the amount

PAGE 86

75 of researcher interaction, influence, or bias, a paper-pencil survey was provided as an alternate option for principals. These self-administered surveys lessened interaction between the researcher and the potential participants and helped minimize social desirability bias (Baumann & Bason, 2004).

PAGE 87

CHAPTER 4 RESULTS Introduction Results presented for the multilevel taxonomy of models begin systematically with Model A and conclude with Model E. Model building was a successive process with each model building upon the former. A summary table of the results is presented demonstrating how variables, such as time measured as preor post-policy, level of poverty, and trend within policy period impact third grade retention in Florida. Other variables, such as a principals retention belief and the degree of support provided in schools were also tested; however, these variables did not improve model fit and were dropped from the overall analysis (Appendix E). A summary table of the results is available at the end of this chapter (Table 4-1). The resulting models, A through E, help answer the following research questions How do principals retention practices change between policy periods? How is this change impacted by poverty level? Do retention practices vary more between policy periods or between principals? What variables reduce this variation? Multilevel Models Model A: Baseline Model A, the baseline model, is often called a one-way random effects ANOVA model or an unconditional means model (UM) (Raudenbush & Bryk, 2002; Singer & Willett, 2003). It is similar to the typical fixed effects ANOVA; however, the residual 76

PAGE 88

77 variance is decomposed into two sources (Raudenbush & Bryk, 2002). The sample distribution of the dependent variable, percent retained, was extremely positively skewed. For this reason, the dependent variable was mathematically transformed (McElroy, 2001; Singer & Willett, 2003) to yield a log-normal distribution, as shown in Equation 4-1. log(Y ij +1) = 0j + e ij (4-1a) 0j = 00 + 0j (4-1b) log(Y ij +1) = 00 + 0j + e ij (4-1c) Hence, log(Y ij +1), the log percent retained for school j at time i, is a function of 00 : the grand mean percent retained over all schools before and after policy 0j : the residual for school j representing the distance of that schools percent retained from the grand mean. These residuals are assumed to be normally distributed with a mean of 0 and a variance of 2 e ij : the residual for school j at time i represent the distance of the school at that time from that schools mean before and after policy. These residuals are also assumed to be distributed normally with a mean of 0 and a variance of 2 e. 1 Model A is a baseline model with no predictors. It serves as a model for comparison to subsequent models. Using Equation 4-1c, the resulting model is shown in Equation 4-2. log(Y ij +1) = 1.66 + 0j + e ij (4-2) Thus, the grand mean log percent retained was 1.66, t(96.160) = 34.800, p < .001, corresponding to a grand mean percent retained of (exp(1.66)-1) = 4.26%. The within-principal variation was significant (p < .001) suggesting the need for predictors, while the between-principal variation was not significant (p = .071). In fact, 98.3% of the 1 Variances for future models are assumed to meet the same criteria.

PAGE 89

78 variability in log percent retained was within-principals. The AIC for the baseline model was 1279.43. Figure 4-1 is a plot that regresses the transformed retention percentages on the principals. It is clear that, over the principals, the predicted regression line suggests little difference while the dispersion of points within principals creates a cloud of variability throughout the graph. It is concluded that efforts should be taken to explain this variability within-principals. 0255075100Participant Number 0.001.002.003.00Log Percent Retained Plus 1 Figure 4-1. Scatterplot of log percent retained plus one Model B: Conditional on Pre/Post Policy Model B was the first in this series of growth models aimed to explain the impact policy has on retention rates. For this, a variable representing time before and after the introduction of the retention policy was added to the baseline model. For my study, the time variable was coded such that Pre/Post_Policy=0 corresponds to the school years

PAGE 90

79 1999-00, 2000-01, 2001-02 and Pre/Post_Policy=1 corresponds to the school years 2002-03 and 2003-04. Distributions were first examined and appear to meet assumptions (Figure 4-2). Pre-policy has a mean of 1.184 which corresponds to exp(1.184)-1 = 2.27%. The standard deviation of .822 suggests that approximately 95% of retention rates in this sample are between (exp(-.46, 2.83)-1), or between (0%, 15.95%). Post-policy has a mean of 2.31 which corresponds to exp(2.31)-1 = 9.1%. With a standard deviation of .595 suggests that approximately 95% of retention rates in this sample are between (exp(1.12, 3.50)-1), or between (2.06%, 32.12%), as shown in Equation 4-3. log(Y ij +1) = 0j + 1j (Pre/Post_Policy) + e ij (4-3a) 0j = 00 + 0j 1j = 10 (4-3b) log(Y ij +1) = 00 + 10 (Pre/Post_Policy) + 0j + e ij (4-3c) For the reduced models, 0j was the intercept and is comprised of the grand mean corresponding to Pre/Post_Policy=0 and the residual, 0j representing the deviation of school j from the grand mean at Pre/Post_Policy=0. Incorporating Pre/Post_Policy permits the retention rates to vary over time for different schools. Hence, 1j is the slope and is comprised of the grand slope, 10 which is the change from pre to post policy. The combined model expresses that the log percent retained for school j at time i is a function of the percent retained for pre-policy, the change in percent retained from preto postpolicy, and the residuals. Using Equation 4-3c, the resulting model is shown in Equation 4-4. log(Y ij +1) = 1.17 +1.14(Pre/Post_Policy) + 0j + e ij (4-4)

PAGE 91

80 A 4.003.002.001.000.00 Log Percent Retained Plus 1 50403020100 Frequency 4.003.002.001.000.00 Log Percent Retained Plus 1 302520151050 Frequency D B 0.001.002.003.00Log Percent Retained Plus 1 0.001.002.003.00Log Percent Retained Plus 1 E C 0255075100New ID 0.001.002.003.00Log Percent Retained Plus 1 0255075100New ID 0.001.002.003.00Log Percent Retained Plus 1 F Figure 4-2. Distribution plots of third-grade retention rates. A) Pre-policy histogram. B) Pre-policy boxplot. C) Pre-policy scatterplot. D) Post-policy histogram. E) Post-policy boxplot. F) Post-policy scatterplot. The estimated mean log percent retained at pre-policy is 1.17, t(153.09) = 21.692, p < .001, corresponding to exp(1.17)-1 = 2.22%. The coefficient for Pre/Post_Policy was

PAGE 92

81 significant at 1.14, t(375.53) = 19.909, p < .001, suggesting that mean log percent retained increases by exp(1.14)-1 = 2.13% from preto post-policy. An intra-class correlation coefficient (ICC) of .079 indicates that approximately 8% of the variance in this model is between principals, while approximately 92.1% of the variance is within principals. This means that there is still considerably more within-principal variability (92.1%), than between-principal variability (7.9%). Incorporating policy reduced the within-principal variation by 51.5%, the variation within schools is 1.81, z = 13.655, p < .001. The proportion of variance within schools was approximately 98%; incorporating pre/post policy as a metric for time reduced it to approximately 92%. Further, model fit improved by 20.8% with the AIC dropping from 1279.43 to 1021.81. The subsequent models will be relative to this model except for the AIC, which is always relative to the baseline model, Model A. Model C: Conditional on Policy and Level of Poverty This model introduced a proxy for school-wide level of poverty using the percent of students eligible for a free or reduced priced lunch as the criterion. 2 Model C is shown in Equation 4-5. log(Y ij +1) = 0j + 1j (Pre/Post_Policy) + 2j (SESMEAN) + e ij (4-5a) 0j = 00 + 0j 1j = 10 2j = 20 (4-5b) log(Y ij +1) = 00 + 10 (Pre/Post_Policy) + 20 (SESMEAN) + 0j + e ij (4-5c) Using Equation 4-5c, the resulting model is shown in Equation 4-6. 2 Using disaggregated SES over years, as opposed to SES for each year, was justified by the lack of variation found in the repeated measures ANOVA described in Chapter 3.

PAGE 93

82 log(Y ij +1) = .463 + 1.13(Pre/Post_Policy) +1.36(SESMEAN) + 0j + e ij (4-6) The estimated mean log percent retained at initial status is .463, t(108.427) = 4.859, p < .001, corresponding to (exp(.463)-1) = .59%. However, this is at SESMEAN=0. The coefficient for Pre/Post_Policy of 1.13, t(374.619) = 19.758, p < .001, suggests that mean log percent retained increases by (exp(1.13)-1) = 2.1% from preto post-policy. The coefficient for SESMEAN of .014, t(97.426) = 8.434, p < .001, suggests that each one-unit change in SESMEAN corresponds to (exp(.014)-1) = .014% for both preand post-policy. An intra-class correlation coefficient (ICC) of .030 indicates that 3% of the variance in this model is between principals, while 97% of the variance is between years within principals. This served to balance-out the ICC, so to speak, compared to the variable that examined the effects before and after policy. Model fit improved by 24.5% (relative to the baseline model) with the AIC dropping to 966.34. Model D: Conditional on Policy, Level of Poverty and Mean Centered SES The model-building process continued with the inclusion of a variable for mean centered SES called SESDIF. This variable is the percent difference between a schools percent of students eligible for a free or reduced price lunch and the mean of SES over the 5-year period for that school. Model D is shown in Equation 4-7. log(Y ij +1) = 0j + 1j (Pre/Post_Policy) + 2j (SESMEAN) + 3j (SESDIF) + e ij (4-7a) 0j = 00 + 0j 1j = 10 2j = 20 3j = 30 (4-7b) log(Y ij +1) = 00 + 10 (Pre/Post_Policy) + 20 (SESMEAN) + 30 (SESDIF) + 0j + e ij (4-7c)

PAGE 94

83 Using Equation 4-7c, the resulting model is shown in Equation 4-8. log(Y ij +1) = .469 + 1.10(Pre/Post_Policy) + .014(SESMEAN) + .022(SESDIF) + 0j + 1j + e ij. (4-8) The estimated mean log percent retained at initial status is .469, t(109.614) = 5.006, p < .001, corresponding to (exp(.469)-1) = .60%. However, this is at SESMEAN=0. The coefficient for Pre/Post_Policy of 1.10, t(380.620) = 19.160, p < .001, suggests that each one-unit change in Pre/Post_Policy corresponds to a (exp(1.10)-1) = 2.0% from preto post-policy. The coefficient for SESMEAN of .014, t(98.421) = 8.670, p < .001, suggests that each one-unit change in SESMEAN corresponds to a (exp(.014)-1) = .014% change in mean log percent retained when Pre/Post_Policy and SESDIF are controlled at a particular value. The coefficient for SESDIF of .022, t(471.591) = 3.442, p < .001, suggests that each one-unit change in SESDIF corresponds to a (exp(.022)-1) = .022% change in mean log percent retained when Pre/Post_Policy and SESMEAN are controlled at a particular value. An ICC of .029 indicates that approximately 97% of the variance in this model is between years within principals, while approximately 3% of the variance is between principals. The variation within schools was 1.780, z = 13.609, p < .001, corresponding to a PRE of 1.5%. The variation between-schools at initial status is .054, z = 2.664, p = .008, corresponding to a PRE of 65.6%. Model fit improved by 24.7% (relative to the baseline model) dropping to 962.86. There was virtually no change in PREs, ICCs, or AIC, nevertheless this model was retained because the parameters and variance components were significant.

PAGE 95

84 Model E: Conditional on Policy, Level of Poverty, Mean Centered SES and Trend within Time The final model formally presented added two variables to account for the policy trend with the preand post-policy time periods. These variables, twt1 and twt2, were coded so that twt1 was centered on pre-policy time and twt2 was centered on post-policy time. Model E is shown in Equation 4-9. log(Y ij +1) = 0j + 1j (Pre/Post_Policy) + 2j (SESMEAN) + 3j (SESDIF) + 4j (twt1) + 5j (twt2) + e ij (4-9a) 0j = 00 + 0j 1j = 10 2j = 20 3j = 30 4j = 40 1j = 50 (4-9b) log(Y ij +1) = 00 + 10 (Pre/Post_Policy) + 20 (SESMEAN) + 30 (SESDIF) + 40 (twt1) + 50 (twt2) + 0j + e ij. (4-9c) Using Equation 4-9c, the resulting model is shown in Equation 4-10. log(Y ij +1) = .465 + 1.11(Pre/Post_Policy) + .014(SESMEAN) + .020(SESDIF) + .034(twt1) + -.226(twt2) + 0j + 1j + e ij (4-10) The estimated mean log percent retained at initial status is 0.465, t(109.806) = 4.952, p < .001, corresponding to (exp(.465)-1) = .59% when Pre/Post_Policy, SESMEAN, SESDIF, twt1 and twt2 are zero. The coefficient for Pre/Post_Policy of 1.11, t(380.595) = 19.333, p < .001, suggests that each one-unit increase in Pre/Post_Policy corresponds to a (exp(1.109)-1) = 2.03% from preto post-policy. The coefficient for SESMEAN of .014, t(380.595) = 19.333, p < .001, suggests that each one-unit change in SESMEAN corresponds to a (exp(.014)-1) = .014% change in mean log percent retained when Pre/Post_Policy, SESDIF, twt1 and twt2 are controlled at particular values. The coefficient for SESDIF of .020, t(469.348) = 2.986, p = .003,

PAGE 96

85 suggests that mean log percent retained changes on average by (exp(.020)-1) = .02% when SESMEAN, Pre/Post_Policy, twt1 and twt2 are controlled at particular values. The coefficient for twt1 of .034, t(385.280) = .720, p = .472, suggests that mean log percent retained changes on average by (exp(.034)-1) = .034% when SESMEAN, SESDIF, Pre/Post_Policy and twt2 are controlled at particular values. The coefficient for twt2 of -.226, t(373.269) = -2.616, p = .009, suggests that mean log percent retained declines on average by (exp(-.226)-1) = -.202% when SESMEAN, SESDIF, Pre/Post_Policy and twt1 are controlled at particular values. An ICC of .030 indicates that approximately 3% of the variance in this model is between years within principals, while nearly 97% of the variance is between principals. Variation within schools is .1.755, z = 13.570, p < .001, corresponding to a PRE of 2.9%, compared to Model B. The variation between-schools at initial status is .055, z = 2.726, p = .01, corresponding to a PRE of nearly 65%, compared to Model B. The AIC dropped to 962.89 improving the model fit improved by 24.7% when compared to Model A. Summary We conclude the model building process with a review of the underlying statistical assumptions on which this study was designed. Again, although this study modeled growth over a 5-year period, an alternative metric for time was used to predict the effects of retention before and after policy implementation. However, within each policy period the trend within time was modeled. Pre-policy contained three time points and because the trend before policy was relatively stable, it met the assumption. Again, post-policy was dichotomous and linear de facto. Normality and homoscedasticity was assessed at multiple points during the model building process and remained unchanged from the assessment prior to the model building.

PAGE 97

86 Table 4-1. Results of fitting a taxonomy of multilevel models for change to the log percent retained data, Models A through E (n = 102) ModelA B C D E Intercept 1.66*** 1.17*** .463*** .469*** .465*** Pre/Post_Policy 1.14*** 1.13*** 1.10*** 1.11*** sesmean .014*** .014*** .014*** sesdiff .022** .020** twt1 .034 twt2 -0.226** Residual 3.72*** 1.81*** 1.82*** 1.78*** 1.76*** Intercept .064~ .155*** .056** .054** .055** Between .064 .156 .056 .054 .055 Within 3.723 1.807 1.816 1.780 1.755 TOTAL 3.787 1.963 1.872 1.834 1.810 Proportionate reduction in error PRE-Residual .515 -0.005 .015 .029 PRE-Intercept -1.430 .637 .656 .648 Relative To ModelA B B B Intra-class correlation coefficient ICC-Between .017 .079 .030 .029 .030 ICC-Within .983 .921 .970 .971 .970 Goodness of Fit AIC 1279.43 1012.81 966.34 962.86 962.89 AIC difference 266.62 313.09 316.57 316.54 Fit improvement (%) 20.80 24.50 24.70 24.70 Relative to ModelA A A A ~ p < .10; p < .05; ** p < .01; *** p < .001

PAGE 98

CHAPTER 5 DISCUSSION Introduction The models showed that time (or the policy period) and poverty at the school level played dramatic roles in the retention rates across this sample. This discussion delves into retention trends to address my overarching question What is the trend of third-grade retention practices before and after implementation of Floridas third-grade retention policy and how is this trend impacted by other variables? The model building process (Figure 3-13) guided my study to understand how particular variables such as policy, and level of poverty impacted retention practices over a 5-year period. This chapter discusses the results from this model building process to answer the more specific research questions. How do principals retention practices change between policy periods? How is this change impacted by poverty level? What variables reduce this variation? Then, we turn our attention the policy implications and recommendations, as well as suggestions for future research. Research Question 1: How Do Principals Retention Practices Change between Policy Periods? The model building process originated with a degenerate model providing a baseline to compare model fit. In considering the baseline model, significant variation was found both between principals and within principals over years that needed 87

PAGE 99

88 explanation. Over 98% of the variation was found within principals over years. In the final model, we gained insight regarding principals retention policies between policy periods. The key feature of Model E was its representation of the data. First, it adjusted for betweenand within-principal SES. Then, it revealed a significant difference between the two policy periods. Furthermore, although it did not reveal a significant trend between the three pre-policy years, it did reveal a significant difference between the two post-policy years. 0 5 101520251 2 345 log(perret+1) p erret A 0 5 101520251 2 345 log(perret+1) perret B Figure 5-1. Examples of mean predicted retention trends over time. Two lines are shown in each graph. One line represents the predicted log percent retained value (log(perret+1)), while the other is unlogged showing percent retained units (perret). Because two metrics are shown in one graph, the y-axis is simply one-unit intervals. The x-axis shows time in years, with years 1, 2, and 3 being pre-policy, and years 4 and 5 being post-policy. A) School from mid-size district, mean SES 52%. B) School from same district, mean SES 91%.

PAGE 100

89 The fact that pre-policy was not significant is worth discussion. This means that between the pre-policy years, the change in mean retention rates can be considered equal. Retention practices were stable before implementation of the state retention policy. On the other hand, the post-policy years (years 4 and 5) had mean retention rates that were significantly different from each other. Two examples of the mean predicted values help us understand how retention trends have changed over time (Figure 5-1). Within both examples, on visual inspection, it is clear why the pre-policy trend was not significant, and the post-policy trend was significant. Research Question 2: How Is This Change Impacted by Poverty Level? Since the level of poverty is widely regarded as a factor that impacts student achievement, school level of poverty was included as a predictor. Using the mean SES of each school over the 5-year period confirmed that, even in this sample, poverty plays an influential role in retention rates for third graders in Florida. Mean SES rates greatly affect retention practices. Schools with the highest levels of poverty (greater than 67%) consistently retained more students over this 5-year period (Figure 5-2). Principals from schools with the lowest range of poverty (33% or less) retained fewer students. Clearly, these principals have had fewer practical experiences with retention over this 5-year period than principals of higher-poverty schools; however, even they have felt the impact of policy with more students experiencing retention in pre-policy years. In further examining the pre-post policy trend, I inspected the poverty levels over time. Here we see the relationship between preand post-policy (Figure 5-3). As poverty increased, the percent of students retained also increased. Poverty is a great predictor of retention rates and so policy impacts students from higher-poverty schools more greatly. In other words, lower-achieving students in higher-poverty schools were more likely to

PAGE 101

90 be promoted pre-policy and because of this, the mandate impacted more children in higher-poverty schools than students in lower-poverty schools. 1999-002000-012001-022002-032003-04Time in Years 0.501.001.502.002.503.00Mean Predicted Values Poverty Groups0-33%34-66%67-100% Figure 5-2. Mean predicted third-grade retention rates by poverty level over time 0 2 4 6 8 10 12 14 16 18 20 5 15 25 35455565758595Mean SES (%)Percent Retained pre post Figure 5-3. Example plotting retention rates using principals mean SES values before and after policy implementation

PAGE 102

91 Research Question 3: What Variables Reduce This Variation? With the focus on change over time, it was hoped that the measure for time would help explain much of the variability. Model B, the Pre/Post Policy model, added growth explicitly into the model with the log units of percent retained showing a significant increase from preto post-policy. The most notable impact of Pre/Post Policy model is improved model fit compared to the baseline model. It was somewhat surprising that variation within principals over the years decreased by only 8% with the proportion of variance decreasing from 98% to 92%. The Pre/Post Policy Model suggested a marked change in retention rates for most schools in 2002-03 suggesting further analysis was required. Attempts were made to move the model building forward beyond policy period and level of poverty examining retention belief and the degree of support, however improved model fit was not achieved. Although model fit was not statistically significant, we can examine these patterns descriptively to understand what they reveal in this context (Onwuegbuzie & Leech, 2004). Nevertheless, we first work to understand what these mean predicted values suggest when considering principals current retention beliefs (Figure 5-4). Since Florida did not have a formal retention/promotion policy before 2002-03, it is interesting to consider possible explanations for the predicted retention patterns over the policy periods, especially considering retention belief. As learned in Chapter 2, the NCLB Act of 2002 is one of most wide sweeping vehicles of educational reform. The mean predicted retention values showed a pattern shift around the 2001-02 school year when NCLB was enacted. It is certainly possible that NCLB had some sort of impact on the beliefs of principals, especially in light of the stronger focus on accountability. Since

PAGE 103

1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values Retention BeliefNoYes 92 Figure 5-4. Mean predicted third-grade retention rates by principals current retention belief over time. the signing of NCLB, the focus on accountability has been prominent. Perhaps principals with the lowest achieving students feel they are in the spotlight. If students in these schools do not meet the AYP mandate, the whole school faces severe penalties, including possible school restructuring and reduced funding. Perhaps Florida principals believe placing their trust in the state mandate to hold back students not reading on grade-level will help their schools achieve AYP. Further information is needed to explain these patterns before and after policy. In this current policy era, the relationship between retention beliefs and practices yields interesting discussion. Past research of retention beliefs found that 74% of principals favored retention for students who were not meeting the grade-level standard

PAGE 104

93 (Byrnes, 1989); similarly, my study found that 69% of principals believe retention can benefit third-grade students who fail the FCAT. However, as shown in figure 5-4, it appears that the Intent works because the Florida retention policy does not permit principals retention beliefs to play a factor in retention decisions. My study shows that regardless of a principals retention belief, students are retained based on their FCAT performance, not a principals retention belief. In a nutshell, as a result of the policy mandate, retention beliefs as reported and analyzed in my study did not predict retention rates. Future research may explore why some principals who do not believe in retention, held back more students before the start of the retention policy than those principals reporting they believe in retention. A possible explanation may be that principals beliefs have swayed since they are now obligated to uphold state policy regardless of their personal beliefs. It could be possible that principals disavowed their personal beliefs so as not to impair their professional judgment. Perhaps, before policy, subtle shifts were the result of teachers or parents overriding the principals belief to veto retention (Jimerson et al., 1997) or pressure to achieve on high-stakes assessments; whereas this option has been severely limited since the implementation of the retention policy. It is perhaps more interesting to discuss these mean predicted retention values in relation to poverty level over time to further illustrate the patterns. Figure 5-5 displays four panels, 1 through 4, plotting principals current retention belief and the mean predicted retention practices of principals over a 5-year period. Panel 1 shows the pattern of all principals using the mean of percent retained (also shown in figure 5-2). Panels 2, 3, and 4 isolate the level of poverty with Panel 2 showing principals who work in schools

PAGE 105

94 with the lowest levels of poverty (lower than 33.9%). Principals of lower-poverty schools have lower retention rates when compared to schools with higher rates of poverty. Panel 3 shows schools in the mid-range of poverty, with between 34.0 to 66.9% students eligible for a free or reduced price lunch, retain more students than their lower poverty counterparts. We can see that principals in these lowerand mid-range poverty schools retain nearly the same numbers of students regardless of their retention belief. This suggests that principals of lowerand mid-range poverty schools are possibly more policy compliant than other types of schools. Finally, panel 4 shows how principals who work in schools with the highest levels of poverty retain students differently before and after the start of the retention policy. This panel also revealed one of the more intriguing findings of this study. Prior to policy, principals of higher-poverty schools who report not believing in retention held back more believers continued to hold back more students. That is, the model predicted that non-retention believing principals of the higher-poverty schools retained more students than retention believers. Perhaps these are the principals who do not believe in retention based on results from high-stakes assessments, but do believe that students can benefit developmentally from more time, or who believe they are giving the gift of time. This gift of time refers to the conventional wisdom that providing children with more time to learn will give them the opportunity to increase their academic potential (Ames, 1986; Graue, Kroeger, & Brown, 2003). In response to variables affecting retention practices, we shift our discussion to the supports offered to students at risk for retention as disaggregated by degree of support

PAGE 106

95 (Figure 5-6). First, I will discuss predicted third-grade retention rates and current degree of support over time. Schools currently using nearly all the supports (5 or 6 out of 6) 1 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 2 3 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 4 Figure 5-5. Principals retention belief by the mean predicted third-grade retention rates over time disaggregated by poverty group. 1) Mean level of poverty for all sample schools. 2) Sample schools with less than 33.9% poverty. 3) Sample schools with between 34.0 to 66.9% poverty. 4) Sample schools with more than 67.0% poverty. (Solid lines equal retention believer; dashed lines equal non-retention believer)

PAGE 107

96 have the highest retention levels. When we compare the current degree of support to past retention rates, it could be said that schools with higher rates of retention now provide as many supports as possible to prevent retention. Since we do not know whether these schools used a similar number of supports in years past, we cannot draw conclusions about whether more or fewer supports will benefit students. However, this variable may permit future research to make such determinations. Interestingly, when including poverty level and policy period in the model building process, degree of support did not account for much of the variation, nor did it achieve significant p-values. Figure 5-7 displays four panels, 5 through 8. Panel 5 shows the pattern of all principals using the mean predicted percent retained (also shown in figure 5-6). Panels 6, 7, and 8 isolate the level of poverty with Panel 6 showing principals who work in schools with the lowest levels of poverty (lower than 33.9%). As we have seen 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values Poverty Groups0-33%34-66%67-100% Figure 5-6. Mean predicted third-grade retention rates by degree of support over time

PAGE 108

97 fluctuate; however, what is worth noting here are the similarities in retention patterns regardless of the degree of support provided within the school. This is especially evident in Panels 6, 7 and 8 where depending on the poverty level of the school, similar patterns of students were retained at each time point, regardless of the number of supports used. Based on the past retention patterns of this sample, there does not appear to be an explicit pattern explaining why particular schools use differing amounts of support. It is worth mentioning that principals of the highest-poverty schools did not support using only 1 or 2 supports in their schools; high-poverty schools reported using a minimum of 3 supports. This is important because the pressure of accountability mandates may make principals of higher-poverty schools feel more obligated to offer their students as many supports as possible. Considering the current supports in use, it was interesting to examine the past retention practices of these schools. Some may believe that the quantity of support leads to success by proxy; however, we do not have data to support this assumption. From these data, we can see that a larger quantity of support does not necessarily raise achievement for studentsmore is not necessarily better. Poverty level greatly influences the mean predicted retention patterns. Panel 5 suggested that doing less better works; however, disaggregating by poverty level loses that inference. Policy Considerations The third-grade retention policy appears to be working as intended. As FLDOE press releases from 2004 and 2005 each report, there are: More 3 rd graders reading at or above grade level than ever before (Florida Department of Education, 2004b, p. 1; 2004c, p. 1). But are they? Third-grade students began taking the FCAT in 1999-00,

PAGE 109

98 5 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 6 7 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 1999-002000-012001-022002-032003-04Time in Years 0.51.01.52.02.53.0Mean Predicted Values 8 Figure 5-7. Degree of support by mean predicted third-grade retention rates over time disaggregated by poverty group. 5) Mean level of poverty for all sample schools. 6) Sample schools with less than 33.9% poverty. 7) Sample schools with between 34.0 66.9% poverty. 8) Sample schools with more than 67.0% poverty. (Thickest lines equal any 5 or 6 supports; medium thickness lines equal any 3 or 4 supports; thinnest lines equal any 1 or 2 supports)

PAGE 110

99 however their scores were not publicly reported during the first two years. Table 5-1 shows the percent of third graders scoring at Level 1 since 2001-02, the year prior to the start of the state policy. Following the 2001-02 administration of the FCAT, before the retention policy, 29% of third graders achieved Level 1 (Florida Department of Education, 2004c, 2005d). Although the percentage of students scoring Level 1 has decreased three percent, dropping from 23% to 20% since the policy start, the number of third graders has also grown. We need to remember why the number of third graders has grown in post-policy yearsretained third-grade students have bloated enrollment figures, so proportionately it does appear that achievement is on the rise. Appearances can be misleading. With the release of the Greene and Winters (Center for Civic Innovation at the Manhattan Institute, 2004) evaluation of Floridas retention policy then-U.S. Secretary of Table 5-1. Third-grade FCAT reading achievement Level 1 School year Level 1 (%) Number of third graders scoring Level 1 Number of tested third-graders 1999-2000 2000-2001 2001-2002 29 53,980 186,139 2002-2003 23 43,265 188,107 2003-2004 22 45,416 206,435 2004-2005 20 40,595 202,975 Note: Authors own calculations using publicly available data from Florida Department of Education ( http://www.firn.edu/doe/sas/fcat.htm ). Education Rod Paige issued a statement praising the report: I applaud schools in Florida and across the nation that are working to help our children by stopping the dangerous practice of social promotion (U.S. Department of Education, 2004a). Regardless of the

PAGE 111

100 actual impact on reading achievement, the law works as it has eliminated social promotion in third grade. Even with these accolades, several concerns are masked within this policy. First, how are third-grade students who score Level 2 supported as they progress through the grades? These are the students on the fringe. Though they did not fail the FCAT in the eyes of the policy, Level 2 students scored with limited success, and technically have not passed the assessment. While it is not advocated here to hold back all students who do not pass the FCAT, we must seriously consider the consequences for both those students who are retained and those who just make it to fourth grade. As we learned in the review of the Florida third-grade retention policy, the FCAT has had an objective impact on retention practices. Prior to the policy, if retention was chosen, it was a subjective decision, including some combination of the classroom teacher, principal and parent to make a retention decision. Some schools may have relied upon test scores and classroom performance; however, these decisions were decided independently, varying from district-to-district, or even school-by-school. With the retention policy, when a third-grade student does not pass the FCAT, the score guides this decision. Teachers and principals have tangential involvement unless a promotion for good cause, where applicable, goes into effect. We need to remember that over 27,000 of Floridas third-grade students were subject to the retention policy after the first year. After experiencing retention, 41% of retained students again failed the FCAT after their second year in third grade (Florida Department of Education, 2004c). I uncovered several unintended consequences when promotions are the result of a good cause exemption. Local school districts maintain control over how many of these

PAGE 112

101 good cause exemptions are implemented. However, in the first year of the retention policy, there were generous amounts of flexibility where districts could exercise some degree of freedom in how they exercised the good cause exemptions. Let us revisit the good cause exemption data presented in table 2-1 and possible implications from the promotional patterns. The newly implemented good cause portfolios were designed to help students who had demonstrated consistent readiness to be promoted to fourth grade; however, due to poor test scores, those students could not be promoted. The classroom teacher with the approval of the school principal and later the district superintendent were able to use a good cause portfolio as a means to by-pass retention. Nearly 1,500 retained third graders were promoted to the fourth grade after the first year of the policy with a good cause portfolio, while after the second year of the policy more than double the number (almost 3,500 retainees) were promoted under the good cause exemption. There are several possible reasons for this increase; however, the most likely may simply be timing. In the first year of the policy, the FLDOE announced the portfolio option only weeks before the close of the first policy year. It is possible that there were students who may have been eligible for this exemption, but due to circumstances, did not receive this option. In the second year of the policy, schools were able to lead a more coordinated effort to begin the good cause portfolio process for at-risk students closer to the beginning of the year. Three times the number of students (n = 1924) were promoted to fourth grade because they had already been retained twice with two or more years of remediation from the first to the second year of the policy. Because the 2004-05 school year is the third school year since this policy start, it is anticipated that there will be a substantial increase

PAGE 113

102 again with this good cause exemption. This is because third graders who were retained and subsequently re-retained have now met the third-grade retention limit and will automatically be promoted to fourth grade, regardless of FCAT performance. Because of the high stakes involved in these retention decisions, research needs to be expanded to the primary grades (K to 3) to study Floridas retention patterns. Are primary grade students being retained in earlier grades to thwart third-grade retention? Or are lower-achieving primary grade students being socially promoted because the new state policy ensures they will be retained in third grade unless their achievement improves? Retention decisions for students in grades K to 2 remain at the discretion of the individual school and district. Also startling is the use of the exemption for SWD who have been retained one time. You may recall that for students to be promoted for good cause to fourth grade, in most cases, the student must be retained at least one time before third grade. The number of SWD who were promoted for good cause more than doubled, increasing from about 3,600 students to nearly 8,000. Prior to the Florida retention policy, research indicated an expanded use of placement in special education programs as a means of meeting accountability pressures (Figlio & Getzler, 2002). Other studies conducted outside Florida have examined retention trends to demonstrate that increasing numbers of students are being placed in special education programs to avoid accountability scrutiny (Allington & McGill-Franzen, 1992; Roderick & Nagaoka, 2005; Roderick et al., 2000). However, we need to further expand research in Florida to understand whether this good cause promotion is used to game the system (Figlio & Getzler, 2002). Even still, national policy has a rule, often called the 2% rule, limiting the percent of students eligible for alternative assessments as part of AYP (Elmore, 2002). Schools

PAGE 114

103 or districts who might be tempted to circumvent the state retention policy by placing more students in special education programs; yet, in the end, all sub-groups of students, including those placed in special education, will have their achievement score disaggregated in AYP calculations. To meet AYP, schools and districts must test, on average, 95 percent of the population to meet the criteria for these calculations (U.S. Department of Education, 2004b). Interestingly, one of the least widely used good cause exemptions is the ability for the retained students to demonstrate proficiency on an alternative assessment. After the first year, about 3,300 retainees were promoted using this option, while just over 3,800 were promoted the following year based on their performance on an alternative assessment. Only students who attended summer school were eligible to take the alternative assessment. As explained in chapter 2, for students who take the SAT-9, they must achieve at the 51 st percentile 1 (something that their Level 2 scoring peers were not held accountable to pass). As retained students progress through the state system, we must continue to study the long-term effects of this policy, including dropout patterns. This retention policy may have in interesting impact on dropout rates in the years to come. By removing a segment from the 2002-2003 cohort of third-grade students (who were first impacted by the state policy), we await Floridas graduation rates for 2012. This is the year that students first affected by the third-grade retention policy would have been scheduled to graduate from high school. Time will tell whether we will see a marked improvement in graduation rates by shifting students who failed the third-grade FCAT to a different cohort. 1 Student taking the SAT-10 must achieve at the 45 th percentile or higher.

PAGE 115

104 Policy Recommendation Floridas third-grade retention policy (Chapter 2) strived to end social promotion by identifying K to 3 students who lack the proficiency to pass the reading portion of the third-grade FCAT and consequently failing the third graders who do not pass the FCAT. In fact, my study found that Floridas retention policy has successfully eliminated social promotion in third grade producing students who should be prepared to enter fourth grade reading on (or close to) grade level. In other words, the policy was intended to solve the problem of students not reading on grade level; however, data show that policy is not accomplishing that goal since post-policy retention rates increased, on average, 300% since the start of the policy (Florida Department of Education, 2005e). In a 2004 memorandum issued by the FLDOE, Florida school districts were provided a lengthy list of requirements as outlined in the Intent to help students meet the policy goal (Warford & Openshaw, 2004). While the Intent does support identifying struggling students before third grade in order to intervene in ways that would reduce the likelihood of later mandated retention in grade, one may wonder how these identified students are also supported when school is not in session. That is, schools must offer summer school, but only to third graders who fail the reading portion of the FCAT. It seems that a key element is missing from the Intent menu of options for schools to select in efforts to support their struggling readers: Summer school should be available to targeted students prior to third grade. Chicagos retention researcher found that students retained in third and sixth grades were actually identified as struggling readers as early as first grade (Roderick & Nagaoka, 2005). Research in Baltimore and Michigan also support increased attendance in summer school programs for struggling readers. In Baltimore, Borman, Benson and

PAGE 116

105 Overman (2005) found a positive relationship between students who regularly attended summer school and student achievement. In other words, students who spent a greater number of weeks in a summer school program showed academic improvements; while in Michigan, researchers studying summer school programs for lower-achieving K to 3 students found significant benefits for these students (Paris et al., 2004). While Roderick and Nagaoka (2005) suggest that alternative intervention approaches, such as summer school for identified struggling readers, can be an expensive prospect, it may not be as costly in the long run when the expenditure of retaining a student for one or more years is considered. Each time a student is retained there is an added cost to educate that pupil. As of 2003-04, the total per pupil expenditure for regular education students in Florida was about $4800. That, then, is the cost of retaining each student. Why not seek an early return on the investment? The FLDOE should consider redirecting funding streams to pay for summer school for early primary grade students, which may help alleviate the need to retain students in third grade. Since Florida already has the READ initiative in place to identify students earlier in their schooling, it appears prudent that the State fund summer school for the most vulnerable early primary grade students in an effort to help students get back on track. As we can see, waiting until students fail before offering summer school as a support option contradicts research (Borman, Benson & Overman, 2005; Paris et al., 2004; Roderick, Jacob, & Bryk, 2004; Roderick & Nagaoka, 2005); however there is another issue on hand, called summer setback, that may also help support the case for extending the school year for those students most at risk for retention. Researchers have found that students from higher-poverty backgrounds may experience summer setback, or

PAGE 117

106 summer reading loss, when after each summer they begin to trail months behind their peers with greater economic advantages (Cooper, 2004; Heyns, 1978; Kim, 2004; McGill-Franzen & Allington, 2001; Phillips & Chin, 2004). The faucet theory offers one possible explanation for summer setback (Entwisle, Alexander, & Olsen, 2001). The academic faucet is turned on during the school year when learning opportunities are more abundant; whereas, in the summer, the academic faucet is turned off when fewer leaning opportunities exist. After comparing test scores to detect gains and losses between higherand lower-poverty students, Entwisle, Alexander and Olsen (2004) found seasonal achievement differences when comparing assessment scores over time. For students from higher-poverty backgrounds, they tended to lag behind their advantaged peers during the summer months. In fact, they had a tendency to begin each new school year behind students from lower-poverty backgrounds. Therefore, it appears that turning off the academic faucet negatively affects students from higher-poverty backgrounds. Students who lose bits and pieces of the previous years learning experience summer setback (Heyns, 1978). However, just how much learning is lost? For some students, summer setback can amount to about two years of loss over a typical school career (Paris et al., 2004). Increasing the amount of reading practice provided all year long may help stave off the effects of summer setback and in turn, may help boost student achievement and even reduce retention rates. For some students, it may come down to limited access to print materials or opportunities for enrichment. During the summer students from high-poverty backgrounds may simply lack access to reading materials (McGill-Franzen & Allington, 2003). Students from higher-poverty backgrounds tend to

PAGE 118

107 have fewer opportunities to engage in reading or other academic activities, such as visiting a local library or museum, during the summer months (Phillips & Chin, 2004). Why not try to circumvent retention by keeping the faucet turned on during the summer months for students most at risk for retention? Waiting until students fail is too late to provide this much needed support. Offering summer school at the earliest point possible may help thwart retention. It seems that if schools are working to identify struggling students, then it would logically follow that these students be provided timely interventions. Because we know that Florida students from higher-poverty backgrounds have an increased likelihood to experience a mandated third-grade retention, it seems as important as ever to try to recoup losses due to summer setback by providing a summer school program. While school day efforts to help identified struggling readers is necessary, it may not be enough for some students. Summer school expands learning time, plus it may help minimize summer setback. Future Research Many intriguing issues emerged from this research. Arguably, the most relevant was methodological. For research questions to be answered with a multilevel model, it is important to choose the correct model for the distribution and to address the model assumptions. My study underwent several analyses to determine the correct transformation of the data. Because these data clearly suggest a nonlinear trend, future research needs to consider analyzing a truly nonlinear model in which the model is adjusted. For this study, the data was adjusted to fit the model using the logarithmic transformation plus 1 unit. Further improvements might be made to address the high incidence of zero-values pre-policy.

PAGE 119

108 In Florida, double retention in third grade is a relatively new phenomenon. Research can continue to monitor promotional patterns as students progress through their schooling. Like Chicago (Allensworth, 2005; Roderick et al., 2000; Roderick & Nagaoka, 2005), research in Florida can study the impact retention has on struggling students learning how they are supported, especially as they reach middle and high school. After promotion to fourth grade, in what ways does the educational system continue to support retainees and double-retainees ensuring equal access to guidance and support throughout their schooling? The Florida Principal Survey provided principals with the opportunity to report upon current practices within their schools. This information serves as a baseline for future research to follow after the state releases the 2004-05 retention rates. We have learned from these analyses about the quantity of the supports available to at risk students. Perhaps we may be able to determine whether we can predict which types of supports work in certain kinds of schools. Other avenues require more research beyond the scope of the data collected for this study. Considering that principals retention beliefs did not impact the multilevel model building in this study suggests that further study in the area of beliefs, or underlying assumptions, is needed, especially related to retention policy enactment. One approach could be to develop a construct to determine the underlying perceptions principals have about retention. Insights from such a construct may help glean reasons why believers and nonbelievers retained similar numbers of students before and after policy implementation.

PAGE 120

109 Conclusion Floridas retention policy has eliminated social promotion in third grade. After reviewing Floridas legislative intent, we know the policy established third grade as the gateway year. The cornerstone of the policy ending social promotion for third-grade students was identifying those students who need a stronger literacy foundation and providing them with extra time and support. However, we must take heed of the numerous supports suggested for schools to implement and consider each student as an individual to determine the best way to support student learning. While most educators and policymakers alike strive to find ways to raise reading achievement and develop stronger readers, we must show prudence in trying anything that has been shown to work in research, as there is a chance that similar results may not be universal. Simply because an intervention has been proven to work with one population of students does not ensure that it will work with all students. As should be plainly obvious, all schools, teachers, and students therein are unique and bring different strengths to the forefront. It is important that we remember the individualism that makes each school, teacher, and student unique. Furthermore, as Elmore (2002) once stated, "low-performing schools, and the people who work in them, don't know what to do. If they did, they would be doing it already" (pp. 33-34). By examining the degree of support in Florida schools, we took stock of the practices of schools, so that future research may assess its benefit. Ultimately, what matters most is ensuring students, regardless of their principals retention belief, or the schools poverty level, receive access to the support they need to be successful. As with many challenges in education, there are no easy answers or quick fixes (Allington & Walmsley, 1995); however, we must persist to understand

PAGE 121

110 what makes a difference for each student. Because there are so many variables, it is difficult to isolate what truly makes a difference. It is for this reason that multilevel modeling was selected for my study. It allowed me to focus on testing the best fit of particular variables, as opposed to merely testing hypotheses. Nevertheless, we, researchers and educators alike, must continue to persevere examining how to best support students in the high-stakes policy environment in which we live. I close with a passage from Laggards in our schools (Ayers, 1909) It does not make much difference what we have to do, whether it is a great thing or a little thing, so long as we feel it is possible for us and that we can do it if we try. There are few more hopeless things in the world than to have it borne in upon us that we are driving against a thing that we cannot do. Yet this is the sort of training that we are giving a large part of our children (p. 220). Ayers was a visionary. We must consider his cautions as we continue studying how to best support those students who struggle with reading.

PAGE 122

APPENDIX A FLORIDA PRINCIPAL SURVEY

PAGE 123

Florida Principal Survey To help with our record keeping, please answer the following questions: 1. If you have the Identification Number we provided to you earlier, please enter it below. If not, you may leave this blank. ___________________ 2. Your School District ___________________________________________ 3. Your name (last name, first name) ________________________________ 4. Your School __________________________________________________ Thank you for participating in this survey. The information gathered will help us analyze how principals interpret and implement the third grade retention policy enacted in 2003. Your opinions are important to us. The information you provide in this survey will remain strictly confidential and will NOT be shared with your school district or the State. We will not refer to specific individuals or school districts by name. School districts will be referred to as small, mid-size, or large. Only members of our research team will have access to your responses. Your participation in this survey is voluntary. A final report will be available next summer. This survey should take approximately 10 15 minutes to complete. If you are have any questions or need your ID number, please e-mail Courtney Zmach (czmach@ufl.edu) for assistance. Thank you for joining with us in this project. By completing and mailing this survey, you are stating that you have read the enclosed informed consent letter, and voluntarily agree to participate in this survey. 112

PAGE 124

113 Directions. Please answer the following questions based on your current interpretation of the third grade retention policy. Some questions provide a space for an "other" answer on a short line. Feel free to use short words or phrases here. Other questions use a rating scale. If at any time during the survey you need to take a break, for one reason or another, the survey is designed such that you should be able to pick up where you left off. For this, you must be on the same computer. Again, your time is most appreciated! Sources of information. Please respond to the following questions. 5. How useful are the following in providing assistance for implementing the retention policy? (Check only one per row) Not useful Somewhat useful Mostly useful Perfectly useful N/A Florida DOE Local School District Support from Assistant Principal Support from third grade teachers Other Media Source (Internet, News, Radio) 6. How important is it for you to stay abreast of legislative changes to retention policies? o Very Important o Somewhat Important o Slightly Important o Not Important Your experiences. Think about your personal experiences when implementing this policy.

PAGE 125

114 7. Have you felt resistance to the retention policy from the following groups? Yes No Community Groups (Church, Business, etc.) Your Teachers (excluding third grade teachers) Third Grade Teachers Other School Based Personnel (Asst. Principal, Curriculum Specialist) Other School Based Personnel (Teaching Assistants, Paraprofessionals) District Staff Parents 8. In your opinion, who should have the "final say" in retention decisions? o Parent o Teacher o Principal o Superintendent o Other (please specify) ______________________________________ 9. Knowing that individual circumstances can exist, what is your response to the following statement? "On the whole, retention in grade can benefit 3rd grade students who score Level 1 on the FCAT" o Yes, retention is a beneficial option for students who fail the FCAT. o No, retention is not a beneficial option for students who fail the FCAT. Good cause exemption portfolios. Think about the good cause exemption portfolios that were decided upon at the end of the 2003-2004 school year. 10. Was there enough time for you to review your students' "good cause" portfolios? o Yes o No o N/A I did not have ANY good cause portfolios to review. o N/A I was not a principal at this school last year. 11. What model did you use for your good cause exemption portfolio? o A portfolio was developed by the State. o A portfolio was developed by our school district. o We use a portfolio developed by another school district. o We use a portfolio developed by another school in our district.

PAGE 126

115 o We developed our own portfolio at our school. o Individual teachers decide what to include in the portfolio. o Not sure How much is this like you? 12. Please respond to the following questions using the scale provided. Where applicable, use the time-frame provided to help gauge your response. 1 Not at all like me 2 3 Somewhat like me 4 5 Very much like me I rely on other people to help me understand the retention policy. I believe ability grouping is important to help monitor our retained and re-retained third grade students. I am comfortable with the speed in which decisions are made regarding the retention policy (at the state level). I am comfortable with the speed in which decisions are made regarding the retention policy (at the district level). I am comfortable with the speed in which decisions are made regarding the retention policy (at the school level). I consult with the FCRR (FL Center for Reading Research @ FSU) website regularly (at least 2 times per month). District officials follow-up with me regularly (at least 1 time per month) to ensure implementation of the retention policy. I feel I am part of the decision making process for this policy. I consult the FLDOE website regularly (at least 2 times per month) to stay current with new retention laws and regulations. I read recommendations from professional journals, such as Reading Teacher, Reading Research Quarterly, Florida Reading Quarterly

PAGE 127

116 I attend state and national conferences, such as IRA (International Reading Assoc.) or ASCD (Assoc. for Supervision and Curriculum Development. Resources for at-risk students. 13. In 2004-2005, how often does your school plan to assess the reading ability of K-3 students? o Weekly o Bi-weekly o Monthly o Quarterly o Other (please specify) ___________________________________ 14. Think about programs or initiatives currently in place at your school designed to help at-risk students. Yes No Not Sure Is the Reading Enhancement and Acceleration Development (READ) Initiative available to students in your school? Does your school emphasize early intervention programs in K 2 to prevent retention in 3rd grade more now than in the past? Do ALL retained 3rd grade students in your school have a current AIP? Do at-risk 3rd graders have access to tutors or mentors trained in reading instruction? Is a transitional class (a class designed for the retained and/or re-retained) available for retained students? 15. Since the retention policy began in 2003, would you say that you are retaining fewer, about the same, or more students in Grades K 2 to help prepare students for the 3rd grade FCAT? Fewer than before policy Same since policy enacted More than before policy Have not compared Kindergarten First Second

PAGE 128

117 16. Since the retention policy began in 2003, would you say that you are retaining fewer, about the same, or more students in Grade 3 ? Fewer than before policy Same since policy enacted More than before policy Have not compared Summer reading camps. Please think back to the summer reading camp held in your district. 17. Who decided the Summer Camp reading curricula for the retained third graders? (Check all that apply.) Summer School site principal Team of District Principals District curriculum leaders Individual teachers Not Sure 18. Were you a Summer Reading Camp site principal? o Yes o No Support for the retained. 19. Which, if any, of the following are used as strategies with retained third grade students? (Check all that apply) o A mentor or tutor with specialized reading training o Extended school day (such as After School Program, Saturday School, or Extended School Year) as defined in the NCLB legislation o Ongoing portfolios that meet state portfolio requirements o None of the above 20. How often does your school plan to review AIP's (Academic Improvement Plan) for retained 3rd graders? o Weekly o Bi-Weekly o Monthly o Quarterly o Not Yet Determined o Other (please specify) ________________________________

PAGE 129

118 21. Are there instructional options that you would prefer to use with retained students, but have not been able to implement for one reason or another? If yes, please share your ideas below. Third grade teachers. Please respond to the following questions regarding your current third grade instructional staff. 22. What factors would you use to determine whether a 3rd grade teacher is "high-performing"? (Check all that apply) o District identified "high-performing" teachers. o Student performance data o Above-satisfactory on performance appraisals o Teacher has completed the Reading Endorsement. o Teacher has completed a Reading Certification program. o Not Sure o Other (please specify) ________________________ 23. Think about your current third grade teachers. How many 3rd grade teachers are NOT TENURED (have four years or less teaching experience)? Please enter the number in the space provided. ____________________ 24. How many third grade teachers does your school currently have? Please enter the number in the space provided. _____________________ Funding. Think about the amount of funding your school receives and the adequacy to meet the financial needs to support your retained students. 25. How are decisions made in your school on how to fund new initiatives and programs for the retained? Who is involved in the decision making process? 26. Who do you believe should be involved in the funding decision making process? _________________________________________

PAGE 130

119 Information. Please write the best response to the following questions about your educational background and the number of students "re-retained". 27. How many years have you been in education? Please include all teaching and administrative experiences. _________________ 28. How many years have you been a principal? (Including 2004-2005 school year) _________________ 29. How many years have you been principal at your current school? (Including 2004-2005 school year) _________________ 30. How many students were re-retained (retained for the second time) in third grade? Please enter a numeral in the space provided. __________________ If you are unsure, please leave this question blank. Thank you! Thank you again for completing this survey. We will be contacting some of you for telephone interviews. Wishing you, your staff and your students a wonderful school year! 31. OPTIONAL: Please feel free to add any additional comments or feedback about the retention policy that you feel are important.

PAGE 131

APPENDIX B REVIEW OF SURVEY PILOT STUDY A pilot study of the closedand open-ended questions established face, content and logical validity, thus ensuring that the content asked accurately reflected the issues that principals believe to be important, and that the questions and wording were clear to principals (Litwin, 1997). Face validity was established by sharing drafts of the survey with colleagues. Their consultation provided the researcher multiple perspectives and unexpected advice. After carefully considering their input, modifications resulted in reducing the number of items asked on the survey, prompting reflection about rating scales and reducing the number of open-ended questions found in early survey drafts. Content and logical validity were established with practicing principals. Two think-aloud interviews were conducted with principals while they took the survey (Desimone & Le Floch, 2004); three volunteers completed the survey and provided written feedback after taking the survey. After analyzing the pilot data, the sample responses were considered in conjunction with the research questions. It was determined that the survey required modification to eliminate questions related to the topic of retention, but not directly related to the research questions of this study. Some questions were reworded using less complicated language. After considering the multitude of reactions from colleagues and pilot participants, including think-aloud interviewees, changes were made to enhance the face, logical and content validity of the survey instrument. This process increased the chance that when principals took the finalized version of the survey their responses 120

PAGE 132

121 would be more closely aligned with their intended meaning. Overall, pilot respondents communicated positive attitudes toward the survey.

PAGE 133

APPENDIX C DATA SOURCES Table C-1. Sources of data School & District Level Data Source School Year Third-grade enrollment by school and district Florida Survey 2 Florida Department of Education (FLDOE) 1999-00, 2000-01, 2001-02, 2002-03, 2003-04 Third-grade retention/ non-promotion by school and district Florida Survey 5 (FLDOE) 1999-00, 2000-01, 2001-02, 2002-03, 2003-04 Free/reduced lunch by school FLDOE 1999-00, 2000-01, 2001-02, 2002-03, 2003-04 Enrollment by school and district FLDOE Schoolresults.org a 1999-00, 2000-01, 2001-02, 2003-04 2002-03 School Type/enrollment NCES/Common Core of Data 2002-03 b Title I recipients FLDOE website 2004-05 b Reading First recipients Just Read FL! office 2004-05 b a Information related to Florida posted at www.schoolresults.org were provided by the FLDOE. b Most current available information from these sources was used in my study. 122

PAGE 134

APPENDIX D SAMPLE FROM STATISTICAL SYNTAX if time=1 twt1 = -1. 1 if time=2 twt1 = 0. if time=3 twt1 = 1. if time=4 twt1 = 0. if time=5 twt1 = 0. if time=1 twt2 = 0. if time=2 twt2 = 0. if time=3 twt2 = 0. if time=4 twt2 = -.5. if time=5 twt2 = +.5. execute. title "MODEL A: RA / Unconditional Means Model (Degenerate)". MIXED logper_ret_plus1 /CRITERIA = CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED = /METHOD = REML /PRINT = G SOLUTION TESTCOV /RANDOM INTERCEPT | SUBJECT(New_ID) COVTYPE(VC) /REGWGT = lgenrol title "MODEL B: Unconditional Growth Model (UGM)". MIXED logper_ret_plus1 WITH PrePost_Policy /CRITERIA = CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED = PrePost_Policy | SSTYPE(3) /METHOD = REML /PRINT = G SOLUTION TESTCOV /RANDOM INTERCEPT | SUBJECT(New_ID) COVTYPE(VC) /REGWGT = lgenrol 1 Syntax created using SPSS 12.0 for Windows. 123

PAGE 135

APPENDIX E RESULTS TABLE FOR ALL MODELS

PAGE 136

Table E-1. Results of fitting a taxonomy of multilevel models for change to the log percent retained data, Models A through F2 (n = 102) A B C D E F1 F2 Intercept 1.66*** 1.17*** .463*** .469*** .465*** .496*** .521** Pre-Post Policy 1.14*** 1.13*** 1.10*** 1.11*** 1.11*** 1.11*** sesmean .014*** .014*** .014*** 1.37*** 1.397*** sesdiff .022** .020** 1.95** 1.952** twt1 .034 -0.226 .033 twt2 -0.226** -0.047** -0.226** Q9 .035 suppsys3grp -0.029 Residual 3.72*** 1.81*** 1.82*** 1.78*** 1.76*** 1.75*** 1.75*** Intercept .064~ .155*** .056** .054** .055** .056** .056** Between .064 .156 .056 .054 .055 .056 .056 Within 3.72 1.81 1.82 1.78 1.76 1.76 1.76 TOTAL 3.79 1.96 1.87 1.83 1.81 1.81 1.81 Proportionate reduction in error PRE-Residual .515 -0.005 .015 .029 .029 .029 PRE-Intercept -1.430 .637 .656 .648 .642 .641 Relative to ModelA B B B B B Intra-class correlation coefficient ICC-Between .017 .079 .030 .029 .030 .031 .031 ICC-Within .983 .921 .970 .971 .970 .969 .969 Goodness of Fit AIC 1279.43 1012.81 966.34 962.86 962.89 947.36 948.03 AIC difference 266.62 313.09 316.57 316.54 332.07 331.40 Fit improvement (%) 20.8 24.5 24.7 24.7 25.95 25.90 Relative to Model A A A A A A ~ p < .10; p < .05; ** p < .01; *** p < .001 125

PAGE 137

LIST OF REFERENCES Alexander, K. L., Entwisle, D. R., & Olsen, L. S. (2004). Schools, achievement, and inequality: A seasonal perspective. In G. D. Borman & M. Boulay (Eds.), Summer learning: Research, policies and programs (pp. 25-51). Mahwah, NJ: Erlbaum. Allen, J., Cary, M., & Delgado, L. (1995). Exploring blue highways: Literacy reform, school change, and the creation of learning communities. New York: Teachers College Press. Allensworth, E.M. (2005). Dropout rates after high-stakes testing in elementary school: A study of the contradictory effects of Chicago's efforts to end social promotion. Educational Evaluation and Policy Analysis, 27(4), 341-364. Allington, R. L. (2001). Does state and federal policymaking matter? In T. Loveless (Ed.), The great curriculum debate: How should we teach reading and math? (pp. 268 298). Washington, D.C.: Brookings Institution Press. Allington, R. L. (2002a). Research on reading/learning interventions. In A. Farstrup & S. J. Samuels (Eds.), What research has to say about reading instruction (3rd ed., pp. 261-290). Newark, DE: International Reading Association. Allington, R. L. (Ed.). (2002b). Big brother and the national reading curriculum: How ideology trumped evidence. Portsmouth, NH: Heinemann. Allington, R. L., & McGill-Franzen, A. (1992). Unintended effects of educational reform in New York State. Educational Policy, 6(4), 397-414. Allington, R. L., & McGill-Franzen, A. (2000). Looking back, looking forward: A conversation about teaching reading in the 21st century. Reading Research Quarterly, 35(1), 136-153. Allington, R. L., McGill-Franzen, A., & Schick, R. (1997). How administrators understand learning difficulties. Remedial and Special Education, 18(4), 223-232. Allington, R. L., & Walmsley, S. A. (Eds.). (1995). No quick fix: Rethinking literacy programs in America's elementary schools. New York: Teachers College Press. American Educational Research Association. (2000). AERA position statement concerning high-stakes testing in PreK-12 education. Retrieved November 12, 2002, from http://www.aera.net/policyandprograms/?id=378 126

PAGE 138

127 Ames, L. B. (1986). Ready or not. American Educator, 10(2), 30-33. Ayers, L. P. (1909). Laggards in our schools. New York: Russell Sage Foundation. Ball, D. L. (1990). Reflections and deflections of policy: The case of Carol Turner. Educational Evaluation and Policy Analysis, 12(3), 247-259. Balow, I. H., & Schwager, M. (1992). Retention in grade: A failed procedure. In R. Reitz (Ed.), Retention in grades: Looking for Alternatives (pp. 9-49). Bloomington, IN: Phi Delta Kappa Center for Evaluation, Development and Research. Barnett, K. P., Clarizio, H. F., & Payette, K. A. (1996). Grade retention among students with learning disabilities. Psychology in the Schools, 33(4), 285-293. Baumann, J. F., & Bason, J. J. (2004). Survey research. In N. K. Duke & M. Mallette (Eds.), Literacy methodologies (pp. 287-307). NY: Guilford. Borman, G.D., Benson, J., & Overman, L.T. (2005). Families, schools, and summer learning. Elementary School Journal, 106(2), 131-150. Bracey, G. W. (2003). Poverty issues get short shrift. In On the death of childhood and the destruction of public schools (pp. 4548). Portsmouth, NH: Heinemann. Bryk, A. S., & Raudenbush, S. W. (1987). Application of hierarchical linear models to assess change. Psychological Bulletin, 101, 147-158. Bush, G. W. (2001). No Child Left Behind Act of 2001. Retrieved February 5, 2003, from United States Department of Education Web site: http://www.ed.gov/nclb/overview/intro/execsumm.html Byrnes, D. A. (1989). Attitudes of students, parents, and educators toward repeating a grade. In L. A. Shepard & M. L. Smith (Eds.), Flunking grades: Research and policies on retention (pp. 132-150). London, England: Falmer Press. Camilli, G., Wolfe, P., & Smith, M. L. (in press). Meta-analysis and reading policy: Perspectives on teaching children to read. Elementary School Journal. Center for Civic Innovation at the Manhattan Institute (2004). An evaluation of Florida's program to end social promotion. (Education Working Paper No. 7). Retrieved January 2, 2005, from http://www.manhattan-institute.org/html/ewp_07.htm#01 Chase, J. A. (1968). A study of the impact of grade retention on primary school children. Journal of Psychology, 70, 169-177.

PAGE 139

128 Clinton, W. J. (1999, January 20). 1999 State of the Union Address. Washington Post, A12. Retrieved September 13, 2004, from http://www.washingtonpost.com/wp-srv/politics/special/states/docs/sou99.htm Cooper, H. (2004). Is the school calendar dated? Education, economics, and the politics of time. In G. D. Borman & M. Boulay (Eds.), Summer learning: Research, policies and programs (pp. 3-23). Mahwah, NJ: Erlbaum. Cooper, H., & Hedges, L.V. (Eds.). (1994). The handbook of research synthesis. New York: Russell Sage Foundation. Darling-Hammond, L. (1990). Instructional policy into practice: "The power of the bottom over the top". Educational Evaluation and Policy Analysis, 12(3), 339-347. Desimone, L. M., & Le Floch, K. C. (2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26(1), 1-22. Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (2nd ed.). New York: Wiley. Dorn, S. (2004). Alternatives for Florida's assessment and accountability systemPolicy brief. Tempe, AZ: Education Policy Studies Laboratory, Arizona State University. Elmore, R. (2002). Unwarranted intrusion. Education Next, 2, 31-35. Retrieved September 23, 2003, from http://www.educationnext.org/20021/30.html Entwisle, D. R., Alexander, K. L., & Olsen, L. S. (2001). Keep the faucet flowing: Summer learning and home environment. American Educator, 25(3), 10-15, 47. Erpenbach, W. J., Forte-Fast, E., & Potts, A. (2003). Statewide educational accountability under NCLB: Central issues arising from an examination of state accountability workbooks and U.S. Department of Education reviews under the No Child Left Behind Act of 2001. Washington, DC: Council of Chief State School Officers. Figlio, D.N. & Getzler, L.S. (2002). Accountability, ability, and disability: Gaming the system (NBER Working Papers No. 9307). Retrieved on January 10, 2006, from http://www.nber.org/papers/w9307.pdf Flinders, D. J., & Thornton, S. J. (Eds.). (1997). The curriculum studies reader. New York: Routledge. Florida Department of Education. (2000a). FCAT achievement levels. Retrieved March 20, 2003, from http://www.firn.edu/doe/sas/fcat/pdf/fcatlevl.pdf

PAGE 140

129 Florida Department of Education. (2000b). Technical assistance paper: Standards for implementation of student assessment programs in Florida school districts. Retrieved June 3, 2005, from http://www.firn.edu/doe/sas/pdf/ethics.pdf Florida Department of Education. (2003a). Q&A technical assistance paper DPS #03-024: Student progression requirements for 2002-2003. Retrieved March 1, 2003, from http://info.fldoe.org/docushare/dsweb/Get/Document-435/TAP17702b1.PDF Florida Department of Education. (2003b). Total number of planned purchases for all districts, reading, grade K-5. Retrieved February 17, 2004, from Florida Instructional Materials Reporting Web site: http://www.fimweb.org/reports/default.asp Florida Department of Education. (2003c). Understanding FCAT reports 2003. Retrieved February 20, 2004, from http://www.firn.edu/doe/sas/fcat/fcinfopg.htm Florida Department of Education. (2004a). 2004 legislation CS/SB 364: Reading deficiencies chart. Retrieved June 9, 2004, from http://info.fldoe.org/dscgi/ds.py/Get/File-2130/04-132.pdf Florida Department of Education. (2004b). Press release: Governor Bush and Education Commissioner Winn announce plan to assist schools affected by hurricanes. Retrieved September 29, 2004, from http://www.fldoe.org/news/2004/2004_09_21-2.asp Florida Department of Education. (2004c). Press release: Governor Bush, Lt. Governor Jennings and Commissioner Horne announce preliminary FCAT results for 3rd and 12th grade. Retrieved April 19, 2004, from http://www.fldoe.org/news/2004/2004_04_19.asp Florida Department of Education. (2004d). Technical assistance paper #12315: Good cause exemptions (FY2004-17). Retrieved June 9, 2005, from http://www.firn.edu/doe/bin00014/pdf/y2004-17.pdf Florida Department of Education. (2005a). Florida school indicators report. Retrieved May 30, 2005, from http://data.fldoe.org/fsir/default.cfm Florida Department of Education. (2005b, May 9). Governor Bush announces Reading First grant recipients. Retrieved June 20, 2005, from http://www.fldoe.org/news/2005/2005_05_09.asp Florida Department of Education. (2005c). Welcome page: Third-grade reading promotion and retention. Retrieved June 8, 2005, from http://www.firn.edu/doe/commhome/progress/proghome.htm

PAGE 141

130 Florida Department of Education. (2005d). Press release. Retrieved May 25, 2005, from http://fcat.fldoe.org/mediapacket/pdf/05gr3gr12pressrelease.pdf Florida Department of Education (2005e). Non-promotions in Florida's public schools, 2003-04. Retrieved September 12, 2005, from http://www.firn.edu/doe/eias/eiaspubs/pdf/nonpromotions.pdf Florida Statutes. (2002). Florida K 20 Education Code: Assessment and accountability: TITLE XLVIII, Chapter 1008.25, Retrieved March 10, 2005, from The Florida Senate Web site: http://www.flsenate.gov/statutes/index.cfm?mode=View%20Statutes&SubMenu=1&App_mode=Display_Statute&Search_String=&URL=CH1008/Sec25.HTM Fullan, M. (2001). The new meaning of educational change. New York: Teachers College Press. Fullan, M. (2002). The change leader. Educational Leadership, 59(8), 16-21. Good, R.H., & Kaminski, R.A. (Eds.) (2002). Dynamic Indicators of Basic Early Literacy Skills (6 th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Goodlad, J. I. (1952). Research and theory regarding promotion and nonpromotion. Elementary School Journal, 53(3), 150-155. Graue, M. E., Kroeger, J., & Brown, C. (2003). The gift of time: Enactments of developmental thought in early childhood practice. Early Childhood Research and Practice, 5(1). Retrieved September 25, 2004, from http://ecrp.uiuc.edu/v5n1/graue.html Grissom, J. B., & Shepard, L. A. (1989). Repeating and dropping out of school. In L. A. Shepard & M. L. Smith (Eds.), Flunking grades: Research and policies on retention (pp. 34-63). London, England: Falmer Press. Guthrie, J. T. (2002). Preparing students for high-stakes test taking in reading. In A. Farstrup & S. J. Samuels (Eds.), What research has to say about reading instruction (pp. 370-391). Newark, DE: International Reading Association. Hall, W. F., & Demarest, R. (1958). Effects on achievement scores of a change in promotional policy. Elementary School Journal, 58(4), 204-207. Hart, G. K. (1996). A policymaker's response. Journal of Literacy Research, 28, 596-601. Heyns, B. (1978). Summer learning and the effects of schooling. New York: Academic Press.

PAGE 142

131 Hiebert, E. H. (2002). Standards, assessments and text difficulty. In A. Farstrup & S. J. Samuels (Eds.), What research has to say about reading instruction (pp. 337-369). Newark, DE: International Reading Association. Holmes, C. T. (1989). Grade-level retention effects: A meta-analysis of research studies. In L. A. Shepard & M. L. Smith (Eds.), Flunking grades: Research and policies on retention (pp. 16-33). London, England: The Falmer Press. Holmes, C. T., & Matthews, K. M. (1984). The effects of nonpromotion on elementary and junior high school pupils: A meta-analysis. Review of Educational Research, 54(2), 225-236. Hox, J. J. (2000). Multilevel analyses of grouped and longitudinal data. In T. D. Little, K.U. Schabel & J. Baumert (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 15-32). Mahwah, NJ: Erlbaum. Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum. International Reading Association. (1999). Position statement: High-stakes assessments in reading. Retrieved September 20, 2002, from http://www.reading.org/resources/issues/positions_high_stakes.html Isaac, S., & Michaels, W. B. (1995). Instrumentation and measurement. In Handbook in research and evaluation (pp. 105-158). San Diego, CA: Educational and Industrial Testing Services. Jackson, G. B. (1975). The research evidence on the effects of grade retention. Review of Educational Research, 45(4), 613-635. Jimerson, S. R. (2001). Meta-analysis of grade retention research: Implications for practice in the 21st century. School Psychology Review, 30(3), 420-437. Jimerson, S. R., Carlson, E., Rotert, M., Egeland, B., & Sroufe, L. A. (1997). A prospective, longitudinal study of the correlates and consequences of early grade retention. Journal of School Psychology, 35(1), 3-25. Kim, J. (2004). Summer reading and the ethnic achievement gap. Journal of Education of Students Placed At-Risk, 9(2), 169-189. Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. Thousand Oaks, CA: Sage. Light, R.J. & Pillemer, D.B. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.

PAGE 143

132 Lindblom, C. E., & Woodhouse, E. J. (1993). The policy-making process (3rd ed.). Upper Saddle River, NJ: Prentice Hall. Linn, R. L. (2004). Accountability models. In S. H. Fuhrman & R. Elmore (Eds.), Redesigning accountability systems for education (pp. 73-95). New York: Teachers College Press. Litwin, M. S. (1997). How to measure survey reliability and validity. Thousand Oaks, CA: Sage. Lobdell, L. O. (1954). Results of a nonpromotion policy in one school district. Elementary School Journal, 54(6), 333-337. Massell, D. (2001). The theory and practice of using data to build capacity: State and local strategies and their effects. In S. H. Fuhrman (Ed.), From the capitol to the classroom: Standards-based reform in the states (pp. 148-169). Chicago, IL: University of Chicago Press. McElroy, E. (2001). Biology 283 lab: Brain size. Retrieved June 28, 2005, from Western Kentucky University, Department of Evolutionary Biology, Genetics, and Biostatistics Web site: http://bioweb.wku.edu/faculty/McElroy/BIOL283/283-5.htm McGill-Franzen, A. (1987). Failure to learn to read: Formulating a policy problem. Reading Research Quarterly, 22(4), 475-490. McGill-Franzen, A. (2000). Policy and instruction: What is the relationship? In M. L. Kamil, P. B. Mosenthal, P. D. Pearson & R. Barr (Eds.), Handbook of reading research (Vol. III, pp. 889-908). Mahwah, NJ: Erlbaum. McGill-Franzen, A., & Allington, R. L. (1993). Flunk'em or get them classified: The contamination of primary grade accountability data. Educational Researcher, 22(1), 19-22. McGill-Franzen, A., & Allington, R. L. (2001). Summer reading. (importance of giving children access to books throughout summer months). Reading Today, 18(6), 10. McGill-Franzen, A., Zmach, C. C., Solic, K., & Love Zeig, J. (in press). The confluence of two policy mandates: Core reading programs and third-grade retention. Elementary School Journal. McLaughlin, M. W. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9, 171-178.

PAGE 144

133 Moskowitz, D. S., & Hershberger, S. L. (Eds.). (2002). Modeling intraindividual variability with repeated measures data: Methods and applications. Mahwah, NJ: Erlbaum. Musella, D. F. (1989). Problems in policy implementation. In M. Holmes, K. A. Leithwood & D. F. Musella (Eds.), Educational policy for effective schools (pp. 93-111). Toronto, Ontario, Canada: Ontario Institute for Studies in Education Press. Nagaoka, J., & Roderick, M. (2004). Ending social promotion: The effects of retention. Chicago, IL: Consortium on Chicago School Research. National Center for Education Statistics. (2002). School locale codes 1987-2000 (NCES 2002-02). Retrieved September 3, 2004, from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=200202 National Center for Education Statistics. (2004). Common core of data: Build a table. Retrieved October 27, 2004, from http://nces.ed.gov/ccd/bat/ National Commission of Excellence in Education. (1983). A nation at risk: The imperative for educational reform. Retrieved September 20, 2004, from http://www.ed.gov/pubs/NatAtRisk/index.html National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4769). Washington, DC: U.S. Government Printing Office. National Research Council. (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy of Sciences. Oakes, J. (1999). Promotion or retention: Which one is social? Retrieved June 14, 2004, from Harvard Education Letter Web site: http://www.edletter.org/past/issues/1999-jf/promotion.shtml Onwuegbuzie, A.J., & Leech, N.L. (2004). Enhancing the interpretation of "significant" findings: The role of mixed methods research. The Qualitative Report, 9(4), 770-792. Retrieved March 1, 2005 from http://www.nova.edu/ssss/QR/QR9-4/onwuegbuzie.pdf Paris, S. G., Pearson, P. D., Cervetti, G., Carpenter, R., Paris, A. H., DeGroot, J., et al. (2004). Assessing the effectiveness of summer reading programs. In G. D. Borman & M. Boulay (Eds.), Summer learning: Research, policies and programs (pp. 121-161). Mahwah, NJ: Erlbaum.

PAGE 145

134 Phillips, M., & Chin, T. (2004). How families, children and teachers contribute to summer learning and loss. In G. D. Borman & M. Boulay (Eds.), Summer learning: Research, policies and programs (pp. 255-278). Mahwah, NJ: Erlbaum. Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage. Reitz, R. (Ed.). (1992). Retention in grade: Looking for alternatives. Bloomington, IN: Phi Delta Kappa Center for Evaluation, Development and Research. Roderick, M. (1994). Grade retention and school dropout: Investigating the association. American Educational Research Journal, 31(4), 729-759. Roderick, M., Bryk, A. S., Jacob, B. A., Easton, J. Q., & Allensworth, E. (1999). Ending social promotion: Results from the first two years. Chicago, IL: Consortium on Chicago School Research. Roderick, M., & Engel, M. (2001). The grasshopper and the ant: Motivational responses of low-achieving students to high-stakes testing. Educational Evaluation and Policy Analysis, 23(3), 197-227. Roderick, M., Jacob, B. A., & Bryk, A. S. (2004). Summer in the city: Achievement gains in Chicago's summer bridge program. In G. D. Borman & M. Boulay (Eds.), Summer learning: Research, policies and programs (pp. 73-102). Mahwah, NJ: Erlbaum. Roderick, M., & Nagaoka, J. (2005). Retention under Chicago's high-stakes testing program: Helpful, harmful, or harmless? Educational Evaluation and Policy Analysis, 27(4), 309-340. Roderick, M., Nagaoka, J., Bacon, J., & Easton, J. Q. (2000). Update ending social promotion: Passing, retention, and achievement trends among promoted and retained students. Chicago, IL: Consortium on Chicago School Research. Ruhl, K. L. (1984). A primer on retention: History, research and a checklist for making the decision. Gainesville, FL: Multidisciplinary Diagnostic and Training Program at the University of Florida. Scott, B. A., & Ames, L. B. (1969). Improved academic, personal, and social adjustments in selected primary-school repeaters. Elementary School Journal, 69, 431-439. Shepard, L. A. (1989). A review of research on kindergarten retention. In L. A. Shepard & M. L. Smith (Eds.), Flunking grades: Research and policies on retention (pp. 64-78). London, England: Falmer Press.

PAGE 146

135 Shepard, L. A., & Smith, M. L. (Eds.). (1989). Flunking grades: Research and policies on retention. London, England: Falmer Press. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press. Spillane, J. P. (1998). A cognitive perspective on the role of the local educational agency in implementing instructional policy: Accounting for local variability. Educational Administration Quarterly, 34(1), 31-57. Tomchin, E. M., & Impara, J. C. (1992). Unraveling teachers' beliefs about grade retention. American Educational Research Journal, 29(1), 199-223. Torgesen, J. (2004). Information relevant to the challenge of providing reading instruction that will accelerate the development of reading skills in students who are retained in 3rd grade as a result of performing at Level 1 on the FCAT. Unpublished manuscript, Florida Center for Reading Research. Tremor, M., & Butler, M. J. (2004). Technical assistance paper related to third grade student progression (FY2005-03). Retrieved April 5, 2005, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-2343/TAP.pdf Tye, B. B. (2000). Hard truths: Uncovering the deep structure of schooling. New York: Teachers College Press. U.S. Department of Education. (1999). Taking responsibility for ending social promotion: A guide for educators and state and local leaders. Retrieved September 1, 2004, from http://www.ed.gov/pubs/socialpromotion/index.html U.S. Department of Education. (2002). No Child Left Behind of 2001. Retrieved February 5, 2003, from http://www.ed.gov/nclb/landing.jhtml?src=pb# U.S. Department of Education. (2004a). Press release: Paige issues statement on report on social promotion. Retrieved May 29, 2005, from http://www.ed.gov/news/pressreleases/2004/12/12082004.html U.S. Department of Education. (2004b). Press release: Secretary Paige issues new policy for calculating participation rates under No Child Left Behind. Retrieved January 29, 2006, from http://www.ed.gov/news/pressreleases/2004/03/03292004.html U.S. Department of Education. (n.d.). Title I, part A program. Retrieved June 19, 2005, from http://www.ed.gov/programs/titleiparta/index.html

PAGE 147

136 Valencia, S. W., & Wixson, K. K. (2004). Literacy policy and policy research that make a difference. In R. B. Ruddel & N. J. Unrau (Eds.), Theoretical models and processes of reading (5th ed., pp. 69-92). Newark, DE: International Reading Association. Warford, J. (2004a). Memorandum: 2004 required instruction and school closures due to hurricanes (DPS #05-041). Retrieved September 30, 2004, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-2455/05-041.pdf Warford, J. (2004b). Memorandum: Reporting requirements for third grade student progression (DPS # 04-098). Retrieved May 20, 2004, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-1978/DPS04-098.pdf Warford, J., & Openshaw, M. L. (2004a). Memorandum: 2004 legislative changes regarding students with reading deficiencies and student progression (DPS # 04-125). Retrieved May 21, 2004, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-2073/DPS04-125.pdf Warford, J., & Openshaw, M. L. (2004b). Memorandum: 2004 legislative requirements regarding students with reading deficiencies and student progressionCS/SB 364 (DPS # 04-132). Retrieved June 9, 2004, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-2129/04-132.pdf Warford, J., & Openshaw, M. L. (2004c). Memorandum: Technical assistance paper for implementation of the READ initiative and intensive acceleration classCS/SB 364 (DPS #04-140). Retrieved June 23, 2004, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-2156/DPS04-140.pdf Warford, J., & Openshaw, M. L. (2005). Memorandum: Third progression technical assistance document (K-12: 2005-60). Retrieved May 26, 2005, from Florida Department of Education Web site: http://info.fldoe.org/dscgi/ds.py/Get/File-2994/k12_05-60.pdf Wothke, W. (2000). Longitudinal and multigroup modeling with missing data. In T. D. Little, K. Schabel & J. Baumert (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples. Mahwah, NJ: Erlbaum.

PAGE 148

BIOGRAPHICAL SKETCH Courtney Caldwell Zmach received the University of Floridas highest graduate student award, the Alumni Graduate Fellowship, before beginning her work toward a Ph.D. in curriculum and instruction, specializing in reading education at the University of Florida, Gainesville. She earned her Bachelor of Arts degree in social sciences from St. Bonaventure University (St. Bonaventure, New York) where she was inducted into two international honor societies: Alpha Kappa Delta and Pi Gamma Mu. Courtney also completed her initial teacher preparation training at St. Bonaventure. She earned her Master of Education degree in curriculum, teaching and learning at the University of Toronto/Ontario Institute for Studies in Education (OISE). Courtney has held elementary education teaching certifications in the Province of Ontario, Canada, as well as the states of Pennsylvania and Florida. Before beginning her doctoral program, she taught for several years in Ontario and Florida. Courtney has also worked as a consultant, across Florida, conducting professional-development sessions for teachers and school administrators. At the University of Florida, she worked as a graduate research assistant and taught reading courses at the undergraduate and graduate levels. Currently, Courtney works as a research analyst, specializing in reading research, at the American Institutes for Research (AIR) in Washington, D.C. 137


Permanent Link: http://ufdc.ufl.edu/UFE0012802/00001

Material Information

Title: Navigating the Complexities of Legislation: How Elementary School Principals Interpret and Implement Florida's Third-Grade Retention Policy
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0012802:00001

Permanent Link: http://ufdc.ufl.edu/UFE0012802/00001

Material Information

Title: Navigating the Complexities of Legislation: How Elementary School Principals Interpret and Implement Florida's Third-Grade Retention Policy
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0012802:00001


This item has the following downloads:


Full Text












NAVIGATING THE COMPLEXITIES OF LEGISLATION: HOW ELEMENTARY
SCHOOL PRINCIPALS INTERPRET AND IMPLEMENT FLORIDA'S
THIRD-GRADE RETENTION POLICY


















By

COURTNEY CALDWELL ZMACH


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2006

































Copyright 2006

by

Courtney Caldwell Zmach















ACKNOWLEDGMENTS

Many people contributed to my educational background, to shape the educator and

researcher that I have become. It is difficult to know where to begin, because each is

important. I thank my steadfast supervisory committee chair (Dr. Richard Allington),

cochair (Dr. Anne McGill-Franzen), and members (Dr. Gregory Camilli, Dr. Thomas

Dana, and Dr. Sandra Russo). Each of them supported my scholarship and development

in becoming a reading researcher. Dissertations can be a lonely journey and I thank those

who spent time listening to me. At different stages and in different ways, Dick Allington,

Anne McGill-Franzen, Greg Camilli, Tom Dana, Sandra Russo, Jenn Graff, Evan Lefsky,

Jeff Miller, Katie Solic, and Lunetta Williams provided listening ears. I value their time,

suggestions, and unwavering support.

My thanks go to the many members of the Just Read, Florida! Office and the

Florida Department of Education for taking time to answer questions and for gathering

data files for my study. I am indebted to Hajnalka Peto and Jenny Williams for the time

they spent transforming state data files, so I could create the databases used in my study.

I thank my parents and family, who supported my decision to uproot my life as a

classroom teacher, to return to school. From as early as I can remember, my parents

shared with me the value of education. Their support and encouragement led me this far.

My former colleagues and students deserve thanks for inspiring me to reach for higher

educational goals. I also express gratitude to my new colleagues at the American

Institutes for Research (AIR), for supporting my continued scholarship.









Of course, this research would not have been possible without the cooperation of

the twelve Florida school districts. Hurricane Season 2004 was an especially trying time

for Floridians. Special thanks go to the elementary school principals who gave their time

to participate in this study.
















TABLE OF CONTENTS



A C K N O W L E D G M E N T S ................................................................................................. iii

LIST OF TABLES ................. .......................................................... vii

LIST OF FIGURES .................................................... ............ ............... viii

A B S T R A C T ............................................ ... ......... ................................... x

CHAPTER

1 INTRODUCTION TO STUDY ..................................................... ....................1

Introduction ...................................................................................... .. ..............1
P policy P problem ................................................ 2
Stu dy A approach ....................................................... 3
P ersp ectiv e s ....................................................... 4
P purposes of this Study ....................................................... 5

2 REVIEW OF THE LITERATURE ....................................................... 6

In tro d u ctio n .................................................................................. 6
Research on R detention ................................................................ .......... 7
High-Stakes Framework .............................................. ................... 18
F lo rid a P o licy C o n tex t ........................................................................................... 3 1
S u m m a ry ......................................................................................................4 6

3 M E T H O D S ............................................................................................................ 4 8

Introduction ................................................................................................. ....... 48
P a rtic ip a n ts ............................................................................................................ 4 9
D ata Collection ....................................................................... ........ 51
P procedures ........................................................................................................ 54
D e s ig n .................................................................................................................... 6 3

4 R E S U L T S .............................................................................7 6

Introduction ................................................................................................. ....... 76
M ultilevel M odels..................................................... 76



v









S u m m a ry ................................................................................. 8 5

5 D ISC U S SIO N ............................................................................... 87

In tro d u ctio n ........................................................................................8 7
R research Q question 1 .............................. ......................... .. ...... .. .......... 87
R research Q question 2 .............................. ........................ .... ...... .. .......... 89
R research Q question 3 .............................. .... ..................... .. ...... .. .......... 91
Policy C considerations ........................................................ .. ............ 97
Policy R ecom m endation ............................................................................ 104
Future R research .................. ....................................... ... ............... 107
C o n c lu sio n ..................................................................................................... 1 0 9

APPENDIX

A FLORIDA PRINCIPAL SURVEY .................................. .................................... 111

B REVIEW OF SURVEY PILOT STUDY ..... .......................... 120

C D A T A SO U R C E S ......................................................................... ..................... 122

D SAMPLE FROM STATISTICAL SYNTAX.......... ............... 123

E RESULTS TABLE FOR ALL MODELS ..................................... ...............124

LIST OF REFEREN CE S ... .... .................. ................ ........................ ............... 126

BIOGRAPHICAL SKETCH ......... ........ ........................... 137
















LIST OF TABLES


Table page

2-1 Florida retention policy good cause statewide exemptions.............................. 36

2-2 Supports identified in Florida's legislative intent ..............................................41

3-1 Time in years: Pre-policy versus post-policy ............... ................... ............... 49

3-2 Degree of support category, Florida Principal Survey ............... ... ............ 54

3-3 Survey response rates by district size.................................... ....................... 57

3-4 Usable sample versus returned responses by district size.............................. 60

3-5 Third-grade percent retained usable sample principals versus the total
population ............. ............................................. ............... 61

4-1 Results of fitting a taxonomy of multilevel models for change to the log percent
retained data, M odels A through E (n = 102).......................................................86

5-1 Third-grade FCAT reading achievement Level 1 .................................................99

C-l Sources of data ......... .. .............................. ..... .... .. .. ........ .... 122

E-1 Results of fitting a taxonomy of multilevel models for change to the log percent
retained data, M odels A through F2 (n = 102)...................................................... 125
















LIST OF FIGURES


Figure p

2-1 H igh-stakes fram ew ork ............................................... ................................... 18

3-1 Item 9, Florida Principal Survey ........................................ ......................... 52

3-2 Item 14, Florida Principal Survey ........................................ ....................... 53

3-3 Item 19, Florida Principal Survey ........................................ ....................... 53

3-4 Employment history, Florida Principal Survey ................ ......................54

3-5 Population versus usable sample mean percent retained by year...........................61

3-6 Mean SES values over 5-year period for usable sample schools..........................62

3-7 M ean retention rates by year ....................................................... ............... 65

3-8 Examples of principals' third-grade retention rates over time..............................66

3-9 Third-grade retention over tim e ........................................ .......................... 67

3-10 Histogram of percent retained before the transformation .............. ............... ...70

3-11 Histogram of percent retained after the transformation .......................................70

3-12 Examining the homoscedasticity assumption .................................. ............... 71

3-13 M ultilevel model for change taxonomy ....................................... ............... 73

3-14 Number of zero percent retained values removed by year.................. ............74

4-1 Scatterplot of log percent retained plus one.......................................................78

5-1 Examples of mean predicted retention trends over time.......................................88

5-2 Mean predicted third-grade retention rates by poverty level over time ...................90

5-3 Example plotting retention rates using principals' mean SES values before and
after policy im plem entation ........................................................................90









5-4 Mean predicted third-grade retention rates by principals' current retention belief
over tim e.............................................................................................. 92

5-5 Principals' retention belief by the mean predicted third-grade retention rates
over time disaggregated by poverty group...........................................................95

5-6 Mean predicted third-grade retention rates by degree of support over time ............96

5-7 Degree of support by mean predicted third-grade retention rates over time
disaggregated by poverty group ........................................ .......................... 98














Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

NAVIGATING THE COMPLEXITIES OF LEGISLATION: HOW ELEMENTARY
SCHOOL PRINCIPALS INTERPRET AND IMPLEMENT FLORIDA'S
THIRD-GRADE RETENTION POLICY

By

Courtney Caldwell Zmach

May 2006

Chair: Richard L. Allington
Cochair: Anne McGill-Franzen
Major Department: Teaching and Learning

Retention policies affecting third graders have become increasingly common as

schools, districts, and states across the country work to comply with the federal mandate

for all third-grade students to read at or above grade level by 2014. Florida is one such

state to enact a high-stakes retention policy as a way to meet accountability challenges.

Third graders in Florida who fail the state assessment, the Florida Comprehensive

Assessment Test (FCAT), are held back in grade.

This exploratory study gained understanding of Florida's third-grade retention

policy by focusing on the elementary school principal (n=102) as the unit of analysis. To

examine the Florida policy, it seemed useful to better understand patterns of third-grade

student retention over a 5-year period, retention beliefs of principals, and degree of

support given to students affected by the policy. The overarching research question of

this study was "What is the trend of third-grade retention practices before and after









implementation of Florida's third-grade retention policy and how is this trend impacted

by other variables?" Using a combination of items from a researcher-designed survey of

elementary-school principals across twelve districts, and data provided by the Florida

Department of Education, I used a multilevel statistical model for change examining

within-person change and between-person differences in change, beginning with the

1999-2000 school year third-grade retention rates. Study variables also included

measures for pre- and post-policy, poverty, trend within time, retention belief, and degree

of support.

Results show that the trend within time and poverty at the school level dramatically

affected retention rates across this sample. Students in higher-poverty schools were

impacted most greatly by the policy. In other words, lower achieving students in

higher-poverty schools were more likely to be promoted pre-policy and because of this,

the mandate had more impact on students in higher-poverty schools than in lower-poverty

schools. Regardless of the actual impact on reading achievement, Florida's retention

policy has eliminated social promotion in third grade.














CHAPTER 1
INTRODUCTION TO STUDY

Introduction

Even with the national agenda to "leave no child behind," substantive segments of

the student population are held back in grade because of mandatory retention policies.

Today, thousands of students in Chicago, New York City, and across Texas and Florida

(and elsewhere) have been affected by the use of high-stakes assessments to make

promotion and retention decisions.

Retention in grade (sometimes called nonpromotion, being held back, flunking, or

failing) is a policy that retains a student in the same grade until the student meets

requirements to move to the next grade (Jackson, 1975; Jimerson, 2001; Shepard &

Smith, 1989). Retention is not a new idea: it is an old and highly contested practice that

has fluctuated across history. Earlier research indicated reasons for retaining students,

including low performance on standardized assessments, youngness for the grade,

physical size, immaturity, and even poor attendance. Today, some believe the

conventional wisdom that if a child does not fare well on a standardized assessment, then

that child is ill prepared for the next grade level. Once again, retention has resurged

because of mandates tied to meeting accountability criteria.

The antithesis of retention is social promotion, where students are promoted from

grade to grade, with same-age peers. Social promotion is something of an enigma, as

many state departments of education have not collected information determining how

widely it has infused the system. Equally troubling, limited information is available






2


regarding how retention decisions are made, or whether standardized assessments are a

determinant for retaining students (U.S. Department of Education, 1999). In the Council

of Chief State School Officers (CCSSO) 2001-2002 Survey of Student Assessment

Programs (SSAP), 10 states reported that standardized assessments played a role in

retention decisions (personal communication, September 14, 2004). However, we know

that this number has grown, as Florida (in 2003) joined the ranks using reading

achievement scores from the Florida Comprehensive Assessment Test (FCAT) for

promotion and retention decisions.

Policy Problem

What truly is the crux of the problem at hand? Not performing on grade level is

seen by some as a social problem (Oakes, 1999). To fix this perceived problem,

lawmakers instituted what they believed to be a solution-retention in grade. If the

children did not learn to read at a proficient level the first time in third grade, will another

year in third-grade be the solution? It would appear that the policy is working when more

students achieve a passing score. Retention in grade has been researched for decades and

proven ineffective (Allington & Walmsley, 1995; Shepard & Smith, 1989); yet Florida

persisted in enacting a retention policy to remedy the perceived social problem.

The challenge we face is more than whether failing a student is justified. It comes

down to how we, as educators and researchers, can most effectively help students who

struggle with reading, especially those who fail to make a passing score on high-stakes

assessments. Nevertheless, retention policies tied to reading achievement are becoming

increasingly common as schools, districts, and states across the country contend with the

No Child Left Behind of 2001 mandate for all third-grade students to read at or above









grade level by 2014 (US Department of Education, 2001). Florida is one such state to

enact a high-stakes retention policy to meet the challenges of accountability.

One wonders how this retention policy benefits Florida's students. As reported in a

dataset provided by the Florida Department of Education (FLDOE), over 27,000 of

Florida's third-grade students were impacted by the retention policy after the 2002-03

school year. Of this number, after 1 year of retention as part of the State's chosen

intervention method to help increase student achievement, only 59% passed the FCAT

after their 2nd year in third-grade (Florida Department of Education, 2004c). There are a

great number of areas awry in education today, but retention used as a reading

intervention is questionable. State-level policymakers believed that Florida's retention

policy (F.S. 1008.25) would help mend the perceived problem of third graders not

reading on grade level, but the research base does not support that belief.

Study Approach

In the early stages of [policy] implementation, summative measures usually are
inappropriate. More appropriate questions for analysis involve the extent to which
necessary resources are available to support implementation, whether there is
evidence of good-faith efforts to learn new routines, or indication of commitment
and support within the implementing system for policy strategies and goals.
-Milbrey McLaughlin (1987, p. 176)

On considering how best to capture the spirit of my study, I decided to collect data

using a survey questionnaire, and to use historical retention data and poverty indicators

provided by the FLDOE. My exploratory study aimed to gain understanding of the

interpretation and implementation of Florida's retention policy by focusing on the

elementary school principal as the unit of analysis. To illuminate the current situation in

Florida, it seemed useful to better understand

* Patterns of third-grade student retention over a 5-year period









* Impact of school-level poverty

* Other variables such as a principals' retention belief and the degree of support
provided for students affected by the new state policy

The overarching research question of my study was "What is the trend of

third-grade retention practices before and after implementation of Florida's third-grade

retention policy and how is this trend impacted by other variables?" To answer this

question, I designed a multilevel model considering factors such as the school's level of

poverty, while determining whether past retention practices appear to influence the level

of current support offered to students at risk for retention or currently retained.

Perspectives

The worldview of the researcher bases socioeconomic status (SES) as a predictor

for students' educational successes. "[Poverty] is a condition, like gravity, that affects

virtually everything" (Bracey, 2003, p. 46). Foci throughout my study presuppose that

economic disadvantage places many schools and their students at risk for failure. The

correlation between student achievement and poverty level is widely accepted (Linn,

2004).

Florida elementary school principals were selected to gather information first-hand,

from a range of schools across the state. Principals are vital stakeholders in making

decisions at the school level, and being informed about the happenings in their schools.

Principals' historical retention practices were built into my study design to examine

retention over time in relation to retention belief. Selection of principals in no way

suggests that principals deserve sole responsibility for matters concerning retention;

however, as the figurehead of leadership in a school, they serve an important role.









Purposes of this Study

Results from my study serve multiple purposes. As a result of the

third-grade retention policy, thousands of third-grade students have been retained or

re-retained (Florida Department of Education, 2004c); however, this policy has brought

forward several mandates and initiatives geared to support students who struggle with

reading. Because of the mandate to use particular supports, examining the relationship

between degree of support available and retention rates may help current principals assess

(or reassess) their practices to provide the interventions proven to benefit students at risk

for failure.

Stakeholders from school, district, and state levels may have a clearer vision of

how this retention policy has been enacted. We know that beliefs held by those who

implement policy play an important role (McGill-Franzen, 2000; Spillane, 1998);

therefore, we must understand how these beliefs affect the retention policy. Regardless

of whether one concurs with or opposes this policy, it is important to understand whether

retention beliefs held by administrators across the State impact the proportion of students

retained, or even how supports are distributed to help students. This research may help

administrative leaders (and district and state leaders) consider patterns of retention in a

different way. Results are unique to Florida and may not be generalizable to other states;

however, other states practicing retention or considering implementing a retention policy

might gain insights that may benefit their locale.














CHAPTER 2
REVIEW OF THE LITERATURE

Introduction

Today in Florida, a retention policy prevents third-grade students from progressing

to the fourth grade when they fail to master grade-level standards (or expected learning)

on the state assessment, the FCAT. Given this policy impact on students, why is

Florida's retention policy (Section 1008.25, Florida Statutes) worth study from the

vantage point of an elementary school principal? As the primary instructional leader of a

school, the principal has the extraordinary task of remaining abreast of policy changes,

while communicating vital information to staff members and parents. The school

principal serves as a gatekeeper of information for the instructional staff, and for the

students they serve and protect. The principal is largely accountable for all the

happenings in his or her school. In the hierarchical system instituted in districts across

Florida, it is important to understand how administrators (in this case, principals) support

those students at risk for retention or already retained in grade.

Over the past decades, other investigations studied the effects of retention policies,

including student achievement, dropout rates of students affected by retention decisions,

and beliefs about retention (Shepard & Smith, 1989). Since Florida implemented the

retention policy in 2003, ending social promotion for third graders who fail to read on

grade level, many facets are worth studying that can add to and extend the existing body

of knowledge. While one can choose to focus on retention at the classroom level

(examining issues related to students, teachers, or classroom instructional practice), I









believe the principal holds direct influence on the aforementioned. Few studies have

examined the role of principals and how they enact retention policies. Since the principal

acts as an intermediary or messenger disseminating information from the district (or

state) to the classroom level, how a principal interprets and implements this retention

policy is vital to our understanding and evaluation of what makes a difference for those at

risk for retention.

Research on Retention

First, we must consider why retention in grade has resurfaced as a commonly

accepted practice, or intervention of choice, in states and districts across our country. To

understand where we are today, let us go back and review a brief history of how

educators, in the United States, came to practice retention.

In the 19th century, as America developed and expanded its public school system,

formalized instruction typically occurred in ungraded one-room schoolhouses with

students ranging in age (Ruhl, 1984), where students progressed through the education

system learning content. By the mid-nineteenth century, the European education system

began to influence multiple aspects of the United States' education structure (Balow &

Schwager, 1992; Ruhl, 1984). Soon, many facets of the United States' system became

graded-everything from schools, students, curricula, teachers (Balow & Schwager,

1992). A graded, or sequential system, turned learning into discrete parts, as students

were held accountable to learn specific information that built upon one year to the next.

As student learning became linked to yearly increments, the introduction of

textbooks helped standardize the curriculum (Jackson, 1975). With the use of textbooks,

it became clear that students did not learn at the same rate. Hence, originated one

consequence when textbook content was not mastered-retention in grade. For students









who did not master the material, this solution required them to repeat the same grade

level to continue learning that material (Holmes & Matthews, 1984; Jackson, 1975).

Decades later, Leonard Ayres described in his book Laggtard in our Schools

(1909) a trend that Dr. William Maxwell, Superintendent of the New York City schools,

reported in 1904. Maxwell observed discrepancies in the normal ages of students within

the New York City school system. He calculated a staggering 39% of children in the

elementary grades over the standard age for a given grade. This inconsistency between

age and grade when students fall backward in grade was termed "retardation" (Ayers,

1909). This fluctuation was the result of retention in grade and research ensued to learn

more about this phenomenon.

One of the first retention studies, The Backward Children Investigation, led by

Ayers in 1907 studied children who were not progressing through school at typical

rates-those who were over-age for the grade. Ironically, Ayers' areas of inquiry are

quite similar to present-day wonderings-learning more about differences in student

characteristics, the age of students when starting school, teacher effectiveness, and

curriculum differences. His research questions included, "How many of the children in

our schools fail to make normal progress from grade to grade and why do they fail? How

many of the children drop out of school before finishing the elementary course and why

do they drop out of school before finishing the elementary course and why do they drop

out? What are the facts and what are the remedies?" (Ayers, 1909, p. 2).

Ayers calculated the "survival" and "mortality" rates of students by establishing the

proportion of students who remained in the school system without dropping out. Based

on his calculations, most students remained in the system through the end of grade school









with a severe drop off in attendance during the junior high years and few retainees

remained to attend high school. The retained were more likely to leave school before

their promoted same age peers. His findings established that students, who do not

succeed, simply drop out (Ayers, 1909).

Evidence has mounted for nearly a century that retention does not benefit students.

In the 1930's and 1940's, researchers conducted studies dispelling the notion that

retention helps children. In fact, Scott and Ames (1969) reported that the "negative

findings on the effects of non-promotion were so uniform in the 1930's and 1940's that

many investigators considered the question closed..." (p. 433). However, Goodlad

(1952) revisited retention when it surfaced as a hot topic of the day with educators asking

the question "What is best for the development of this child?" (p. 150).

Over fifty years ago, Goodlad recognized that retention was not a one-size-fits-all

solution asserting that "promotion on the basis of fixed minimum standards is not

adequate" (Goodlad, 1952, p. 150). He tested hypotheses dealing with differences in

social and personal adjustment between children who repeated a grade and those who did

not repeat. Based upon his findings, he found promotion to be a more justifiable

educational practice, taking into account the best interests of the child. Goodlad

recommended that 1) children be treated as individuals, not subjected to system-wide

policy, 2) teachers use facts related to achievement, intelligence as well as human growth

and development in decision making, and 3) "[the] instructional needs of the pupil should

take precedence over matters of administrative expediency in dealing with questions

involving promotion and nonpromotion" (p. 154).









Using Goodlad's (1952) recommendations, Lobdell (1954) conducted a study

where one school district adhered to criteria guiding both teachers and principals in

making retention decisions for low achieving students. Criteria for these decisions were

both general and specific. Briefly, general criteria included what would be best

long-term for the child, including 1) holding a child back only one time before sixth

grade, 2) avoiding repetition of sixth grade and finally, 3) children should not spend the

repeated year with the same teacher as the previous year. Specific criteria suggested that

many aspects be taken into account beyond the score on a standardized assessment,

including current grades, intelligence scores, age, size, social characteristics and the

attitude of the parents toward the child's progress (Lobdell, 1954).

Essentially, all children were to be treated individually when making retention

decisions. Lobdell (1954) asserted that "holding back a child for two or three or more

years beyond his normal graduation age, as practiced in past generations, can in no way

be defended" (pp. 335-336). Both the teacher and the principal were involved in making

these decisions; however, the role of the principal was more of a judge. Principals

insured that the facts were present and that the best interests of the child were taken into

account (Lobdell, 1954). Retained students who were subjected to these criteria showed

short-term success, however the long-term effects were not discernable from Lobdell's

available data. Like Goodlad, Lobdell (1954) concluded that broad policies for retention

needed to be replaced with criteria to guide the decision making process, as this was

believed to be in the best interests of the child.

Changes in promotional policies was the subject of the Hall and Demarest (1958)

piece in The Elementary School Journal. They identified three levels of promotional









policy: 1) Grade-standard: students learned a predetermined amount to move to the

next grade, 2) Continuous-promotion: as in social promotion, where students move

through the grades with same-age peers, and 3) Continuous-progress: students are

retained based upon individual retention decisions (Hall & Demarest, 1958). This study

examined the policy shift in Phoenix, Arizona treating children as individuals to make

retention decisions. The move to "continuous-progress" was seen as a positive shift for

children (Hall & Demarest, 1958).

Research on retention was sparse in the 1960s. However, as the decade neared an

end, Chase (1968) and Scott and Ames (1969) published their work on the subject. Both

studies omitted students retained for reasons such as low intelligence, brain damage, or

emotional disturbance as these were not viewed as necessarily correctable via retention

and included only those who were immature because of youngness or behavior. Findings

from these studies supported retention as a means to increase maturity.

Nonetheless, over the years, research has consistently shown that retention causes

negative long-term consequences. Efforts to increase achievement by means of retention

simply present more hurdles for the student to overcome. Grissom and Shepard (1989)

created a structural equation model, or causal model, in hopes of explaining the causes

for dropping out of school. Although they found that substantially more students who

were retained drop out of school early compared to their promoted peers, other factors, in

addition to flunking, contributed to the dropping out, such as the need for the student to

work or attend to other family matters. Research on dropping out of school has been

studied in more recent times with similar results to Ayers (Roderick, 1994). Roderick

(1994) also found that the retained were more likely to dropout leaving school early.









There are other long-term educational outcomes worth consideration. Retained students

are more likely to be placed in special education programs (Barnett, Clarizio, & Payette,

1996; Guthrie, 2002; McGill-Franzen & Allington, 1993), and students placed in special

education or other remedial programs rarely escape their special education label

(McGill-Franzen, 1987; McGill-Franzen & Allington, 1993).

Retention Research in Chicago

The Consortium on Chicago Schools Research at the University of Chicago

studying the implementation of Chicago Public Schools' 1996 retention policy provides a

wealth of insight into the effectiveness of that policy (e.g., Roderick, Bryk, Jacob, Easton,

& Allensworth, 1999; Roderick & Engel, 2001; Roderick, Nagaoka, Bacon, & Easton,

2000). When students in Chicago did not meet promotional gate cut-off scores on the

standardized assessment, Iowa Tests of Basic Skills (ITBS), they were held back.

Promotional gates in Chicago occur at third, sixth and eighth grades.

After analyzing 2 years of retention records, these researchers determined that

many children were being placed in special education programs after the retention

decision was reached. In fact, they calculated that between 17 and 20% of retainees were

recommended for special education placements (Nagaoka & Roderick, 2004). One

caveat Nagaoka and Roderick (2004) acknowledged was the difficulty discerning

whether these students had unidentified difficulties prior to the policy or whether these

students were being pushed into special education programs to improve accountability

reports. Both explanations are plausible (Allington & McGill-Franzen, 1992;

McGill-Franzen & Allington, 1993). In addition, one must strongly consider that lack of

teacher expertise is another possible explanation for these referrals. When teachers use









all they have in their bag of tricks, they commonly turn elsewhere for guidance-special

education is an intuitive choice.

Some retained students did not meet the criteria to move through the promotional

gates after their second attempt. These students remained in the same grade for a third

year in the same grade; these students are called the double retainees. The Chicago team

worked to determine whether the double retainees were better off after these retentions

(Roderick et al., 2000). After equating the scores of the third-grade ITBS to the

fourth- and fifth-grade tests, the researchers found little conclusive evidence to suggest

that double retentions benefited Chicago's third graders. In fact, after three years in third

grade, about 80% were promoted to fourth grade, while about 10% of students were

placed in special education programs.

Thus far, results from the study in Chicago confirm past research efforts that

retention does not work. In fact, they have found "little evidence that students who were

retained did better than their low-achieving counterparts who were promoted" (Nagaoka

& Roderick, 2004, p. 45). Retained students who were offered additional chances to

meet the promotional criteria struggled to reach the cut point. Consortium researchers

support early intervention; however, they caution that this does not mean high-stakes

assessments need to be moved to earlier grades (Roderick et al., 1999).

Reviews of Retention Research

Jackson (1975) presented the first review of research on retention by dividing thirty

studies published between 1911 and 1973 into three groups according to the research

design. Based on his analyses of naturalistic, pre-post and experimental design, he found









that retention may benefit some students, although more students seem to benefit from

promotion. Jackson also suggested that the retention is unfounded as an intervention.

Next, we examined evidence from using meta-analysis, a research method that

takes into account the quantitative outcomes of multiple research studies (Cooper &

Harris, 1994; Light & Pillemer, 1984). The steps of meta-analysis are important to

evaluate highly contested issues, as it helps compensate for biases in the sample and

findings (Camilli, Wolfe, & Smith, in press; Holmes & Matthews, 1984). One benefit of

meta-analysis, and the reason why these studies are presented here, is that this technique

is much more "comprehensible to the reader than lengthy recounting of each individual

study's methods and results" (Shepard & Smith, 1989, p. 16).

Two noteworthy meta-analyses conducted over a decade ago aimed to learn more

about the effects of retention on students (Holmes, 1989; Holmes & Matthews, 1984).

The 1984 meta-analysis systematically reviewed 44 studies that fit the selection criteria.

Essentially, research studies needed to have specific features for the meta-analytic

calculations. Holmes and Matthews (1984) required original studies to contain these

criteria: 1) report the effects of retaining students in grade school or junior high, 2)

provide enough data to calculate effect sizes, and 3) compare retained students to

promoted students. After statistically combining the results of these studies,

overwhelmingly the evidence confirmed that retention does not work (ES = -.37). The

sign of the effect size indicates whether combined outcomes of the studies had a positive

or negative effect on students. An effect size of -.37 suggests that the negative effects of

retention far outweigh the positive.









Only 5 years after Holmes and Matthews published these results, Holmes (1989)

performed another meta-analysis using the original forty-four studies along with an

additional nineteen studies conducted since that review. In the 1989 analysis, 9 studies

showed positive effects, benefiting the retained. Although the overall effect size was

slightly smaller in the 1989 analysis (ES = -. 15 vs. ES = -.37), the overall direction of the

results confirm that retention does not support the development of students academically

or personally (Holmes, 1989). Additionally, Holmes (1989) conducted a secondary

analysis to ascertain why there was a difference between these effect sizes merely 5 years

later. He found that studies fell into two types, positive and negative. Most of the

positive studies were conducted in suburban schools with few minority students,

contained remediation plus intervention, and compared grade level peers as opposed to

same age peers. When studies were matched on IQ, achievement tests, socioeconomic

status, gender, and grade, the effect size was -.30, after controlling for possible

differences in the samples.

More recently, Jimerson (2001) conducted a meta-analysis examining the results of

studies dating from 1990 to 1999. Compared to the meta-analyses cited above, selection

criteria for this analysis included accepting only studies with matched comparison

groups. With an effect size equaling -.31, meta-analytic methods repeatedly substantiate

that retention is not an effective method to increase student achievement (Jimerson,

2001). In his conclusion, Jimerson (2001) recommended moving beyond questioning

whether to retain students and to move forward considering which interventions are

effective to remediate students who struggle academically.

Retention Trends









Studies exist documenting trends of retention practices over time. Of particular

interest here is Allington and McGill-Franzen's study (1992) investigating

school-extending practices in the New York State. These researchers wondered whether

there was a shift in retention to Grades K to 2 in an effort to delay the predicted

third-grade retention. This clever move allowed a delay in public accountability reports

(Allington & McGill-Franzen, 1992). Early identification of students in Grades K to 2

who would likely be retained in third grade were held back in earlier grades in efforts to

remediate them prior to the mandated flunking in third-grade, if they failed the state

assessment.

Retention Beliefs

"The contradiction between research and actual classroom practice is deep-seated

in a belief system that has been delivered over the past 90 years" (Reitz, 1992), or over

the past 100 years now. Two studies included here, focus on educators' beliefs about

retention, were conducted during an era when "social promotion" was a more acceptable

option. Byrnes (1989) conducted a study on the attitudes of students, parents and

educators on the practice of retention through two means, survey and interviews, in a

large city in the southwestern United States. We concentrate here on the questionnaire

sent to forty-five principals and assistant principals with a 78% return rate. Findings

from the principal questionnaire determined that most principals (74%) favored retention

for students who were not meeting the grade level standard. Principals also favored

immaturity (54%) as a cause for holding students back in grade. Contrary to popular

belief, in that study retention was not seen in a negative light. Significant differences

were detected when respondents were asked who should have the final judgment in

retention decisions. Teachers felt this was their responsibility; whereas principals felt









this fell in their domain. Alternatives to retention were explored with teachers and

principals choosing options such as smaller class sizes and more individualized

instruction compared to other ideas that they perceived as changing the current school

practices. For instance, options not selected included "flexible entry age, transitional

maturity classes, and multi- and non-graded school structures" (Byrnes, 1989, p. 114).

Conclusions from this study showed that teachers, principals and parents believe

retention benefits those students not performing on standard or who were thought to lack

maturity. Byrnes (1989) described retention as an intuitive choice, even though research

still did not conclusively suggest a benefit for students.

Another study using a multi-method approach focused on the attitudes of teachers

of grades K to 7 (Tomchin & Impara, 1992). Using a questionnaire and interviews,

Tomchin and Impara found that primary grade teachers (grades K to 3) are more likely to

retain students, compared to teachers from middle grades (grades 4 to 7). Primary grade

teachers do not believe harm will follow a retention decision, whereas teachers in older

grades were more skeptical. On the questionnaire, almost 98% of teachers disagreed with

the statement that "Children should never be retained" (Tomchin & Impara, 1992). As

principals are typically former classroom teachers, it is of interest in the present study to

determine the retention beliefs held by principals, especially now that social promotion is

constrained by policy. "Parents, teachers, and principals seem to play a crucial role in the

decision-making process, and generally a veto from any of these consultants can result in

promotion instead of retention, regardless of performance on competency tests"

(Nikalson, 1984 and Rose et al., 1983 as cited in Jimerson, Carlson, Rotert, Egeland, &

Sroufe, 1997, p. 4). In Florida, a veto is not possible.









High-Stakes Framework

To understand my policy study, one must be familiar with the high-stakes

framework (Figure 2-1) within which retention is situated. Also, as students' reading

success is pivotal to the national goal for students to read on grade level by 2014, we see

how retention interconnects with the literature of educational reform, accountability, and

high-stakes standardized assessments. We begin with a review of the essentials to

understand this high-stakes framework beginning with an overview of national

educational reform exploring past and present movements focusing on standards-based

reform, seeing how reform entwines with accountability and standardized assessments.

Next, we examine the uses and possible misuses of standardized assessments and the

relationship to retention. Finally, by focusing on how policy is interpreted and

implemented, we work toward a common understanding. Adding to the background of

my study, efforts were made to focus on elementary school principals and the State of

Florida within each area of the framework.


Figure 2-1. High-stakes framework









Reform Movements

In considering where we are today, let us remember past policy, or "first

generation" policy (McLaughlin, 1992 as cited in McGill-Franzen, 2000, p. 800). The

current talk about educational equity, and raising achievement are not new ideas. These

have been policy goals for at least 40 years, when the federal government passed the

Elementary and Secondary Education Act (ESEA) of 1965, in particular Title I, aimed to

improve the education for students from impoverished backgrounds (McGill-Franzen,

2000). For 40 years, high-need schools enrolling at least 40% of families from

economically disadvantaged backgrounds have received funds from the largest federal

funding program, Title I. Economic disadvantage is determined by the percentage of

students who are eligible for a free or reduced price lunch. One goal of Title I funding

has been to close the achievement gap by providing all students with the opportunity to

attain a high-quality education, regardless of their economic disadvantage or advantage.

Educational attainment for all students was on the national educational agenda. With

reform movements continuing to permeate the United States' educational system and

gain strength, there is impetus for broad, sweeping transformations and having them

happen instantaneously. Reforms are one method of instituting change.

A Nation at Risk: The Imperativefor Educational Reform is widely regarded as the

origin of modern American educational reform (National Commission of Excellence in

Education, 1983). However, such reports moved us into the next generation of education

policy (McGill-Franzen, 2000). It called for the American public to restructure the

educational system and raise standards (National Commission of Excellence in

Education, 1983). Specifically, one recommendation applied to retention in grade.

Recommendation C, Number 8 suggested redesigning the grade leveling system









lessening importance on age restrictions for placing and promoting students, instead it

emphasized that academic progress and students' needs should serve more to guide

promotion and graduation decisions (National Commission of Excellence in Education,

1983).

Politically, support for retention appears to be bipartisan. By the late 1990s, many

lawmakers took note of the commission's advice and scrutinized students' academic

progress. In his 1998 State of the Union Address, then-President Clinton "joined a host

of other political leaders, from the Democratic mayor of Chicago to the Republican

governor of Texas, all calling for an end to the promotion of students whose achievement

does not meet the expectations for that grade" (National Research Council, 1999, p. 41).

We note here that when President Clinton publicly denounced "social promotion" again

in his 1999 State of the Union Address, he also praised the efforts of Chicago Public

Schools for raising student achievement when it ended social promotion (Clinton, 1999).

Although five years later, it remains to be seen whether the Chicago policy has helped the

students held back in grade (Nagaoka & Roderick, 2004).

The United States Department of Education (hereafter, USDOE) later released the

document, Taking Responsibility for Ending Social Promotion: A Guide for Educators

and State and Local Leaders (1999), where President Clinton directed states and

localities to practice retention as a way to increase student achievement. Here, he

outlined the steps and rationale for ending social promotion, or the practice of simply

moving children from grade-to-grade with their age cohort. To accomplish this goal,

President Clinton described a plan that included rigorous standards aligned to the









curriculum, reduced class sizes, well-prepared teachers, and extra support for students

through after-school or summer programs (U.S. Department of Education, 1999).

National policy spotlights how children learn to read, the best pedagogy for

teachers, and measuring reading growth as children learn how to read. Policymakers

have been led to believe that there is one way, or a best way, for teaching students how to

read as evidenced by a review of research that validates only experimental or quasi-

experimental research (Allington, 2002b). As an example, the Congressionally requested

report of the National Reading Panel (National Institute of Child Health and Human

Development, 2000) focused their review of existing research using a specific set of

methodological guidelines. In conducting their analyses, they concentrated on five

components of literacy, including 1) phonemic awareness, 2) phonics, 3) fluency, 4)

vocabulary, and 5) comprehension, using studies that met stringent guidelines. The

emphasis on systematic and explicit scientifically-based reading research has some

policymakers believing that only particular instructional methods work best for teaching

children to read. While the National Reading Panel (NRP) provided many interesting and

worthwhile recommendations, the report restricted review of other components known to

be essential to teaching children to read such as writing instruction and independent

reading. This review of research was central to recommendations set forth in the

reauthorization of the Elementary and Secondary Education Act, the No Child Left

Behind Act of 2001 (Bush, 2001).

The resounding theme of "every child" is clearly seen in the No Child Left Behind

(NCLB) framework (Bush, 2001), the most recent wide-sweeping educational reform.

Four pillars, as described on the NCLB website, are the cornerstones of this reform,









including 1) stronger accountability, 2) more local freedom, 3) proven methods, and 4)

choices for parents (U.S. Department of Education, 2002). Each of these, when joined

together, stimulate change as efforts to reform the American educational system take

place.

Central to implementing NCLB is understanding the funding structure of this

mandate. The two main sources of federal funding are Title I and Reading First grants.

Title I funds were always designated for use in schools with higher levels of

impoverished students; however, NCLB also provided a host of new stringent guidelines

regulating how these funds may be spent. For our purposes here, it is fundamental to

know that these funds must be spent upon scientifically-based methods and strategies, as

defined by NCLB (and the NRP). In fiscal year 2004, the federal government

appropriated $18.5 billion dollars for Title I.

With the advent of NCLB came another federal grant called Reading First. As the

name implies, this grant is geared specifically at boosting literacy achievement for

students in the primary grades. Beginning with fiscal year 2002, $900 million dollars

were appropriated for this grant with similar funds available for the next five fiscal years

(U.S. Department of Education, n.d.). Florida was one of the first states awarded a

Reading First grant and as of May 2005, Florida's Department of Education has received

an additional $100 million of federal support for districts (Florida Department of

Education, 2005b). However, to receive monies districts (or consortium of districts)

must win a sub-grant from the state's Reading First funding. To obtain these highly

competitive grants, applicants must document how the monies will be spent while

adhering to the State grant approved by the federal government.









In Florida, Reading First grants are reviewed by the FLDOE, as well as faculty

affiliated the Florida Center for Reading Research (FCRR) at Florida State University

(Florida Department of Education, 2005b). Eligibility for Reading First funds differ from

Title I. Title I focuses on improving the educational achievement for students from high

poverty backgrounds; Reading First focuses efforts on educational achievement for

students who struggle with learning how to read in a way that prohibits reaching

accountability. Reading First schools must come from districts with 15% or more of

enrolled students from high-poverty backgrounds, and schools must have 10% or more of

economically disadvantaged students, as opposed to the 40% level of economic

disadvantage to receive Title I. As of the 2004-05 school year, Florida has over 400

Reading First schools.

Link to Accountability

Accountability has been part of policy discussion for years, however the signing of

the NCLB Act of 2001 (Bush, 2001) pushed it into prominence. One key element of this

reform is a rigid accountability requirement-schools and districts across the nation must

meet what is called Adequate Yearly Progress (AYP) by 2014. Federal policy provides

"a single definition of adequate yearly progress, the amount by which schools must

increase their test scores to avoid some sort of sanction-an issue that in the past has

been decided jointly by states and the federal government. And the federal government

has set a single target date by which all students must exceed a state-defined proficiency

level-an issue that in the past has been left almost entirely to states and localities"

(Elmore, 2002, p. 31). State departments of education have the enormous task of

developing accountability plans to comply with the federal mandate (Erpenbach,

Forte-Fast, & Potts, 2003). Hence, the era of accountability is upon us and there is much









information to disseminate from federal, state and district policy initiatives-this

concerns every elementary school principal across our nation.

Accountability legislation can include both rewards and sanctions (Massell, 2001).

NCLB provides serious sanctions for schools and districts not meeting AYP that increase

in severity with each year a school or district does not meet the standard. For some, the

final penalty will result in a full restructuring, meaning that school personnel may be

fired, schools could be converted into a charter school, or even turned over to a private

enterprise. NCLB holds the system responsible for educating children. However, in the

case of retention in grade as a measure of success, the onus of accountability falls on the

student (Massell, 2001). Similarly, McGill-Franzen (1987) asserted in her case about

students' placement in compensatory and special education programs, that we "place the

burden of the problem on the student" (p. 488). I suggest the same is true of retention.

Many problems that students face are out of their control and as a result, they suffer.

Although the system pays, in a sense when accountability is publicized, it is the student

who may carry the life-long consequence of failing a grade.

Standardized Assessments

In this section, we see how standardized assessments are inextricably tied to both

reform and accountability. To add to the context of my study, the reading portion of

Florida's standardized test, the FCAT, will be the focus of this section. The FCAT was

first administered in spring 1999 as part of the state's response to national education

reform. In 1999, Governor Jeb Bush brought high-stakes to Florida using the

criterion-referenced portion of the FCAT as part of the state reform plan, the A+ Plan for

Education, to boost student achievement. This plan assigned letter grades to successful

schools providing monetary rewards, while sanctioning low performing schools (Dorn,









2004). In 2003, Florida attached another high-stake consequence-retention in grade-for

third graders failing the FCAT-SSS, or FCAT.

The FCAT is given in grades 3 to 10, with two parts making up the reading portion

of the FCAT (Florida Department of Education, 2003c). One part is criterion-referenced,

assessing the state mandated standards-based curriculum, the Sunshine State Standards

(SSS). Officially called the FCAT-SSS, most often it is simply called the FCAT. Scores

are reported as Achievement Levels (Level 1 (low) to Level 5 (high)), and

Developmental Scale Scores (DSS), which accumulate over time beginning in third

grade. According to Florida state law, students who score Level 1 fail the assessment.

Interestingly, the FCAT-SSS is said to be one of the most challenging state assessments

currently in use, especially in terms of passage length and text difficulty of the passages

(Hiebert, 2002).

The other portion of the FCAT is norm-referenced (FCAT-NRT), comparing

Florida students to a national sample. Florida's norm-referenced test is a modified

version of the Standard Achievement Test (SAT). Classroom teachers typically

administer both parts of the FCAT with scoring conducted by contractors hired by the

state. Florida's assessments are secure, meaning the tests are highly protected before,

during and after administration (Florida Department of Education, 2000b).

It is widely endorsed that one assessment should not be the sole determinant in a

high-stakes decision, such as retention in grade (American Educational Research

Association, 2000; International Reading Association, 1999). Although Florida, too,

claims to support this, children receive mandatory retention upon achieving Level 1 on

the FCAT (Warford & Openshaw, 2004b). Technically, the FCAT is not the sole









determinant for retention, however other options would not be considered if it were not

for a student failing the FCAT. It is the reverse of the adage-innocent until proven

guilty. Instead, Florida third graders are guilty of failure, unless they qualify for a good

cause exemption, described in more detail later.

Researchers have been studying retention for decades using standardized

assessment data. With many retention decisions tied to high-stakes assessments, it is

important to consider the legitimacy of these decisions and the relationship to education

placements. It has been argued that some placement practices, such as retention and

inclusion in special education programs, are the result of standardized assessments and

the stakes attached to them (McGill-Franzen & Allington, 1993). After studying

low-achieving children in multiple contexts, concern surfaced that some schools retained

students before third grade to delay accountability penalties by classifying more students

in special education programs. These efforts appeared to increase the reported

achievements on a high-stakes assessment in these schools (McGill-Franzen & Allington,

1993).

Lastly, the State of Florida has relied on core reading programs as the panacea to

help students read more proficiently, with a goal of increased performance on FCAT to

meet accountability measures. In 2002, the State Textbook Adoption Committee found

six core reading programs meeting the adoption criteria, meaning these programs are

research-based and aligned to the states' reading curricula (Florida Department of

Education, 2003b). In a study of third grade core reading programs used in Florida, two

programs were selected for a content analysis focusing on the instruction provided to

students (McGill-Franzen, Zmach, Solic, & Love Zeig, in press). Using school-level









data, these researchers worked to understand the relation between the percentage of

third-grade students scoring Level 1 on the FCAT to the core program used, determining

if schools using a particular program had an academic advantage. Using each school's

percentage of students eligible to receive free or reduced price lunch as a proxy for

poverty, McGill-Franzen et al. (in press) found that school level of poverty was a better

predictor for academic achievement than the core program used. High-stakes

standardized assessments have many unintended consequences, particularly for those

most at risk for retention-students attending schools with higher rates of poverty.

High-Stakes Retention Policy

Built on the idea of setting high expectations for all students, standards-based

reform resulted in standardization of the curriculum. It seems inherently logical that we

want to test this knowledge to determine the effectiveness of instruction on students'

learning as measured by standardized evaluations. However, assessments became

high-stakes after attaching consequences and rewards. When then-President Clinton

publicly called for the end of social promotion in his 1999 State of the Union address, it

was clear that federal policy makers felt a decisive measure was needed to establish

accountability (National Research Council, 1999). Thus, the emphasis on standardized

assessments given in schools across the nation changed to hold students and schools

accountable to achieve certain academic standards at specific points in a school career.

Assessments moved from low- to high-stakes as they became used for retention and

promotion decisions, or in grading schools. Retention policies based on students'

performance on high-stakes assessments are becoming more common.









Policy Interpretation and Implementation

The last stage of the high-stakes framework described here involves how policy is

interpreted and implemented. Policy logic, as described by Allington (2001), "proceeds

on the assumption that implementing particular policies will have some intended effect"

(p. 275)-there are two essential facets to this logic. The first presumes that intended

policy will shape (or reshape) instruction. The subsequent presumption is that this

reshaping will generate the desired results via change-sometimes in instructional

practices (Allington, 2001). Those who study educational change know the complexities

of this process (Allen, Cary, & Delgado, 1995; Flinders & Thornton, 1997; Fullan, 2001;

Tye, 2000). Principals who handle change the best are more successful in implementing

new programs and policies (Fullan, 2002).

Vast amounts of research have occurred on the interpretation and implementation

of educational policy with teachers. Thinking about the school-level context, research

has shown that the policies introduced by state lawmakers often have few similarities to

actual practice (McGill-Franzen, 2000). Researchers have encountered teachers who

were attempting to implement policy, but in actuality were creating their own policies

based upon the intended legislation (Spillane & Jennings, 1997, cited in McGill-Franzen,

2000, p. 901). One would suspect that the same is true of school administrators. As

policies are transformed from legislative intent to practice, there can be some loss in the

translation. "What actually is delivered or provided under the aegis of a policy depends

finally on the individual at the end of the line" (McLaughlin, 1987, p. 174). If we

characterize the typical elementary school teacher as the "end of the line," then the school

principal acts as the intermediary between the district and the teacher. This junction









serves an important purpose in understanding what ultimately happens at the end of the

line.

Resistance to policy is one reason for a policy's demise, when in fact, for many, it

is a lack of understanding on the part of those charged with the implementation

(Darling-Hammond, 1990). Policy analysts call for awareness on how policy is enacted

locally. This insight can help guide the successful implementation of the said policy.

The institutionalization of a policy depends on this understanding. Other problems in

policy implementation exist. Lack of a core understanding is illustrated in the case study

example of a teacher named Carol Turner, who in her mind, perceived that she was

successfully implementing the new California Mathematics Curriculum Framework

(Ball, 1990). With a lack of understanding of the new mathematics framework and a new

textbook in hand, Carol was under the misguided impression that following the new

textbook constituted enacting the new curriculum. Although it is unclear how a textbook

can be the impetus for change, nonetheless in this case, it was viewed as the "messenger

of change" (Ball, 1990, p. 257).

The notion of a messenger is vital. Much of the literature overlooks the role of the

principal, who can be seen more in the role of policy messenger, than anything else.

Nevertheless, even as a messenger, who may be deemed powerless by an instructional

staff, the principal is the gatekeeper of knowledge through which the details of the policy

permeate to the classroom teacher (Ball, 1990). As the instructional leader of a school,

how a teacher learns the essentials of a policy is the responsibility of the principal or their

designee. Valencia and Wixson (2004) explain a model illustrating a multi-level system

through which policies are shaped as they move toward the core of education-the









classroom. As policy trickles through the system, there are many possible outcomes.

Depending upon factors such as opinion or perception of a policy, there can be positive or

negative conclusions to a particular policy (Valencia & Wixson, 2004).

We know implementation of policy is not simple-it is "a complexly interactive

process without beginning or end" (Lindblom & Woodhouse, 1993, p. 11). It takes

direction to achieve the desired results. With that said, past research has found that when

it comes to policy implementation there are many school principals who feel less than

adequate when enacting new policy (Musella, 1989). Senior administrators in a large

school district requested Musella observe an in-service session as principals learned

about a new governmental policy. As Musella witnessed this training, he noticed

concerns regarding ambiguity of the policy goals, nonspecific policy guidelines,

frustration about the lack of input in the policymaking process, inadequate funding

resources, and insufficient time to enact changes. In an effort to share his observations,

without pointing out the obvious faults in their process, Musella shared research on

typical reasons for opposition to change (Zander, 1962, cited in Musella, 1989). A

strikingly obvious, but often overlooked reason for policy failure is that "those

responsible for changing are not involved in the planning" (Musella, 1989, p. 95). The

principals observed felt powerless; they were not part of the change process; however,

they were expected to enact a major governmental policy.

How administrators implement and understand new policy is important (Allington,

McGill-Franzen, & Schick, 1997). Simply because a policy exists, does not ensure that it

is acted upon, or adequately understood. Allington et al. (1997) described many

important characteristics in their qualitative study of administrators' understanding of









learning disabilities. Although many of the administrators interviewed supported state

and federal programs, one essential finding of this study was that administrators

perceived "creating and funding almost any intervention was someone else's

responsibility" (Allington et al., 1997, p. 231). These findings are fundamental to the

third-grade retention policy in Florida. How principals implement the retention policy

may rest more upon their beliefs, as policy implementers do "not always do as told"

(McLaughlin, 1987, p. 172).

Florida Policy Context

Although the federal government endorses the use of retention in grade, this is a

policy matter for local and state policymakers, such as a state department of education or

a school district. The Florida policy mentioned thus far in my study was not the first

attempt to enact a statewide retention policy. In 1999, the Florida Legislature did not

pass a law for retention decisions based solely on a student's FCAT score for

fourth-grade students (Florida Department of Education, 2003a). Instead, student

performance throughout the school year was deemed appropriate to make promotion and

retention decisions, not simply a score from the FCAT. At that time, the Legislature

decided that retention decisions should be left to the discretion of individual school

districts. In August 1999, in lieu of a formal policy, the FLDOE recommended three

options to school districts for students who were not meeting their district's progression

plan. Options included to, "(1) remediate students before the beginning of the next

school year and promote, (2) promote and remediate in the following year with intensive

remediation, and (3) retain and remediate" (Florida Department of Education, 2003a, p.

4). Even after making these recommendations to districts, the State tried again to enact a

retention policy and this time met success.









Policy Enactment

The 2002 Legislature passed a rewrite of the Florida School Code (F.S. 1008.25)

instituting a mandatory retention policy into effect on January 7, 2003, immediately

ending social promotion in Florida schools (Florida Statutes, 2002). Of interest to this

study is the mandatory third-grade retention policy enacted during the 2002-03 school

year.

"The new law set the Grade 3 reading FCAT as the critical gateway to identify

students who, after remediation, are still unable to demonstrate reading proficiency and

clearly need more time to learn the basic skill of reading" (Florida Department of

Education, 2003a, p. 2). Here is what third-grade students encounter: The law asserts that

third-grade students, who do not achieve Level 1 on the reading portion of the FCAT,

fail. Students performing at Level 1, as defined by the State, are said to have experienced

little success with the standards-based curriculum, the Sunshine State Standards (Florida

Department of Education, 2000a). This policy focuses on identifying students who need

a stronger literacy foundation "...regardless of the reason that is causing it-even a

learning disability, limited English proficiency, or a disadvantaged background-needs to

be addressed and corrected before the student can be expected to move successfully on to

the more difficult work of the higher grades" (Tremor & Butler, 2004, p.1). The

Legislation has mandated school districts to "allocate remedial and supplemental

instruction resources to students in the following priority: (1) first-students who are

deficient in reading at the end of grade 3, and (2) next-students who fail to meet

performance levels required for promotion" (Florida Department of Education, 2000a, p.

2).









Retention loopholes

As mentioned earlier, loopholes (or narrow exceptions) written into this law, called

the "good cause exemptions," allow some children to forego retention (Florida

Department of Education, 2004d). Currently, six good cause exemptions allow

promotion to fourth grade, if documented. Third-grade students meeting one of the

following good cause options are eligible for promotion to fourth grade, even after

scoring Level 1 on the FCAT

1. Limited English proficient (LEP) students having two years or less of English
instruction

2. Students with disabilities (SWD) not participating in the state assessment because
their individualized education plan (IEP) or Section 504 plan indicates that FCAT
is not an appropriate assessment

3. Students scoring at the appropriate level on an alternative assessment

4. Students meeting district determined criteria to score the equivalent to Level 2 on
FCAT via a good cause portfolio

5. Students with disabilities (SWD) previously retained in grade receiving two or
more years of remediation

6. Students who have been retained two years in any grade, kindergarten to grade
three, and have received two or more years of remediation

(Florida Department of Education, 2004d)

Here is further explanation for Options 3 through 6. Option 3, the FLDOE

recommends districts use the Stanford Achievement Test (SAT) as the alternative

assessment for students who fail the FCAT-SSS. Because of the challenging nature of

the FCAT, "this is one of the reasons why, in third-grade, a student must achieve at the

51st percentile on the nationally normed SAT9 test of reading in order to insure that they

have sufficient reading ability to achieved above Level 1 on the FCAT" (Torgesen, 2004,

p. 2). On May 17, 2005, the FLDOE provided a new cut score for the FCAT-NRT, now









version 10 of the SAT, to at or above the 45th percentile as announced in a memorandum

for District School Superintendents (Warford & Openshaw, 2005). Districts electing to

use the SAT-10 as their alternative assessment will adhere to the 45th percentile or above,

while districts still using the SAT-9, described by Torgesen (2004), must continue using

the 51st percentile or above as the criteria. Depending on the version, SAT-9 or SAT-10,

students must meet the designated cut-point to be eligible for a promotion with good

cause.

Option 4 allows a good cause portfolio designed by school districts. Each district

had the opportunity to create a portfolio for classroom teachers to compile on behalf of

students identified at risk to fail the FCAT. The FLDOE notes that only teachers may

initiate and compile these portfolios. The state recommended four districts, Citrus, Clay,

Orange and Pasco, as examples for other districts to use as models to establish their

portfolio criteria (Florida Department of Education, 2005). Good cause portfolios are

designed to be an on-going, collection documenting students' classroom performance

equivalent to Level 2 or higher on the FCAT. If a student fails the FCAT, the teacher

submits a portfolio to the school principal to determine if the portfolio meets the criteria

to promote the student for good cause.2 "If the school principal determines that the

students should be promoted, the principal must recommend it in writing to the district

superintendent. The district superintendent must accept or reject the school principal's

recommendation in writing" (Florida Department of Education, 2004a, p. 8).


1It is unclear how the FLDOE determined the new cut-point for the SAT-10. Based on information provided by
the test publisher, Harcourt Assessment, the SAT-10 cut-point mandated by the FLDOE is lower than the
equivalent calculated by the publisher. Harcourt equates a 51st percentile rank on the SAT-9 to a 46th percentile
rank for the total reading score, not the 45th percentile.
2 For more about good cause portfolios, please visit www.firn.edu/doe/commhome/progress/proghome.htm.









The latter two good cause promotion exemptions are complex with numerous

scenarios by which a student could be promoted. Complexities arise when understanding

the options available to those students previously retained in K to 3. Option 5 relates to

students with disabilities (SWD) retained in any grade, kindergarten to grade three. For

these students, they may be promoted to fourth-grade after being retained one time;

however, they must have received at least two years of reading remediation.

Hypothetically, a SWD who was retained in first grade and received extra remediation

during the year of retention and in second grade could be promoted to fourth grade after

failing the third-grade FCAT. Another hypothetical student may have been identified in

second grade as a struggling reader. However, this SWD student received remediation in

second grade and was promoted to third grade. After a second year of remediation, this

student would not be eligible for promotion because the child was never retained. After a

second year in third grade, this child would be eligible for a promotion with good cause,

if they still do not pass the third-grade FCAT.

The final good cause exemption, option 6, allows promotion for good cause to any

"students who have received the intensive remediation in reading for two or more years,

but still demonstrate a deficiency in reading and who were previously retained in K-3 for

a total of two years" (Florida Department of Education, 2004a, p. 7). This good cause

promotion set a limit on the number of times a student may be retained in K to 3,

provided they received the appropriate remediation (Warford & Openshaw, 2004b),

although Florida does not currently limit the number of retentions a student may

experience in elementary school. As can be seen, good cause exemptions are stringent









and most complex. In a sense, upon failing the FCAT, students' promotional decisions

are treated on a case-by-case basis.

Lastly, state law requires districts to report the specific type of Good Cause

Exemption used to promote a student each year, beginning with the 2002-03 retentions

(Warford, 2004b). Using data provided by the FLDOE, Table 2-1 disaggregates the

number of Florida students promoted with good cause after the first and second years of

the policy.

Table 2-1. Florida retention policy good cause statewide exemptions
2002-03 2003-04
# % # %
1) Limited English proficient students with 2,974 22.80 2,511 11.79
fewer than 2 years in English as a second
language program

2) Students with disabilities (SWD) not 1,016 7.79 1,647 7.73
participating in statewide assessment as per
individualized education plan (IEP)

3) Students who demonstrate proficiency on an 3,307 25.35 3,845 18.05
alternate assessment

4) Students who demonstrate proficiency 1,514 11.61 3,468 16.28
through a portfolio

5) Students with disabilities (SWD) retained 3,637 27.88 7,906 37.12
once with 2 or more years of remediation

6) Students retained twice with 2 or more years 598 4.58 1,924 9.03
of remediation

TOTAL PROMOTED WITH GOOD CAUSE 13,046 21,301
Note. Data provided courtesy of Florida Department of Education (L. Fleming, personal
communication, June 6, 2005).

Let us compare the numbers of students scoring Level 1 on the FCAT to the

numbers of students promoted for good cause to understand the impact of this policy.

Over 43,000 students scored Level 1 on the FCAT in 2002-03, and the FLDOE reported,









as shown in Table 2-1, that after the 2002-03 school year over 13,000 students were

promoted with good cause to fourth grade. Based on this information, it would appear

that nearly 30,000 students were retained after the first year of policy implementation.

However, according to data also collected by the FLDOE, state files document that over

27,000 third-grade students were retained after policy implementation and after

application of good cause exemptions and we will rely upon this number.3

Policy Guidelines

Much of the retention policy echoes the mandates set forth in NCLB. Identifying

students who struggle with reading and evidencing their reading achievement are central

to both national and statewide reform efforts. However, like retention, determining the

best technique to help students who struggle with reading has a long and contentious past

(Allington & McGill-Franzen, 2000). Rather than debate here which techniques are best,

I present the mandates and suggestions endorsed by the FLDOE in the 2004 Legislative

Intent and other technical reports disseminated to Florida schools via the state paperless

communication system (Florida Department of Education, 2004a; Tremor & Butler,

2004; Warford & Openshaw, 2004a, 2004b, 2004c, 2005).

The Legislative Intent (hereafter, Intent) for the state third grade policy (F.S.

1008.25, Section 6) has three main areas of focus, including: 1) determining reading

proficiency, 2) parental notification, and 3) developing comprehensive plans to address

the Intent goal for all students to read at or above grade level. Two new initiatives were

also announced by FLDOE officials in the 2004 Intent (Florida Department of Education,

3 School districts reported data to the State during the same collection period in August 2003. The reason for
these discrepant numbers is unclear; however, for the purposes of my study we will rely on the numbers
disaggregated by school (i.e., approximately 27,000 third grade students retained in 2002-03), not the total
extrapolated by the researcher, even though these should have been nearly the same. One FLDOE official
suggested the amount may be discrepant due to students moving. Although this suggestion would only apply to
state numbers if students were moving out-of-state, not within the state.









2004a; Warford & Openshaw, 2004a). One, called Reading Enhancement and

Acceleration Development (READ) Initiative, is a retention prevention program aimed at

students in grades kindergarten to three, including newly retained third graders. Multiple

lines of legislation outline several steps and sub-steps that comprise this initiative (s.

1008.25(7) (b) 7, F.S.; Bill page 17, line 1 page 19, line 23) (Florida Statutes, 2002).

Another initiative calls for establishing classrooms for "retained third grade students who

subsequently score at Level 1 on the reading portion of FCAT" (F.S. 1008.25(7) (b) 8;

Bill page 18, lines 1 23) (Florida Statutes, 2002). These classrooms, called an Intensive

Acceleration Class, have specifications outlined in the legislation. A brief overview of

how at risk students are identified, how parents are notified and how students are

supported follows. 4

Identification

First, all students in Kindergarten through Grade Three (hereafter, K to 3) have

their reading proficiency determined and monitored. The FLDOE is focused on

identifying students in K to 3 who lack the required proficiency to pass the third grade

FCAT, defining these students as "exhibiting a substantial deficiency in reading" and

"must be given intensive reading instruction following the identification." To make

diagnoses, elementary schools are required to assess their students "regularly" to identify

the "exact nature of the student's difficulty in learning to read." Reading First school

students must be assessed using the statewide assessment system, the Dynamic Indicators

of Basic Early Literacy (DIBELS; Good & Kaminski, 2002). Non-Reading First schools

must also identify their students, however in addition to the DIBELS, these schools may

4 To provide the reader the opportunity to hear the tone of the 2004 Legislative Intent, specific terms or wording
used in the Legislative Intent were included here using quotation marks.









select from other assessments. Newly identified students are then subject to a

remediation plan that must be provided "during regular school hours." All of this can be

likened to a critically-ill person being treated as an out-patient.

Notification

Upon labeling a student as reading "deficient," parents must be notified as soon as

possible with a "description and explanation, in terms understandable to the parent." It is

expected that parents be consulted in developing an academic improvement plan (AIP)

and that parents are provided a list of the supplemental instructional services and supports

to be provided "until the deficiency is corrected." Schools are to keep parents apprized of

their child's progress. It is also required that parents be provided strategies to help their

child with reading. At the end of third grade, parents are also alerted that if the services

and supports do not correct the deficiency, "as demonstrated by scoring at Level 2 or

higher on the statewide assessment test in reading for grade 3," then the student must be

retained, unless they qualify for promotion with good cause.

Retention plus remediation

In addition to diagnosing students and sharing this news with parents, the Intent

specifies how to remediate students. Schools and districts must follow the regulations of

the READ Initiative for those students who qualify. This Initiative aims to prevent

retention for students in K to 3, and for the newly retained third graders, it offers an

accelerated, or intensive approach, to help prepare students for promotion to fourth grade.

Each student must receive extra support during the regular school day, which is in

addition to the mandatory 90-minutes of daily, uninterrupted reading instruction utilizing

a scientifically-based program. These are the state approved and Florida Center for

Reading Research (FCRR) reviewed core reading programs focusing on the five areas of









reading identified by the NRP: 1) phonemic awareness, 2) phonics, 3) fluency, 4)

vocabulary, and 5) reading comprehension (National Reading Panel, 1999). Ongoing

progress monitoring must also be provided. All core subject areas, such as science,

mathematics, social studies, are to be incorporated into the school day, in addition to the

above.

Double retention plus remediation

In 2004-05 a new initiative began for students who were already retained in third

grade, but did not pass the FCAT again. Each district must establish an Intensive

Acceleration Class and qualified students must participate. The goal of this program is to

accelerate learning for students to gain two years worth of material in one year.

Theoretically, the FLDOE suggests that successfully remediated students could advance

to fifth grade and by-pass fourth grade. This child could potentially rejoin his or her

same age cohort. The Intent does not suggest that students would advance to the fourth

grade via this program. To accomplish this extraordinary feat, the Intent explicitly

describes several rules for districts (Florida Department of Education, 2004a, pp. 11-12).

It recommends reduced teacher-student ratios, however ratios are not defined. The

FLDOE notes that class sizes are expected to be smaller than other third-grade classes.

Teacher-student contact time is expected for most of the day. In other content area

subjects, such as Mathematics or Science, students are expected to be taught from the

fourth-grade Sunshine State Standards. As well, the fourth-grade Language Arts strand is

used. This provision helps ensure students will have the necessary background

knowledge to cope if promoted directly to fifth grade. Students in an Intensive

Acceleration class also have services of a speech-language pathologist, if needed.









Students are to be assessed weekly. Districts must monitor all students taking part in this

Initiative.

Central supports

Several supports are embedded within the Intent for all identified students with

reading difficulties. State law requires districts to offer certain supports, while others are

suggested, but not mandated. Six supports (Table 2-2), central to my research, and their

purpose within the Florida retention policy context, are described here (Florida

Department of Education, 2004a).

Table 2-2. Supports identified in Florida's legislative intent
Required Recommended

Focus on early intervention Extended learning opportunities
Current academic improvement plan Transitional class available for retained
(AIP) students
On-going portfolios meeting state Mentor or tutor with specialized training
requirements

Focus on early intervention. Students who are identified as early as kindergarten

are provided extra supports. As part of the establishment of the READ initiative

described earlier, third grade students are provided an accelerated curriculum and

students in grades K to 2 are provided intervention to foster their reading development.

The central tenet here is to help prevent third-grade retention. No laws exist to retain

children before third grade. Prevention can benefit students experiencing difficulty

(Allington, 2002a; Allington & Walmsley, 1995).

Current academic improvement plan (AIP). An Academic Improvement Plan

(AIP) formally documents the remedial strategies provided to the student over a period.

The FLDOE requires schools to collaborate with parents when making or revising an









AIP. The AIP must name which of the five areas of reading identified by the NRP (1999)

is lacking.

On-going portfolios meeting state requirements. All retained third graders must

have an "active, ongoing portfolio" that may be used as part of the good cause promotion

options. As mentioned earlier, these portfolios are district-created, so there is expected

variation across districts. Essentially, these teacher initiated and compiled good cause

portfolios must contain evidence that accurately reflects whether the student can achieve

comparable to Level 2 on the third-grade FCAT. Both the school principal and district

superintendent will review and review these portfolios.

Extended learning opportunities. All students who have trouble with reading

must be provided extra intensive support beyond the 90-minute reading block. The

FLDOE advocates the use of extended learning outside the regular school day, although

by definition, this would be in addition to the extra support that must be provided during

the regular day beyond the 90-minutes of instruction. Extended learning may occur

before- or after-school, or on the weekend, such as a Saturday School. Another option

includes the use of an extended school year beyond the minimum 180-days. Extended

learning, as suggested by the FLDOE, is different from summer school. All students who

fail the third grade FCAT must be provided the opportunity to attend summer school, or

as they are called in Florida, Summer Reading Camp.

Transitional class available for retained students. The Intent mandates that

retained students must be offered the choice of a transitional type setting with the purpose

of producing learning gains. The goal of such classes, as noted by the FLDOE, is "what

is being provided to help the student catch up, not where it is being provided" (Florida









Department of Education, 2004a, p. 13) [emphasis in original]. Configurations may vary,

and schools may choose transitional classes that contain third- and fourth-grade students

or re-retained third graders only. Districts may elect to offer the transitional classroom at

a central location. This type of class is akin to transitional classes preparing students for

first grade (e.g., pre-first grade, junior first grade, readiness room) (Shepard, 1989).

Mentor or tutor with specialized training. All parents of retained students must

be provided with either "supplemental tutoring in scientifically research-based reading

services in addition to the reading block" (Florida Department of Education, 2004a, p.

10). This tutoring may happen before and/or after school. Alternatively, students may be

provided with a mentor or tutor with specialized reading training, as opposed to using a

scientifically research-based program.5

The FLDOE provided many alternatives for districts to select with their allocated

funds. Other options recommended in the Intent include: reduction in teacher-pupil ratio,

although the FLDOE does not endorse a capacity or class size. The Intent also

recommends more "frequent" progress monitoring, which may include the assessments

found within the scientifically research-based programs. A frequency is not prescribed.

Implementing the policy

Policymakers acknowledge the complexities of the process (Hart, 1996). To cope

with the policy fluctuations, local school districts are provided with communications and

meetings as a means to guide changes and aid understanding in a timely manner. They

serve as a way to distribute the recipe with state policymakers telling districts what needs



SIn September 2004, the FLDOE surveyed districts to learn which of the three options they selected to provide
parents of retained students. Districts are required to provide at least one of the following: 1) tutoring with a
research-based program, 2) a tutor or mentor trained in reading, or 3) a "Read at Home" contract.









attention and how to do it. To assist struggling readers, Florida mandates schools and

districts use special classes, use certain types of curriculum materials, as well as provide

summer school and make certain services available. Timely explanations are needed to

assist policy implementers to understand mandates as this law continues to evolve.

The FLDOE is committed to providing updates and technical assistance papers

concerning this policy via an open-access paperless communication system. Although

many documents are intended for school district superintendents or other district leaders,

interested members of the public are free to register for an e-mail service that delivers

documents upon release. As of June 2005, twenty-four notices dating back to August 26,

2002 were posted on the FLDOE website regarding this policy, plus related attachments

(Florida Department of Education, 2005c), although many other documents have been

distributed since the last entry on January 11, 2005. Recent documents are presumably

available elsewhere on the FLDOE website.

Following the first year of policy implementation (2003-04), the FLDOE sent an

electronic memorandum to Florida's district superintendents outlining amendments of

two Florida state educational laws (F.S. 1002.20 and 1008.25) (Warford & Openshaw,

2004a). As part of this paperless communication system, a carbon-copy (cc:) of this

memorandum was also sent electronically to the Assistant Superintendents for

Curriculum and Instruction, Directors of Student Services, Directors of Elementary

Education, Directors of Exceptional Student Education, Elementary School Principals,

and Elementary Guidance Supervisors of each district. Revisions to 2004-05 student

progression plans require changes, many of which relate or interrelate to the third-grade

retention policy. Essentially, the memorandum provides a lengthy list of requirements









for each elementary school with a copy of the bill attached. Specifically, the FLDOE

requires each school to

assess the reading ability of each K-3 student, provide parents with notification of
any reading deficiency, implement a detailed academic improvement plan (AIP),
and provide intensive reading instruction; revise the required notice to parent of
third grade students with substantial reading deficiencies to include information
about additional evaluations, portfolio review and assessment to determine whether
the student is ready for promotion, and information on the district's specific criteria
and policies for mid-year promotion; define mid-year promotion; make a technical
correction related to students who were previously retained in grade three; for third
graders who are retained, require appropriate intensive interventions, including the
provision of summer reading camps; specify the activities and supports to be
provided to retained third graders, including the use of a state-identified reading
curriculum that meets certain specifications; provide for intensive acceleration for
students currently retained who score Level 1 in reading; require a report to the
State Board of Education on the interventions provided; and require the option of
placement in a transitional setting for retained third graders (Warford & Openshaw,
2004a, p. 1).

After school districts had opportunity to review the wide-ranging edicts from the

memorandum outlining the legislative changes, the FLDOE issued another memorandum

with a chart clarifying the old and new legislative intents (Warford & Openshaw, 2004b).

This chart, covering 14 pages, provides comparison of the original intent to the current

modifications (Florida Department of Education, 2004a; Warford & Openshaw, 2004b).

This serves as evidence that the FLDOE does provide support to districts in their efforts

to decipher and enact changes in a timely manner.

Even with these technical assistance reports, questions and confusion exist as

implementing this policy continues. In October 2004, members of the FLDOE Just

Read! Florida6 office traveled to multiple sites across the state holding meetings, offering

assistance to principals and district-level administrators. I had the opportunity to attend


6 Members of the Florida Department of Education Just Read! Florida office participated in writing the
Legislative Intent for the retention policy. This office oversees the technical assistance of this policy. Also,
these October meetings were originally slated for September 2004, but were rescheduled due to hurricanes.









the October 7th meeting held in Tampa, Florida. The purpose of the meeting was for

attendees to receive clarifications to their implementation questions, and for attendees to

provide input prior to the impending passage of the mid-year promotion rule. Several

matters were discussed at length; however, I will provide just two examples. First, the

FLDOE, as described earlier, introduced two new initiatives, READ and Intensive

Acceleration, for the 2004-05 school year. When these initiatives were introduced and as

they were written in the Legislative Intent, they were referred to by these titles.

However, the FLDOE released another document that described these initiatives as

a "tiered" system. At the meeting, members of the FLDOE used the titles

interchangeably, causing confusion. After explaining that "Tier 1" (identified as at risk

for retention) and "Tier 2" (retained once) students participate in the READ Initiative,

while "Tier 3" (retained two times in the same grade) students participate in Intensive

Acceleration Classes, confusion appeared to ease. Another area of questioning related to

the number of times students may be retained. Participants received advice regarding

how to handle specific cases for students with multiple retentions explaining how

students who received two years of intensive intervention may not be retained in a grade

for more than two years. After two years of intensive intervention, students would be

eligible for a good cause exemption. The observed scenario was reminiscent of what

Musella (1989) described in his work observing an in-service training. Similarly, I heard

concerns regarding ambiguity of the interventions and unclear policy guidelines.

Summary

Themes resonating from the past research on retention indicate that 1) students

need to be treated as individuals when making a life-altering retention decision and 2)






47


students who are at risk for retention or retained need adequate support to help them

achieve their highest potential. As we can see, retention is deeply rooted within the

high-stakes framework presented and examined within this chapter. In Florida, simply

because FCAT achievement data appears to point toward success, how has the state

managed to raise achievement while drastically increasing the number of students

retained in grade? In order gain more insight into this policy, we work to understand how

Florida elementary school principals provide support within the given framework.














CHAPTER 3
METHODS

Introduction

Historically, retention has been a dichotomous debate with little middle ground;

however, in the previous decade, it has gained acceptance with more states using it as a

component of a state accountability scheme. Florida is one such state to enact a policy

preventing promotion in grade when students do not meet set criteria. The impetus of

this policy is to help the state meet the national challenge to have all third-grade students

reading on grade-level by 2014. Florida is determined to beat the national goal, calling

for districts to have all third-graders reading on level by 2012. This retention policy is

relatively new with little known about school-level implementation.

The foundation of my study was rooted in a multilevel model considering factors,

such as level of poverty in their schools, while determining whether the past retention

practices of a principal influenced the level of current support offered to students at risk

for retention or currently retained. This multilevel model allows the reader to look

broadly-discerning whether retention practices have changed since the start of this

policy. Using 5 years of percent retained data enabled examination of differences in

retention rates before and after policy implementation. As recommended by Allington

(2001) for the study of policy, data from pre-policy and post-policy provide opportunity

for comparison. To better understand the school years discussed in the balance of this

study, Table 3-1 displays the school years and their relation to the start of the retention

policy.









Table 3-1. Time in years: Pre-policy versus post-policy
Year School Year Retention Policy

1 1999-2000

2 2000-2001 Pre-Policy

3 2001-2002

4 2002-2003
Post-Policy
5 2003-2004


Participants

Population

To study the Florida third grade retention policy, Florida principals were selected.

This State has 67 traditional public school districts encompassing over 1,800 elementary

schools. Each school district covers an entire county.

Sample

For this study, districts were selected both purposefully and randomly for a total of

920 possible public elementary schools.1 The six districts selected with purpose are

among the largest of Florida's school districts, while the other six districts were selected

at random from the remaining available districts. This sample included districts from

each geographic region of the State the panhandle, the north, the south, landlocked

areas, the Gulf coast, and Atlantic coastal regions. Within and across all districts,

combinations of urban, suburban and rural schools exist (National Center for Education

Statistics, 2002).

Prior to submitting proposals to conduct research in these districts, ten of the

twelve districts expressed interest in participation, responding positively with a letter of


1 Charter schools were not included in this study.









interest to the researcher. The other two districts welcomed submission of the research

request. Of these, all districts willing agreed to partake in the study after independent

review by district representatives. Across all districts, I sought formal school district

approval to request principals' voluntary participation in the Florida Principal Survey.

One large district restricted access to a cluster of thirty elementary schools participating

in a district research evaluation, reducing the available sample from 920 elementary

school principals to a maximum of 890 possible elementary school principals.

Confidentiality Protection

In the district approvals, it was agreed that I may refer to districts by enrollment

group, as opposed to specific district names. Based on their PK to 12 district enrollments

gathered using the Common Core of Data Build a Table tool (National Center for

Education Statistics, 2004), I classified these 12 districts into three groups: 1) small, 2)

mid-size, and 3) large. Districts with fewer than 30,000 students enrolled were classified

as small; districts with greater than 30,001 students enrolled were deemed mid-sized.

The other six districts, with over 100,000 students enrolled, were classified as large. This

classification system may be unique to Florida as its districts are countywide. To protect

the identities of the participating districts, specific enrollment figures are not displayed.

Also, specific school and principal names will not be revealed, nor released publicly.

Each principal was assigned a unique identification code as an additional safeguard to

protect the identities of the respondents. Appendix A shows the Florida Principal Survey

that participants received.









Data Collection

Survey Instrument

As this policy stretches across Florida, survey design was selected as a data

collection method because it is representative, objective, quantifiable and systematic

(Isaac & Michaels, 1995). The survey questionnaire, researcher created, referred to as

the Florida Principal Survey contains three distinct sections. Part 1 has four items

requesting respondents to identify their name, school and district, as well as the unique

identification number provided in their consent letter. Part 2 contains items for principals

to share their attitudes and practices related to the retention policy. Most topics for the

survey were derived after reviewing the Intent memo the FLDOE sent to schools and

districts outlining the legislative changes (Warford & Openshaw, 2004b).

For my study, the focus of item construction was the support available for at risk

students, including those at risk for retention, those currently retained, or those retained

again in third grade.2 The closed-ended items in this study were fixed-alternatives. One

item offered two response options (Figure 3-2) and six items were categorical by design

with response choices, "yes, no, or not sure" (Figures 3-3 and 3-4). However, the "not

sure" option was not selected by this sample. For the purposes of this study, these items

were treated as dichotomous, "yes, no." Part 3 of the survey requested the principal to

enter the years of career experience as an educator, the total years as a principal, and the

length of time spent as principal at their current school. A summary of the pilot survey

review follows in Appendix B.


2 Other items included in the survey are for a separate report.









Retention belief

Item 9 served to identify the retention belief of principals as related to the use of

one standardized assessment result to determine a retention decision. As current policy

bases the decision on FCAT performance, it was salient to this study to understand

principals' responses. By design, principals were asked to select between the two

options. Principals were provided the response choices shown in figure 3-2. They could

click or check the option best matching their current belief.

Directions: Knowing that individual circumstances can exist, what is your response to
the following statement?

"On the whole, retention in grade can benefit 3rd grade students who score Level 1 on
the FCAT"

Item Response Choices
Number
Q9 Yes, retention is a beneficial option for students who fail
the FCAT.

No, retention is not a beneficial option for students who fail
the FCAT.
Figure 3-1. Item 9, Florida Principal Survey

Resources for at risk students

The next two groupings of items provided an opportunity for principals to report on

the presence of different types of supports or monitoring systems currently used in their

schools. As legislation has strongly suggested that schools and districts use particular

supports to remediate their students, I wanted to learn which supports were being used by

schools. Survey items are presented in figures 3-3 and 3-4.

Degree of support

Based on the data collected, it would not be possible to make determinations about
























Figure 3-2.


Item 14, Florida Principal Survey


Directions: Which, if any, of the following are used as strategies with retained third
grade students?

Item Survey Item
Number
Q19a A mentor or tutor with specialized reading training
Q19b Extended school day (such as After School Program,
Saturday School, or Extended School Year) as defined in the
NCLB legislation
Q19c Ongoing portfolios that meet state portfolio requirements

Figure 3-3. Item 19, Florida Principal Survey

the quality of these supports as implemented in each school; however, I decided to

examine the number of supports reported as implemented to determine if any patterns

exist. A new variable created for the planned analyses lends itself to help explain

retention patterns for schools using fewer or more supports, that is the degree of support

provided in each school. A categorical variable called suppsys3grp was created based on

principals' responses to the item numbers in Table 3-2. Schools reporting use of any 1 or

2 supports provided a lower degree of support options; schools using any combination of

3 or 4 supports provided a medium degree of support options; and schools using 5 or 6

supports provided a higher degree of support options.


Directions: Think about programs or initiatives currently in place at your school
designed to help at-risk students.

Item Survey Item
Number
Q14b Does your school emphasize early intervention programs in K
2 to prevent retention in 3rd grade more now than in the past?
Q14c Do ALL retained 3rd grade students in your school have a
current AIP?
Q14e Is a transitional class (a class designed for the retained and/or
re-retained) available for retained students?









Table 3-2. Degree of support category, Florida Principal Survey
Item
number Support type

Q14b Focus on Early Intervention
Q14c Current Academic Improvement Plan (AIP)
Q14e Transitional Class available for retained students
Q19a Mentor or Tutor with specialized training
Q19b Extended Learning Opportunities
Q19c On-going portfolios meeting state requirements


Employment history

Between these items (Figure 3-4), there was enough information, for the purposes

of this study, to determine how long the respondent held a principalship at their current

school. The purpose of the length of the principalship at the same school will be

described in more detail later.


Item Number 28. How many years have you been a principal?
(Including 2004-2005 school year)

Item Number 29. How many years have you been principal at your current school?
(Including 2004-2005 school year)



Figure 3-4. Employment history, Florida Principal Survey

Procedures

The first phase of the data collection began after the start of the 2004-05 school

year with an on-line survey sent electronically to the elementary school principals of the

twelve participating districts. Approximately 50% of the districts volunteered to have a

district official contact their principals to acknowledge their approval of the study. All

participating principals had Internet access.









Principals' e-mail addresses were determined using multiple sources. Most

districts do not maintain databases to share e-mail addresses outside their school district

Intranet. Small districts were willing to provide or check e-mail addresses gathered.

Some e-mail addresses were gathered from district and school websites, while hundreds

were created using a "formulaic" method (e.g., a seven character name followed by the

district URL or FirstName.LastName@district.kl2.fl.us) suggested by district officials.

Nearly two hundred schools were telephoned to request the e-mail address of their

principal. Surprisingly, the least effective method of obtaining accurate an e-mail address

was calling, as many school employees did not readily know this information. Every

reasonable effort was made to ensure the survey reached the intended sample. A

three-stage follow-up sequence to collect data was planned (Dillman, 2000).

Research decisions during the collection period

During the data collection period, four hurricanes inundated Florida. Every school

district in the state was affected by at least one hurricane resulting in school closures. In

the 12 districts from this study, the cumulative number of district closures due to the four

hurricanes was evenly distributed by district size with an average of 24 days lost.3

Two areas of data collection were potentially problematic. First, because of the

impact of the hurricanes, a decision was needed whether to continue data collection in

these districts. On September 29, 2004, the FLDOE announced that the FCAT would

proceed as planned with some provisions for those districts most greatly affected to have

more time to administer the assessment (Warford, 2004a). The State Education

Commissioner John Winn encouraged districts "to restore a sense of normalcy by


3 To protect the identities of the districts, specific numbers are not given.









continuing to focus on student achievement" (Florida Department of Education, 2004b, T

4). Because of these announcements, data collection proceeded, as there was no hold on

the policy under study.

The second problematic area was the low response rate from the on-line survey.

After 5 weeks of data collection, a 9% response rate was achieved. A decision was

reached to offer an alternate paper-pencil survey after learning that districts would permit

a paper version of the survey to be sent to the nearly 800 non-responding principals under

current district approvals. All non-responding principals received a copy of the paper

survey along with a postage-paid envelope to return the survey directly to the researcher.

The on-line format continued to be available for those who chose that format. The paper

survey offered in October was nearly identical to the on-line format. Questions used the

same wording and were asked in the same order. Minor modifications, were made where

the on-line survey instructed respondents to "click" the appropriate response. For

example, the paper survey was changed to "check" the response. Non-responding

principals were contacted using a two-stage follow-up (Dillman, 2000). I e-mailed

principals two times after the paper surveys were mailed requesting their participation

using the survey format of their choice.

These revisions resulted in an overall 29% return rate. Of the 255 principals who

returned the survey, 137 principals completed the on-line format, while 118 principals

completed the paper-pencil format. An additional 2% of the principals declined

participation stating lack of time as a reason. Table 3-3 shows the response rates

achieved using the district size classification. All returned surveys, including those

returned without names, were included in these calculations. Compared to the large









districts, a higher percentage of principals from small and mid-sized districts responded

favorably to this survey.

Table 3-3. Survey response rates by district size
Small N %
Responded 18 41.9
Declined 2 4.7
Non-response 23 53.4
Total 43
Mid-Size N %
Responded 46 40.0
Declined 4 3.5
Non-response 65 56.5
Total 115
Large N %
Responded 191 26.1
Declined 15 2.0
Non-response 526 71.9
Total 732

Usable Sample Procedure

While awaiting the close of the survey data collection, several databases of

information needed for this study were gathered, including the third grade percent

retained calculations and the school-wide percent of students eligible for a free or

reduced price lunch.4 Of the 255 returned surveys, each district was identified; however,

seven surveys were not linked to a principal and were therefore not included, bringing the

maximum available to 248 surveys. Every reasonable effort was exhausted to locate

missing data. All missing pieces of data were hand-searched to crosscheck for accuracy.

Prior to determining the final sample, a 10% random sample was selected to ensure the

accuracy of the databases compiled for the study and detected no errors in data entry or

data calculation.


4 See Appendix C for a list of the data, including the source and year.









For each type of data collected, a systematic procedure determined the usability of

a principal in this study. Principals were evaluated to determine how they met the

screening criteria. Essentially, each case, or principal, was examined to determine the

level of missing data. The criteria included 1) holding a principalship at their present

location for 5 or 6 continuous years, 2) assessing the completeness of third grade

retention rates, 3) assessing the completeness of school-wide free or reduced price lunch

eligibility rates, and 4) assessing the completeness of survey items used in these analyses.

Each piece of the screening criteria is described below, as well the how I arrived at the

total usable sample. At the end of each procedure a sub-total is provided.

First, as surveys were returned to the researcher, they were entered into a database

structured at the person-level where each principal represented a row (or case) (Kreft &

de Leeuw, 1998; Singer & Willett, 2003). Then, using the names of the responding

principals, I created a matrix of each principal's employment history for school years

1999-2000 through 2004-2005. Three components constituted a principal's employment

history which included: 1) the number of years of principalship total as reported on item

28; 2) the number of year of principalship held at their current school, as of 2004-2005,

as reported on survey item 29; and 3) the principal needed to still work at the same

school, as revealed by the name of the school they provided on the survey questionnaire.

Of those who responded to the survey, 121 principals held a principalship for at least 5 or

6 continuous years at the same school and thereby met the time requirement.

The second factor considered third grade retention percentages for the 5-year

period examined in this study. Using data files provided by the Florida Department of

Education, I created two sets of spreadsheet files. One file contained raw numbers of









students retained in third grade from each of the 5 years used in this study, and the other

file contained the numbers of third graders enrolled each year. I merged these files to

calculate the percent of third grade students retained in grade for each year examined in

this study. There were cases when one or both of the state provided data files did not

contain either the raw number of students enrolled or the enrollment needed to determine

the percentage. There were 6 principals removed due to missing data from one or more

time point. This reduced the usable sample to 115 principals.

The third factor considered the level of missing free or reduced price lunch

eligibility percentages for each school. Following the same procedure as determining the

percent of students retained in third grade, free and reduced price lunch eligibility

databases were created for each of the 5 years from files provided by the FLDOE. Of the

114 principals, there was one principal with many years of free or reduced price lunch

eligibility missing; therefore, to avoid erroneous conclusions, this principal was removed.

The total usable was reduced to 114 principals.

The survey items used in this study underwent an inspection to assess the

completeness of the closed-item responses. The four principals who did not respond to

item 9 were removed, as were two others who did not respond to at least half of the items

used to determine the degree of support groups. The total usable sample was reduced to

108 principals.

Finally, after one final inspection of the 108 principals, I reexamined all cases and

removed two outliers. Since only two schools that primarily serve students with

disabilities responded to the survey, these outliers were removed due to extremely small

third grade student populations. An additional inspection of the data revealed 4









principals with extreme values. These schools contained percent retained values that

appeared to be anomalous. Without revealing too much identifying information, an

example is provided. One school contained near zero percent retained for two years,

while in the next year nearly a quarter of third grade students were retained. To avoid

erroneous conclusions, these principals were removed from the final analysis. In all, there

is a usable sample of 102 principals in these analyses. The final distribution of the usable

sample by district size is provided in Table 3-4.

Table 3-4. Usable sample versus returned responses by district size
District Size Total Sent Returned Usable Percent Usable
Small 43 18 11 61.1
Mid-Size 115 46 21 45.7
Large 732 191 70 36.6
Total 890 255 102 40.0

Usable Sample Representativeness

Population and sample data were compared to determine the probability that the

sample represents the percent retained for the population of all state elementary schools.

Since total population data was readily available in files provided by the FLDOE for each

of the 5 years of retention data used in this study, I compared the usable sample

principals (n = 102 school principals) to the state-level totals5 using non-directional

z-tests. Table 3-5 displays means, standard deviations, standard errors, z-scores and

p-values demonstrating that there was not enough evidence to conclude that the usable

sample mean percent retained was unequal to the total population mean percent retained.6

For example, as displayed in Table 3-5, for Year 5 (School Year 2003-04) the population


SThe number of schools in the Florida system varies by year. See Table 3-9 for the number of schools statewide
for years 1999-00 to 2003-04 as reported on the Florida School Indicator's website.

6 In other words, the means were statistically equal.











(u = 10.722, c = 3.514) was not significantly different from the usable sample

(M= 9.127, s = 5.165), z = -.454,p = .650. Figure 3-5 visually displays the pattern

between mean percent retained of third graders over the 5-year period for each principal

comparing the population of principals in the state and the sample of principals usable in

this study.

Table 3-5. Third-grade percent retained usable sample principals versus the total
population
Year M s S.E. of n o oG, N z-score p-value
M (pop
S.E.)
1 3.077 4.036 .043 72 3.329 1.781 .004 1665 -.141 .888

2 3.118 3.394 .030 100 3.039 1.229 .003 1714 .064 .949

3 3.359 3.974 .035 102 3.338 1.552 .004 1776 .014 .989

4 11.855 7.124 .064 102 14.377 5.250 .012 1829 -.480 .631

5 9.127 5.165 .045 102 10.722 3.514 .008 1848 -.454 .650

Note. Weighted by third grade enrollment at each year


I I I I I
1999-00 2000-01 2001-02 2002-03 2003-04
Time in Years


- Sample
SPopulation


Figure 3-5. Population versus usable sample mean percent retained by year


16-


14-


12-





( 8-


6-


4-











Additional Variables

Four other variables were designed for the analyses that follow. Two are measures


of school-wide level of poverty; while two others consider effects of policy and time.


Socioeconomic status. For this study, the percent of students eligible for


free/reduced priced lunch at the school-level was used as a proxy for the socioeconomic


status (SES) of the school population. Prior to analysis, I examined the data to determine


the variability in school SES for each principal over the 5 years. Since SES lacked


variability over years, I created a new variable called SESMEAN (Figure 3-6).


1 000



0 800 -



0600 -



0 400 -



0 200



0 000

1999-00 2000-01 2001-02 2002-03 2003-04
Time in Years
Cases weighted by Igenrol


Figure 3-6. Mean SES values over 5-year period for usable sample schools

Mean Centered SES. An additional proxy for poverty used in the analyses that


follow is a variable called SESDIFF. This variable subtracts the raw percentage of


students eligible for a free or reduced price lunch in a school for each year from the mean


percentage of students eligible for free or reduced price lunch for that school over years.









Pre/Post Policy. This is a dummy coded variable used to make a distinction

between the pre- and post-policy periods. The years before the start of the retention

policy (i.e., years 1, 2, and 3) have been dummy coded as 0; while the final two years

(i.e., years 4 and 5) have been coded as 1 (Table 3-1). This Pre/Post Policy variable

estimates the effects of the policy.

Trend within Policy Period. A final coding scheme was developed to separately

analyze trends over years for pre- and post-policy periods. The variable twtl was coded

to analyze the differences during the pre-policy years; while the variable twt2 was coded

to analyze the post-policy differences. Each time point contains a 1-point difference and

each trend is centered on zero (Appendix E).

Design

Research Questions

Here, I investigate the possible explanations to the overarching research question

"What is the trend of third-grade retention practices before and after implementation of

Florida's third-grade retention policy and how is this trend impacted by other variables?"

To answer this question, I use a multilevel statistical model for change (viz., hierarchical

linear model, growth curve model, or mixed models) examining within-person change

and between-person differences in change beginning with the 1999-2000 school year

(Bryk & Raudenbush, 1987; Singer & Willett, 2003). Using longitudinal data, I ask

* How do principals' retention practices change between policy periods?

* How is this change impacted by poverty level?

* What variables reduce this variation?

The next sections explain how I answered these questions.









Multilevel Framework

A repeated measures analysis of variance (ANOVA) was conducted as a

preliminary analysis. Here, the unit of analysis is time where the repeated measure

ANOVA treats time as levels of a factor. After inspecting these data structured at the

person-level, it was decided that multilevel modeling might lead to a more valid

interpretation of the data. In making this decision, I consulted the literature on multilevel

modeling (Bryk & Raudenbush, 1987; Hox, 2002; Kreft & de Leeuw, 1998; Singer &

Willett, 2003). Kreft and de Leeuw (1998) explain several reasons why one would

choose a multilevel model over a typical ANOVA or linear regression. Figure 3-7 shows

one limit of examining person-level data. Examining the mean rates of retention may

produce misleading results, as it does not consider each individual. Technically, there is

variation both between years for a principal and variation between each principal. This

can lead to inflated Type I error rates and significant findings that are spurious.

Multilevel modeling easily manages unbalanced cases with missing values (Kreft

& de Leeuw, 1998; Singer & Willett, 2003). In fact, unbalanced data is thought to be

interpreted more readily using multilevel modeling compared to a technique such as

repeated measure analysis of variance (Singer & Willett, 2003). Additionally, in a

technique such as an ANOVA, typically cases with missing data are deleted (i.e., listwise

or pairwise), whereas they are not in a multilevel model (Hox, 2000; Moskowitz &

Hershberger, 2002).

The techniques of Applied Longitudinal Data Analysis: Modeling Change and

Event Occurrence (Singer & Willett, 2003) informed the design of this study. According

to Singer and Willett (2003), the first step in building a multilevel model is to determine

if it is an appropriate technique. To make this determination, they recommend creating










empirical growth plots to inspect the trajectories using a small sample. To create these

growth plots, the data was restructured in a person-period data set where multiple records

exist for each individual. If there is sufficient variability between these growth plots,

then multilevel modeling for change may be appropriate.

14.0


12.0-


S10.0-


8.0-


6.0-


4.0-


2.0- I I I
1 2 3 4 5
Time in Years


Figure 3-7. Mean retention rates by year. Results of repeated measures analysis of
variance of principals' year-by-year retention rates

The figures shown were instrumental in this decision making process. First, the

records shown in figure 3-8 display the range of possibilities, illustrating that there is

sufficient evidence showing variability amongst principals' retention rates for both the

intercepts and the slopes. Figure 3-9 displays the variability amongst all the principals of

this study. Most principals' retention rates begin near 0% and rise sharply over the

5-years displayed. Some principals had seemingly higher rates of retention before the

state policy, which then declined over time, while a few remained more constant over

time. Nevertheless, there is sufficient variation over years to warrant further inspection.












Singer and Willett (2003) further explained three methodological characteristics


complementary to multilevel modeling in the context of a longitudinal analysis


examining change over time. Here, I state their recommendations and share how I


considered their advice. First, they recommended three or more waves of data to detect


change. For these analyses, I selected 5 years of data to model change over time. Three


years were pre-policy and the latter two were the available data since Florida


implemented the retention policy. For my study, I planned to inspect time within each


policy period, as well as time before and after policy implementation. Second, they


suggested that data should change systematically over time. With the introduction of the


retention policy in 2002-03, the systemization of change over time was exemplified by


40'

30'

20'










0-* -
40'

S30'

20'






40'

S30'

20'

410
1 0. ------------ -----



1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Time in Years Time in Years Time in Years


Figure 3-8. Examples of principals' third-grade retention rates over time










the introduction of this policy. Post-policy retention rates changed based upon FCAT

results. Finally, they advised researchers to use a noteworthy metric to measure time.

For this study, the third grade retention outcome at the end of the school year was used to

measure policy years. Since promotional status changes on a year-by-year basis, the

school year is the metric for time. Study years were coded as either pre- or post-policy.

25.0
Linear Regression

20.0"

15.0

10.0 -

5.-

0.0 --



-10.0"I I
1 2 3 4 5
Time in Years
Figure 3-9. Third-grade retention over time. Observed variation in fitted trajectories for
all principals (n = 102)

Model Assumptions

All statistical analyses are based on underlying assumptions. Because a multilevel

model is more complex, so are the assumptions on which this model was based. Singer

and Willet (2003) identified three key assumptions for researchers to consider when

determining the tenability for fitting a model to data. These assumptions are the shape,

normality, and homoscedasticity.7

First, the shape, or linearity, of the dependent variable was inspected to determine

whether the means fall on a straight line as a function of the independent variable. That



7 All analyses were weighted by the log of school-level third-grade enrollment at each year.









is, does the mean of percent retained for pre- and post-policy connect with a straight line?

For variables such as Pre-Post Policy, "there [was] nothing to assess because a linear

model is de facto acceptable for dichotomous predictors" (Singer & Willett, 2003, p.

128). Simply put, any two points can be connected by a line.

Second, a key assumption was that these data were distributed normally. That is,

the distribution of the dependent variable is normal for both pre- and post-policy.

Violations of normality are usually robust; however, for these data, the violation was

severe for the dependent variable, percent retained. Figure 3-10 shows the extreme

positive skew of the data distribution. The minimum value was 0, and the maximum

value was 34.65, suggesting a normal median at approximately 17. However, the

clustering of data points near zero resulted in a median of 4.41. The mean of this skewed

distribution was 6.29. Using the standard deviation of 6.19 for interpretation would lead

to the impossible statement that approximately 95% of retention rates in this sample are

between -6.35% and 19.73%. We can refer back to figure 3-9 to see predicted percent

retained with negative values.

The chosen solution to this problem was a mathematical transformation of the

dependent variable into natural log units (McElroy, 2001; Singer & Willett, 2003). Since

it is mathematically impossible to take the natural logarithm of a zero value, an alternate

metric was needed since this sample contained forty-nine values associated with a zero

value. Rather than completely remove these cases using listwise deletion (Wothke,

2000), a decision was reached to add 1 to the percent retained value before taking the

logarithmic transformation. The natural logarithm of percent retained plus one

normalized the distribution. This transformation of the data permitted zero-values









(Figure 3-11). The resulting distribution had a mean of 1.65, a median of 1.74, and a

standard deviation of .917. To assist interpretation, log values can be converted back to

the original scale of measurement by taking the antilog and subtracting one (Singer &

Willett, 2003). Now, it is more feasible to argue that 95% of the population logpercent

retainedplus one are between (exp(.00) -1) = .00 and (exp(3.57) -1) = 34.52.

Finally, Singer and Willett (2003) also recommended examining the

homoscedasticity, the assumption of equal conditional variances. In other words, we

assume that the variance of the dependent variable is the same at all values of predictor,

which is to say that the spread of values around the mean dependent variable is the same

for all values of the predictor. So, for these data, there are two mean log percent retained

values, one pre-policy and the other post-policy. The assumption is that the variation

around these means would be the same. Boxplots were examined to determine whether

this assumption was met. For the assumption to be met, the interquartile ranges should

not be considerably wider or narrower than the other. Figure 3-12 shows the interquartile

ranges for two predictors. Visually, we see near equal variances; therefore,

homoscedasticity appears to have been met.

Model Building Process

The rationale for building a multilevel model was twofold: 1) to explain variation

in principal retention rates over 5-years while acknowledging the violation of

independence since 5-years worth of data are nested within each principal and 2) to add

predictors sequentially to test the significance of that predictor, as well as changes in

model fit and variance reduction. Many procedures assume statistical independence

make some sort of independence assumption. For example, when comparing two groups,













400





300 -


U

S200





100 -



Mean 6.6008
Std. Dev. = 6.44589
0 'N = 2,281
0.00 10.00 20.00 30.00 40.00
Percent Retained

Cases weighted by Igenrol

Figure 3-10. Histogram of percent retained before the transformation



250-



200-



= 150-


V 1

100- 3







Mean= 1.6533
0- Std. Dev. = 0.9174
N= 2,281
0.00 1.00 2.00 3.00 4.00
Log Percent Retained Plus 1

Cases weighted by Igenrol

Figure 3-11. Histogram of percent retained after the transformation











differences between groups should be due to the variable of interest and not some shared


feature of group membership (unless those groups are explicitly included in the model).


Violations of this assumption can lead to incorrect standard error estimates resulting in a


Type I error rate unequal to the conventional nominal a =.05 (i.e., the potential for


spurious statistical significance) (Singer & Willett, 2003).


200- 200-



100- 100-



0. 00 00-



-100- -100-
197
0350 219

-200- -200-
I I I
Pre-Pohcy Post-Policy No Yes
Pre-Post Policy Retention Belief


Figure 3-12. Examining the homoscedasticity assumption. Left panel represents the
residual by Pre-Post Policy. Right panel represents the residual by
Retention Belief.

Within these models, I used the following terms to represent the variables and


predictors. In these analyses, log(Y,+1) was the natural log of the percent retained for


time i in school with an addition of 1 to permit log transformations of zeros.8 Singer and


Willett (2003) recommended using a scale to help make the intercept more interpretable.


In all models, the intercept parameter flo indicated the average percent retained for


principal where 0 corresponds to Pre-Policy. fl, was the regression coefficient relating


time to log percent retained for principal j. yoo was the grand mean log percent retained at


8 The addition of 1 to the log is implied unless stated otherwise.









Pre/PostPolicy=0. 10o was the grand mean slope over principals. e, was the residual for

principal j at time i. There are seven main models in this taxonomy labeled Model A

through Model F2. Figure 3-13 provides an overview of the model taxonomy included in

this study.

To make meaning from this model building process different statistics were

examined. Results from these models yield intra-class correlations (ICC) and

proportionate reductions in error (PRE). ICC values provided opportunity to compare

values associated within a particular model. These values are not compared to other

models, whereas PRE values provided context for model comparison. An ICC represents

the proportion of total variance that is between-principals for a particular model. A PRE

represents the proportionate reduction in variance for a particular variance estimate (e.g.,

the residual variance) relative to another model. For consistency, all models and

sub-models were compared back to Model B, which is the time model called Pre/Post

Policy.

Finally, there are many ways to ascertain the fit of the model relative to the fit of

other models, including Akaike's Information Criteria (AIC), Bayesian Information

Criteria (BIC), and log-likelihood (LL) statistic (Singer & Willett, 2003). This research

used Akaike's Information Criteria (AIC) to provide an overall comparison of the model

fit. The AIC has the disadvantage of a more subjective interpretation (i.e., no

corresponding p-value) and the advantage of being able to compare non-nested models

(e.g., one model with a predictor X compared with another model with predictor X and

not Y) (Singer & Willett, 2003). Using a smaller-is-better approach, these goodness of fit

values allow comparison to Model A, the baseline model.



































Figure 3-13. Multilevel model for change taxonomy

Study Limitations

Limitations may hinder valid interpretation of these results. First, one study

assumption needed to be addressed prior to these analyses. Given the violation of

normality, I had to make a decision regarding whether to select a different model to

accommodate the data (e.g., specify a binomial distribution with a logit link) or to

transform the data to accommodate the model. The latter was selected. Initially, these

data were normalized taking the log of the dependent variable, percent retained. This

would have resulted in dropping fifty-seven 0% retained values from this study, simply

because it is mathematically impossible to take the natural logarithm of zero. Figure 3-14

displays the frequencies of values that would have been removed due to 0% retained by










year. Interestingly, here we see that 0% retained is largely a product of pre-policy

retention practices, not post-policy.


20-



15



10



5



0-
1 2 3 4 5
Time in Years

Figure 3-14. Number of zero percent retained values removed by year

This decision could have been justified because of the positive skew of the

distribution resulting in the majority of values being close in proximity to the scores

being removed. In other words, this was not a case of removing outliers; instead, it was a

more sensible technique to handling these data than either permitting the skew or

proposing an additional transformation of the data to permit zero-values. However, this

technique was not selected because of the richness of the data potentially lost. Because

of the pre-post policy focus of my study, it seemed prudent to select a metric that retained

these values. Therefore, prior to transforming the percent retained, 1 unit was added to

all the values. This adjustment permitted the 0% retained values, and normalized the

percent retained distributions.

Finally, after the data collection began, four hurricanes inundated Florida, which

may account for the lower than anticipated return on the survey. To minimize the amount






75


of researcher interaction, influence, or bias, a paper-pencil survey was provided as an

alternate option for principals. These self-administered surveys lessened interaction

between the researcher and the potential participants and helped minimize social

desirability bias (Baumann & Bason, 2004).














CHAPTER 4
RESULTS

Introduction

Results presented for the multilevel taxonomy of models begin systematically with

Model A and conclude with Model E. Model building was a successive process with

each model building upon the former. A summary table of the results is presented

demonstrating how variables, such as time measured as pre- or post-policy, level of

poverty, and trend within policy period impact third grade retention in Florida. Other

variables, such as a principals' retention belief and the degree of support provided in

schools were also tested; however, these variables did not improve model fit and were

dropped from the overall analysis (Appendix E). A summary table of the results is

available at the end of this chapter (Table 4-1). The resulting models, A through E, help

answer the following research questions

* How do principals' retention practices change between policy periods?

* How is this change impacted by poverty level?

* Do retention practices vary more between policy periods or between principals?

* What variables reduce this variation?

Multilevel Models

Model A: Baseline

Model A, the baseline model, is often called a one-way random effects ANOVA

model or an unconditional means model (UM) (Raudenbush & Bryk, 2002; Singer &

Willett, 2003). It is similar to the typical fixed effects ANOVA; however, the residual









variance is decomposed into two sources (Raudenbush & Bryk, 2002). The sample

distribution of the dependent variable, percent retained, was extremely positively skewed.

For this reason, the dependent variable was mathematically transformed (McElroy, 2001;

Singer & Willett, 2003) to yield a log-normal distribution, as shown in Equation 4-1.

log(Y, +1) = lo + e,, (4-1a)

floj = oo + Poj, (4-1b)

log(Y, +1) = yoo + po, + e, (4-1c)

Hence, log(Y,+1), the log percent retained for school at time i, is a function of

yoo: the grand mean percent retained over all schools before and after policy

uoj : the residual for school representing the distance of that school's percent
retained from the grand mean. These residuals are assumed to be normally
distributed with a mean of 0 and a variance of a2o.

* e,: the residual for school at time i represent the distance of the school at that time
from that school's mean before and after policy. These residuals are also assumed
to be distributed normally with a mean of 0 and a variance of a2e.

Model A is a baseline model with no predictors. It serves as a model for

comparison to subsequent models. Using Equation 4-1c, the resulting model is shown in

Equation 4-2.

log(Y, +1)= 1.66 + oj + e, (4-2)

Thus, the grand mean log percent retained was 1.66, t(96.160) = 34.800, p < .001,

corresponding to a grand mean percent retained of (exp(1.66)-1) = 4.26%. The

within-principal variation was significant (p < .001) suggesting the need for predictors,

while the between-principal variation was not significant (p = .071). In fact, 98.3% of the


1 Variances for future models are assumed to meet the same criteria.










variability in log percent retained was within-principals. The AIC for the baseline model

was 1279.43.

Figure 4-1 is a plot that regresses the transformed retention percentages on the

principals. It is clear that, over the principals, the predicted regression line suggests little

difference while the dispersion of points within principals creates a cloud of variability

throughout the graph. It is concluded that efforts should be taken to explain this

variability within-principals.



3.00

0 1








0.0 o
0 25 5 0 75 100
S0o O9 o o o
30.00 o0 oo o o o a o





e 4 S o o og o d p o
o o0 0 0S 0 0
38 o o 008 o0 0 oo
0 o 0 0o 0 0 00 0 o

ai j ,- o o o o o o o o u
















Model B: Conditional on Pre/Post Policy
Model B was the first in this series of growth models aimed to explain the impact










policy has on retention rates. For this, a variable representing time before and after the




time variable was coded such that Pre/Post Policy corresponds to the school years
0 o 9 o oft







itr.oon o te r non o o of loi model. For my one









1999-00, 2000-01, 2001-02 and Pre/Post Policy=l corresponds to the school years

2002-03 and 2003-04.

Distributions were first examined and appear to meet assumptions (Figure 4-2).

Pre-policy has a mean of 1.184 which corresponds to exp(1.184)-1 = 2.27%. The

standard deviation of .822 suggests that approximately 95% of retention rates in this

sample are between (exp(-.46, 2.83)-1), or between (0%, 15.95%). Post-policy has a

mean of 2.31 which corresponds to exp(2.31)-1 = 9.1%. With a standard deviation of

.595 suggests that approximately 95% of retention rates in this sample are between

(exp(1.12, 3.50)-1), or between (2.06%, 32.12%), as shown in Equation 4-3.

log(Y, +1) = foj + /l(Pre/Post Policy) + e,, (4-3a)

lo = yoo + pUo
fli =10o, (4-3b)

log(Y, +1) = yoo +ylo(Pre/Post Policy) + poj + e,j. (4-3c)

For the reduced models, foj was the intercept and is comprised of the grand mean

corresponding to Pre/PostPolicy=0 and the residual, uoj, representing the deviation of

school from the grand mean at Pre/Post Policy=0. Incorporating Pre/Post Policy

permits the retention rates to vary over time for different schools. Hence, flj is the slope

and is comprised of the grand slope, 71o, which is the change from pre to post policy. The

combined model expresses that the log percent retained for school at time i is a function

of the percent retained for pre-policy, the change in percent retained from pre- to

post- policy, and the residuals. Using Equation 4-3c, the resulting model is shown in

Equation 4-4.

log(Y, +1) = 1.17 +1.14(Pre/PostPolicy) + uPo + e,j. (4-4)




































Log Percent Retained Plus 1


300






100


0
0






000-
a
nc







000-


0

i 300- C o 0
oo

300 0
0 00 0 0
0 o

00 o oo 8 o OO OO


00o 0 0 0 oo oo
0 CD 0 00 1
Now oo o o
0 8 o o o

-- 0 0 QD

100 0 0 0 0
0 & Oc 0 0



000- GDO 00ED 00 0 00 GO GCMSO ME 0

0 25 50 75 100

C New ID


Log Percent Retained Plus 1





300




200-
10-












L
3000-
0-


O









--I -


300


a)

200"
a)
a)


100.

0-

j000
000


o O 0 0 o o

o 0 o 0 0 0 0







O
o o 6oo 0o o 0

o o @0 0 o 0 0 O 0O

.o ?o ,o0
0c &00 0 Q F 0
0 0 0 0 00 0

0
0 0 0 0 00
00 0



0
.o0


0 25 50

New ID


75 100


Figure 4-2. Distribution plots of third-grade retention rates. A) Pre-policy histogram.

B) Pre-policy boxplot. C) Pre-policy scatterplot. D) Post-policy histogram.

E) Post-policy boxplot. F) Post-policy scatterplot.



The estimated mean log percent retained at pre-policy is 1.17, t(153.09) = 21.692,



p < .001, corresponding to exp(1.17)-1 = 2.22%. The coefficient for Pre/Post Policy was









significant at 1.14, t(375.53) = 19.909, p < .001, suggesting that mean log percent

retained increases by exp(1.14)-1 = 2.13% from pre- to post-policy. An intra-class

correlation coefficient (ICC) of .079 indicates that approximately 8% of the variance in

this model is between principals, while approximately 92.1% of the variance is within

principals. This means that there is still considerably more within-principal variability

(92.1%), than between-principal variability (7.9%). Incorporating policy reduced the

within-principal variation by 51.5%, the variation within schools is 1.81, z = 13.655,

p < .001. The proportion of variance within schools was approximately 98%;

incorporating pre/post policy as a metric for time reduced it to approximately 92%.

Further, model fit improved by 20.8% with the AIC dropping from 1279.43 to 1021.81.

The subsequent models will be relative to this model except for the AIC, which is always

relative to the baseline model, Model A.

Model C: Conditional on Policy and Level of Poverty

This model introduced a proxy for school-wide level of poverty using the percent

of students eligible for a free or reduced priced lunch as the criterion.2 Model C is shown

in Equation 4-5.

log(Y, +1) = loj + f,(Pre/Post Policy) + f2,(SESMEAN) + e, (4-5a)

loj = yoo + uoJ
&1J = 710 (4-5b)
2]j =720,

log(Y, +1) = yoo + yo(Pre/Post Policy) + y2o(SESMEAN) + uo, + e,,. (4-5c)

Using Equation 4-5c, the resulting model is shown in Equation 4-6.




2 Using disaggregated SES over years, as opposed to SES for each year, was justified by the lack of variation
found in the repeated measures ANOVA described in Chapter 3.









log(Y, +1) = .463 + 1.13(Pre/Post Policy) +1.36(SESMEAN) + /oj + e, (4-6)


The estimated mean log percent retained at initial status is .463, t(108.427) = 4.859,

p < .001, corresponding to (exp(.463)-1) = .59%. However, this is at SESMEAN=0. The

coefficient for Pre/Post Policy of 1.13, t(374.619) = 19.758, p < .001, suggests that mean

log percent retained increases by (exp(1.13)-1) = 2.1% from pre- to post-policy. The

coefficient for SESMEAN of .014, t(97.426) = 8.434, p < .001, suggests that each

one-unit change in SESMEAN corresponds to (exp(.014)-l) = .014% for both pre- and

post-policy.

An intra-class correlation coefficient (ICC) of .030 indicates that 3% of the

variance in this model is between principals, while 97% of the variance is between years

within principals. This served to balance-out the ICC, so to speak, compared to the

variable that examined the effects before and after policy. Model fit improved by 24.5%

(relative to the baseline model) with the AIC dropping to 966.34.

Model D: Conditional on Policy, Level of Poverty and Mean Centered SES

The model-building process continued with the inclusion of a variable for mean

centered SES called SESDIF. This variable is the percent difference between a school's

percent of students eligible for a free or reduced price lunch and the mean of SES over

the 5-year period for that school. Model D is shown in Equation 4-7.

log(Y, +1) = flo + fl,(Pre/Post Policy) + f2,(SESMEAN) + &3,(SESDIF) + e,, (4-7a)

oj = yoo + /Uo
lj YO10
,2j = 20 (4-7b)
/3j = 30,

log(Y, +1) = yoo + yo(Pre/Post Policy) + y2o(SESMEAN) + y30(SESDIF) + (4-7c)
Uoj + e,.










Using Equation 4-7c, the resulting model is shown in Equation 4-8.

log(Y, +1) = .469 + 1.10(Pre/Post Policy) + .014(SESMEAN) + (4-8)
.022(SESDIF) + o, + /j + e,.

The estimated mean log percent retained at initial status is .469, t(109.614) = 5.006,

p < .001, corresponding to (exp(.469)-1) = .60%. However, this is at SESMEAN=0. The

coefficient for Pre/Post Policy of 1.10, t(380.620) = 19.160, p < .001, suggests that each

one-unit change in Pre/PostPolicy corresponds to a (exp(1.10)-1) = 2.0% from pre- to

post-policy. The coefficient for SESMEAN of .014, t(98.421) = 8.670, p < .001, suggests

that each one-unit change in SESMEAN corresponds to a (exp(.014)-) = .014% change

in mean log percent retained when Pre/Post Policy and SESDIF are controlled at a

particular value. The coefficient for SESDIF of .022, t(471.591) = 3.442, p < .001,

suggests that each one-unit change in SESDIF corresponds to a (exp(.022)-1) = .022%

change in mean log percent retained when Pre/PostPolicy and SESMEAN are controlled

at a particular value.

An ICC of .029 indicates that approximately 97% of the variance in this model is

between years within principals, while approximately 3% of the variance is between

principals. The variation within schools was 1.780, z = 13.609, p < .001, corresponding

to a PRE of 1.5%. The variation between-schools at initial status is .054, z = 2.664,

p = .008, corresponding to a PRE of 65.6%. Model fit improved by 24.7% (relative to

the baseline model) dropping to 962.86. There was virtually no change in PRE's, ICC's,

or AIC, nevertheless this model was retained because the parameters and variance

components were significant.









Model E: Conditional on Policy, Level of Poverty, Mean Centered SES and Trend
within Time

The final model formally presented added two variables to account for the policy

trend with the pre- and post-policy time periods. These variables, twtl and twt2, were

coded so that twtl was centered on pre-policy time and twt2 was centered on post-policy

time. Model E is shown in Equation 4-9.

log(Y, +1) = flo + f,(Pre/Post Policy) + f2,(SESMEAN) + &y3(SESDIF) + (4-9a)
j4,(twtl) + ,5J(twt2) + e, ,

/Oj = yoo + /O]
1ij = o10
32 j 20
1 = 730 (4-9b)
,4J 740
Pfl 750,

log(Y, +1) = Yoo + yo(Pre/Post Policy) + y2o(SESMEAN) + y3o(SESDIF) + (4-9c)
y4o(twtl) + y50(twt2) + iu0 + e,,..


Using Equation 4-9c, the resulting model is shown in Equation 4-10.

log(Y, +1) = .465 + 1.11(Pre/Post Policy) + .014(SESMEAN) + (4-10)
.020(SESDIF) + .034(twtl) + -.226(twt2) + uoj + /j, + e, .

The estimated mean log percent retained at initial status is 0.465, t(109.806) =

4.952, p < .001, corresponding to (exp(.465)-1) = .59% when Pre/Post Policy,

SESMEAN, SESDIF, twtl and twt2 are zero. The coefficient for Pre/PostPolicy of

1.11, t(380.595) = 19.333, p < .001, suggests that each one-unit increase in

Pre/PostPolicy corresponds to a (exp(1.109)-1) = 2.03% from pre- to post-policy. The

coefficient for SESMEAN of .014, t(380.595) = 19.333, p < .001, suggests that each

one-unit change in SESMEAN corresponds to a (exp(.014)-l) = .014% change in mean

log percent retained when Pre/PostPolicy, SESDIF, twtl and twt2 are controlled at

particular values. The coefficient for SESDIF of .020, t(469.348) = 2.986, p = .003,









suggests that mean log percent retained changes on average by (exp(.020)-1) = .02%

when SESMEAN, Pre/Post Policy, twtl and twt2 are controlled at particular values. The

coefficient for twtl of .034, t(385.280) = .720, p = .472, suggests that mean log percent

retained changes on average by (exp(.034)-) = .034% when SESMEAN, SESDIF,

Pre/PostPolicy and twt2 are controlled at particular values. The coefficient for twt2 of

-.226, t(373.269) = -2.616, p = .009, suggests that mean log percent retained declines on

average by (exp(-.226)-1) = -.202% when SESMEAN, SESDIF, Pre/Post Policy and

twtl are controlled at particular values.

An ICC of .030 indicates that approximately 3% of the variance in this model is

between years within principals, while nearly 97% of the variance is between principals.

Variation within schools is .1.755, z = 13.570, p < .001, corresponding to a PRE of 2.9%,

compared to Model B. The variation between-schools at initial status is .055, z = 2.726,

p = .01, corresponding to a PRE of nearly 65%, compared to Model B. The AIC dropped

to 962.89 improving the model fit improved by 24.7% when compared to Model A.

Summary

We conclude the model building process with a review of the underlying statistical

assumptions on which this study was designed. Again, although this study modeled

growth over a 5-year period, an alternative metric for time was used to predict the effects

of retention before and after policy implementation. However, within each policy period

the trend within time was modeled. Pre-policy contained three time points and because

the trend before policy was relatively stable, it met the assumption. Again, post-policy

was dichotomous and linear de facto. Normality and homoscedasticity was assessed at

multiple points during the model building process and remained unchanged from the

assessment prior to the model building.










Table 4-1. Results of fitting a taxonomy of multilevel models for change to the log
percent retained data, Models A through E (n = 102)
Model- A B C D E
Intercept 1.66*** 1.17*** .463*** .469*** .465***
Pre/Post Policy 1.14*** 1.13*** 1.10*** 1.11***
sesmean .014*** .014*** .014***


sesdiff
twtl
twt2


Residual
Intercept
Between
Within
TOTAL


3.72***
.064-
.064
3.723
3.787


1.81***
.155***
.156
1.807
1.963


1.82***
.056**
.056
1.816
1.872


.022**


1.78***
.054**
.054
1.780
1.834


.020**
.034
-0.226**

1.76***
.055**
.055
1.755
1.810


Proportionate
reduction in error
PRE-Residual .515 -0.005 .015 .029
PRE-Intercept -1.430 .637 .656 .648
Relative To Model- A B B B
Intra-class correlation
coefficient
ICC-Between .017 .079 .030 .029 .030
ICC-Within .983 .921 .970 .971 .970

Goodness of Fit
AIC 1279.43 1012.81 966.34 962.86 962.89
AIC difference 266.62 313.09 316.57 316.54
Fit improvement (%) 20.80 24.50 24.70 24.70
Relative to Model- A A A A
-p< .10; *p<.05; **p< .01; ***p< .001














CHAPTER 5
DISCUSSION

Introduction

The models showed that time (or the policy period) and poverty at the school level

played dramatic roles in the retention rates across this sample. This discussion delves

into retention trends to address my overarching question "What is the trend of third-grade

retention practices before and after implementation of Florida's third-grade retention

policy and how is this trend impacted by other variables?" The model building process

(Figure 3-13) guided my study to understand how particular variables such as policy, and

level of poverty impacted retention practices over a 5-year period. This chapter discusses

the results from this model building process to answer the more specific research

questions.

* How do principals' retention practices change between policy periods?

* How is this change impacted by poverty level?

* What variables reduce this variation?

Then, we turn our attention the policy implications and recommendations, as well

as suggestions for future research.

Research Question 1: How Do Principals' Retention Practices Change between
Policy Periods?

The model building process originated with a degenerate model providing a

baseline to compare model fit. In considering the baseline model, significant variation

was found both between principals and within principals over years that needed







88


explanation. Over 98% of the variation was found within principals over years. In the

final model, we gained insight regarding principals' retention policies between policy

periods. The key feature of Model E was its representation of the data. First, it adjusted

for between- and within-principal SES. Then, it revealed a significant difference between

the two policy periods. Furthermore, although it did not reveal a significant trend

between the three pre-policy years, it did reveal a significant difference between the two

post-policy years.


25 -

20

15 -
log(perret+1)
-- perret


0
5-



1 2 3 4 5
A

25

20 -

15 -* log(perret+1)

10 /- perret

5 --_

0 ----
1 2 3 4 5
B

Figure 5-1. Examples of mean predicted retention trends over time. Two lines are shown in
each graph. One line represents the predicted log percent retained value
(log(perret+l)), while the other is unlogged showing percent retained units
(perret). Because two metrics are shown in one graph, they-axis is simply
one-unit intervals. The x-axis shows time in years, with years 1, 2, and 3 being
pre-policy, and years 4 and 5 being post-policy. A) School from mid-size district,
mean SES 52%. B) School from same district, mean SES 91%.









The fact that pre-policy was not significant is worth discussion. This means that

between the pre-policy years, the change in mean retention rates can be considered equal.

Retention practices were stable before implementation of the state retention policy. On

the other hand, the post-policy years (years 4 and 5) had mean retention rates that were

significantly different from each other. Two examples of the mean predicted values help

us understand how retention trends have changed over time (Figure 5-1). Within both

examples, on visual inspection, it is clear why the pre-policy trend was not significant,

and the post-policy trend was significant.

Research Question 2: How Is This Change Impacted by Poverty Level?

Since the level of poverty is widely regarded as a factor that impacts student

achievement, school level of poverty was included as a predictor. Using the mean SES of

each school over the 5-year period confirmed that, even in this sample, poverty plays an

influential role in retention rates for third graders in Florida. Mean SES rates greatly

affect retention practices. Schools with the highest levels of poverty (greater than 67%)

consistently retained more students over this 5-year period (Figure 5-2). Principals from

schools with the lowest range of poverty (33% or less) retained fewer students. Clearly,

these principals have had fewer practical experiences with retention over this 5-year

period than principals of higher-poverty schools; however, even they have felt the impact

of policy with more students experiencing retention in pre-policy years.

In further examining the pre-post policy trend, I inspected the poverty levels over

time. Here we see the relationship between pre- and post-policy (Figure 5-3). As poverty

increased, the percent of students retained also increased. Poverty is a great predictor of

retention rates and so policy impacts students from higher-poverty schools more greatly.

In other words, lower-achieving students in higher-poverty schools were more likely to