This item is only available as the following downloads:
A COMPARATIVE STUDY OF READING ACHIEVEMENT 200 2 TO 2013 By S YNTHIA DELINA LAFONTAINE A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR TH E DEGREE OF DOCTOR OF EDUCATION UNIVERSITY OF FLORIDA 2014
2014 Synthia Delina LaFontaine
3 To Charles, for his unconditional love and support
4 ACKNOWLEDGMENTS The guiding words shared almost a half century ago by my grandfather, Andrew Jackson McCurley, (1875 1 980 ) have cu lminated in this dissertatio n. As a member of the founding family of Denton County, Texas and the leader, preacher and teacher of McCurley Prairie, Texas, he guided many people to prioritize the values of ethics and education I am es pecially grateful to my sister, Lanette Eastwood, who made tremendous personal sacrifices so I could live and thrive to pursue higher education I am also deeply grateful to Charles my husband, for recognizing the importance of this academic endeavor, and devoting four years of his life to my completion of this degree I thank Dr. Bernard Oliver, my committee chair, for his perfect balance of supervision, insight, patience, and guidance. I have much gratitude for Dr. Linda Eldridge initial inspiration and ongoing exemplary facilitation of advanced learning that ignited and nurtured my commitment to these studies. I thank Dr. Linda Behar Hornstein for facilitating comprehension of statistics and effectively concise communications. Las tly, I am appreciative of the work and support of the entire dissertation committee, Dr. Bernard Oliver, Dr. Linda Eldridge, Dr Linda Behar Hornstein, and Dr Eileen Oliver, for their work, guidance and support.
5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 7 LIST OF FIGURES ................................ ................................ ................................ .......... 9 LIST OF ABBREV IATIONS ................................ ................................ ........................... 10 ABSTRACT ................................ ................................ ................................ ................... 13 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 14 FCAT Scores ................................ ................................ ................................ .......... 14 School Grades ................................ ................................ ................................ ........ 14 Reading as an Accountability Measure ................................ ................................ ... 18 Statement of the Problem ................................ ................................ ....................... 21 Purpose of the Study ................................ ................................ .............................. 22 Significance of the Study ................................ ................................ ........................ 23 Definition of Terms ................................ ................................ ................................ .. 24 2 LITERA TURE REVIEW ................................ ................................ .......................... 26 Test Based Accountability ................................ ................................ ....................... 26 FDOE Reports ................................ ................................ ................................ ........ 28 Understanding Data ................................ ................................ ................................ 33 Accuracy in Calculations and Conclusions ................................ ....................... 33 Public Perception ................................ ................................ ............................. 36 Accurate Performance Reporting in Florida ................................ ...................... 37 Policy Changes ................................ ................................ ................................ ....... 39 Improvement Requires Accurate Data ................................ ............................. 40 Data Driving Instruction ................................ ................................ .................... 41 State Level Data Usage ................................ ................................ ................... 42 Summar y ................................ ................................ ................................ ................ 46 3 METHODOLOGY ................................ ................................ ................................ ... 47 Research Methodology ................................ ................................ ........................... 47 Research Question One ................................ ................................ ................... 47 Resear ch Question Two ................................ ................................ ................... 48 Research Question Three ................................ ................................ ................ 49 Variables of Consideration ................................ ................................ ...................... 50 Origin and Sources of Data ................................ ................................ ..................... 51 Instrumentation ................................ ................................ ................................ ....... 52
6 Assumption s ................................ ................................ ................................ ........... 52 Limitations ................................ ................................ ................................ ............... 52 Summary ................................ ................................ ................................ ................ 53 4 RESULTS AND ANALYSIS OF DATA ................................ ................................ .... 55 Research Questions ................................ ................................ ............................... 55 Analysis ................................ ................................ ................................ .................. 56 Research Question One ................................ ................................ ......................... 62 Research Question Two ................................ ................................ ......................... 64 Research Question Three ................................ ................................ ....................... 70 Summary ................................ ................................ ................................ ................ 78 5 CONCLUSION ................................ ................................ ................................ ........ 80 Summary of Results ................................ ................................ ................................ 80 Implications ................................ ................................ ................................ ............. 84 Recommendations ................................ ................................ ................................ .. 87 APPENDIX A UNIVERSITY OF FLORIDA INSTITUTIONAL REVIEW BOARD (UFIRB) APPROVAL ................................ ................................ ................................ ............ 89 B ADDITIONAL DATA CHARTS ................................ ................................ ................ 90 C DQ C RESOURCES ................................ ................................ ................................ 93 REFERENCES ................................ ................................ ................................ .............. 95 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 100
7 LIST OF TABLES Table page 1 1 Elementary school grade points ................................ ................................ ......... 15 1 2 Sample school grade report ................................ ................................ ............... 16 1 3 School grade changes ................................ ................................ ........................ 17 1 4 School grades 2012, 2013, with Safety Net ................................ ........................ 19 2 1 FCAT reading achievement of student groups by grade level ............................ 29 2 2 FCAT reading achievement levels, 2008 and 2012 ................................ ............ 30 2 3 FCAT reading scores for ELL students and for all students ............................... 32 2 4 Student groups with FCAT reading achie vement declines, 2011 2013 .............. 33 4 1 Students tested ................................ ................................ ................................ ... 56 4 2 Perc ent change in number of students tested, 2001 2013 ................................ 57 4 3 Percent change in number of students tested annually ................................ ...... 57 4 4 Percentage at level three and above, FCAT reading, 2002 2013 ....................... 58 4 5 School grades by school levels ................................ ................................ .......... 61 4 6 ................................ ....................... 61 4 7 ................................ ........................... 63 4 8 ................................ ................................ 64 4 9 Research question one summary of findings ................................ ...................... 64 4 10 FCAT reading by student groups ................................ ................................ ........ 68 4 11 Research question two summary of findings ................................ ...................... 69 4 12 Research question three summary of findings ................................ .................... 74 4 13 ................................ .... 76 4 14 ................................ ................. 76 4 15 reading 2002 to 2013 .................. 77
8 4 16 ................................ ...................... 77 5 1 DQC state actions ................................ ................................ .............................. 85 B 1 Number and percentage of students tested ................................ ........................ 90 B 2 FCAT reading by groups: Changes 2003 to 2013 ................................ .............. 91 B 3 FCAT reading by grade levels: Changes 2003 to 2013 ................................ ...... 92
9 LI ST OF FIGURES Figure page 4 1 Comparison of school grades and FCAT reading averages by group s .............. 94
10 LIST OF ABBREVIATIONS AMO Annual Measurable Objectives. The targets set yearly for each school and group in Florida to reduce by 50% the proportion of students scoring below level three on FCAT 2.0 in reading and math and below level 4 on the Florida Alternate Assessment (FAA) in reading and math by 2016 17, using 2010 11 as the baseline. All schools and group s are evaluated annually about meeting these targets (Fl orida Department of Education, 2013i) AYP Adequate Yearly Progress. A means of measuring and reporting annual progress made by public school students in core subjects as demonstrated on standardized assessments (United States De partment of Education, 2010) CCSS The Common Core State Standards A nationally standardized criteria for k12 educational content in each subject area. The CCSS are to be fully implemented with the beginning of the 2013 14 school year (Florida Department of Education, 2013b) DQC Data Quality Campaign A Washington, DC, based organization that supports the development and effective use of statewide longitudinal data systems (Data Quali ty Campaign, 2013a) DSS Developmental Scale Score. A on FCAT considering expectations for progression through the grade levels (Florida Department of Education, 2013m) ELL English Langua ge Learner. A designation of s tudents who are learning English as a second language (Florida Department of Education, 2013e) EOC End of Course examinations. S ubject area assessments to determine student achievement levels in some subject areas and grade levels (Florida Department of Education, 2013m) ESE Exceptional Student Education. Educational program for students with exception al educational needs that may be cognitive, emotional, or physical (Florida Department of Education, 2013e) ESEA Elementary and Secondary Education Act. An act passed in 1965 that emphasizes equal access to education establishes high standards and accountab ility and au thorizes federally funded education programs that are administered by the states. In 2002, Congress amended ESEA and reauthorized it as the No Child Left Behind Act (NCLB) (United St ates Department of Education, 2010)
11 ESOL English for Speakers of Other Languages. An educational program for ELL students (Florida Department of Education, 2013e) FAA Florida Alternative Assessment. An assessment de sig ned for ESE students whose participation in the general statewide assessment (FCAT, FCAT 2.0, and EOC) is not approp riate even with accommodations (Florida Department of Education, 2013n) FCAT Florida Comprehensive Assessment Test A test of student schools in selected grades and subjects. The FCAT measures student achievement in reading, mathematics, science, and writing (Florida Departm ent of Education, 2013d) FCAT 2.0 The FCAT 2.0 measures student achievement of the Next Generation Sunshine State Standards. Florida began the transition from the FCAT to the FCAT 2.0 with reading in 2011 Science began in the spring of 2012 (Florida Department of Education, 2013c) FDOE T he Florida Department of Education. LEA Local Education Agency. A public board of education, other public institution, or agency with administrative control and direction for a public school (United States Department of Education, 2010) NAEP National Assessment of Educational Progress. The largest nationally representative and continuing assessment of conducted periodically in mathematics, reading, science, writing, the arts, civics, economics, geography, and U.S. histor y (United States Department of Education, 2010) NCES National Center for Education Statistics. Data service s provided by the United States Department of Education, Institute of Education Sciences (United States Department of Education, 2012a) NCLB No Child Left Behind. An act that is the 2002 reauthorization of the Elementary and Secondary Education Act (ESEA) i ncorporat ing accountability, research, and local control / flexibility in a federal effort to support educational efforts to improve achievement for all public school students (United States Department of Education, 2010) NGA that is organized to identify priority issues and deal collectively with matters of public policy and governance a t the state and national levels (National Governor's Association, 2013)
12 NGSSS T he standards adopted in 2007, subsequently c alled the Next Generation Sunshine State Stand ards in Reading/Language Arts the FCAT 2.0 r eading a ssessment (Florida Department of Education, 2013a) PSAT Preliminary Scholastic Aptitude Test. A standardized pencil and paper test that is des igned to indicate how students will perform on the SAT (College Board, 2012) RTTT Race to the Top. Educational reform funding program that emerged ESEA that requir es states to meet specific criteria (United States Department of Education, 2012b) SAT Scholastic Aptitude Test. The SAT and SAT Subject Tests are designed to assess academic readiness for college (College Board, 2012) SBOE State Board of Education. SSS Sunshine State Standards. Educational standards approved by the SBOE in 1996 (Florida Department of Education, 2013m) TAG, TAP Technical Assistance Guide, Technical Assistance Paper. Documents explaining test scores and accountability calculations
13 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education A COMPARATIVE STUDY OF READING ACHIEVEMENT 2002 TO 2013 By Synthia Delina LaFontaine May 2014 Chair: Bernard Oliver Major: Educational Leadership Th is quantitative study compares the educational outcomes in reading of student groups in the state of Florida from 2002 through 2013 as demonstrated on the Florida Comprehensive Assessment Test (FCAT) in r eading The progress of students who are African American economically d isadvantaged ( as designated by the state ) Hispanic, or White, in scoring at or above achievement level three, the level of success defined by the state as satisfactory, i s us ed to determine and compare the results of the educational system f or each group and for the student population ov erall The purpose is to provide a n exploratory s student achievement in reading and closing longstanding achievement gaps. The study shows inconsistent pr ogress for students achieving satisfactory learning goals in reading and in closing the reading achievement gap for students from 2002 to 2013, while state reports indicated increased levels of success.
14 CHAPTER 1 INTRODUCTION FCAT Scores Between 2002 and 2013, the reading achievement of studen ts in grades three through ten of as measured through an annual administratio n of the Florida Comprehensive Assessment Test (FCAT). FCAT became FCAT 2.0 when the standards assessed in th e test were updated in 20 1 1 After each annual That office published the test results in tables and charts, and provided summaries of student groups by achievement levels and by content areas assessed on the FCAT/FCAT 2.0. Tables and graphs show score results in aggregated and disaggregated compilations for all students tested by racial, socio economic, and learner profile groups, grade levels, schools, and districts Learn er profile groups include students receiving special educational services (ESE) and students learning English as a second language (ELL) FCAT scores are divided into five achievement levels. Scores at level three or above are considered sufficient achieve ment for accountability purposes. (Florida Department of Education, 2014b) School Grades After FCAT scores are determined the Bureau of Accountability Reporting uses test score results to determine each school grade Grades accountability system Gr ade guides e xplain the overall school grade scoring system and changes that may have been made each year The current School Grade system is based on an 800 point scale. The FCAT scores of students scoring at level
15 Student scores in reading and math are 600 of the 800 points possible. S cores in read ing are 300 of the 800 points possible for school grades. S cores of students who fall into the lowest quartile are factored two or three times in the calculation (Florida Department of Education, 2014) Table 1 1 illustrates th e categories used by the FDOE for school grade calculations and reporting. Table 1 2 is a sample school grade report. Table 1 1. Elementary school grade points. Reading Math Writing Science Performance FCAT 2.0, FAA Performance FCAT 2.0, FAA Performance FCAT 2.0, FAA Performance FCAT 2.0, FAA 100 points 100 points 100 points 100 points Learning Gains All Students FCAT 2.0, FAA Learning Gains All Students FCAT 2.0, FAA 100 points 100 points Low 25% Learning Gains FCAT 2.0 Low 25% Learning Gains FCAT 2.0 100 points 100 points (Florida Department of Education, 2013j) Additional requirements: Learning gains requirement for the l ow est 25% reinstated. Test at l east 90% of students, 95% to earn A performance threshold in r eading (25%) will be applied. G rade lowered one letter grade if not met (greater weight on reading). The points earned result in grade s according to the following scale: B = 495 524 points C = 435 494 po ints D = 395 434 points F = < 395 points (Florida Department of Education, 2013i)
16 Table 1 2 Sample school grade report. Florida School District Florida Elementary School 2009 2010 Reading Math Writing Science Grade Points Earned % Meeting High Standards (FCAT Level 3 and Above) 68% 58% 88% 20% 234 Writing and Science: Takes into account the % scoring 4.0 and above on Writing and % scoring 3 and above on Science. Sometimes the District writing and/or science average is substituted for the writing and/or science component. % of Students Making Learning Gains 62% 57% 119 3 ways to make gains: Improve FCAT Levels Maintain Level 3, 4, or 5 Improve more than one year within Level 1 or 2 Adequate Progress of Lowest 25% in the School? 53% (YES) 70% (YES) 123 Adequate Progress based on gains of lowest 25% of students in reading and math. Yes, if 50% or more make gains in both reading and math. FCAT Points Earned 476 Percent Tested = 100% Percent of eligible students tested School Grade* C Grade based on total points, adequate progress, and % of students tested (Florida Department of Education, 2014)
17 The State Board of Education frequently changes the school grade point system. (Florida Department of Education, 2013j) Table 1 3 illustrates the history of grade point system changes Table 1 3 School grade changes. 1999 the first time. Based on current year data. Included performance in reading, mathematics, and writing, and additional criteria dropout rates, attendance, and discipline data. Minimum a nd higher performing criteria set for reading, writing, and math. To receive an A or B, schools were required to meet the higher performing criteria and additional requirements. 2002 School grades calculation was changed to the points based model we have today 50% performance and 50% learning gains. Learning gains were added (reading and math). FCAT administered in grades 3 to 10. Learning gains for the lowest 25% in reading added. Learning gains target in reading for the lowest 25% added. 2005 Additional students added to the calculation. Students with disabilities added to learning gains for students with scores. ELLs added to learning gains. 2007 Science added to school grading measures. Learning gains for the lowest 25% in mathematics added (along with learning gains target for the lowest 25% in math). Bonus points for high school retakes added. 2010 Implement new High School measures (50% of HS Grade). Acceleration participation and success. Graduation rate all students and at risk. Col lege readiness SAT, ACT, and PERT. 2012 New FCAT 2.0 and End of course (EOC) assessments with new achievement levels. Students with disabilities and English language learners in performance measures. New middle school acceleration measure. Use federal graduation rate. Allow weighted learning gains for some students. Safety net protection. 2013 New Science FCAT 2.0 and Geometry and Biology EOC assessments and achievement levels. Safety net provision continued. Reading threshold implemented. (Florida Department of Education, 2013o)
18 Technical Assistance Guides provide additional school grade system details. For example, the scores of students who do not attend the same school for most of the school year are not in cluded in a (Florida Department of Education, 2014) School grade results are reported by categories designated by the FDOE, but that information does not include race ethnicity, special education (ESE), English language learners (ELL), or socioeconomic status (Florida Department of Education, 2014) Reading as an Accountability Measure The FCAT reading results are the focus of this study b ecause of the central imp ortance the FDOE has placed on them. Florida selected reading achievement on FCAT as an accountability measure for schools and as criteria for student promotion and graduation. Table 1.3 shows that the FDOE identified reading, math and writing as the subjects to measure for accountability in 1999. In 2002, the FDOE chose to emphasize progress for the lowest performing students by assigning additional point value for their achievement in reading and math (F lorida Department of Education, 2013o) Florida statute requires students be retained at the end of grade three if the FCAT reading score indicates a reading deficiency. The statute identifies achieving at a level less than achievement level t wo as th e indicator of a reading deficiency. The law specifies that students achieving level one or two on FCAT reading must receive extensive remediation and ongoing progress monitoring (Florida Senate, 2014) The FDOE requires stud ents in grades three through nine to take FCAT reading. FCAT reading is the only FCAT required to be taken for all of those years. FCAT
19 reading is the only assessment that requires achievement of level two or above for promotion in grade three. Students wh o do not pass FCAT reading in nin th grade are required to retake the test up to five times until passed, or pass the ACT or SAT, or they are unable to receive a standard diploma (Florida Department of Education, 2014e) Safety Net Changes are made to the school grade system by the Florida legislative body. O ne of the change s implemen ted in 2012 and 2013 was policy legislates that no school grade could be lowered by more than one letter from the prior year regardless of the drop in performance resulting from FCAT scores and other school grade criteria Table 1 4 shows the effect of the policy on school grades in 2013 Table 1 4 School grades 2012, 2013, with Safe ty Net. Grade 2012 2013 Without Safety Net 2013 With Safety Net A 1,242 (48%) 756 (29%) 756 (29%) B 609 (23%) 389 (15%) 680 (26%) C 494 (19%) 818 (31%) 718 (27%) D 212 (8%) 390 (15%) 353 (14%) F 40 (2%) 262 (10%) 108 (4%) Total 2,597 2,615 2,615 (Smiley & Vasquez, 2013) T he scores of student groups by race and socio economic status are not included in school grade calculations. The school grades are used to publicly identify a accountability status. This study analyze d the Department of Education tables and charts to compare the read ing achievement scores among all students and students in African American economically disadvantaged (as designated by the state), Hispanic, and White groups with each other to determine the reading achievement of the students and the changes in the achievement gaps during this period. Th is study include d an overview of the
20 comparison of the FCAT score results and the school grades that were made public to clarify the level of transparency and specificity in the student and school performance communicated by the FDOE to the public. The division that reports the test results is the Division of Accountability, Research and Measurement (ARM). Their mission is to provide data t hat informs policy and to support high standards t hat lead to (Florida Department of Education, 2013a) The division asserts its intent to support the FDOE mission of ncreasing the proficiency of all students within one seamless, efficient system, by providing them with the opportunity to expand their knowledge and skills through learning opportunities and research valued by s (2013a) The d ccountability system that measures student progress toward the following goals; highest student achievement, seamless articulation and max imum access, skilled workforce and economic development, and quality efficient services for (2013a) This division is responsible for evaluating and calculating achievement scores that indicate the status of the state, the school districts, and the schools in meeting the Reporting, which administers accountability programs such as school grades, AYP determina tions, alternative schools' improvement ratings, determinations of schools' D ifferentiated A ccountability program status, and identification of Opportun ity (2013a)
21 The accountability programs opera te to fulfill requirements imposed by the legislature. New statutes may be created or existing ones may be revised annually. Therefore, the testing and accountability progr ams are subject to policy and practice revisions annually. Statement of the Problem Re ports of test results and accountability ratin gs includ ing school grades published by the FDOE desc ribe conclusions with related consequences for school systems and schoo ls. The consequences range from recognitions to interventions. Recognitions include awarding supplemental funding to schools from special funding allocations under the Florida School Recognition program. Under the program, schools that earn a grade of A o r B, or move up one grade, earn additional funding of up to $75.00 per student (Florida Department of Education, 2014c) The interventions occur under the Differentiated Accountability program initiated in 2 008. The Differentia ted Accountability program categorizes schools by school grade s and history The categories range from Prevent I to Intervene. The consequences for the schools escalate. For example, schools in the Prevent I cat egory are visited by Differentiated Accountability staff periodically to ensure the use of improvement strategies approved by the FDOE. Also, their school improvement plans are audited. Sch ools in the Intervene category have their school improvement plan s audited, are visited more frequently for longer periods of observation and training, and could be forced to replace staff, and/or to repurpos e or clos e the school (Florida Department of Education, 2014d)
22 The FDOE annually releases FCAT scores, FCAT score summary reports, School Grade reports, and School Grade Technical Assistance Papers, School Grade Guides and media release/press packets The FDOE reports of FCAT scores and school grades emphasize the averag e scores of all student group s to illustrate progress. The reports show scores averaged for all student race socio economic English language learner (ELL), and exceptional education (ESE) groups that consistently show progress. When the reports also show disaggregated scores that do not show commensurate progress for some groups, they do not address the achievement deficiencies or achievement gaps between the s tudent groups (Florida Department of Education, 2014b; Florida Department of Education, 2014) Purpose of the Study The purpose of the study is to conduct a quantitative analysis of t he progress demonstrated by reading achievement on FCA T/FCAT 2.0 from 2002 to 201 3 by students overall and by racial and socio economic groups. The progress of each group is compared. The overall results of the analysis are compared to the progress consistently reported by the FDOE ely. An exemplary compilation available to the public for study is the National Center for Educational Statistics compilations of student performance data in all core subject areas and the related data points, The Condition of Education 2012 (U.S. Department of Education, 2012b) The center also reports on the reading ability of students, graduation or dropout rates, college readiness rates, among others, without reference to the potential for statistical incons istency of scores Th e study illustrate s and emphasizes the variances
23 in performance in reading on FCAT/FCAT 2.0 demographic group s Research Questions This study addresse d the following research questions. 1. What do the FCAT/FCAT 2.0 reading scores of 2002 to 2013 disclose about the achievement of students overall? 2. What are the differences when comparing FCAT/FCAT 2.0 reading scores of 2002 to 2013 of students by race and socio economic groups? 3. To what degree do stud ents achieve passing scores on FCAT/FCAT 2.0 reading when analyzed by racial and socio economic groups? The null hypotheses that guided the study are: H 0 1 : There will be no statistically significant difference d emonstrated by the FCAT/FCAT 2.0 reading scor es of students overall over the period of 2002 through 2013. H 0 2 : There will be n o significant differences when c omparing FCAT/FCAT 2.0 reading scores of students overall with students of racial and socio economic groups over the period of 2002 through 201 3. H 0 3 : T here will be no significant progress demonstrated by FCAT/FCAT 2.0 reading scores among students in racial and socio economic groups. Significance of the Study O ver two and half million students are enrolled in over three thousand six hundred publ ic schools in 67 school districts Florida. Findings from this study will show the actual progress that ha s been made in reading with transparency across racial and socio economic group ucational accountability system is based on a
24 high stakes test that has consequences for districts and schools to meet annual accountability measures. La dd explained at the I nternational Conference for Improving Education through Accountability and Evaluation, itizens in this democratic society have a ri ght to expect that schools will be held accoun table for effectively serving public interests (Ladd, 2012) Bey ond the accountability stakes, three primary stakeholder groups have an interest in an accurate under standing of student achievement and M embers of the firs t stakeholder group are current students and their families. M embers of the second stakeholder group are t axpayers who are funding the schools. The curre nt taxpaying citizens are funding public education with the implicit expectation that services are delivered with fiduciary responsibility. The concept of fiduciary responsibility requires accurate analysis and reporting of performance. M embers of the thir d stakeholder group are future citizens The quality of education today will have a significant impact upon the intellectual reasoning and processing capacity of the citizens of the future. Future citizens need to be capable of critical decision making and participa te in society as civic leaders or citizens who defin e the social and economic health of communities Definition of Terms Achievement Levels Five categories of achievement that represent the success students demonstrate with the content assessed o n FCAT The achievement l Achievement l evels range from one to five with l evel one being th e lowest and l evel five being the highest. To be considered on grade level, or making satisfacto ry progress learning, students must achieve l evel three or higher (Florida Department of Education, 2013m)
25 School Grades Florida schools are assigned a grade based primarily upon student achievement data from the FCAT 2.0, end of course (EOC) assessments, and the Florida Alternate Assessment (FAA). Florida uses grades to communicate how well s chool s and districts are p erform ing relative to state standards. Schoo l grades utilize a point system; points are awarded for students who score satisfactory or higher and/or make annual learning gains. The assessment based components of grades are based on achievement in reading, math, writing, and science, annual learning gains, and the progress of the lowest quartile. School grades for secondary schools include assessments graduation rates, acceleration, and college readiness (Florida Department of Education, 2013j) Learning Gains Students demonstrate learning gains in one of three ways: Improve achievement levels from one to two, two to three, or four to five; or maintain within the achievement levels of three, four, or five; or one or two (does not include retained students). In the school grade system, s chools earn one point for each percent of students who make learning gains in reading and one point for each percent of students w ho make learning gains in math (Florida Department of Education, 2013k) Low 25%, Lowest quartile Special attention is given to the reading and math gains of students in t he lowest 25 percent in levels one, two, or three in each school. Schools earn one point for each percent of the lowest performing students who make learning gains in reading and math from the previous year. It takes at least 50% to make adequate progress for this grou p. If a school has less than 50% of this group making gains, schools can still make adequate progress for the group if they demonstrate improvement over the prior year (Florida Department of Education, 2013k) Sa fety Net A rule imposed by the State Board of Education that stipulates no individual school's grade will drop more than one letter grade in any one year (Florida Department of Education, 2013l)
26 CHAPTER 2 LITERATURE REVIEW Th is study is a quan titative analysis of the achievement of student groups i n reading on /FCAT 2.0 from 2002 to 2013 The review of literature covers relevant topics to this study, including : Test based accountability FDOE r eports U nderstanding data and P olicy changes Test Based Accountability Test based accountability is fundamental to th e questioning of the measures of academic achievement by students in F over time Kathryn Ryan of test based accountabil ity as practiced in the United States in comparison to S chool Self Evaluation and Inspection (SSE/I) accountability systems employed in many European nations describes intended advantages and resulting disadvantages of the test based accountability systems (Ryan, Gandha, & Ahn, 2013) Ryan explains that the test based accountability nature of NCLB was dependent upon measuring performance with high stakes assessments based on questionable standards, resulting in negative sanct ions while it was intended to stimulate school impro vement (Ryan, Gandha, & Ahn) Ryan explain s the result ha s been minimal, and that improvements are questionable Most pointedly, from analysis of performance data compilations, including (Nation's Report Card, 2014) Ryan concluded that students have not reached intended levels of mastery in reading and the achievement gaps between st udent racial and socio economic groups that the law was meant to resolve has not been reduced (Nation's Report Card) Ryan suggest s test based accountability
27 does not serve to improve schools She emphasiz e s an analysis of s chool safety, leadership effectiveness, and school improvement strategies a re import a nt steps in improving t eaching, learning and student outcomes characteristics that are often overlooked by te st based accou ntability systems (2013, p. 5) Ryan recommend s test based accountability as well as measures to improve the efficacy of instruction, manage better and advance the quality of both s chool leadership and management (2013) Unintended Negative Consequences Unintended unf oreseen negative results of test based accountability systems that emerged from the NCLB A ct of 2001 are emphasized in numerous studies. The negative r esults range from generally less effective school improvement effort s to s pecific repercussions to teachers and student groups. Volante and Jafaar summarize a comparison of the negative and positive consequences They conclude negative outcomes outweigh the positive (2010, p. 170) Negative results of test based accountability include: H igh stakes test ing of the NCLB accountability era that has not increased student achievement (Aimer & Berliner, 2003; Betebenner & Linn, 2009; Linn, 2010; Volante & Jaafar, 2010; Ryan & Shepard, 2008) Failure to reduce the achievement gap (Smyth, 2008; Haretos, 2005; Braun, Chapman, & Vezzu, 2010; Linn, 2010; Volante & Jaafar, 2010; Ryan & Shepard, 2008) Increased dropout and reduced graduation rates for some racial and socio economic groups and for all students o verall (Aimer & Berliner, 2003; Volante & Jaafar, 2010) Increased retentions, especially in the elementary intermediate grades and for some racial and socio economic student groups (Aimer & Berliner, 2003; Haretos, 2005; Volante & Jaafar, 2010) Teachers and counselors roles are l ess facilitative of development and consequently have become more judgmental (Duffy, G iordano, Farrell, Paneque, & Crump, 2008)
28 Reduced resource allocations for the arts, social sciences and physical education (Duffy, Giordano, Farrell, Paneque, & Crump, 2008) M ovement of highly qualified staff to more hi ghly ranked schools (Volante & Jaafar, 2010) S anctions for s chools in lower income and racially diverse communities (Maleko & Gawlik, 2011) L ack of progress in developing tools that assist in understand ing how to promote learning for traditionally struggling groups (Haretos, 2005; Volante & Jaafar, 2010) T est based teaching that compromises student interest and limits content scope (Volante & Jaafar, 2010) A fail ure to engage essential external support for school improvement efforts (Ryan, Gandha, & Ahn, 2013) The negative consequences for students who do not meet adequate yearly progress (AYP) is particularly damaging in racially diverse schools. The standards for meeting AYP are set based on the achievement of student groups with differing levels of learning history. The sanctions and negative consequences, such as remedial classes and re tention, fall heavily upon racial and socio economic groups with low er scores. FDOE Reports The literature review included the study of measurements of student achievement provided by the FDOE in F CAT/FCAT 2.0 test score report summaries press releases and t echnical assistance documents. The FCAT/FCAT 2.0 scores in reading for all students in the racial, socio economic and learner profile groups are available as Microsoft (MS) Excel spreadsheets online at the FDOE website. Learner profile groups include ESE and ELL. The scores on the FDOE website include grades three through ten on FCAT/FCAT 2.0 in reading, writing, mathematics and science. The scores are displayed in aggregate and disaggregate forms. In aggregate form, without
29 considering any other data, the scores show a general trend of improvement in educational outcomes for students for each year of the study (Florida Department of Education, 2013h) However, when additional data is considered, such as changes in student groups or grade levels, significant progress is not consistently supported (2013h) Table 2 1 exemplifies the difference when the scores of all students are compared the scores of students designated as economically d isadvantaged (econ. dis.) by the state. The table illustrates the percentage of students who scored at level three or above on FCAT in reading in the fifth, eighth, and tenth grade levels, in 2007, 2010, and 2013. The table shows that the average scores of all students are consistently higher than the scores of students in this group, and that the difference is greater at the higher grade levels. Table 2 1. FCAT reading achievement of student groups by grade level. Grade FCAT reading % level 3 E con. d is. 2007 All 2007 Econ. d is. 2010 All 2010 Econ. d is. 2013 All 2013 5 60% 72% 60% 69% 50 % 60% 8 34% 49% 43% 55% 45% 56% 10 19% 34% 26% 39% 41% 54% (Florida Department of Education, 2013e) The problem presented by the lower scores of some student groups and in some grade levels is not directly addressed by FDOE FCA T score reports or school grade reports. Technical Assistance Guides (TAG) explain changes to achievement levels T able 2 2 shows the achievement level score s in 2008 and in 2012.
30 Table 2 2 FCAT reading achievement levels, 2008 and 2012. 2008 Grade Level 1 Level 2 Level 3 Level 4 Level 5 3 100 258 259 283 284 331 332 393 394 500 4 100 274 275 298 299 338 339 385 386 500 5 100 255 256 285 286 330 331 383 384 500 6 100 264 265 295 296 338 339 386 387 500 7 100 266 267 299 300 343 344 388 389 500 8 100 270 271 309 310 349 350 393 394 500 9 100 284 285 321 322 353 354 381 382 500 10 100 286 287 326 327 354 355 371 372 500 2012 Grade Level 1 Level 2 Level 3 Level 4 Level 5 3 140 181 182 197 198 209 210 226 227 260 4 154 191 192 207 208 220 221 237 238 269 5 161 199 200 215 216 229 230 245 246 277 6 167 206 207 221 222 236 237 251 252 283 7 171 212 213 227 228 242 243 257 258 289 8 175 217 218 234 235 248 249 263 264 296 9 178 221 222 239 240 252 253 267 268 302 10 188 227 228 244 245 255 256 270 271 302 (Florida Department of Education, 2013d) The FDOE publishes media packets to explain the results of testing each year. A typical press release is the report is the FCAT/FCAT 2.0 Media Packet, 2013, made available to the public in June of 2013 on the FDOE website (Florida Department of Education, 2013j) The packet explains the transition from th e FCAT to the FCAT 2.0 testing. The report addresses the classification of students scoring at achievement level three or above. In prior years, scoring at achievement level three or above was classified as scoring at or above grade level, and demonstratin g sufficient academic success with the standards tested to graduate and be prepared for college or career level pursuant to Section 1008.22(3)(c)5., F.S. a level three score that is a graduation (Florida Department of Education, 2011b) In reading, the score at levels three, four, or
31 five, indicate that student s are able to read at or above the expected capacity for the grade level in which they were enrolled and at which they were tested. The 2013 media packet explains the terminology being used has changed to state that students scoring at level three demonstr Next Generation Sunshine State Standards (Florida Department of Education, 2013f) Students scoring th the Next Generation Sunshine State Standards (2013f) while s tudents scoring at level five had Next Generation (2013f) The FCAT 2013 media packet presents data for grade levels and for the different racial groups as well as learner profile and grade level classifications. The racial groups include African American Hispanic, and Whi te. The learner profile classifications include ESE and ELL. The grade level reports are organized into three groups. The elementary group includes grade levels three, four, and five. The secondary levels include the sixth, seventh and eighth grade group a nd the ninth and tenth grade group (Florida Department of Education, 2013f) demonstrated satisfactory progress in reading in 2011, 57% showed that l evel of progress in 2012, and 57% showed that level of progress in 2013, without any mention of (2013f) The report shows the progress of students in the three main racial group s, African American Hispanic, and White also remained stagnant. The percent of African American students showing satisfactory progress in reading ranged from 36% to 38% over the th ree years
32 The percent of Hispanic students showing satisfactory progress in reading ranged from 51% to 53% over the three years The percent of White students showing satisfactory progress in reading ranged from 68% to 69% over the three years (2013f) All groups showed similar scores of minimal improvement in percentage of students achieving satisfactory levels of success with the reading standards over the three years. As few as 10% of some student groups have a chieved satisfactory levels of success According to FCAT demographic score reports available, t he ELL group in grade three s howed declining percentages of students achieving level three or above over the three year period (2013f) Table 2 3 shows a comparison of students in the ELL group with all students who scored at level three or above on FCAT in reading for grade levels three, five, eight, and ten. The lack of progress for ELL students, compared to all studen ts, is not discussed in FDOE FCAT score reports or FDOE School Grade reports. Table 2 3. FCAT reading scores for ELL students and for all students. Grade level Student Group 2011 % level 3 2012 % level 3 2013 % level 3 3 All 57 56 57 3 ELL 18 22 21 5 All 60 61 58 5 ELL 22 20 18 8 All 56 55 53 8 ELL 12 10 10 10 All 54 50 52 10 ELL 11 9 11 (Florida Department of Education, 2013e) The narratives of the media packet do not address the lack of significant change in the scores, showing the lack of progress or declines, among students demonstrating a satisfactory level of success with the standards (2013f) Table 2. 4 shows the percentage achieving level 3 or above on FCAT in reading of several groups of
33 elementary school students, o nly where declines in performance occurred in the period of 2011 to 2013 The declines were not addressed by the FDOE media packets (Florida Department of Education, 2013f) Table 2 4 Studen t groups with FCAT reading achievement declines, 2011 2013. Grade Student group 2011 2012 2013 % level 3 3 African American 38 37 38 3 Econ. Dis. 46 45 46 3 ELL 21 22 18 3 Hispanic/Latino 51 52 50 3 White 71 69 71 3 Total Students 57 56 57 4 African American 39 43 41 4 Econ. Dis. 48 52 49 4 Hispanic/Latino 55 57 56 4 White 70 73 71 4 Total Students 59 62 60 5 Hispanic/Latino 53 58 56 5 Total Students 58 61 60 3, 4 & 5 Econ. Dis. 47 49 48 3, 4 & 5 Hispanic/Latino 53 56 54 3, 4 & 5 Total Students 58 60 59 (Florida Department of Education, 2013e) Understanding Data Accuracy in Calculations and Conclusions valid interpretation of student performance as does the educational reform for which such an accountability system was put in place: ensuring that our students are achieving rigo (Huff & Plake, 2010, p. 130) M isguided use of test score results t hat determine allocation of school resources could be avoided if data was interpreted with higher accuracy (Wainer, 2011) practices of using data for decision making show s the process often includes
34 misinterpretation of the data and the conclusions. Ultimately, this leads to the misguided decisions, policies, and allocation of reso urces (Duffy, Giordano, Farrell, Paneque, & Crump, 2008; Volante & Jaafar, 2010; Wainer, 2011) In Uneducated Guesses on the Advance Placement (AP) Calculus test scores were analyzed and the results were publicized in the 1988 movie Stand and Deliver Wainer explain s that an analysis reliminary Scholastic Aptitude Test (PSAT) scores in math and subsequent scores on AP Calculus tests provides information that can be success of the Garfield students is c learly validated, Wainer poses the question if the teacher, Jaime Escalante, was a particularly good teacher or a good recruiter. The data analysis shows that, indeed, Escalante recruited almost 100% of the students in the school who scored in the highest levels on the PSAT in math. This leads to an initial conclusion that he was an exceptionally crafty recruiter, as opposed to exceptional teacher. Wainer explains that the percentage of students who were recruited from the high scoring PSAT student group th at passed the AP Calculus exam was 100%, while about 80% of the students from that same high scoring PSAT group in a school in a more elite community passed the test. Overall, predictive analysis shows that 53 of Mr. the test, while 83 students actually succeeded. The conclusion is that Mr. Escalante was both an exceptional recruiter and teacher T his type of conclusion from predictive analysis is particularly important to policymaking and resource allocation. Wainer suggest s more extensive data gathering
35 and the importance of using performance data more effectively to improve schools (Wainer, 2011) Wainer explains as an example that predictive analysis can be used to determine score requ irements for entry into AP courses, so the classrooms, scheduled class times, and number of teachers designated for AP courses is controlled by the number of students with the capacity for success in the course. The relevance of to this st udy is founded in the call for more advanced analysis of test scores and the conclusions that are drawn from them. Wainer suggests methods of data analysis of test scores that are based in science, logic, objectivity, and fundamental common sense. Wainer rec ommends five criteria for da ta analysis, which are important to evaluation of school s : 1. details specifically address an important topic in current school accountability ratings in Florida: AP course enrollment versus success passing AP tests. 2. A characterization of student ability. The school being held accountable for factors that it cannot control. Wainer suggests that accurate dat a analysis can reveal what a student is capable of achieving through predictive analysis. 3. A functional connection between success and qualifications. Wainer suggests schools should be evaluated based on what students have the capacity to achieve correlated to what the students actually achieved as a result of the education provided by the school. 4. A model for selection of students into AP courses. Wainer suggests that school systems can best utilize resources by offering AP courses to students who are most l ikely to succeed in those courses and identifying that student group through aptitude testing. 5. A measure of teaching and motivation. Wainer argues that once data are used logically to make student placement decisions, the teacher efficacy and student motiv ation factors that elude accurate analysis are more likely to emerge for objective consideration. (Wainer, 2011)
36 In addition to the fundamental relevance of the importance of accurate data analysis of test scores to evaluating t he quality of education to this study, several points the AP courses, industry certification examinations, school rankings, and state funding. Public Perception A Har vard University team led by Matthew M. Chingos of the Program on Education Policy and Governance published its research related to the impact of tests scores to public perception of the quality of schools in Citizen Perceptions of Government Service Qualit y: Evidence from Public Schools In this report, the team n students receive in its public schools. The team found that accountability reports about student achievement levels in schools pre vailed as the primary factor influencing citizen opinion of the quality of education received by students in the schools. The Harvard team analyzed perceptions of school quality compared to the actual school quality, among citizens with offspring in school s compared to citizens without offspring enrolled in the schools. reports abou t school quality (Chingos, Henderson, & West, 2010) The report expresses the importance of citizens having access to accurate information to evaluate government performance correctly and concludes that accountability programs have a of an oversample of Florida residents confirms that public accountability systems can at perceptions are
37 (2010, p. 2) Accurate P erformance Reporting in Florida Bracey concurs with Wainer about in sightful, thoughtful, logical, data analysis of test scores but oppose s the use of analysis of the results of aptitude testing and achievement testing to determine school quality. Bracey cite s potentially serious flaws in both aptitude and achievement test s as the cause for concern (Bracey, 2007) 2007 Phi Delta Kappan article, Another Way to Game the System focuses on the importance of informed data analysis of test scores by accountability systems and the public at la rge (2007) He describe s (p. 412) Brace y describe s research performed by Walter Haney of the Center for the Study of Testing, Evaluation and Education Policy, in the Lynch School of Education at Boston College. In a paper presented at the Hechinger Institute in 2006, Haney presented research on s research on the use of National Assessment of Educational Progress (NAEP) as a quality check of state assessment programs that rev eal ed breaches in the capacity of both NAEP and FCAT test scores to document school quality. Haney describe s the accountability reports (Haney, 2006, p. 2) fourth grade students. While students nationwide only showed a two point increase from Moreover, the race gap in achievement was even more impressive: nationwide, White
38 students showed a three point increase and African American students showed a four point increase, while in Florida, White students showed a four point increase and Florida African American students showed an unprecedented nine point increase. African American increase was almost one third the standard deviation. Haney and Bracey both document the governo to erroneous interpretation of the test scores. Haney shows that the policy to retain third grade stude t, FCAT, caused the jump. Haney reviewed enrollment data in Florida schools and established transition ratios to reflect the percentage of students who were promoted from one grade level to the next. The data aligned with the policy. F urther analysis of student de mographic groups emphasized that excessive proportions of African American and Hispanic third grade students had been retained under the new policy in the year prior to the incredibly impressive fourth grade NAEP scores. C onclusions from the test score dat a th at the in 2005, under the same fundamental principles of accountability and reporting used today, were seriously flawed. It appeared that students improved in achievement because f ewer students were tested. The students who were not tested were the students who had been retained. The retained students would have probably scored poorly on the test. A disproportionately large percentage of the students who were retained were Hispanic and African American students, 15 to 20%, versus White students, at 4 to 6% (Haney, 2006)
39 Haney question s test scores. Haney note s grade nine and (p. 3) Linn e xplain ed F based on the students test performance s c scores, results in many schools with grades of A in the state system that fail to make AYP as requi red by federal l aw (Linn, 2008) Policy Changes There is a wide range of literature supporting major policy changes to improve student achievement for all student groups and to reduce the achievement gaps between student gro ups. Ryan promote s an overhaul of the accountability systems to include revision of educational policy to result in the significant learning gains needed in the U.S. (2013, p. 12) Linn calls for an overhaul of edu cational policy on student testing and system accountability, as well (Linn, 2010) Broad policy changes are also encouraged by the 2010 study conducted by Braun and associates of Boston College, of the achievement gap betwe en African American appears there is a need for both fresh thinking on education reform and a more on (Braun, Chapman, & Vezzu, 2010) The study evaluated the same ten states studied prior to NCLB to compare pre and post NCLB outcomes. The study compared student test scores, as well as policy changes aimed toward reducing the achievement
40 gap. Overall, substantial gaps in achievement wer e found and it was determined that NCLB, and related test based accountability policy changes, made very little difference in achievement of African American students (Braun, Chapman, & Vezzu) Improvement Requires Accur ate Data The importance of data in terms of understanding what students know, what they have learned, and what they need to learn is reiterated across much of the literature. (2011) summary emphasize s the importance of using analysis of test scor es to fuel continuous improvement efforts (Lezotte, 2011) C onclusions drawn from data analysis is intent upon classifying and categorizing schools and instructional systems to benefit executive level decision and policymaking, improving school s. His research teams point out characteristics that can transform scho ols to achieve high levels of effectiveness. The ten points that Daggett conveys are: 1. Create a culture that embraces the belief that all students need a rigorous and relevant curriculum and all children can learn. 2. Use data to provide a clear unwavering f ocus to curriculum priorities that is both rigorous and relevant by identifying what is essential, nice to know, and not necessary. 3. Provide students real world applications of the skills and knowledge taught in the academic curriculum. 4. Create a framework to organize curriculum that drives instruction toward both rigor and relevance and leads to a continuum of instruction between grades and between disciplines. 5. interest, learnin g style, aptitude, and needs.
41 6. Set high expectations that are monitored and hold both students and adults accountable for above. 7. Sustained professional development that is focused on the im provement of instruction. 8. Obtain and leverage parent and community involvement successfully in schools. 9. Establish and maintain safe and orderly schools. 10. Offer effective leadership development for administrators, teachers, parents, and community. (Daggett, 2005, pp. 3 4) Data Driving Instruction Daggett emphasize d the importance of needs and monitor progress in meeting them He also stressed t he importance of data analysis informing a commitment to continuous improvement, and professional development in skills needed to interpret data reports accurately (Daggett, 2005) einforced widely. As recently as 2008, a study by Dennis in the Journal of Adolescent and Adult Literacy pointed out that standardized testing data are used to make instructional placements yet the test results do not reveal learner needs sufficiently to i nform i nstructional decisions The authors found that standardized assessments, such as FCAT, are similar to pass/fail tests, indicating if students can read on grade level or not. These tests, ing problems. S instructional needs in reading vary widely, from basic word decoding to advanced comprehension and analysis of text. Students scoring below achievement level 3 on FCAT reading were placed in the same remedial reading classes. Althou gh supplemental tests were performed, Dennis (2008) found that the teachers did not
42 u nderstand how to use results to make instructional program decisions. Students with very different instructional needs often receive the same instruction which is often in effective (Dennis, 2008) State Level Data Usage (NGA) promotes the efforts of the National Data Quality Campaign (DQC) to enhance the capacity of school districts to use data to improve instr uction The DQC stresses the importance of role of state support and coordination The DQC focuses on the key factors to be considered in the use of data to enhance educational sy stems and schools effectively. K ey points in the publication speak directly t o the research topics. The publication stresses the use of data by school districts to improve teaching and learning is not occurring sufficiently well or quickly readin ess levels for all students are not at the point the educational incentives initiated by federal and state governments over the last decade had been planned to cause. The publication explains the efforts to report and use data to make decisions that lead t o improved teaching and learning have not been genuinely motivated. The efforts have been compliance measures. School districts nationwide have reported data to meet state requirements but states have not facilitated the use of data that would actually imp rove teaching and learning (Data Quality Campaign, 2013a) States and districts are not helping one another resolve this issue, according to the DQC. A few states have taken key actions by supporting district efforts to analyze data to evaluate policies and programs, understand patterns of performance, and determine effective allocation of resources, but most have not. Some states have helped districts develop early warning systems that help educators align resources to meet lear ner needs, but most have not.
43 The DQC reports that state and district collaboration about the use of data is important to accomplishing these shared objectives that are important to improving schools: Maximize data investments and reduce costs and burden. Ensure cross district and cross state comparability. Meet the needs of all stakeholders. Equalize and enhance district capacity. (2013a) To resolve these issues, the DQC identifies four guiding principles for states th at speak to this research study: Collaboratively identify district data capacity to inform state data efforts. Transform data into actionable information and ensure district access. Ensure data literacy among educators through preservice and in service pol icies and practices. Maximize efficiency and minimize burden in data collection. (2013a) These four principles are interrelated in practical application to the importance of using data to understand school performance accurately. Much like the adage that problems cannot be solved if they are not acknowledged, schools cannot be improved if the need for improvement is not understood with accuracy. The third principle, ensure data literacy among educators through preservic e and in service policies and practices, is fundamental to all the principles (2013a) If the key practitioners in the system do not understand how to analyze data and draw conclusions that inform decisions well, the mer it of the other principles is greatly reduced. did not facilitate clear public understanding of the learning progress achieved by all students in the educational system. Ther efore, the third principle the need to ensure data literacy among educators through preservice and in service policies and practices, is particularly poignant to this study (2013a) As mentioned briefly in the introduct ion,
44 the DQC lists ten actions states must take to be able to use data effectively to improve schools: 1. Link state K 12 data systems with early learning, postsecondary education, workforce, social services, and other agencies. 2. Create stable, sustained suppo rt for state longitudinal data systems. 3. Develop governance structures to guide data collection, sharing, and use. 4. Build state data repositories (e.g., data warehouses) that integrate student, staff, financial, and facility data. 5. Implement systems to provid e all stakeholders with timely access to the information they need while protecting student privacy. 6. Create progress reports with individual student data that provide information educators, parents, and students can use to improve student performance. 7. Crea te reports that include longitudinal statistics on school systems and groups of students to guide school district and state level improvement efforts. 8. Develop a purposeful research agenda and collaborate with universities, researchers and intermediary groups to explore the data for useful information. 9. Implement policies and promote practices, including professional development and credentialing, to ensure educators know how to access, analyze, and use data appropriately. 10. Promote strategies to raise awar eness of available data and ensure that all key stakeholders, including state policymakers, know how to access, analyze, and use the information. (Data Quality Campaign, 2013c) As of late 2013, the FDOE had not successfully impl emented two of the ten actions. The FDOE had not completed actions three and five Although the FDOE had school systems and groups of students to guide school dist rict and state level (Data Quality Campaign, 2013c) Further, the FDOE had not facilitated parental use of the longitudinal data to evaluate their
45 (Data Quality Campaign, 2013c) The NGA Issue Brief, Using Data to Guide Sate Education Policy, designates similar requirements for effective state and district use of data to inform improvement in schools effectively. The Issue Brief also ident ifies the problems associated with poor data analysis and communication by reiterating that problems that are not identified cannot be solved, because state policy will not evolve to meet needs that are not known to exist. The Brief states that over $500 m illion has been spent by the United States federal government to support the effective use of data in school systems and the return on that investment is dismal. The brief stresses the importance of data reporting being easy to access and accurate (National Governor's Association, 2012) The overarching concern of this type of study is data quality. Betebenner and Linn discussed related data quality issues in an exploratory seminar, Measurement Challenges within the Race to the T op Agenda, which addressed the nature of data quality concerns in communicating educational progress. Betebenner and Linn propose, (Betebenner & Linn, 2009) Betebenner and Linn reinforce concerns about insufficient research to motivate consistency in standardized practices for analyzing educational data and reporting educational progress. In a separate article publ ished the same year, Betebenner describes the needs of the public and the school based stakeholders in deriving meaning from test scores and accountability systems as determined through surveys: understanding student growth (Bet ebenner, 2009)
46 Summary The review of literature establishes that the movement of test based accountability driven by the NCLB Act of 2001 has resulted in outcomes that support questioning the achievement of students by groups and the closing of the a chievement gap in Florida. Under NCLB: Student achievement has not been increased nationwide. The achievement gap between student groups has not been diminished. Student achievement data is not used effectively to improve schools systems. Instruction has not improved and learning has not advanced. Effective use of data to inform the public and the school system decision makers about learning progress is not an established standard in the Florida school system, and should be. The representation of learning progress in Florida is not disclosed with focus on achievement issues that are problematic and call for system reforms. This review of literature confirms the need for analysis of student achievement that is used for accountability purposes. The purpose o f this s tudy is reinforced by the literature: there is a need to identify the problem in the variances of student achievement among student group s
47 CHAPTER 3 METHODOLOGY This chapter provides the methodology and design of the research used for this study. Initially, the research methodology is explained, followed by descriptions of the variables of consideration, representation of findings, origin and sources of the data, instrumentation, assumptions, and limitations. The chapter concludes with a summary o f the methodology used. Research Methodology To achieve the intended outcome for this research, descriptive statistics and statistically analysis were used with FCAT score summar y reports in reading. The percentage of students reported as making sufficie nt progress by achieving level three or above considered by year and by student groups is provided The s tatistical analysis was performed using SPSS Version 21. In general, each research question required the following analysis: 1. Evaluation of the data that are the basis of the information the state provides FCAT/FCAT 2.0 scores in reading in aggregated and di saggregated comparative charts for the years of 2002 to 2013. 2. Analy sis of the performance in reading of s tudents by groups in grades three through ten from 2002 to 2013 to identify trends of achiev ement among student group s. 3. Compar ison of the achievement of student group s and consider ation of the results in comparison to public reports released by the FDOE about the progress Research Question One R esearch question one require d determination if the FCAT/FCAT 2.0 reading scores for all students overall indicate d significant learning progress from 2002 to 2013. The reports of the percentage of students scoring at level three and above on FCAT in
48 reading for each year from 2 002 to 2013 was obtained from the FDOE website. The percentage of students who scored at and above level three for all students was compared from 2002 to 2013 a nd for each school year in between So, 2002 was compared to 2013 2002 was compared to 2003; 2003 was compared to 2004, etc. to determine the statistical significance of the differences across the y ears and each pair of years within the time period. Statistical analysis wa s used to determine the significance of the differences of the scores Where there was a statistically significant wa s determined to be s ignificant. Where the difference was not statistically significant, th determined not to be significant. T test procedures were used for this process because the independent variable consists of only two categories. This st ep compare d the scores of one group of students over two time periods in each procedure. P aired t tests are selected because the units of analysis are the same. Statistical significance was set at the standard criterion for assessing significance in a t test, .05 O ne tailed tests were used for the d irectional test The degre e of freedom for comparing the eight grade levels wa s t (7). Research Question Two Research question two require d the analysis of the differences between the percentage of students i n four groups who achieved level three and above on FCAT in reading from 2002 to 2013 and each one year span within that period. The reports of the percentage of students who achieved a level three and above on FCAT in reading w ere obtained from the FDOE w ebsite. The results for four student groups w ere analyzed for question two: African American economically disadvantaged, Hispanic, and White. The percentage of students in each group achieving level three and above
49 on the test w ere statistically compared. ANOVA, a one way analysis of variance, wa s the statistical tool used for analysis of the differences and determin ation of the significance of the differences ANOVA is commonly used when the independent variable has three or more categories. Subsequently, where the difference in outcomes of the four groups ha d statistical significance, the next step determined which pair of the groups ha d the significant difference. P ost hoc Tuke y tests we re used to determine which pair of the groups compared cause the significance. Research Question Three Research question three require d analysis of the learning progress indicated by the scores within each of the student group s studied: African A merican economically disadvantaged, Hispanic, and White. The reports of the percentage of students within each group who achieved a level three and above on FCAT in reading w ere obtained from the FDOE website. The percentage of students who achieved level three and above w ere compared from 2002 to 2013, and for each one year span within that period, for each student group. The comparisons w ere statistically analyzed to determine if significant progress occurred across the time period, and if significan t pr ogress occurred in each one year span within that period. So, the percentage of students achieving level three and above in 2002 w as compared to 2013, and 2002 w as compared to 2003, 2003 w as compared to 2004, 2004 w as compared to 2005, etc., for each group Paired t test procedures were used to compare percent of students scoring at or above level three on FCAT in reading i n each group, across the years, and within each year span, to determine if significant progress wa s made.
50 Variables of Consideration To address the research questions data points of FCAT/FCAT 2.0 scores analyzed include 1. The statewide results of FCAT/FCAT 2.0 in r eading from 2002 to 2013 2. The results of FCAT/FCAT 2.0 in r eading disaggregated by student profiles 3. The total number of students tested and the percentage all students tested who scored at a chievement level three and above 4. The number of students tested in the African American, economically disadvantaged, Hispanic, and White groups and the percentage in each group who scored at achi evement level three and above The FCAT score reports are presented on the FDOE Demographics website with s cores categorized by learner profiles that include student demographic informat ion, including socio economic status, ESE status, ELL status, and racia l/ethnic group classification echnical assistance guide s we re studied to determine the processes used by the state to produce the score reports. The processes used by the state achievement learner profile classification, these classifications we re compared over the 12 ye ar period to identify consistent groups to compare across the years. Th e state designates that students scoring at level three and above are meeting the learning standards satisfactorily, also described as performing at or above grade level (Florida Department of Education, 2013d) In addition to test scores, enrollment membership data are considered to compare quantities of students tested and determine the impact to outcome if present. Upon analysis of the technical assistance guides and enrollment data, clarification of equ itable comparison groups we re ac hieved by determining if there we re variances in the c omponents of the score reports.
51 The r esearch question s require d comparison of results for each year. Upon clarification of equi table comparison groups, the amount of change that has occurred each year, by each of the categories, wa s analyzed. Ultimately, the amount of change per year for each of the 12 years, and for the 12 year period overall, for each of the student profile groups wa s determined. The r esearch question s require d descriptive analysis of the test scores in the score categories listed, across the 12 year period to identify the progress in stud ent achievement by grade level. For this purpo se, comparative analysis determined the results for the FCAT/FCAT 2.0 t est score data for: 1. Each student group and for all students in aggregate 2. A ll grade levels in aggregate 3. Each year and for the 12 year period in aggregate 4. Achievement of level three or above 5. In r eading The null hypothesis is that there wa s no significant progress over the years for student s overall no significant differences of achievement among demographic group s of students, and no significant progress of stude nts in the racial and socio economic group s studied Origin and Sources of Data All data analy zed for the study w ere obtained online from the FDO E website The FDOE website offers public access to FCAT/FCAT 2.0 score reports by district and school, according to student group s, and for the subject area studied, r eading, over the years of study, 2002 to 2013. The FCAT/FCAT 2.0 results and the school grades for each level of school and all schools overall, for all years of the study, were obtained from the FDOE website. scores in
52 unidentifiable groups mee ting accountability criteria for school grades and NCLB/AYP purposes. The FDOE website also provides all TAGs for the score reports. Instrumentation The state of Florida chose FCAT/FCAT 2.0 as the primary source of determining student achievement for most public school students during the years of the study, 2002 to 2013. As related in the introduction, reading achievement on FCAT has been d esignated by the state for promotion criteria in third grade special remedial program criteria, and graduation criteria, as well as accountability criteria for schools (Florida Department of Education, 2013f) Therefore, this study focuse d on FCAT/FCAT 2.0 achievement in reading to determine learning progress over those years. In this study, student FCAT/FCAT 2.0 scores by achievement at level three and above as defined by the FDOE we re analyzed by student groups and in aggregate overall, in r eading Assumptions The validity and reliability of the FCAT/FCAT 2.0 are assumed without question achievement. Assumptions were made that the FDOE Technical Assistance Guides disclose all pertinent and relevant information and updat ed modifications to understand the wa y the FCAT/FCAT 2.0 scores are derived, compiled, and presented for public use Limitations The study limitations include access to information that is made publicly available b y the FDOE. Additionally, the limitations presented by the use of the FDOE imposed upon this study. The FDOE ch
53 designation of achievement levels across the 2010 and 2011 sch ool years. Therefore, there are limitations to the validity of the achievement of level t hree and above as the definitive indicator for successful learning. However, the communicated progress to the public, of achievement of level three on FCAT/FCAT 2.0 the measure to investigate. The designation is also primary basis for assigning school grades the means of communicating school quality and progress Additional l imitations occur with the comparison of groups that contain duplicate student scores. The groups selected for study are the students classified as African American economically disadvantaged, Hispanic, and White. Duplication occurs because economic disadv antage occurs across all groups. Scores for economically disadvantaged students are duplicated within the scores for the African American Hispanic, and White students. Additionally, scores for African American Hispanic, and White students are duplicated within the scores for the economically disadvantaged students. The significance of the limitation is reduced with acknowledgement and offset by the expectation of learning progress for all student groups, regardless of grouping. Summary In summary, this st udy used the FCAT/FCAT 2.0 test scores published by the FDOE and com pare d those scores among student groups to identify achievement progress overall and by group s. The study consider ed the progress made by all student s a nd by students in the African Americ an economically disadvantaged, Hispanic, and White groups and compare d that progress to the designation of quality and progress the state attributes to schools under the school grade system : the percentage of students achieving level three and above. The study use d paired t tests
54 to compare progress of students overall for the years studied, ANOVA to compare defined groups to analyze progress of the efforts to close the achievement gap and paired t tests to determine progress within each group selected fo r the study. Compilation s of the results in tables scores and the progress over time by students in Florida
55 CHAPTER 4 RESULTS AND ANALYSIS OF DATA The results of the research study are presented in this chapter. The purpose of the analysis of the data was to clarify the progress in reading achievement represented by FCAT /FCAT 2.0 scores over the 12 years from 2002 to 2013 The student groups for whom the progress was analyzed were students overall and African American economically disadvantaged (as defined by the state), Hispanic and White. The analysis was performed to determine the progress of the students overall, compare the progress in closing t he achievement gap between the student groups, and determine the progress achieved by each of the students groups studied. The researcher analyzed the FCAT/FCAT 2.0 results in reading of stude nts in grades three through ten focused on the percentage of stu dents achieving level three or above. The FCAT/FCAT 2.0 reading results were selected because the FDOE requires the testing every year from grade three through grade ten, uses the scores in calculations that determine third grade promotion, student program placement in grades three through ten and as criteria for graduation with a standard diploma (Florida Department of Education, 2013f; Florida Department of Education, 2014e) Research Questions The results and analysis for the following research questions were addressed : 1. What do the FCAT/FCAT 2.0 reading scores of 2002 to 2013 disclose about the achievement of students overall? 2. What are the differences when comparing FCAT/FCAT 2.0 reading scores of 2002 to 2013 of students by race and socio economic groups? 3. To what degree do students achieve passing scores on FCAT/FCAT 2.0 reading when analyzed by racial and socio economic groups?
56 Analysis From the reports provided by the FDOE, the study analyzed data showing t he number of students tested and the percen tage of students scoring at level three and above in reading by grade level for all students and for each group of African American economically disadvantaged (Econ. Dis.), Hispanic, and White students for grade levels three through ten f rom 2002 to 2013. Table 4 1 provides an ove All students in regular educational programs in public schools in Florida are required to take FCAT in reading in grades three through ten (Florida Department of Education, 2013f) The changes in the number of students tested overall represents the trend in changing numbers of students w ithin each demographic group The Hispanic group increased over the period. The e conomically disadvantaged group increased over the period, as well. The White group decreased over the period by over 100,000 students. The African American group decrease d slightly from 2002 to 2013. Table 4 1. Students tested. 2002 2003 2004 2005 2006 2007 African American 357,502 362,469 363,352 363,991 359,725 354,511 Hispanic 303,648 323,656 340,947 358,304 372,249 381,595 Econ. Dis. 701,361 720,572 774,982 752,495 737,151 733,034 White 781,188 783,520 784,030 779,581 763,126 742,230 All 1,498,688 1,533,913 1,559,082 1,580,536 1,582,232 1,571,818 2008 2009 2010 2011 2012 2013 African American 352,127 350,214 350,113 353,129 352,991 352,415 Hispanic 386,813 393,101 414,203 443,894 454,338 466,627 Econ. Dis. 754,239 803,361 873,660 906,218 933,227 946,833 White 723,516 711,615 700,322 678,717 668,774 660,085 All 1,560,826 1,582,232 1,571,818 1,570,656 1,573,009 1,578,282 (Florida Department of Education, 2013g)
57 Table 4 2 shows the number of students in each group studied by percentages The F DOE did not release data for economically disadvantaged students for 2001. Table 4 2. Percent c hange in number of students tested 2001 2013 Percent Change 2001 2013 African American 3.22 Hispanic 70.19 Econ. Dis.* 35.00 White 14.45 All Students 9.38 *Change is calculated from 2002 to 2013 for economically disadvantaged students. (Florida Department of Education, 2013g) Table 4 2 displays the significant increase of over 70% in the population of the Hispanic students there was a 35% increase in the numb er of economically disadvantaged students From 2001 to 2013, t he public schools decreased over 14%. Overall, the number of students tested increased by over 9% during the time of the study. The changes each year are reflected in Table 4 3. Table 4 3. Percent c hange in numbe r of students tested annually. Percent change 2001 02 2002 03 2003 04 2004 05 2005 06 2006 07 African American 4.71 1.39 0.24 0.18 1.17 1.45 Hispanic 10.74 6.59 5.34 5.09 3.89 2.51 Econ. Dis. 2.74 7.55 2.90 2.04 0.56 White 1.24 0.30 0.07 0.57 2.11 2.74 All 3.87 2.35 1.64 1.38 0.11 0.66 2007 08 2008 09 2009 10 2010 11 2011 12 2012 13 African American 0.67 0.54 0.03 0.86 0.04 0.16 Hispanic 1.37 1.63 5.37 7.17 2.35 2.70 Econ. Dis. 2.89 6.51 8.75 3.73 2.98 1.46 White 2.52 1.64 1.59 3.09 1.46 1.30 All 0.70 1.37 0.66 0.07 0.15 0.34 *FDOE did not release data about the economically disadvantaged students in 2001. (Florida Department of Education, 2013g)
58 Table 4 4 displays the percent age of students achieving level three and above on FCAT in reading, 2002 to 20 13 for the groups studied and students overall for all grade levels The FDOE states that achieving level three on the test signifies demonstrating satisfactory performance (Florida Department of Education, 2013f) Achievement levels are significant in understanding the communications provided to the public about system success because th e y are used to derive the grades of A, B, C, D, or F for schools and districts (Florida Department of Education, 2013i) Table 4 4. Percentage at level three and above, FCAT reading 2002 2013. Level three 2002 2003 2004 2005 African American 27.38 32.14 32.13 34.50 Econ. Dis. 33.25 36.00 38.88 39.75 Hispanic 37.13 39.75 42.00 44.50 White 59.63 62.75 63.50 64.63 All 39.34 42.66 44.13 45.84 Level three 2006 2007 2008 2009 African American 38.75 38.75 41.13 43.00 Econ. Dis. 43.38 43.75 46.25 49.13 Hispanic 49.63 49.63 51.88 54.38 White 67.25 68.50 70.88 72.13 All 49.75 50.16 52.53 54.66 Level three 2010 2011 2012 2013 African American 43.13 35.75 36.75 37.75 Econ. Dis. 50.63 43.88 44.75 45.75 Hispanic 57.00 50.88 51.75 52.75 White 72.13 67.88 68.25 68.63 All 55.72 49.59 50.38 51.22 2002 2013 African American 36.76 Econ. Dis. 42.95 Hispanic 48.44 White 67.18 All 48.83 (Florida Department of Education, 2013g)
59 School Grades and Student Achievement Tables 4 5 and 4 6 show that a greater percentage of schools are rated as satisfactory than the percentage of students rated as meeting standards for reading on FCAT in Florida. The tables achievement with the award of school grades. O ver the study period, the education system was able to cause an increase of less than three percentage points in the percent of elementary school students a ble to demonstrate meeting the standards in reading. Although only 5 9% of elementary school students achieved level three in reading on FCAT in 2013 over 82% of elementary schools received a school grade of A, B, or C Table 4 5 shows the percent of schools achieving the grades of A, B, or C, and therefore being considered at or above meeting minimum accountability performance s Bureau of School Improvement. This demonstrates the nature of the conclusion of the H arvard study reviewed earlier: the perception of school quality is not consistent with the performance of the students accurate, transparent tool for stakeholders to use to understand the learning progress of students in schools. Averaged over the years of 2002 2013 the state communicated that over 94% of elementary schools performed at the A, B, or C, levels of periodically by the FD OE. Less than 65% of elementary school studen ts achieved level three or above on FCAT in reading during that period. Averaged over the same period, almost 94% of mi ddle schools were awarded a school grade of A, B, or C, while the
60 percent of middle school students who achieved level three or above in reading on FCAT averages out to be below 57%. Over the period, 79 % of high schools were awarded the A, B, or C grades while less than 41% of high school students achieved a level three or higher in reading on FCAT. The percentage of schools with satisfacto ry accountability ratings of A, B, or C, is much greater that the percentage of students that have achievement at levels considered satisfactory when all student groups and grade levels are considered.
61 Table 4 5. School grades by school levels. % ABC Sch ools 2002 2003 2004 2005 2006 Elementary 89.59 95.61 95.59 94.12 97.37 Middle 94.86 95.97 91.48 92.44 98.66 High 82.60 81.40 72.63 70.26 79.84 All 49.09 50.35 50.21 50.34 51.64 % Change 2.56 0.27 0.25 2.59 % ABC Schools 2007 2008 2009 2010 2011 Elementary 95.20 95.65 97.43 94.53 94.09 Middle 89.71 95.87 97.01 95.46 94.84 High 66.33 78.28 67.65 85.36 93.64 All 50.45 51.74 51.89 52.14 52.46 % Change 2.31 2.56 0.29 0.47 0.62 % ABC Schools 2012 2013 Average 2002 13 Change 2002 13 Elementary 91.18 82.81 94.58 7.57 Middle 86.90 79.08 93.93 16.63 High 98.12 97.57 79.64 18.13 All 51.70 49.44 51.09 0.72 % Change 1.45 4.36 (Florida Department of Education, 2013f) Table 4 6. All 2002 2003 2004 2005 2006 Elementary 56.00 60.33 65.00 68.00 69.33 Middle 48.67 51.33 50.67 51.00 57.00 High School 32.00 33.00 33.00 34.00 36.00 All Students 47.25 50.13 51.63 53.13 56.38 2007 2008 2009 2010 2011 Elementary 69.67 69.67 72.00 71.00 58.00 Middle 58.00 60.33 62.33 63.33 56.33 High 37.50 42.00 42.00 43.50 51.50 All Students 57.25 59.25 60.88 61.25 55.75 2012 2013 Average 2002 13 Change 2002 13 Elementary 59.67 59.00 64.81 5.36 Middle 56.67 57.33 56.08 17.81 High 51.00 53.50 40.75 67.19 All Students 56.38 57.00 55.52 20.63 (Florida Department of Education, 2013g)
62 Research Question One Research question one required determination if the FCAT/FCAT 2.0 reading scores for all students overall indicate d significant learning progress from 2002 to 2013. The percentage of students scoring at level three and above on FCAT in reading for each year was compared from 2002 to 2013 and for each schoo l year in between. So, 2002 was compared to 2013, 2002 was compared to 2003; 2003 was compared to 2004, etc., to determine the statistical significance of the differences across the years and each pair of years within the time period. Statistical analysis was used to determine the significance of the differences of the scores. Where a statistically significant difference was determined learning progress wa s determined to be significant. Where the difference wa s not statistically significant, the learning progress was determined not to be significant. T test procedures were used for this process because the independent variable consists of only two categories. This step compared the scores of one group of students over two time periods in each proc edure. Paired t tests wer e selected because the units of analysis are the same. Statistical significance was set at the standard criterion for assessing significance in a t test, .05. One tailed tests were used for the directional test. The degree of freed om for comparing the eight grade levels was t (7). The percentage of students that achieved at least a level three changed significantly across time, t (7) = 4.26, p = .004. The percentage of students who achieved at least a level three in 2002 ( M = 46.00, SD = 9.47) was significantly lower than the percentage of students in 2013 ( M = 57.00, SD = 2.62). Several paired t test procedures were conducted to determine whether there was a change in the percentage of students who achieved at least a level three fro m year to
63 year. The f indings in Table 4 8 reveal the percentage of students who achieved at least a level three increased significantly from 2002 to 2003, t (7) = 4.71, p = .00 1 The percentage also increased significantly 2005 to 2006, t (7) = 1.96, p = 045 and 2008 to 2009, t (7) = 2.30, p = .022. The students progressed significantly from 2002 to 2013 in the ability demonstrating reading skills at or above grade level on FCAT, but, considering the time span, this would be expected. Considering that state accountability system is based on statute, p olicy, and procedures designed to support annual improvement in schools it would also be expected that teaching and learning would advance each year for all students in the state. However, this analysis reveals that in most years of the study, the percenta ge of students able to demonstrate the ability to read on or above grade level on FCAT advanced only one or two percentage points. Significant progress was not made each year. Ultimately, by the end of the twelve year study, over 40% of students are unable to read on grade level in Florida schools. Research Question One Findings The findings in T able 4 7 indicate the percentage of students who scored at satisfactory levels in reading on FCAT progressed significantly over the span of the study, 2002 to 2013. However, T able 4 8 shows that significant progress did not occur every year. The data shows inconsistent progress of all students overall The findings fail to reject the null hypothesis for question one. Table 4 9 summarizes the findings Table 4 7. All 2002 M 2002 SD 2013 M 2013 SD t Sig. 2002 vs. 2013 47.25 10.46 57.00 2.62 3.28 .006
64 Table 4 M SD M SD t Sig. 2002 vs. 2003 47.25 10.46 50.13 11.54 4.81 0.001 2003 vs. 2004 50.13 11.54 51.63 13.87 1.06 0.162 2004 vs. 2005 51.63 13.87 53.13 14.68 1.47 0.092 2005 vs. 2006 53.13 14.68 56.38 15.13 1.96 0.045 2006 vs. 2007 56.38 15.13 57.25 14.14 0.73 0.240 2007 vs. 2008 57.25 14.14 59.25 12.26 1.82 0.506 2008 vs. 2009 59.25 12.26 60.88 13.37 2.30 0.022 2009 vs. 2010 60.88 13.37 61.25 12.42 0.70 0.252 2010 vs. 2011 61.25 12.42 55.75 3.20 1.63 0.735 2011 vs. 2012 55.75 3.20 56.38 4.10 0.92 0.194 2012 vs. 2013 56.38 4.10 57.00 2.62 0.92 0.194 Table 4 9. Research question one summary of findings Year In crease 2002 to 2013 Significant increase 2002 to 2003 Significant increase 2003 to 2004 Not a significant increase 2004 to 2005 Not a significant increase 2005 to 2006 Significant increase 2006 to 2007 Not a significant increase 2007 to 2008 Not a significant increase 2008 to 2009 Significant increase 2009 to 2010 Not a significant increase 2010 to 2011 Not a significant increase 2011 to 2012 Not a significant increase 2012 to 2013 Not a significant increase Research Question Two The second research question sought to determine whether there wo uld be any differences among the studied groups in th e achievement of a level three and above on
65 FCAT in reading from 2002 to 2013. To answer this research question, the percentage of students who achieved level three on FCAT in reading was obtained for students in each of the groups of the study: African American, economically disadvantaged, Hispanic and White. The percentages were compared for each group for each year from 2002 to 2013. Statistical analysis was conducted to determine if there were statistically significant differences in the percentages of students achieving level three and above on FCAT in reading among the student groups. O ne way analysis of variance (ANOVA) procedures were conducted. ANOVAs were conducted because the independent variable, student groups, consisted of four categories the four student groups of the study Where a statistically sign ificant difference was determined in the initial calculation, p ost hoc Tukey tests were used to determine which pairwise comparisons were contributing to the overall difference. The findings in Table 4 10 reveal that percentages of students who achieved at least a level three varied significantly across groups from 2002 to 2013. In 2002, White students had a significantly higher mean percentage than African American students ( p = .000), Hispanic students ( p = .002), and economically disadvantaged students ( p = .000). Similarly, in 2003, White students had a significantly higher mean percentage than African American students ( p = .000), Hispanic students ( p = .003), and economically disadvantaged students ( p = .000). In 2004, post hoc Tukey results indicate t hat White students had a significantly higher mean percentage than African American students ( p = .001), Hispanic students ( p = .030), and economically disadvantaged students ( p = .011). Note that although the differences were statistically significant, th e differences were not as large as the previous years. In 2005, White
66 students had a significantly higher mean percentage than African American students ( p = .003) and economically disadvantaged students ( p = .016); there were no significant differences be tween White and Hispanic students. Similarly, in 2006, White students had a significantly higher mean percentage than African American students ( p = .006) and economically disadvantaged students ( p = .027). As in 2005, there were no significant differences between White and Hispanic students. The same pattern of findings occurred in 2007: White students had a significantly higher mean percentage than African American students ( p = .002) and economically disadvantaged students ( p = .012); as in 2005 and 2006 there were no significant differences between White and Hispanic students. I n 2008, the pattern reverted to that of 2004: White students had a significantly higher mean percentage than African American students ( p = .001), Hispanic students ( p = .035), and economically disadvantaged students ( p = .004). In 2009, White students did not differ significan tly from Hispanic students. White students still had a significantly higher mean percentage than African American students ( p = .002) and economi cally disadvantaged students ( p = .017). Again, in 2010, White students did not differ significantly from Hispanic students. White students, however, still had a significantly higher mean percentage than African American students ( p = .001 ) and economicall y disadvantaged students ( p = .017). From 2011 to 2013, White students had a significantly higher mean percentage than African American students ( p = .000 during all three years), Hispanic students ( p = .000 during all three years), and economically disadv antaged students ( p = .000 during all three years). Improvements in the schools and in classroom instruction have not served all students in all student groups. The improvements in schools and in classroom instruction have failed to
67 address the learning ne eds of all students. The gap in learning to read at or above grade level is significant across the studied groups for each year of the study. Research Question Two Findings Table 4 1 1 shows a summary of the findings derived from the statistical analysis co mparing the percentage of students who achieved level three and above on FCAT in reading from 2002 to 2013. Statistical difference was found among the student groups each year of the study. The percentage of students who achieved level three was significan tly higher than African American students and economically disadvantaged students each year of the study. The percentage of student s who achieved level three was significantly higher than Hispanic students in all years of the study except 2005, 2006, 2007, 2009 and 2010. The data shows significant difference s in achievement among groups for most years of the study and rejects the null hypothesis.
68 Table 4 10 FCAT reading by student groups. % Level three African American Hispanic Econ. Dis. White Year M SD M SD M SD M SD F Sig. 2002 27.38 10.28 37.13 10.52 33.25 11.63 59.63 10.89 13.53 0 2003 29.88 11.45 39.75 11.42 36.00 11.96 62.75 11.90 12.08 0 2004 32.13 14.41 42.00 14.68 38.88 15.25 63.50 13.81 6.95 0.001 2005 34.50 15.79 44.50 15.73 39.75 16.03 64.63 14.33 5.79 0.003 2006 38.75 16.47 49.63 15.69 43.38 16.67 67.25 14.64 4.94 0.007 2007 38.75 15.20 49.63 14.66 43.75 15.52 68.50 13.67 6.20 0.002 2008 41.13 13.89 51.88 12.99 46.25 13.99 70.88 11.48 7.84 0.001 2009 43.00 15.19 54.38 14.01 49.13 15.25 72.13 13.09 6.06 0.003 2010 43.13 14.47 57.00 13.15 50.63 14.20 72.13 11.61 6.76 0.001 2011 35.75 3.96 50.88 3.68 43.88 3.91 67.88 3.31 107.77 0 2012 36.75 4.77 51.75 4.59 44.75 5.01 68.25 3.45 71.06 0 2013 37.75 3.92 52.75 3.01 45.75 3.37 68.63 2.62 129.29 0
69 Table 4 11. Research question two summary of findings Year African American Econ. Dis. Hispanic White 2002 Significantly lower Significantly lower Significantly lower Significantly higher 2003 Significantly lower Significantly lower Significantly lower Significantly higher 2004 Significantly lower Significantly lower Significantly lower Significantly higher 2005 Significantly lower Significantly lower Not significantly different Significantly higher 2006 Significantly lower Significantly lower Not significantly different Significantly higher 2007 Significantly lower Significantly lower Not significantly different Significantly higher 2008 Significantly lower Significantly lower Significantly lower Significantly higher 2009 Significantly lower Significantly lower Not significantly different Significantly higher 2010 Significantly lower Significantly lower Not significantly different Significantly higher 2011 Significantly lower Significantly lower Significantly lower Significantly higher 2012 Significantly lower Significantly lower Significantly lower Significantly higher 2013 Significantly lower Significantly lower Significantly lower Significantly higher
70 Research Question Three Research question three required analysis of the learning progress indicated by the scores within each of the student groups studied: African American economically disadvantaged Hispanic and White. The reports of the percentage of students within each g roup who achieved a level three and above on FCAT in reading were obtained from the FDOE website. The percentage of students who achieved level three and above were compared from 2002 to 2013, and for each one year span within that period, for each student group. The comparisons were statistically analyzed to determine if significant progress occurred across the t ime period, and if significant progress occurred in each one year span within that period. So, the percentage of students achieving level three an d above in 2002 was compared to 2013, and 2002 was compared to 2003, 2003 was compared to 2004, 2004 was compared to 2005, etc., for each group. Paired t test procedures were used to compare percent of students scoring at or above level three on FCAT in re ading in each group, across the years, and within each year span, to determine if significant progress was made. The findi ngs in Table 4 1 3 indicate the percentage of African American students who achieved at least a level three changed across time, t (7) = 3.95, p = .003 The percentage of African American students who achieved at least a level three in 2002 ( M = 27.38, SD = 10.28) was significantly lower than the percentage of African American students in 2013 ( M = 37.75, SD = 3.92). Several paired t test procedures were conducted to determine whether there was a change in the percentage of African American students who achieved at least a level three from year to year. The fin dings in Table 4 1 3 reveal the percentage of African American students who achieved at least a level three increased significantly from 2002
71 to 2003, t (7) = 4.18, p = .00 2 and from 2004 to 2005, t (7) = 2.52, p = .0 20 The percentage also increased significantly from 2005 to 2006, t (7) = 2.05, p = 0.04 ; 2007 to 20 08, t (7) = 2.13, p = .035 ; 2008 to 2009, t (7) = 1.97, p = .045 ; 2010 to 2011, t (7) = 1.96 p = .045 ; and 2012 to 2013 t (7) = 2.16, p = .034. A paired t test procedure was conducted to determine whether there was a change in the percentage of Hispanic students who achieved at least a level three from 2002 to 2013. The findi ngs in Table 4 1 4 indicate the percentage of Hispanic students who achiev ed at least a level three changed across that time, t (7) = 4.99, p = .002. The percentage of Hispanic students who achieved at least a level three in 2002 ( M = 37.13, SD = 10.52) was significantly lower than the percentage of Hispanic students who achieve d at least a lev el three in 2013 ( M = 52.75, SD = 3.01). Several paired t test procedures were conducted to determine whether there was a change in the percentage of Hispanic students who achieved at least a level three from year to year. The findings in T able 4 14 show the percentage of Hispanic students who achieved at least a level three increased significantly from 2002 to 2003, t (7) = 5.27, p = .001; 2005 to 2006, t (7) = 2.68, p = .016 ; 2 007 to 2008, t (7) = 2.26, p = .029; 2008 to 2009, t (7) = 3.21, p = .007 ; and 2009 to 2010, t (7) = 4.93, p = .001 A paired t test procedure was conducted to determine whether there was a change in the percentage of economically disadvantaged students who achieved at least a level three from 2002 to 2013. The findings in Table 4 1 5 indicate the percentage of economically disadvantaged students who achieved at least a level three changed across time, t (7) = 3.95, p = .003 The percentage of economically disadvantaged students who achieved at least a level three in 2002 ( M = 33.25, SD =
72 11.63) was significantly lower than the percentage of economically disadvantaged students in 2013 ( M = 45.75, SD = 3.37). Several paired t test procedures were conducted to determine whether there was a change in the percenta ge of economically disadvantaged students who achieved at least a level three from year to year. The findings in Table 4 1 5 reveal the percentage of economically disadvantaged students who achieved at least a level three increased significantly from 2002 to 2003, t (7) = 4.92, p = .00 1 ; 2007 to 2008, t (7) = 2.50, p = .0 20 ; 2008 to 2009, t (7) = 3.02, p = .0 08 ; and 2009 to 2010, t (7) = 2.12, p = .036 A paired t test procedure was conducted to determine whether there was a change in the percentage of White students who achieved at least a level three from 2002 to 2013. The findings shown in Table 4 1 6 indicate the percentage of White students who achieved at least a level three changed across time, t (7) = 2.86, p = .0 12 The percentage of White students who achieved at least a level three in 2002 ( M = 59.63, SD = 10.89) was significantly lower than the percentage of White students in 2013 ( M = 68.63, SD = 2.62). Several paired t test procedures were conducted to determine whether there was a change in the percentage of White students who achieved at least a level three in from year to year. The fi ndings in Table 4 1 6 reveal the percentage of White students who ach ieved at least a level three increased significantly from 2002 to 2003, t(7) = 4.0 8 p = .00 3 and from 2007 to 2008, t (7) = 2.19, p = .033. Significant improvements in the percentage of students able to demonstrate the ability to read on grade level on F CAT occurred in some years of the study for some groups H owever the gains were minimal and ultimately led to less than half of the
73 students being able to read on grade level in two of the groups studied Less than 38% of students in the African American g roup and less than 46% of the students in the economically disadvantaged group were able to demonstrate grade level capacity to read on FCAT at the end of the study. The economically disadvantaged group includes students from all racial, ethnic, and learne r profile (ESE, ELL) groups. This indicates the educational system did not sufficiently meet the learning needs of the majority of African American and economically disadvantaged students in reading. Research Question Three Findings In summary, all student groups showed significant progress in raising the percentage of students who achieved level three and above on FCAT in reading across the span of the study period, from 2002 to 2013. However, not all groups showed significant progress each year in increas ing that percentage. The null hypothesis was not rejected by the data analysis. Table 4 12 shows a summary of the findings. The cells with the word significant indicate that there was significant progress in raising the percentage of students who achieved level three and above on FCAT in reading for the time period indicated in each row by the student groups of each column.
74 Table 4 12. Research question three summary of findings Comparison ye ar s All students African American Econ. Dis. Hispanic White 02 to 13 Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant Significant African American students showed significant progress in improving the percentage of students achieving satisfactory performance levels over the entire study period, from 2002 to 2013, and from 2002 to 2003, 2004 to 2005, 2005 to 2006, 2007 to 2008, 2008 to 2009, and 2010 to 2011. Economically disadvantaged students showed significant progress in improving the percentage of students achieving satisfactory performance levels over the entire study period, from 2002 to 2013, and from 2002 to 2003, from 2007 t o 2008, from 2008 to 2009, and from 2009 to 2010. Hispanic students showed significant progress in improving the percentage of students achieving satisfactory performance levels over the entire study period, from 2002 to 2013, and from 2002 to 2003, from 2 005 to 2006, from 2007 to 2008, from 2008 to 2009, and from 2009 to 2010. White students showed significant progress in improving the percentage of students achieving satisfactory performance levels over the entire study period, and
75 from 2002 to 2003, and from 2007 to 2008. The data shows that the group with the higher percentage of students achieving satisfactory performance levels in reading on FCAT, White, improved that percentage less each year than the other groups. While significant improvements in th e percentage of students with satisfactory achievement in reading on FCAT occurred for some groups across the study, the improvement was minimal. Most importantly, less than half of the students in the African American and the economically disadvantaged gr oups demonstrated capacity to read on grade level. The school system did not sufficiently meet the learning needs in reading of the majority of students in those groups.
76 Table 4 1 3 African American students FCAT reading 2 002 to 2013 t Sig. 2002 vs. 2013 3.95 0.00 3 2002 vs. 2003 4.18 0.002 2003 vs. 2004 1.79 0.090 2004 vs. 2005 2.52 0.020 2005 vs. 2006 2.05 0.040 2006 vs. 2007 0 0.499 2007 vs. 2008 2.13 0.035 2008 vs. 2009 1.97 0.045 2009 vs. 2010 0.15 0.444 2010 vs. 2011 1.96 0.045 2011 vs. 2012 1.6 0 0.775 2012 vs. 2013 2.16 0.034 Table 4 1 4 Hispanic s tudents FCAT reading 2002 to 2013 three t Sig. 2002 vs. 2013 4.18 0.004 2002 vs. 2003 5.27 0.001 2003 vs. 2004 1.35 0.219 2004 vs. 2005 1.83 0.109 2005 vs. 2006 2.68 0.032 2006 vs. 2007 0 0.999 2007 vs. 2008 2.26 0.058 2008 vs. 2009 3.21 0.015 2009 vs. 2010 4.93 0.002 2010 vs. 2011 1.68 0.137 2011 vs. 2012 0.98 0.361 2012 vs. 2013 1.18 0.275
77 Table 4 1 5 Economically Table 4 1 6 White s tudents FCAT reading 2002 to 2013. t Sig. 2002 vs. 2013 2.86 0.012 2002 vs. 2003 4.08 0.003 2003 vs. 2004 0.61 0. 280 2004 vs. 2005 1.35 0.110 2005 vs. 2006 1.63 0.124 2006 vs. 2007 1.21 0.133 2007 vs. 2008 2.19 0.033 2008 vs. 2009 1.45 0. 099 2009 vs. 2010 0 0.499 2010 vs. 2011 1.4 0 0.102 2011 vs. 2012 0.57 0.292 2012 vs. 2013 0.48 0.322 t Sig. 2002 vs. 2013 3.95 0.003 2002 vs. 2003 4.92 0.001 2003 vs. 2004 1.88 0.051 2004 vs. 2005 0.96 0.185 2005 vs. 2006 1.80 0.575 2006 vs. 2007 0.29 0.389 2007 vs. 2008 2.50 0.020 2008 vs. 2009 3.02 0.008 2009 vs. 2010 2.12 0.036 2010 vs. 2011 1.81 0.565 2011 vs. 2012 1.11 0.152 2012 vs. 2013 1.32 0.113
78 Summary The statistical analysis findings show inconsistency in the progress of the percentage meeting standards in reading by students overall, comparatively among racial and socio economic groups, and by each of the student groups studied, from 2002 to 2013. For all students combined the percentage of students who achieved at least a level three in reading in 20 13 was significantly high er than the percentage of students in 20 02 However, si gnificant change did not occur every year. In comparing achievement at level three in reading for each group each year, a significantly higher percentage of White students achieved level 3 and above than most student groups each year since 2002. In considering the progress made in closing the achievement gap, in all years of the study, one group had a significantly higher percentage of students at achievement level three and above than at least two other groups of students. Although the percentage o f Black, Hispanic, and economically disadvantaged students who achieved at least a level three in reading in 2013 was significantly higher than in 2002, the increase in the percentage of students who achieved at least a level three in reading from one scho ol year to the next was not consistent among the groups. The data analysi s reveals that the educational system in the state of Florida has not closed the achievement gap in reading nor met learner needs sufficiently to cause consistent progress in the per centage of students who can meet the reading standards over the period of the study. This is not consistent with school accountability ratings and school grade reports. The percentage of schools with A, B, or C ratings in 2013 was over 57% while the perce ntage of students demonstrating grade level capacity in reading on FCAT was less than 50% for some groups The 2013 School Grade report
79 published by the state does not mention the performance of the students by groups (Florida Depar tment of Education, 2014f) The FDOE d id not inform stakeholders and serve improvement efforts with transparency and accuracy about the limitations in achievement progress in reading for all student groups over the study period.
80 CHAPTER 5 CONCLUSION Summary of Results This study analyzed the reading achievement of student groups in Florida from 2002 to 2013 to determine the significance of the learning progress of all students, the reduction in the achievement gap between student groups and the specific progress of groups of students. Inconsistent progress was evident for all students and for all student groups and the gaps in achievement among groups was not significantly reduced. The first research question asked if there was significant progress for all students across the years of the study. Statistical analysis found that the learning progress of all students has been inconsistent and significant progress has not occurred in mo st school years of t he study. The second research question asked if there was a reduction in the gaps of achievement among the student groups studied : African American, economically disadvantaged, Hispanic, and White. Statistical analysis revealed that t he reduction in the gaps of achievement between student groups has not been accomplished. The scores show significant differences among student groups has persisted each school year. The school system has not advanced in meeting learner needs and causing lea rning to occur with equity for all student groups for the span of the study. The third question asked if each of the student groups showed significant learning progress over the period of the study. The statistical analysis found that no student groups hav e made significant learning progress e very year of the study. In general, the study considered if the FDOE fully disclosed the achievement progress of the students to the public with transparency Through review of the data and
81 school grade reports, the s tudy found the FDOE has not used the data to inform improvements effectively and does not publish reports that fully and accurately disclose achievement data to the public. The FDOE committed to the steps and elements of the Data Quality Campaign (DQC) as part of the commitment to meet federal standards for state educational systems (Data Quality Campaign, 2013a) The DQC of September 2013 promotes the primary focus of their work as agents of change romotes the development of state longitudinal data systems that not only collect data but also transform them into actionable information that answers critical questions (Data Quality Campaign, 2013d) The FDOE uses FCAT achievement data as the primary foundation for communicating educational system success to the public though the school grade program. The communication of schools grades to the pub lic is made without explanation of the FCAT performance of all s tudents by racial, socio economic and learner profile (ESE, ELL) group s. Beyond that, t he criteria for school grades have be en State Board of Education (SBOE) For example, m ost recently, i n 2012 and again in 2013 the S BOE changed the sc hool grade criteria by adding a temporary measure, referred to as a safety net, preventing schools from falling more than one grade level in one year to avoid excessive numb ers of schools labeled as failing This practice is inconsistent with the DQC Action 7: Create reports using longitudinal statistics to guide systemwide improvement efforts. Create reports that include longitudinal statistics on school systems and groups of students to guide school district and state level improvem ent efforts. All stakeholders need information on school, district, and state performance to gauge progress and make decisions to support continuous improvement at all education levels. (Data Quality Campaign, 2013b)
82 T he longitudinal data show a decrease in enrollment of White students, the demographic group with an average of almost 70% of students scoring at or above level three over the years of the study. An average of less than 55% of Hispanic students achieve d the sa me level of proficiency over the study years, with an increase of almost 70% in enrollment An average of l ess than 50% of economically disadvantaged students scored at or above achievement level three over the study years, while the same group represents (Florida Department of Education, 2013e) However, the FDOE Media packets communicating the school grades and promoting the success of the educational system to the public do n ot attempt to explain the deficiency in student success among those groups (Florida Department of Education, 2013j) The DQC criteria for improving schools are to publicize longitudinal success educating all student groups with specificity and transparency to e m power stakeholders to collaborate in system reform (Data Quality Campaign, 2013d) T he Florida SBOE publications have not identif ied student achievement issues in s chool grade reports The SBOE has changed the school grading criteria with t he stated justification of prevent ing school failure labels f rom being applied where students do not achieve proficiency in the standards of instruction (Florida Department of Education, 2011b) The result is reduced transparency of disclosure about achievement where student learning needs are unmet. This study verifies achievement among Florida students has not consistently increased for students overall or among many group s, even though state communications consistently report school improvement among schools. Interestingly,
83 t he need for transparency and accuracy in reporting school improvement was emphasized by the former Commissioner of Education, Tony Bennett, during the S BOE meeting in which he proposed the change in sc safety net to prevent school grades from dropping more than one letter in one year. Mr. and how that gets reporte (Smiley & Vasquez, 2013) The statistical analysis in this study shows the educational system has not resulted in the levels of success for African American or economically disadvant aged student s as it has for the other group s studied However, the diminished progress for some student groups is not included in the report of school grades t hat communicate school performance to the public. As the former commissioner emphasizes, how data (Smiley & Vasquez, 2013) is important for the public being able to understand school performance. As noted i n the Harvard study included in the literature review, the public does not fully understand schoo l performance in Florida. The study is limited to evaluating the data concerning the subjects of the study that is ratings. It is not the purpose of the study to investigate the validity of the calculations from which the data is derived. The purpose is focused on the clarification of meaning of the student performance data that is made available to the public that is aligned with the research questions. The study is limited to the evaluation of the meaning of the data disclosed by the state. The study is also limited by the changes in school grades, the changes in the cut scores used to determine achievement levels, and the lack of
84 transparency with which raw FCAT scores are converted to scale scores and developmental scale scores. Although these issues did not directly affect the study result s in response to the research questions, these limitations contribute to the problems in understanding student achievement and school, district, and program performance in Florida. Implications The accountability requirements in the educational systems ha v e been increasingly calling for transparency in reporting progress and utility in acquiring and communicating student and system performance data. The implications of the lack of consistency in student performance and publicized reports of school and distr ict performance are that Florida is not using data effectively to support improvement in the educational systems that It is commonly understood that a problem cannot be solved until it is acknowledged. Upon acknowledgement o f a problem, there are a myriad of strategic approaches and steps to be planned and implemented by concerned stakeholders in an underperforming system services so more students demonstrate proficie ncy meeting reading standards cannot start until the problem of underachievement and inequity in achievement are acknowledged studied and addressed The state of Florida, along with all other states, committed to the DQC ten s tate a ctions Florida has not fulfilled all of the ten state actions The two actions that Florida has not fulfilled are essential to school improvement processes. Table 5 1 shows the
85 Table 5 1. DQC state actions Action Reported Sta tus 1 Link state K 12 data systems with early learning, postsecondary education, workforce, social services and other critical agencies Done 2 Create stable, sustained support for robust state longitudinal data systems Done 3 Develop governance structures to guide data collection, sharing and use Not done 4 Build state data repositories (e.g., data warehouses) that integrate student, staff, financial and facility data Done 5 Implement systems to provide all stakeholders with timely access to the information they need while protecting student privacy Not done 6 Create progress reports with individual student data that provide information educators, parents and students can use to improve student performance. Done 7 Create reports that include longitudinal statistics on school systems and groups of students to guide school district and state level improvement efforts. Done 8 Develop a purposeful research agenda and collaborate with universities, researchers and intermediary groups to explore the data for useful information. Done 9 Implement policies and promote practices, including professional development and credentialing, to ensure educators know how to access and use data appropriately. Done 10 Promote strategies to raise awareness of available data and ensure that all key stakeholders, including state policymakers, know how to access, analyze and use the information Done (Data Quality Campaign, 2014a; Data Quality Campaign, 2014b) The FDOE has not fulfilled action three and action five. Action three is important because data governance is vital to the management of data, so that school system organizations and agencies can define the roles and responsibilities essential to the process o f institutionaliz ing data quality and effective use of data to improve the system Without a data governance strategy, there is no clear ownership of the data, no clear business proc esses for collecting and reporting data, and no accountab (Data
86 Quality Campaign, 2013c) It is vital that the FDOE fulfill the commitment t o an effective data governance. Action five calls for state educational departments to implement data systems that give all stakeholders timely access to all of the student data needed to inform decision making. It is important that p arents, teachers, an d appropriate stakeholders can access student level longitudin al data so that it may be used to evaluate learning and educational systems, determine needs, and monitor progress in improvement efforts (Data Quality Campaign, 2013c) Th is study focused on the problem of understanding the ac hievement data that shows the underachievement and inequity in reading achievement by students in and understanding of the problem. The reduced disclosure and acknowledgem ent of the problem impedes the ability to understand and attempt to resolve the underlying causes of the deficiencies in the educational system. The progress of the system has not been s ignificant in each of the past twelve years It is concluded that the efficacy of the system will not be sufficient without major policy and practice changes. A publicly transparent and accurate method of student achievement data acquisition, reporting, and analysis, with highly accurate disclosure of student learning progre ss, is needed in Florida. Statute, policy, and regularly practiced procedures to implementation of effective improvement plans and procedures. The commitment to the ten state actions of the DQC has not been fulfilled in practice and the effect is evident in the student achievement data when analyzed. The
87 and teachers have resulted in inco nsistent progress in student achievement in reading for over a decade. System improvements in the acquisition and implementation of effective student achievement assessment and reporting measures are needed to support effective improvements. Recommendation s This study focused on the data showing student achievement in reading in m eeting the learning standards with clarity as to student groups and progress over time The inconsistency in progress of all students and persistent deficits in achievement by some student groups demonstrates why a dditional research and action planning and implementation is needed to improve measures to understand and improve learning progress in the system. A committed effort of educational leaders to reform education accountability policy and learning progress monitoring practices is needed. There are a variety of statute, policy, and agency resources needed to support and facilitate research of the problem and implementation of steps to change accountability policy, evaluate school and student performance responsibly, and report and effectively use longitudinal data to improve schools Since the state has committed to the standards of the DQC, it would be logically advantageous to use the federal resource for the research and policy reform concepts needed. The DQC provides resources to support the s tatute and policy reforms need ed in Florida It is recommended that the state organize and engage leadership in the FDOE systems to study and develop strategic plans to implement the policy recommendations and practice plans provided by the DQC. The DQC provides planning tools, policy support reports, and publications providing implementation
88 guidance that are research base programs to use data to support improvement in schools. DQC resources that are recommended for the state to use are listed in Appendix C Most importantly, the state needs to make a commitment to reforms that alter current practices and lead to the use of student achievement data to inform improvement. The importance of the commitment to reform is superior to all other actions because the state has resources, systems and prior commitments that c ould have l ed to improvements but resulted in the limited progress documented in this study. The unity of politicians, legislators, educational leaders and school staff members in making a commitment to reformed improvement efforts, strategically planning their imple mentation and establishing ongoing progress monitoring and intervention systems is vital to overcome the limited learning progress documented in the study. This study analyzed the percentage of students demonstrating grade level reading skills on FCAT, f rom 2002 to 2013. The analysis focused on all students as well as the students in the African American, economically disadvantaged, Hispanic, and White groups The study found that the progress of student s in all groups was not consistently significant, and the achievement gap between the groups was not significantly diminished. The progress program, the School Grade reports and the FCAT Score reports was not supported through statistica l analysis of the percentage of students achieving level three or above on FCAT reading The study found, in concurrence with the l iterature that the progress promised by NCLB and the test based accountability systems used over the years of NCLB did not result in the intended advancements
89 APPENDIX A UNIVERSITY OF FLORIDA INSTITUTIONAL REVIEW BOARD (UFIRB) APPROVAL
90 APPENDIX B ADDITIONAL DATA CHARTS Table B 1 Number and percentage of students tested Number tested 2002 2003 2004 2005 2006 2007 African American 357,502 362,469 363,352 363,991 359,725 354,511 Hispanic 303,648 323,656 340,947 358,304 372,249 381,595 Econ. Dis. 701,361 720,572 774,982 752,495 737,151 733,034 White 781,188 783,520 784,030 779,581 763,126 742,230 All 1,498,688 1,533,913 1,559,082 1,580,536 1,582,232 1,571,818 Number tested 2008 2009 2010 2011 2012 2013 African American 352,127 350,214 350,113 353,129 352,991 352,415 Hispanic 386,813 393,101 414,203 443,894 454,338 466,627 Econ. Dis. 754,239 803,361 873,660 906,218 933,227 946,833 White 723,516 711,615 700,322 678,717 668,774 660,085 All 1,560,826 1,582,232 1,571,818 1,570,656 1,573,009 1,578,282 % tested 2002 2003 2004 2005 2006 2007 African American 23.85 23.63 23.31 23.03 22.74 22.55 Hispanic 20.26 21.10 21.87 22.67 23.53 24.28 Econ. Dis. 46.80 46.98 49.71 47.61 46.59 46.64 White 52.12 51.08 50.29 49.32 48.23 47.22 % tested 2008 2009 2010 2011 2012 2013 African American 22.56 22.13 22.27 22.48 22.44 22.33 Hispanic 24.78 24.84 26.35 28.26 28.88 29.57 Econ. Dis. 48.32 50.77 55.58 57.70 59.33 59.99 White 46.35 44.98 44.55 43.21 42.52 41.82 (Florida Department of Education, 2013g)
91 Table B 2 FCAT r eading by group s: Changes 2003 to 2013. % Level three 2003 Change 2002 3 2004 Change 2003 4 2005 Change 2004 5 2006 Change 2005 6 African American 32.14 17.42 32.13 0.06 34.50 7.39 38.75 12.32 Hispanic 39.75 7.07 42.00 5.66 44.50 5.95 49.63 11.52 Econ. Dis. 36.00 8.27 38.88 7.99 39.75 2.25 43.38 9.12 White 62.75 5.24 63.50 1.20 64.63 1.77 67.25 4.06 All 42.66 8.43 44.13 3.43 45.84 3.90 49.75 8.52 % Level three 2007 Change 2006 7 2008 Change 2007 8 2009 Change 2008 9 2010 Change 2009 10 African American 38.75 0.00 41.13 6.13 43.00 4.56 43.13 0.29 Hispanic 49.63 0.00 51.88 4.53 54.38 4.82 57.00 4.83 Econ. Dis. 43.75 0.86 46.25 5.71 49.13 6.22 50.63 3.05 White 68.50 1.86 70.88 3.47 72.13 1.76 72.13 0.00 All 50.16 0.82 52.53 4.74 54.66 4.05 55.72 1.94 Level three 2011 Change 2010 11 2012 Change 2011 12 2013 Change 2012 13 Average 2002 13 African American 35.75 17.10 36.75 2.80 37.75 2.72 36.76 Hispanic 50.88 10.75 51.75 1.72 52.75 1.93 48.44 Econ. Dis. 43.88 13.33 44.75 1.99 45.75 2.23 42.95 White 67.88 5.89 68.25 0.55 68.63 0.55 67.18 All 49.59 10.99 50.38 1.58 51.22 1.67 48.83 (Florida Department of Education, 2013d)
92 Table B 3 FCAT reading by g rad e l evels : Changes 2003 to 2013. All Students % Level three 2002 2003 2004 2005 2006 2007 Elementary 56.00 60.33 65.00 68.00 69.33 69.67 Middle 48.67 51.33 50.67 51.00 57.00 58.00 High School 32.00 33.00 33.00 34.00 36.00 37.50 Level three All Grades 47.25 50.13 51.63 53.13 56.38 57.25 All Students % Level three 2008 2009 2010 2011 2012 2013 Elementary 69.67 72.00 71.00 58.00 59.67 59.00 Middle 60.33 62.33 63.33 56.33 56.67 57.33 High School 42.00 42.00 43.50 51.50 51.00 53.50 Level three All Grades 59.25 60.88 61.25 55.75 56.38 57.00 (Florida Department of Education, 2013g)
93 APPENDIX C DQC RESOURCES Why Data Matter in ESEA Reauthorization: Recommendations to Ensure Data Are Used to Improve Student Achievement Key Elem ents for Strengthening State Laws and Policies Pertaining to Student Data Use, Privacy, and Security: Guidance for State Policymakers Roadmap for High School Feedback Reports Roadmap for Teacher Access to Student Level Longitudinal Data Measuring the Ed ucation Pipeline: Common Data Elements Indicating Readiness, Transition and Success The Power of State Data: Message Map Pivotal Role of Policymakers as Leaders of P 20/Workforce Data Governance Recommendations for Statewide Longitudinal Data System Use in the Reauthorization of the Elementary and Secondary Education Act Supporting Education Policy and Practice through Common Data Standards Investing in Educator Data Literacy Improves Student Ac hievement: Evidence of Impact Supporting Education Policy and Practice through Common Data Standards Using Data to Improve Teacher Effectiveness: A Primer for State Policymakers Using Data to Increase College and Career Readiness: A Primer for State Pol icymakers Preparing Every Citizen for the Knowledge Economy: A Primer on Using Early Childhood, K 12, Postsecondary and Workforce Data From Compliance to Service: Evolving the State Role to Support District Data Efforts to Improve Student Achievement Da ta: The Missing Piece to Improving Student Achievement Leveraging the Power of State Longitudinal Data Systems: Building Capacity to Turn Data into Useful Information (Data Quality Campaign, 2014b)
94 Figure 4 1 Comparison of school grades and FCAT reading averages by group s. (Florida Department of Education, 2013g) 47% 50% 52% 53% 57% 58% 60% 61% 62% 56% 57% 57% 47% 50% 52% 53% 56% 57% 59% 61% 61% 56% 56% 57% 27% 32% 32% 35% 39% 39% 41% 43% 43% 36% 37% 38% 33% 36% 39% 40% 43% 44% 46% 49% 51% 44% 45% 46% 37% 40% 42% 45% 50% 50% 52% 54% 57% 51% 52% 53% 60% 63% 64% 65% 67% 69% 71% 72% 72% 68% 68% 69% 0% 10% 20% 30% 40% 50% 60% 70% 80% 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 School Grades All Students FCAT All Students FCAT Black Students FCAT Econ. Dis. Students FCAT Hispanic Students FCAT White Students FCAT 2.0
95 REFERENCES Aimer, A., & Berliner, D. C. (2003). The effects of high stakes testing on student motivation and learning. Educational Leadership 60 no5 F 32 8. Betebenner, D. W. (2009). Norm and criterion referenced student growth. Educational Measurement: Issues and Practice 42 51. Betebenner, D. W., & Linn, R. L. (2009). Growth in student achievement: Issues of measurement, longitudinal data analysis, and acc ountability. Exploratory Seminar: Measurement Challenges within the Race to the Top Agenda (pp. 1 23). Educational Testing Service. Bracey, G. W. (2007). Another way to game the system. Phi Delta Kappan 412 413. Braun, H., Chapman, L., & Vezzu, S. (2010). The Black White achievement gap revisited. Education Policy Analysis Archives, 18(21) 1 96. Chingos, M. M., Henderson, M., & West, M. R. (2010, June 6). Citizen perceptions of government service quality: Evidence from public schools. Retrieved from DASH: Digital Access to Scolarsh at Harvard: http://www.hks.harvard.edu/pepg/PDF/Papers/PEPG10 16_Chingos Henderson West.pdf College Board. (2012). College Board. Retrieved January 12, 2013, from SAT: http://sat.collegeboard.org/home Daggett, W. R. (2005, June 12). Successful schools: From research to action plans. Retrieved December 12, 2013, from International Center for Leadership in Education: http://www.ode.state.or.us/initiatives/elearning/nasdse/characteristicshighschools .doc Data Quality Campaign. (2013a November 24). Who we are Retrieved December 12, 2013, from Data Quality Campaign: http://www.dataqualitycampaign.org/who we are Data Quality Campaign. (2013b). Creating a longitudinal data system: Using data to improve student achievement. Retrieved Feb ruary 12, 2012, from http://www.dataqualitycampaign.org/find resources/creating a longitudinal data system Data Quality Campaign. (2013c, September 12). State analysis by state action Retrieved December 12, 2013, from Data Quality Campaign: http://www.dat aqualitycampaign.org/your states progress/10 state actions?action=ten
96 Data Quality Campaign. (2013d, September 24). DQC fact sheet. Retrieved December 12, 2013, from Data Quality Campaign: http://www.dataqualitycampaign.org/files/DQC%20Fact%20Sheet%20Oct%2 020 13.pdf Data Quality Campaign. (2014a, March 24). State analysis Florida. Retrieved from Data Quality Campaign: http://www.dataqualitycampaign.org/files/pdf/stateprofiles/FL.pdf Data Quality Campaign. (2014b, March 20). Resources. Retrieved from Data Qua lity Campaign: 2014 Dennis, D. V. (2008). Are assessment data really driving middle school reading instruction? What we can learn from one student's experience. Journal of Adolescent & Adult Literacy, 51(7) 578 587. doi:10.1598/JAAL.51.7.5 Duffy, M., Gior dano, V., Farrell, J., Paneque, O., & Crump, G. (2008). No Child Left Behind: Values and research issues in high stakes assessments. Counseling & Values, 53(1) 53 66. Florida Department of Education. (2011b). State board of education. Tallahassee, FL: Flo rida Department of Education. Retrieved from http://www.fldoe.org/board/meetings/2011_12_19/109422.pdf Florida Department of Education. (2013a, November 24). Accountability, Research, and Measurement Retrieved December 5, 2013, from Florida Department of Education: http://www.fldoe.org/arm Florida Department of Education. (2013b, September 20). Common core state standards Retrieved December 5, 2013, from Florida Department of Education: https://www.fldoe.org/schools/ccc.asp Florida Department of Education (2013c, September 20). FCAT 2.0 Bureau of k 12 assessment: FCAT 2.0 Retrieved December 12, 2013, from Florida Department of Education: http://fcat.fldoe.org/fcat2/ Florida Department of Education. (2013d, September 20). FCAT Bureau of k 12 assessment; F CAT Retrieved December 12, 2013, from Florida Department of Education: http://fcat.fldoe.org/fcat/ Florida Department of Education. (2013e, December 6). FCAT Demographics Retrieved December 6, 2013, from Florida Department of Education: https://app1.fldo e.org/fcatdemographics/ Florida Department of Education. (2013f, December 6). FCAT Media packet 2013. Retrieved December 6, 2013, from http://fcat.fldoe.org/mediapacket/2013/pdf/2013FCAT20_Media.pdf
97 Florida Department of Education. (2013g, December 6). FCA T Student performance results: Demographic report Retrieved December 6, 2013, from Florida Department of Education: http://app1.fldoe.org/FCATDemographics/ Florida Department of Education. (2013h, September 20). Florida comprehensive assessment test Retr ieved December 12, 2013, from Florida Department of Education: http://fcat.fldoe.org/resultsEOC/ Florida Department of Education. (2013i, September 20). Florida school grades. Retrieved December 12, 2013, from Florida Department of Education: http://school grades.fldoe.org/pdf/1213/AMO_TAP2013.pdf Florida Department of Education. (2013j, September 12). Florida school grades. Retrieved December 12, 2013, from Florida Department of Education: http://schoolgrades.fldoe.org/pdf/1213/SchoolGradesPressPacket.pdf F lorida Department of Education. (2013k, December 5). Independent Review of 2010 FCAT Results Retrieved December 12, 2013, from Florida Department of Education: http://www.fldoe.org/schoolgrades.asp Florida Department of Education. (2013l, August 12). Media room Retrieved December 12, 2013, from Florida Department of Education: http://www.fldoe.org/news/2013/2013_07_16.asp Florida Department of Education. (2013m, September 02). Office of assessment Retrieved December 12, 2013, from Florida Department of Education: http://www.fldoe.org/asp/ Florida Department of Education. (2013n, September 20). Office of assessment; Florida alternate assessment Retrieved December 12, 2013, from Florida Department of Education: http://www.fldoe.org/asp/altassessment.as p Florida Department of Education. (2013o). School grades model and historical background Retrieved December 12, 12, from http://www.fldoe.org/ppt/HistoryofSchoolGrades.ppt Florida Department of Education. (2014, March 19). Florida school grades Retrieve d from Florida Department of Education: http://schoolgrades.fldoe.org/index.asp Florida Department of Education. (2014b, March 19). The Florida Comprehensive Assessment Test (FCAT) Retrieved from Florida Department of Education: http://fcat.fldoe.org/fcat / Florida Department of Education. (2014c, March 19). School Recognition Program Retrieved from Florida Department of Education: http://www.fldoe.org/faq/default.asp?Dept=177&ID=613
98 Florida Department of Education. (2014d, March 19). Differentiated Accoun tability Retrieved from Florida Department of Education: http://www.fldoe.org/news/2008/2008_07_29/diffaccountimplem.pdf Florida Department of Education. (2014e, March 21). FCAT 2.0 Graduation Requirements Retrieved from Florida Department of Education: 2014 Florida Department of Education. (2014f, March 28). Florida school grades. Retrieved from Florida Department of Education: http://schoolgrades.fldoe.org/pdf/1213/SchoolGradesPressPacket.pdf Florida Senate. (2014, March 21). 2011 Statutes Retrieved fr om Florida Senate: http://www.flsenate.gov/laws/statutes/2011/1008.25 Haney, W. M. (2006, September 8 10). Evidence on education under NCLB (and how Florida boosted NAEP scores and reduced the race gap). Retrieved December 5, 2012, from Boston College: Res earch at Boston College: http://www.bc.edu/research/nbetpp/statements/nbr6.pdf Haretos, C. (2005). The No Child Left Behind Act of 2001: Is the definition of "Adequate Yearly Progress" adequate? Kennedy School Review 629 46. Huff, K., & Plake, B. (2010). Innovations in setting performance standards for k 12 test based Accountability. Measurement, 8(2/3) 130 144. doi:10.1080/15366367.2010.508690 Ladd, H. (2012). School accountability: To what ends and with what effects? Rome, Italy: Keyn ote address for Conference on Improving Accountability and Evaluation: Lesson from Around the World. Retrieved January 24, 2014, from http:////fds.duke.edu/db/attachment/2050 Lezotte, L. (2011, June 12). Effective schools. Retrieved December 12, 2013, from Effective Schools Research: http://effectiveschools.com/images/stories/gettingitdone_lezotte.pdf Linn, R. L. (2008). Methodological issues in achieving school accountability. Journal of Curriculum Studies, 40(6) 699 711. doi:10.1080/00220270802105729 Lin n, R. L. (2010). A new era of test based educational accountability. Measurement, 8(2/3) 145 149. doi:10.1080/15366367.2010.508692 Maleko, G., & Gawlik, M. (2011). No Child Left Behind: What we know and what we need to know. Education, 131(3) 600 624. Na tional Governor's Association. (2012, February 13). Issue brief: Using data to guide education state policy and practice. Retrieved December 12, 2013, from National Governor's Association Center for Best Practices: http://nga.org/cms/center
99 National Govern or's Association. (2013, August 12). About. Retrieved December 5, 2013, from National Governor's Association: http://www.nga.org/cms/about Nation's Report Card. (2014, January 24). Nation's Report Card Retrieved January 24, 2014, from About: http://nation sreportcard.gov Ryan, K., & Shepard, L. (2008). The future of test based educational accountability. New York: Routledge. Ryan, K., Gandha, T., & Ahn, J. (2013, November). School self evaluation and inspection for improving US schools? Retrieved December 1 2, 2013, from National education policy center: http://nepc.colorado.edu/files/pb inspectorate final.pdf Smiley, D., & Vasquez, M. (2013, July 16). Miami Herald schools. Retrieved December 12, 2013, from Miami Herald Digital Newspaper: http://www.miamihera ld.com/2013/07/16/3502744/state approves safety net for.html Smyth, T. (2008). Who is No Child Left Behind leaving behind? Clearing House, 81(3) 133 137. U.S. Department of Education. (2012b, June 10). Condition of education. Retrieved February 12, 2012, from Institute of Educational Sciences, National Center for Educational Statistics, U.S. Department of Education: http://nces.ed.gov/programs/coe/ United States Department of Education. (2010, December 6). Elementary and secondary education. Retrieved Dece mber 12, 2013, from United States Department of Education: http://www2.ed.gov/nclb/index/az/glossary.html?src=az#15 United States Department of Education. (2012a, September 20). National center for education statistics Retrieved December 2013, 2013, from Institute of Education Sciences: http://ies.ed.gov/aboutus/ United States Department of Education. (2012b, September 23). Race to the top fund Retrieved December 12, 2013, from United States Department of Education: http://www2.ed.gov/programs/racetotheto p/index.html Volante, L., & Jaafar, S. (2010). Assessment reform and the case for learning focused accountability. Journal of Educational Thought, 44(2) 167 188. Wainer, H. (2011). Uneducated guesses: Using evidence to uncover misguided education policies Princeton, NJ: Princeton University Press.
100 BIOGRAPHICAL SKETCH Synthia Delina LaFontaine attended the public schools of Fort Worth, Texas. She earned a Bachelor of Arts in Human Learning and Development from Te xas Wesleyan University in 1996, with an endorsement in Bilingual Education. In 2003, Synt hia earned a M aste r of Education degree in curriculum and i nstruction, English for Speakers of Other Languages, (ESOL), from t he University of Florida. In 2008, she earned an Educational S pecialist degree in educational l eadership from NOVA Southeastern University. In 2014, she earned a Do ctor of Education degree in educational l eadership from the University of Florida. In Texas, Synthia taught as a bilingual teacher in Fort Worth Independent School (ESOL) at Tarrant Count College before moving to Florida in 2000. In Florida, she taught as an ESOL teacher at East Naples Middle School and ta ught adults ESOL at administrative positions in CCPS include d a site based administrator at Lorenzo Walker, a distr ict based teacher trainer, and five years as the Coordinator of School I mprovement Subsequently, she became an improvement specialist with the Florida Department of Education, Bureau of School Improvement, Differentiated Accountability Region V team supporting improvement efforts in the public schools of Broward, P alm Beach and Dade counties