<%BANNER%>

Cost Efficiency of Supplemental Educational Services in a Selected School District

Permanent Link: http://ufdc.ufl.edu/UFE0045152/00001

Material Information

Title: Cost Efficiency of Supplemental Educational Services in a Selected School District
Physical Description: 1 online resource (100 p.)
Language: english
Creator: Kohler, Traci
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2013

Subjects

Subjects / Keywords: after -- educational -- programs -- school -- services -- supplemental
Human Development and Organizational Studies in Education -- Dissertations, Academic -- UF
Genre: Educational Leadership thesis, Ed.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Prior to recent competitive grants, the NCLB Act of 2001 was the accountability standard for which equity and efficiency were measured. From the viewpoint of efficiency, NCLB aimed to increase the accountability of school districts by transforming resources into performance outputs via student academic performance. In this context, the importance of educational efficiency was a main principle of NCLB.  The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the school district.  A quasi-experimental design was used to analyze the Reading FCAT developmental scale scores from 2008 to 2009 wherein student achievement results were divided into four categories of students: students who signed up for SES tutoring in reading but did not receive services, students who received greater than zero but less than thirteen hours, students who received thirteen or greater hours, and students who did not receive SES tutoring in reading.  To answer the research questions addressed in this study, parametric and nonparametric statistical analyses were conducted. A correlation coefficient was run to determine if there were statistically significant increases in FCAT Reading developmental scale scores,based upon the subjects’ participation in SES reading only. Computations were performed using SAS software and tested at a .05 level of significance. In determining the effects of differing levels of SES participation hours, Spearman rho measured the statistical dependence between SES reading hours and FCAT Reading results. To analyze student learning gains in reading when comparing students who participated in SES tutoring with students who did not a Satterthwaite was applied.  Results of this cost-efficiency of Supplemental Educational Services study demonstrated that maximum learning gain output was achieved only after a certain threshold of SES reading intervention exposure occurred. Moreover, specific student grade levels or student characteristics were significant factors with regard to SES enrollment.In addition, students who received SES reading program exposure demonstrated reading learning gains statistically significant from students who did not receive exposure.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Traci Kohler.
Thesis: Thesis (Ed.D.)--University of Florida, 2013.
Local: Adviser: Wood, R C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2013
System ID: UFE0045152:00001

Permanent Link: http://ufdc.ufl.edu/UFE0045152/00001

Material Information

Title: Cost Efficiency of Supplemental Educational Services in a Selected School District
Physical Description: 1 online resource (100 p.)
Language: english
Creator: Kohler, Traci
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2013

Subjects

Subjects / Keywords: after -- educational -- programs -- school -- services -- supplemental
Human Development and Organizational Studies in Education -- Dissertations, Academic -- UF
Genre: Educational Leadership thesis, Ed.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Prior to recent competitive grants, the NCLB Act of 2001 was the accountability standard for which equity and efficiency were measured. From the viewpoint of efficiency, NCLB aimed to increase the accountability of school districts by transforming resources into performance outputs via student academic performance. In this context, the importance of educational efficiency was a main principle of NCLB.  The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the school district.  A quasi-experimental design was used to analyze the Reading FCAT developmental scale scores from 2008 to 2009 wherein student achievement results were divided into four categories of students: students who signed up for SES tutoring in reading but did not receive services, students who received greater than zero but less than thirteen hours, students who received thirteen or greater hours, and students who did not receive SES tutoring in reading.  To answer the research questions addressed in this study, parametric and nonparametric statistical analyses were conducted. A correlation coefficient was run to determine if there were statistically significant increases in FCAT Reading developmental scale scores,based upon the subjects’ participation in SES reading only. Computations were performed using SAS software and tested at a .05 level of significance. In determining the effects of differing levels of SES participation hours, Spearman rho measured the statistical dependence between SES reading hours and FCAT Reading results. To analyze student learning gains in reading when comparing students who participated in SES tutoring with students who did not a Satterthwaite was applied.  Results of this cost-efficiency of Supplemental Educational Services study demonstrated that maximum learning gain output was achieved only after a certain threshold of SES reading intervention exposure occurred. Moreover, specific student grade levels or student characteristics were significant factors with regard to SES enrollment.In addition, students who received SES reading program exposure demonstrated reading learning gains statistically significant from students who did not receive exposure.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Traci Kohler.
Thesis: Thesis (Ed.D.)--University of Florida, 2013.
Local: Adviser: Wood, R C.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2013
System ID: UFE0045152:00001


This item has the following downloads:


Full Text

PAGE 1

1 COST EFFICIENCY OF SU PPLEMENTAL EDUCATIONAL SERVICES IN A SELECTED SCHOOL DISTRICT By TRACI LYNN KOHLER A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMEN TS FOR THE DEGREE OF DOCTOR OF EDUCATION UNIVERSITY OF FLORIDA 2013

PAGE 2

2 2013 T raci L ynn Kohler

PAGE 3

3 To my family

PAGE 4

4 ACKNOWLEDGMENTS I owe a tremendous debt of gratitude to my family. I would like to thank my mother, Sandy De Young, for raising me to belie ve that I could accomplish anything that I wanted. She instilled in me the importance of a high quality education and worked tirelessly to provide that opportunity for my brother and me. Although my father and step father passed away long before I started o n this doctoral journey, their influence helped me become the self confident, high achieving adult that I am today. A special thank you t o my brother, Scott, whose life saving kidney donation enabled me to continue being a mother, stepmother, wife, daughter sister, and aunt to the most important people in the world, my family. I would also like to thank my husband, Stefan, for his continued love and encouragement. His gentle insistency to keep writing kept me focused as life got increasingly complicated. I could not have accomplished this dream without him My sons, Michael and Hayden, had to sacrifice time spent with me in order for me to complete my dissertation. I will be forever grateful for their patience. My stepchildren, Chad, Alyssa and Haley, whose excitemen t upon learning of my doctoral endeavor kept me motivated to finish. I would like to thank Department of Human Development and Organizational Studies in Education I give my deepest appreciation to my chair, Dr. R. Craig Wood. His expert guidance and academic advice were invaluable throughout this process. I would also like to acknowledge the help and dedication of Angela Rowe. My final acknowledgement is extended to my friends and cohort members from Collier County Public Schools. The support and friendship provided to me has proven to

PAGE 5

5 be a life changing experience. I especially want to thank Susan McNally whose humor and perspective I admire. I thank God for his blessings. With him, all things a re possible.

PAGE 6

6 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF ABBREVIATION S AND DEFINITIONS ................................ ............................. 9 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 13 Statement of the Problem ................................ ................................ ....................... 19 Purpose of the Study ................................ ................................ .............................. 21 Summary ................................ ................................ ................................ ................ 21 Limitations ................................ ................................ ................................ ............... 22 Organization of the Study ................................ ................................ ....................... 22 2 REVIEW OF THE LITERA TURE ................................ ................................ ............ 25 History of After School Programs ................................ ................................ ........... 25 Other After School Programs ................................ ................................ .................. 26 21st Century Community Learning Centers ................................ ...................... 27 The After School Corporation (TASC) ................................ .............................. 31 Quality Indicators of Effective After Schoo l Tutoring ................................ ............... 40 Policy Reports on SES ................................ ................................ ............................ 46 Tennessee State Wide SES Effectiveness ................................ ............................. 48 Los Angeles Unified School District SES Effectiveness ................................ .......... 49 Chicago Public Schools SES Effectiveness ................................ ............................ 51 Milwaukee Public Schools SES Effectiveness ................................ ........................ 53 Educational Funding ................................ ................................ ............................... 56 Cost Efficiency in Education ................................ ................................ ................... 61 Summary ................................ ................................ ................................ ................ 64 3 METHODOLOGY ................................ ................................ ................................ ... 65 Purpose of the Study ................................ ................................ .............................. 66 Overview of the Method ................................ ................................ .......................... 67 Participating District Statistics ................................ ................................ ................. 68 Data Organization ................................ ................................ ................................ ... 69 Procedures ................................ ................................ ................................ ............. 70 Summary ................................ ................................ ................................ ................ 71 4 RESEARCH FINDINGS ................................ ................................ .......................... 73 Research Questions ................................ ................................ ............................... 74

PAGE 7

7 Analysis and Quantitative Results ................................ ................................ .......... 74 Researc h Question 1 : How Did Reading Participation in SES Correlate to Student Reading Learning Gains in a Selected School District? ................... 77 Research Question 2 : Research Question 2: How Did Different Levels of Reading Participation Hours within S ES Correlate to Student Learning Gains in Reading in a Selected School District? ................................ ........... 77 Research Question 3 : Research Question 3: How Did Reading Learning Gains of Students Who Participated in SES Compare to Reading Learning Gains of Similar Students Who Did Not Participate in SES in a Selected School District? ................................ ................................ .............. 78 Summary ................................ ................................ ................................ ................ 79 5 CONCLUSIONS AND RECO MMENDATIONS ................................ ....................... 87 Conclusions ................................ ................................ ................................ ............ 87 Implications ................................ ................................ ................................ ............. 88 Recommendations ................................ ................................ ................................ .. 92 Summary ................................ ................................ ................................ ................ 93 LIST OF REFERENCES ................................ ................................ ............................... 94 BIOGR APHICAL SKETCH ................................ ................................ .......................... 100

PAGE 8

8 LIST OF TABLES Table page 1 1 2008 09 Florida Department of Education Title I Supplemental Educational Services ................................ ................................ ................................ .............. 23 4 1 Population by grade level ................................ ................................ ................... 80 4 2 2008 09 FCAT reading developmental scale scores ................................ ......... 80 4 3 Students with disabilities ................................ ................................ .................... 80 4 4 English language learners ................................ ................................ .................. 80 4 5 Predictor variables on the 2008 09 FCAT reading dev elopmental scale score gains ................................ ................................ ................................ ................... 80 4 6 Reading SES participation hours ................................ ................................ ........ 81 4 7 Subgroup frequency ................................ ................................ ........................... 81 4 8 Grade level ................................ ................................ ................................ ......... 81 4 9 Subgroups by grade level ................................ ................................ ................... 82 4 10 Reading SES participation ................................ ................................ .................. 83 4 11 Reading developmental scale score gains by subgroup ................................ ..... 83 4 12 Students with disabilities by subgroup ................................ ................................ 84 4 13 Students with disabilities and SES reading participation ................................ .... 84 4 14 English language learners by subgroup ................................ ............................. 85 4 15 English language learners and SES reading participation ................................ .. 85 4 16 SES reading participation and reading developmental scale score .................... 85 4 17 Subgroup T wo: SES reading participation developmental scale scores ............. 86 4 18 Subgroup T hree: SES reading participation developmental scale scores .......... 86 4 19 Subgroup T wo reading developmental scale score gains ................................ .. 86 4 20 Subgroup T hree reading developmental scale score gains ................................ 86 4 21 General SES reading participation and reading developmental scale scores .... 86

PAGE 9

9 LIST OF ABBREVIATION S AND DEFINITIONS AYP Adequate Yearly Progress: Identified proficiency goals ea ch year with the target of 100 % proficiency for all students in at least reading/language arts and mathematics by 2013 14. Annual AYP goals were determined by individual state accountability systems and take into consideration student achievement scores by certain subgroups of students as well as student scores taken as a whole. Other academic indicators such as graduation and attendance rates were also measured. 1 ESEA Elementary and Secondary Education Act: First enacted in 1965 and remains the largest fed erally appropriated funding source for K 12 education 2 FCAT Florida Comprehensive Assessment Test : M easured content knowledge and skills in reading, writing, science, and mathematics. The content knowledge and skills assessed by FCAT were alig ned to the core curricular content established in the Next Generation Sunshine State Standards. NCLB No Child Left Behind Act of 2001 : S tated purpose is to ensure that all children have a fair, equal, and significant opportunity to achieve minimum proficie standards and assessments. 3 NGSSS Next Generation Sunshine State Standards : E stablished the core content/standard of the curricula in language arts, science, mathematics and social studies to be taught in Florida and specified the skills that K 12 public school students were expected to acquire. 4 1 U.S. Department of Education, Paige Outlines Adequate Yearly Progress Provisions Under No Child Left Behind Press Release. (July 2002). http://www2.ed.gov/news/pressreleases/2002/07/07242002.html ( accessed October 10, 2010). 2 U.S. Departm ent of Educati on, "Cross cutting Guidance for t he Elementary a nd Secondary Education Act September 1996 Achieved Information," United States Department o f Education http://www2.ed.gov/l egislation/ESEA/Guidance/pt1.html (accessed October 17, 2012). 3 No Child Left Behind Act of 2001 (2002):20 USC §6301. 4 Florida Statute §1003.41(7) (2005).

PAGE 10

10 P ROVIDER A ny public or private (nonprofit or f or profit) entity that met the S schools), private schools, schoo l districts, educational service agencies, institutions of higher education, faith based organizations, community based organizations, business groups, and individuals were among the types of entities that were eligible apply to the S tate for approval to p rovide SES. 5 SES Supplemental Educational Services : A provision contained within NCLB that required tutoring or other supplemental academic enrichment services that were in addition to the school day; and were of high quality, research based, and specific ally designed to increase the academic achievement of eligible children on the academic assessments and attain proficiency in meeting the 6 TITLE I P rovided federal funding under NCLB to ensure that all children had a fair, equal, and significant opportunity to obtain a high quality education and reach, at a minimum, proficiency on challenging s tate academic achievement standards and state academic assessments. 7 5 U.S. Department of Education, Supplemental Educational Services Non Regulatory Guidance ( 2009 ). 6 N o Child Left Behind Act of 2001 ( 2002 ):20 USC §1116. 7 No Child Left Behind Act of 2001, (2002):20 USC §1001

PAGE 11

11 Abstract of Dissertation Presented to the Graduate Scho ol of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education COST EFFICIENCY OF SUPPLEMENTAL EDUCATIONAL SERVICES IN A SELECTED SCHOOL DISTRICT By T raci L ynn Kohler May 2013 Chair: R. Craig Wood Major: Ed ucational Leadership Prior to recent competi tive grants the NCLB Act of 2001 was the accountability standard for which equity and efficiency were measured. From the viewpoint of efficiency, NCLB aimed to increase the accountability of school districts by transforming resources into performance outputs v ia student academic performance. In this context, the importance of educational efficiency was a main principle of NCLB. The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the school district. A qua si experimental design was used to analyze the Reading FC AT developmental scale scores from 2008 to 2009 wherein student achievement results were divided into four categories of students : students who signed up for SES tutoring in reading but did not receive services, students who received greater than zero but less than thirteen hours, students who received thirteen or greater hours, and students wh o did not receive SES tutoring in reading

PAGE 12

12 To answer the research questio ns addressed in this study parametric and n onparametric statistical analyse s were conducted. A correlation coefficient was run to determine if there were statistically significant increases in FCAT Reading developmental scale scores, based upon the Computations were performed using SAS software and tes ted at a .05 level of significance. In determining the effects of differing levels of SES participation hours, Spearman rho measured the statistical dependence between SES reading hours and FCAT Reading results To analyze student learning gains in reading when comparing students who participated in SES tutoring wit h students who did not a Satterthwaite was applied Results of this cost efficiency of Supplemental Educational Services study demonstrated that maximum learning gain output was achieved only aft er a certain threshold of SES reading intervention exposure occurred. Moreover, specific student grade levels or student characteristics were significant factors with regard to SES enrollment. In addition, students who received SES reading program exposure demonstrated reading learning gains statistically significant from students who did not receive exposure.

PAGE 13

13 CHAPTER 1 INTRODUCTION Under the No Child Left Behind Act (NCLB) of 2001 1 school districts that received federal funding must maintain adequate yea rly progress (AYP) of students or face strict sanctions. One sanction mandated that the school district provide Supplemental Educational Services or SES. 2 Specifically, students who enrolled in Title I schools that failed to meet AYP benchmarks for three c onsecutive years had the option of receiving free, after school tutoring interventions via SES. Once approved by state departments of education, public and private SES providers were eligible to provide the academic interventions. NCLB policy makers claime d that the competitive, market based nature of SES would provide quality tutoring service choices and ultimately, positively impact student academic achievement. However, anticompetitive forces appeared to be prevalent among SES providers. 3 As more school districts complied with federal sanctions, the sheer number of SES providers grew exponentially. Only a few firms had the revenue and size to capture most of the market activity. In a study of a large urban school district (from 2004 to 2006), researchers suggested that the larger SES firms had larger class sizes and charged higher hourly rates. 4 Because funding for SES was based upon eligible per 1 No Child Left Behind Act of 2001 (2002):20 USC § 1001. 2 No Child Left Behind of 2001 (2002):20 USC §1116. 3 Matthew Steinberg Patricia Burch a nd Joseph Donovan, "Supplemental Educational Services and NCLB: Policy Assumptions, Market Practices, Emerging Issues," Educational Evaluation and Policy Analysis 29, no. 115 (2007). 4 Carolyn Heinrich Robert Meyer and Gregory Whitten, "Supplemental Educa tional Services Under No Child Left Behind: Who Signs Up, and What Do They Gain?," Educational Evaluation and Policy Analysis 32, no. 2 (June 2010)

PAGE 14

14 pupil allocations, students actually received fewer hours of services when SES providers charged higher hourly rates. During the 2008 09 school year, SES providers in the State of Florida provided an average of fourteen hours of tutoring in reading and ten hours of tutoring in mathematics per student. 5 The SES providers in the selected school district for this stud y provided an average of fourteen hours in reading and seven hours of mathematics tutoring during the same time period, with a median of thirteen hours and seven hours respectively. 6 A survey of Title I administrators in 2006 07 indicated that a majority o f states had monitoring systems in place for SES providers. Of those, only a few states had databases containing student achievement and participation information that permitted rigorous evaluations of achievement effects. 7 These concerns pointed to a mism atch between policy implementation of SES and the concerns for quality, efficiency, and equity claimed as priorities in federal law. The NCLB Act of 2001 was the new accountability standard for which equity and efficiency were measured. From an equity stan dpoint, this act created a mandate by which students from economically disadvantaged backgrounds were afforded educational services that were typically associated with those opportunities of white, middle class students. From the viewpoint of efficiency, N CLB aimed to increase the 5 Deepak Gajre, Program Specialist IV, Florida Department of Education, Bureau of Federal Educational Progra ms, SES question, e mail message to Traci Dami, November 02, 2010. 6 Ibid 7 Scott Naftel Georges Vernez, Karen Ross, Kerstin Carlson Le Flock, Christopher Beighley, and Brian Gill, State and Local Implementation of the No Child Left Behind Act Volume VII T itle I School Choice and Supplemental Educational Services: Final Report (Washington, D.C. U.S. Department of Education, Office of Planning, Evaluation and Policy Development, 2009), RAND.

PAGE 15

15 accountability of school districts by transforming resources into performance outputs via student academic performance. 8 In this context, the importance of educational efficiency was a main principle of NCLB. As in past educationa l policy reform efforts, student achievement results were purported to incr ease proportionally with corresponding increases in financial support. In 1983, the National Commission on Excellence in Education released the report, A Nation at Risk 9 The report focused many policy makers away from fiscal equity and toward economic efficiency. However, given historical educational research regarding fiscal equity coupled with the emergence of economic efficiency, researchers then During the remainder became the political basis for educational reform efforts. Researchers began studying whether improving schools would require additional funding. Researchers such as Hanushek argued it did not. After adjusting for inflation, the amount of per student spending almost tripled during 1960 and 2000. Yet, the academic performance of high school seniors from 1970 until 2003 remained nearly flat. 10 As researchers began to look toward increasing efficiencies within the organization, management, and operation 8 Je Ik Cho, "An Efficiency and Productivity Study in the Presence o f the No Child Left Behind Act in Pennsylvania School Districts" (PhD diss., Pennsylvania State University, University Park, PA), 2009. 9 National Commission on Excellence in Education. A Nation at Risk: The Imperative for Educational R eform Washington, D. C., 1983. 10 Eric Hanushek, "The Failure of Input based Schooling Practices," Economic Journal 113, no. 485 (2003)

PAGE 16

16 During the 1990s, the nation embarked on a new school reform path. The foundation of effe ctive school research held that all children can be taught the intended curriculum and held to high academic standards. 11 Subsequently, a movement toward establishing standards for student achievement grew. State legislatures began to discuss means of stand ardizing assessments to measure student outcomes. Eventually, some state legislation created rewards and sanctions for school districts based upon student outcomes on state assessments. By the year 2000, the majority of states had state level accountabilit y systems in place. 12 However, state assessments remained varied in content, methodology, rewards and sanctions. Collecting measurable student achievement results differentiated school accountability policies from prior school reform efforts which include d specific program endorsements and resource increases. Initial accountability reports in the 1990s indicated that states with accountability sanctions showed improvements of 0.2 standard deviations in eighth grade test scores on the National Assessment of Educational Progress (NAEP) than states who did not employ accountability consequences. 13 11 See e.g. Levine, Daniel U. and Lawrence W. Lezotte. Unusually Effective Schools: A Review and Analysis of Research and Practice ( Madison, WI : University of Wisconsin, Madison, 1990), The National Center for Effective Schools Research and Development Social Policy 12 no. 2 (1981). 12 Eric Hanushek, 2003 13 Eric A. Hanushek, "Why the Federal Go vernment Should be Involved in School Accountability," Journal of Policy Analysis and Management 1, no. 24 (Winter 2005).

PAGE 17

17 The 2000 Presidential Campaign of George W. Bush focused on the successes of school accountability measures and promoted the increase of student testing along with sch ool district sanctions as a means of improving the educational system in the United States. Once elected, President Bush made outcome based testing a legislative priority. Subsequently, the No Child Left Behind Act of 2001 14 received bipartisan support in C ongress, and President Bush signed it into law in January 2002. Prior to current Race to the Top competitive grant funding 15 NCLB was the largest federal appropriated funding source of elementary and secondary education in the United States. 16 The bipartis an passage of NCLB reauthorized the Elementary and Secondary Education Act (ESEA) of 1965. 17 For the first time in history, federal funding was tied to student academic outcomes. Students from low income backgrounds, with disabilities and/or from minority b ackgrounds were afforded educational opportunities usually associated with white, middle class students. 18 After school tutoring was one of those opportunities. Under NCLB, local school districts that failed to meet AYP were required to set aside 20 % of the 19 Title I was and remains a 14 No Child Left Behind Act of 2001, (2002):20 USC 15 Patrick M c Guinn, "Stimulating Reform: Race to the Top, Competitive Grants and the O bama Education Agenda," Educational Policy 26, no. 1 (January 2012) 16 U.S. Department of Education, 10 Facts About K 12 Education Funding, Washington, D.C., 2005. 17 Ibid 18 No Child Left Behind Act of 2001 (2002):20 USC §1001. 19 No Child Left Behind Act of 2 001 (2002):20 USC §1116.

PAGE 18

18 federally funded formula grant provided to school districts to help in the educational advancement of students who attend high poverty schools. 20 The 20 % set aside was earmarked for t wo specific sanctions, public school choice and SES. Public school choice was offered to students who attended schools that had not made AYP for two consecutive years. Part of the 20 % was required to fund the transportation of students to their public scho ol of choice. Supplemental Educational Services were offered to students that attend ed schools that had not made AYP for three consecutive years. 21 In addition, this a cademic assistance was was consis tent 22 The money to finance SES was provided by redirecting Title I funding. 23 As stated earlier, NCLB required a 20 % set aside. Districts were required to spend a minimum of 5 % of the set aside solely on S ES programs. 24 The per child cost of SES was determined by dividing the number of the five to seventeen who were from families living below the poverty level. 25 Bec ause of this calculation, the max imum per child expenditure for Supplemental E duca tional S ervices varied broadly across states and districts within states. In addition, no 20 No Child Left Behind Act of 2001, (2002):20 USC §1002. 21 No Child Left Behind Act of 2001 (2002):20 USC §1116. 22 Ibid 23 Ibid 24 Ibid 25 Ibid

PAGE 19

19 administrative cost or any other cost associated with SES was to be charged to the s et aside. 26 During Fiscal Year 2008 09, Florida was one of six states chosen by the U.S. Department of Education to implement a Differentiated Accountability Pilot to help states develop better ways to target interventions in schools identified as in need o f improvement. 27 Students in those schools designated as Schools in Need of Improvement (SINI) were given targeted supplemental education services before they were given a choice to transfer to a school not designated as a School in Need of Improvement. 28 Th is change switched NCLB support from school choice being the first sanction followed by supplemental educational services as the second sanction. Statement of the Problem Districts varied broadly in the amount spent per child in Title I supplemental educat ional services. In a sample of 100 school districts across the nation, 32 % spent less than $600 per student enrolled while 17 % spent $1,200 or more during the 2005 06 school year. 29 During the 2008 09 school year, Florida spent an average of $1,268 per stud ent enrolled in a SES program, with a range extending from $848 to $1,405. 30 During the same time period (2008 09), the selected school district in this study spent 26 No Child Left Behind Act of 2001 (2002):20 USC §1116 27 Florida Department of Education Florida's Differentiated Accountability Model Guidance for Implementation 2008 09 School Year (2008 ). 28 Ibid 29 S cott Naftel et al. 2009. 30 Deepak Gajre, November 02, 2010.

PAGE 20

20 an average of $1,023 per pupil. 31 Nationally, overall SES expenditures nearly doubled to $375 million from $192 million (2003 04 to 2005 06), mostly because of increased district participation via NCLB sanctions. 32 However, the average maximum per pupil allocation for SES decreased from $1,434 in 2004 05 to $1,135 in 2005 06. 33 Despite the increasin g number of eligible students and rising set aside expenditures for SES due to NCLB sanctions, student participation rates remained low. Based upon state reported data of 2005 06, 31 % of districts that were required to offer SES had reported no students pa rticipating in SES; 25 % reported participation rates of less than 5 % ; 21 % reported participation rates between 2 and 20 % and 24 % reported participation rates higher than 20 % 34 During the 2008 09 school year, the Florida Department of Education reported a 1 3.58 % participation rate. 35 Given the large economic impact SES has on Title I budgets, it is surprising that relatively few studies have been conducted on the academic and/or cost efficiency of Supplemental Educational Services. Of those that reported acad emic outcomes, studies indicated few or no positive correlations with increased student achievement on state assessments. 36 31 Deepak Gajre, November 02, 2010 32 S cott Naftel et al. 2009. 33 Ibid 34 Andrew Abrams et al., 2009. 35 Deepak Gajre, Program Specialist IV, Florida Department of Education, Bureau of Federal Educational Programs, SES question, e mail me ssage to Traci Dami, November 10 2010. 36 See e.g. Steven M. Ross Allison Potter, Jangmi Paek, and Dawn McK ay, "Implementation and Outcomes of Supplemental Educational Services: The Tennessee

PAGE 21

21 Purpose of the Study The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the selected school district. The following questions were addressed in this study: 1. How did reading participa tion in SES correlate to student reading learning gains in a selected school district? 2. How did different levels of reading participation hours within SES correlate to student learning gains in reading in a selected school district? 3. How did reading learning gains of students who participated in SES compare to reading learning gains of similar students who did not participate in SES in a selected school district? Summary Race to the Top competitive grants as well as remaining NCLB programs (Title I, Part A) f ocus ed on decreasing the achievement and opportunity gaps for students. Funded with a set aside from Title I, Part A SES programs targeted low income and low performing students in academic based after school programs. Few national and local studies docum ented the positive academic impact of SES as defined within federal accountability requirements. The efficiency of Title I, Part A set aside funds to support the SES structure warranted study in terms of significant academic learning gains. State Wide Evaluation Study," Journal of Education for Students Placed at Risk 13 (2008), Jordan H. Rickles and Melissa K. Barnhart, The Impact of Supplemental Educational Services Participation on Student Achievement: 2005 06 (Los Angeles, CA: Los Angeles Unified School District, Program Evaluation and Res earch Branch, 2007), LAUSD and Carolyn Heinrich, Robert Meyer and Greg Whitten, June 2010

PAGE 22

22 Limitations Th i s analysis included a case st udy approach that analyzed cost efficiency of Supplemental Educational Services in a broad sense. No attempt was made to analyze individual SES programs. Generalizability to other SES programs within school districts is not cle ar. Lastly, the identification and classification criteria of English Language Learners differ from state to state. No attempt was made to adjust or alter terminology included in the reported research Organization of the Study Chapter 1 introduced the pr oblems being addressed in this study and presents past educational reform efforts and sections from the Federal Constitution relevant to this study. Also included is the structure of Title I, Part A required set asides. Chapter 2 contains a review of the r elated literature including after school research, previous SES studies, educational funding, and the concept of cost efficiency in education. Chapter 3 includes a discussion on the concept of student learning gains, and the methodology used in this study. Chapter 4 provides the results of the methodology as analyzed using 5 includes conclusions, implications, and recommendations for further study related to the problem addressed in this study The Tables contain data and computations used in the methodology, formulas, and analysis to reach the conclusions stipulated in this study.

PAGE 23

23 Table 1 1 2008 09 Florida Department of Education Title I Supplemental Educational Ser vices District n umber District n ame Number of s chools r equired to offer SES Number of s tudents e ligible for SES Unduplicated n umber of s tudents s erved in r eading and/or m ath SES 01 Alachua 24 6 235 973 02 B aker 1 384 0 03 B ay 7 2 486 945 04 B radford 4 1 047 104 05 B revard 22 8 478 1 528 06 B roward 75 40 074 3 722 07 C alhoun 1 489 50 08 C harlotte 2 1 117 385 09 C itrus 4 1 439 297 10 C lay 3 1 251 233 11 C ollier 14 7 792 434 12 C olumbia 6 2 396 332 13 D ada 171 105 089 16 224 14 D esoto 3 2 048 306 15 D ixie 2 891 138 16 D uval 52 18 380 4 275 17 E scambia 19 6 889 1 807 18 F lagler 2 1 242 221 19 F ranklin 2 615 39 20 G adsden 11 4 922 201 21 G ilchrist 1 347 58 22 G lades 1 282 15 23 G ulf 2 525 36 24 H amilton 2 657 105 25 H ardee 4 1 768 226 26 H endry 3 1 513 229 27 H ernando 9 4 897 556 28 H ighlands 6 2 287 415 29 H illsborough 100 52 808 4 211 30 H olmes 5 1 632 156 31 Indian River 3 1 088 218 32 J ackson 6 2 318 324 33 J efferson 1 843 69 34 L afayette 1 345 40 35 L ake 10 5 072 850 36 L ee 16 8 206 1 525 37 L eon 8 3 016 455 38 L evy 8 2 939 258 39 L iberty 2 650 36 40 M adison 5 1 612 206 41 M anatee 10 4 713 846 42 M arion 23 9 359 1 442 43 M artin 3 1 264 264

PAGE 24

24 Table 1 1. Continued District n umber District n ame Number of s chools r eq uired to offer SES Number of s tudents e ligible for SES Unduplicated n umber of s tudents s erved in r eading and/or m ath SES 44 M onroe 3 1 010 59 45 N assau 2 845 165 46 O kaloosa 4 959 216 47 O keechobee 5 2 317 250 48 O range 46 26 766 3 526 49 O sceola 9 6 437 719 50 P alm B each 78 43 138 4 441 51 P asco 17 9 258 1 421 52 P inellas 41 16 478 2 823 53 P olk 46 23 567 2 210 54 P utnam 7 3 054 402 55 St. Johns 5 1 777 266 56 S t L ucie 10 4 056 991 57 Santa Rosa 4 1 759 560 58 S arasota 6 2 542 577 59 S emin ole 10 4 358 543 60 S umter 1 1 118 115 61 S uwannee 1 509 161 62 T aylor 3 1 179 152 63 U nion 0 0 0 64 V olusia 31 12 254 1 398 65 W akulla 1 88 16 66 W alton 3 1 140 309 67 W ashington 2 975 93 68 S chool for D eaf /B lind 1 83 0 74 FAMU L ab School 1 196 2 State t otal 988 487 268 66 169 Sources: 2008 09 Title I Schools Identified for Improvement, Corrective Action, or Restructuring under Section 1116, Florida Department of Education, 2008 09 Survey 3 Student Demographics, Florida Department of Education 2008 09 Survey 3 Title I Supplemental Educational Services, Florida Department of Education

PAGE 25

25 CHAPTER 2 REVIEW OF THE LITERA TURE The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine t he extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the selected school district. SES was designed as an after school tutoring program mandated through NCLB to increase the ac ademic achievement of students who attended low performing Title I schools. The purpose of this chapter is to present a review of the literature pertaining to the history and best practices of other after school programs. The current status of SES effectiv eness nationwide will then be reviewed. Finally, the literature relevant to educational funding and the concept of cost efficiency in education will be presented. History of After S chool Programs After school programs arose in response to historical labor force changes and the creation of formal schooling in the late Nineteenth Century. 1 Compulsory education laws along with a declining need for child participation in an industrialized labor force increased the amount of time children had outside of school. By the turn of the century, after school programs included activities beyond those of childcare. Besides providing academic needs. 2 The demands for after school programs c ontinued to grow as 1 Robert Halpern, t Kind of Child Development Institution: The History of After School Programs for Low Teachers College Record 104 no. 2, (March 2002). 2 Ibid

PAGE 26

26 m aternal employment rose in the Twentieth C entury. Thirty eight percent of mothers with children ages six to seventeen were employed in 1955, with recent estimates ranging close to 80 % 3 Since school age children spent most of their wak ing hours in discretionary activities other than in school parents, educators and policy makers were concerned with the positive and negative developmental consequences of student activity during this discretionary time. During the 1990s, the provision, fu nding and acceptance of organized after school programs greatly increased. The percentage of public schools and after school programs more than tripled between 1987 and 1999, increasing from approxima tely 16 % to 47 % 4 In 2006, 10 % of school aged children attended after school programs. 5 Other After School Programs An increasing number of studies have been published in recent years due to the magnitude of student participation in after school programs. The majority of studies investigated whether participation in formal after school programs led to academic improvements and to a lesser extent, other type of developmental gains. An analysis performed by Granger reported positive effects on average across studies, but the 3 U.S. Department of Labor, Women in the Labor Force: A Databook, (2005) 4 Karen DeAngelis and R obert Rossi. Schools Serving Family Needs: Extended Day Programs in Public and Private Schools National Center for Education Stati stics Issue Brief. NCES, 97 590. 5 P. R. Carver and I. U. Iruka, After school Programs and Activities National Ce nter for Edu cational Statistics, NCES, 2006 07

PAGE 27

27 majority of studies did not find that program participants demonstrated higher academic performance than nonparticipants. 6 One review of four major after school programs demonstrated that participant attendance was often intermittent, ave raging one to two days per week. 7 Results raised questions as to the amount of program exposure (treatment intervention) actually received by participants. Low levels of participant attendance may have explai ned some nonsignificant impact findings in after school programs. 8 21st Century Community Learning Centers In 1994, the federal government took an increased role in after school programs Department of Education. Subseque ntly, Congress authorized the 21 st Century Community Learning Centers (21 st CCLC) program and opened schools for use by local communities. 9 In 1998, 21 st CCLC programs were refocused and emphasis was placed on providing academic, enrichment, and recreation al activities in public schools during after school hours. 10 Centers were opened before school, on weekends, and during the 6 R. C. Granger, After school Programs and Academics: Implications for Po licy, Practice, and Research Social Policy Report 2, no. 22 (2008). 7 Thomas J. Kane. The Impact of After school Programs: Interpreting the Result s of Four Recent Evaluations Working Paper, Los Angeles: University of California and William T. Grant Foundation, 2004. 8 Susan J. Bodilly and Megan K. Beckett Making Out of School Time Matter: Evidence for an Action Agenda ( Santa Monica, CA: RAN D Corpor ation, 2005), RAND. 9 U.S. Department of Education, When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program (Washington, D.C.: Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, 2005), NCEE. 10 Ibid

PAGE 28

28 summer. In 2002 and in concert with the passage of NCLB, 21 st CCLC funding was restructured into block grants for State Departments o f Education to allocate as appropriate. 11 Functioning under new accountability requirements of NCLB, the specific focus for 21 st CCLC was to create community learning centers that provided academic enrichment opportunities for children, particularly student s who attended high poverty and low performing schools, to meet state and local student standards in core academic subjects (mainly reading and math), to offer students a broad array of enrichment activities that complement their regular academic programs, and to offer literacy and other educational services to the families of participating children. 12 Earlier analyses of 21 st CCLC found few significant impacts on the behaviors and academics of students. 13 The United States Department of Education released it s final report on 21 st CCLC in April 2005. Twenty six 21 st CCLC in twelve school districts were included. The research design utilized a randomized controlled field study. Elementary students were randomly assigned to either a 21 st CCLC group or to a contr ol group. The study found that elementary students who were randomly assigned to the 21 st CCLC were no more likely to have higher academic achievement as evidenced by reading test 11 U.S. Department of Education, 21 st Century Community Learning Centers Non regulatory Guidance (2003). 12 No Child Left Behind Act of 2001, (2002):20 USC §4201. 13 U.S. Department of Education, When Schools Stay Open La te: The National Evaluation of the 21st Century Community Learning Centers Program 2005

PAGE 29

29 scores or grades relative to the control group. 14 However, the treatment grou p was more likely to feel safe after school 15 In 2010, the Center for Research in Educational Policy in conjunction with the Virginia Department of Education completed an evaluation of 21 st CCLCs in the state. Based upon analyses for grades three through e ight that included Standards of Learning (SOL) test data, the 21 st CCLC program had a statistically significant positive influence on reading and mathematics for students designated as students with disabilities (SWD) who attended 30 or more days compared to matched control students. 16 However, the effect was negative for students designated as Limited English Proficient (LEP). 17 The Florida Department of Education with the help of researchers at the University of Florida conducted a study on the impact of 21 st CCLCs in the state of Florida. One hundred twenty two programs participated in three surveys which took place between November 2003 and April 2004. 18 14 U.S. Department of Education, When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program 2005. 15 Ibid 16 Todd Zoblots ky and Ying Huang, Evaluation of 21st Century Community Learning Centers 2007 2008 Supplemental Technical Report Analysis for Grades 3 8 (Memphis, Tennessee: Virginia Department of Education 2010 ), CREP. 17 Ibid 18 James J. Zhang, David S. Fleming, and Bra ndy L. Bartol, The Sunshine State Does Great Things for Its Children: Assessing the Effectiveness of the 21st Century Community Learning Centers (21st CCLC) Program (Gainesville, FL: Uni versity of Florida, College of H ealth and Human Performance, 2004).

PAGE 30

30 Each program in the state was required by the Florida Department of Education to address specific absol ute priorities. The absolute priorities were: 1. To target services to students who attend schools that have been identified as being in need of improvement, 2. To support the reading initiative, specifically Just Read! Florida, 3. To disseminate and market 21 st CC LC program to the appropriate population, and 4. To operate the program jointly between one or more local education agencies and one or more community based organizations. 19 In addition competitive priorities were encouraged and included the following: 1. T o use a significant portion of the program funds to address substantial problems within D or F schools as identified by the Florida Department of Education 2. T o provide nutrition, health, and fitness programs that will help decrease the national obesity epidemic, and 3. T o form partnerships with institutions of higher education, libraries, and other private and/or public profit and nonprofit entities with technological expertise to improve the use of technology. 20 For the 2003 04 school year, forty three programs were granted funding. Of the forty three programs, forty one 21st CCLCs operated during the regular school year. Approximately 75 % of participating students were on free or reduced price lunch. The number of students with limited English proficiency represente d 12.5 % of the program participants. The number of students requiring special education represented 11.2 % of the program participants. 19 F lorida Department of Education, Request for Proposal/Application 21st Century Community Learning Centers Program ( 2003). 20 Ibid

PAGE 31

31 Student participation among ethnic groups remained similar over time. African Americans accounted for 46.4 % Hispanics a ccounted for 23.7 % Caucasians accounted for 15.8 % Asians/Pacific Islanders accounted for 1.1 % American Indians accounted for 0.2 % and other ethnicities accounted for 12.8 % many of which were from multiethnic backgrounds. In terms of the number of opera tional days per site, each site was opened approximately twenty two days per month. This number remained steady over the three survey periods. The average 21st CCLC program required annual budgets that ranged from $150,000 to $200,000. Program participant costs averaged $2.56 per hour, $8.15 per day, and $37.96 per week. Academic achievement results were analyzed using independent group analysis of variance (ANOVA). Surveys were periodically conducted. Dependent variables for reading included reading stand ardized test scores and average reading grade in school. Results of the study indicated significant reading mean score differences among elementary, middle and high school participants. The achievement results of e lementary school students demonstrated inc reases in the fourth testing period. The achievement results of m iddle school students demonstrated decreases from the baseline to the first and second testing period, and increases from the second testing period to the third test period. Lastly, the achie vement results of h igh school students demonstrated increases in the second and third periods. At the time publication, statewide standardized test scores were not released, making a pre and postanalysis not possible in the study. The After School Corpora tion (TASC) TASC was founded in 1998 with a goal of increasing the availability and quality of after school services in New York City. Because after school programs were historically

PAGE 32

32 after school programs as a public sector responsibility in which participants incurred no out of pocket expenses. 21 TASC programs were built upon partnerships with public schools and other public and private partners. Four foundations (Charles Stewart Foundation, the Ca rnegie Corporation of New York, the William T. Grant Foundation, and the Atlantic Philanthropies) supported external evaluations of TASC to assess program effects on students and program practices linked to student success. 22 ation, evaluation data were collected from ninety six TASC after school programs and host schools. The student sample contained 52,000 after school participants and 91,000 nonparticipating students who attended host TASC schools. 23 The final report examined outcomes of TASC after school projects as a whole and not on projects individually. Overall, the evaluation was framed The e following: 1. The supply of affordable, high quality after school programs open to all children and youth is not adequate to meet the demand. 2. The private sector and parents lack the resources and institutional access required to equitably meet the need for after school services. 3. Public sector investment can play a central role in expanding and improving after school opportunities for children and youth. 21 Elizabeth R. Reisner, Richard N. White, Christina A. Russell and Jennifer Birmingham, Building Quality, Scale and Effectivene ss in After school Programs (Washington, D.C.: Policy Studies Associates, Inc ., 2004 ), Policy Studies Associates, Inc. 22 Ibid 23 Ibid

PAGE 33

33 4. An intermediary organization, such as TASC, is necessary to promote, cultivate, coordinate, and support lo cal partnerships that provide high quality after school programs. Strategies that the intermediary uses to provide support need to include training, technical assistance, and volunteer recruitment. These supports will promote program quality, including con tinuity in staffing. 5. By developing strong local partnerships, TASC will identify and increase a wide range of public and private supports that can be effectively targeted to expand and improve after school services including training and technical assistan ce. 6. Using financial resources drawn from many sources and a program design based on core elements that promote good practice in both education and youth development fields, TASC will award grants to nonprofit organizations to operate afterschool projects w hose designs are research based and comprehensive in scope. 7. The TASC projects will extend after school services to children and youth in their host schools, using the resources provided by TASC and others to support nonprofit sponsoring organizations. TASC grants will explicitly promote collaboration between schools and communities, with the after school projects bridging these two cultures. 8. In their first year of operations, projects will establish procedures, staffing, and schedules to permit the delivery of planned activities and services. In subsequent years, the projects will focus on refining the quality and effectiveness of their activities and services, as they enlarge their capacity to serve more students. 9. As a result of effective program designs th at address the needs of working parents and their school age children, enrolled students will attend the after school projects regularly as part of an extended learning platform supported by their schools. 10. By participating regularly in after school activit ies that are distinctive from but connected to the school day, students will improve their academic skills and knowledge, gain new experiences, take advantage of opportunities for recreation and artistic expression, and develop psychosocial skills that pro mote positive youth development. 11. The increased availability of high quality after school services will: (a) provide participants with opportunities for educational enrichment and their own education.

PAGE 34

34 12. Through the documentation of after school evaluator will collect evidence describing the delivery and effectiveness of after schoo l services from participating schools, parents, and students. 13. The evidence, in combination with organized, broad based advocacy, will persuade the public and policymakers that afterschool programs are necessary, feasible, and affordable. 14. Support for the pr ograms will become widespread, and public officials will increasingly invest in universally available and sustainable programs as a public sector responsibility. 24 Overall, the evaluation concentrated on three areas: quality services, student benefit and id entification of best practices. Data were collected through surveys, site visits and review of administrative records. Quality services focused on areas such as high levels of after school program attendance, the development of strong relationships between host school programs and parents, and the ability to increase fiscal independence. Student benefits mainly focused on student academic growth and school change and m apped backward predominant program characteristics or components. Students who participated in TASC programs as compared to New York Public Schools overall were lower achieving in school, came from lower inco me families and were of black or Hispanic backg rounds. On average, 32 % of students in TASC host schools enrolled in the after school program while 59 % of students in TASC host high 24 Elizabeth R. Reisner Richard N. White, Jen nifer Birmingham and Megan Welsh, Building Quality and Supporting Expansion of After school Projects: Evaluation Results From the TASC After school Program's Second Year (Washington, D.C.: Policy Studies Associates, Inc., 2001). Policy Studies Associates, Inc.

PAGE 35

35 schools enrolled. 25 However, the high school figure was inaccurate in that some high schools automatically enrolled all stu dents in the TASC program. In terms of program attendance, evaluators categorized participating students as Elementary and middle school TASC participants were considered active if they attended at least sixty days during the school year, or an average of three days per week. 26 During the 2001 02 school year, 71 % of students met the criteria as active participants. 27 For high school students, a different criterion A mini mum of twenty days 28 During the 2001 02 school year, 47 % of students met the criteria as active participants. 29 In order to build strong relationships with parents and host sc hool TASC programs, site coordinators sponsored numerous activities such as cultural and recreational events and parenting classes. Analyses demonstrated increases in project sponsored activities across the six years and were usually connected to a private nonprofit organization. 30 High school projects were more likely to engage outside community agencies through student internship opportunities. 25 Elizabeth R. Reisner et al. 2004. 26 Ibid 27 Ibid 28 Ibid 29 Ibid 30 Ibid

PAGE 36

36 TASC programs were encouraged to gain gradual financial independence from the grant awards. In 2001 02, TASC awar d recipients were expected to finance 30 % of total operating funds from sources other than TASC. 31 During that same time period, TASC grants provided program operational funds at $980 per student, down from $1,000 per student in 1999 2000. 32 After school act ivities included academic enrichment, homework assistance, the arts and recreation. Taken as a whole, evaluators examined the impact of these activities in terms of academic performance and school attendance. Data sources included test scores, surveys and attendance records. In the area of mathematics, TASC participants made greater gains than did similar nonparticipants. 33 Students who actively participated in TASC for a year demonstrated greater gains (although not statistically significant) then did nonpa rticipating students, and students who actively participated in TASC for two or more years demonstrated substantial gains (effect size of 0.42) than nonparticipating students. 34 Specifically, students with disabilities who attended TASC projects gained more points than expected on math tests after one year of participating in TASC (effect size of .24). 35 Another subgroup, students identified as English Language Learners (ELL) also gained more points than expected in math after a year of TASC participation 31 El izabeth R. Reisner et al. 2004. 32 Ibid. 33 Ibid 34 Ibid 35 Ibid

PAGE 37

37 (ef fect size of 0.11). 36 Evaluators considered an effect size of 0.10 or larger to be substantive. In the area of reading, TASC students did not demonstrate significant achievement differences when compared to nonparticipating students. 37 The evaluators sugges ted that the disparity between the mathematics and reading results was due to the fact that mathematics success depends on a skill building process; whereby, students benefit ed from completing homework and obtaining assistance with it. In con trast, literac y success required more complex skills and experiences. Homework assistance alone was not enough to scaffold student literacy learning. TASC participation influenced attendance patterns in grades PreK 8. Students who experienced one year of TASC exposure d emonstrated an increase of 0.53 % in attendance as compared to an increase of 0.11 % for nonparticipants. 38 The net difference equated to an increase of three quarters of a school day over the school year for TASC participants. For those students who particip ated in TASC for two years, student attendance yielded an increase of 0.75 % or an increase of 1.2 days over the school year. 39 School attendance associated with TASC participation was noteworthy in the middle grades where effect sizes were greater than 1. 0 and analyses controlled for 36 Elizabeth R. Reisner et al 2004. 37 Ibid. 38 Ibid 39 Ibid

PAGE 38

38 declined between fifth and eighth grades, active TASC participants did not. The greatest change took place between seventh and eighth grade where the attendance gain of all participants to nonparticipants measured 1.5 % creating a net gain of 2.7 school days in a school year. 40 The TASC evaluation also centered on best practices associated with the greatest academic benefits for students. Analy sis was limited due to the structure of TASC programs which offered only slight variations among host sites. Factors such as a full time site coordinator, academic pressure of host schools to increase student academic performance, and TASC training and tec hnical assistance remained constant among TASC programs. However, evaluators found certain project characteristics that were associated with positive participant outcomes. TASC participants who demonstrated more positive 1 year gains in reading and mathema tics were enrolled in programs that offered high intensity of academic and cognitive development activities (effect size of 0.26 in math and 0.15 in reading.). 41 Data were collected through interviews and site visits, and results suggested that multidiscipl inary activities which often culminated in performances, major products or publications fostered rigorous, student centered activities. 42 TASC staffing and supervisory practices were also positively associated with student learning. TASC participants perfor med better academically than other TASC participants (effect size of 0.13 in math and 0.26 in reading) when the site coordinator 40 Elizabeth R. Reisner et al., 2004 41 Ibid 42 Ibid

PAGE 39

39 had a teaching certificate. 43 The difference may be due to the availability of the site coordinator to promote rigorous instruct ion through coaching and modeling and to the professional understanding of the site coordinator to connect and enhance student learning experiences from the school day. 44 In addition, TASC programs that had at least 25 % of staff with 4 year degrees demonstr ated greater student academic increases than in TASC programs with a lower percentage of staff with 4 year degrees (effect size of 0.14 in math and 0.13 in reading). 45 In TASC programs where the site coordinator required staff to submit lesson plans, partic ipants demonstrated greater academic gains on standardized tests than other programs in which the site coordinator did not require lesson plans (effect size of 0.14 in math and 0.17 in reading). 46 Evaluators suggested that lesson preparation may have influe nced the quality of the participants learning experiences. Lastly, qualitative findings in the evaluation suggested certain programmatic features were important to TASC success and included: 1. T he partnership between nonprofit sponsoring organizations and ho st schools 2. T he location of after school 3. T he employment of a full time project site coordinator and 43 Elizabeth R. Reisner et al., 2004 44 Ibid 45 Ibid 46 Ibid

PAGE 40

40 4. T he expectation that participants would attend the after school project almost every day 47 Quality Indicators of Effective After School Tutoring The needs of low income children have been a major influence on the development of after school programs. Title I of the Elementary and Secondary Education Act was created in response to data trends that indicated low incom e children were at risk for academic failure and therefore needed additional instructional time to supplement regular school day educational activities. 48 The No Child Left Behind Act of 2001 required states to place academic standards and assessment measu res in place so that all students achieved proficiency in reading and mathematics. States were required to provide Supplemental Educational Services (SES) to low income student s in Title I schools that did achieve adequate yearly progress toward this goal. regular instructional day, there was interest among researchers and educators in effectiveness of after school strategies in improving student achievement. Specifically, NCLB required all programs and stra tegies to be grounded within scientifically objective procedures to obtain reliable and valid knowledge relevant to educational y through experime ntal or quasi experimental methods. 49 In addition, legislation required the creation and application of rigorous State 47 Elizabeth R. Reisner et al., 2 004 48 Robert H. Finch, History of Title I ESEA (Washington, D.C.: United States Department of Health, Education, and Welfare, Office of Education, 1969), DHEW. 49 No Child Left Behind Act of 2001 (2002):20 USC §9101

PAGE 41

41 academic standards. 50 In 2010, the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) identified fifty thr ee successful after school programs in terms of increasing student academic proficiency. In all fifty three programs, staff had broad knowledge of National and State standards in core content (reading/language arts and mathematics) areas. 51 A research synth esis conducted by McRel found after school programs could significantly increase student achievement in reading and mathematics on average by one tenth of a standard deviation compared to students who did not participate in after school programs. 52 While af ter school programs varied in the approaches to improve student reading proficiency, a few common features emerged. Improved student attendance and improved academic performance were positively correlated to the number of student visits to after school pro 53 However, other research suggested student learning gains did not occur until a student 50 No Child Left Behind Act of 2001 (2002) : 20 USC §1001 51 Denise Huang Jamie Cho, Hannah Nam, Christine Oh, Aletha Harven and Seth Leon, What Works? Common Practices in High Functioning After school Programs Across the Nation in Math, Reading, Science, Arts, Technology, and Homework A study by the National Partnership (Los Angeles: University of California, Graduate School of Evaluation, Standards, and Student Testing, 2010), CRESST, 768. 52 McR el The Effectiveness of Out of School Time Strategies in Assisting Low Achieving Students in Reading an d Mathematics: A Research Synthesis (Washington, D.C .: United States Department of Education, Institute of Education Sciences, 2004), McR el # 2003 19. 53 Marco Munoz, Outcome based Community School Partnerships: The Impact of the After school Programs on Non academic and Academic Indicators ED468973.

PAGE 42

42 participated in a significant number of hours in an after school program, at times requiring fifty participation hours. 54 The quali ty of staff in after school programs played a role in student academic outcomes. Content specific lead teachers or supervisors monitored instructional quality and assisted other staff in making modifications to program components according to student needs 55 In some successful after school programs, staff were often certified teachers at the day school attended by the students. 56 Other common features in successful after school programs included developing academic and social skills and implementing a well d efined reading curriculum. Programs tended to structure activities in a safe environment that informally connected youth with a cross age mix of peers and with positive adult role models while implementing a formal reading curriculum. 57 Project Accelerated Literacy (PAL) provided scaffolded learning through eight major components of literacy: read aloud to children, shared reading, guided reading, independent reading, modeled writing, shared writing, guided writing and independent writing. 58 Other programs bu 54 Darrell Morris, Shaw, and Perney "Helping Low Readers In Grades 2 and 3: An After school Volunteer tutoring Program," Elementary School Journal 91, no. 2 (1990). 55 Denise Huang et al. 2010. 56 McR el 2004. 57 J enni fer Birmingham, Ellen Pechman, Christina Russell, and Monica Mielke, Shared Features of High Performing After school Programs: A Follow Up to the TASC Evaluation (Washington, D.C.: Policy Study Associates, Inc 2005 ), Policy Study Associates, Inc. 58 Mary E. Interrante Hausner, "The Impact of Kindergarten Intervention Project Accelerated Literacy on Emerging Literacy Concepts and Second Garde Reading Comprehension" (PhD diss., Loyola University, Chicago, IL), 2000.

PAGE 43

43 literacy skills through reading, storytelling, and writing activities and through formal KidzLit Passport to Success and the Madison Square Garden Literacy Challenge. 59 Research ers such as Cooper and Fashola studied different types of after school program structures ( after school extended day and summer programs) to identify evidence of effectiveness. 60 An additional meta analysis by McRel identified five possible characteristics of after school programs that had moderating influences on effect sizes. 61 However, the relationship between program delivery length (time and hours) and instructional strategy utilized had limited research findings. 62 Typical after school programs had shor t program delivery durations (less than forty five hours) as compared to summer school programs and therefore, were associated with lower effect sizes. 63 Yet, after school programs tended to conduct more small group and one on one instruction. 64 Because smal l groups and one on one instruction were associated with 59 Jennifer Birmingham et al. 2005 60 See e.g. Olatokunbo Fashola, Review of Extended Day and After School Programs and Their Effectiveness (Baltimore, MD: Johns Hopkins University, 1998), CRESPAR, 24 and Cooper, Ha rris, Kelly Charlton, Jeff Valentine, La ura Muhlenbruck and Geoffrey Borman, "Making t he Most of Summer School: A Meta Analytic and Narrative Review," Monographs of the Society for Research in Child Development 65, no. 1 (2000). 61 McR el 2004. 62 Ibid 63 Ibid 64 Patricia Lauer, Motoko Akiba, Stephanie Wilkerson, Helen Apthorp, David Snow and Mya Martin Glenn, "Out of School Time Programs: A Meta Analysis of Effects for At Risk Students," Review of Educational Research 76, no. 2 (Summer 2006)

PAGE 44

44 more positive effects compared to large groups, the benefits of summer schools of longer duration may have been offset by the use of large group instruction. 65 Several studies suggested that the effec tiveness of after school programs might vary depending on the grade levels of students. 66 For reading, the largest positive effect size (.26) occurred for students in the lower elementary grades. 67 The results suggested the importance of early intervention f or students who were struggling with reading acquisition. The focus within program activities whether on academics only or academics plus social produced mixed findings. 68 Boys in first through third grades displayed poorer reading and math grades, poorer w ork habits and more behavior problems when provided a diverse array of activities. 69 Yet, overall common practices identified in high functioning after school programs reported offering three to four activities each day. 70 Most programs included homework ass istance or other academic activities (e.g., math, reading, writing), to enrichment (e.g., cooking, arts and crafts, computers) to recreation 65 Patricia Lauer, Motoko Akiba, Stephanie Wilkerson, Helen Apthorp, David Snow and Mya Martin Glenn, "Out of School Time Programs: A Meta Analysis of Effects for At Risk Students," Review of Educational Research 76, no. 2 (Summer 2006) 66 See e.g. Harris Cooper et al., 2000, and Jean Baldwin Grossman, Karen Walker and Rebecca Raley, Challenges and Opportunities in After school Programs: Lessons for Policy Makers and Funders (Philadelphia, PA: Manpower Demonstration Research Corporation), P/PV. 67 Patricia Lauer et al., Summer 2006 68 Ibid 69 Kim Pierce, Daniel Bolt, and Deborah Lowe Vandell, "Specific Features of Af ter school Program Quality: Associations with Children's Functioning in Middle Childhood," American Journal of Community Psychology 45 (2010). 70 Denise Huang et al. 2010.

PAGE 45

45 (e.g., sports, dance, games). Some high performing programs focused on exposing participants to more humanistic com ponents including opportunities that built positive cross age peer relationships and contact with positive adult role models. 71 Effect sizes were significant in terms of program duration which measured the potential for student exposure to after school acti vities. Programs that were more than forty five hours in duration had statistically significant effect sizes for reading and mathematics. 72 However, programs that lasted more than 210 hours for reading and more than 100 hours for mathematics had effect size s that were not significantly different from zero. 73 One of the largest positive effect sizes (.50) occurred in programs that used one on one tutoring in reading. 74 In 2005, the Northwest Regional Educational Laboratory (NWREL) conducted an extensive literat ure review on literacy practices and outcomes within an after school context. Researchers concluded that tutoring and homework help components in after school settings, particularly those focused on literacy skills, had a ng scores. 75 71 Jennifer Birmingham et al. 2005 72 Patricia Lauer et al., Summer 2006 73 Ibid 74 Ibid 75 Brenda Britsch, Nicky Martin, Amy Stuczynski, Bethany Tomala and Patti Tucci, Literacy in After school Programs (Austin, TX: Northwest Regional Educational Laboratory, Southwest Educational Development Laboratory, 2005), NWREL.

PAGE 46

46 Policy Reports on SES In 2009, the United States Department of Education contracted with RAND and American Institutes for Research (AIR) to prepare an updated report on the trends of Title I parental school choice options and supplemental educa tional service components through 2006 07. Data were gathered from state level interviews and a nationally representative sample survey of Title I school district officials, principals, teachers and parents as well as student achievement data from nine sch ool districts. Approximately 3.3 million students or 13 % of K 12 students in Title I schools were eligible for Title I supplemental educational services in 2006 07. 76 This was a 2 % increase (11 % to 13 % ) than in the previous year. 77 Nationwide, 14 % of Title I schools were required to offer SES during this same time period. 78 In addition, 29 % of high poverty Title I schools (schools with 75 % or more of students receiving free or reduced priced lunch) had students eligible for SES, compared with 3 % of low poverty Title I schools. 79 A larger proportion of urban districts (25 % ) had schools required to offer supplemental services than did suburban districts (12 % ) or rural districts (6 % ). 80 In terms of grade levels served, 11 % of Title I elementary schools were eligibl e to offer SES as compared to 24 % of Title I middle schools and 11 % of Title I high 76 Scott Naftel et al. 2009 77 Ibid 78 Ibid 79 Ibid 80 Ibid

PAGE 47

47 schools. 81 Of those eligible Title I elementary schools, 75 % offered SES to students. 82 At eligible Title I high schools, approximately 33 % offered SES to students. 83 In 2009, the National Longitudinal Study of No Child Left Behind (NLS NCLB ) provided student achievement data from nine large urban school districts to the United States Department of Education in order to examine the relationship between SES and stu dent achieveme nt. Using a quasi experimental fixed effects model, the analysis compared achievement of individual students before and after participating in SES with those of nonparticipating students. 84 On average, students participating in SES during one school year ha d an effect size measuring 0.08 of a standard deviation unit in reading. 85 For those students that had two or more years of services, the effect size was .18 of a standard deviation in reading, with the significance level set at the 5 % level. 86 In addition, students with 81 Scott Naftel et al. 2009 82 Ibid 83 Ibid 84 Ron Zimmer, Brian Gill, Paul Razquin, Kevin Booker, and J.R. Lockwood III, State and Local Implementation of the No Child Left Behind Act: Volume I Title I School Choice, Supplemental Educational Services, and Student Achievement (Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, 2007), Washington, D.C. 85 Stephanie Stullich, Andrew Abrams, Elizabeth Eisner, and Erica Lee, Title I Impl ementation Update on Recent Evaluation Findings (Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development and Program Studies Service, 2009), Washington, D.C. 86 Ibid.

PAGE 48

48 disabilities, African American students, and Hispanic students all demonstrated positive achievement effects from participating in SES. 87 Tennessee State Wide SES Effectiveness In 2008, the State of Tennessee through the Center for Research on Educational Policy, University of Memphis, conducted an SES study on the effectiveness of the program statewide. Student achievement scores of SES students were compared to those of non SES student s. Researchers utilized a multi linear regression model to participation, prior achievement, and other characteristics such as free or reduced price lunch, gender, and ethnicity. 88 The Tennessee Comprehens ive Assessment Program (TC AP) was a criterion referenced test mandated by the State of Tennessee for assessing AYP in Reading/Language Arts and Mathematics, in grades 3 8. Students took the multiple choice assessment each spring. The Tennessee SES evaluation included subjects and s tudents having sufficient prior year scores, involved with providers that had at least 20 students and received tutoring from only one provider. Because the sample size for students in grades 9 12 was too small to support valid inferences and students in g rade 3 had no prior achievement scores, the final sample size included students in grades 4 8. Tutoring information was then merged with student data. 87 Stephanie Stullich, Andrew Abrams, Elizabeth Eisner, and Erica Lee, Title I Implementation Update on Recent Evaluation Findings (Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development and Program Studies Service, 2009), Washington, D.C 88 Steven M. Ross et al., 2008

PAGE 49

49 Final statistical analyses demonstrated the effects of tutoring on student achievement to be small and no t significantly different from zero. 89 However, the researchers acknowledged their underlying assumption that thirty to forty hours of after school tutoring was not likely to produce strong effect sizes. Consequently, student participation hours were not in cluded in the study. Los Angeles Unified School District SES Effectiveness Los Angeles Unified School District (LAUSD) through its department of Program Evaluation and Research Branch (PERB) conducted a study on the impact of SES participation on student achievement for the 2005 06 school year. Almost a year later, In 2005 06, approximately 11 % (23,086) of eligible students (216,192) applied to an SES program. 90 Due to the fact that on e third of students who applied to an SES program did not actually attend the program, final participation numbers resulted in 14,759 students being served. 91 English only students were more likely to apply, while students designated as Fluent English Profi cient (RFEP) were less likely to apply. 92 In addition, students with disabilities (SWD) were more likely to apply than students without disabilities. 93 In total, English/Language Arts (ELA) programs accounted for 6,538 students, and mathematics programs acco unted for 8,221 students. 89 Steven M. Ross et al., 2008 90 Jordan H. Rickles and Melissa K. Barnhart, 2007. 91 Ibid 92 Ibid 93 Ibid

PAGE 50

50 In terms of attendance, students were classified as low, medium or high attendance participants. Students with low attendance participated in 1 % to 49 % of a icipated in 50 % to 89 % participated in 90 % to 100 % 94 Based upon final student data, approximately 19 % of students were classified as low attendance participants, 14 % of students were medi um attendance participants and 67 % of students were high attendance participants. 95 Attendance distribution was nearly identical for ELA and math programs. Researchers adjusted for differences in student achievement/test performance data for previous perfor mance and other student characteristics. The results were compared to differences of an average student in a given group to standard scale score deviations of the average student in the total population. With a p value <0.05, results indicated that a diffe rence in adjusted test scores existed between students who attended SES and those who applied but did not attend. 96 ELA test scores demonstrated a 0.02 scale score improvement for students who attended an SES program and a 0.02 scale score improvement for students who applied but did not attend. 97 The impact of program attendance was less clear. Students with low to medium attendance did not have scores statistically different from students who did not attend 94 Jordan H. Rickles and Melissa K. Barnhart, 2007 95 Ibid 96 Ibid 97 Ibid

PAGE 51

51 the program. 98 Of students that participated in SE S programs, students who attended who attended at least 90 % 99 The majority of improved student performance occurred among elementary students. In addition, students from English only backgrounds had higher participation rates. Chicago Public Schools SES Effectiveness After three years of implementing SES tutoring, the Chicago Public School system through its Office of R esearch, Evaluation and Accountability released its 2007 program evaluation report. Student achievement data were analyzed from the 2005 06 school year and included 41,645 students from 324 schools. 100 However, over 230,000 CPS students were eligible for SES services, and approximately 75,000 registered to receive tutoring. 101 Students were prioritized according to district goals. Initial SES services were offered to third grade and high school students. Remaining SES slots were offered to students who performe d the lowest on the 2005 Iowa Tests of Basic Skills (ITBS). 102 Overall, approximately 61 % of all SES participants scored at or below the 25 th percentile on the ITBS reading subtest. 103 Students in SES programs were 98 Jordan H. Rickles and Melissa K. Barnhart, 2007 99 Ibid 100 Chicago Publ ic Schools, SES Tutoring Programs: An Evaluation of Year 3 in the Chicago Public Schools (Chicago: Office of Research, Evaluation, and Accountability, 2007), CPS. 101 Ibid 102 Ibid 103 Ibid

PAGE 52

52 more likely to be African American and less l ikely to be white or Asian as compared to the rest of the school district. 104 The impact of SES on student achievement was analyzed based upon changes in reading and mathematics scale scores from the 2004 05 ITBS to the new state test, Illinois Standards Ach ievement Test (ISAT), in 2005 06. Only students in grades 3 through 8 were included in the analyses due to the fact that ISAT was administered to those limited grade levels. 105 Additional anal yses criteria excluded English Language L earners and SES participa nts that received less than thirty hours of tutoring. 106 Based upon 2005 06 ISAT reading performance scale scores, the achievement of SES participants demonstrated a significant improvement (0.8 adjusted points) as compared to eligible students who did not participate in SES. 107 However, the results had only a small effect on scores given the average student scored +/ 15.60 adjusted scale score points different from zero 108 Overall, model interactions suggested that students with disabilities gained more throug h SES participation and that sixth and seventh graders benefited more from SES in their math achievement. 109 104 Chicago Public Schools, SES Tutoring Programs: An Evaluation of Year 3 in the Chicago Public Schools (Chicago: Office of Research, Evaluation, and Accountability, 2007), CPS 105 Ibid 106 Ibid 107 Ibid 108 Ibid 109 Ibid

PAGE 53

53 Forty two SES programs tutored students during the 2005 06 school year. The cost per student ranged from $375 to $1,866.94. 110 The number of tutoring h ours available to students ranged from 32 to 120. 111 To measure the cost effectiveness among SES providers, the residual achievement scale scores obtained in the analysis of effect size were compared with the relative cost per student in each program. 112 With the exception of three programs, students tutored in more expensive programs performed similarly to or did not perform as well as did students in the least expensive program. 113 Spending more money did not consistently demonstrate greater positive changes in student achievement. Milwaukee Public Schools SES Effectiveness Milwaukee Public Schools in Milwaukee, Wisconsin, accounted for nearly four fifths of the schools identified for improvement in the state of Wisconsin during 2002 to 2007. 114 To evaluate the ef fect of SES, gains in student achievement from five different standardized tests were analyzed from school years 2004 05, 2005 06, 2006 07, and 2007 08, depending on the grade level of students. Tests included an older version of the Wisconsin Knowledge an d Concepts Examination (WKCE) in grade 8, and the 110 Chicago Public Schools, SES Tutoring Programs: An Evaluation of Year 3 in the Chicago Public Schools (Chicago: Offic e of Research, Evaluation, and Accountability, 2007), CPS 111 Ibid 112 Ibid 113 Chicago Public Schools, 2007. 114 Carolyn Heinrich, Robert Meyer and Greg Whitten, June 2010

PAGE 54

54 TerraNova in grades 6, 7, and 9. 115 After 2005, the revised version of the WKCE was administered in grades 3 through 8 and grade 10. 116 In 2005 06, all students enrolled in schools required to offer SES took fa ll and spring semester AGS Group Reading Assessment and Diagnostic Evaluation assessments. 117 In 2007 08, ninth grade students took a ThinkLink examination. 118 To identify the effects of SES on student achievement, eligible students who registered for SES were compared with eligible students who did not register for SES. Logistical regressions were calculated to account for differences between the groups, including differences in student characteristics. 119 Analyses also included fixed effects modeling specificat ion to account for the possibility that students who participated in SES differed from nonparticipants in ways that correlated with achievement growth. 120 Linearly rescaling test scores across in different years and grades provided data of similar variance a cross grades. 121 The estimated effects of attending an SES on changes in reading and mathematics in 2004 05, 2005 06, and 2006 07 school years were calculated using a statistical significance p value at < .05. 122 Both matched and unmatched student 115 Carolyn Heinrich, Robert Meyer and Greg Whitten, June 2010 116 Ibid 117 Ibid 118 Ibid 119 Ibid 120 I bid 121 Ibid 122 Ibid

PAGE 55

55 achievement r esults were reported for middle and high schools students. After matching, results demonstrated no statistically significant differences in changes in test scores for students who attended SES compared to those students who did not attend any SES. 123 In ter ms of intensity of service, as measured by number of hours in SES attendance, results demonstrated some relationship to increased student achievement. 124 Student attendance hours in SES ranged from 1 to 110 hours of tutoring with the average number of hours as low as thirteen for high schools students and as high as thirty for middle school students. 125 Registered Hispanic students who were not English proficient and attended middle school were significantly more likely to attend SES. 126 One positive estimated ef fect was documented in terms of total hours attended and reading achievement scores. In 2005 06, for each additional hour 0.087 of a test unit. 127 In middle school, students who received at least 20 hours of SES as compared to fewer than 20 hours demonstrated significantly higher math scores in 2004 05. 128 123 Carolyn Heinrich, Robert Meyer and Greg Whitten, June 2010 124 Ibid 125 Ibid 126 Ibid 127 Ibid 128 Ibid

PAGE 56

56 Educational Funding With the passage of the Elementary and Secondary Education Act in 1965, a variety of federal programs w ere created to support the goals of federal education while targeting additional resources to meet the needs of school aged children who were economically and educationally disadvantaged. Title I, Part A was and continues to be the largest program that the U.S. Department of Education appropriated for elementary and secondary programs. In 2004 05, Title I, Part A encompassed 33 % of federal funds to states on the basis of statutory formulas. 129 Statutory formula also considered ifferences and per pupil expenditure factor amounts. 130 After reserving funds for state level activities and other required set asides, states suballocated funds to school districts and other subgrantees. 131 For Title I, school districts distributed funds to schools with high concentrations of low income students, typically based upon free or reduced price lunch program. 132 Districts could have provided schools with different low income per pupil amounts as long as schools with higher poverty rates received high er allocations. 133 129 Jay Chambers, Irene Lam, Kanya Mahitivanichcha, Phil Esra, Larisa Shambaugh, American Institutes for Research, State and Local Implementatio n of the No Child Left Behind Act Volume VI Targeting and Used of Federal Education Funds (Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, 2009), AIR, ED 00 CO 0120. 130 Jay Chambers et al. 2009. 131 Ibid 132 Ibid 133 Ibid

PAGE 57

57 Based upon data collected by the Study of Educational Resources and Federal Funding (SERFF), the American Institutes for Research (AIR) examined Title I funding from 1997 98 to 2005. After adjusting for inflation, Title I allocations per poor child substantially increased in the highest poverty school districts; however, this change largely reflected overall allocation growth in Title I appropriations during that time period. 134 In fact, the overall share of Title I funds going to the highes t poverty schools districts only changed slightly between 1997 98 and 2004 05, increasing from 50 % to 52 % 135 When analyzed from a Title I per poor child allocation perspective, the highest poverty school districts received an average of $1,579 per child as compared to $1,256 in the lowest poverty school districts. 136 In terms of distribution revenues, districts in the highest poverty quartile which represented 49 % age children received 38 % of all federal funds, 26 % of state revenue s, and 15 % of local revenue. 137 In contrast, districts in the lowest poverty quartile which represented 7 % age children received 12 % of all federal funds, 22 % of state funds, and 37 % of local funds. 138 The discrepancy between local funding percentages accounted for the fact that local funds primarily drew from property taxes where lower poverty school districts had a higher 134 Jay Chambers et al. 2009 135 Ibid 136 Ibid 137 Ibid 138 Ibid

PAGE 58

58 revenue source. State revenues compensated partially but not fully for local funding disparities. In 2004 05, highest poverty school districts received over three times more federal funding per student ($1,449) than the lowest poverty school district ($388). 139 Funding differences per student at the state level indicated similar results ($5,478 vs. $3,973). 140 However local revenue fund results exhibited a different funding imbalance. The highest poverty school districts received less than half as much local revenue per student ($3,098) as compared to the lowest poverty school district ($6,475). 141 At the school level, federal funding per low income student decreased due to the fact that different poverty measures were used. Allocations at the school level utilized free or reduced price lunch data as compared to district level analysis which utilized Census Bureau estima tes of poor school age children. 142 The free or reduced price lunch program had a higher income threshold and therefore included more children. In 2004 05, the average Title I district allocation was $1,499 but only $796 per low income student at the school level. 143 In addition, district level Title I funding allowed districts to retain some funds for program administration and required districts to retain some funds for certain programs such as SES 139 Jay Chambers et al. 2009 140 Ibid 141 Ibid 142 Ibid 143 Ibid

PAGE 59

59 In 2004 05, within district funding demonstrated that low in come student allocations were similar or higher in higher poverty Title I schools as compared to lower poverty Title I schools. 144 In a sample of 242 districts (8,536 Title I schools), districts provided an average of $634 per low income student in Title I s chools with above average district poverty rates as compared to $558 in Title I schools with below average poverty rates, a 14 % higher than average allocation to above average poverty Title I schools. 145 An additional analysis indicated that in school distri cts with ten or more Title I schools, 49 % of districts demonstrated a positive relationship between Title I funds per low income student and school poverty while most of the remaining districts demonstrated no statistically significant relationship. 146 Yet, other within district studies indicated that poor and minority students have not received their fair share. 147 In an era of high academic standards, providing support to schools and students in the greatest academic need still remains an educational fiscal and policy challenge, which may lead to a new focus within fiscal equity litigation. 148 144 Jay Chambers et al. 2009 145 Ibid 146 Ibid 147 See e.g., Ross Rubenstein, Amy Ellen Schwartz and Leanna Stiefel, Rethinking the Intr adistrict Distribution of School Inputs to Disadvantaged Students (Berkeley, CA: University of California, Berkeley, Boalt Hall School of Law, 2006) and William Owings and Leslie Kaplan, "The Alpha and Omega Syndrome: Is Intra district Funding the Next Rip eness Factor?," Journal of Education Finance 36, no. 2 (Fall 2010). 148 William Owings and Leslie Kaplan, Fall 2010

PAGE 60

60 But does money matter in education? Researchers such as Ledges, Lane and Greenwald concluded that spending additional money on education increased student achievement. 149 On the other hand, Hanushek suggested that money matters little in student achievement. 150 Still others have analyzed fiscal expenditures in education and concluded that what really mattered is how school districts and school sites actually spent money, basi cally quality over quantity. 151 A 1996 study conducted in Florida examined resource allocation patterns among high performing high schools as compared to the remaining high schools in the state. Due to the equalization school funding formula used in Florida, data indicated a statistically insignificant result between per pupil spending and per pupil spending for instruction. 152 However, the percentage of expenditures devoted to instruction was lower in the high performing high schools. These results indicated t he high performing schools may have spent more money on resources not directly linked to instruction or may have spent instructional resources differently as compared to the other high schools. 153 149 Larry V. Hedges, Richard D. Laine, and Rob Greenwald, "Does Money Matter? A Meta Analysis of Studies of the Effects of Differential School In puts on Student Outcomes," Educational Researcher 23, no. 3 (April 1994). 150 Eric A. Hanushek, "The Impact of Differential Expenditures on School Performance," Educational Researcher (May 1989). 151 Lawrence O. Picus, In Search of More Productive Schools: A Gui de to Resource Allocation in Education (Eugene, OR: ERIC Clearinghouse on Educational Management, University of Oreg on, 2001), ERIC. 152 Lawrence O. Picus, 2001. 153 U.S. Department of Education. National Center for Education Statistics. Assessment and Analysis of School Level Expenditures, Working Paper No. 96 19, by Joel D. Sherman, Cla yton Best, and Lawrence Luskin, Washington, D.C.: 1996.

PAGE 61

61 Cost Efficiency in Education Regardless of what impact additi onal funds might have, it is important that existing resources be used as efficiently as possible. Improving student performance, with or without funds, requires four ingredients: 1. Reallocation of existing resources 2. Incentives for improved performance 3. Dev systems and 4. A more market based budgeting environment 154 Through the creative use of categorical funds, elimination of classroom aides and reallocation of resources such as the elimination of one or more teaching positions, researchers Odden and Busch concluded that schools generated an additional $82,000 to $349,000 in school based funds. 155 Subsequently, schools intensified use of remaining staff and funneled these additional funds toward o ther mechanisms such as increased professional development activities that ultimately resulted in improved student performance. 156 Accountability systems were based on an assumption that a focus on student outcomes would lead to behavioral changes by student s, teachers, and schools due to a heightened public reporting process. Another assumption of accountability systems was dependent on the development of explicit incentives that would lead to innovation, 154 Lawrence O. Picus, 2001. 155 Allen Odden and Carolyn Busch, Financing Schools for High Performance: Improving the Use of Edu cational Resources ( Jossey Bass, 1998). 156 Allen Odden and Carolyn Busch 1998

PAGE 62

62 efficiency, and fixes to any observed performance pro blems. 157 Unfortunately, incentives that appeared to have the most success were sanctions for low performing schools while high performing schools were exempt from sanctions. 158 Market based structures are a powerful tool for improving the performance and effi ciency of an organization. However, current models for market structures in public education such as school choice, vouchers, and supplemental educational services have yielded mixed r esults. Researchers such as Pic us suggested that schools implementing su ccessful programs will meet goals while those selecting inappropriate programs will go out of business as evident in the premise of competition within market based economies. 159 However, according to Steinberg, anticompetitive forces appeared to be prevalent among SES providers. 160 As more school districts complied with federal sanctions, the sheer number of SES providers grew exponentially. Only a few firms had the revenue and size to capture most of the market activity. With a lack of competition for student enrollment and few state accountability mechanisms in place for SES, SES providers had little incentive to function efficiently. 161 157 Eric A. Hanushek and Margaret Raymond, The Confusing World of Educational Accountability (Palo Alt o, CA: Hoover Institution, Stan ford University, 2001), Hoover Institution. 158 Law rence O. Picus, 2001. 159 Ibid 160 Matthew Steinburg, Patricia Burch, and Joseph Donovan, "Supplemental Educational Services and NCLB: Policy Assumptions, Market Pra c tices and Emerging Issues," Educational Evaluation and Policy Analysis 29, no. 2 (June 2007). 161 Matthew Steinburg, Patricia Burch, and Joseph Donovan, June 2007.

PAGE 63

63 As a theory of fair competition within a just society, the distributive results of a market economy are just even if unequal (as long as they flow from just institutions). 162 Hence, society provides equality of opportunity. Yet, education functions within a social and political context where complex moral theories about justice apply; thereby impacting implementation of policy and its design. 163 Educational funding formulas attempted to address the various views of justice through the treatment of equals equally (horizontal equity) as well as the treatment of unequals unequally (vertical equity). 164 But, did this view equate to efficie ncy, especially economic efficiency? Research literature has traditionally argued that in many special education funding cases, efficiency counted, but that was not all that counted. Social justice many times overrode efficiency. 165 Should increasing access to after school tutoring among disadvantaged students also override efficiency? Additional research is needed to fully evaluate the current models of market based mechanisms and the impact on efficiency and social justice in public education. 162 Kenneth Strike, "Equality of Opportunity and School Finance: A Com mentary on Ladd, Satz, and Brig house and Swift," Education Finance and Policy 3, no. 4 (Fall 2008). 163 Patricia Burch, Caroly n Heinrich, Annalee Good, Mary Stewart, Equal Access to Quality in Federally Mandated Tutoring: Preliminary Findings of a Multisite Study of Supplemental Educational Services (Madison, WI: University of Wisconsin, Madison 2011 ), Wisconsin Center for Educa tional Research. 164 R. Craig Wood, Educational Finance Law: Constitutional Challenges to State Aid Plans An Analysis of Strategies (Dayton, OH: Education Law Association, 2007). 165 Kenneth Strike, Fall 2008

PAGE 64

64 Summary This chapter was divided into four sections. The first section provided a review of the related literature including after school research. The second section provided the relatively limited results of previous SES studies, including those conducted at the stat e and federal levels. The third section provided a summary of research perspectives in educational funding. The final section summarized the concept of cost efficiency in education. Chapter 3 includes a discussion on the concept of student learning gains, and the methodology used in this stud y.

PAGE 65

65 CHAPTER 3 METHODOLOGY Under the No Child Left Behind Act (NCLB) of 2001, school districts that received federal funding must maintain adequate yearly progress (AYP) of students or face strict sanctions. One sanction mandated that the school district provide Supplemental Educational Services or SES. Specifically, students who enrolled in Title I schools that failed to meet AYP benchmarks for three consecutive years had the option of receiving free, after school tutorin g interventions via SES. Once approved by state departments of education, public and private SES providers were eligible to provide the academic interventions. NCLB policy makers claimed that the competitive, market based nature of SES would provide qualit y tutoring service choices and ultimately, positively impact student academic achievement. However, anticompetitive forces appeared to be prevalent among SES providers. 1 As more school districts complied with federal sanctions, the sheer number of SES prov iders grew exponentially. Only a few firms had the revenue and size to capture most of the market activity. In a study of a large urban school district (from 2004 to 2006), researchers suggested that the larger SES firms had larger class sizes and charged higher hourly rates. 2 Because funding for SES was based upon eligible per pupil allocations, students actually received fewer hours of services when SES providers charged higher hourly rates. During the 2008 09 school year, SES providers 1 Matthew Steinberg Patricia Burch and Joseph Don ovan, "Supplemental Educational Services and NCLB: Policy Assumptions, Market Practices, Emerging Issues," Educational Evaluation and Policy Analysis 29, no. 115 (2007). 2 Carolyn Heinrich Robert Meyer, and Gregory Whitten, June 2010

PAGE 66

66 in the State of Fl orida provided an average of fourteen hours of tutoring in reading and ten hours of tutoring in mathematics per student. 3 The SES providers in the selected school district for this study provided an average of fourteen hours in reading and seven hours of m athematics tutoring during the same time period, with a median of thirteen hours and seven hours respectively. 4 A survey of Title I administrators in 2006 07 indicated that a majority of states had monitoring systems in place for SES providers. Of those, o nly a few had databases containing student achievement and participation information that permitted rigorous evaluations of achievement effects. 5 These concerns pointed to a mismatch between policy implementation of SES and the concerns for quality, effici ency, and equity claimed as priorities in federal law. Purpose of the Study The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the selected school district. The following questions are addressed in this study: 1. How did reading participation in SES correlate to student reading learning gains in a selected school district? 2. How did different levels of reading participation hours within SES correlate to student learning gains in reading in a selected school district? 3 Deepak Gajre, Novemb er 02, 2010. 4 Ibid 5 Scott Naftel et al. 2009.

PAGE 67

67 3. How did reading learning gains of students who participated in SES compare to reading learnin g gains of similar students who did not participate in SES in a selected school district? Overview of the Method This study utilized existing student FCAT scores and hours spent in SES reading programs to assess the efficiency of SES tutoring in grades 4 1 2 in a selected school district in an effort to align this study with an anticipated increase in school districts becoming their own SES provider in the state of Florida. A letter dated May 6, 2011, and sent to the United States Department of Education by Frances Haithcock, Chancellor of Public Schools in Florida, requested a waiver to the prohibition on approving an that identified schools and LEAs may be able to establi sh that they have an effective program that can help improve the academic achievement of students and should not be prevented automatically from gaining approval simply because of their improvement 6 In February 2012, the United Stated Department o f Education approved 7 However, both Florida S tatute 8 and Florida 6 Frances Haithcock, Tallahass e e, Florida, to T helma Melendez de Santa Anna, May 6, 2011, U.S. Department of Education, Office of Elementary and Secondary Education, Washington, D.C. accessed http://www.fldoe.org/bsa/pdf/2011WaiverRequest DINI SINI.pdf 7 Michael Yudin, Washington, D.C. to Gerar d Robinson, February 22, 2012, Florida Department of Education, Commissioner, Tallahassee, FL accessed http://www2.ed.gov/policy/eseaflex/implementation letters/fl implementation letter.pdf 8 Florida Statute §1008.331 (2012).

PAGE 68

68 Administrative Code 9 required SES services. Subsequently, t he 2012 13 Request for Application for Supplemental Educational Services 10 from the Florida Department of Education requested application classification information from potential SES providers which included an opportunity for school districts (or Local Ed ucational Agency, LEA.) Included in this study were the results of nationally recognized after school programs and SES program results gathered from federally commissioned reports, school district program evaluations and peer reviewed journal articles. P articipating District Statistics The selected school district in this study had a total student enrollment of 43,343. The number of students who received free or reduced price lunch was 26,986 or 62 % of the student population. Title I funds were provided t o secondary schools with 59 % or higher rate of students eligible for free or reduced price lunch while elementary schools with 75 % or higher rate of students eligible received funds. During the 2008 09 school year, all schools in the selected school distri ct with 75 % or higher rate of students eligible for free or reduced price lunch received funds. During the same time period, 79 % (n= 14) of Title I schools were required to offer SES. The ethnic makeup of the student population district wide was comprised of 44 % Hispanic, 40 % white, 12 % African American, 2 % mixed, 1 % Asian and 1 % Indian. In grades 9 12, the total annual dropout rate was 2.0 % The 4 year graduation rate in this district was 80.4 % English was not the first language for 13 % of the student pop ulation with 6,000 students in the 9 Florida Administra tive Code 6A 1.039 (2011). 10 Florida Department of Education, Request for Applications for Supplemental Educational Service Providers 2012 2013 School Year (Tallahassee, FL: FLDE, 2012).

PAGE 69

69 English Language Learner (ELL) program. Forty eight percent of the student population c ame from non English speaking homes. Collectively, these students spoke sixty one different languages and hailed from 147 different co untries of origin. Exceptional student education constituted 20 % of the student population, with 13 % encompassing students with disabilities and 7 % involving students with gifted abilities. The district employed nearly 3,000 highly qualified teachers with 49 % having advanced degrees. Data Organization Prior to submitting a Research and Data Release Procedure application to the selected school district, permission to conduct research was obtained from the University of Florida Institutional Review Board. The study did not involve direct student contact but evaluated existing FCAT scores and time spent in SES services. Student identification numbers were deleted. Therefore, individual students were not identifiable and informed consent was not required. The me thodology included using reading development scale scores from 2008 Accountability Program (2008) which reversed the order of school choice and SES sanctions in Title I schools after two consecutive years of failing to make adequate yearly progress. Thus, data from these years of FCAT administration provided greater validity when compared to other published SES program reports during the same or prior timeframe. Developmental sca le scores (DSS) were selected as the unit of measure because the data were an effective tool in tracking student progress over a

PAGE 70

70 gains. 11 00 for all grade levels tested and were only calculated when a student had two years of FCAT data. 12 SES data regarding eligible students, participating students, and attendance hours were gathered from the selected Procedu res The reading developmental scale score from the 2008 and 2009 FCAT Reading administrations were used as pre and post tests. A qua si experimental design was utilized. The experimental group participated in SES reading programs, and students in the contro l group did not participate in the tutoring program. Students who did not participate in both the 2008 and 2009 administration of the FCAT were eliminated from the study. Students who received tutoring in SES reading programs were placed in the experimenta l group. The control group was randomly selected utilizing the Statistical Analysis System (SAS) software and was resampled for each analysis. Included in the control group were students who were qualified to receive SES tutoring in reading, but the parent s of these students chose not to enroll them as well as students who applied for SES tutoring in reading but did not receive tutoring services in reading. To answer the research questions addressed in this study both parametric and nonparametric statistica l analyses were conducted. A correlation coefficient was run to determine if there were statistically significant increases in FCAT Reading 11 Cornelia Orr, Assessment and Accou n tability Briefing Book FCAT, Scho ol Accountability and Teacher Certification Tests ed. (Tallahass e e, FL: Florida Department of Education, 2007). 12 Ibid

PAGE 71

71 developmental scale scores from the 2008 to 2009 administration, based upon the ng only. Computations were performed using SAS software and tested at a .05 level of significance. In determining the effects of differing levels of SES reading participation hours on on coefficient was applied using subgroups of students. Subgroups included students who enrolled in reading SES but received zero hours of tutoring, students who were enrolled in reading SES with greater than zero hours but less than thirteen hours of tuto ring, students who were enrolled in reading SES with greater than thirteen hours of tutoring and students who were not enrolled in reading SES. Spearman rho measured the statistical dependence between SES reading hours and FCAT Reading developmental scale scores. To analyze student learning gains in reading when comparing students who participated in SES reading tutoring (greater than zero hours) with students who did not receive SES reading tutoring a Satterthwaite was applied to determine if a significan t relationship existed. Additionally, a nonparametric technique, Chi Square, was applied to determine if other student characteristics such as English for Speakers of Others Languages ( ESOL ) or students with disabilities ( SWD ) status impacted SES participa tion. This nonparametric statistical procedure was applied when data were measured on nominal (classificatory) scales. Summary This chapter provided an overview of the study by discussing the need to evaluate SES to determine its impact on improving the re ading achievement of students receiving after school tutoring. Demographic information was presented on the selected

PAGE 72

72 school district as well as data elements utilized in this study. Finally, a discussion of the procedures was described. Results of the data analysis are analyzed in Chapter 4.

PAGE 73

73 CHAPTER 4 RESEARCH FINDINGS The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, FCAT data were obtained from the selected school district. In reporting the results of this study, student achievement data were not identified by student name; however, student characteristics such as grade level, ESE and ESOL status were considered. This study utilized FCAT data from the selected school district in a quasi experimental design. FCAT Reading developmental scale scores for 2008 and 2009 were used as pre and posttests Students who attended Title I sc hools that were required to provide SES were divided into two groups receiving free or reduced price lunch in grades 4 12. Both groups were required to have FCAT scores from the 2008 and 2009 administration for reading. Group One the experimental group, r eceived some SES tutoring hours in reading. Group Two the control group, did not receive any SES tutoring hours in reading. In the experimental group and the control group, each 2009 score and the net result was used in the analysis. Subsequent group analyses of student achievement results included four categories: students who signed up for SES tutoring in reading but did not receive services, students who received greater than zero hours but less than thirteen SES tutoring in r eading, students who received thirteen or greater hours of SES tutoring in reading, and students who did not receive SES tutoring in reading.

PAGE 74

7 4 Research Questions The following research questions were addres sed in the study: 1. How did reading participation in SES correlate to student reading learning gains in a selected school district? 2. How did different levels of reading participation hours within SES correlate to student learning gains in reading in a selecte d school district? 3. How did reading learning gains of students who participated in SES compare to reading learning gains of similar students who did not participate in SES in a selected school district? Analysis and Quantitative Results FCAT reading data fr om the selected school district were analyzed to assess the cost efficiency of the SES program in raising reading student achievement. The data file contained complete data by grade level for 5,033 students who had FCAT reading scores in 2007 08 or 2008 09 and who attended a Title I school that was required to offer SES. Table 4 1 reported the number of students represented at each grade level. Table 4 2 reported the mean and standard deviation of reading developmental scale scores (DSS) in 2008 09. Reading DSS were slightly more variable on 2007 08 (mean score = 1512.45, SD = 348.86) as compared to 2008 09. Tables 4 3 and 4 4 reported the number of students who were identified as students with disabilities (SWD) and students actively enrolled in an ELL prog ram, respectively. SWD data indicated a 1.68 % higher identification rate between the Title I student population and the overall student population in the selected school district. In addition, students actively enrolled in an ELL program constituted 17.54 % of the Title I student population used in this study. Of the 5,033 student population, 1,027 (23.98 % ) students had enrolled in SES reading. When assessing the reading gains for students who participated in reading SES, the experimental group further elim inated 859 students. These students enrolled

PAGE 75

75 in SES but did not receive tutoring services. The final experimental data set contained 348 students with recorded SES reading hours greater than zero. Table 4 5 reported the 2008 09 FCAT reading developmental s cale score gains by grade level with two predictor variables of 2007 08 FCAT reading developmental scale score gains and SES reading participation. In every grade level, the 2008 FCAT Reading score predicted the 2009 FCAT Reading score, with the exception of grade four. In grade four, reading SES participation predicted 2009 FCAT Reading scores. Table 4 6 reported the mean, median and mode of reading SES hours for students. The average number of SES reading hours was fourteen while the most frequently occur ring number of hours was twenty However, the median of thirteen SES reading hours was selected in further data analyses to eliminate the effect of outliers. Based upon thirteen hours of SES reading, the student population of 5,033 was divided into four su bgroups. Subgroup One enrolled in SES reading but received zero hours of tutoring. Subgroup Two enrolled in SES reading and received more than zero hours but less than thirteen hours of tutoring. Subgroup T hree enrolled in SES reading and received thirteen or more hours of tutoring. Subgroup F our did not enroll in SES reading. Table 4 7 reported the frequency with which students were identified in each subgroup. Of the students who received greater than zero hours of SES tutoring in reading ( Subgroup Two vs Subgroup Three ), S ubgroup T hree had the largest participant numbers. Table 4 8 reported that the type of subgroup affiliation is significantly related to grade level. Table 4 9 reported the percentage of students by subgroup for each grade level, four to twelve Of the students who received greater than

PAGE 76

76 zero hours of SES tutoring in reading ( S ubgroups T wo and T hree combined), grade four had the largest student participant numbers. Table 4 10 reported that student participation in SES reading is statist ic ally 8 FCAT devel opmental scale score in reading. To further disaggregate the data, Table 4 11 reported the mean developmental scale score in reading by grade level and subgroup. Students in grades four through seven who received gr eater than zero hours of SES tutoring in reading on average made greater learning gains than those who did not receive any reading tutoring. In grade eight, students who received thirteen or more hours in SES tutoring in reading on average had the largest learning gain scor e in reading. Grades nine and ten did not have student s in every subgroup. In grade eleven students who enrolled in SES reading but did not receive tutoring on average had the largest learning gain score in reading. In grade twelve stud ents who did not enroll in SES tutoring for reading had on average the largest learning gain score in reading. Tables 4 12 through 4 15 reported data on students with special characteristics. Table 4 12 reported the relationship between students with disa bilities (SWD) and enrollment within subgroups. SWD received SES tutoring in reading ( S ubgroups T wo and T hree combined) at a higher rate than nondisabled students at 8.67 % and 6.61 % respectively. However, Table 4 13 reported no statistically significant re lationship related to SWD status and SES participation in reading. Table 4 14 reported th e relationship between English Language L earner (ELL) students and enrollment within subgroups. ELL students enrolled in SES reading but did not receive services ( Subg roup One ) at a higher percentage than non ELL students, 19.37 % compared to

PAGE 77

77 16.58 % The percentage of ELL students who received SES reading ( Subgroups Two and T hree ) was higher than non ELL students. Table 4 15 reported the significant relationship related to ELL status and participation in SES reading. Research Question 1: How D id R eading P articipation in SES C orrelate to S tudent Reading L earning G ains in a S elected S chool D istrict? Of the 5,033 student population, 1,027 (23.98 % ) students had enrolled in SE S reading. When assessing the reading developmental scale score gains for students who participated in reading SES, the experimental group further eliminated 859 students. These students enrolled in SES but did not receive tutoring services. The final expe rimental data set contained 348 students with recorded SES reading hours greater than zero. To analyze reading learning gains; however, the data set further eliminated 39 students who did not have both FCAT 2008 and 2009 reading scores, resulting in a stud ent population total of 309. Table 4 16 reported SES reading participation and the Analyses indicated there was not a statistically significant relationship related to SES reading participation and reading learning gains with a correlation coefficient = .05 ( p value = 0.35). Research Question 2: How D id D ifferent L evels of R eading P articipation H ours within SES C orrelate to S tudent L earning G ains in R eading in a S elected S c hool D istrict? The number of students who had FCAT reading scores for two consecutive years (2008 and 2009) and received greater than zero hours of SES reading was 309. Overall, the average number of SES reading hours was fourteen while the most frequently occurring number of hours was twenty However, the median of thirteen SES reading hours was selected in this data analyses to eliminate the effect of outliers. The

PAGE 78

78 first group of students (n=90) received greater than zero hours of SES reading but less tha n thirteen (Table 4 17). The second group of students (n= 219) received greater than or equal to thirteen hours of SES reading (Table 4 18). Tables 4 19 and 4 20 reported SES reading participation levels and resulting reading learning gains utilizing a Spe statistically significant relationship for students with SES reading participation greater than zero hours but less than thirteen hours and FCAT Reading developmental scale s core gains with a correlation coefficient = .12 ( p value = 0.22). However, a statistically significant relationship was found for students with SES reading participation greater than or equal to thirteen hours and FCAT reading learning gains with a correla tion coefficient = 0.13 ( p value = 0.05). Research Question 3 : How D id R eading L earning G ains of S tudents W ho P articipated in SES C ompare to R eading L earning G ains of S imilar S tudents W ho Did N ot P articipate in SES in a S elected S chool D istrict? Of the 5,0 33 student population, 4,248 students had both FCAT 2008 and 2009 reading scores. As described previously, 309 students had the requisite reading scores and received SES tutoring in reading which left a remaining 3,939 students who did not receive SES read ing but had the required FCAT reading scores for this analysis. Table 4 21 reported the average Developmental Scale Score (DSS) gain for students who did and did not receive SES tutoring in reading. Students who received SES in reading on average gained 17 5.1 DSS points ( SD = 222.2) while students who did not receive SES in reading on average gained 132.0 DSS points ( SD = 207.6). There was a significant relationship with regards to DSS reading gain when student participation in SES reading was compared to t he DSS reading gain of students who did not participate in SES reading ( p value = 0.001).

PAGE 79

79 Summary The purpose of these analyses was to examine the cost efficiency of the SES reading program in a selected school district where Title I schools were required to offer the SES program during the 2008 09 school year. A Spearman rho correlation coefficient was used in multiple analyses to examine the statistical dependence between student participation in SES reading and reading developmental scale score gains as well as varying student participation hours in the SES reading program and reading learning gains. Also analyzed were the reading developmental scale score means for students who received SES in reading and the DSS of students who did not receive SES readi ng. The analyses concluded that a statistically significant relationship existed for students with SES reading participation greater than or equal to thirteen hours and DSS reading learning gains on the 2009 FCAT Reading test. Overall, there was a signific ant relationship with regards to DSS reading gain on average when student participation in SES reading was compared to the DSS reading gain on average of students who did not participate in SES reading on the 2009 FCAT Reading test. In Chapter 4 quantitati ve results of the study were presented. Chapter 5 presents a discussion of the findings, conclusions and implications, as well as recommendations for future policy considerations.

PAGE 80

80 Table 4 1 Population by g rade l evel Grade l evel Frequency Percent Cumulative f requency Cumulative p ercent 4 995 19.77 995 19.77 5 923 18.34 1918 38.11 6 587 11.66 2505 49.77 7 618 12.28 3123 62.05 8 606 12.04 3729 74.09 9 337 6.70 4066 80.79 10 334 6.64 4400 87.42 11 361 7.17 4761 94.60 12 272 5.40 5033 100.00 Table 4 2 2008 09 FCAT r eading d evelopmental s cale s cores Basic st atistical m easures Location Variability Mean 1607.813 Std d eviation 318.14590 Median 1632.000 Variance 101217 Mode 1672.000 Range 2513 I nterquartile r ange 389.00000 Table 4 3 Students with d isabilities ESE status Frequency Percent Cumulative f requency Cumulative p ercent No 4294 85.32 4294 85.32 Yes 739 14.68 5033 100.00 Table 4 4 English l anguage l earners ESOL status Frequency Percent Cumulative f requency Cumulative p ercent No 4150 82.46 4150 82.46 Ye s 883 17.54 5033 100.00 Table 4 5 Predictor v ariables on the 2008 09 FCAT r eading d evelopmental s ca le s c ore g ains Source DF Type III SS Mean s quare F v alue Pr > F Grade 4 Read_DSS_FY08 1 40245167.20 40245167.20 1429.06 <.0001 SES p articipation 3 300927.55 100309.18 3.56 0.0139 Grade 5 Read_DSS_FY08 1 42191984.64 42191984.64 1146.25 <.0001 SES p art icipation 3 19555.39 6518.46 0.18 0.9119 Grade 6 Read_DSS_FY08 1 27636193.88 27636193.88 816.57 <.0001 SES p articipation 3 30721.26 10240.42 0.30 0.8235

PAGE 81

81 Table 4 5. Continued Source DF Type III SS Mean s quare F v alue Pr > F Grade 7 Read_DSS_FY0 8 1 26477168.60 26477168.60 893.86 <.0001 SES p articipation 3 43096.06 14365.35 0.48 0.6929 Grade 8 Read_DSS_FY08 1 12541201.23 12541201.23 813.65 <.0001 SES p articipation 3 69461.72 23153.91 1.50 0.2130 Grade 9 Read_DSS_FY08 1 9739526.589 9739526.58 9 428.17 <.0001 SES p articipation 1 15023.846 15023.846 0.66 0.4171 Grade 10 Read_DSS_FY08 1 14024931.57 14024931.57 385.66 <.0001 SES p articipation 3 129159.79 43053.26 1.18 0.3161 Grade 11 Read_DSS_FY08 1 7456069.860 7456069.860 120.43 <.0001 SES p articipation 3 456306.954 152102.318 2.46 0.0641 Grade 12 Read_DSS_FY08 1 1551429.223 1551429.223 24.89 <.0001 SES p articipation 3 252091.806 84030.602 1.35 0.2666 Table 4 6 Rea ding SES p articipation h ours Basic s tatistica l m easures Location Variability Mean 14.36207 Std d eviation 6.17758 Median 13.00000 Variance 38.16248 Mode 20.00000 Range 28.00000 Interquartile r ange 9.50000 Table 4 7 Subgroup f requency a Subgroups Frequency Percent Cu mulative f requency Cumulative p ercent 1 859 17.07 859 17.07 2 80 1.59 939 18.66 3 268 5.32 1207 23.98 4 3826 76.02 5033 100.00 a Sample size = 5033. Table 4 8 Grade l evel Statistic DF Value Prob Chi s quare 24 181.5173 <.00 01

PAGE 82

82 Table 4 9 Subgroups by g rade l evel Subgroups : Frequency Percent Row p ct Col p ct Grade l evel 4 5 6 7 8 9 10 11 12 Total 1 179 3.56 20.84 17.99 136 2.70 15.83 14.73 116 2.30 13.50 19.76 94 1.87 10.94 15.21 88 1.75 10.24 1 4.52 47 0.93 5.47 13.95 72 1.43 8.38 21.56 99 1.97 11.53 27.42 28 0.56 3.26 10.29 859 17.07 2 16 0.32 20.00 1.61 17 0.34 21.25 1.84 6 0.12 7.50 1.02 9 0.18 11.25 1.46 15 0.30 18.75 2.48 0 0.00 0.00 0.00 6 0.12 7.50 1.80 10 0.20 12.50 2.77 1 0.02 1.25 0. 37 80 1.59 3 88 1.75 32.84 8.84 75 1.49 27.99 8.13 20 0.40 7.46 3.41 27 0.54 10.07 4.37 24 0.48 8.96 3.96 0 0.00 0.00 0.00 1 0.02 0.37 0.30 31 0.62 11.57 8.59 2 0.04 0.75 0.74 268 5.32 4 712 14.15 18.61 71.56 695 13.81 18.17 75.30 445 8.84 11.63 75.8 1 488 9.70 12.75 78.96 479 9.52 12.52 79.04 290 5.76 7.58 86.05 255 5.07 6.66 76.35 221 4.39 5.78 61.22 241 4.79 6.30 88.60 3826 76.02 Total 995 19.77 923 18.34 587 11.66 618 12.28 606 12.04 337 6.70 334 6.64 361 7.17 272 5.40 5033 100.00

PAGE 83

83 Table 4 10 Reading SES p articipation Data s ource DF Type III SS Mean s quare F v alue Pr > F 1 234514579.7 234514579.7 7015.43 <.0001 SES p articipation 1 143641.9 143641.9 4.30 0.0382 Table 4 11 Reading d evelopmental s cale s core g ains by s ubgroup Sub groups N o bs N Mean Std d ev Minimum Maximum Grade l evel = 4 1 179 177 218.5141243 171.6824205 366.0000000 850.0000000 2 16 14 289.4285714 118.1463983 103.0000000 492.0000000 3 88 85 237.61176 47 211.3661399 477.0000000 982.0000000 4 712 622 240.0241158 191.1596219 513.0000000 1287.00 Grade l evel = 5 1 136 133 86.6390977 220.0086547 496.0000000 1170.00 2 17 17 155.2941176 250.6555816 183.0000000 744.0000000 3 75 73 138.1232877 200.51073 30 682.0000000 676.0000000 4 695 624 93.5560897 211.9913004 588.0000000 1209.00 Grade l evel = 6 1 116 116 140.7155172 216.1217775 365.0000000 925.0000000 2 6 5 189.0000000 110.5282769 33.0000000 335.0000000 3 20 20 228.3000000 189.9626279 350.0000 000 614.0000000 4 445 403 142.3424318 207.3407484 409.0000000 1026.00 Grade l evel = 7 1 94 90 154.0777778 160.8836713 174.0000000 724.0000000 2 9 7 221.2857143 287.3381946 12.0000000 797.0000000 3 27 22 154.7272727 323.0720371 534.0000000 811.00000 00 4 488 422 137.2890995 190.9872882 600.0000000 858.0000000 Grade l evel = 8 1 88 84 143.0952381 152.0285816 151.0000000 776.0000000 2 15 12 131.1666667 106.0830149 58.0000000 315.0000000 3 24 17 167.6470588 119.9285940 94.0000000 385.0000000 4 4 79 439 144.3189066 174.2832245 549.0000000 1063.00 Grade l evel = 9 1 47 45 44.1777778 153.7817652 400.0000000 329.0000000 4 290 240 63.6791667 151.3883904 382.0000000 776.0000000 Grade l evel = 10 1 72 72 23.9444444 260.6478866 769.0000000 693.000 0000 2 6 6 12.0000000 252.6190808 379.0000000 270.0000000 3 1 1 172.0000000 172.0000000 172.0000000 4 255 224 6.4553571 170.7664016 590.0000000 662.0000000 Grade l evel = 11 1 99 70 179.5857143 255.6030943 433.0000000 1100.00 2 10 7 33.2857143 1 95.0587213 194.0000000 286.0000000 3 31 21 106.9523810 277.9045657 731.0000000 563.0000000 4 221 111 76.2612613 251.2245179 661.0000000 758.0000000

PAGE 84

84 Table 4 11. Continued Sub groups N o bs N Mean Std d ev Minimum Maximum Grade l evel = 12 1 28 16 51.8 750000 294.6629430 558.0000000 470.0000000 2 1 1 76.0000000 76.0000000 76.0000000 3 2 1 389.0000000 389.0000000 389.0000000 4 241 51 99.6470588 249.2545545 639.0000000 833.0000000 Table 4 12 Students with d isabi lities by s ubgroup Subgroups : ESE_stat Frequency Percent Row p ct Col p ct No Yes Total 1 737 14.64 85.80 17.16 122 2.42 14.20 16.51 859 17.07 2 68 1.35 85.00 1.58 12 0.24 15.00 1.62 80 1.59 3 216 4.29 80.60 5.03 52 1.03 19.40 7.04 268 5.32 4 3273 65.03 85.55 76.22 553 10.99 14.45 74.83 3826 76.02 Total 4294 85.32 739 14.68 5033 100.00 Table 4 13 Students with d isabilities and SES r eading p articipation Statistic DF Value Prob Chi Square 3 5.0913 0.1652 Likelihood Ratio Chi Square 3 4.7299 0.1927 Mantel Haenszel Chi Square 1 0.0007 0.9784 Phi Coefficient 0.0318 Contingency Coefficient 0.0318 Cramer's V 0.0318

PAGE 85

85 Table 4 14 English l anguage l earners by s ubgroup Subgroups : ESOL sta tus Frequency Percent Row pct Col p ct No Yes Total 1 688 13.67 80.09 16.58 171 3.40 19.91 19.37 859 17.07 2 50 0.99 62.50 1.20 30 0.60 37.50 3.40 80 1.59 3 203 4.03 75.75 4.89 65 1.29 24.25 7.36 268 5.32 4 3209 63.76 83.87 77.33 617 12.26 16.13 69.88 3826 76.02 Total 4150 82.46 883 17.54 5033 100.00 Table 4 15 English l anguage l earners and SES r eading p articipation Statistic DF Value Prob Chi s quare 3 38.9932 <.0001 Likelihood r atio Chi s quare 3 34.1478 <.0001 Mantel Haenszel Chi s quare 1 13.0029 0.0003 Phi c oefficient 0.0880 Contingency c oefficient 0.0877 Cramer's V 0.0880 Table 4 16 SES r eading p articipation and r eading d evelopmental s cale s core Spearman c orrelation c oeff icients Pr > lrl under H0: Rho=0 Number of o bservations G ain SES r eading G ain 1.00000 309 0.05317 0.3515 309 SES r eading 0.05317 .03515 309 1.00000 348

PAGE 86

86 Table 4 17 Subgroup Two: SES r eading p articipation d evelopmental s ca le s cores Subgroup 2 Variable N Mean Std d ev Median Minimum Maximum Reading SES h ours 90 6.96667 2.68349 6.00000 1.00000 2.00000 Gain 90 182.93333 215.95223 188.00000 379.00000 797.00000 Table 4 18 Subgroup Three: SES read ing participation developmental scale scores Subgroup 3 Variable N Mean Std d ev Median Minimum Maximum Reading SES h ours 219 17.89954 4.30372 20.00000 13.00000 29.00000 Gain 219 171.93607 225.14168 173.00000 731.00000 982.00000 Table 4 19 Subgroup Two r eading d evelopmental s cale s core g ains Spearman c orrelation c oefficients, N = 90 Pr > lrl under H0: Rho=0 Reading SES h ours Gain Reading SES h ours 1.00000 0.12848 0.2275 Gain 0.12848 0.2275 1.00000 Table 4 20 Subgroup Three reading developmental scale score gains Spearman Correlation Coefficients, N = 219 Pr > lrl under H0: Rho=0 Reading SES Hours Gain Reading SES Hours 1.00000 0.13119 0.0525 Gain 0.13119 0.0525 1.00000 Table 4 21 General SES r eading p articipation and r eading d evelopmental s cale s cores Variable: Gain SES h ours N Mean Std d ev Std e rr Minimum Maximum Subgroups 1 & 4 (No SES r eading h ours) 3939 132.0 207.6 3.3070 769.0 1287.0 Subgroups 2 & 3 (Some SES r eading h ours) 309 175.1 222.2 12.6412 731.0 982.0

PAGE 87

87 CHAPTER 5 CONCLUSIONS AND RECO MMENDATIONS Prior to recent competitive grants at the federal level, the NCLB Act of 2001 was the accountability standard for which equity and efficiency wer e measured. From an equity standpoint, this act created a mandate by which students from economically disadvantaged backgrounds were afforded educational services that were typically associated with those opportunities of white, middle class students. From the viewpoint of efficiency, NCLB aimed to increase the accountability of school districts by transforming resources into performance outputs via student academic performance. 1 In this context, the importance of educational efficiency was a main principle of NCLB. The purpose of this analysis was to study the cost efficiency of the SES program in a selected school district. To determine the extent with which the federally funded SES program impacted FCAT reading developmental scale scores, the following r esearch questions were addressed in this study: 1. How did reading participation in SES correlate to student reading learning gains in a selected school district? 2. How did different levels of reading participation hours within SES correlate to student learning gains in reading in a selected school district? 3. How did reading learning gains of students who participated in SES compare to reading learning gains of similar students who did not participate in SES in a selected school district? Conclusions This study i ncluded 5,033 students who had FCAT reading scores in 2007 08 or 2008 09 and who attended a Title I school that was required to offer SES. Based upon the median of thirteen hours of SES reading participation, the student population of 1 Je Ik Cho, 2009.

PAGE 88

88 5,033 was divided int o four subgroups. Subgroup One enrolled in SES reading but received zero hours of tutoring. Subgroup T wo enrolled in SES reading and received more than zero hours but less than thirteen hours of tutoring. Subgroup T hree enrolled in SES reading and received thirteen or more hours of tutoring. Subgroup F our did not enroll in SES reading. The analyses presented herein demonstrated that the cost efficiency of Supplemental Educational Services in a selected school district was that maximum learning gain output w as achieved only after a certain threshold of SES reading intervention exposure occurred. Moreover, specific student grade levels or student characteristics were significant factors with regard to SES reading enrollment. In addition, students who received SES reading program exposure demonstrated reading learning gains statistically significant from students who did not receive exposure. Implications Districts varied broadly in the amount spent per child in Title I supplemental educational services. In a s ample of 100 school districts across the nation, 32 % spent less than $600 per student enrolled while 17 % spent $1,200 or more during the 2005 06 school year. 2 During the 2008 09 school year, Florida spent an average of $1,268 per student enrolled in a SES program, with a range extending from $848 to $1,405. 3 During the same time period (2008 09), the selected school district in this study spent an average of $1,023 per pupil. 4 Nationally, overall SES expenditures nearly doubled to $375 million from $192 mil lion (2003 04 to 2005 06), mostly because of increased 2 S cott Naftel et al. 2009. 3 Deepak Gajre, November 02, 2010. 4 Ibid

PAGE 89

89 district participation via NCLB sanctions. 5 However, the average maximum per pupil allocation for SES decreased from $1,434 in 2004 05 to $1,135 in 2005 06. 6 Market based structures are a powerful tool for improving the performance and efficiency of an organization. However, current models for market structures in public education such a s school choice, vouchers, and supplemental educational s ervices have yielded mixed r esults. Researchers such as Pic us suggested that schools implementing successful programs will meet goals while those selecting inappropriate programs will go out of business as evident in the premise of competition within market based economies. 7 However, according to Steinberg, anticomp etitive forces appeared to be prevalent among SES providers. 8 As more school districts complied with federal sanctions, the sheer number of SES providers grew exponentially. Only a few firms had the revenue and size to capture most of the market activity. With a lack of competition for student enrollment and few state accountability mechanisms in place for SES, SES providers had little incentive to function efficiently. 9 The lack of incentive potentially changed when a letter dated May 6, 2011, and sent to the United States Department of Education by Frances Haithcock, Chancellor of Public Schools in Florida, requested a waiver to the prohibition on approving an 5 S cott Naftel et al. 2009. 6 Ibid 7 Lawrence O. Pi cus, 2001. 8 Matthew Steinburg, Patricia Burch, and Joseph Donovan, June 2007 9 Ibid

PAGE 90

90 that iden tified schools and LEAs may be able to establish that they have an effective program that can help improve the academic achievement of students and should not be prevented automatically from gaining approval simply because of their improvement 10 In February 2012, the United Stated Department of Education approved 11 However, both Florida state statute 12 and Florida Adminis trative Code 13 still required SES services. Subsequently, the 2012 13 Request for Application for Supplemental Educational Services 14 from the Florida Department of Education requested application classification information from potential SES providers which included an opportunity for school districts (or Local Educational Agency, LEA) to participate. In the selected school district for this study, students who received SES reading program exposure demonstrated reading learning gains statistically significa nt from students who did not receive exposure. When school districts become an SES provider for its students, school districts may consider conducting analyses to determine significance between student learning gains after student participation in traditio nal SES tutoring programs and learning gains after student participation in a school district sponsored SES tutoring program. 10 Frances Haithcock, May 6, 2011 11 Michael Yudin, February 22, 2012 12 Florida Statute §1008.331 2012 13 Florid a Administrative Code 6A 1.039 2011 14 Florida Department of E ducation 2012.

PAGE 91

91 SES program exposure was a significant factor in the selected school district. The analyses presented herein demonstrated that the cost efficiency of Supplemental Educational Services achieved maximum learning gain output only after a certain threshold of SES reading intervention exposure occurred. Given each SES provider in a school district received the same maximum per child expen diture for supplemental educational services, additional cost effectiveness studies may be required to identify the least cost alternatives or approaches in delivering SES services. Specific student grade levels or student characteristics were significant factors with regard to SES reading enrollment in the selected school district. In every grade level, the 2008 FCAT Reading score predicted the 2009 FCAT Reading score, with the exception of grade four. In grade four, reading SES participation predicted 200 9 FCAT Reading scores. Florida law states that third graders who are not proficient in reading at 15 Fourth grade students that were promoted from th e SES program, thus potentially impacting student learning gains on state assessments. Additionally, retained third grade students have two years of FCAT scores. Future stu dies may consider the impact on student learning gains for retained third graders who participated in SES tutoring. In the selected school district, English was not the first language for 13 % of the student population with 6,000 students in the English La nguage Learner (ELL) program. Forty eight percent of the student population came from non English speaking homes. 15 Florida Statute §1008.25(5) (2002).

PAGE 92

92 Collectively, these students spoke sixty one different languages and hailed from 147 different countries of origin. ELL students enrolled in S ES reading programs but did not receive services at a higher percentage than non ELL students, 19.37 % compared to 16.58 % Yet, the overall percentage of ELL students who received SES tutoring in reading remained higher than non ELL students. Analyses indic ated that there was a significant relationship related to ELL status and participation in SES reading. Future studies may analyze the learning gains of ELL students specifically who participated in SES programs. Further analyzes may identify instructional strategies that address English language acquisition and academic achievement in an after school environment. Recommendations Through the literature review and data analyses, several public policy recommendations can be made either directly or indirectly f rom the research presented in this study. These include: 1. Legislators and policymakers should be more aggressive in monitoring the impact of SES programs on student learning gains. 2. Reexamine the NCLB waiver of school districts providing SES services as stud ent learning gain data become available. 3. Legislators and policymakers should mandate a minimum threshold of program exposure hours for students in SES programs. 4. Legislators and policymakers should increase the extent to which funds generated for SES are di rected to students with confounding characteristics such as ELL or retention status to allow for greater program exposure.

PAGE 93

93 Summary Many researchers and legislators have argued that improving the performance of public education requires additional funding t o support students with the greatest ac ademic and socio economic needs. Since the passage of NCLB, legislators have attempted to link the financing of the Supplemental Education Services (SES) program to improved academic performance. Linking educational ou tcomes to school financing requires the integration of cost efficiency concepts to achieve maximum outcomes with minimum possible inputs. Federal waivers now allow school districts to become an SES provider. It is the responsibility of legislators and poli cymakers to provide resources and m andates within Title I, Part A to support successful SES program structures and student participation criteria based upon results and best practices identified in national and state studies, including this study conducted in a selected school district.

PAGE 94

94 LIST OF REFERENCES Abrams Andrew S tephanie Stulli ch, Elizabeth Eisner, and Erica Lee 2009. Title I Implementation Update on Recent Evaluation Findings W ashington, D.C. : ED Pubs. Baldwin Jean Grossman, Karen Walker and Rebecca Raley. 2001. Challenges and Opportunities in After school Programs: Lessons for Policy Makers and Funders Philadelphia, PA: Manpower Demonstration Research Corporation Birmingham, Jennifer Ellen Pechman, Chris tina Russell, and Monica Mielke. 2005. Shared Features of High Performing After school Programs: A Follow Up to the TASC Evaluation Washington, D.C.: Policy Study Associates Bodilly, Susan J. and Megan K. Beckett. 2005. Making Out of School Time Matter : Evidence for an Action Agenda Santa Monica, CA: RAND Corporation Britsch, Brenda Nicky Martin, Amy Stuczynski, Bethany Tomala and Patti Tucci. 2005. Literacy in Af ter school Programs Austin, TX: Northwest R egional Educational Laboratory Burch, Patricia, Carolyn Heinrich, Annalee Good, and Mary Stewart. 2011. Equal Access to Quality in Federally Mandated Tutorin g: Preliminary Findings of a Multisite Study of Supplemental Educational Services Madison, WI: University of Wisconsin, Madison Carver, P. R. and I. U. Iruka. 2 006. After school Programs and Activities Washington, D.C.: U.S. Department of Education Nationa l Center for Educational Statistics Chambers, Jay Irene Lam, Kanya Mahitivanichc ha, Phil Esra, Larisa Shambaugh. 2009. State and Local Implementation of the No Child Left Behind Act Volume VI Targeting and Used of Federal Education Funds Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development Chicago Public Schools. 2007. SES Tutoring Programs: An Evaluation of Year 3 in the Chicago Public Schoo ls Chicago IL : Office of Research, Evaluation, and Accountability Cho, Je Ik. 2009. An Efficiency and Productivity Study in the Presence of the No Child Left Behind Act i n Pennsylvania School Districts U niversity Park, PA: Pennsylvania State University Cooper, Harris, Kelly Charlton, Jeff Valentine, Laura Muhlenbruck and Geoffrey Borman. 2000. Making the Most of Summer School: A Meta Analytic and Narrative Review Monographs of the Society for Research in Child Development 65 ( 1). DeAngelis, Karen and Robert Rossi. 1 997. Schools Serving Family Needs: Extended Day Programs in Public and Private Schools W ashington, D.C.: U.S. Department of Educati on, Office of Educational Research and Improvement.

PAGE 95

95 Edmonds, R. R. 1 981. Ma king public schools effective Social Policy 12 ( 2). Fashola, Olatokunbo 1998. Review of Extended Day and After School Programs and Their Effectiveness Baltimore, MD: Johns Hopkins Uni versity Finch, Robert H. 1969. History of Title I ESEA Washington, D.C.: United States Depart ment of Health, Education, and Welfare Florida Administrative Code 2011. 6A 1.039 Florida Department of Education. 2003. R equest for Proposal/Application 21st Century Community Learning C enters Program Tallahassee, FL: Florida Department of Education. Florida Department of Education 2008. Florida's Differentiated Accountability Model Guidance for Implementation 2008 09 School Year Tallahassee, FL: Florida Department of Education. F lorida Department of Education. 2012. Request for Applications for Supplemental Educational Service Providers 2012 2013 School Year Tallahassee, FL: Florida Department of Education. Florida Statute 2002. §1008.25(5) Florida Statute 2005. §1003.41 (7) Florid a Statute 2012. §1008.331 Gajre, Deepak November 2, 2010. Program Specialist IV, Florida Department of Education, Bureau of Federal Educational Programs, SES question, e mail message to Traci Dami Gajre, Deepak. November 10, 2010. Program Specialist IV, Florida Department of Education, Bureau of Federal Educational Programs, SES question, e mail message to Traci Dami Granger, R. C. 2008. After school Progra ms and Academics: Implications for P olicy, Practice, and Research Social Policy Report 2 (22) : 1 19. Haithcock, Frances. May 6, 2011. K 12 Public Schools Chancellor, Florida Department of Education, waiver request letter to Thelma Melendez de Santa Anna United States Department of Education, Office of Eleme ntary and Secondary Education, Washington, D.C. accessed http://www.fldoe.org/bsa/pdf/2011WaiverRequest DINI SINI.pdf Halpern, Robert 2002. A Different Kind of Child Development Inst itution: The History of After School Pro grams for Low Income Children Teachers College Record 104 (2).

PAGE 96

96 Hanushek, Eric A. 2005. Why the Federal Government Should be Invo lved in School Accountability. Journal of Policy Analysis and Management 1 (24). Hanushek, Eric. 2003. The Failure of Input based Schooling Practices Economic Journal 113 ( 485). Hanushek, Eric A. 1989. The Impact of Differential Expen d itures on School Performance Educational Researcher 18 ( 4 ) : 45 62 Hanushek, Eric A. and Margaret Raymond. 2001. The Confusing World of Educational Accountability Palo Alt o, CA: Hoover Institution Hedges, Larry V., Richar d D. Laine, and Rob Greenwald. 1994. Does Money Matter? A Meta Analysis o f Studies of the Effects of Differential Scho ol Inputs on Student Outcomes Educational Researcher 23 ( 3). Heinrich, Carolyn, Robert Meyer and Greg ory Whitten. 2010. Supplemental Education Services Under No Child Left Behind: Who Signs Up, and What Do They Gain? Educational Evaluation and Policy Analysis 32 ( 2). Huang, Denise, Jamie Cho, Hannah Nam, Christine Oh, Aletha Harven, and Seth Leon. 2010. What Works? Common Practices in High Functioning After school Programs Across the Natio n in Math, Reading, Science, Arts, Technology, and Homework A stu dy by the National Partnership Los Angeles: University of California, Graduate School of Evaluation, Standards, and Student Testing Interrante Hausner, Mary E. 2000. The Impa ct of Kindergarten Intervention Project Accelerated Literacy on Emerging L iteracy Concepts and Second Grad e Reading Comprehension Chic ago, IL: Loyola University Kane, Thomas J. 2004. The Impact of After school Programs: Interpreting the Resu lts of Four Recent Evaluations Working Paper, Los Angeles: University of California William T. Grant Foundation Lauer, Patricia Motoko Akiba, Stephanie Wilkerson, Helen Apthorp, David Snow and Mya Martin Glenn. 2006. Out of School Time Programs: A Meta Analysis of Effects for At Risk Students Review of Educational Research 76 (2). Levine Daniel U. and Lawrence W. Lezotte. 1990. Unusually Effective Schools: A Review and Analysis of Research and Practice Madison, WI: Univer sity of Wisco nsin, Madison McR el 2004. The Effectiveness of Out of School Time Strategies in Assisting Low Achieving Students in Reading and Mat hematics: A Research Synthesis Washington, D.C.: Unit ed States Department of Education, Institute of Education Sciences McGuinn, Patrick. 2012. Stimulating Reform: Race to the Top, Competitive Grants a nd the Obama Education Agenda Educational Policy 26 ( 1).

PAGE 97

97 Morris, Dar rell Shaw and Perney 1990. Helping Low Readers i n Grades 2 and 3: An After school Volunteer tuto ring Program Elementary School Journal 91 ( 2). Munoz, Marco 2002 Outcome based Community School Partnerships: The Impact of the After school Programs on N on a cademic and Academic Indicators Louisville, KY: Jefferson County Public Schools Naftel Scott, Georges Vernez, Karen Ross, Kerstin Carlson Le Flock, Chris topher Beighley, and Brian Gill. 2009. State and Local Implementation of the No Child Left Behind Act VolumeVII Title I School Choice and Supp lemental Educ ational Services: Final Report Washington, D.C. : U.S. Department of Education, Office of Planning, Evaluation and Policy Development National Commission on Excellence in Education. 1983. A Nation at Risk: The I m perative for Educationa l Reform Washington, D.C. : National Commission on Excellence in Education. No Child Left Beh ind Act of 2001 2002. 20 USC §1001. No Child Left Behind Act of 2001 2002 20 USC §1002. No Child Left Behind Act of 2001 2002 20 USC §1116. No Child Left Behind Act of 2001 2002 20 USC §4201 No Child Left Behind Act of 2001 2002 20 USC §6301 Odden, Allen, and Carolyn Busch. 1998. Financing Schools for High Performance: Improving th e Use of Educational Resources San Fran cisco CA: Jossey Bass Orr, Cornelia. 2007. Assessment and Accountability Briefing Book Tallahassee, FL: Florida Department of Education, 2007 Owing s, William, and Leslie Ka plan. 2010. The Alpha and Omega Syndrome: Is Intra district Fu nding the Next Ripeness Factor? Journal of Education Finance 36 ( 2). Picus, Lawrence O. 2 001. In Search of More Productive Schools: A Guide to Resource Allocation Washington, D.C.: E d ucation Resources I n formation Center. Pierce, Kim Daniel Bolt, and Deborah Lowe Vandell. 2010. Specific Features of After school Pr ogram Quality: Associations with Children's Functioning in Middle Childhood American Journal of Community Psychology 45 Reisner, Elizabeth R. Richard N. White, Christina A. Russell and Jennifer Birmingham. 2004. Building Quality, Scale and Effectivene ss in After school Programs Washington, D .C.: Policy Studies Associates, Inc

PAGE 98

98 Reisner, Elizabeth R., Richard N. White, Jennifer Birmingham and Megan Welsh. 2001. Building Quality and Supporting Expansion of After school P rojects: Evaluation Results From the TASC After school Program's Second Year Washington, D.C.: Policy Studies Associates, Inc. Rickles Jordan H. and Melissa K. Barnhart. 2007. The Impact of Supplemental Educational Servi ces Participation on Student Achievement: 2005 06 Los Angeles, CA: Los Angeles Unified School District, Program Evaluation and Research Branch Ross, Steven M., Allison Potter Jangmi Paek, and Dawn McKay. 2008. Implementation and Outcomes o f Supplemental Educational Services: The Tennesse e State Wide Evaluation Study Journal of Education for Students Placed at Risk 13 Rubenstein, Ross, Amy Ellen Schwartz and Leanna Stiefel. 2006. Rethinking the Intradistrict Distribution of School Inputs to Disadvantaged Students Berkeley, CA: University of California, Berkeley, Boalt Hall School of Law Steinburg, Matthew, Patri cia Burch, and Joseph Donovan. 2007. Supplemental Educational Services and NCLB: Policy Assumptions, Market Practices, Emergi ng Issues. Educational Evaluation and Policy Analysis 29 ( 115). Strike, Kenneth 2008. Equality of Opportunity and School Finance: A Commentary on Ladd, Sa tz, and Brighouse and Swift Education Finance and Policy 3 (4). Stullich, Ste phanie Andrew Abrams, E lizabeth Eisner, and Erica Lee 2009. Title I Implementation Update on Recent Evaluation Findings Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development and Program Studies Service U.S. Department of Education. 2003. 21 st Century Community Learning Centers Non regulatory Guidance Washin gton, D.C .: U.S. Department of Education. U.S. Department of Educati on. 2005. 10 Facts About K 12 Education Funding, Washin gton, D.C .: U.S. Department of Education. U.S. Department of Education. 1996. Assessment and Analysis of School Level Expenditures Working Paper No. 96 19, Washington, D.C. U.S. Department of Education. 2009. Supplemental Educational Services Non Re gulatory Guidance Washin gton, D.C .: U.S. Department of Education. U.S. Department of Education. 2005. When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program Washington, D.C.: I nstitute of Education Sciences and National Center for Education Evaluation U.S. Department of Labor. 2005. Women in the Labor Force: A Databook Washin gton, D.C .: U.S. Department of Education.

PAGE 99

99 Wood, R. Craig. 2007. Educational Finance Law: Constitutional Challenges to State Aid Plans An Analysis of Strategies Dayton, OH: Education Law Associ ation Yudin, Michael. February 22, 2012. Acting Assista nt Secretary, U.S. Department of Education, Office of Elementary and Secondar y Education, ESEA flexibility approval l e tter to Gerard Robinson Florida Department of Education, Commissioner, Tallahassee, FL. accessed http://www2.ed.gov/policy/eseaflex/implementation letters/fl implementation letter.pdf Zhang, James J. David S. Fleming, and Brandy L. Bartol. 2004. The Sunshine State Does Great Things for Its Children: Assessing the Effectiveness of the 21st Century Co mmunity Learni ng Centers (21st CCLC) Program Gainesville, FL: Uni versity of Florida, College of H ealth and Human Performance Zimmer, Ron Brian Gill, Paul Razquin, Kevin Boo ker, and J.R. Lockwood III. 2007. State and Local Implementation of the No Child Left Behind Act: Volume I Title I School Choice, Supplemental Educational Services, and Student Achievement Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development Zoblotsky Todd and Ying Huang. 2010. Evaluation of 21st Cen tury Community Learning Centers Memphis, TN : C enter for Research in Educational Policy

PAGE 100

100 BIOGRAPHICAL SKETCH Traci Lynn Kohler grew up in Appleton Wisconsin and received a Bachelor of Science from the University of Wisconsin Madison in 1992, with a major in elementary education and a minor in health education She completed her Master of Science in curriculum and instruction from Florida Gulf Coast University in 1999. Traci taught middle school geography and reading from 1994 until 2001 for Collier County Public Schools in Naples, Florida. In 2001, she left the classroom and began her work as a staff developer/instructional technology specialist wit h Collier County Public Schools. In 2004, she transitioned to the role of an administrator as the Coordinator for Staff Development and School Improvement. In 2008 she became the Director for Federal and State Grants. Since 2009, Traci has been the Direct or of Staff Development, Instructional Technology and Media Services for Collier County Public Schools in Naples, Florida. Traci is married and has two sons, one stepson and two stepdaughters