1 SCHOOL LEVEL AUTONOMY: WHO PARTICIPATES AND WHO BENEFITS By J ESSICA S. HAYNES A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOC TOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2013
2 2013 Jessica S. Haynes
3 To Mom, Dad, Kate, Peter, and Brooke Lynn, all of whom have helped me to be a better and stronger person
4 ACKNOWLEDGMENTS This thesis would not have been possible without the inp ut and guidance of many professors, colleagues, and family members. I am very grateful to Dr. Damon Clark for his instruction on empirical research and helping me to obtain essential data. I would, also, like to thank Jonah Rockoff for being kind enough to share data that he had already collected. This data has been invaluable to the project. My deepest thanks I owe to Dr. Romano and Dr. Kenny. Dr. Kenny has helped me sort through the empirical challenges and taught me that simple and robust methods a re best. I could not have finished the project without his kind words of encouragement. I am also deeply thankful and will probably become increasing so in the years that follow to Dr. Romano who read even the earliest drafts of my research and continued to read the many drafts that followed. I appreciate his consistent direction on how to be a more thorough, clear, and concise researcher. In the process of writing this paper, I have also received support and insights from my colleagues in the program. Thank you to Dr. Edward See, Dr. Shourjo Chakravorty, Dr. Michelle Phillips, Ying Tang, and Derek Drayer. A special thank you to Shourjo for the many walks and coffees that helped me to work through this process. I am also very grateful for my officemate s and friends, Michelle and Ying. I will always remember the Rosie the Riveter poster -"We Can Do It!" I owe my deepest gratitude to my family. I am particularly thankful to my parents who have constantly supported me and given me the depth to walk my own path. I am grateful to my sister, Kate, whose joy and strength are an inspiration. I could not have made it through this process and the last few years without her friendship. Finally, I am
5 thankful to Peter, without whom I would not have had the c ourage to finish and who reminded me why I end eavor to be a good researcher.
6 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURES ................................ ................................ ................................ .......... 9 ABSTRACT ................................ ................................ ................................ ................... 10 CHAPTER 1 BUDGETARY AUTONOMY THEORETICAL MODEL ................................ ............ 12 1.1 Introduction ................................ ................................ ................................ .... 12 1.2 Literature Review ................................ ................................ ........................... 13 1.3 Program Specifics ................................ ................................ .......................... 19 1.4 Model Specifica tion ................................ ................................ ........................ 25 1.4.1 General Specification ................................ ................................ .......... 25 1.4.2 Centralized Control ................................ ................................ .............. 30 1.4.3 Autonomy ................................ ................................ ............................ 31 1.5 Who Selects Autonomy ................................ ................................ .................. 34 1.5.1 General Selection ................................ ................................ ................ 34 1.5.2 Two Types of Autonomous Schools ................................ .................... 35 184.108.40.206 The selection decision for non rent consuming schools ......... 37 220.127.116.11 T he selection decision for rent consuming schools ................. 40 1.6 The Impact of Autonomy on Test Scores ................................ ....................... 43 1.6.1 Impact on School Level Te st Scores ................................ ................... 43 1.6. 2 Impact on Individual Type's Test Score ................................ ............... 44 1.7 Conclusion ................................ ................................ ................................ ..... 46 2 ESTIMATING TEST SCORE IMPACTS ................................ ................................ 53 2.1 Introduction ................................ ................................ ................................ .... 53 2.2 Data ................................ ................................ ................................ ................ 57 2.3 Identification Strategy and Selection Equation ................................ ............... 64 2.4 Selection Equation Empirical Results ................................ ............................. 73 2.5 Test Sc ore Results ................................ ................................ ......................... 78 2.6 Conclusion ................................ ................................ ................................ ..... 83 3 ESTIMATING EXPENDITURE IMPACTS ................................ ............................. 102 3.1 Introduction ................................ ................................ ................................ .. 102 3.2 Data ................................ ................................ ................................ .............. 102 3.3 Empirical Methodology ................................ ................................ ................. 106 3.4 Results ................................ ................................ ................................ ......... 112
7 3.5 Conclusion ................................ ................................ ................................ ... 114 4 SIMULATION OF TEST SCORE IMPACTS AND PROGRAM COSTS ................ 123 4.1 Introduction ................................ ................................ ................................ .. 123 4.2 The Computable Model ................................ ................................ ................ 124 4.2.1 District Parameters ................................ ................................ ............ 124 4.2.2 Principal Input Parameters ................................ ................................ 127 4.2.3 Student Parameters ................................ ................................ ........... 130 4.3 Calibrat ing The Model ................................ ................................ .................. 131 4.4 Results ................................ ................................ ................................ ......... 133 4.4.1 Best Fits ................................ ................................ ............................ 134 4.4.2 Auto nomy Decisions ................................ ................................ .......... 136 4.4.3 Costs of the Program ................................ ................................ ......... 136 4.4.4 Student level impacts ................................ ................................ ........ 139 4.5 Conclusion ................................ ................................ ................................ ... 141 APPENDIX A WHY NEW YORK CITY PROVIDES A GOOD BASIS FOR THIS STUDY? ......... 152 B OPTIMAL INPUT ALL OCATIONS ................................ ................................ ........ 154 C PROPENSITY AUTONOMOUS ................................ ................................ ............ 161 D TEST SCORE IMPACTS ................................ ................................ ...................... 175 LIST OF REFERENCES ................................ ................................ ............................. 180 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 184
8 LIST OF TABLES Table page 1 1 Defi nition of school groups ................................ ................................ ................. 48 1 2 Group 1: Characteristic effects on autonomy decision and test scores ............. 49 1 3 Group 2: Characteri stic effects on autonomy decision and test scores ............. 50 1 4 Characteristic effects summary ................................ ................................ .......... 52 2 1 Connecting the theoretical model to empirical analysis ................................ ...... 85 2 2 Tests of exogeneity of principal female ................................ .............................. 86 2 3 Descriptive statistics ................................ ................................ ........................... 87 2 4 Balance of observed variables after treatment ................................ ................... 88 2 5 Balance of observed variables in the pre treatment period ................................ 89 2 6 Selection of autonomy ................................ ................................ ........................ 90 2 7 First stage: Generated instrument ................................ ................................ ..... 92 2 8 The impact of autonomy on school level test scores ................................ .......... 93 2 9 Heterogeneous test score impacts ................................ ................................ ..... 95 3 1 Definitions of expenditure categories ................................ ................................ 115 3 2 Summary statistics ................................ ................................ ........................... 117 3 3 Comparison of means ................................ ................................ ...................... 119 3 4 Comparison of mean s pre treatment ................................ ................................ 120 3 5 The impact of princi pal female on selection ................................ ...................... 12 1 3 6 The impact of autonomy on expenditure allocation ................................ .......... 122 4 1 Mapping determinants of the theoretical into the computable model ................ 143 4 2 District wide impact of autonomy program: Var yin g budgetary reward ................................ ................................ ................................ .......... 144 4 3 District wide impact of autonomy program: Varying budgetary reward and risk aversion ................................ ................................ ...... 146 4 4 District wide impact of autonomy .................... 151
9 LIST OF FIGURES Figure page 2 1 The interaction of auton omy and free and reduced lunch status on test scores ................................ ................................ ................................ ................. 96 2 2 The interaction of autonomy and principal experience on test scores ................ 98 2 3 The interaction of autonomy and school size on math scores .......................... 100
10 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy SCHOOL LEVEL AUTONOMY: WHO PARTICIPATES AND WHO BENEFITS By Jessica S. Haynes May 2013 Chair: Richard Romano Major: Economics School level budgetary autonomy grants traditional public school principals increased co ntrol to manage the budget of the schools they lead. Increased school level autonomy is being implemented in school districts with the hope that principals will be able to apply their local knowledge to more effectively execute budgets. This dissertation analyzes a school level budgetary autonomy program implemented in New York City Public Schools. I utilize both theoretical and empirical methods to examine the decision to participate in the program, the resulting impacts on student achievement, and chan ges in expenditure patterns. Simulations are employed to examine how test score impacts and program costs vary with risk aversion and the strength of incentives. Estimates show that a principal's years of administrative experience positively impacts pro gram participation for both middle schools and elementary schools. In elementary schools, the probability a principal will select autonomy is increasing with the size of the school. I use a generated instrumental variables procedure to account for select ion and improve the efficiency of estimated test score impacts and expenditure changes. In elementary schools, estimates show that increased autonomy improves both math and reading scores. Autonomy does not have a statistically significant
11 impact on stud ent achievement in middle schools. The magnitude of test score impacts depends on the characteristics of a particular school. The percentage of free and reduced lunch (FRL) students e nrolled in a school negatively a ffects the impact of autonomy. The res ults show that for math scores this negative impact is significant enough to reverse the gains from autonomy if more than 68% of students are free and reduced lunch eligible. The reading results show that test score gains can be achieved even when 100% of students qualify for FRL. To gain an understand ing of how autonomy improved student achievement in New York City Public schools, I examine changes in expenditure patterns. I analyze the impact on 7 mutually exclusive expenditure categories: teachers, other classroom expenditures instructional support services, leadership, other direct services, centralized services, and capital services. The estimates show that autonomy increases the amount of funds contributed to teachers. Further, autonomy decrea ses expenditures on leadership, centralized services and capital services.
12 CHAPTER 1 BUDGETARY AUTONOMY THEORETICAL MODEL 1.1 Introduction School administrators are key agents in the daily operation of schools. The job of a school administrator is com posed of multiple tasks. One of the key functions of a 1 The amount of leeway principals have to make budgetary allocations, however, varies greatly across schools and districts. Further, it is often c laimed that public school principals do not have sufficient autonomy, especially over school budgets. This can, some believe, explain why public schools have been found to perform worse than pri vate schools (Evans and Schwab 1995, Ro use1998 ) and char ter s chools (Hoxby and Rockoff 2004, Hoxby 2004, Hoxby, K ang, and Murarka 2009 ). It might also explain why decades of research finds only small educational returns to allocating more resources to the public school sector (Hanushek 2003 ). In response to the s e c oncern s education policies are increasingly focused on giving more budgetary authority to principals to ensure that resources are directed to their most efficient use. This chapter examines how increased freedom to allocate funds coupled with performance est scores. The model developed here is based on an autonomy program implemented in New York City schools that increased the budgetary autonomy of the principals who selected into the program. 2 The model examines not only t he selection decision of the principals, but 1 H a category including the task of managing the budget, significantly impacts student achievement. 2 Se e Appendix A for a discussion of other autonomy programs and why studying New York City is preferable.
13 how principals allocate expenditures and their own effort. Key determinants of a the effectiveness of the principal, the composition of the student population, and th e volatility of test scores. In section 1.2 I discuss the trend toward decentralized decision making and its known impacts, as well as the importance of the principal's decision making Section 1.3 details the NYCPS autonomy program that provides the basis for the model. The theoretical model is sp ecified in section 1.4 and the predictions that result a re presented in section 1.5 1.2 Literature Review There is a broad body of research that examines the performance of alternative structures of orga nizations characterized by the d egree of decentralized decision making in public institutions 3 Inherent in the question of how decentralized decision making functions within particular institutions is an analysis of how agents use private information or behave in self interested ways that produce suboptimal outcomes. 4 I do not attempt a full review of this literature, but instead focus on three areas of research more closely aligned with this project. 5 In doing so, I review literature on decentralized 3 See e.g. J. Green and J.J. Laffont, Incentives in Public Decision Making, (Amsterdam: North Holland, 1979); Theodore Groves, Yongmiao Hong, John McMil Quarterly Journal of Economics, CIX (1994): 183 209; P. European Economic Review 40 (1996): 61 Annals of Public and Cooperative Economics 80(2009): 247 273. 4 For a seminal paper on the principal agent problem, see Sanford J. Analysis of the Principal Econometrica, 51 (1983): 7 46. 5 These papers will be more useful in addressing the question of why the form of autonomy implemented by New York City Department of Education (NYCDOE) p roduces different outcomes than centralized (district level) control. This directs the focus away from the related, but different, question of what form of autonomy produces optimal outcomes in public schools.
14 decision making in schools, both in traditional public schools and charter schools and the role/impact of school administrators. Decentralized decision making has long been a fixture in initiatives and programs designed to improve education outcomes. The premise of these initiatives is that decision makers at a more local level will have a better understanding of the particular needs and challenges of the students they serve and will, therefore, be able to more 4) present a simple theor etical model that addresses the importance of institutions in the production of education quality. They aptly direct it is the institutions of the education system that allocate the rights of decision making in the system and determine the incentives 6 Their model highlights that school level autonomy likely results in two countervailing effects. Autonomy increases the informational content of decisions and, as a result, improves the efficiency of resource allocations. On the other hand, autonomy increases the latitude for an educational agent to divert funds from quality the other depends on the typ e of autonomy that has been granted. They suggest that school level autonomy on standard setting and testing would lead to the inefficient diversion effect dominating the efficiency achieving information effect. Increased control of process and personnel decisions, however, would lead to the opposite. The implementation of decentralized decision making programs varies widely. Some programs devolve decision making authority from the district level to smaller groups of schools. For example, in 1969 the N 6
15 districts (CSDs). 7 More prevalent, however, are programs that aim to move decision making authority to the school level. Within public scho ols, charter schools provide the maximum amount of school level autonomy. Beyond charter schools, there are programs that increase the autonomy of public schools that remain under some district control. Many of these programs are classified under the ter ms Site Based Management (SBM) or Site Based Budgeting (SBB). 8 Even within programs focused on school level autonomy, the specifics vary greatly. The intended decision makers may either be principals, teachers, and/ or parents. Some SBM/SBB programs req uired schools to set up site based councils composed of all three types of decision makers. Decentralized decision making programs, also, differ widely in the scope of autonomy conferred on the decision maker. For example, some programs grant only budget ary autonomy, while others provide schools the ability to alter the length of the school day, 9 institute different curriculum, or change the textbooks. Relatively few studies examining school level autonomy programs, such as SBM/SBB programs, focus o n measuring the impact of such programs on student outcomes or resource allocations. Among the studies that analyz e programs granting school level budgetary autonomy, there are two that focus on student outcomes, i.e. 7 The histor y of this reform is discussed briefly in Leanna Stiefel, Amy Ellen Schwartz, Carole Portas, and 8 (2003): 407. High Schools remained under centralized control. 8 For a good review of literature on and examples of SBM/SBB programs, see Stiefel et al. (2003): 405 407. 9 Generally, charter schools have this type of leeway. The Pilot program in Boston allowed schools the ability to make this type of change as well.
16 Stie fel et al. (2003) and Loeb and St runk (2007). Stiefel et al. examines a site based management program in New York City schools that increased budgetary autonomy. The program was introduced in 1996 under the name Performance Driven Budgeting (PDB). PDB combined budgetary autonomy with p erformance incentives. Schools that expressed interest and were approved by their CSD superintendent joined the program. Stiefel et al. employ a difference in difference strategy that compares the student test scores in PDB schools to those in non PDB sc hools. 10 As a result, the program impacts they report include any selection bias that may be present. 11 They find that the program increased school level math and English scores in Grade 4 by approximately 0.056 and 0.059 of a standard deviation, respectiv ely. A similar result holds for Grade 5 English scores (0.057). The impact on Grade 5 math scores is not statistically significant. Loeb and Strunk examine the impact of autonomy on NAEP test scores at the state level in particular on the percent pas sing at the Basic Level and the percent passing at the Proficient Level Their paper does not focus on a particular initiative, but instead traces a general decentralization trend occurring between 1993 and 2000. They reported level of autonomy on the NCES Schools and Staffing Survey. 12 Strunk and Loeb look specifically at the interaction between accountability principal 10 All results are for elementary schools only. 11 The estimates provided in their paper do not attempt to separate the selection effect from the causal effect of school based management. The researchers define this combined effect as the impact of the recommended by their district superintendents as in good shape fiscally and interes ted in trying a new 12 Principals select a n umber between 0 and 5, with 5 being maximum autonomy. Accountability is also measured on an index from 0 to 5. States like Florida, which have high school exit exams, performance incentives, and annual exams, receive a 5 on the accountability rank.
17 autonomy is more successful under systems o f stronger accountability. 13 Principal control of spending has a statistically significant and negative impact on both percent passing at the Basic and Proficient levels. When principal autonomy, however, is interacted with accountability, the impact is p ositive and significant. Stiefel et al. also examine the impact of budgetary autonomy on expenditures. They use three measures of expenditures: the total amount of reimbursable funds, the share of total revenues that are reimbursable funds, and th e share of classroom expenditures as a percentage of total tax levy and operating aid revenues. The final specification is used to see if autonomy causes schools to allocate more resources to the classroom where it is most directly consumed by students. The results are mixed and most estimates are not statistically significant. 14 The only significant results show that planning to become autonomous reduced the share and level of reimbursable expenditures. 15 Since autonomous public schools are intermediate between traditional public schools and charter schools studies analyzing the impact of charter school attendance on student outcomes provide evidence of the impacts broader autonomy may present T here is now some convincing evidence that charter schools can improve educational outcomes (Hoxby and Rockoff 2004, Hoxby, Kang, Murarka 2009, Angrist et al. 2009). 13 See Strunk and Loeb, Table 9, 36. 14 that the planned PDB 1996 97 dummy variable is the only one that has a significant impact on any of our expenditure va riables. Its effect is negative and significant on the reimbursable per pupil expenditure ( 0.01201) and on the share of reimbursable expenditure as a percent of total expenditures ( 1.4453). Its effect is still negative, but insignificant on total per p upil expenditure and the share of classroom teacher expenditure. The implemented PDB 1997 98 and planned PDB 1998 99 show mixed, statistically 15 How reimbursable expenditures may be connected to the production of school quality is not explained.
18 These studies identify the impact of charter school attendance on student test scores by using lotteries in oversubscribed schools. These schools are oversubscribed because tell us the implications of charter schools for the wider population. 16 This sample identifies effects for students entering better charter s chools, i.e. the charter schools most strongly demanded by parents. Clark (2009) uses a regression discontinuity design that does not hinge on lotteries 17 to identify the impact of grant maintained schools in the UK, a program very much like U.S. charter s scores. He finds that grant maintained schools produce relatively large improvements in student achievement. Even in light of these findings, w e know relatively little about how charter schools and other autonomous schools achi eve these r esults and the types of changes that best serve a particular population of students (Angrist et al. 2010). Angrist et al. (2010) provides evidence that a particular charter school in New England increased student performance for a student popul ation traditionally not served by charters. 18 This school This study examines the impact that budgetary autonomy alone will have on student achievement. Beyond stud ies that address the impact of autonomous decision making in public schools, the re is a growing body of research that analyzes the role of the principal in the 16 This is noted in Angrist et al. (2009). 17 To participate in the program, parents at the school had to vote whether to become a grant maintained school or remain a traditional school. A majority vote determined program participation. 18 The schools population was predominately Hispanic and contained a larger proportion of LEP and special education students.
19 production of education (see e.g. Blank 1987, Horng, Kalorgrides, and Loeb 2009, Hanushek and Riv kin 2009 ). While it is difficult to draw general results from this comes (Horng, Klasik, and Loeb 2009 ). Since organiz ational management is a category that inclu des managing the school budget, this provides some support for pursuing a better understanding of the effects of budgetary autonomy 1.3 Program Specifics The program used to motivate the model of autonom y present ed in this chapter, which is examined empirically in Chapter 2 is the Autonomy Zone/Empowerment Sc hools Program implemented in New York City public schools. The program began in 2004 2005 sc hool year, as a pilot program with 29 schools 19 The initial fun ding for the program came from The Fund for Public Schools, a non profit group that channels private donations to New York City public school initiatives. 20 The Fund contributed about $1.3 billion, approximately $43 dollars per student, to cover initial co sts. The pilot program lasted for two years, with 19 additional schools entering the program in the second year. of autonomy and flexibility in decision making in exchange for g reater accountability 19 The number of schools entering by year comes from a data set of Empowerment Sc hools and entry dates provided by the NYCDOE. 20 January 30, 2011, http://schools.nyc.gov/rdonlyres/3062CAC4 0056 4C27 A77D COF5498EC599/0/Annualreport2005.pdf
20 21 successful and as a result the program was expanded under the Empowerment Schools Program (Empowerment Network). 22 In 2006 2007 and 2007 2008, an additio nal 216 23 and 246 24 schools, respectively, joined the program. Throughout the evolution of this program, the key component has been increased budgetary and decision making autonomy for the principal s who participate The principal alone has the ability to a pply to join the program. Once the principal decides to apply he/she submits a contract including perfo rmance goals to the district. 25 Following these submissions, the district must approve the participation of the school. 26 Between 2004 and 2007, 22 sch ools that wished to participate were turned down. 27 The program increased budgetary autonomy through two main channels : 1) by providing increased level of discretionary funds and 2) increased procurement caps 21 Ibid., 17. 22 2009, http://schools.nyc.gov/NR/rdonlyres/6AFCD437 0386 4907 AE52 FDE6E8CB2E48/57528/ESOBrochure2009.pdf 23 The NYCDOE website officially reports 332 schools participated in th e first year (non pilot) of the Empowerment Schools program. Based on my data, I only account for 264 schools (29+19+216) in the first year of this phase of the program. This is because I do not include charter schools in the dataset. Charter schools we re given the choice to join the program at this phase. These schools did not join to gain autonomy, since by definition they already have extensive autonomy, but instead to collaborate with other schools on what works. 24 This brings my total to 505 schools entering the program between the start of the 2004 and 2007 school years. In my data another 30 schools enter in 2008 bringing the total to 535 schools. My count of schools in the program in 2008, however, is 524 because 11 schools exit the program in 2 008. This does not correspond to the number reported by the NYCDOE because charter schools are excluded in my study. 25 more than 300 principals signed sweeping new accountability agreements that gave them more control over school budgets and curriculums and freedom from regulations that many schools have found 26 27 I c annot identify these schools and do not know the reason NYC decided to block their participation.
21 First, the program increased the level of d iscretionary funds available to the participating principal by reallocating monies from the district budget to the school 28 and by offering a budgetary reward tied to student achievement. 29 Schools on average gained control of approximately $250,000 more in discretionary funds. Of this $250,000, 60% was in the form of additional funds and 40% was newly unrestricted funds that were previously 30 Since the average school has about 766 students in New York City, this increased unrestricted expenditures per student by about $326. As a percentage of a school s total budget 31 this increased control of the budget by 2.5%. The additional funds only amount to about a 1.5% increase in the budget of the schoo l. Schools, of course, allocated these funds in diverse ways. 32 The following shows h ow two schools chose to use their funds. 33 In School A, with 384 [e] xtra [c] urricular [a] ctivities, incl uding Yearbook and Choir, [p] artially [f] unded two coaches to provide professional development to teachers to increase instruction quality, 28 Examples of this can be found in School Allocation Memorandum. For instance, School Allocation Memorandum No. 52, FY06 shows previously centralized funds b eing disbursed to autonomous schools Department of Education, last revised July 26, 2005, http://schools.nyc.gov/offices/d_chanc_oper/budget/dbor/allocationmemo/fy05 06/datafiles/sam52.pdf acc essed through Lexis Nexis, which tracks the same movement of Arts funding. 29 (proficient) on their quality reviews will be eligible for rewards and recognitions, including ex FAQ. 30 Autonomy Zone FAQ. 31 The average total budget of a school is about 10 million. 32 I investigate if there are common changes in the distr ibution of funds in Chapter 3 33 ment of Education, accessed January 31, 2012, http://schools.nyc.gov/NR/rdonlyres/44CC868E 9EAE 46FC AA51 62BE7813A2DC/30315/Empowerm entSchoolsbrochure.pdf
22 students and $228,000 in additional d and 1 new paraprofessional, started a drama program, and focused on professional development for math teachers to increase the quality of math instruction received by While the increase in per is moderate arguably from another component of the program. T he program also doubled the procurement cap for expenditures without dis trict approval from $2,500 to $5,000 34 T he program also expanded the ability of the principal to purchase services from vendors not contracted with the district. S chools gained the ability to buy services from vendors not contracted with the district for up to $25,000. 35 I argue that t hese changes provide a considerable increase in school level autonomy. Part of the increase in discretionary funds came from a budgetary reward for rogress reports and +(well d 36 The grade a school receives on their Progress Report Cards is determined in part by an increase in student achievement above the previous year. All schools in the district receive a Progress 34 Autonomy Zone FAQ. 35 Ibid. 36 Ibid.
23 Report Card 37 but only autonomous schools are eligible for a budgetary reward as a result. 38 performance. Therefore, a schoo l that enters the program in the 2004 2005 school year could receive this reward at the start of the 2005 2006 school year. This characteristic of the program, therefore, not only increased the level of discretionary funds, but also strengthened the align ment between the incentives of the principal and student achievement. For a school to earn an A on their Progress Report, the raw score they of this required improvemen t is dependent on the current percentile rank of the school in terms of student achievement and the composition of the student body. In this way, the district is attempting to not unduly penalize schools that for instance already have a high percentile ra nk and therefore are not likely to make large gains, as well as schools with more difficult to educate student populations. These required gains are aligned with the conditions required to achieve Adequate Yearly Progress 39 Progre ss Report Card is comprised of ratings of the school environment (15%), student For middle schools and elementary schools, which is the focus of this study, student progress is measure d by average student improvement from the prior year to the current year on 37 The district has produced these reports for individual schools since 1994. The standardization of these reports, however, was also part of this initiative. http://query.nytimes.com/gst/fullpage.html?res=9C06E6D71238F932A35753C1A9629C8B63&scp=34&sq =progress%20report%20cards&st=cse. 38 There are other programs that use the Progress Report cards for perso nal performance rewards for principals and teachers. 39 This requirement comes from the No Child Left Behind legislation.
24 English/Language Arts and Math exams. 40 contingent upon test scores in middle schools and elementary schools. Schools also gained increase d decision making flexibility beyond the increases in their discretionary budgets and procurement limits. Schools received greater flexibility in how to allocate funds earmarked for a particular use. For instance, schools received the money allocated for professional development, but were required to spend it toward the same purpose. 41 Finally, in selecting to participate in this program, principals also agreed to Princ ipals who failed to meet the expected gains in scores faced consequences as a result. The possible consequences ranged from district interventions to leadership changes and school closure. 42 The accountability agreements lay out sanctions -including re moval -43 The district meted out punishments particularly for schools participating in the program that 40 The model simplifies this setting by modeling the budgetary rewards as tied directly to the average gains in standardized test scor es. Since this accounts for 60% of the Progress Report Score in middle and elementary schools, this simplification should still provide a reasonable approximation of the incentives faced by a principal. Further, the interpretation of the production funct ion presented in the model as student test scores could be expanded to include other elements. 41 Professional development activities such as workshop series, mentoring/team teaching, study groups and tuition r eimbursement Per session for teachers attending training to become highly qualified Additional teachers for push in or pull out academic intervention services Extension of the school day to enable longer blocks of instruction coupled with more intensive su pport January 31, 2011, http://schools.nyc.gov/offices/d_chanc_oper/budget/dbor/allocationmemo/fy05 06/ datafiles/sam76.pdf. The term "per session" refers to time outside of normal schools hours, e.g. after school or during summer. 42 Autonomy Zone FAQ. 43
25 receive a grade of D or F on their progress report or a grade of C in three consecutive yea rs. Further, since principals in the Empowerment Schools network are more directly responsible for the outcomes of their students, consequences of poor performance may also be in the form of decreased professional status leading to less desirable career a dvancement. Conversely, meeting the expected gains may allow principals to signal their ability and result in more rapid promotion. 44 1.4 Model Specification 1.4.1 General Specification The model developed in this chapter captures key elements of the a utonomy program implemented in New York City Public Schools. The theoretical analysis directs us toward testable hypotheses. This provides a basis for the empirical investigation in Chapter 2. Consider a model in which the principal of the school is the sole decision maker. The principal seeks to maximize her own utility. The principal, first, selects between autonomy and centralized control. U nder autonomy the principal undertakes an increased role in budgetary allocation decisions and faces sharper incentives tied to student outcomes. Alternatively, centralization grants the principal a more limited decision making role without outcome based incentives. All school principals share the same underlying increasing, twice differentiable, and concave ut ility function though parameterized by principal characteristics They care directly about the wage they earn a generalized concept of economic rents the quality of education and the personal costs and/or rewards 44 See Cullen and Mazzeo (2007) for an analysis of internal principal labor markets.
2 6 incurred in the process associated with educating students, as well as other components that vary with the educational regime, as discussed below. In both educational regim es, a principal is motivated by the salary they earn as a result of their employment in NYCPS. It is assumed throughout that the wage rate is assumption allows the analysis to focus on the impact of allocating expenditures between categories that produce test scores and other categories that may be preferred by an individual principal. As a result, the model does not address questions about if the autono my program causes principals to leave the profession or changes the composition of principals in the district. 45 Principals differ along two dimensions: personal characteristics and the student composition of the schools they serve. Personal characteristi cs include their effectiveness and th eir tastes for rents, quality, and professional exposure The student populations of each school are composed of two types of students Each student type is typified by two parameters: the variance of their test scores and their test taking effectiveness I assume type 1 students are high effectiveness and type 2 students are low effectiveness. Each school is composed of type 1 students and type 2 students The number of each type is fixed within a school. Therefore, the total number of students served by a school is 45 The data from NYCPS does not indicate that the composition of principals is changing.
27 The quality of education is measured by gain s in the average performance o f determined by the following Cobb Douglas production function ( 1 1) denote s per student principal effort directed toward type i The form of Equation 1 1 has the desirable feature that expenditures and effort are complementary. An increase Expenditures and effort are subscripted by i because the principal ca n direct these inputs toward a particular student type though expenditure allocation is mandated in the non autonomous regime developing programs, providing instructional support, managing the staff and budget, and/or performing other administrative functions, such as discipline. 46 The production function is assumed to exhibit nonincreasing returns 47 to scale in inputs, ffort and expenditures will produce less than a proportionate increase in test scores. This is a good approximation of the fact that increasing test scores becomes more difficult as inputs are increased. To capture the differing test taking ability of s tudent types, the student level effectiveness parameter varies across types. 46 ee Horng, Klasik, and Loeb (2009). The study shows that principals in schools with a high percentage of low income students spent more time in their day on administrative tasks than principals at lower poverty schools. This suggests that allocating effor t to a student type is a relevant concept. 47 maximization problem.
28 and their stock of prior knowledge that contributes to test taking ability. are the high effect iveness type, where The random component of test scores varies across types, captures random individual performance and unobserved variation in test taking ability within types. T he random component can also refl ect variation in test precision Under both organizational structures, t he pr incipal knows the distribution of the error terms, but cannot affect the distribution s Since the quality of educ ation is measured as the gain in average test scores, each school faces an exogenous target level score set by the district. Therefore, sch ool quality is measured as ( 1 2) where is the average score of group i students and represents the target scores. 48 49 Since student test scores contain a random component, the principal faces some uncertainty tied to a sch 48 previous period plus the gain expected by the district. In the case of NYC schools, as well as other accountability structures in 49 Linking school quality to a target test score is warranted, particularly in reference to New York City Schools, because all schools were given performance targets generated as function of their performance in the prior year and weighted by the student population in the prior year. For a good explanation of the Winters (2009).
29 principals have constant absolute risk averse (CARA) preferences represented by a negative exponential utility function of the following generic form (1 3) where is the constant coefficient of absolute risk aversion, is the strength of the quality. 50 Since the principal knows only the distribution of potential gains and gains im pact her utility in both regimes, the principal maximizes her expected utility. M equivalent D ue to the specification of the utility function and the normal dis tribution of the errors, the resulting certainty equivalent is of a simple, additive form The certainty equivalent is the minimum/nonrandom level of utility the principal would accept rather than facing an uncertain payoff as a result of testing performa nce randomness. The certainty equivalent will be less than expected utility given the uncertainty. 51 Given that wages enter as an additive constant and the participation constraint izing ( 1 4) Increases and decreases in salary can still be examined through the personal costs a principal may incur from joining the autonomy program. Note that principals will differ with respect to their characteristic vector as well as the student bodies they serve. In addition, the decision variables, constraints and incentive structures vary across the two organizational options, i.e. centralized 50 The personal cost term, P, is discussed below. 51 For a general discussion of the certainty equivalent and exponential utility, see Patrick Bolton and Mathias Dewatripont, Contract Theory (Cambridge: The MIT Press, 2005), 137 139.
30 control and autonomy. These differences are described in the two fol lowing subsections. 1.4.2 Centralized Control Under centralized control, the district mandates all expenditure decisions and no rents can arise The principal maintains the ability to select her preferred levels of targeted efforts given these expenditu re allocations. These effort allocations are not observable to the district or the researcher. I assume that the district cares about maximizing educational quality, but requires expenditure equalization across types due to equity concerns. Quality maxi mization subject to the equity constraint then implies a balanced budget with and where Z is an exogenous amount of per pupil funding Given these fixed allocations, the principal then proceeds to set targeted effort levels that maximize her utility. Since the centralized setting leaves the principal with zero rents, ( 1 5) where is the co nvex disutility of her effort vector Under centralized management, the personal costs, P, enter the utility functions as The effort cost function is defined as ( 1 6) Although exerting effort is privately costly for the principal, increased effort increases the productivi ty of expenditures and benefits the principal through her preference for quality.
31 The principal therefore maximizes the following certainty equivalent when the school is centralized (1 7) by choosing her targeted efforts 52 Her choice of effort does no t impact the uncertainty cost she faces, which are given by the last term in Equation 1 7 1.4.3 Autonomy If the principal selects autonomy, then her utility function is given by ( 1 8) As in the centralized setting, the principal still c ares about the quality of education measured by and faces the same cost of effort function Now, however, the principal is not sub ject to the equity or balanced budget constraints imposed by the district in the centralized setting. This allows principals the opportunity to consume some of the budget as rents, I refer to these as generalized rents to indicate that the fun ds may be spent on programs the principal values or believes are advantageous to the student population. By definition, these are not allocated in any way that increases student test scores, the measure of school quality. Note that if and can be directly consumed, then the consumption of rents would be equivalent to the consumption of wages. As discussed previously in section I, I suppress the notation for wages and allow to be less than 1. 52 With CARA preferences and the normality of the uncertainty in the payoff, the certainty equivalent equals the mean payoff minus one half the variance of the payoff.
32 In contrast to the cen tralized setting, the principal also incurs a further personal impact capturing under autonomy relative to the centralized setting accounts for the career impact of average test score gains or losses. If her efforts produce average scores above the target scores expected by the district the principal may receive professional recognition in the form of a promotion or a raise. Similarly, missing the performance targ et could lead to sanctions for the principal, ranging from being removed or reassigned to a lesser position to increased scrutiny by central administration and/or parents. 53 These under autonomy make clea r the dynamic nature of the effects in reality A portion of the personal costs incurred in my static model can, therefore, be attributed to the discounted value of the rewards (costs) of the one period gain (loss) in test scores. 54 Under autonomy, the principal is given a budgetary reward for gains in test scores in addition to the standard funding NZ The reward is dependent upon the size of the gain in scores and flows directly into the rents she enjoys. Further, the reward is assumed to be ( 1 9) where is a coefficient set by the district that translates school quality Q into a dollar amount The reward is not received by the principal until the scores have been real ized. The principal is then constrained by the following budge t constraint 53 The specifics of these costs are somewhat ambiguously defined by the New York City Department of Education. While principals were told that in undertaking autonomy failure to meet goals could result in removal from their current position, this punishment has not been used in practice. 54 Cullen and Mazzeo (2007) examine the extent to which the internal labor markets of school districts re wards principals for student achievement. They find evidence of a competitive labor market that appears to reward performance.
33 (1 10) This implies that the total budgetary reward cQ, is consumed as rents. 55 56 T generalized rents are (1 11) These g eneralized rents are the total budget funds not assigned to the production of scores. The structure of these rewards creates a regime where the principal faces stronger incentives under autonomy even if she does not care about rents. If the principal does not care about rents then the budgetary reward will not aff ect the 57 The move to autonomy, however, has a second exposure effect that still pr ovides incentives for improving quality without a principal having a preference for rent s. Under autonomy, the princ ipal maximizes the following certainty equivalent (1 12) by choosing efforts and expendit ures subject to Constraint 10 As with centralized control, the principal's allocation decisions do not impact the uncertainty costs they bear. The uncertai nty costs unde r autonomy, the last term of Equation 1 12 will always 55 Since the budgetary reward is given after test score gains (or losses) have been realized, this prevents schools in the model from spending a portion of the budget they do not yet have. 56 By forcing the budgetary reward to be consumed as rents, I model one of the dynamic elements of the environment while maintaining a simple one period setting. This specification likely understates the value of this budgetary reward, since the reward may be used productively in a dynamic model. Principals may also elect to fund arts or special interest programs that are not directly related to the production of test scores. 57 The principal will always be on her budget constraint in this setting.
34 exceed those under centralized control if the principal cares about the budgetary reward or the additional impact on their career, i.e. if 1.5 Who Selects Autonomy The pri ncipal's effort and allocation decisions are regime specific and determine the expected test score effects for students. Once we know how school and principal characteristics vary with the organizational regime they select we can the n analyze the expecte d program impacts In selecting between autonomy and centralized control, the principal is weighing the impact on consumption of generalized rents, school quality, and other personal costs and benefits. 1.5 .1 General Selection A principal selects the or ganizat ional regime that yields the higher expected utility For a principal to select autonomy over centralization, the following condition must hold ( 1 13) where and inty equivalent given autonomy and centra lized control respectively. Equation 1 13 can be rewritten to make the tradeoffs the principal considers explicit ( 1 14) where The potential benefits of autonomy are the consumption of genera lized rents the budgetary reward received for producing quality and improved average test scores Autonomy, also, carries with it added career impact, to producing quality. Schools that fail to m eet the target
35 score may cost principals future promotions or invite other negative sanctions. The principal will also compare their effort costs under the two regimes. Finally, under autonomy there is an increased weight to uncertainty. 1.5 .2 Two Type s of Autonomous Schools The principal makes a selection decision contingent upon the maximum well being she can achieve under each regime. Given autonomy, the principal's optimal allocations can take two different forms. These forms are the result of the latitude autonomy grants the principal to pursue diverse goals. She can devote her resources, both efforts and expenditures, to improving average test scores and/or to consuming generalized rents. The first case is characterized by principals who pursue improved test scores such that they exhaust their total budget in the process. The second case entails the consumption of some generalized rents from the initial budget 58 Whether an autonomous principal will choose to consume rents depends on the following condition ( 1 15) If Condition 15 holds, the principal will find it optimal to spend her total initial budget. Condition 15 yields a few direct implications. First, if a principal does not care about rents then the principal will spend all of the budget and no generalized rents will be consumed. Assuming for at least some schools in the district, Condition 15 relative ly more type 1 students are less likely to consu me rents. Smaller schools are 58 generate improvements in test scores.
36 less likely to consume rents because each student has a larger marginal impact on their e, more career oriented, and more concerned with school quality. 59 The empirically relevant case is for type 2 student s to be a majority. Condition 15 further suggests that duced to consume generalized rents. This is because it is more costly to increase scores for weaker students. Empirical evidence on principal sorting within a district has shown that principal experience and credentials, two components of effectivenes s, are inversely related to school size and the proportion of type 2 students (Branch, Hanushek, and Rivkin (2009) and Horng, Kalgorides, and Loeb (2009)). This suggests that the parameters that push a principal toward consuming rents are positi vely corre lated. As a result, Condition 15 illustrates that given autonomy, the already high performing schools are more likely to exhaust their budgets on test performance improvement than the low performing schools. To simplify the succeeding discussion of resul ts, I define two groups of schools based on Condition 15 Group 1 is characterized by high effectiveness principals, relatively more high effectiveness students, and small school size. Group 1 schools optimally choose not to consume any generalized rents Group 2, therefore, is composed of schools with low effectiveness principals, more low test effectiveness students, and a large school size. The differences between these groups are 59 This equates to higher values of and respectively.
37 summarized in Table 1 1 For the remaining analysis, I address the em pirically relevant case 60 where type 1 students are the smaller student group i.e. 18.104.22.168 The s electio n decision for non rent consuming schools Knowing which schools will balance their budgets given autonomy, group 1 schools, and which sc hools will take rents, group 2 schools, is helpful for characterizing principal's selection between centralization and autonomy. Beginning with group 1 schools, their selection condition reduces to a comparison of expected average test scores under each r egime and the increased uncertainty cost. Group 1 schools face this simplified condition because they optimally choose to consume no rents. The selection condition for group 1 schools can be written as ( 1 16) where and autonomy and centralized control, respectively. Due to the increased flexibility granted under autonomy and the sharper incentives (and the budgetary exhaustion), the will necessarily be higher than under centralized control. Since a principal still faces the added impact of not meeting the target score, and the uncertainty cost, the gains in average test scores must be large enough to cover these incre ased costs. The impact of each parameter on a group 1 principal's propensity to select au tonomy is summarized in Table 1 2. Table 1 2 tracks the role of each parameter in 60 One way to think about these two student groups is in terms of free and reduced lunch status. The average school in New York City is composed of 78.2% free and reduced lunch eligible and 21.8% non eligible.
38 the tradeoffs the principal faces. Column 1 reports the impact on the value of th e test score difference. The value of the test score difference is ( 1 17) Column 2 reports the parameters impact on minus the cost of effort. I examine the negative of this difference so that a plus sign means autonomy is more likely to be select ed. Column 3 reports the difference in the uncertainty costs between regimes. These costs are weighted by the relevant taste parameters. This difference is expresse d on the right hand side of Equation 1 16 61 The group 1 schools with the highest propens ity to select autonomy will have low they face and uncertainty costs will always be greater under autonomy. Higher values of principal effectiveness, and the size of the school, increase the likelihood a group 1 principal will select autonomy. 62 will impact a group 1 school's propensity to become autonomous through the budgetary reward and the uncertainty they face. An increased preference for rents increases the weight of uncertainty and therefore make principal's less likely to select autonomy. This characteristic's net impact on the budgetary reward is related to the relationship be tween the average test score a school can expect given autonomy and the value of the target score. Group 1 principals know that the value of the difference between the expected test score under autonomy and centralized control will be positive. What ma tters for the 61 The proof of these comparative static results is c ontained in Appendix C. 62 Note that there is an upper bound on the size of the school that will choose not to consume rents, but the largest of these schools will be the most likely to select autonom y.
39 budgetary reward, however, is how the test score achieved with autonomy compares to the target score. Therefore, will positively impact how the principal values scores across regimes only if the average score expected under aut onomy exceeds the target score. This effect creates a disincentive for schools that have difficulty achieving test score gains to join the program. Combining the impact on the budgetary reward and uncertainty costs yields the overall impact on selection. When the expected score is below the target score, preference for rents will have an overall negative impact on propensity to select autonomy. When the expected score, however, is greater than the target score the impact is indeterminate. The role of the career impact parameter, is symmetric to the proceeding discussion. The strength of the career impact will positively impact the value of the test score difference if the average score achieved under autonomy exceeds the target score. If t he average score a principal earns under autonomy is less than the target score, then their career impact will negatively impact the value of the test score across regimes. Therefore, the career impact will have a determinate effect on the propensity to s elect autonomy only when the expected score given autonomy is less than the target score. The other preference parameter, however, always has an indeterminate effect student achievement positively impact the gains in expected average test scores that result under autonomy. The parameter, however, also increases the uncertainty cost. The preference for student achievement does not interact with the target score The target score is not intrinsically important to a principal.
40 22.214.171.124 The selection decision for rent consuming schools If instead a principal will choose to consume generalized rents from the current budget, it is necessary to further consi der the choices a principal in a group 2 school would make. Group 2 schools, again, are characterized as having low principal effectiveness, large numbers of students, and a strong majority of the lower test effectiveness type. These schools' selection de cision can be reduced to the following tradeoffs ( 1 18) The individual components of a group 2 principal's selection decision are: rents, the value of test scores, cost of effort, and uncertainty cost. The relative uncertainty costs group 2 sch ools face are the same as those faced by group 1 schools. Group 2 schools, however, tend to be larger. The uncertainty costs, therefore, are likely lower for these schools. For group 2 schools, the impact of all characteristics except risk aversion on t he propensity to select autonomy is indeterminate. The results are s ummarized in Table 1 3 The indeterminacy is the result of the increased tradeoffs that are available. Under autonomy with rents, the principal can choose to consume a little generaliz ed rents and still increase test scores or they can choose to consume a larger amount of rents to the detriment of test scores. The increased flexibility allows the principal to use characteristics other than risk aversion to support either goal. To gain some insight about rent consuming schools decisions, I look at individual components of the autonomy decision, beginning with the optimal amount of rents
41 consumed. The amount of generalized rents a principal will consume is decreasing in ffectiveness, their preference for quality, and their career impact a group 2 school will consume less rents under autonomy. The composition of the student body can also compel a principal to consume rents. First, principals at the helm of larger schools will have a propensity to consume more rents. Also, schools with relatively more group 2 students, the type with the lower marginal productivity, will have a proclivity for consuming more generalized rents. The impact of a principal's taste for rents, on the amount of rent they consume depends on the relationship between the budgetary reward, the other taste parameters, i.e. and and the value of the output elasticity of expenditures. In particular, the impact of the principal's ta ste for rents depends on ( 1 19) The stronger the incentives tied to the budgetary reward are relative to other incentives tied to the avera ge test score and the output elasiticity of effort, the more preference for rents will positi vely impact test scores. If 1 9 holds, then the principal's preference for rents will decrease the consumption of rents because the budgetary reward wins out, othe does not impact the amount of rents she pursues. Beyond generalized rents, group 2 principals also compare the value of the quality they can expect under each regime. Th e average test scores group 2 schools expect to achieve under autonomy may be either greater than, less than, or equal to those they expect under centralization. These schools are balancing the marginal
42 benefit of average test score effects against the fo rgone rents. There are few unambiguous characteristic effects on the value of test scores for group 2 schools. All of the unambiguous effects relate to one of the parameters that is only in play under autonomy, and There are two different defined impacts for the tas te for rents parameters. If 19 holds and the expected average score given autonomy exceeds the target score, then the principal's preference for rents, will increase the value of the test score diff erence between regimes. This result stems from the interaction of two effects of the impact on the actual scores and the impact on how the principal values that score (its relation to the targ et score). When 19 holds the allocation of expe nditures and effort is increasing with and therefore the expected average test score is also increasing. The second effect, the reward for producing the test score, will also have a positive impact on the value of the test score difference in this case. The career impact, will have a determinate impact on the value of the test score difference when the expected average score exceeds the target score. This is because always positively impacts the average test s core under autonomy and the test score gain will positively impact the principal's well being when they earn some budgetary reward as a result. Another component of the principal's decision to select autonomy is the impact on effort costs. Changes in the principal's cost of effort mirror the changes in the expected school level test scores discussed in the preceding paragraphs. If the expected average test score is increasing, then the principal's cost of effort must be increasing. The final compon ent of a group 2 principal's selection decisions is the
43 change in uncertainty costs. The uncertainty costs are the same for group 1 and group 2 schools. Therefore, all the results from group 1 schools continue to hold. The analysis of group 2 schools s election decisions shows that autonomy allows for schools to have many different experiences within the program. 1.6 The Impact of Autonomy on Test Scores 1.6.1 Impact on School Level Test Scores A central intention of this research is to understand how autonomy impacts student performance. I begin by examining the impact on school level test scores. For group 1 schools, expected average test scores necessarily increase given autonomy, For group 1 schools who select autonomy, the expecte d average test scores must increase by a sufficient margin to cover uncertainty costs and increased sensitivity to the target score. The following comparative statics results suggest how these gains are impacted by principal and student characteristics. Increases in all characteristics except risk aversion increase the expected average score for group 1 principals who choose autonomy. Autonomy simply allows the principal to more effectively incorporate the characteristics into their allocation decisions Therefore, in comparison to centralized control increasing the value of student and principal characteristics has a larger, positive impact on student test scores. These re sults are summarized in Table 1 2 The expected average score impacts for group 2 schools' characteristics are more complex relative to those for group 1 due to the increased tradeoffs discussed in the previous section. The following results provide further insight into the types of schools that will pursue higher test scores after selecting autonomy. As with group 1 schools, a principal who faces greater career impact will produce higher test scores
44 under autonomy than centralized control as long as the expected average test score exceeds the target score. Also, like group 1 scho impact. The only other determinate impact is for the principal's taste for rents. If 19 holds then taste for rents will increase the expected average test score. It is not clear whether the difference between exp ected test scores is increasing or decreasing with ; school size, N; the ratio of type 1 to type 2 students, ; and the effectiveness of the principal, k. 1.6. 2 Impact on Individual Type's Test Score Another aim of this paper is explore whether autonomy has differential impacts by student type. All differential impacts stem from the type specific allocation of inputs possible under both regimes. Given centralize d control, only efforts can be assigned to a specific type. Some of t he dist ributional effects are contingent on whether the school is classified as group 1 or group 2. In Appendix B I show that: PROPOSITION 1: Given autonomy, a principal will contr ibute more total inputs i.e. more of both effort and expenditures, to type 1 students relative to type 2 students. COROLLARY 1: The expected test score of type 1 students will exceed those of type 2 student given autonomy. Proposition 1 and Corollar y 1 make sense in light of the fact that assigning funds than assigning funds to type 2 students. Accountability designs that focus on the lowest 25% or a minimum lev el of proficiency will cause principals to focus allocations to achieve test score increases for the population with the greatest impact on those targets. Autonomy generally increases the ability of the principal to respond to a given
45 set of incentives. The incentives set forth are paramount to the progra m outcomes. Also, in Appendix B I prove that PROPOSITION 2: Autonomous, group 1 schools will contribute more total inputs i.e. more of both expenditures and effort, to type 1 students than they would receive under centralized control. As a result, type 1 students are expected to achieve higher test scores under autonomy with no rents consumed than they would have under centralized control. For type 2 students, however, it is difficult to determ ine the ultimate impact on their expected scores. Although
46 type's test scores under either regime. All other characteristics positively impact both type 1 and type 2 students. For group 2 schools, the results are summarized in the last three columns of Table 1 3 The comparative statics results are of the form Career impact, positively impacts the scor es of both types. When 19 holds, the principal's taste for rents also positively impacts the scores of both types. As with group 1 s chools, risk aversion does not impact test scores. The remaining characteristics all have indeterminate effects. The results on rent consumption, propensity to select autonomy, and the ultimate test score impacts of aut onomy are summarized in Table 1 4. Table 1 4 indicates that selection is impacted by principal and student characteristics. This table also shows that selection and the individual characteristics of the school and principal are critical to understanding test score impacts. 1.7 Conclusion Autonomy allows the principal to better utilize their own preferences and characteristics, as well as the characteristics of the student body they serve. This can benefit students when the incentives are such that student achievement is valued more highl y than generalized rents. If, however, the incentives tied to test scores are not sufficiently strong the principal may choose to use the latitude for their personal benefit. In other words, how the principal values their own gains relative to those of s tudents greatly impacts the success of such a program. Schools that have smaller student populations, more effective principals, and principals who care more about their students and their careers will be likely to select
47 autonomy without consuming gener alized rents. Group 1 schools will always improve the expected school level test score and the test score expected for the more able student type. For these schools, there is also the possibility that principals will increase effort sufficiently to posit ively impact less able students scores as well. The success of such programs depends on how well districts understand the impact of student and principal characteristics. Further research is needed to determine principal's tastes for career, student achi evement, and generalized rents.
48 Table 1 1. Definition of school groups Principal and school c haracteristics Group 1 Group 2 Generalized r ents No Yes Principal e ffectiveness, k High Low Career i High Low Taste for q High Low Type r atio, n1/n2 High Low School s ize, N Low High
49 Table 1 2. Group 1: Characteristic effects on autonomy decision and test scores Conditions Value of test score difference 1 Cost of Effort Difference2 Uncertainty cost difference 2 Propensity to select autonomy Average score impact 3 Type 1 score impact Type 2 score impact Risk no effect no effect no effect no effect no effect Principal effectiveness, k + no effect + + + + Taste for S A,1* >G 4 + indeterminate + + + S A,1* G + indeterminate + + + S A,1*
50 Table 1 3. Group 2: Characteristic effects on autonomy decision and test scores Conditions Rents Value of test score difference 1 Cost of effort difference 2 Unce rtainty cost difference 2 Propensity to select autonomy Average score impact 3 Type 1 score impact Type 2 score impact Risk no effect no effect no effect no effect no effect no effect Principal effectiveness, k I ndetermina te indeterminate no effect indeterminate indeterminate indeterminate indeterminate Taste for (2 4 and S A,2* >G 5 + indeterminate + + + (2 and S A,2* G + indeterminate + indeterminate (2 and S A,2* G + indeterminate + + + S A,2*
51 1 For Group 2 schools the value of the average test score expecte d under autonomy may be greater than, less than, or equal to the value expected under centralized control. 2 These columns examine impacts on minus the cost difference. For example, minus the cost of effort is (C(e^A,2*) C(e^C*)). This is done for co nsistency, i.e. so that a + means more likely to choose autonomy and means less likely. 3 All test score impacts consider the difference between the score under autonomy and the score under centralized control. These results come from comparative stat 4 This condition states that the weight of the budgetary reward exceeds the weight of the generalized rents. 5 This condition states that the average score expected under autonomy exceeds the target score se t by the district. 6 It is assumed that n1/n2 <1. 7 This result hinges on the assumption that the standard deviation of type 1 student's test scores is less than or equal to tha t of type 2 students. If the standard deviation of the type's test sc ores are symmetric, then the ratio of student's has no effect. Indeterminacies: There are three reasons for the indeterminacies observed in this table. et score. ies exist because the relationship between the amount of expenditures spent on type 1 given autonomy with rents cannot be compared to the per student allocation under centralization.
52 Table 1 4. Characteristic effects summary Test score ef fects 1 Propensity to select autonomy No rents schools Rent consuming schools Propensity to consume no rents No rents schools Rent consuming schools Average High ability type Low ability type Average High ability type Low ability type Risk no effect no effect no effect no effect no effect no effect no effect Principal Effectiveness, k + + indeterminate + + + indeterminate indeterminate indeterminate Taste for indeterminate/ indeterminate + + + +/ +/ Taste for + indeterminate indeterminate + + + indeterminate indeterminate I ndeterminate Career + indeterminate/ indeterminate + + + + + + School Size, N + indeterminate + + + indeterminate indeterminate I nd eterminate Type Ratio, n1/n2 2 + indeterminate indeterminate + + indeterminate indeterminate I ndeterminate Notes: This table summarizes results from th e previous tables, i.e. Table 1 2 : Group 1 Char acteristic Effects and Table 1 3 : Group 2 Characteri stic Effects. The table indicates the impact of each parameter on the decision to consume rents, the decision to select auto nomy, and the test score impact given autonomy has been selected. 1 All test score impacts consider the difference between the sco re under autonomy and the score under centralized control. 2 It is assumed that n1/n2 <1.
53 CHAPTER 2 ESTIMATING TEST SCORE IMPACTS 2.1 Introduction To examine the efficacy of the proposed theoretical mechanisms and the predictions of the model, I analyze the impact of increased budgetary autonomy on average test scores. The analysis is two fold. Since the theoretical results suggest that the outcomes, I use an instrumental variables approach to effectively control for the impact of this selection. I am, first, interested in the program impacts separate of any selection bias. For policies that would universally introduce autonomy, estimating the latter is relevant. y, I do not wish to overestimate increases in average test scores. I am, also, interested in how well the determinants of the theoretical model predict program selection. For instance, the model implies that a principal will be less likely to select the program if they are more risk averse. The first stage of an instrumental variables approach is useful for exploring this second question. The causal relationship of student test scores to increased autonomy is captured using equations like the following o ne for school level test scores, of a school i testing in year t: (2 1)
54 The variable is a dummy variable equal to 1 if a school participates in the Autonomy Zone/Empowerment Schools program, and 0 otherwise. The terms and are year of test and borough specific test effects. 1 is the average test score for i.e. how effectively the school ge nerates test scores due to inputs other than the observed, contemporaneous covariates. This may include how the level of expenditures in the previous years has created an environment with adequate technology or if the classrooms are well lit and pleasant. is a vector of characteristics describing the student body of a particular school. 2 and are vectors of the characteristics of the teachers and principal at a particular school. 3 is an error term t hat reflects random fluctuations in average test scores.If were randomly assigned an Ordinary Least Squares (OLS) empirical model would capture the average causal effect of increased autonomy on school level test scores. Because principals a re able to choose to participate in the Autonomy Zone/Empowerment schools network, OLS estimates would be biased by the correlation between the choice of autonomy and unobserved variables such as a ern for their career, or a preference for consuming generalized rents. The preceding theoretical investigation implies that characteristics of the school, both observable and unobservable, impact this 1 School specific fixed effects are not used because the proposed instrument is school specific and does not vary substantially over the 5 year period used in this study. 2 The specifics of these variables are discussed in Section 2.2. 3 does not include a principal's risk aversion since theory indicates it is irrelevant to the production of student test scores.
55 decision. In the following section, I analyze the cov ariate balance between autonomous and traditional public schools to provide further evidence that selection bias is likely. The preceding theoretical investigation also suggests an instrument for participation in the autonomy program. Risk aversion alte join the program, but does not impact test scores. As a proxy for risk aversion, I use a ection 2. 3 the validity of this excluded instrument is discussed If the empirical results show genuine increas es in average test scores, i.e. increases after having controlled for selection bias, this will suggest that on average the schools are either consuming zero or minimal generalized rents and resource allocation by the principal is effective If, instead, the results show test score decreases this will suggest that considerable rents are being consumed and the program does not effectively improve student achievement. Since the theoretical model suggests that test score impacts vary with the characteristics of a school and its principal, I include specifications with interaction terms for observable parameters. An investigation of the types of expenditure reallocations that occur given autonomy are pursued in the following chapter. Table 2 1 connects the t heoretical parameters to the parameters utilized in this empirical analysis. The first stage will provide further insight into the type of schools that select autonomy. Since the theoretical model provides indeterminate results on the relationship of se veral parameters to the propensity to select autonomy, these results are of particular interest. The general form of the selection equation is (2 2)
56 where aversion. I expect to negat ively as an excluded instr ument is provided in Section 2. 3 In Section 2. 2 I present the dataset used in this empirical investigation, as well as the balance of o bserved covariates across autonomous and non autonomous schools. Section 2. 3 explains the empirical specifications used to obtain the impact of autonomy on student test scores. Se ction 2. 4 reports results on characteristics that up of the autonomy program. Section 2. 5 reports the test score results of thes e analyses. Finally, section 2. 6 concludes.
57 2.2 Data The data used in this analysis comes from 3 main sources: the New York City Department of the Accountability and Overvie w Reports (AORs), a dataset on schools autonomy decisions 4 and other demographic information. The primary dataset is the AORs for the 2002 2003 to 2006 2007 5 school years. This dataset c ontains student and teacher demographic data, as well as average test scores on NYC English and mathematics exams for grade 4 and grade 8. I add to this the autonomy decisions and information on principal experience and demographics to form a rich, unbal anced panel 6 composed of 867 elementary and middle schools in New York City. Charter schools are excluded from the dataset. 7 4 This dataset was prepared by New York City Department of Education research analysts. 5 The fact that the panel ends in 2006 2 007 ensures that the effect of the autonomy program is not conflated with the effect of the Fair Student Funding program implemented at the beginning of the 2007 2008 school year. Information on this program can be accessed at http://schools.nyc.gov/Offic es/mediarelations/NewsandSpeeches/2006 2007/20070508_fsf.htm. 6 The panel is necessarily unbalanced because between the 2002 2003 and 2006 2007 school years schools between the 2002 2003 and 2008 2009 school year. By the 2005 2006 school year, there were 178 new schools. 7 Charter schools are given a level of autonomy that exceeds that given to public autonomous schools. This paper is designed to see how a moderate level of autonomy works.
58 The Accountability and Overview Report is part of the New York State School Report Card. These report cards are released to the public at the end of each school year in compliance with No Child Left Behind requirements. 8 The AORs provide information on accountability measures, including the average test score of students for yearly English and mathematics exams administered in the spring. These reports also provide a snapshot of the conditions in a particular school. The student and teacher demographic data 9 obtained from the AORs includes information such as the number of students enrolled in a school, class size, the racial mak eup of the school, the number of students receiving free or reduced school lunches, the number of uncertified teachers, and the average age of a teacher. These data also contain important indicators of behavioral problems, such as the number of suspension s and the annual attendance rate. 8 No Child Left Behind Act, Section 1116.a.1.c. accessed from U.S. Department of Education website, http://www2.ed.gov/policy/elsec/leg/esea02/pg2.html#sec1116. 9 The counts used to determine these statistics are obtained earl y in October of the school year.
59 The dataset containing scho ol s autonomy decisions records the year a school entered the program and if the school selected to exit the program. There are three types of schools that select autonomy: 1) existing tradit ional public schools, 2) new traditional public schools, and 3) charter schools. Traditional public schools selecting the program gain increased decision making and budgetary control. Charter schools by definition already have more expansive autonomy tha n this program provides. Charters autonomous schools. The program also offers these schools access to network with other schools about policies and programs that wor k. As noted, charter schools are excluded. While a few schools chose to exit the program, all of these decisions occurred outside of the 2002 2003 to 2006 2007 time span used in this study. Therefore, in this study, once a school selects autonomy they r emain in this organizational regime.
60 Of the 867 schools in the dataset, 642 schools are elementary schools. 367 of the schools are middle schools. 10 There are 228 (35.5%) and 143 (40%) schools participating in the autonomy program in elementary and midd le school s respectively. The data on principals contains demographic information, including ethnicity, gender, and birth date, and a complete accounting of their employment history within New York City schools (NYCS). This data also allows me to capture different schools, as well as across employment categories. For instance, I can track the number of years a current principal worked as teacher or assistant principal in NYCS. The data does not contain any information on a prin outside of NYCS. Therefore, if a principal enters the school system after working as an administrator or teacher in another d istrict, my calculation of the principal's experience will be too low. Table 2 2 provides summary s tatistics for these variables. I choose not to include the behavioral indicators, i.e. the attendance and suspensions rate, because they are highly correlated with lagged test scores. Further, the variables measuring the proportion of students of a parti cular race/ethnicity are too highly correlated with the classifications for free and reduced lunch and English Language Learners. 11 10 There are 142 schools that serve both elementary and middle school students. 11 Specifications including all variables do not produce statistically different results for the coefficient on the autonomy dummy. Since the first stage coefficients are of interest, however, the refined set of covariates is preferred.
61 Covariate balance. To examine whether selection is an empirically relevant issue, I compare group means for autonomous a nd centralized schools, both before and after th e treatment period. Table 2 4 reports descriptive statistics for autonomous and centralized schools in NYCS after treatment. Autonomous schools differ substantially from centralized schools. Autonomous sch ools are staffed with higher proportions of female teachers, younger teachers, and slightly more uncertified teachers. 84.2% of teachers in autonomous schools are female compared to 82.5% of teachers in non autonomous schools. 12 The average age of teacher s in these schools is 40.51 versus 41.26 in their centralized counterparts. The difference in the percentage of uncertified teachers is small. The mean percentage of uncertified teachers is 10.3% in autonomous schools, as opposed to 9.6% in centralized schools. 13 The average principal differs across these types of schools as well. Principals in autonomous schools have approximately a year less of administrative experience than those in centralized schools (9.44 versus 8.57). Principals, however, are on average the same age. 12 It is important to note that this dataset contains only elementary and middle schools. 13 Although this difference is statistically significant, it may be an ar tifact of the smaller size of autonomous schools.
62 The composition of the student body also differs to some extent across organizational regimes. The mean racial makeup of the two types of schools is not statistically different. The share of students qualifying for free and reduced lunch status, however, is lower in autonomous schools (0.677 compared to 0.704). Autonomous schools also tend to be smaller than their non autonomous counterparts. The average autonomous school is composed of 690.224 students versus 725.133 in the non au tonomous schools. Autonomous schools also have a higher percentage of suspensions per student (6% versus 3.7%). The average test scores of these schools diverge on English exams, but not on mathematics exams. The English scores are higher in autonomous schools with a z score of 0.154 compared to 0.07. 14 14 The z scores were calculated before excluding charter schools from the dataset.
63 Table 2 5 reports the differences in means during the pre treatment period. The pre treatment period is defined as the two years prior to the start of the autonomy program. Therefore, this is a compar ison of schools that will later cho o se increased autonomy to those that will later choose to remain centralized. The pre treatment test score means show a somewhat different picture than the post treatment mean difference discussed above. These group mea ns, however, also suggest that schools that choose autonomy are fundamentally different even before the program. Schools that become autonomous have statistically higher mathematics and English z scores. The English z scores are higher by 0.192 standard deviations. The mathematics z scores differ even more substantially. The mathematics z scores are 0.323 standard deviations higher for the autonomous schools. These lagged scores represent baseline ave a baseline advantage in both subjects tested. In addition to the higher baseline test scores, teachers were slightly older in schools that eventually became autonomous over this period (41.7 compared to 41.3). The percentage of students in a school cl assified as ethnically Asian or Pacific Islander is 1% higher for these schools. Similar to the group means in the treatment period, these schools have a lower percentage of students eligible for free and reduced lunch. Students in schools that later sel ect autonomy also have less absences than students in schools that will choose to remain centralized. In contrast to the treatment period, schools that eventually select increased autonomy are slightly larger in this period. The schools that will select autonomy have an average number of students that is approximately 38 students larger than those that will remain under district control.
64 From the preceding examination of differences in means, it is clear that autonomous schools differ markedly from cent ralized schools. These recorded differences, of course, miss any additional differences that are unobservable to the researcher. These differences, however, provide suggestive evidence of selection. The following identification strategies address the li kely selection issue. 2.3 Identification Strategy and Selection Equation To examine the impact of autonomy on the average test scores of schools, it is consideration is whether the selection occurs on observables. This requires that conditional on observed covariates, take average test score with and without autonomy. This is the ignorability of treatment assumption. 15 If ass ignment to autonomy is a deterministic function of the observed covariates, then autonomy will be independent of outcomes after partialling out the covariates. Selection on observables is unlikely to hold. The model developed in Chapter 1 demonstrates th e importance of unobservable taste parameters such as a principal's taste for quality, taste for rents, and career impact. 15 See Jeffrey M. Wooldridge, Econometric Analysis of Cross Section and Panel Data, (The MIT Press: Cambridge, 2001), p. 607 608 for a discussion of this assumption.
65 If, however, these unobservables are time invariant, then a fixed effects model can be used to meet the ignorability of treatment ass umption. Since this assumption is not likely to hold, a two stage least squares (2SLS) instrumental variables model is pursued. I use insights gained from the theoretical model to choose an excluded instrument. A standard 2SLS procedure is a strong test of the existence of test score impacts, which are prone to be small. In response to this concern, the empirical specification relies on a more efficient estimator. This specification is an instrumental variables procedure with a generated instrument. T he assumptions and limitations of each method are considered in the following paragraphs. The reduced form equation includes time dummies to account for district wide policy changes that affected all schools and borough dummies to capture differences that depend on unobserved, time invariant characteristics of a particular region. 16 (2 3) The variable of interest is a dichotomous variable, equal to 1 if the school is autonomous in a particular period and zer o otherwise. In the time period examined, all schools that enter the autonomy program remain autonomous in all subsequent periods. I include the lagged test score to control for school level inputs that may persist from the previous year. The vectors and represent student, teacher, and principal characteristics of a school in a particular time period. 16 This is t he same as Equation 1 It is repeated here for convenience.
66 contains variables related to the proportion of a student type, the school size, and other variab les related to the difficulty of educating a student population. To examine the impact of the proportion of type 2 students, i.e. low test effectiveness students, in a school, I use the proportion of students eligible for free and reduced lunch. 17 The pre diction is that schools with more type 2 students will tend to have lower test scores and be less likely to select autonomy. I also control for the proportion of English learners, the average class size of a school, and the total number of students enroll ed. School size and the proportion of students eligible for free and reduced lunch are suggested by the preceding theoretical analysis. contains controls for teacher quality. These are included because teacher quality is likely to ha ve an impact on average test scores. Teacher quality variables are not directly considered in the theoretical model. Better teachers should increase the achievement of students. 18 I include the average age of the teachers to proxy for teacher experience. I also include the fraction of uncertified teachers currently teaching in the school. These variables should have opposite impacts on average test scores. More experienced teachers should positively impact average test scores. 19 More uncertified teache 17 The fact that lower socioeconomic status students often perform less effectively on standardized tests is well documented. This lowe r test effectiveness may be the result of test bias, lower household educational inputs, or differences in test ability. For some limited evidence that attempts to sort out the impact of the allocation of resources versus type specific effects, see Brown (1991). Mizala et al. (2007) point out that ranking school quality by average test scores is highly correlated with ranking school quality by average socioeconomic status. 18 Measures of teacher quality, however, have generally provided mixed and insignifi cant impacts on test scores. 19 Many studies find no effect.
67 In the empiric al specification, I proxy for principal effectiveness using years of experience. I measure principal experience as the years the individual has been employed as a principal. This definition is preferred to one including classroom experience because skil ls required in a leadership position are different than skills required in the classroom. Recent research analyzing the impact of principal characteristics on student test scores supports the idea that time spent as a principal has a greater impact than time spent teaching (Clark, Martorell, and Rockoff 2009; Branch, Hanushek, and Rivkin 2008; have a shorter time horizon to draw in impacts of the autonomy program. Also, more senior principals tend to have higher salaries, so any monetary impacts are likely to be a smaller percentage of their incomes. 20 If conditional on the included covariate s the take up of autonomy was reduced form equation would provide causal estimates. In other words, if was a deterministic function of the included covariate s and fixed effects then this model would consistently estimate the expected effect of autonomy on a randomly drawn school from the population, i.e. the average treatment effect (ATE). The theoretical model, however, suggests that there are unobservable c for test scores or career impact, which will impact selection of autonomy. 20 coefficients on variables of interest are not statistically different.
68 Since the autonomy program instituted in New York City public schools creates only partial autonomy, i.e. it does not allow the pri ncipal complete control of the budget, the test score effects may be small. To improve my ability to find what may be relatively small effects, I use an instrumental variables specification designed to increase the efficiency of the estimates. 21 This pro cedure is a two step process. The first step is to generate predictions for the probability that a principal will select autonomy conditional on observed covariates and an excluded instrument Then, the second step is to use the fitted prob abilities from the previous step as excluded instruments in a standard 2SLS procedure. To produce efficient instruments for 2SLS, the instrument for is of the form (2 4) where is a known parametric form. 22 The nonlinea rity this generates improves the efficiency of estimation. I assume that the binary response model is a probit model. 23 21 See Jeffrey M. Wool dridge, Econometric Analysis of Cross Section and Panel Data, (The MIT Press: Cambridge, 2001), p. 623 624. 22 Id. at p. 439 442. 23 The parametric form does not have to be correctly specified because we are using the fitted probabilities obtained from this process as excluded instruments.
69 The role of the excluded instrument in the first step is to insure that the fitted probabilities are not so highly correla ted with the other covariates that multicollinearity prevents precise estimates. For the excluded instrument to be a good instrument, the instrument needs to be partially correlated with autonomy after the other exogenous variabl es have been netted out an d orthogonal to the error in the second stage equation of the 2SLS procedure. This amounts to principal female having an impact on test scores only through its impact on autonomy. The determinants of the theoretical model 24 suggest a potential area to s earch for an excluded instrument. Since the instrument must impact the likelihood that a school becomes autonomous, but not the average test score produced, the level of risk aversion is suggested. In the proposed theoretical model, the level of risk ave rsion only impacts how the principal deals with the uncertainty costs they face. More risk averse principals have less tolerance for uncertainty. Therefore, the level of risk aversion will help to determine whether a principal decides to become autonomou s or not. Higher values of risk aversion, modeled as constant absolute risk aversion (CARA), make a principal less likely to join the program. Since the principal cannot affect the variation in he realized average score. 24 See Chapter 1.
70 majority of experimental and field studies on risk behavior find that women are more risk averse than men. 25 26 Studies that frame gambles as invest ment or insurance decisions show less consistent results. Even in studies that do not find men more risk prone than women, gender differences often are recorded. For instance, Moore and Eckel (2003) find that with weak ambiguity in the payoff and/or leve l of risk men are significantly more risk averse than women for gambles involving losses Women are also less ambiguity averse than men for such gambles. Similarly Schubert et al. (2000) and Moore and Eckel (2003) find that women are more risk averse in investment decisions, but less risk averse in insurance contexts. There is consistent evidence that risk attitudes differ by gender. Further, risk attitudes have been shown to impact occupational and compensation decisions. Bonin et al. (2007) show tha t lower levels of risk aversion make an individual more likely to select an occupation with low earnings risk. 25 For a summary of empirical findings on gender and risk aversion, see Catherine Eckel and P. Experimental Econo mics Results, eds. Charles R. Plott and Vernon L. Smith(Amsterdam: Elsevier Science B.V., 2008), 1078 1086. Table 1 of this chapter summarizes the findings. 26 Whether risk attitudes are innate or the product of social learning is still an open question. Booth, social setting. Placing undergraduate Microeconomics students in all female classes significantly reduces their risk aversion. In the context of this paper, however, the distinction is not particularly important. As
71 have an independent effect on student test scores after con trolling for the selected covariates. This condition cannot be directly tested. Table 2 2 provides support for the exogeneity of this instrument The first two columns show that for the subsample of non autonomous schools, the gender of the principal do es not have a significant impact on student test scores after controlling for the other covariates. Similarly, the second two columns show that in the pre treatment period, i.e. before the autonomy program was an option, principal female does not have a s ignificant impact on student test scores. I generate the probability of participation from the following probit specification (2 5) In practice, the lack of var iation in autonomy decisions du r ing the pre treatment period causes the probit model to per fectly fit the data in this period 27 Therefore, the pre tre atment data are dropped from this specification and this model is estimated using data from the 2004 2005 to 2006 2007 school year. 27 This i s a result of the fact that the probit specification constrains the predicted values to be between 0 and 1.
72 The results of this fir s t step are reported in Table 2 6 Ther e are separate specifications of the test score results for reading and math scores. As a result, the lagged score value used in estimating Equation 2 5 contain either reading or math scores. 28 With regard to the relationship of a principal's gender and t he probability to select autonomy, the pooled and middle school only samples show that a principal being female is significantly and negatively related to participation in the autonomy program. 29 These impacts have the expected sign. If f emale principal s are more risk averse, then being a female principal should make a principal less apt to select autonomy. The remaining results of this selection model are discuss ed in the Section 2. 4 The second step of the generated IV procedure is to use the fitted pro babilities generated from Equation 2 5 as the excluded instrument in a standard 2SLS. The first stage equation therefore, is (2 6) To make sure these fitted are partially correlated with autonomy conditional on the other covariates, I check the coefficient on the excluded instrument in the first stage o f the 2SLS procedure. Table 2 7 shows that the coefficient on the excluded instrument is significant for both the reading a nd math specifications. Table 2 7 also reports that the est imate is significant for elementary and middle schools separately. In addition, the fitted probabilities must be exogenous in the test score model. 28 Both scores are not included because they are highly correlated. 29 The coefficients generated when using the elementary school only sample are not significant. This is likely the result of there not being adequate variation in principal gender.
73 The second stage is (2 7) where is the fitted value from Equation 2 6 The other c ovariates are the same as those described at the beginning of this section. is the program impact of autonomy. The empirical specification addresses the principal's self selection into the autonomy program and the efficiency necessary to ide ntify small impacts. 30 2.4 Selection Equation Empirical Results The selection equation, Equation 2 5, discussed in the preceding section is of interest beyond its role in the empirical strategy. The estimates it produces demonstrate how characteristics o f teachers, students, and principals alter participation in autonomy. The r e sults are presented in Table 2 6 The preceding discussion noted that all of the specifications except for those that rely on the elementary only sample show that a principal bei ng female is significantly and negatively related to participation in the autonomy program. Th is result is, again, expected if female principals are on average more risk averse than male principals. 30 The results of other specifications are provided in the Appendix for comparison.
74 The theoretical model also suggests a few other importan t characteristics for selection: principal experience, school size, and the composition of the student body. All specifications confirm that principal experience is positively related to the decision t o select autonomy. In Table 2 6 the coefficient on years of administrative experience is positive and significant for all six specifications. The probit model implies that an additional year of experienc e for the average school is associated with a 0.1% increase in participation. The coefficients, furthe r, show that principal experience has the same magnitude of effect on selection in elementary and middle schools. complementary to inputs and student ability. Therefore, principal exper ience increases average test score. The positive coefficients found in all specifications are expected.
75 The proportion of free and reduced lunch students, however, does not have statistically significant effects on participation. The theoretical model predicts that t of generalized rents consumed is sufficiently small. One way to categorize students that proxies for free and reduced lunch students stands in for the proportion students 31 in a school. This measure does not have a statistically significant impact on socioeconomic status being too crude a categorization of baseline ability or it may reflect that some schools selecting autonomy are consuming sufficient generalized rents. When schools are using the program to consume generalized rents, they are forced to tradeoff increases in test scores for these rents. Th is mitigates the impact of the student body on the selection decision. 31 In the model, these students are referred to as type 2 students.
76 The size of a school does not have a statistically significant impact on selection in specifications that use the pooled and middle school sample. In the elementary specifications, however, school size has a positive a nd significant effect Table 2 6 columns 3 and 4 show that an additional student above the average increases the probability that a school will select autonomy by 0.002 percent. The coefficient is not statistically di fferent from the coefficient in middle schools or the pooled sample. The results provide some evidence that larger schools are more likely to select. The coefficient on school size is positive in all specification, although the effect is statistically si gnificant only in the elementary school subsample. The theoretical model predicts that school size will have a positive impact on selection where no rents are consumed and that its effect will be indeterminate when rents are consumed. Beyond the paramete rs directly suggested by the theoretical model, a few other instruments have consistent impacts. Mean age of teachers proxies for the experience program. 32 The coeffici ent on mean age of teachers is negative in all specifications. In the pooled sample, a one year increase in teacher age causes a 0.1% or 0.2% decrease in the probability a school will become autonomous in the reading and math specifications, respectively. In the elementary subsample, this coefficient is marginally significant in the math specification with a coefficient of 0.002. Overall, these results suggest that younger groups of teachers will make a principal more likely to select autonomy. 32 S tudies of charter schools find that teachers in charter schools tend to be younger and less and teacher labor markets: Evidence from charter school entr
77 Anoth er teacher characteristic that effects selection is the proportion of uncertified teachers. The results of the pooled model imply that the proportion of uncertified teachers decreases the likelihood of selecting autonomy. This characteristic has one of t he largest impacts on selection. The pooled sample estimates show that a small increase in the proportion from the average of 10% will cause approximately an 10% decrease in the probability that a school becomes autonomy. The impact is not significant in either elementary school specification. The impact is stronger in middle schools. In the middle school math specification, a small increase in uncertified teachers will decrease the probability of selecting autonomy by 16%. This difference may be relat ed to the fact the teaching is more subject specific in middle schools than elementary schools. Overall, a younger, more certified group of teachers causes principals to be more likely to select autonomy. This may occur because younger teachers are less resistant to change. The proportion of English Language Learners (ELLs) also negatively impacts the decision to select autonomy. In the pooled, elementary, and middle school reading specifications, the impact is negative and significant, ranging fro m 0.046 to 0.058. In the middle school math specification, the coefficient is negative, but not significant. Budgetary autonomy would be limited to some extent by these students because their funding is earmarked, Title I funds. The district can not provide autonomy over federally mandated funds. Economics, Vol. 96 (June 2012), 431 448. Jackson finds that teachers moving to charter schools have 4.76 years less experience than teachers in traditional public schools.
78 Taken as a whole, the selection results are largely consistent across specifications. A principal being female, an increase in the proportion of uncertified teachers, an increase in mean teacher age, o r an increase in the proportion of ELL students causes the principal to be less likely to select autonomy. In turn, principal experience and school size have a positive impact on program participation. 2.5 Test Score Results The estimates of the impact o f autonomy on average test s cores are summarized in Table 2 8 I report results for all schools, as well as for elementary and middle schools separately. 33 The pooled sample shows that there is large, significant and positive effect of joining autonomy o n reading scores. The coefficient shows that the average reading score of a school is improved by 0.121 standard deviations as a result of becoming autonomous. In elementary schools, increased autonomy improves both math and reading scores. The average math score of a school is increased by approximately 0.167 of a standard deviation. This result, however, is only marginally significant. The average reading score is increased by 0.174 of a standard deviation if an elementary school becomes autonomous. The magnitude of these impacts is large. Class size interventions have been found to improve test scores by approximately 0.2 of a standard deviation. 34 33 Since elementary and middle schools may provide distinctly different challenges for budgetary management and decision making, this distinction will allow me to better capture information about when autonomy is most effective. 34 Further, policies that aim to decrease class size are expensive to i mplement.
79 The evidence presented shows that the increase in budgetary autonomy does not have a statistically sig schools. There is, however, a significant positive impact in elementary schools. The results suggest that both math and reading scores may increase substantially as a result of greater budgetary and decision making autonomy. In addition to mean impacts, the theoretical analysis points out that test score populations 35 are apt to realize larger gains with autonomy. Table 2 9 shows how the test score impact varies with the size of the school, the experience of the principal, and the proportion of free and reduced lunch students. All the other controls discussed in section 2.3 are included. These results shows that the impact of autonomy is positive and significant for reading scores. 35 In the theoretical model, this refers to schools with more type 1 students, i.e. more of the type with higher baseline ability. In the context of the empirical specifications, this can be thought of as schools with more high SES students.
80 For each specification, the results imply that the proportion of low socioeconomic students significantly impacts the outcomes of increased autonomy. On both reading and math exams, the proportion of free lunch student in an autonomous school has a The estimates reported in column II show that if more than 68% of students in the schools qualify for free and reduced lunch the math score gains of autonomy are washed out. 36 The average school in the dataset has approximately 70% free and reduced lunch students (0.696). For autonomous schools, the average is 67.7% and the maximum is 100%. There ar e therefore schools that select autonomy, but have student body compositions which will offset the math score gains. The joint specification, reported in column IV, estimates that schools with an even smaller majority of free and reduced lunch eligible st udents will not realize the test score gains associated with autonomy. A school with 56% of free and reduced lunch students will cause there to be no impact of autonomy on math scores. The results of this joint specification are further explained in Figu re 2 1a. This figure illustrates the impact of autonomy on math scores for schools with different levels of free and reduced lunch students. 37 36 This result is coming from 1.217 1.798*proportion frl=0 (i.e. proportion frl=0.6768). 37 This presentation of the interaction term results is modeled after the discussion of interaction in terms in Organization, Vol. 58 (2004), 807 820.
81 The impacts on reading scores present a stronger case for the benefits of autonomy when dealing with high lev els of low SES students. In columns VI and VIII, the estimates show that even schools with 100% free and reduced lunch students can expect increases in reading scores. 38 Figure 2 1b demonstrates the interaction of autonomy and the proportion of FRL studen ts for schools with different proportions of these students. The flexibility afforded by autonomy allows the principal to better respond to the needs of low SES students in the production of reading scores. This is in contrast to the math score results w hich suggest that schools with high proportions of low SES students will perform worse than centralized schools. 38 In column VI, the proportion of frl students would have to exceed 1 for the gains to be reversed (2.660 2.51).
82 The interaction of increased autonomy with principal experience shows that autonomy significantly increases test scores for most levels of expe rience. Principal experience appears to have a small, negative impact when paired with autonomy. The effect is only statistically significant in the reading scores specification reported in column V. In column I, the estimate implies that a principal wi th 11.89 years of experience would counteract the positive impacts of autonomy on math scores. The coefficient, however, is not statistically different from zero. In all other specifications containing the interaction of principal experience and autonom y, principals would need to have accrued between 52.5 and 235 years of experience 39 to neutralize the benefits of autonomy. The overall impact of autonomy for a principal with a particular leve l of experience is reported in F igure 2 2a and 2 2b for math an d reading scores, respectively. The impact of autonomy on reading scores is positive and statistically significant. The average principal in the autonomy zone only has 9.44 years of experience. The impact of autonomy on math scores is positive and margi nally significant for up to 20 years of experience. 40 39 Based on the column V estimate, it would require 52.5 years of experience to offset the positive effect of autonomy (1.680 autonomy would never be offset by experience. In column IV, it would require 232.5 years of experience to negate the positive impact of autonomy (0.93 142 years of experience to reverse the gains (2.84 40 The estimated test s core impact for a principal with 20 years of experience is 0.861 with a standard error of 0.521.
83 The estimates of the interaction between school size and autonomy from the generated instrument specification all support that there are greater gains to autonomy in larger schools. In column I II, the estimate suggests that in an autonomous school each additional student increases the average test score by 0.0006 of a standard deviation. The interaction of autonomy and school size does not have a statistically significant impact in any other sp ecification. The other specifications, i.e. column IV, VII, and VIII, all have positive signs. 41 Figures 2 3a and 2 3b present the overall result of autonomy for schools of different sizes. Figure 2 3a shows that for schools with an enrollment in excess of 300 students the impact of autonomy is positive and significant. The impact of autonomy on reading scores is even stronger. Figure 2 3b shows that the impact is positive and significant for a school between 100 and 1600 students. 42 2.6 Conclusion Thi negatively impacted by the principal being female, the mean age of the faculty, the proportion of uncertified teachers, and the proportion of English Language Learners enrolled in a sc hool. In addition, more years of administrative experience and larger These results on selection provide support for risk aversion being negatively related to a princip impact this decision. 41 The largest autonomous school in the dataset has 1675 students. The largest non autonomous school has 2707 students. Therefore, the positive joint impact of school size and autonomy should be considered within these parameters. 42 The minimum schools size of an autonomous school is 76. The maximum school size is 1675.
84 The test score impacts imply that increased budgetary and decision making autonomy generally has positive or insignificant effects. Further, the i mplications of autonomy differ across elementary and middle schools. Elementary schools have larger, positive impacts of this program. Finally, there is substantial heterogeneity in the impact of autonomy given observable characteristics of the student b ody. A higher proportion of free and reduced lunch students coupled with autonomy negatively affects the ability of these schools to improve student test scores. The results, however, provide estimates for which the overall impact is positive for a broad range of school sizes and principal experience. These effects are stronger on reading scores than math scores. Although the proportion of free and reduced lunch students decreases the positive effects of autonomy, for reading scores the positive impa ct is never overwhelmed.
85 Table 2 1. Connecting the theoretical model to empirical analysis Observed by r esearcher Empirical s pecification Risk a Yes Gender Principal e ffectiveness, k Yes Years of administrative e xperience Taste for r No Error term Taste for q No Error term Career i No Error term School s ize, N Yes School Size Type r atio, n1/n2 Yes Share of s tudents on Free and Reduced Lunch
86 Table 2 2 Tests of exogeneity of princ ipal female Non autonomous schools Pre treatment Reading Math Reading Math Principal female 0.004 0.002 0.00007 0.0008 (0.005) (0.005) (0.009) (0.011) Years of administrative experience 0.0006* 0.0004 0. 001 0.0002 (0.0004) (0.0004) (0.0008) (0.0009) Principal age 0.00002 0.0006* 0.00015 0.0005 (0.0003) (0.0004) (0.0006) (0.0007) School size 0.000003 0.000006 0.000007 0.000003 (0.000007) (0.00001) (0.00001) (0.00001) Lagged test score 0.923*** 0.949*** 0.967*** 0.937*** (0.007) (0.007) (0.013) (0.014) Mean teacher's age 0.0001 0.003*** 0.0006 0.002 (0.007) (0.0008) (0.002) (0.002) Proportion of uncertified teachers 0.059 0.057 0.108* 0.045 (0.035) (0.038) (0.063) (0.068) Class size 0.0002** 0.0002* 0.0006*** 0 .0008** (0.00008) (0.0001) (0.00009) (0.0004) Proportion of FRL students 0.024*** 0.026*** 0.02 0.041** (0.008) (0.009) (0.017) (0.019) Proportion of ELL students 0.017 0.07** 0.044 0.092 ( 0.025) (0.029) (0.055) (0.064) Number of schools 628 628 656 656 R squared 0.922 0.911 0.941 0.919 Year dummies Yes Yes Yes Yes Borough dummies Yes Yes Yes Yes Note: The standard errors reported are Huber White robust. This specification includes both elementary and middle schools. significant at 1%; ** significant at 5%; *** significant at 10%
87 Table 2 3. Descriptive statistics Teacher v ariables Mean Standard d eviation Minimum Maximum Share of female t eachers 0. 832 0.118 0.25 1 Mean age of t eachers 41.082 3.375 26.313 52.481 Share of uncertified t eachers 0.101 0.072 0 0.615 Principal v ariables Age of p rincipal 50.179 7.576 28 81 Female p rincipals 0. 708 0.449 0 1 Years of administrative e xperience 8.831 6.217 0 53.038 Student v ariables Class s ize 24.773 4.113 10.507 49.859 Share of s tudents on Free and Reduced Lunch 0.696 0.279 0.029 1 Share of students c lassified as English Language Learners 0.109 0.094 0 0.549 School s ize 714.13 340.742 63 2707 Test s cores (z scores) English 0.072 0.427 1.373 1.685 Mathematics 0.094 0.433 1.287 1.647
88 Table 2 4. Balance of observed variables after treatment Variables Autonomous Centralized Statistically different Share of female teachers 0.842 0.825 *** Mean age of teachers 40.561 41.255 *** Average number of absences 0 .103 0.096 *** Age of principal 49.842 50.24 Years of administrative experience 9.444 8.567 *** Principal's gender 0.667 0.71 Class size 27.684 25.958 ** Share of black students 0.413 0.386 Share of Hispanic stu dents 0.37 0.344 Share of white students 0.116 0.145 Share of Asian/Pac. Islander students 0.007 0.006 Share of students on Free and Reduced Lunch 0.677 0.704 *** Share of students classified as English Language Learn ers 0.107 0.11 School size 690.224 725.133 *** Average number of absences 12.038 12.311 Suspensions per student 0.06 0.037 *** Mathematics z score 0.063 0.094 English z score 0.154 0.07 ** significant at 1%; significant at 5%; *** significant at 10%
89 Table 2 5. Balance of observed variables in the pre treatment period Variables Autonomous Centralized Statistically different Shar e of female t eachers 0.845 0.826 *** Mean age of t eachers 41. 719 41.3 ** Share of uncertified t eachers 0.111 0.105 Age of p rincipal 51.641 51.291 Years of administrative e xperience 7.554 7.815 Principal's g ender 0.691 0.666 Class s ize 27.303 27.368 Share of black s tudents 0.386 0.377 Share of Hispanic s tudents 0.337 0.349 Share of white s tudents 0.159 0.149 Share of Asian/Pac. Islander s tudents 0.007 0.006 ** Share of s tudents on Free and Reduced Lunch 0.696 0.724 Share of students c lassified as English Language Learners 0.097 0.097 School s ize 798.581 760.773 Average number of a bsences 11.746 12.298 *** Suspensions per s tudent 0.02 0.216 Mathematics z score 0.149 0.09 ** English z score 0.134 0.052 ** significant at 1%; ** significant at 5%; *** significant at 10%
90 Table 2 6. Selection of autonomy Pooled Elementary Middle Instruments Reading Math Reading Math Reading Math Principal female 0.011** 0.010* 0.00 7 0.006 0.011** 0.016* (0.006) (0.006) (0.007) (0.006) (0.006) (0.011) Years of administrative 0.001*** 0.001*** 0.001*** 0.001*** 0.001*** 0.001** Experience (0.0004) (0.0004) (0.0005) (0.0005) (0.0004) (0.0008) Principal age 0.0005 0.0005 0.0001 0.00004 0.0005 0.001* (0.0004) (0.0004) (0.0004) (0.0004) (0.0004) (0.0007) School size 0.00001 0.00001 0.00002* 0.00002* 0.00001 0.00002 (0.00001) (0.00001) (0.00001) (0.00001) (0.00001) (0.00002) Lagged test score 0.0 1 0.0004 0.015* 0.002 0.01 0.010 (0.008) (0.007) (0.009) (0.008) (0.008) (0.013) Mean teacher's age 0.001* 0.002** 0.001 0.001 0.001* 0.0009 (0.0008) (0.0008) (0.001) (0.001) (0.0008) (0.001) Proportion of uncertified 0.105** 0.10 9** 0.079 0.083 0.105** 0.176** Teachers (0.048) (0.049) (0.061) (0.062) (0.048) (0.089) Class size 0.0001 0.0001 0.002* 0.001 0.0001 0.0001 (0.00008) (0.0001) (0.001) (0.001) (0.00008) (0.0001) Proportion of FRL 1 0.011 0.004 0.00 4 0.006 0.011 0.008 Students (0.011) (0.011) (0.011) (0.011) (0.011) (0.023) Proportion of ELL 2 0.050* 0.058** 0.046* 0.054** 0.050* 0.088 Students (0.028) (0.028) (0.029) (0.030) (0.028) (0.065) Number of schools 832 832 642 642 367 367 R squared 0.242 0.24 0.249 0.244 0.242 0.267
91 Table 2 6. Continued Pooled Elementary Middle Reading Math Reading Math Reading Math Controls Yes Yes Yes Yes Yes Yes Year dummies Yes Yes Yes Yes Yes Yes Borough dummies Yes Yes Yes Yes Yes Yes 1. FRL stands for free and reduced lunch. 2. ELL stands for English Language Learners. This is the proportion of students who are not native speakers and enrolled in classes to further their comprehension of English. significant at 1 %; ** significant at 5%; *** significant at 10%
92 Table 2 7. First stage: Generated instrument Pooled Elementary Middle Reading Math Reading Math Reading Math Generated instrument 1.061*** 1.018*** 1.115*** 1.003*** 1.02*** 1.008*** (0.117) (0.121) (0.134) (0.14) (0.168) (0.17) Partial F statistic 19.03 18.21 13.72 12.15 8.67 8.9 Controls Yes Yes Yes Yes Yes Yes Year dummies Yes Yes Yes Yes Yes Yes Borough dummies Yes Yes Yes Yes Yes Yes Notes: The standard errors reported are Huber White robust. The coefficients are used only to show that the generated instrument meets the condition that it is partially correlated with autonomy after all the other variables have been controlled for. significa nt at 1%; ** significant at 5%; *** significant at 10%
93 Table 2 8. The impact of autonomy on school level test scores Elementary Middle Variables Reading Math Reading Math Reading Math Autonomy 0.121** 0.038 0.174** 0.167* 0.079 0.0 91 (0.059) (0.067) (0.07) (0.089) (0.061) (0.078) Years of administrative 0.0004 0.0007 0.00006 0.0003 0.0004 0.0007 E xperience (0.0005) (0.0006) (0.0007) (0.0007) (0.0007) (0.0008) Principal age 0.0003 0.0006 0.0007 0.001* 0.0005 0.0002 (0.0004) (0.0005) (0.0006) (0.0006) (0.0007) (0.0007) School size 0.00001 0.0002* 0.00001 0.00001 0.00002 0.00002 (0.0001) (0.0001) (0.00002) (0.00002) (0.00001) (0.00001) Lagged test score 0.891*** 0.955*** 0.867*** 0.929*** 0.930*** 0.99 9*** (0.011) (0.011) (0.016) (0.014) (0.015) (0.016) Mean teacher's age 0.0008 0.003*** 0.0005 0.004*** 0.0008 0.001 (0.001) (0.001) (0.001) (0.001) (0.001) (0.002) Proportion of uncertified 0.101* 0.099 0.005 0.09 0.086 0.156* T eacher s (0.058) (0.064) (0.094) (0.102) (0.082) (0.096) Class size 0.0001 0.0001 0.001 0.001 0.0001 0.0002 (0.0001) (0.0002) (0.001) (0.001) (0.0001) (0.0002) Proportion of FRL 0.027** 0.029** 0.022 0.037** 0.036* 0.018 S tudents (0.013) (0.014) ( 0.015) (0.017) (0.02) (0.021) Proportion of ELL 0.054 0.135*** 0.05 0.185*** 0.062 0.018 S tudents (0.034) (0.039) (0.4) (0.048) (0.052) (0.064)
94 Table 2 8. Continued Pooled Elementary Middle Reading Math Reading Math Re ading Math Number of schools 867 867 642 642 367 367 R squared 0.908 0.905 0.89 0.883 0.937 0.927 Year dummies Yes Yes Yes Yes Yes Yes Borough dummies Yes Yes Yes Yes Yes Yes Notes: The standard errors reported are Huber White ro bust. significant at 1%; ** significant at 5%; *** significant at 10%
95 Table 2 9. Heterogeneous test score impacts 1 Math Reading (I) (II) (III) (IV) (V) (VI) (VII) (VIII) Au tonomy 0.107 1.217*** 0.435* 0.93* 1.680*** 2.660*** 0.858** 2.84*** (0.313) (0.449) (0.264) (0.546) (0.411) (0.806) (0.334) (1.00) Autonomy*experience 0.009 0.004 0.032** 0.02 (0.012) (0.012) (0.012) (0.015) Autonomy*FR L 2 1.798*** 1.66*** 2.297** 2.13** (0.516) (0.539) (0.885) (0.958) Autonomy*school size 0.0006** 0.0004 0.0003 0.0002 (0.0002) (0.0003) (0.0004) (0.0004) Notes: The reported standard errors are Huber White robust. 1. These results are for the pooled sample containing both elementary and middle schools (N=832). 2. FRL stands for free and reduced lunch. significant at 1%; ** significant at 5%; *** significant at 10%
96 Figure 2 1a. The interaction of aut onomy and free and reduced lunch status on math scores
97 Figure 2 1b. The interaction of autonomy and free and reduced lunch status on reading scores
98 Figure 2 2a. The interaction of autonomy an d principal experience on math scores
99 Figure 2 2b. The interaction of autonomy and principal experience on reading scores
100 Figure 2 3a. The interaction of autonomy and school size on math sco res
101 Figure 2 3b. The interaction of autonomy and school size on reading scores
102 CHAPTER 3 ESTIMATING EXPENDITURE IMPACTS 3.1 Introduction This chapter explores the link between increased budgetary autonomy and the assignme nt of funds to particular uses. The empirical results of the previous chapter establish that the autonomy program in New York City Public Schools increased student achievement for elementary schools. To gain an understand ing of how budgetary autonomy imp roves the production of student achievement, I examine directly the reassignment of funds across categories. The estimates show that schools in the program increase expenditures on teachers. Autonomous schools also decrease the percentage of expenditure s on leadership services, such as assistant principal and office staff salaries, other direct services, such as transportation and maintenance, and centralized services. 3.2 Data The data used in this analysis comes from 3 main sources: New York City D Accountability and Overview Reports, and a dataset containing principal employment history. The primary dataset is the SBERs for the 2002 2003 to 2006 2007 1 school years. These report s contain detailed expenditures made by a school, including both dataset is combined with student and teacher demographic data, as well as information 1 The fact that the panel ends in 2006 2007 ensures that the effect of the autonomy progra m is not conflated with the effect of the Fair Student Funding program implemented at the beginning of the 2007 autonomy. Information on this program can be a ccessed at http://schools.nyc.gov/Offices/mediarelations/NewsandSpeeches/2006 2007/20070508_fsf.htm.
103 on principal experi ence, to form a rich, unbalanced panel 2 composed of 901 schools in New York City. Charter schools are excluded from the dataset. In the SBERs, the expenditures are categorized by the student type served, the function, and the funding sourc e. Within a scho ol, there are two student types: 1) general education students and 2) full time special education students. I confine the analysis to the funds spent on general education students. Since this analysis is linked with understanding how autonomous schools altered their behavior to create gains in test scores, focusing on changes in allocations directed at the students who make up the largest portion of the Progress Report Card index should capture any significant change in resource allocations. 3 The repor ts catalogue expenditures within 5 major functional categories and 47 subcategories. The largest of the 5 categories is Direct Services to Schools. These expenditures account for all spending that takes place at the school site and directly contributes t o student education. Included in this category are expenditures on text books, teachers, administrative staff, school support staff, and instructional materials. On average, Direct Services spending is $11600 4 per student, which accounts for approximatel y 89% of total expenditures within a school. In schools that join the Autonomy Zone or select the ESO network, average per student expenditures in this category are $13093.04 and accounts for around 94% of total expenditures, compared 2 The panel is necessarily unbalanced because between the 2002 2003 and 2006 2007 school years school system opened 291 schools between the 2002 2003 and 2008 2009 school year. By the 2005 2006 school year, there were 178 new schools. 3 The reports also contain enrollment by student type. This allows me to construct per student expenditures within each functional category. 4 All expenditures have been converted into 2007 dollars using the CPI U tables.
104 to non autonomous sc hools where the values are $11470.21 and 88%, respectively. The difference in the amount of per student expenditures and percentages between autonomous and non autonomous schools provides evidence that autonomous schools not only were able to spend more t otal per student, but were also able to spend a greater portion within the school itself. To simplify the analysis of expenditure changes, I aggregate the 47 subcategories into seven mutually exclusive categories. Five of these categories fall under Direc t Services: teachers, other classroom expenditures, leadership (net of are expenditures to a central office or service and expenditures on capital services. These seven categories on average account for approximately 96 percent of total expenditures in a school. This strategy accounts for about 95% and 97% of total expenditures for autonomous and non autonomous schools, respectively. Table 3 1 contains an explana tion of the types of expenditures that fall under a particular expenditure categories. Of the seven categories, the largest is expenditures on teachers. This accounts for approximately 43% and 39% of total expenditures in non autonomous and autonomous sch ools, respectively 5 Table 3 3 compares the percentage of expenditures allocated to a particular function across autonomous and non autonomous schools. This provides a first look at some of the differences in resource allocations. 6 5 The next largest subcategory is debt services which accounts for 7.17% and 5.53% of autonomous and non autonomous school spending, respectively. 6 When comparing these means, however, it is important to note that characteristics of autonomous and non autonomous schools differ in several important respects. The most significant difference is that autonomous schools are nearly 200 students smaller t han non autonomous schools on average.
105 Table 3 3 shows that the percentage of funding is statistically different across autonomous and non autonomous schools for 6 of the 7 expenditure categories greater budgetary autonomy, the le adership category defined above does not include funds accounted for with the 7 categories is the result of the difference in the percentage of expenditures allocated to versus non autonomous schools. 7 In non autonomous schools, the average per student expenditure on the principal is $294.32. In autonomous schools, the mean per student expenditure on principal services is nearl y double this amount, $557.39. The difference in per student expenditures in part reflects the fact that autonomous schools tend to be smaller than non autonomous schools. Table 3 3 illustrates that autonomous schools assignment of funds differs from t hat of non autonomous schools. This preliminary comparison of the types of schools does not explain the mechanism that creates the difference in expenditure patterns. To examine whether these expenditure differences are the result of underlying diffe renc es in t he schools, Table 3 4 provides the means for non autonomous and autonomous schools in the pre treatment period. The pre treatment period is defined as the period in the dataset before schools had the option of becoming autonomous, i.e. the 2002 200 3 to 2003 2004 school year. In the pre treatment period, the percentage of funds allocated differs for four categories of expenditures: teachers, leadership, support services, and other direct 7 budget for both types of schools.
106 services. Schools that will later choose to become autonomous spend smaller percentages of their funds on teachers, leadership, and other direct services than schools that will remain centralized. The schools that later become autonomous spend more on support services in this period. These differences in expenditu re allocations, as well as the results of theoretical and other empirical work 8 suggest that selection into autonomy is an important consideration. The student and teacher demographic data 9 obtained from the AORs includes information such as the numbe r of students enrolled in a school, class size, the racial makeup of the school, the number of students receiving free or reduced school lunches, the number of uncertified teachers, and the average age of a teacher. These data also contain important indic ators of behavioral problems, such as the number of suspensions and the annual attendance rate. Finally, this data is supplemented with information and the experience o f the principal in both administration and teaching. The student and teacher covariates included are the same as those utilized to estimat e test score impacts. Table 3 2 reports the summary statistics for these variables. 3.3 Empirical Methodology To exa mine whether resource allocations are systematically altered under the Autonomy Zone/Empowerment Schools Organization, I model school level resource expenditures as functions of school and student characteristics. The dependent variable is specified as the percentage of per student expenditures allocated to a particular 8 See chapter 1 for a theoretical discussion of a principal's selection decisi on. For empirical work that shows schools that select autonomy differ from nonautonomous schools, at least on observable variables, see chapter 2 section 2.3. 9 The counts used to determine these statistics are obtained early in October of the school year
107 resource category. 10 These expenditures include funds from state, federal and local sources, as well as funds that are earmarked for particular functions. Further, schools with different ch aracteristics with have different costs of attaining some level of education outcomes. As a result, school and student characteristics are used to control for potential differences in cost and funding allocations between schools. For instance, schools wi th larger populations of special needs students, e.g. special education, limited English proficient (LEP), or other at risk students, are likely to have higher per student funding to address the higher costs of attaining some level of educational outcome. An example of this is Title I funding for schools with a higher proportion of lower socioeconomic status students. 11 Although the selected controls will help untangle changes stemming from budgetary autonomy from underlying funding changes, the possibi lity remai ns that the decision to become autonomous is endogenously determined. To provide estimates robust to this possibility, I utilize an instrumental variables procedure The methodology is the same as the empirical method used to estimate test scor e impacts in the previous chapter. The causal relationship between autonomy and the percentage of funds allocated to a particular category is captured using equations like the following one for school level expenditure percentages of school i in year t: (3 1) 10 I have also examined changes in the level of expenditures. The percentage change specification is preferable since autonomous schools are granted larger budgets. 11 In the case of NYC schools, there is also a significant amount of nonprofit funding for which these schools would be eligible, such as funds from the Bill and Melinda Gates Foundation or The Fund for Public Schools.
108 is defined as the percentage of total expenditures within a school that are dedicated to a pa rticular category I include time dummies to account for district wide policy changes that affected all schools and school dummies to capture differences in resource allocations that depend on unobserved, time invariant characteristics of a particular school. From the preliminary data analysis, we know that schools participating in the autonomy program have larg er budgets, i.e. higher levels of per student expenditures. Therefore, it is possible that autonomous schools may choose to allocate increased funds in the same or ne arly the same proportions. The dependent variable will provide evidence of systematic changes in expenditure patterns. If autonomous schools seem to be shifting resources in similar patterns, this may point to underlying inefficiencies in centralized allocations. The variable of interest is a dichotomous variabl e, equal to 1 if the school is autonomous in a particular period and zero otherwise. The coefficient on this variable will provide evidence of changes in resource allocations that result from increased budg etary autonomy. All schools entering the program remain in the program through the end of the dataset. As previously mentioned, I control for characteristics of students, teachers, and principals that may alter the level and/or most efficient use of funds. There are several channels through which the se characteristics may alter expenditures. First, budgets in New York City schools, as in many other districts, are determined by complex funding formulas coming from the national, district, and state level. These formulas are for the most part driven by the composition of students within a school and prior performance
109 of these students. The second channel through which these characteristics influence expenditures allocation results from the specific needs of a school and is contingent on the level of au tonomy in resource allocation. Particular student and teacher populations, as well as attributes of the principal, will cause the most efficient use of funds to differ. is a vector of contemporaneous student characteristics including the p roportion of students of a particular race category, the proportion eligible for free or reduced lunch, the proportion classified as English learners, and the proportion classified as special education. is a vector of one year lagged student characteristics. This vector includes the number of students suspended, the average number of absences for student in school i, the proportion of students repeating a grade, and class size 12 These variables are lagged one year because expenditures in th e current year are likely to influence these outcomes. Also, included is the percentage of resources spent in the previous year This value added approach should capture the fact that prior allocations may alter the need for expenditures i n the current year. is a vector of principal characteristics including the number of years of experience as a teacher and administrator. These measures are all contemporaneous and designed to assess the ability of the principal to eff ectively allocate funds. The expenditures in the leadership category are net of principal salary. Therefore, changes in this category will not be the result of a new principal with a higher level of experience or an existing principal moving up a step on the salary schedule. Changes in leadership allocations could represent the principal hiring more experienced assistant principals, 12 Class size is simply determined by general education enrollment divided by the number of teachers.
110 hiring better/more administrative support staff, or even improvements to administrative offices. is a vecto r of teacher characteristics in the previous year. This includes the mean ages of teachers within a school and the proportion of teachers who are unlicensed. The mean age of teachers is a proxy for teacher experience. The proportion of unlicensed teache rs controls for the impact of teachers with very little training. Teachers with little training may require the school to spend more funds on teacher training. Teacher characteristics in the previous year could act as a constraint on the type of resource allocations that will be effective. I do not control for characteristics in the current year because I am interested in whether autonomous principals decide to spend more or less on teachers. This specification will not identify whether this change is t he result of the number or experience level of teachers. Given this specification, will be the causal effect of autonomy if the remaining unobserved effect is mean independent of treatment. Since principals were able to choose to participa te in the autonomy program, this seems unlikely to hold. Instrumental variables. Since principals directly made the decision to join the autonomy program, dealing with this selection is essential to obtaining credible causal estimates of changes in reso urce expenditures. As with the estimation of test score impacts, I rely on the principal s level of risk aversion to suggest a good instrument. Principals in the district are likely to have varying levels of risk aversion. More risk averse principals s hould be less apt to join a program with increased personal responsibility for student outcomes. Several empirical
111 studies have shown that on average women are more risk averse than men. 13 Therefore, female principals should be less likely to join the pro gram than male principals. This relationship is confirmed by the first stage regressions presented in Table 3 5 The excluded variable in t he first stage equation below is equal to 1 if the principal in a school in a particular year is female and zero i f the principal is a male. 14 To improve the efficiency of th e estimates, I estimate a model which utilizes the non linearity of fitted probabilities from a first stage probit model to instrument for program participation. First, I estimate the following b inary response model, (3 2) for a subsample of the dataset from 2005 to 2007, i.e. the three years in which the autonomy program is available. is a vector of dummy variables for each New York City borough. Th e fitted values from Equat ion 3 2 are then used as the excluded instrumen t to estimate the following a standard two stage least squares model: (3 3) (3 4) where is t he fitted value from Equation 3 2 and is t he fitted value from Equation 3 3 Equatio ns 3 3 and 3 4 also include time dummies ( and ), as well as borough dummies 13 For a summary of t hese studies, see Eckel and Grossman (2002). 14 I do not use the year the policy became available as an instrument, due to the small number of participating schools in the pilot program.
112 3.4 Results Table 3 6 reports the estimates of the impact of autonomy on a particular expenditure category. Column (1) provides ordinary least squares (OLS) estimates as a baseline for comparison. Column (2) presents the results from the generated instrumental variables procedure. This is the preferred specification because it accounts for selection bias. Generally, the results show that aut onomous schools spend a larger percentage of their funds on teachers than schools that remain centralized as a result of being autonomous. Autonomous schools spend a smaller percentage on leadership, other direct services, and centralized services than ce ntralized schools. T he estimates show that the percentage of expenditures allocated to centralized services decreases given autonomy. The column (2) estimate shows that centralized expenditures are 2.563 percentage points less in autonomous schools. This result is consistent with the intention of the autonomy program. The program was designed to shift resources away from centralized control and increase funds to direct services. The allocation of funds to teachers increases by 2.78 2 percentage points (or 0.53 standard deviations) as a result of autonomy. The corresponding dollar increase in per student funding would be approximately $381.16. 15 The two estimates, 1.036 and 2.78 2 are not statistically different for one another. The increased percenta ge of funding assigned to teachers may reflect that teaching services are a more productive 15 The average total per student expenditures in autonomous schools are $14,222.57. The average per student expenditures are lower in non autonomous schools, $12,186.74. Therefore this dollar equivalent is after already accounting for the difference in levels.
113 input than other inputs. It is not clear from these estimates whether the school is hiring more experienced teachers or more total teachers. 16 There is also evide nce of changes in the percentage of expenditures allocated to other direct services. In column (2), the percentage of expenditures in this category decreases by 2.13 percentage points given autonomy. This decrease corresponds to a 0.65 of a standard devi ation. The OLS estimate is not stat istically significant. The two estimates, however, are statistically different. Other direct s ervices includes a range of expenditures, such as expenditures on food services, transportation, school safety, computer sys tem support, custodial services, building maintenance, leases, and energy costs. Schools with autonomy decrease leadership expenditures by 1.36 percentage points (0.52 standard deviations) according to generated instrumental variables estimate. This resul t is marginally statistically significant at the 10 percent level. The OLS and IV estimate are statistically different and of opposite signs. Decreases in leadership ex penditures may suggest that funds previously assigned to leadership under centralized control have a more productive use for students. Autonomous schools do not spend different percentages of their total expenditures on other classroom services, support services, or capital services. This may suggest that the centralized setting was alread y aptly allocating this funding or that autonomy to assign these funds did not actually increase as a result of the program. 16 Experimental evidence from Project STAR finds large (approxima tely 0.2 of a standard deviation) increases in student achievement resulting from smaller classes. Few studies have found significant positive effects of teacher experience beyond the first 3 years of experience.
114 3.5 Conclusion The changes in expenditures patterns provide further insight about why autonomy improved student achievement in New York City Public schools. The selection of schools into the autonomy program is again an important consideration. The generated instrumental variables procedure deals with this selection. The estimates show that autonomy increases the amount of fun ds contributed to teachers. Further, autonomy decreases expenditures on leadership, centralized, and capital services. Future research is needed to examine the types of changes in teachers that occur as a result of the increased funding.
115 Table 3 1. Definitions of expenditure categories Function Description of f unction I. Direct services to s chools Services provided directly to public school students and staff, and which take place primarily in the school building during the school day, during the school year. A. Classroom i nstruction School based direct instructional services provided primarily in classrooms. i. Teachers All teachers who provide direct instruction on full time, part time or per diem basis or during their preparation pe riods. ii. Classroom Includes spending on paraprofessionals, other classroom staff, textbooks, library books, instructional supplies and equipment, professional development, contracted instruction, and summer and evening services B. Instr. suppor t s ervices Direct services to students that supplement the basic classroom instructional program. This includes counseling services, attendance and outreach services, drug prevention, referral and evaluation services, support for after school and student activities, and support for parental involvement. C. Lea dership/supervision/s upport Includes salaries of full time and per diem assistant principals, salaries of deans and program coordinators, secretaries, school aides, and the supplies and materials to maintain administrative offices. This category is net of principal's salary since the principal cannot set his/her own salary. D. Other direct s ervices Includes food services, transportation, school safety, computer system support, custodial servic es, building maintenance, leases, and energy costs.
116 Table 3 1. Continued Function Description of Function II. Central Includes sabbaticals/leaves, instructional support and administration paid to the district office, instructional suppor t paid to the central office, operational offices (e.g. Office of the Chief Financial Officer), central leadership (e.g. school board/ Chancellor's office), and funds spent for retiree health and welfare. III Capital s ervices i. Debt s ervice Pay ment for long term debt in support of school construction
117 Table 3 2. Summary statistics Variable Mean Standard Deviation Minimum Maximum Auto nomy 0.04 0.196 0 1 Percentage of expenditures by c ategory Teachers 41.5 5.2 16.2 64.4 Other c lassroom 10.8 3.4 2.4 34.1 Leadership 8.7 2.6 0.9 24.1 Support s ervices 7.6 2.6 0.4 26.1 Other direct s ervices 14.7 3.3 2.8 35.1 Centralized s ervices 7.5 1.3 2.6 13.1 Capital s ervices 6 .0 1.9 0.9 12.9 Teacher v ariables Mean a ge 41.1 3.9 25 58.1 Proportion of uncertified teachers 0.108 0.078 0 1 Principal v ariables Principal experience (admin) 8.5 6.0 0 53.0 Principal experience (teacher) 11.5 8.4 0 61.0 Student v ariables Class size 24.7 4.4 1 49.9 Proportion white 0.386 0.269 0 1
118 Table 3 2. Continued Variable Mean Standard deviation Minimum Maximum Student variables (continued) Proport ion black 0.345 0.304 0 1 Proportion Hispanic 0.145 0.216 0 0.945 Proportion FRL 0.696 0.279 0.029 1 Proportion ELL 0.109 0.099 0 1 Proportion repeating 0.029 0.043 0 1 Mean absences 12.3 3.8 1.1 40 Mean suspensions 0.038 0.064 0 0.794 School size 708.3 343.5 63 3770 N=4120 Notes: FRL stands for free and reduced lunch status. ELL stands for English language learners.
119 Table 3 3. Comparison of means Non Auto Auto Difference Standard Error I. Direct Services A. Classroom instruction i. Teachers 42.707 38.763 3.944*** 0.32 ii. Other classroom 10.544 10.8 0.256 0.203 B. Leadership 7.397 8 .127 0.73*** 0.147 C. Support services 8.364 9.74 1.376*** 0.158 D. Other direct services 15.092 13.793 1.299*** 0.203 II. Centralized services 7.579 7.069 0.51*** 0.078 III. Capital services 5.314 6.584 1.27*** 0.129 Total 96.997 94.876 Note: The means reported are a the percentage of expenditures allocated to a particular category. The total percentage accounted for is less than 100 because the principal's salary is removed.
120 Table 3 4. Comparison of means pre tre atment Non Auto Auto Difference Standard Error I. Direct services A. Classroom instruction i. Teachers 44.073 43.25 0.823*** 0.248 ii. Other classroom 11.056 11.536 0.256 0.203 B. Leadership 7.458 7.153 0.305** 0.13 C. Support services 8.184 8.603 0.419*** 0.158 D. Other direct services 15.072 14.703 0.369*** 0.167 II. Centralized services 7.408 7.359 0.049 0.078 III. Capital services 4.207 4.19 0.017 0.06 Total 97.458 96.794 Note: The means reported are the percentage of expenditures allocated to a particular category. The total percentage accounted for is less than 100 because the principal's salary is removed.
121 Table 3 5 Expenditure c ategory Teachers 0.243** (0.120) Other c lassroom 0.240** (0.121) Leadership 0.234* (0.121) Support s ervices 0.249** (0.121) Other direct s ervices 0.233* (0.121) Centralized s ervices 0.250 ** (0.120) Ca pital s ervices 0.250** (0.121) Controls Yes Year dummies Yes School dummies Yes The estimates reported are the impact of a principal being female on the propensity to select autonomy. The standard errors report are Huber White robust.
122 Table 3 6 The impact of autonomy on expenditure allocation OLS Generated i nstrument Dependent variable c ategory (1) (2) Teachers 1.036*** 2.782* (0.257) (1.51 3) Other c lassroom 0.292** 0.203 (0.174) (0.998) Leadership 0.276* 1.364* (0.143) (0.734) Support s ervices 0.411** 0.562 (0.164) (0.828) Other direct s ervices 0.152 2.130** (0.176) (0.883) Centralized s ervices 0 .763*** 2.563*** (0.066) (0.46) Capital s ervices 0 .192*** 0.231 (0.045) (0.273) Controls Yes Yes Year dummies No Yes School dummies No Yes The standard errors report are Huber White robust.
123 CHAPTER 4 SIMULATION OF TEST SCORE I MPACTS AND PROGRAM COSTS 4.1 Introduction The theoretical analysis presented in Chapter 1 suggests that increased autonomy allows the principal to better respond to their characteristics and the characteristics of the student body they serve. Whether this creates gains for student achievement, however, depends largely on how the budgetary reward relates to the principal's tas tes. In this chapter, I extend the analysis by var ying the budgetary reward to simulate what outcomes are predicted within a given p opulation of schools and students. The simulation model allows autonomous schools to select into the program based on th e decision making described in C hapter 1 The population of schools and students comes from New York City Public Schools. I draw a ra ndom sample of 100 New York City Public Schools and where possible use observed data from these schools to populate the model. This analysis provides a further understanding of the costs of the program, including monies that are consumed as rents and the direct cost of doling out the budgetary reward. The simulations also illustrate the impact of an autonomy program on different types of students. I focus in particular on the diverse impact low and high socioeconomic students are likely to experience. Th e results show that the level of rents consumed varies dramatically with the budgetary reward and principal risk aversion. These simulations demonstrate that the rewar d and low levels of risk aversion, that to achieve the largest test score impacts some rents are consumed, and that the impact on high socioeconomic students is
124 always of a larger magnitude than the impact on low socioeconomics students, even when the impa ct is negative. S ection 4. 2 translates the theoretical model proposed in chapter 1 into a computable model used for the simulations presented in this paper. This section delineates how observable and unobservable parameters from the theoretical model are identified. Section 4. 3, then, provides a description of calibrating the model. Section 4. 4 presents results. The paper concludes in Section 4. 5. 4.2 The Computable Model In this section, I map the determinants of the theoretical model into computabl e and observable input parameters. I explain how I construct student types, define the principal parameters, and per student expenditures. I also introduce a simple calibration procedure. By calibration, I mean that I select parameters that generate pre dictions which closely match the empirically observed counterparts. The computable model allows me to examine how predicted choices, test score impacts, rents and other program costs will vary with the budgetary incentive and risk aversion. The section is arranged into three subsections. These subsections decompose the input parameters into the following categories: (1) district parameters, (2) principal parameters, and (3) student parameters. 4.2.1 District Parameters The district determines the leve l of per pupil funding and the budgetary reward coefficient I assume the output elasticities of effort and expenditures are fixed at the district level. Output elasticities of effort and e xpenditures. The test score producti on function used in the theoretical model,
125 (4 1) is assumed to exhibit non increasing returns to scale. For the computable equivalent, I assume the special case of constant returns to scale holds, i.e. I utilize best estimates from the existing literature to approximate the output elasticity of expenditures and then use these estimates to compute the output elasticity of efforts. For an estimate to qualify as a best estimate, I follow the classification suggested in 8 meta analysis. First, higher levels of aggregation tend to create more biased estimates. Therefore, the estimate must come from an analysis of school level or classroom level data. Second, the cumulative nature of education requires that previous inpu ts are adequately controlled for. This can be satisfied at minimum by including lagged test scores or through a successful quasi experimental design. The best estimates suggest a value of between 0.029 and 0.057. 1 2 3 To test how sensitiv e 1 I first examined the studies from Hanus hek (1998) that focus on the impact of per pupil funding on elementary and/or middle school test scores. The majority of those studies, however, do not meet the criteria for a best estimate. The estimates used in this simulation come directly from the fo llowing two papers: Guryan(2000) and Holmlund, McNally, and Viarengo (2010). 2 Guryan produces robust estimates of the impact of per pupil expenditures on average test scores for 4th grade students. Estimates range from .033 .057. All estimates are sta tistically significant and positive. The median estimate is .034. This means a $1,000 increase in per pupil expenditures causes a 34 point increase in average test scores. Findings for 8th grade test scores are not consistent. He finds both positive an d negative effects and very few are statistically significant. 3 Holmlund, et al. directly respond to the concerns raised in the Hanushek paper. They meet the conditions he sets forth for a quality analysis, i.e. they control for previous test scores of s tudents and they have school level expenditures. Controlling for previous test scores will control for the fact that within school funding allocation is redistributive. They examine the impact of per pupil expenditures on test scores for 11 year olds in English primary schools. For the specifications that include year dummies and school fixed effects, the coefficients range from 0.029 0.051. The lowest coefficient estimate comes from a model that includes school specific trends. They find that $1,000 increase improves test scores by about of a standard deviation.
126 the specification is to changes in I compute the model for the minimum (0.029) and maximum (0.057) values of this range. Per pupil funding. The level of actual per pupil funding within a school is directly observable from school level ex penditure data. This data comes from the New Expenditure Reports (SBERs). The level of per pupil funding varies across schools due to differences in complex funding algorithms designed to account for the s ize and makeup of a student population. Budgetary reward coefficient. The budgetary reward coefficient c is varied across simulations. As a starting point, I vary c from 1 to 10,000,000. To give c some context it is useful to think in terms of the am ount of budgetary reward a school would receive for a given c and change in school level scores. To determine the amount of budgetary reward a school receives, the coefficient is multiplied by the gain in the school level z score. If c=1, this means that would be a substantial change, it is a useful benchmark to think of the budgetary reward that this would produce. For instance a 1,000,000 reward coefficient coupled with a following year, i.e. it would lead to a 1% increase in the budget. In per student terms, this would lead to additional $140.06 per student 4 The Empowerment Schools Organization reports that on average autonomous schools gained control of an additional $250,000 in discretionary funds. 5 4 5
127 4.2.2 Principal Input Parameters In the theoretical model each school has one principal in any given period. In the observed data, there are two cases that do not fit this simplification: (1) schools that have more than one principal leading a school simultaneously and (2) schools that experience a change in the principal during the year. In the computable model, I deal with these realities by limiting the sample to schools with one principal in the first case and using the characteristics of the principal at the beginning of the year in the second. The characteristics of a principal are def ffectiveness k, risk aversion concern for her career and her taste for rents and student achievement The first characteristic can easily be mapped into concepts that can be derived from aversion, I rely on the risk aversion literature. For simplicity, I assume that the career impact parameter does not vary across principals. The two taste parameters cannot be observed. I set =1. I estimate the taste for student achievement with data for schools that are always centralized. Principal effectiveness. is likely the product of both ability and experience, I ideally would create an index by combining a proxy for these two factors 6 I create an index of principal experience only. 6 Empirical studies such as Cla rk ( http://www.caldercenter.org/upload/working paper 38_final.pdf have not found a strong connection between college selectivity, a proxy for principal ability, and student perform ance.
128 To generate this index, I use the employment histories of all prin cipals in NYC public schools from 1982 to 2007. This data contains detailed information on education history, positions held, and demographic characteristics. The typica l career track of a principal involves starting as a classroom teacher, then becoming an assistant principal, and finally a principal. As a result, principal experience could be measured in many ways. I measure principal experience as the years the indiv idual has been employed as a principal. This definition is preferred to one including classroom experience because skills required in a leadership position are different than skills required in the classroom. Recent research analyzing the impact of pri ncipal characteristics on student test scores supports the idea that time spent as a principal has a greater impact than time spent teaching (Clark, Martorell, and Rockoff 2009; Branch, Hanushek, and Rivkin 2008; Coelli and Green 2009). Further, in recen t years NYC schools has made the path to becoming a principal more direct, i.e. diminishing the amount of time a prospective principal is required to teach and be an assistant principal. Therefore, including teaching and assistant principal experience may bias the ranking against principals who have entered the NYC School system post 1992. There are a few observations that have very high years of experience. I cross I discard any observations that would make a principal less t han 25 at the beginning of her career. 7 8 The resulting experiences are then normalized and made non negative. 7 For instance, I would discard the observation if years of experience is 40 and the principal is 50 years of age.
129 Risk aversion. I vary the coefficient of absolute risk aversion (CARA) across sp ecifications. The value of the CARA chosen is critical to making sure principals are appropriately risk averse. If the value is too high, then principals will make decisions in a manner that does not align with observed outcomes. A higher value of will make a principal less likely to select autonomy. On the other hand, if the principals are not risk averse enough, then risk aversion does not have a significant impact on their decision process and modeling them as risk neutral agents would have sufficed. 9 To understand how risk averse a particular value of CARA makes principals, it is useful to relate it to the risk premium and the size of the gamble. 10 In this model, the size of the gamble is the combined career impact, loss of quality, an d possible loss of budgetary reward that the principal faces. Quantifying the size of the gamble is, therefore, difficult. I start by using a coefficient of .0001 for all principals. If the individual is equally likely to gain or lose $10,000 in this ga mble, this CARA would equate to a risk premium of 43% of the gamble 11 I, then, try multiple specifications to see if I can obtain a better fit with observed outcomes. I increase the parameter in discrete intervals from .0001 to .05. Principal taste par ameters. selection of organizational regime and the resulting test scores. It is obvious that none 8 The youngest principal in the employment records is 26.5 years old. 9 As cited earlier, preliminary empirical evidence suggests See Appendix I. 10 See Babcock, et al. (1993) for a discussion of reasonable constants of ARA. 11 This calculation is based on the following relationship between CARA and the risk premium where the CARA i s 0.0001 and there is an equal probability of winning or losing $10,000. See Babcock et al. (1993) for an explanation.
130 of these taste parameters are observable. For the comparison of organizational regimes, which is the purpose of this analysis, it is the relative values of these parameters that are particularly important. For simplicity, I assume that all taste parameters are fixed across principals. Since both and the taste for quality are relate and estimate the other. I set equal to 1 for all principals. impacts the optimal outcomes under both centra lization and autonomy. Therefore, obtaining a reasonable approximation of obtain estimate of is discussed in Section I.IV. This procedure will rely on th e fact that will impact the observed average test scores for schools that never select autonomy. is set equal to 1. This value assumes that all principals value an increased budget a nd/or pure rents. 4.2.3 Student Parameters In the model a school is defined by its student population. The model assumes there are two types of students. In reality, schools are composed of many types of students that can be split along numerous charact eristics. I divide students along a SES students. To split a lunch status. On average, 74 percent of students enroll in the free and reduced lunch program in the NYCPS data Since students are either classified as free and reduced lunch eligible or not this division accounts for all students in a school. The model,
131 therefore, is populated with the actual school size. School size is an important input parameter in the model. Preliminary evidence suggests that smaller school s select autonomy. T he values of and are observable. Student types also differ by their test effectiveness and the randomness of their test scores. 12 This effectiveness parameter encompasses both endowed ability and the cumulative impacts of educa tion. Since depends at least partially on the ability of the student, it cannot be observed. Therefore, I estimate these parameters. The proced ure is described in section 4.3 erved test score variation where potentially observable determinant. A reasonable estimate of does not follow readily from existing literature. Further, there is not c onsistent evidence on whether low or high socioeconomic status students will have more volatile scores. Therefore, I set 4.3 Calibrating The Model I calibrate the parameters of the model that cannot be observed or estimated by setting them so that the model's predictions match observed educational outcomes 13 taste for quality and the student effectiveness parameters With these pa rameters 12 The fact that lower socioeconomic status students often perform less effectively on standardized tests is well documented. This lowe r test effectiveness may be the result of test bias, lower household educational inputs, or differences in test ability. For some limited evidence that attempts to sort out the impact of the allocation of resources versus type specific effects, see Brown (1991). Mizala et al. (2007) point out that ranking school quality by average test scores is highly correlated with ranking school quality by average socioeconomic status. 13 This is one use of model calibration. See Vanni, et al. (2011) for a general dis cussion of calibration methods.
132 in place, the second stage entails making discrete adjustments in the budgetary and the risk coefficients to approximate observed decisions and test scores. For the first stage, I minimize a well defined difference between the observed average tes t scores and the average test score predicted by the model for schools that never become autonomous. By only populating this stage of the calibration with data from non student typ es can be estimated separately from their career impact or taste for the budgetary reward parameters and I impose the assumption that > From these specifications, I then select the combination of parameter values that minimize s the sum of square errors, i.e. that minimizes (4 2) where j represents a school, y(j) is the observed school predicted school level test score for a given set of parameters The parameter values s elected through this process are then used in the second step of the calibration process. The second step of the process involves varying the budgetary reward and the risk aversion coefficient s Goodness of fit is again measured by minimizing the differe nce between observed and predicted school level test scores. In addition, I use the number of correctly predicted regime choices, i.e. autonomy or centralized control, as a secondary measure of goodness of fit. The two measures capture slightly different
133 attributes of the model. Since observable regime choices only tell us whether a school selected auto nomy or remained centralized, I can not obse rve if an autonomous school chooses to take rents. Within the second step of the process there are three itera tions. First, I fix the value of the risk aversion coefficient at 0.0001 and vary the budgetary reward between c=1 and c=1,000,000,000. This allows an isolated look at the impact of the budgetary reward. Then I vary both the risk aversion constant and t he budgetary reward simultaneously. 14 The risk aversion coefficient varies discretely between .0001 and .05. Finally, I try to improve the fit of the model by varying risk aversion with principal gender. In this specification, I set the risk aversion coe fficient equal to 0.0002 and 0.0001 for female and male principals, respectively. 4.4 Results These simulations illustrate how the budgetary reward set by the district, as well aversion, will impact the costs and outcom es produced score is largest for moderate levels of the budgetary reward and low levels of risk aversion, that to achieve the largest test score impacts rents are c onsumed, and that the impact on high socioeconomic students is always of a larger magnitude than the impact on low socioeconomics students, even when the impact is negative. In the following subsections, I explain where the model is most predictive of the outcomes observed in New York City schools, as well as examine a broader set of simulations to track how the budgetary incentive and risk aversion change the outcomes 14 In this specification, I check the impact of the test score output elasticities on the fit of the model. The fit does not appear particularly sensitive.
134 that this model would predict. The first subsection d iscusses the best fit results. Sec tion 4.4 .2 discusses how autonomy decisions vary with the budgetary reward and r isk aversion. Then, section 4.4.3 analyzes predicted changes in the costs of imple menting the autonomy program. Section 4.4.4 focuses attention on student impacts, exploring b oth the average score impacts and the type specific impacts. 4.4.1 Best Fits The two measures of best fit, i.e. how well the model approximates observed average test scores and observed regime choices, provide different views of where the model performs b est. In the dataset used for these simulations only 12 of the 100 schools select autonomy. In most specifications, the model predicts a larger take up of the autonomy program. The best fit by autonomy decisions occurs where there are stronger incentives i.e. when the budgetary reward and/or the risk coefficient is larger. In model 1, where the risk coefficient is fixed at 0.0001, the specification with the best fit occurs where c=5,000,000. In this specification, the model correctly predicts 25% of re gime choices. This result is reported in T able 4 2 Varying the risk aversion coefficient with gender does not change how the model fits with regard to this measure. The results for this speci fication are reported in T able 4 4 The best overall fit occ urs in the specification where both the risk aversion coefficient and the budgetary reward vary reported in T able 4 3 There are three cases that all provide an equally good fit. All of these occur where the risk aversion coefficient is pushed to the ma ximum value of 0.05. The corresponding budgetary incentives are also large, at c=1,000,000 c=5,000,000, and c=10,000,000. The results that align with this measure of fit suggest that the test score impact will be positive for both types of students and that the effect will be largest for type 1
135 students. When c=1,000,000 and are relatively small impa cts. The district, however, is able to improve scores while decreasing school expenditures by $ 2 9700 000 The district "saves" money because schools that do not meet the target score lose control of part of their budget. These average impacts are ver y small because only 2 schools select autonomy. When observed school level z scores are compared to those predicted by the model, the best fit with the model occurs at c=10, except when risk aversion varies by gender and the best fit occurs at c=1. Lowe r incentives appear to yield test score results closer to observed results. Focusing solely on the results where the test score outcomes align well with those observed, the average impact on a type 1 and a type 2 student are 0.337 and 0.132, respe ctively. With such low incentives, where the school only receives an increase of $10 for one standard deviation increase in test scores, schools respond by choosing to consume rents. In these cases, all schools choose to be autonomous and take rents. Th e result of this consumption of rents is lower test scores than would be obtained if all schools remained centralized for all student types. The type 1 students experience a larger decrease in test scores than type 2 students. This result is robust to ch anges in the risk aversion coefficient. Even pushing the risk aversion constant to 0.05 does not counteract the utility a principal can gain from consuming rents. By offering autonomy with low incentives, the district reduces school budgets by a negligi ble amount, $349. The rents consumed by schools outweigh this savings. For the cases where c=10, the total rents consumed in the district are $856,588,8 00
1 36 In the specification where risk aversion varies with gender, the impacts are similar. The avera ge impact on type 1 and type 2 students are 0.352 and 0.138, respectively. The total rents consumed are $ 856 588 800 4.4 .2 Autonomy Decisions The number of schools selecting autonomy varies widely in the simulations. The take up ranges from 100 school s to 2 schools. In T able 4 2 where the risk aversion coefficient is set low at 0.0001 for all principals, the take up ranges only from 100 to 82 schools. This specification clearly illustrates that the budgetary incentive significantly impacts the takin g of rents. The number of autonomous schools predicted to take rents ranges from 100 to 0 schools. Setting the budgetary incentive high reduces the amount of rents consumed. For values at or above c= 10 000 000 no schools consume rents and 82 schools se lect autonomy. Even with 82 schools selecting the program the gains When risk aversion is allowed to vary as well, it becomes clear that risk aversion greatly reduces the number of schools taking rents when co upled with moderate to strong budgetary incentives. The results of t his specification are in Table 4 3 Beginning at c=100,000, which equates to a $14 per student reward for an increase of my and choose not to consume any rents. The proportion of schools choosing autonomy with no rents increases with the strength of the budgetary incentive. 4.4.3 Costs of the Program The program modeled in this analysis has two costs. First, there is the direct cost of implementing the budgetary reward. The direct cost of implementing the
137 llowing year and penalize the school by reducing it. The second cost of this program is the pure rents that principals gain from the ability to consume their baseline budget when they select autonomy. The results tables present the sum of pure rents con sumed in the district. Rents. Even at low levels of risk aversion the rents consumed by autonomous schools can be pushed to zero as long as the budgetary reward is very strong. For instance, rents are zero for all levels of risk aversion when c=10,000,00 0. This occurs because schools are willing to tradeoff the consumption of pure rents for access to an increased budget in the following year. The maximum amount of rents consumed predicted by the model is $856,588,800. For budgetary rewards at or below c=10,000, this maximum amount of rent is consumed. Within these results, however, the model shows that positive test score impacts can be achieved even with maximum rents being consumed. These positive impacts are the result of the principal changing th eir allocations under autonomy. These reallocations include the principal's effort allocations. The largest positive impact with maximum rents occurs where c=10,000. For this value, test scores be under centralized control. The model also shows that given sufficiently strong incentives rents can be driven to zero. Where rents are zero, the average test score impacts are quite small, ranging from 0.004 solely the result of very few schools selecting autonom y. As discussed in section 4.4 b., when c=10,000,000
138 negligible. The largest test score impacts occur where the level of rents consumed is still high, between $476 million and $706 million. In Table 4 2 when schools consumed a total of $476 million in rents, approximately $9.7 million per school 15 or $13,605.44 per student, the average test score impact is 0.7 Where $7 06 million is consumed, which is corresponds to $9.289 million per school 16 or $13,010.47 per student, the average test score impact is 0.765 The range of outcomes predicted highlights that within this program some rents are necessary to achieve signific ant test score impacts. Direct cost of incentives. In addition to the rents consumed, the direct cost of the budgetary incentives varies with the budgetary reward and risk aversion parameters. In Table 4 2 where only the budgetary reward varies, the d irect costs range from $293,490 to $7,960,000,000. The direct cost of incentives is negative when the district is penalizing schools for not meeting the target score in larger amounts than they are rew arding score gains. The results in Table 4 3 show t hat there are cases where the cost of the budgetary reward is negative, i.e. the district reduces the budgets of the schools, and the test score gains are positive. When c=100,000, this holds for all levels of risk aversion. This also occurs for stronger values of c at high levels of risk aversion. 15 $476 million/49 schools consuming rents 16 $706 million/76 schools consuming rents
139 4.4.4 Student level impacts Average impacts. The average pro gram impact produced in these simulations ranges widely from all schools in the district select autonomy and choose to consume rents as well. 17 All of these cases occur where the budgetary reward is 1, 10, or 100, i.e. the school principal the target score. This result shows both that autonomy with inadequate incentives can decrease average student achievem ent and that these negative results occur only with very low incentives. The largest impacts on average student achievement occur at moderat e levels of incentives. Table 4 2 shows that the largest average score impact of this incentive level, this means that on average remaining centralized. To achieve this level of increase, schools are on average receiving an additional $25,778.01 18 for their subsequent year budget. This is equivalent to approximately $36.10 per student. In all three tables when the budgetary reward is 100,000, the average score impact is positive and strong, ranging from 0.3351 cases the total direct cost of incentives is negative. This means that there are schools choosing autonomy that will fail to meet their target scores and suffer budgetary losses, yet still find it utility maximizing to select autonomy. The target score is set equal to the score the school would achieve if they remained centralized. In these cases, the 17 Note that all three tables report results where 100 schools select autonomy with rents and there are sizable positive test score impacts. See e.g. Table I, Table II, and Table III for c=1,000 and c=10,000. 18 98 schools select autonomy in this case. Therefore, on average schools receive an addition 2,526,245/98 in budgetary reward.
140 principal is trading off the budgetary reward for consumption of pure rents and the test score gains above what centralized control would allow. In Table 4 2 the average $2,934.90 19 or $4.11 per student. Very strong incentives, i.e. budgetary rewards of 10,000,000 or more, have small test score impacts. The test score impact s range from 0.001 direct costs of incentives are often immense. In Table 4 2 where the risk aversion constant is low and the budgetary reward is allowed to vary to an unreasonably 20 high level of 1 billion, the direct cost of incentives r anges from $79.6 million to $7.96 billion. In the subsequen t specifications, i.e. Tables 4 3 and 4 4 the maximum level of budgetary reward is 10 million. This budgetary reward would result in $100,000 e target score. In Table 4 2 at high levels of risk aversion, 0.02 and 0.05, the direct cost of incentives is negative. The model predicts a range of average test score impacts. The predictions, however, show for moderately strong incentives we can ex pect the test score impacts to be positive on average. Type impacts. The two fundamental insights the simulations provide about type impacts is that both type's tests scores change in the same direction and that the impact is always larg er for type 1 stud ents. Table 4 2 shows that for low levels of incentives, where test impacts are negative, both types' test scores decline as a result of the 19 Since all schools in the district choose autonomy the impact on the average school is 293,490/100. 20 These cases are pursued only to help provide bounds on predicted results.
141 principals' allocations. Type 2's test scores decline approximately 2/5 to 1/2 21 the amount that type 1's test sco res decline. This result is rep eated in the results of Table 4 3 and 4 4 The largest difference occurs for strong incentives coupled with low levels of risk aversion Table 4 2 where risk aversion is low, shows that for any budgetary reward above 10 m illion, the impact for type 1 students (0.010) will be 10 times as large as the impact on type 2 students (0.001). In Table 4 3 the largest impact occurs for budgetary incentives above 1,000,000 and risk aversion at 0.02. The impact for type 1 students (0.0055) will be a little more than 6 times the size of the impact on type 2 students (0.0009). 4.5 Conclusion These simulations demonstrate is largest for moderate levels of the budgetary reward and low le vels of risk aversion, that to achieve the largest test score impacts rents are consumed, and that the impact on high socioeconomic students is always of a larger magnitude than the impact o n low socioeconomics students. Ultimately, how a district should engage these results depends on the importance they or outside agencies are placing on student achievement and their ability to pay the requisite rewards. Autonomy in and of itself creates the possibility of either positive or negative student achievement impacts. For the impacts of autonomy to improve student scores, there need to be adequate incentives. 21 In columns 1 and 2, the ratio of the type 2 to type 1 impact is 0.392 and 0.395, respectively. In column 2, the ratio is 0.473.
142 Future research is needed to analyze the interaction of different incentive structures with autonomy. Further, the computable model in this paper gener ally predicts larger scores impacts than are observed. Future research is needed to determine why this difference occurs. One possibility is that the model does not adequately account for the fact that the value added for type 1 students may be smaller t han the value added for type 2 students.
143 Table 4 1. Mapping determinants of the theoretical into the computable model Observed by Researcher Notation Data Source Values Used District Parameters Output elasticity of effort No literature 0.02 9 0.057 Output elasticity of expenditures No literature 0.971 0.943 Per pupil funding Yes Z NYCPS Data Actual per pupil funding Budgetary reward coefficient No c Selected by researcher 1 10,000,000 Principal Input Parameters Principal ef fectiveness Yes k NYCPS Data Years of principal experience Risk aversion No Selected by researcher 0.001 0.05 Taste for quality No Estimated Taste for career impact No Selected by researcher 1 Taste for rents No Selected by researcher 1 S tudent Parameters School size Yes N NYCPS Data Number of students enrolled Number of high test effectiveness students Yes n 1 NYCPS Data Number of non FRL students Number of low test effectiveness students Yes n 2 NYCPS Data Number of FRL students Test effectiveness No I i Estimated Notes: FRL refers to free and reduced lunch eligible.
144 Table 4 2. District wide impact of autonomy program: Varying budgetary r Budgetary reward c onstant 1 10 100 1000 10000 100000 500000 1000000 5000000 Risk aversion c onstant 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 Autonomous s chools 100 100 100 100 100 100 98 94 81 Schools taking r ents 100 100 100 100 100 97 76 49 1 Average score i mpact 1 0.192 0.183 0.068 0.131 0.154 0.398 0.765 0.700 0.197 Type 1 i mpact 2 0.352 0.334 0.112 0.262 0.303 0.798 1.125 1.097 0.420 Type 2 i mpact 2 0.138 0.132 0.053 0.088 0.104 0.263 0.643 0.566 0.122 Direct cost of i ncentives 3 36.1 2 349.31 2042.33 3412.60 59609.49 293490 2526245 7015125 39.8 million Rents (in millions) 856.59 856.59 856.59 856.59 856.59 846 706 476 12.8 Choices predicted c orrectly 12 12 12 12 12 12 12 16 25 Score d ifference 4 10 3.285 102.42 152.657 266.550 268.371 296.222 265.964 266.214 265.321 ticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minu s the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This va lue is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
145 Table 4 2. Continued Budgeta ry reward c onstant 10000000 50000000 100000000 500000000 1000000000 Risk aversion c onstant 0.0001 0.0001 0.0001 0.0001 0.0001 Autonomous s chools 82 82 82 82 82 Schools taking r ents 0 0 0 0 0 Average score i mpact 1 0.003 0.003 0.003 0.003 0.003 Type 1 i mpact 2 0.010 0.010 0.010 0.010 0.010 Type 2 i mpact 2 0.001 0.001 0.001 0.001 0.001 Direct cost of i ncentives 3 79.6 million 398 million 796 million 3.98 billion 7.96 billion Rents (in millions) 0 0 0 0 0 Choices predicted c orrectly 24 24 24 24 24 Score d ifference 4 265.304 265.304 265.304 265.304 265.304 simulations assume constant returns to scale. Therefore, the output elasticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minus the average score that would result from all scho ols staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This value is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
146 Table 4 3. District wide i mpact of autonomy program: Varying budgetary reward and risk aversion c Risk aversion c onstant 0.0001 0.0002 0.0005 0.001 0.002 0.005 0.01 0.02 0.05 Budgetary reward c onstant=1 Autonomous s chools 100 100 100 100 100 100 100 100 100 Schools taking r ents 100 100 100 100 100 100 100 100 100 Average scor e i mpact 1 0.1916 0.1916 0.1916 0.1916 0.1916 0.1916 0.1916 0.1916 0.1916 Type 1 i mpact 2 0.3516 0.3516 0.3516 0.3516 0.3516 0.3516 0.3516 0.3516 0.3516 Type 2 i mpact 2 0.1378 0.1378 0.1378 0.1378 0.1378 0.137 8 0.1378 0.1378 0.1378 Direct cost of i ncentives 3 36.1 36.1 36.1 36.1 36.1 36.1 36.1 36.1 36.1 Rents (in millions) 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 Choices predicted c orrectly 12 12 12 12 12 12 12 12 12 Score d ifference 4 103.2846 103.2846 103.2846 103.2846 103.2846 103.2846 103.2846 103.2846 103.2846 Budgetary reward c onstant=10 Autonomous s chools 100 100 100 100 100 100 100 100 100 Schools taking r ents 100 100 100 100 100 100 100 100 100 Average score i mpact 1 0.1825 0.1825 0.1825 0.1825 0.1825 0.1825 0.1825 0.1825 0.1825 Type 1 i mpact 2 0.3338 0.3338 0.3338 0.3338 0.3338 0.3338 0.3338 0.3338 0.3338 Type 2 i mpact 2 0.1317 0.1317 0.1317 0.1317 0.1317 0.1317 0.1317 0.1317 0.1317 Direct cost of i ncentives 3 349 349 349 349 349 349 349 349 349 Rents (in millions) 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 Choices pr edicted c orrectly 12 12 12 12 12 12 12 12 12 Score d ifference 4 102.4243 102.4243 102.4243 102.4243 102.4243 102.4243 102.4243 102.4243 102.4243 ticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minu s the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This va lue is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
147 Table 4 3. Continued Risk Aver sion Constant 0.0001 0.0002 0.0005 0.001 0.002 0.005 0.01 0.02 0.05 Budgetary Reward Constant=100 Autonomous Schools 100 100 100 100 100 100 100 100 100 Schools Taking Rents 100 100 100 100 100 100 100 100 100 Aver age Score Impact 1 0.0678 0.0678 0.0678 0.0678 0.0678 0.0678 0.0678 0.0678 0.0678 Type 1 Impact 2 0.1123 0.1123 0.1123 0.1123 0.1123 0.1123 0.1123 0.1123 0.1123 Type 2 Impact 2 0.0528 0.0528 0.0528 0.0528 0.0528 0.0528 0.0528 0.0528 0.0528 Direct Cost of Incentives 3 2040 2040 2040 2040 2040 2040 2040 2040 2040 Rents (in millions) 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 Choices Predicted Correctly 12 12 12 12 12 12 12 12 12 Score Difference 4 152.6567 152.6567 152.6567 152.6567 152.6567 152.6567 152.6567 152.6567 152.6567 Budgetar y Reward Constant=1000 Autonomous Schools 100 100 100 100 100 100 100 100 100 Schools Taking Rents 100 100 100 100 100 100 100 100 100 Average Score Impact 1 0.1315 0.1315 0.1315 0.1315 0.1315 0.1315 0.1315 0.1315 0.1315 Type 1 Impact 2 0.2619 0.2619 0.2619 0.2619 0.2619 0.2619 0.2619 0.2619 0.2619 Type 2 Impact 2 0.0877 0.0877 0.0877 0.0877 0.0877 0.0 877 0.0877 0.0877 0.0877 Direct Cost of Incentives 3 3410 3410 3410 3410 3410 3410 3410 3410 3410 Rents (in millions) 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 Choices Predicted Correctly 12 1 2 12 12 12 12 12 12 12 Score Difference 4 266.5504 266.5504 266.5504 266.5504 266.5504 266.5504 266.5504 266.5504 266.5504 ticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minu s the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This va lue is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
148 Table 4 3 Continued Risk a ver sion c onstant 0.0001 0.0002 0.0005 0.001 0.002 0.005 0.01 0.02 0.05 Budgetary reward c onstant=10000 Autonomous s chools 100 100 100 100 100 100 100 100 100 Schools taking r ents 100 100 100 100 100 100 100 100 100 Average s core i mpact 1 0.1539 0.1539 0.1539 0.1539 0.1539 0.1539 0.1539 0.1539 0.1539 Type 1 i mpact 2 0.3031 0.3031 0.3031 0.3031 0.3031 0.3031 0.3031 0.3031 0.3031 Type 2 i mpact 2 0.1037 0.1037 0.1037 0.1037 0.1037 0.1037 0.1037 0.1037 0.1037 Direct cost of i ncentives 3 59600 59600 59600 596 00 59600 59600 59600 59600 59600 Rents (in millions) 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 856.59 Choices predicted c orrectly 12 12 12 12 12 12 12 12 12 Score d ifference 4 268.3709 268.3709 268.3709 268.3709 268.3709 268.3709 268.3709 2 68.3709 268.3709 Budgetary reward c onstant=100000 Autonomous s chools 100 100 100 98 96 93 92 88 67 Schools taking r ents 97 97 97 95 93 91 90 86 66 Average score i mpact 1 0.3978 0.3978 0.3978 0.3793 0.3652 0.3550 0.3546 0.3410 0.3351 Type 1 i mpact 2 0.79 85 0.7985 0.7985 0.7156 0.6662 0.6397 0.6396 0.6690 0.6913 Type 2 i mpact 2 0.2632 0.2632 0.2632 0.2664 0.2642 0.2594 0.2589 0.2308 0.2155 Direct cost of i ncentives 3 293000 293000 293000 295000 296000 427000 427000 430000 705000 Rents (in million s) 846 846 846 838 825 791 784 741 605 Choices predicted c orrectly 12 12 12 12 14 17 18 20 35 Score d ifference 4 296.2219 296.2219 296.2219 296.231 296.2301 284.1405 284.1396 284.0428 261.2036 ticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minu s the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This va lue is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
149 Table 4 3. Continued Risk a ver sion c onstant 0.0001 0.0002 0.0005 0.001 0.002 0.005 0.01 0.02 0.05 Budgetary reward c onstant=500000 Autonomous s chools 98 95 92 84 74 45 25 14 3 Schools taking r ents 76 74 72 66 56 32 16 8 1 Average score i mpact 1 0.7646 0.7075 0.7040 0.7119 0.6776 0 .6080 0.4541 0.4176 0.3348 Type 1 i mpact 2 1.1253 0.9718 0.9703 1.1015 1.1039 1.1688 0.8769 0.8942 0.7687 Type 2 i mpact 2 0.6434 0.6187 0.6145 0.5811 0.5344 0.4196 0.3121 0.2575 0.1891 Direct cost of i ncentives 3 2530000 2400000 2190000 2060000 2050000 684 000 48700 647000 2970000 Rents (in millions) 706 673 662 611 533 343 198 110 204 Choices predicted c orrectly 12 15 18 24 32 57 69 78 85 Score d ifference 4 265.9643 266.1878 266.3936 266.039 266.0334 255.2313 257.7673 260.9889 252.2111 Budgetary r ewar d c onstant=1000000 Autonomous s chools 94 92 79 64 43 26 13 7 2 Schools taking r ents 49 48 39 28 15 6 1 0 0 Average score i mpact 1 0.6999 0.6991 0.6638 0.5811 0.5142 0.3919 0.3446 0.0020 0.0010 Type 1 i mpact 2 1.0973 1.0970 1.1005 1.0074 1.1452 0.8915 0.7 911 0.0055 0.0026 Type 2 i mpact 2 0.5664 0.5654 0.5171 0.4379 0.3022 0.2241 0.1946 0.0009 0.0005 Direct cost of i ncentives 3 7020000 6760000 5980000 5660000 5040000 2630000 625000 795788.3 5938103 Rents (in millions) 476 468 385 302 181 84.6 20.3 0 0 C hoices predicted c orrectly 16 18 27 40 55 72 81 85 86 Score d ifference 4 266.214 266.4529 265.3688 264.6787 264.4359 250.7437 254.4595 258.5102 252.2116 ticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minu s the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This va lue is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
150 Table 4 3. Continued Risk a ver sion c onstant 0.0001 0.0002 0.0005 0.001 0.002 0.005 0.01 0.02 0.05 Budgetary reward c onstant=5000000 Autonomous s chools 81 78 67 56 41 25 15 7 2 Schools t aking r ents 1 1 0 0 0 0 0 0 0 Average score i mpact 1 0.1969 0.1969 0.0035 0.0035 0.0033 0.0030 0 .0025 0.0020 0.0010 Type 1 i mpact 2 0.4197 0.4198 0.0100 0.0098 0.0093 0.0082 0.0068 0.0055 0.0026 Type 2 i mpact 2 0.1220 0.1220 0.0013 0.0013 0.0013 0.0012 0.0011 0.0009 0.0005 Direct cost of i ncentives 3 39800000 38400000 35200000 32900000 28500000 14700 000 4414382 3978754 29700000 Rents (in millions) 12.8 12.8 0 0 0 0 0 0 0 Choices predicted c orrectly 25 28 37 44 57 73 81 85 86 Score d ifference 4 265.3211 265.5443 265.392 264.3618 263.7245 249.6506 254.2768 258.5102 252.2115 Budgetary reward c onstan t=10000000 Autonomous s chools 82 79 70 56 41 24 13 7 2 Schools taking r ents 0 0 0 0 0 0 0 0 0 Average score i mpact 1 0.0035 0.0035 0.0035 0.0035 0.0033 0.0029 0.0024 0.0020 0.0010 Type 1 i mpact 2 0.0098 0.0099 0.0100 0.0098 0.0093 0.0080 0.0066 0.0055 0. 0026 Type 2 i mpact 2 0.0013 0.0013 0.0013 0.0013 0.0013 0.0012 0.0010 0.0009 0.0005 Direct cost of i ncentives 3 79.6 million 76.9 million 70.6 million 65.7 million 56.6 million 24.5 million 3.78 million 7.96 million 59.4 million Rents (in millions) 0 0 0 0 0 0 0 0 0 Choices predicted c orrectly 24 27 34 44 57 74 81 85 86 Score d ifference 4 265.3044 265.529 265.3856 264.3618 263.7853 252.1375 257.1524 258.5102 252.2115 ticity of effort is jointly determined. 1. This value is the average school level score given the selection that has occurred minu s the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized. 3. This va lue is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
151 Table 4 4. District wide impact o Budgetary reward c onstant 1 10 100 1000 10000 100000 500000 1000000 5000000 10000000 Autonomous s chools 100 100 100 100 100 100 98 94 81 82 Schools taking r ents 100 100 100 100 100 97 76 49 1 0 Average scor e i mpact 1 0.192 0.183 0.068 0.131 0.154 0.398 0.765 0.700 0.197 0.003 Type 1 i mpact 2 0.352 0.334 0.112 0.262 0.303 0.798 1.125 1.097 0.420 0.010 Type 2 i mpact 2 0.138 0.132 0.053 0.088 0.104 0.263 0.643 0.566 0.122 0.001 Direct cost of i ncentive s 3 36.12 349.31 2042.33 3412.60 59609.49 293489.97 2526245.3 7015124.5 39767064 79608088 Rents (in millions) 856.59 856.59 856.59 856.59 856.55 846.4 706.29 475.72 12.8 0 Choices predicted c orrectly 12 12 12 12 12 12 14 16 25 24 Score d ifference 4 10 3.809 104.335 167.077 290.544 292.422 296.307 292.975 293.052 291.642 291.643 principals. 1. This value is the average school level score given the selection that has occurred minus the average score that would result from all schools staying centralized. 2. This value is the average type i score given the selection that has occurred minus the average type i score that would result from all schools staying centralized 3. This value is the budgetary reward multiplied by the gain in test scores summed over all 100 schools in the sample. 4. This the squared difference between the predicted and observed school level score summed over all schools.
152 APPENDIX A WHY NE W YORK CITY PROVIDES A GOOD BASIS FOR THIS STUDY? Although the practice of increasing autonomy for schools is neither new nor solely practiced in New York City Public Schools (NYCPS), focusing the empirical specification on these schools provides a suffic ient number of observations, a clearly defined program, and access to a rich set of data. New York City is currently the largest school district in the U.S. and serves over 1 million students. The analysis in this paper focuses on school level decisio ns and their requisite outcomes. This study analyzes outcomes from the 2004 2005 school year through the 2007 2008 school year. The total number of schools in NYCPS in each of these years was 1,205, 1,408, 1,429, and 1,454 1 respectively. Further, appro ximately a third of these schools select autonomy over this period. A total of 505 schools choose autonomy between 2004 2007. Other school districts in the U.S. that have implemented autonomy programs include: Cincinnati, Milwaukee, Houston, Seattle Oakland, Boston, Chicago, Portland, Minneapolis, St. Paul, Prince Williams County (Virginia), Okaloosa (Florida), and Hawaii. 2 The majority of these districts pursue a district wide approach that increases the autonomy of all schools. Houston, Seattle, Chicago, Portland, Minneapolis, Prince William County, Okaloosa, and Hawaii have all increased autonomy district wide. New York City, however, first increased autonomy for only a few schools under the 1 These figures are obtained from the annual report, Characteristics of the 100 Largest Public Elementary and Second School Districts in the United States generated from the Common Core of Data. See Table A 1 in each year s report. 2 A school district in Edmonton, Canada implemented a school level autonomy in 1976. The district maintained control of standa rd setting, but the individual principals made operating decisions. This reform provided a template for several of the U.S. reforms. (Ouchi, 2006)
153 Autonomy Zone Program and then expanded the number of schools under the Empowerment Schools Organization (ESO). New York City provides intra district variation in school level autonomy. Further, some autonomy reforms did not result in real increases in school level autonomy due to other district struct ures. Through repeated conversations with ESO facilitators I feel reasonably confident that the program implemented in NYCPS increased autonomy at the school level. Finally, focusing the empirical analysis on NYCPS provides a rich set of data. NYCPS m aintains school level data on expenditures, the characteristics of teachers, principals, and students, as well as test score data.
154 APPENDIX B OPTIMAL INPUT ALLOCATIONS Maximizing the objective func tions expressed in Equations 1 7 and 1 12 subject to th e budget constraints, the optimal resource allocations given a particular regime are determined. Under centralized control, the district mandates expenditures such that the budget is exhausted, i.e. no generalized rents beyond the budgetary reward are po ssible. The principal is forced to pursue improvements in average test scores with the latitude only to set efforts. Her optimal effort allocations will be ( B 1) r preference for and the effectiveness of the student type, The per pupil funding and output elasticity of expenditures also increase effort allocations. The effort allocation is decreasing in the size of the school and the size of the targeted populations. Given autonomy, however, the principal has the option to pursue diverse goals. She can devote her resources, both efforts and expenditures, to improving average test scores a nd/or to consuming generalized rents. This latitude results in two sets of optimal allocations. The first case is characterized by principals who pursue improved
155 test scores such that they exhaust their total budget in the process. The second case enta ils the consumption of some generalized rents. 1 Whether an autonomous principal will choose to consume rents depends on the following condition (B 2) If Condition B 2 holds, the principal will find it optimal to spend her total b udget. If not, Condition B 2 yields a few direct implications. First, if a principal does not care about rents then the principal will spend all of the budget an d no generalized rents beyond the budgetary reward will be consumed. Assuming for at least so me schools in the district, Condition B 2 schools, and schools with relatively more type 1 students ar e less likely to consu me rents. Smaller schools are less likely to consume rents because each student has a principals that are more effective, more career oriented, a nd more concerned with school quality. 2 The empirically relevant case is for type 2 students to be a majority. that a school will be induced to consume general ized rents. 1 generat e improvements in test scores. 2 This equates to higher values of and respectively.
156 Evidence on principal sorting within a district shows that principal experience and credentials, two components of effectiveness, are inversely related to school size and the proportion of type 2 students. 3 This suggests that the parameter s that push a principal toward consuming rents are positivel y correlated. As a result, Condition B 2 suggests that given autonomy, the already high performing schools are more likely to exhaust their budgets than the low performing schools. To simplify t he succeeding discussion of results, I define two groups of schools based on Condition B 2 Group 1 is characterized by relatively high effectiveness principals, relatively more high effectiveness students, and relatively small school size. Group 2, ther efore, is composed of schools with low effectiveness principals, more low test effectiveness students, and a large school size. The differences between these groups are summarized in Table 1 1. The optimal expenditure allocations of autonomous principals in group 1 schools will be (B 3) Since the principal exhausts her budget, the strength of the incentives ( and ) does veness k does not impact the allocation. The principal simply divides up the budget in a way that yields the highest test scores. In making this assessment, the principal will examine the opportunity cost of assigning a dollar of expenditur es to a particular type. The comparison of the relative impact across types is evident in the ratio of the student 3 See for example Branch, Hanushek, and Rivkin (2009) and Horng, Kalgorides, and Loeb (2009)
157 students and is concerned with the size of the stu dent group. For group 2 schools, the expenditure allocations differ because the strength of her preferences and the budgetary incentive impact the level of the expenditures. An autonomous principal will assign (B 4 ) Whether the allocation of expenditures to both types is increasing or decreasing in the strength of her taste for rents depends on the relationship between the incentives tied only to test score performance, the incentives t ied to consumption of rents, and the output elasticity of effort If (B 5) then the allocation of expenditures is increasing in the taste for rents. The stronger the incentives that are dependent on increasing average test scores, i.e. the higher and effectiveness will also increase the allocations. The marginal pr oduct of expenditure is higher for more effective principals. These results can be summarized as expenditure size of the school. rt allocations will depend on her expenditure choices. Therefore, I represent her optimal effort allocation as a function of
158 (B 6) The marginal product of effort is increasing in the expenditures she all ocates. Her effort allocation is, also, increasing in her career impact, preference for rents, and her effectiveness. The effectiveness of a student type increases her allocation to that type. Effort allocations are decreasing in the size of the school and the targeted student population. A central objective of this model is to examine how an autonomous school functions in contrast to one under centralized control. Comparing Z, the per student expenditure gi ven centralized control, to Equations B 3 and B 4 the per student expenditure given autonomy, yields a few insights. For the remaining analysis, I address the empirically relevant case 4 where type 1 students are the smaller student group ASSUMPTION 1. PROPOSITION 1. Given aut onomy, a principal will contribute more total inputs to type 1 students relative to type 2 students. Proof. Using Equation B 3 and assumption 1, it follows directly that the level of expenditures devoted to type 1 students is greater in group 1 schools. Comparing to the expressions only differ in the denominators. The denominator of is less than the denominator of i.e. 4 One way to think about these two student groups is in terms of free and reduced lunch status. The average school in New York City is composed of 78.2% free and reduced lunch eligible and 21.8% non eligible.
159 by assumption 1 and the fact that by definiti on. Therefore, the allocation of expenditures accorded to type 1 students exceeds the allocation to type 2 students in group 1, autonomous schools. Similarly, using Equation B 4 we can show that the expenditure allocation apportioned to type 1 students is greater in group 2 schools. For to exceed for these schools the following must hold This expression can be re written as By assumption 1 and the fact that the above condition holds. As a result, the leve l of expenditures received by type 1 students is higher than the level received by type 2 students in group 2, autonomous schools. Further, this means that expenditures allocated to group 1 students exceed those allocated to group 2 students in all autono The total package of per ted effort. Using equation B 6 and the result that in all autonomous schools, it follows that the efforts contributed to type 1 students will be gr eater than the amount contributed to type 2 students. Since both effort and expenditure allocations are greater in all autonomous schools, the total level of inputs is greater for type 1 students.
160 COROLLARY 1. The expected test score of type 1 students w ill exceed those of type 2 student given autonomy. Proof. From Equation 1 1 the expected score of a type i student is represented by Since both effort, expenditures, and endowed ability are greater for type 1 students than type 2 st udents, it follows directly that expected test score of type 1 student will be COROLLARY 2. Autonomous, group 1 schools will contribute more total inputs to type 1 students than they would receive under centralized control. Proof. Group 1 schoo ls spend the total budget. Therefore, it is not possible for the per student allocation of both types to be less than Z. From proposition 1, it follows that >Z. Comparing Equation B 1 to Equation B 6 the expressions differ by the incenti ves and the expenditure allocation. Since the expenditure allocation for ty pe 1 students is greater in Equation B 6 and the incentives are stronger, > Therefore, the smaller, higher ability group receives more inputs. Autonomy pushes principals to assign more funds to the type 1 student group Accountability designs that focus on the lowest 25% or a minimum level of proficiency wi ll of course cause principals to focus allocations to achieve those outcomes. Autonomy generally increases the ability of the principal to respond to a given set of incentives.
161 APPENDIX C PROPENSITY AUTONOMOUS I. Group 1 Group 1 schools do not opti mally choose to consume any rents. As a result, group 1 schools compare three factors across regimes: 1) the value of the test score gains, 2) the optimal effort costs they will incur, and 3) the uncertainty costs they must face. ecision is represented by (C 1) If the value of Equation C 1 is greater than 0 the principal will choose to participate in the autonomy program. The first two terms above capture the value of test score gains under a utonomy and centralized control, respectively. The following pair of terms regime. Finally, the remaining pair of terms represent that the principal will examine t he difference in uncertainty costs. The Value of Test Score Gains Given that the school selects autonomy and chooses not to consume any rents, the value of test score gains is (C 2)
162 Under centralized control, the va lue of test score gains will be (C 3) For group 1 schools, i.e. the schools that find it optimal not to consume rents, Therefore, the test score gains are greater under autonomy. Since the incentives are also greater, the value of the test score gains under autonomy will exceed the value of the test score gains under centralized control. How much greater is the value of the test score gain given autonomy? Value of test score gain given autonomy (C 4) (C 5) Value of test score gain given centralized control (C 6) The difference in test score values can be written as
163 (C 7) Comparative Statics I denote the difference between the value of test score gains as Risk No effect (C 8) Preference for rents, Preference for rents will positively impact the value of the test score difference if the average score achieved under autonomy exceeds the target score. Even though no
164 rents are consumed by group 1 schools, the preference for rents impacts how a prin cipal values the budgetary reward. If the average score a principal earns under autonomy is less than the target score, then their preference for rents will negatively impact the value of the test score across regimes. This effect creates a disincentive for schools have difficulty achieving test score gains to join the program. Some of the other comparative statics make this relationship more clear. For instance, schools with less effective principals will have greater difficulty achieving test score ga ins. Preference for Student Achievement, test scores between autonomy and centralized control. Note that preference for student achievem ent does not interact with the target score The target score is not intrinsically important to a principal. The change in incentive structure is what makes Career Impact, The strength of the career impact will positively impact the value of the test score difference if the average score achieved under autonomy exceeds the target score. If the average score a principal earns under autonomy is less than the target score, then their career impact will negatively impact the value of the test score across regimes. This effect creates a disincentive for schools have difficulty achieving test score gains to join the program. Some of the other comparative statics make this re lationship more clear. For instance, schools with less effective principals will have greater difficulty achieving test score gains. School Size, To analyze the effect of school size and not changes in the ratio of student types, I assume th at the increase in school size creates an equal change in each student
165 population and therefore leaves the ratio of the types unchanged. The mathematical equivalent of these assumptions are (1) and (2) The impact of school siz e on average test scores varies across organization regimes. In group 1, autonomous schools, school size increases the expected school level test score, In centralized schools, however, school size decreases the expected school level test score, See Appendix B for the derivation of these effects. From the school level test score effects, it is directly apparent that Ratio of student types, Cost of Effort Difference The optimal cost of ef fort for group 1 schools that select autonomy is The optimal cost of effort for centralized schools is Comparative Statics Risk aversion, Risk aversion does not impact
166 optimal to allocate more effort to their students because of the student achievement gains they can realize. Preference for rents, A preference for rents will always increase the difference in the cost of effort across regimes. This is because autonomy regime is the only avenue for the principal to purs ue any consumption of rents. Preference for student achievement, are willing to spend in attaining it. Career Impact,
167 The increased im to be greater under autonomy than under centralized control. School Size, N Ratio of student types, n1/n2 and Uncertainty Cost Difference The differen of resources. This difference is denoted Comparative Statics Risk aversion, Since the uncertainty cost is always greater under autonomy, an increase in a princ No effect Preference for student achievement, Career impact,
168 School size, Since Ratio of student types, If then the ratio of the types does not impact the difference in uncertainty costs. If, however, one type has a larger standard deviation in the ir test score than the other type the ratio will matter. If will increase the uncertainty cost. Propensity to Select Autonomy Risk aversion, Risk aversion only affects th e uncertainty cost that the principal bears. Since the uncertainty cost under autonomy always exceeds the cost under centralized control, an increase in risk aversion will make the principal less likely to select autonomy. difference in effort costs. The impact on the value of the test score difference is greater than the change in effort costs. The overall impact of principal makes a principal more apt to select autonomy. This result holds because and Taste for rents, II. Group 2 Group 2 schools will choose to consume rents given autonomy. For group 2 school s, the principal compares four factors across regimes: (1) the level of rents that can be consumed given autonomy, (2) the value of the difference between student achievement, (3) the difference between effort costs, and (4) the difference in uncertainty costs. The only factor that remains unchanged from the analysis of group 1 schools is the uncertainty costs. The value of uncertainty costs cannot be altered by The Level of Generalized Rents
169 The generalized rents consumed by an autonomous principal are Comparative Statics Risk aversion, Risk aversion does not impact the level of rents consumed by the principal. Since the effectiveness of the principal increases the alloca tion of expenditures for both types, k decreases the amount of rents consumed. Taste for rents, The sign of depends on the sign of Preference for student achievement, Career impact, amount of rents consumed. School size, Ratio of student types,
170 where and The Value of Test Score Difference For group 2 schools, the school level test score produced under autonomy does not have to be greater than the test score produced under centralized control. A principal may optimally choose to forgo test score gains and consume more rents The value of the test score difference is Comparative Statics Risk Aversion, and To know the impact of prin student achievement, it is necessary to know how the per student expenditures under autonomy relate to the per student Z spent under centralized control. There are two possible scenarios, given
171 Knowing that this relationship is important, I re write by substituting Taste for rents, centralized control, and If and then If and the result is indeterminate. If and the result is again indeterminate. Both results are indeterminate because there is no a priori relationship between the target score G and other parameters. If and then and Preference for Student Achievement,
172 well as the test score impact, it is necessary to know the relationship between and Career Impact, where If then If, however, then the impact is ind eterminate. School Size, The impact of school size is indeterminate. See Appendix B. Ratio of student types, Cost of Effort The optimal cost of effort for group 2 autonomous schools is The optimal cost of effort for centraliz ed schools is Comparative Statics Risk Aversion, Since the relationship between expenditures under centralized control and expenditure s under autonomy is indeterminate for group 2 schools, the ultimate impact of k cannot be established. Taste for rents,
173 where and er centralized control The impact of taste for rents is wholly determined by its impact under autonomy. If then This means that in this case, the autonomy through a higher effort cost. If then In this case, the effort cost is an incentive to select autonomy. Preference for S tudent Achievement, This relationship is indeterminate due to the relationship between expenditures under autonomy and centralized control. Career Impact, where and Therefore, Th e career impact increases the effort costs a principal will occur. This makes autonomy less desir able when considering only effort costs. School Size,
174 This relationship is indeterminate as a result of the relationship between expenditure s under autonomy and expenditures under centralized control.
175 APPENDIX D TEST SCORE IMPACTS I. Group 1 Preference for Rents, where and As a result, it follows directly that To find the ultimate impact of au tonomy, I compare this impact to centralized control. Since it is not possible to consume rents under centralized control, and School Size, If By differencing and we c an establish the impact of a school becoming autonomous on average test scores. Comparing the two regimes in this way is beneficial because the status quo was centralized control, i.e.
176 Therefore the impact of school size on test scores is positive given a school selects autonomy. Note: There is an upper bound on the size of schools that will choose autonomy without rents. See equation (13) in Section IV. Ratio of student types, To analyze changes in the ratio of type 1 to t ype 2 students, I fix the school size and analyze changes in Expected school level test scores can be re written by substituting for The impact of changes in the ratio of student types can therefore be analyze d by where and
177 The first component of the partial derivative above represents how increases in the ratio of type 1 to type 2 students will change expenditure and effort allocations and through th is ch annel impa ct school stu dent achievement, this decrease causes the marginal benefit of spending money on level test score of the allocation change is negative, i.e.
178 by the second term, school level test score. II. Group 2 Taste for rents, where and By substituting and the effect on test scores can be simplified as The sign of depends on the sign of
179 If then If then To know the test score impact, the change in the autonomous test score is preference for rents does not enter their decision making under centralized control, itive when and negative when The impact on school level test scores follows the same pattern. School Size, The know how school size affects the school level test score impact, I compare to To sign this impact, it is necessary to know the relationship between and Since may be greater than or less than Z, the effect of school size is indeterminate.
180 LIST OF REFERENCES Abdulkadirog lu, Atila, Joshua Angrist, Susan Dynarski, Thomas Kane, and Parag Pathak 2009. "Accountability and Flexibility in Public Schools: Evidence from Boston Charters and Pilots," National Bureau of Economic Research Working Paper No. 15549. Angrist, Joshua Susan Dynarski, Thomas Kane, Parag Pathak, and Christopher Walters 2010, "Who Benefits from KIPP?," National Bureau of Economic Researc h Working Paper No. 15740. Bishop, John and Ludger 2004. a Simple Model of Educational Production. Education Economics 1 2(1): 18 39. Bonin, Holger, Thomas Dohmen, Armin Falk, David Huffman, and Uwe Sunde 2007. Sectional Earnings Risk and Occupational Sorti n g: The Role of Risk Labour Economics 14: 926 937. Booth, Alison, Lina Sosa, and Patrick Nolen 2011 "Gender Differences in Risk Aversion: Do Single Sex Environments Affect Their Development?" Institute for the Study of Labor, IZA Discussion Paper No. 6133 Branch, Gregory F., Eric A. Hanushek, and Stephen G. Rivkin 2009. "Estimating Principal Effectiveness. National Center for Longitudinal Data in E ducation Research Working Paper 32. Braumoeller, Bear F 2004. Hypothesis Testing and M ultiplication Interaction Terms. International Organization 58: 807 820 Brown, Bruce. 19 91. How Gender and Socioeconomic Status Affe ct Reading and Math Achie vement. Economics of Education Review 10(4): 343 357. Clark, Damon, Paco Martorell, and Jonah Rockoff 2009. "School Pr incipals and School Performance. National Center for Longitudi nal Data in Education Research Working Paper 38 Cullen, Julie B. and Michael J. Mazzeo. 2007. "Implicit Performance Awa rds: An Empirical Analysis of the Labor Market f or Public School Administrators. UC San Diego and Northwestern Unpublished Manuscript Currie, J., and Duncan Thomas 2001. "Early Test Scores, Soc ioeconomic Status, and Future Outcomes." Research in Labor Economics 20: 103 132. Eckel, Catherine and Phillip Grossman 2008. Men, Women and Risk Aversion: Experimental Evidence. Chapter 113 1078 1086. Handbook of Experimental Economics Results ed Charl es R. Plott and Vernon L. Smith, Amsterd am: Elsevier Science B.V.
181 Eckel, Catherine, and Philip Grossman 2002 Differences in the Economic Decisions of Men and Women: Experimental Evidence Amsterdam: Elsevier Science B.V. Eckel, Catherine Philip Grossman, Nancy Lutz, and V. Padmanabhan 1998. "Playing It Safe: Gender Differences in Risk Aversion." The Network on Preferences and Norms, John D. and Catherine IZ MacArthur Foundation Working Paper. Eckel, Catherine and Evan Moore 2003. Depa rtment of Economics, Virginia Tech Unpublished Manuscript Evans, William N. and Robert M. Schwab 1995. "Finishing High School and Starting College: Do Cathol ic Schools Make a Difference." Quarterly Journal of Eco nomics, 110 (4): 941 74. Guryan, J. 2000 Does Money Matter? Regression Discontinuity Estimates from Education Finance Reform in Massachusetts. National Bureau of Economic Research Working Paper No. 8269. Hanushek, E ric 2003. "The Failure of Input Based Schooling Policies," Economic Journal 113(485): F64 F98. Harbaugh, Wt. K. Krause, and L. Vesterlund. 2002 "Risk Attitudes of Children and Adults: Choices Over Small and Large Probability Gains and Losses." Experimental Economics 5: 53 84. Holmlund, Helena, Sandra McNally, and Martina Viarengo. 2010. "Does Money Matter for Schools?" Economics of Education Review 29: 1154 1164. Holt, C., and Susan K. Laury 2002. "Risk Aversion and Incentive Effects." American Economic Review, 92(5): 1644 1655. Horng, Eileen L., Daniel Klasik, and Susanna Loeb 2009. "Principal Ti me Use and School Effectiveness." National Center for Longitudinal Dat a in Education Working Paper 34. Hoxby, Caroline M. and Jonah E. Rockoff 2004. harter Schools on Student Achiev Harvard University and Columbia Business School Unpublished Manuscript. Hoxby, Caroline M. 2004. "A Straightforward Comparison of Charter Schools and Regular Public Schools in the United States." Harvard Universi ty mimeo. Hoxby, Caroline M. and Sonali Murarka 2009. "Charter Schools in New Y ork City: Who Enrolls and How They Aff ect Their Students' Achievement. National Bureau of Economic Research Working Papers No. 14852.
182 Jackson, C. Kirabo. 2012. Competition and Teacher Labor Markets: Evidence from Charter School Entry in North Carolina. Journal of Public Economics 96: 431 448. Karnon, J. and Tazio Vanni 2011. "Calibrating Models in Economic Evaluation: A Comparison of Alternative Measu res of Goodness of Fit, Parameter Search Strategies, and Convergence Criteria." Pharmacoeconomics 29(1): 51 62. Loeb, Susanna and Katherine Strunk 2007. "Accountability and Local Control: Response t o Incentives with and without Authority over Res ou rce Generation and Allocation. Education Finance and Policy 2(1): 10 39. Mizala, Alejandra, Pilar Romaguera and Miguel Urquiola 2007 Socioeconomic Status or Noise? Tradeoffs in the Generatio n of School Quality Information. Journal of Develop ment Economics 84(1): 61 75. New York City Department of Education. 2004. City Department of Education Website ( accessed July 15, 2009 ) New York City Department of Education 2005. hools Annual Rep ort 2005. http://schools.nyc.gov/rdonlyres/3062CAC4 0056 4C27 A77D COF5498EC599/0/Annualreport2005.pdf (accessed January 30, 2011). New York City D epartment of Education. 2005. cation Memorandum No. 52, FY 06. http://schools.nyc.gov/offices/d_chanc_oper/budget/dbor/allocationm emo/fy05 06/datafiles/sam52.pdf (accessed January 30, 2011). New York City Department of Education. 2006. School Allocation M emorandum No. 76, FY06. http://schools.nyc.gov/offices/d_chanc_oper/budget/dbor/allocationmemo/fy05 06/datafiles/sam76.pdf ( accessed January 31, 2011) New York City Department of Education. 2009. owerment Schools Brochure. http://schools.nyc.gov/NR/rdonlyres/6 AFCD437 0386 4907 AE52 FDE6E8CB2E48/57528/ESOBrochure2009.pdf (accessed October 12, 2010). New York City Department of Education. 2010. http://schools.nyc.gov/NR/rdonlyres/44CC868E 9EAE 46FC AA51 62BE7813A2DC/30315/EmpowermentSchoolsbrochure.pdf (accessed January 31, 2012). Niederle, M., and L. Vesterlund. 2007 Do Women Shy Away From Competition? Do Men Compete Too Much?" Quarterly Journal of Economics 122 (3): 1067 1101. Niederle, M., and A. H. Yestrumskas 2008. Gender Differences in Seeking Challenges: The Role of Institutions. National Bureau of Economic Research Working Paper No. 13922
183 Ouchi, William G 2006. "Power to the Principals: Decentralization in Three Large School Districts. Organization Science 17 (2): 298 307. Rouse, Cecilia. 1998. "Private School Vouchers and Student Achievement: A n Evaluation of the Milwaukee Pare ntal Choice Program." Quarterly Journal of Economics 113(2): 553 602. Schubert, R., M. Brown, M. Gysler, and H.W. Brachinger. 1999 "Financial Decision Making: Are Women Really More Risk Averse?" American Economic Review 89(2): 381 385 Springer, M atthew G. and Marcus A. Winters 2009. "New York City's S chool Wide Bonus Pay Program: Early E vidence from a Randomized Trial. National Center on P erformance Incentives Working Paper No. 2009 02. Stiefel, Leanna, Amy E. Schwartz, Carole Portas, and Dae Yeop Kim 2004. Journal of Education Finance 28: 403 24. Vanni, T., Jonathan Karnon, Jason Madan, Richard White, W. John Edmunds, An na M. Foss, and Rosa Legood. 2011. "Calibrating Models in Economics Evaluation: A Seven Step Approach." Pharmacoeconomics 29(1): 35 49. Wooldridge, Jefferey M 2001 Econometric Analysis of Cross Section and Panel Data Cambridge: The MIT Press.
184 BIOGRAPHICAL SKETCH Jessica Stephanie Haynes was born in 1983 in Clearwater, Florida. In December 2005 she graduated cum laude from the University of Florida with a Bachelor o f Arts degree in economics and women's s tudies. In August 2007 she started a do ctorate in economics at the University of Florida. She received her Ph.D. from the University of Florida spring 2013. Her research interests include economics of education, law and economics, and other public economics topics In August 2013, she joined the Department of Economics at Auburn University.