<%BANNER%>

The Relationship of Three State Level Factors to Academic Achievement in Math and Reading after Controlling for Demograp...

Permanent Link: http://ufdc.ufl.edu/UFE0041688/00001

Material Information

Title: The Relationship of Three State Level Factors to Academic Achievement in Math and Reading after Controlling for Demographic Variables in Aggregated Data
Physical Description: 1 online resource (33 p.)
Language: english
Creator: Porter, Phillip
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: achievement, hqt, naep, nclb, nrt, retention
Human Development and Organizational Studies in Education -- Dissertations, Academic -- UF
Genre: Research and Evaluation Methodology thesis, M.A.E.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: States have approached education through many different policies. NCLB increases the level of reporting and accountability in every state, but still allows the states some flexibility in how they meet the goals required by NCLB. Comparing achievement across the states may provide some insight as to the efficacy of these different state factors that may affect achievement. The state tests required by NCLB are not useful for comparing achievement from state to state, but the NAEP, which NCLB requires every state to participate in, does provide achievement information which may be used to compare the states. State factors of interest are the use of norm-referenced tests, retention policies, and the percentage of highly qualified teachers in each state. It was found that the use of norm-referenced tests was associated with a lower level of math achievement. Retention policies had little effect on achievement, although there was a significant relationship between the percentage of students achieving at proficient or above and allowing local school districts to set retention policies. The percentage of highly qualified teachers had a strong relationship with improved academic achievement in both math and reading.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Phillip Porter.
Thesis: Thesis (M.A.E.)--University of Florida, 2010.
Local: Adviser: Miller, M David.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0041688:00001

Permanent Link: http://ufdc.ufl.edu/UFE0041688/00001

Material Information

Title: The Relationship of Three State Level Factors to Academic Achievement in Math and Reading after Controlling for Demographic Variables in Aggregated Data
Physical Description: 1 online resource (33 p.)
Language: english
Creator: Porter, Phillip
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: achievement, hqt, naep, nclb, nrt, retention
Human Development and Organizational Studies in Education -- Dissertations, Academic -- UF
Genre: Research and Evaluation Methodology thesis, M.A.E.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: States have approached education through many different policies. NCLB increases the level of reporting and accountability in every state, but still allows the states some flexibility in how they meet the goals required by NCLB. Comparing achievement across the states may provide some insight as to the efficacy of these different state factors that may affect achievement. The state tests required by NCLB are not useful for comparing achievement from state to state, but the NAEP, which NCLB requires every state to participate in, does provide achievement information which may be used to compare the states. State factors of interest are the use of norm-referenced tests, retention policies, and the percentage of highly qualified teachers in each state. It was found that the use of norm-referenced tests was associated with a lower level of math achievement. Retention policies had little effect on achievement, although there was a significant relationship between the percentage of students achieving at proficient or above and allowing local school districts to set retention policies. The percentage of highly qualified teachers had a strong relationship with improved academic achievement in both math and reading.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Phillip Porter.
Thesis: Thesis (M.A.E.)--University of Florida, 2010.
Local: Adviser: Miller, M David.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0041688:00001


This item has the following downloads:


Full Text

PAGE 1

1 THE RELATIONSHIP OF THREE STATE LEVEL FACTORS TO ACADEMIC ACHIEVEMENT IN MATH AND READING AFTER CONTROLLING FOR DEMOGRAPHIC VARIABLES IN AGGREGATED DATA By PHILLIP CARROLL PORTER A THESIS PRESENTED TO THE GRADUAT E SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS IN EDUCATION UNIVERSITY OF FLORIDA 2010

PAGE 2

2 2010 Phillip Carroll Porter

PAGE 3

3 To Geoffrey thanks for letting me use the computer

PAGE 4

4 AC KNOWLEDGMENTS I would like to thank my committee members, Dr. M. David Miller and Dr. David Therriault for their patience, flexibility, and guidance. last forever. I would also like to thank my parents, Sanford and Colleen Porter, for giving me a love of reading and learning, and my wife, Eliza, and children, Geoffrey and Margot, for giving me motivation and support.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 6 ABSTRACT ................................ ................................ ................................ ..................... 7 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ ...... 9 2 REVIEW OF LITERATURE ................................ ................................ .................... 13 Comparing Student Achievement ................................ ................................ ........... 13 Factors That Affect Student Achievement ................................ ............................... 14 3 METHODS ................................ ................................ ................................ .............. 17 Data ................................ ................................ ................................ ........................ 17 Sample ................................ ................................ ................................ .................... 18 Analysis Approach ................................ ................................ ................................ .. 19 4 RESULTS ................................ ................................ ................................ ............... 21 Demograph ics ................................ ................................ ................................ ......... 21 Norm referenced Tests ................................ ................................ ........................... 21 Retention Policies ................................ ................................ ................................ ... 22 Highly Qua lified Teachers ................................ ................................ ....................... 22 Combined ................................ ................................ ................................ ............... 22 5 DISCUSSION ................................ ................................ ................................ ......... 27 LIST OF REFERENCE S ................................ ................................ ............................... 30 BIOGRAPHICAL SKETCH ................................ ................................ ............................ 33

PAGE 6

6 LIST OF TABLES Table page 3 1 Unweighted Descriptive St atistics for Reading and Math Samples in 50 States and the District of Columbia. ................................ ................................ ... 20 3 2 Descriptive Statistics for NRT and Retention. ................................ ..................... 20 4 1 Regression Results of Gender, Poverty, Race, ELL, and Disability On Math Achievement. ................................ ................................ ................................ ...... 23 4 2 Regression Results of Gender, Poverty, Race, ELL, and Disability On Reading Achieve ment. ................................ ................................ ....................... 23 4 3 Regression Results of Poverty, and Race On Math Achievement. ..................... 24 4 4 Regression Results of Poverty, and Race On Re ading Achievement. ............... 24 4 5 Regression Results of Poverty, Race, and NRT On Math Achievement. ........... 24 4 6 Regression Results of Pover ty, Race, and NRT On Reading Achievement. ...... 24 4 7 Regression Results of Poverty, Race, and Retention On Math Achievement. ... 25 4 8 Regression Results of Poverty, Race, and Retention On Reading Achievement. ................................ ................................ ................................ ...... 25 4 9 Regression Results of Poverty, Race, and HQT On Math Achievement. ........... 25 4 10 Regression Results of Poverty, Race, and HQT On Reading Achievement. ...... 26 4 11 Regression Results of Poverty, Race, NRT, Retention, and HQT On Math Achievement. ................................ ................................ ................................ ...... 26 4 12 Regression Results of Poverty, Race, NRT, Retention, and HQT On Reading Achievement. ................................ ................................ ................................ ...... 26

PAGE 7

7 Abstract of Thesis Presented to the Gr aduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Master of Arts in Education THE RELATIONSHIP OF THREE STATE LEVEL FACTORS TO ACADEMIC ACHIEVEMENT IN MATH AND READING AFTER CONTROLLING FOR DEMOGRAPHIC VARIABLES IN AGGREGATED DATA By Phillip Carroll Porter May 2010 Chair: David Miller Major: Research and Evaluation Methodology States have approached education through many different policies. NCLB increases the level of reporting and accountability in every state, but still allows the states some flexibility in how they meet the goals required by NCLB. Comparing achievement across the states may provide some insight as to the efficacy of these different stat e factors that may affect achievement. The state tests required by NCLB are not useful for comparing achievement from state to state, but t he NAEP, which NCLB requires every state to participate in, does provide achievement information which may be used to compare the states. State factors of interest are the use of norm refer enced tests, retention policies, and the percentage of highly qualified teachers in each state. It was found that the use of norm refer enced tests was associated w ith a lower level of math achievement. Retention policies had little effect on achievement, although there was a significant relationship between the percentage of students achieving at proficient or above and allowing local school districts to set retent ion policies. The

PAGE 8

8 percentage of highly qualified teachers had a strong relationship with improved academic achievement in both math and reading.

PAGE 9

9 CHAPTER 1 INTRODUCTION In 2001 the enactment of the No Child Left Behind Act of 2001 (NCLB) brought about s ignificant changes in American education. NCLB was a set of amendments to the Elementary and Secondary Education Act of 1965 with the goal ensuring that all students had a high quality education, regardless of minority status or any disadvantages the stud ent may be working under (NCLB 2002) There are four main pillars to NCLB: accountability, freedom, proven methods, and more choices for parents (U S DOE 2010 ). To provide stronger accountability, NCLB required more testing than most states were p reviously administering, and increased reporting of those test results More freedom for states and communities was provided by leaving many of the implementation details up to the various states. The individual states also have more freedom in how federal money is used in their schools. NCLB emphasizes rigorous scientific research for determining the value of educational methods. Federal money is provided for this purpose and the U.S. Department of Education provides information about this research to the states. Parents have more choice under NCLB If their children are attending schools which are not living up to the academic or safety requirements of NCLB they may transfer their children to another school in the school district, with the school district paying any transportation costs. Although, in pra ctice, very few parents have ex ercised this option (U.S. DOE 20 07a). NCLB also provides for extra services, again paid for by the school district, for students at underperforming schools. NCLB made many state level education policies mandatory ; state assessments, highly qualified teachers, reporting of yearly progress, and financial consequences for

PAGE 10

10 failure to meet these goals (NCLB 2002 ). Prior to NCLB differences in these policies from state to state were fruitful grounds for research into the relationship between state policies and student achievement (Goertz & Duffy, 2001 ; Carnoy & Loeb 2002; Braun, 2004; Marchant Paulson & Shunk, 2006 ). With these policies implemented in every state they become less useful predictors of student achievement differences between the states. Even with the flexibility provided by NCLB there are some policy differences bet ween the states that could allow for an explanation of some of the differences in student achievement across states. NCLB allows states to set their own academic standards and develop their own tests of those standards (NCLB 2002). This paper will evalu ate three of these policies which may affect student achievement, the use of norm refer enced testing, the use of retention policies, and the percentage of highly qualified teachers. The first area of interest is the use of norm referenced versus crit erion referenced testing. Norm referenced tests compare student scores to some normative group to provide information on how the students did relative to other students. Criterion referenced tests compare student scores to some objective criterion. Unde r NCLB states may determine these criteria, and many states have chosen to set the bar quite low in relation to NAEP and other indicators of student achievment (Bandeira, Blankenship & McLaughlin, 2009) The use of norm referenced tests may reveal these low criteria when students in states with low criteria are compared to other groups. The second area of interest is retention policies. NCLB has no requirements related to student retention, permitting each state to set their own policy, or to have no p olicy at all, governing the retention of students from year to year.

PAGE 11

11 The third area of interest is the percentage of highly qualified teachers. Under degree, a license or certificate in the subject taught, and passed a state certification test in the subject taught ( NCLB, 2002 ) While NCLB does require every elementary teacher and every secondary teacher of core academic classes to be highly qualified, not every state ha s reached this goal. This variation between states on the percentage of highly qualified teachers allows for comparisons of student achievement. One of the benefits of NCLB for researchers is the large amount of data now available from each state. Each state reports data on participation in state assessments, student achievement, and highly qualified teachers. When comparing student achievement between states it would seem that this stockpile of information would be invaluable. One of the drawbacks to the level of flexibility provided by NCLB is that each state is able to set their own standards for their own assessments. Comparing student achievement based on these state assessments would be like, as 2007). Fortunately NCLB provides a solution to the problem that it has given us. One of the requirements of NCLB is that every state participate in the National Assessment of Educational Progress (NAEP). Since 1969 NAEP has been assessing students in America on mathematics, reading, science, writing, and a variety other subjects. The main NAEP tests for mathematics and reading, the tests most comparable to those required by NCLB, are administered every four years to 4 th 8 th and 12 th grade students. NAEP testing does not attempt to reach every student, but instead attempts to obtain a representative

PAGE 12

12 sample for each state (NCES 20 10 ). The results are based on samples from each state rather than the entire population of each state but do give us a consistent measuring stick from state to state. This study will examine the relationship between state policies and student achievement. Three state level policies will be examined: the use of norm referenced testing, retention policies, and the percentage of highly qualified teachers. Student achievement will be measured using mathematics and reading NAEP scores.

PAGE 13

13 CHAPTER 2 REVIEW OF L ITERATURE Comparing Student Achievement There is a large amount of variation from state to state in the percentage of students who achieve proficien t or above on the NCLB mandated state P art of this is due to differences i n the actual achievement of the students taking the tests. However, the differences in content standards, test rigor, and cutoff scores make the differences in student achievement practically impossible to isolate (Linn Baker & Betebenner, 2002). Despit e the lack of consistency from state to state, every year media outlets run stories trying to make state comparisons based on tests which are not comparable (Ho 2007). This is no t to say that states may not be properly compared w ith each other based on st udent achievement, only that the state tests are not the proper measure of achievement for making these comparisons. The NAEP provides an assessment which is consistent from state to state and allows for such co m p arisons (NCES 2010 ). The problems with u sing the state tests for comparisons are evident when student achievement as measured by the state tests is compared to student achievement as measured by the NAEP (Ho 2007). This disagreement between the state tests and th e NAEP is not necessarily indic a t i ve of faulty assessment by either the states or by NAEP. Each state is developing their own assessments for their own purposes, based on different content standards and educational goals in additio n to those required by NCLB (L ee, 2007 ). The NAEP is not specifically aligned with any expected.

PAGE 14

14 Several studies have used NAEP scores to compare states with different accountability systems (Carnoy & Loeb, 2002; Goe r tz & Duffy, 2001 ) and to examine the effects of high stakes testing (Marchant, Paulson & Shunk, 2006; Braun 2004 ). Carnoy et al. and Goetz et al. found that stronger accountability systems were associated with higher student achieveme nt. Braun found that high stakes testing was linked to higher achievement. Marchant looked closer and found c ontent area differences in the relationship between high stakes testing and achievement, with a stronger relationship for math achiev e men t than for reading achievement. Establishing proficiency levels is left up to the individual stat es by NCLB and there are nearly as many different ideas on what proficiency looks like as there are states The NAEP level of basic is more similar to the level of proficient used on the various state tests (Mosquin & Chromy, 2004) School policies have been found to have stronger effects for students scoring basic or above than for students scoring proficient or above on the NAEP ( Marchant, Paulson & Shunk, 2006 ). Factors That Affect Student Achievement Demographic differences p lay a large role in explaining differences in student achievement. Any study of student achievement will need to control for dem o graphic variables (Konstantopoulos & Hedges 2008; Marchant, Paulson & Shunk, 2006 ). The most widely used demographic variab les are race and socio economic status (SES) H owever other important demographic variables include gender (Lubienski 2001 ), English Language Learner status (Paulson & Marchant, 2009 ), and disability status ( CEP, 2009 ).

PAGE 15

15 There are many factors which inf luence student achievement. For the purposes of this study three state level factors were selected; the use of norm refer enced testing, the use of retention policies, and the percentage of highly qualified teachers. The first of these state level factors is norm refer enced testing. T he virtues of norm refer enced testing and criterion refer enced testing are a subject of considerable debate Norm refer enced testing has many benefits, such as increased credibility, trackin g of student growth, and being comparable across states (Schafer 2003 ). Some researchers would argue that norm refer standards (Trousdale Penuel & Khanna, 1999). Angoff (1974) implies that the argument standards set for criterion refer enced tests are the norms that the test makers hop e to achieve. State level retention policies are the second factor of int erest for this paper. "Given data from over 85 years of research consistently demonstrating either ineffectiveness or negative effects associated with the practice of grade retention, one has to wonder how this practice has persisted." (Silberglitt Apple ton, Burns & Jimerson, 2006). The practice still persists, so researchers still continue to examine it. Retention has been shown to have negative effects while the student is still in elementary school (Hong & Yu, 2007 ) and even through adolescence (Jime rson & Ferguson, 2007 ). The final state level factor of interest for this paper is the percentage of highly qualified teachers in each state. Little research has been done on the effectiveness of is a large body of work

PAGE 16

16 showing that increasing the quality of teachers is associated with increases in student achievement (Boyd et al. 2007; Wayne & Youngs, 2003 ).

PAGE 17

17 CHAPTER 3 METHODS Data Demographics were collected from NCES reports on NAEP reading and mathematics testing of 4 th grade students in 2005. The demographics included in this study were race, gender, poverty, English Language Learner (ELL) status, and disability status (including 504). The variable of race was defined as the percentage o f test takers who were white. Gender was defined as the percentage of test takers who were male. Poverty was defined as the percentage of test takers who were eligible for the National School Lunch Program. ELL status was the percentage of test takers w ho were classified as English Language Learners by their schools. Disability status is the percentage of students who were on an Individualized Education Plan or 504 plan. Student test scores were collected using NAEP Data Explorer reports on NAEP readin g and mathematics testing of 4 th grade students in 2005 (NCES, 2010b) The variables for student scores represent the percentage of test takers scoring at or above proficient in each state, and the percentage of test takers scoring at or above basic in ea ch state. The state policies evaluated in this paper were the use of norm referenced testing, retention policies, and the use of highly qualified teachers Data on the use of norm referenced testing were provided by the NCES Profile of State Assessment St andards and was reported dichotomously (NCES, 2007) Data on retention policies were provided in the Education Commission of the States report, Student Promotion/Retention Policies ( Zinth 2005) States were divided into three groups, those which had sta te level retention policies, those which had local retention policies,

PAGE 18

18 and those which had no required retention policies. Data on the percentage of highly qualified teachers in each state came from the 2007 State and Local Implementation of the No Child Left Behind Act, Volume II Teacher Quality under NCLB: Interim Report and was recorded as a continuous variable (U.S. DOE 2007 b ) Sample Table 3 1 contains means and standard deviations for race, gender, poverty, ELL status and disability status. The mean is the unweighted average percentage of students with that characteristic across the 50 states and the District of Columbia. The test takers were evenly split by gender, and mostly white, with a large minority who were eligible for subsidized lunch, and much smaller minorities who were ELL, or classified as having a disability by their school. Table 3 2 contains descrip tive statistics for the use of norm refer enced tests (NRT) and retention policies The number of states is reported for the use of norm refer enced tests and retention policies. For the percentage of highly qualified teachers, the average percentage of highly qualified teachers across each of the 50 states and the District of Columbia is 89.0% (SD=12.4) excluding the state of Delaware which did not report the percentage of highly qualified teachers in 2005. Norm referenced tests were used as a part of their state testing programs in 20 states. The three app roaches to retention policies were fairly evenly split. A state level policy was in place in 16 states, 18 states required a retention policy, but allowed local of s tate level retention policy. The average percentage of highly qualified teachers in each state was 89%, with Alaska as the lowest having 34% of its teachers being highly

PAGE 19

19 qualified, and only Wisconsin meeting the goal of having 100% of teachers as highly q ualified. Analysis Approach Achievement data for 4 th grade students from the 2005 administration of the reading and math sections of the NAEP were used to examine three state level factors thought to affect student achievement. After controlling for demo graphic differences, the use of norm refer enced testing, the use of retention policies, and the percentage of highly qualified teachers were evaluated individually and then together using regression Race and socioeconomic status are the most commonly used predictors of student achievement, but data was also available about the gender, ELL status and disability status of the test takers. The literature review suggested that these could also be significant demographic variables so it was deter mined to evaluate all five of these variables and select those which were significantly related to reading and math achievement in our sample as control variables for the later analyses. Once demographic variables were selected the use of norm referenced testing, the use of retention policies, and the percentage of highly qualified teachers were each evaluated individually while controlling for demographics in a regression model where race and poverty were included along with the appropriate state level f actor The literature review suggested that when using NAEP scores to compare states that using the percentage of students who achieved basic or above was more appropriate than the proficient or above achievement level (Mosquin & Chromy, 2004) S ome s chool policies had stronger relationships depending on which level of achievement was being evaluated (Marchant, Paulson & Shunk, 2006 ) For these reasons both basic or above and proficient or above were included as achievement outcomes.

PAGE 20

20 A final, combine d, regression model was analyzed which included race, poverty status, and the three predictors; norm referenced testing, the use of retention policies, and the percentage of highly qualified teachers. Table 3 1. Unweighted Descriptive Statistics fo r Reading and Math Samples in 50 States and the District of Columbia. Reading Math N Mean SD Mean SD Male 51 50.5 0.9 50.9 0.9 Poverty 51 42.6 11.7 43.3 11.7 White 51 65.3 19.8 64.8 19.9 ELL 51 5.6 5.8 6.3 6.3 Disability 51 10.1 2.8 12.4 2.7 Ta ble 3 2. Descriptive Statistics for NRT and Retention. N n % NRT 51 20 39.2% Retention 51 State 16 31.4% Local 18 35.3% None 17 33.3%

PAGE 21

21 CHAPTER 4 RESULTS Demographics Table s 4 1 and 4 2 contain the parameter estimates, standard errors, p values, and R squared from the complete demographic analysis. The demographic analysis indicated that race and poverty status were the only two significant demographic predictors of reading scores at either level, and for math scores at the basic or above level. Poverty s tatus was the only significant demographic predictor of math scores at the proficient or above level. It was decided to use race and poverty status as demographic variables for all additional models. To provide a valid R squared for use when compari ng later models, another analysis was conducted using only poverty status and race. Table s 4 3 and 4 4 contain the parameter estimates, standard deviations p values, and R square d when only an a lyzing lunch status and race. Norm refer enced Tests Table s 4 5 and 4 6 contain the parameter estimates, standard errors, p values, and R squared from the analysis of the use of norm refer enced tests (NRT) in each state. Controlling for race and lunch status, the use of norm referenced tests was a significant predictor of math scores for both proficient or above and basic or above. It was not found to be a significant predictor of reading scores at either level. Th e use of NRT tests was associated with a three point decrease in the percentage of students reaching proficient or above and basic or above in math.

PAGE 22

22 Retention Policies Table s 4 7 and 4 8 cont ain the parameter estimates, standard errors, p values, and R squared from the analysis of the use of retention policies in each state. Controlling for race and lunch status, the use of retention polici es was not a significant predictor of math scores at either level. Having a state required retention, but allowing local controls over the details of that policy was found to be a significant predictor of proficient or above reading scores (p=.0 124 ), but not for basic or above reading scores. States requiring a retention policy, but allowing local control over the details of that policy was associated with a 2.50 point increase in the percentage of test takers reaching proficient or above in readi ng Highly Qualified Teachers Table s 4 9 and 4 10 contain the parameter estimates, stand ard errors p values, and R squared from the analysis of the percentage of highly qualified teachers (HQT) in each state. Controlling for race and lunch status, the percentage of highly qualified teachers was a significant predictor of math and reading scores at all levels. Each percentage point increase in the percentage of highly qualified teachers was associated with a 1 .19 increase in the percentage of test takers reaching proficient or above or basic or above in either test subject. Combined Table s 4 11 and 4 12 contain the parameter estimates, standard deviations p values, and R squared from the combined analysis of the use of norm refer ence tests (NRT) the use of retention policies, and the percentage of highly qualified teachers (HQT) in each state.

PAGE 23

23 For the combined model, the relationships found in the individual models were still observed with the exception of the significant relati onship between retention policies being set at the local level and the percentage of test taker achieving proficient or above on the NAEP reading test. Table 4 1 Regression Results o f Gender, Poverty, Race, E LL a nd Disability On Math Achievement. Table 4 2. Regression Results o f Gender, Poverty, Race, E LL a nd Disability On Reading Achievement. Basic+ (R 2 =.60) Proficient+ ( R 2 =.70) B SD p B SD p Male 0.39 0.84 0.6457 0.62 0.62 0.4380 Poverty 0.35 0.08 <.0001 0.52 0.08 <.0001 White 0.15 0.06 0.0092 0.07 0.05 0.2195 ELL 0.02 0.13 0.8835 0.02 0.12 0.8585 Disability 0.01 0.28 0.9663 0.08 0.27 0.7604 Basic+ (R 2 =.75) Proficient+ (R 2 =.80) B SD p B SD p Male 0.61 0.70 0.3885 0.59 0.48 0.2266 Poverty 0.42 0.07 <.0001 0.40 0.05 <.0001 White 0.16 0.05 0.0009 0.07 0.03 0.0211 ELL 0.09 0.12 0.4314 0.07 0.08 0.3699 Disability 0.07 0.22 0.7605 0.01 0.15 0.9798

PAGE 24

24 Table 4 3 Reg ression Results o f Poverty, a nd Race On Math Achievement. Table 4 4. Regression Results o f Poverty, a n d Race On Reading Achievement. Table 4 5. Regression Results o f Poverty, Race, a nd N RT On Math Achievemen t. Table 4 6. Regression Results o f Poverty, Race, a nd N RT O n Reading Achievement. Basic+ (R 2 =.60) Proficient+ ( R 2 =.70) B SE p B SE p Poverty 0.35 0.08 <.0001 0.52 0.08 <.0001 White 0.15 0.06 0.0092 0.07 0.05 0.2195 Basic+ (R 2 =.75) Proficient+ (R 2 =.80) B SE p B SE p Poverty 0.42 0.07 <.0001 0.40 0.05 <.0001 White 0.16 0.05 0.0009 0.07 0.03 0.0211 Basic+ (R 2 =.67) Proficient+ ( R 2 =.73) B SE p B SE p Poverty 0.37 0.07 <.0001 0.55 0.07 <.0001 White 0.12 0.04 <.0001 0.02 0.04 0.6290 NRT 3.03 1.40 0.0118 3.20 1.31 0.0188 Basic+ (R 2 =.78) Proficient+ (R 2 =.81) B SE p B SE p Poverty 0.41 0.06 <.0001 0.39 0.04 <.0001 White 0.15 0.04 0.0003 0.07 0.03 0.0133 NRT 1.94 1.22 0.1166 1.51 0.83 0.0747

PAGE 25

25 Table 4 7. Regression Results o f Poverty, R ace, a nd Retention On Math Achievement. Table 4 8. Regression Results o f Poverty, Race, a nd Retention On Reading Achievement. Table 4 9. Regression Results o f Poverty, Race, a nd H QT On Math Achievement. Basic+ (R 2 =.64) Proficient+ ( R 2 =.70) B SE p B SE p Poverty 0.36 0.08 <.0001 0.54 0.08 <.0001 White 0.15 0.05 0.0036 0.05 0.04 0.2633 Local 0.04 1.77 0.9811 0.42 1.68 0.8060 State 0.42 1.99 0.8317 0.54 1.8 9 0.7746 Basic+ (R 2 =.78) Proficient+ (R 2 =.83) B SE p B SE p Poverty 0.41 0.07 <.0001 0.39 0.04 <.0001 White 0.18 0.04 <.0001 0.09 0.03 0.0011 Local 2. 76 1.44 0.0619 2.50 0.96 0.0124 State 1.34 1.61 0.4076 1.07 1.07 0.3223 Basic+ (R 2 =.71) Proficient+ ( R 2 =.74) B SE p B SE p Poverty 0.35 0.07 <.0001 0.55 0.07 <.0 001 White 0.10 0.04 0.0354 0.01 0.04 0.9224 HQT 0.19 0.06 0.0020 0.14 0.06 0.0135

PAGE 26

26 Table 4 10. Regression Results o f Poverty, Race, a nd H QT On Reading Achievement. Table 4 11. Regression Results o f Poverty, Race, N RT Retention, a nd H QT On Math Achievement. Table 4 12. Regression Results of Poverty, Race, NRT, Retention, and HQT On Reading Achievement. Basic+ (R 2 =.84) Proficient+ (R 2 =.85) B SE p B SE p Poverty 0.39 0.06 <.00 01 0.39 0.04 <.0001 White 0.13 0.04 0.0007 0.05 0.03 0.0720 HQT 0.18 0.04 0.0003 0.12 0.03 0.0007 Basic+ (R 2 =.75) Proficient+ ( R 2 =.77) B SE p B SE p Poverty 0.36 0.07 <.0001 0.56 0.07 <.0001 White 0.06 0.05 0.1890 0.02 0.05 0.6218 NRT 2.93 1.32 0.0319 2.88 1.33 0.0355 Local 2.22 1.62 0.1777 1.37 1.62 0.4038 State 0.93 1.77 0.6042 0.20 1.78 0.9129 HQT 0.20 0.06 0.0017 0.15 0.06 0.0167 Basic+ (R 2 =.85) Proficient+ (R 2 =.86) B SE p B SE p Poverty 0.39 0.06 <.0001 0.39 0.04 <.0001 White 0.13 0.04 0.0016 0.05 0.03 0.0595 NRT 1.85 1.06 0.0887 1.18 0.76 0.1313 Local 1.07 1.29 0.3296 1.47 0.93 0.1223 State 0.15 1.41 0.4076 0.42 1.02 0.6788 HQT 0.15 0.05 0.0028 0.09 0.03 0.0084

PAGE 27

27 CHAPTER 5 DISCUSSION These results are generally consistent with the existing literature on these three state level factors and student achievement. The relationship between highly qualified teachers and student achievement found in this study match the relationships found in other studies. While the results of the analysis of retention policies initially seems to be different from previous results, upon closer examination the results of this study are consistent with previous research. The relationship between the use of norm refer enced testing and student achievement found in this study is unusual. It is unexpected that states using norm refer enced tests should have lower achievement scores than states not using norm refer enced tests. The literature makes no predictions about an effect on student achievement, but does indicate that norm refer enced tests and criterion refercriterion refer enced tests often measure different things, with criterion refer enced tests being more specifically directed to the goals and standards of the state using them (Trousdale Penuel & Khanna, 199 9) The theoretical underpinning of the decision to examine this variable is that states using norm refer enced tests would have an external evaluation of their standards and would have higher state standards resulting in higher achievement (Scha fer 2003) Some states have moved to remove norm refer enced testing from their state testing programs for financial reasons without providing any information on any academic effects this may t change and math scores will rise slightly, providing an academic reason for dropping norm refer enced tests from state testing plans (FL DOE 2010) A particularly interesting tidbit here is that there is a larger effect found at the profici ent or above than at basic or above, opposite the pattern

PAGE 28

28 found with the other variables Generally, school factors have less influence on higher achieving students (Marchant, Paulson & Shunk, 2006 ) Those higher achieving students have factors outside o f school which lead to their academic success. When examined in isolation letting local school districts decide on retention policies is associated with greater reading achievement at the proficient or above level, with reading achievement at the basic or above level approaching significance (p=0.06). The literature shows that the retention of students results in lower achievement (Silberglitt Appleton, Burns & Jimerson, 2006 ; Hong & Yu, 2007; Jimerson & Ferguson, 2007 ) To better understand this apparent conflict between the results of this study and previous studies, the various state and local retention policies were more closely reviewed. T he state policies are more dependent on the results of state te sts and provide fewer opportunities for local school officials to advance students. Local retention policies are less reliant on test scores and give local school officials more leeway in promoting students. Assuming that the state level retention polici es will result in more students being retained than the local retention policies, these results are not unexpected. Previous studies on retention have focused on individual instances of retention, not policies leading to retention. This previous research suggest s that no system of retention is bett er than any system of retention. These results indicate that p erhaps local control with more flexibility, over rete ntion will yield better results than having no retention at all. The percentage of highly qua lified teachers was related to significant gains in student achievement across the board as the literature indicates (Boyd et al. 2007; Wayne & Youngs, 2003 ) Additionally for proficient or above, including the percentage

PAGE 29

29 of highly qualifi ed teachers in the model removed the significant effects of race on achievement, suggesting that increasing teacher quality is an effective way of reducing the achievement gap between white and minority students, at least for higher achieving students. When all of these factors are included in a single model the relationship between local retention policies and reading achievement at the proficient or above level loses significance. Most of the difference in R squared between the revised demographic mod el and the full model comes from the percentage of highly qualified teachers. For a state looking at policy changes intended to improve student achievement in reading and math, the percentage of highly qualified teachers is clearly the most important fact or to improve. Dropping norm refer enced testing as a part of the state testing program may be helpful to math achievement, although these differences are slight, and may be different based on the ps y chometric characteristics of the tests being r eplaced and the replacement tests Improving the achievement of higher performing students is more difficult, mostly because a larger proportion of their variance is due to factors outside of school.

PAGE 30

30 LIST OF REFERENCES Angoff W. H. (1974). Criterion ref erencing, norm referencing and the SAT College Board Review, 92 2 5 Bandeira de Mello, V., Blankenship C., and McLaughlin, D.H. (2009). Mapping State Proficiency Standards Onto NAEP Scales: 2005 2007 (NCES 2010 456). Washington, DC : U.S. Department of Educatio n. Boyd, D, Lankford, H, Loeb, S, Rockoff, J, & Wyckoff, J. (2007). The narrowing gap in New York City teacher qualifications and its implications for student achievement in high poverty schools Washington D.C.:National Center for Analysis of L ongitundinal Data in Education Research. Braun, H.( 2004). Reconsidering the impact of high stakes testing Education Policy Analysis Archives 12 ( 1). Carnoy, M & Loeb, S. (2002). Does external accountability affect student outcomes? A cross state ana lysis. Educational Evaluation and Policy Analysis 24 305 331. Center on Education Policy [CEP] (2009 ). Has progress been made in raising achievement for students with disabilities ? State Test Score Trends T h r ough 2007 2008, Part 4. Washington D.C.: Aut hor Florida Department of Education [FL DOE ]. (2010). Frequently Asked Questions R etrieved on March 2010. Available online at http://www.fldoe.org/faq/default.asp?ALL=Y&Dept=179&Cat=95 Goertz, M & Duffy, M. (2001). Assessment and accountability in t he 50 states. Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania. Ho, A. D. (2007). Discrepancies between score trends from NAEP and state tests: A scale invariant perspective. Educational Measurement: Issues and Practic e 26 (4), 11 20. Hong G. & Yu B. (2007). Early learning in elementary years. Educational Evaluation and Policy Analysis 29 239 261. Jimerson, S. R., & Ferguson, P. (2007). A longitudinal study of grade retention: Academic and behavioral outcomes of retained students through adolescence. School Psychology Quarterly 14 (3), 314 339. Konstantopoulos, S, & Hedges, L.V. (2008). How large an effect can we expect from school reform? Teachers College Record 11 0 ( 8).

PAGE 31

31 Lee, Jaekyung (2007). Do n ational and st ate a ssessments converge for educational accountability? A m eta a nalytic s ynthesis of m ultiple m easures in Maine and Kentucky, Applied Measurement in Education 20 ( 2 ) 171 203. Linn, R L Baker, E L & Bete benner, D W (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001 Educational Researcher 31 ( 6), 3 16. Lubienski, S.T. (2001). A second look at mathematics achievement gaps: Intersections of Race, Class and Gender in NAEP Data Paper presented at the annual meeting of the American Educational Research Association, Seattle. Marchant, G J Paulson, S E & Shunk, A.( 2006). Relationships between high stakes testing policies and student achievement after cont rolling for demographic factors in aggregated data. Education Policy Analysis Archives 14 (30). Mosquin, P. & Chromy, J. (2004). Federal sample sizes for confirmation of state tests in the No Child Left Behind Act Washington, DC: American Institutes for Research. National Center for Education Statistics [NCES]. (2007). NAEP State Mapping 2005 Washingotn D.C.: Author National Center for Education Statistics [NCES]. (2010a). NAEP Data Explorer R etrieved on March 2010. Available online at http://nces.ed.gov/nationsreportcard/about National Center for Education Statistics [NCES]. (2010b). NAEP Overview R etrieved on March 2010. Available online at http://nces.ed.gov/nationsreportcard/naepdata No Child Left Behind Act of 2001 [NCLB] 20 U.S.C. §6301 (2002). Paulson, S. E., & Marchant, G. J. (2009). Background variables, levels of aggregation, and standardized test scores. Edu cation Policy Analysis Archives, 17 (22). Schafer W. D. ( 2003 ). A state perspective on multiple measures in school accountability Educational Measurement: Issues and Practice 22 ( 2 ), 27 31 Silberglitt, B, Appleton, JJ, Burns, MK, & Jimerson, SR. (2006) Examining the effects of grade retention on student reading performance: A longitudinal study. Journal of School Psychology 44, 255 270. Trousdale, D., Penuel, W.R. & Khanna, R. (1999 ). Arguing for equity: The politics of testing English language learners in San Fransisco Paper presented at the annual meeting of the American Educational Research Association, Montreal.

PAGE 32

32 U.S. Department of Education [U.S. DOE ]. (2010). No Child Left Behind. R etrieved on March 2010. Available online at http://www.ed.gov/nclb/landing.jhtml U.S. Department of Education [U.S. DOE ]. (2007 a ). State and local implementation of the No Child Left Behind Act Volume I Title I School Choice, Supplemental Educational Services, and Student Achievment Washington D.C: Author U.S. Department of Education [U.S. DOE ]. (2007 b ). State and local implementation of the No Child Left Behind Act Volume II Teacher Quality Under NCLB: Interim Report Washington D.C: Author Wayne, AJ, & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research 73 89 122. Zinth, K (2005). Student promotion/retention policies. Denver: Education Commission of the States.

PAGE 33

33 BIOGRAPHICAL SKETCH Phillip Carroll Porter was born in Frankfurt Germany, but moved early and often. His family settled in Gainesville, FL while he was sti ll in high school and despite various excursions to far away places, two years as a missionary in Canada and three years as a student at Brigham Young University, he always finds his way back. Six seasons of gh to make him a Gator, but he is delighted to finally have a degree from the University of Florida. Before graduate school Phillip was the band director at P.K.Yonge for five years. While in graduate school Phillip was fortunate to teach Measurement a nd Assessment in Education for three semesters, and also to work with Dr. Regina Bussing and Dr. Cyndi Garvan as a re search assistant.