Citation
End-of-Course Geometry Exam Results Compared with National Academy Foundation (NAF) Enrollment, Gender, and the Cost-Effectiveness of the Program in a Selected School District

Material Information

Title:
End-of-Course Geometry Exam Results Compared with National Academy Foundation (NAF) Enrollment, Gender, and the Cost-Effectiveness of the Program in a Selected School District A Two-Year Study
Creator:
Stalcup, Kimberly A
Place of Publication:
[Gainesville, Fla.]
Florida
Publisher:
University of Florida
Publication Date:
Language:
english
Physical Description:
1 online resource (1 p.)

Thesis/Dissertation Information

Degree:
Doctorate ( Ed.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Educational Leadership
Human Development and Organizational Studies in Education
Committee Chair:
WOOD,R C
Committee Co-Chair:
ELDRIDGE,LINDA BURNEY
Committee Members:
MOUSA,BRUCE E
DANA,THOMAS M

Subjects

Subjects / Keywords:
academies -- cost-effectiveness -- engineering -- exams -- naf -- stem
Human Development and Organizational Studies in Education -- Dissertations, Academic -- UF
Genre:
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Educational Leadership thesis, Ed.D.

Notes

Abstract:
The college and career ready student was the focus of major legislative discussion as well as policy and program implementation at the local school board levels. School grades were calculated partially based on post-secondary opportunities given to students while enrolled in high school. For this reason, numerous prescriptive programs to drive student success emerged in secondary schools such as the National Academy Foundation (NAF). This study focused on the correlation between student enrollment in National Academy Foundation (NAF) engineering courses and their respective end-of course exam grades in a select math course. Gender was also analyzed within engineering academy enrollment in eight southwestern high schools to determine performance variance. Six of these high schools were newly implementing the NAF Engineering Academy in course and pathway selections in the 2014-15 school year. End of course Geometry exam scores correlating to enrollment in NAF Engineering courses were compared to determine the success in those EOCs compared to those students who were not enrolled in NAF engineering courses over a two-year period. Success among the gender subgroups in each southwestern Florida high school was also studied to determine future recruitment initiatives. Limitations and adaptations were discussed including representation of gender subgroup in the Academy of Engineering, as well as funding. Cost-effectiveness of the program was determined to validate the expenditures of the NAF program for student benefit relating to performance on the Geometry end-of-course exam. The positive impact that NAF engineering enrollment may play on performance of the Geometry EOC could be a precursor to market the measurable impact to students. Specifically, the students enrolled in NAF (as reported in this study) scored at means one level of proficiency higher than non-NAF engineering students (Level 4 to Level 3 in comparison). While gender did not show significant differences in the marginal means, NAF enrollment yielded a higher mean Geometry EOC score among males and females in the study. Immersion into STEM focused curricula was tied to critical thinking, decision-making, and connection to real-world applicable learning. The benefits of NAF Engineering course expenses were justified in the increase performance based on enrollment in the program. Keywords: dissertation review, End of Course Exams, National Academy Foundation, engineering, STEM, cost-effectiveness ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis:
Thesis (Ed.D.)--University of Florida, 2017.
Local:
Adviser: WOOD,R C.
Local:
Co-adviser: ELDRIDGE,LINDA BURNEY.
Statement of Responsibility:
by Kimberly A Stalcup.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
LD1780 2017 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

END OF COURSE GEOMETRY EXAM RESULTS COMPARED WITH NATIONAL ACADEMY FOUNDATION (NAF) ENROLLMENT, GENDER, AND THE COST EFFE CTIVENESS OF THE PROGRAM IN A SELECTED SCHOOL DISTRICT : A TWO YEAR STUDY By KIMBERLY A. STALCUP A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION UNIVERSITY OF FLORIDA 2017

PAGE 2

2017 Kimberly A. Stalcup

PAGE 3

To my dad, my mom, my husband, my two children and my family who are my team. Thank you for being my cheerleaders.

PAGE 4

4 ACKNOWLEDGMENTS The desire to better oneself and empower others to do the same is the underlying reason I desired to be an educator. Although I work daily to help students seek a purpose, I have a support network that continue to uplift and remind me of my own purpose. I seek balance and I have vowed not to live to work but to work to live. Many people have influenced my decisions and have motivated me to achieve my goals but even more of an impact are those who remind me the true goals in life. As I worked on my docto ral coursework and research, I vowed not to entirely disrupt the most important thing in my life: the importance of family. However, my husband, J.R. Stalcup and my two sons, Brayden and Bryen Stalcup have supported me and have served as my cheerleader s t hroughout this process. Although we embarked on many adventures while I was in school, there were times that I had to devote to my studies. They were supportive, helpful, and my rock in times of need. Even when J.R. and I both were immersed in our studie s, we believed that we were showing our children the i mportance of education. I wholeheartedly thank my mother, Karen Sweatlock, for her continuous unconditional support and love. She has believed in me from the beginning of my existence, along my educat ional path, my quest to be a better person, and continues to nurture my family. I thank her for giving me the praise that only a mother can give. Even though my father is amongst the angels, I thank John W. Sweatlock for his strength he gives me to keep going and for watching over me as I progress through my education, administrative career, and life on this earth. I know you are my guardian angel and watch over my children and ensure their safety I wish to thank my mentors who always gave me opportuni ty and unwavering encouragement: Mr. Tim Kutz, Mr. Gary Brown, Mr. Ken Fairbanks, and Dr. Shari Huene Johnson. I cannot thank the

PAGE 5

5 district CTE department and my husband, J.R. enough for their help with gathering research data and statistical brainstorming I also thank my past leaders who outlined leadership qualities and helped me shape the leader I wish to be. I also thank Mr. Matt Buchler for his support in completing the requirements for this degree and for the opportunities for growth and advancement in the future. I wish to thank those who have brought a sense of balance and adventure to my life: Kenny Sweatlock, Kirstin Talley Coverdale, and Natalie Kane. I also thank my colleagues who became lifelong friends: Cherie Allison, Clay Terrell, and Mic hal Durik. I thank my family near and far who have been true supporters of my aspirations and dreams and continue to fill my life with great memories. Last but not least, I thank my University of Florida professors who opened my mind to think differently challenge the status quo, and create d a bigger path for me. I humbly thank my chair, Dr. R. Craig Wood and co chair, Dr. Linda Eldridge for their guidance, patience, and professional direction I thank my committee Dr. Bruce Mousa, and Dr. Thomas Dana for their encouragement and constructive feedback. This is not only dedicated to the people I have mentioned but I dedicate this work to all my past students, teachers, UF cohort members, colleagues, professors, and educators at all levels with whom I hav e had the pleasure to cross paths. You have made me continuously learn and love what I do.

PAGE 6

6 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURES ................................ ................................ ................................ .......... 9 LIST OF ABBREV IATIONS AND DEFINITI ONS ................................ ........................... 10 ABSTRACT ................................ ................................ ................................ ................... 12 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ ..... 14 Statement of the Problem ................................ ................................ ....................... 17 Purp ose of the Study ................................ ................................ .............................. 18 Organization of the Study ................................ ................................ ....................... 20 Research Questions ................................ ................................ ............................... 21 Data Analy sis Plan ................................ ................................ ................................ .. 22 Assumptions and Limitations ................................ ................................ .................. 24 2 REVIEW OF THE LITERA TURE ................................ ................................ ............. 26 History of Education Accountability ................................ ................................ ......... 27 Florida End of Course Testing ................................ ................................ ................ 28 Academy Schools, Charters and Imbedded Academy Programs ........................... 32 STEM, CTE Initiatives and Equity in Programs ................................ ....................... 36 History and Demand for Career Academies ................................ ........................... 42 Cost Estimation and Funding ................................ ................................ .................. 46 Summary ................................ ................................ ................................ ................ 47 3 METHODS ................................ ................................ ................................ .............. 49 Research Design ................................ ................................ ................................ .... 49 Data Analysi s Plan ................................ ................................ ................................ .. 51 Research Questions ................................ ................................ ............................... 52 Summary ................................ ................................ ................................ ................ 53 4 PRESENTATION OF RESULTS ................................ ................................ ............. 56 Variance of Results 2014 15 ................................ ................................ ................... 56 Variance of Results 2015 16 ................................ ................................ ................... 57 Limitations in Results ................................ ................................ .............................. 58

PAGE 7

7 Cost E ffectiveness Analysis ................................ ................................ .................... 60 5 CONCLUSIONS AND IMPL ICATIONS ................................ ................................ ... 67 Significance of the Study ................................ ................................ ........................ 67 Im plications of the Study ................................ ................................ ......................... 68 Limitations and Assumptions ................................ ................................ .................. 69 Conclusions ................................ ................................ ................................ ............ 70 APPENDIX A RESEARCH REQUEST ................................ ................................ .......................... 73 B SUPPLEMENTAL FIGURES ................................ ................................ .................. 74 LIST OF REFERENCES ................................ ................................ ............................... 75 BIOGRAPHICAL SKETCH ................................ ................................ ............................ 78

PAGE 8

8 LIST OF TABLES Table page 3 1 FSA EOC Corresponding Course List 2014 15 and 2015 16 ............................. 54 3 2 Geometry Score Ranges and Level 2014 15 ................................ ..................... 54 3 3 2014 15 Florida End of Course Assessments Achievement Level Policy Definitions ................................ ................................ ................................ ........... 54 3 4 2015 16 Florida End of Course Assessments Achievement Level Policy Definitions ................................ ................................ ................................ ........... 55 3 5 2015 16 FSA EOC Scale Scores for Each Achievement Level .......................... 55 4 1 2014 15 Geometry EOC scores of NAF/Non NAF Students Descriptive Statistics ................................ ................................ ................................ ............. 62 4 2 2014 15 Geometry EOC scores of NAF/Non NAF Students by Gender Descriptive Statistics ................................ ................................ .......................... 63 4 3 Univariate Analysis of Variance Tests of Between Subjects Effects 2014 15 Geometry EOC vs. NAF vs. Gender ................................ ................................ ... 63 4 4 2014 15 Estimated Marginal Means: Gender ................................ ..................... 63 4 5 2014 15 Estimated Marginal Means : NAF ................................ .......................... 63 4 6 2014 15 Estimated Marginal Means: Gender and NAF ................................ ...... 64 4 7 Univariate Analysis of Variance Tests of Between Subjects Effects 2015 16 Geometry EOC vs. NAF vs. Gender ................................ ................................ ... 64 4 8 Descriptive Statistics: 2015 16 Mean Geometry EOC Scores for NAF/Non NAF and Gender Comparisons ................................ ................................ .......... 65 4 9 2015 16 Estimated Marginal Means: NAF ................................ .......................... 6 5 4 10 Univariate Analysis of Variance Tests of Between Subjects Effects 2015 16 Geometry EOC vs NAF vs. Gender ................................ ................................ ... 66 4 1 1 2015 16 Estimated Marginal Means: Gender ................................ ..................... 66 4 12 2015 16 Estimated Marginal Means: Gender and NAF ................................ ...... 66

PAGE 9

9 LIST OF FIGURES Figure page 4 1 2014 15 Estimated Marginal Means of Scale Graph Comparison ...................... 64 4 2 2015 16 Estimated Marginal Means of Scale Graph Comparison Gender NAF identifier ................................ ................................ ................................ ...... 65 B 1 Flor ida Education Finance Program (FEFP) Equation ................................ ........ 74 B 2 2015 District of Study Budget Expense Allocations ................................ ............ 74

PAGE 10

10 LIST OF ABBREVIATIONS AND DEFINITIONS Career and Technical Education (CTE) CTE provided students of all ages with the academic and technical skills, knowledge and training necessary to succeed in future careers and to become lifelong learners. In total, about 12.5 million high school and college students were enrolled in CTE across the natio n. CTE prepared these learners for the world of work by introducing them to workplace competencies, and ma de academic content accessible to students by providing it in a hands on context. In fact, the high school graduation rate for CTE concentrators was a bout 90% 15 percentage points higher than the national average. 1 Computer Based Testing (CBT) Many Florida statewide assessments are now being administered using a computer based format. In 2015, the FSA Grades 8 10 ELA Writing, Grades 5 10 ELA Reading, Grades 5 8 Mathematics, Algebra 1 EOC, Geometry EOC, and Algebra 2 EOC were given in a computer based format, with paper based accommodations offered for eligible students. When taking the test on the computer, students make their answer choices u sing the mouse or keyboard, and they may use various CBT tools, such as the strikethrough tool or the highlighter tool, as they work. Once they have completed the test, they submit their answers electronically. Before exiting the assessment and submitting their responses, students are taken to a screen that identifies questions that are answered, unanswered, and marked for review. 2 End of Course Exam (EOC) Many Florida courses are required for graduation with a required passing score on the end of course culminating exam. These statewide assessments are now being administered and required for graduation as of this publication in the following subjects: Algebra 1 EOC and Geometry EOC. Florida Standards Assessments (FSA) With the Florida standards in pl ace to help Florida students succeed, the Florida Standards Assessments (FSA) in English Language Arts (ELA), Mathematics, and end of course (EOC) subjects (Algebra 1, Algebra 2, and Geometry) serve d Florida students by measuring education gains and progre ss. 3 1 Career and Technical Education (2017). Advance CTE. https://www.careertech.org/cte 2 Understanding Florida Standards Assessments Reports. (2015). Office of Assessment. Florida Department of Education. http://www.fldoe.org/core/fileparse.php/5663/urlt/UnderstandingFSAReports.pdf 3 Florida Standards Assessments. (2016). Florida Department of Education. < http://www.fldoe.org/accountability/assessments/k 12 student assessment/fsa.stml >

PAGE 11

11 National Academy Foundation (NAF) NAF: a national network of education, business, and community leaders who work ed together to ensure high school students were col lege, career, and future ready. Science, Technology, Engineering and Math (STEM) STEM : an educational program developed to prepare primary and secondary students for college and graduate study in the fields of science, technology, engineering, and mathematics (STEM). In addition to subj ect specific learning, STEM aimed to foster inquir ing minds, logical reasoning, and collaboration skills. T score The score that students receive d the first year the FSA assessment was administered. T scores were reported using a norm referenced score scale known as a T score scale. 4 4 Understanding Florid a Standards Assessments Reports. (2015). Office of Assessment. Florida Department of Education. < http://www.fldoe.org/core/fileparse.php/5663/urlt/UnderstandingFSAReports.pdf >

PAGE 12

12 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education END OF COURSE GEOMETRY EXAM RESULTS COMPARED WITH NATIONAL ACADEMY FOUNDATION (NAF) ENROLLMENT, GENDER, AND THE COST EFFECTIVENESS OF THE PROGRAM IN A SELECTED SCHOOL DISTRICT : A TWO YEAR STUDY By Kimberly A. Stalcup December 2017 Chair: R. Craig Wood Cochair: Linda Eldridge Major: Educational Leadership The college and career ready student was the focus of major legislative discussion as well as policy and program implementation at the local school board levels. School grades were calculated partially based on post secondar y opportunities given to students while enrolled in high school. For this reason, numerous prescriptive programs to drive student success emerged in secondary schools such as the National Academy Foundation (NAF). End of course Geometry exam scores corre lating to enrollment in NAF Engineering courses were compared to determine the success in those EOCs compared to those students who were not enrolled in NAF engineering courses over a two year period. Gender was also analyzed within engineering academy en rollment in eight southwestern high schools to determine performance variance and future recruitment initiatives Cost effectiveness of the program was determined to validate the expenditures of the NAF program for student benefit relating to performance o n the Geometry end of course exam. T he positive impact that NAF engineering enrollment

PAGE 13

13 may play on performance of the Geometry EOC could be a precursor to market the measurable impact to students. Specifically, the students enrolled in NAF (as reported in this study ) scored at means one level of proficiency higher than non NAF engineering s tudents (Level 4 to Level 3 in comparison). While gender did not show significant differences in the marginal means, NAF enrollment yielded a higher mean Geometry EOC score among males and females in the study. Immersio n into STEM focused curricula was tie d to critical thinking, decision making, and connection to real world applicable learning. The benefits of NAF Engineering course expenses were justified in the increase performance based on enrollment in the program.

PAGE 14

14 CHAPTER 1 INTRODUCTION Th e college and career ready student was the focus of major legislative discussion as well as policy and program implem entation at the local school board levels. In the state of Florida, s chool grades were partially calculated based on post secondary opport unities given to students while enrolled in high school such as industry certifications 1 For this reason, numerous prescriptive programs to drive stude nt success emerged in secondary schools such as the National Academy Foundation (NAF) 2 NAF grew from on e NAF Academy of Finance in New York City to hundreds o f academies across the country that focused on growing industries including: finance, hospitality & tourism, information technology, engineering, and health sciences. During the 2014 15 school year nea rly 82,000 students atte nded 667 NAF academies across thirty eight states, including DC and the US Virgin Islands. In 2014, NAF academies reported 97 percent of seniors graduated with 93 percent of those graduates that planned to go to college. 3 According to NAF, an academy helped increase school and district graduation rates and g ave students the confidence to take charge of their futures and succeed in college, career, and beyond. 4 It is proposed that t he NAF educational design was flexible en ough for differing high schools and communities in size for succ essful implementation. NAF worked with high schools and school districts to 1 Accountability Update (2015). Florida Departmen t of Education. < http://www.fldoe.org/core/fileparse.php/5637/urlt/AccoReportTechMeeting2016.pdf > 2 Nati onal Academy Foundation. www.naf.org 3 Ibid. 4 Ibid.

PAGE 15

15 implement the NAF educational design into the school setting and provided the framework for the academies 5 There w ere numerous academy programs carrying different brands and names including the Project Lead the Way 6 program to incorporate Science, Technology, Engineering, and Math (STEM) initiatives in schools. N four essential elements of practice: academy development & structure, curriculum & instruction, advisory board, and work based learning. NAF reported that these elements build on each other to engage students, support school and district priorities, and g ave businesses the oppor tunity to connect with students and offer internships. 7 NAF academies were structured as small, focused learning communities that fit within and claimed to enhance high school systems, allowing NAF to become an integral part of a plan for higher achievemen t for a fee. However, the budget crisis facing many school systems placed increased pressure on any expenses that are not already part of the routine funding in a school system. NAF aca under 5 percent of the costs per student in hi gh school. 8 significantly improved college and career ready results, short term budget considerations could diminish the ability of the school board to implement NAF 5 www.naf.org 6 See www.pltw.org 7 www.naf.org 8 Ibid.

PAGE 16

16 academies. 9 Project Lead the Way (PLTW) also ran as an ancilla ry program to facilitate the engineering programs. For this reason, this study implemented a cost effective analysis in an attempt to justify NAF engineering expenses including the costs of PLTW professional development based on student performance. Grant s, in house technology resources, and donor finances provide d supplies and needed resources in the selected school district studied 10 NAF promoted open enrollment for its academies in order to maximize every s claimed flexible structure encouraged teacher collaboration across subject areas and fostered personalization to meet student, school, district, and state needs and goals. NAF claimed that it provides a rigorous, industry validated career themed curriculum that incorporates current industry instructional practices foster cross curriculum collaboration so students can make conne ctions across subject areas. 11 The NAF curricula was created in partnership with industry professionals and designed around projects that help students acquire valuable workplace skills and see their education as a step toward long term career options. 12 The program claimed to empower teachers to expand the boundaries of the classroom in non traditional ways that ensure d lessons have real world application to growing 9 NAF Strategic Plan 2012 16. www.socialimpactexchange.org/sites/www.socialimpactexchange.org/files/NAF%202012 16%20plan%20Final 2.pdf 10 Career and Technical Education Director Interview (2016). Career and Technical Education Director. Name and school distri ct removed for Anonymity. 11 National Academy Foundat ion. www.naf.org 12 National Academy Foundation. (2012). National Academy Foundation Strategic Plan 2012 2016. < http://www.socialimpactexchange.org >

PAGE 17

17 industries. The advisory boards claim ed to provide a bridge between schools and the workplace where businesspeople and community leaders volunteer on local advisory boards to play an active role in developing their future workforce by shaping talent in high school. In addition, a dvisory board members collaborate with educators to inform curricula and help organize work based learning activities within the community NAF claimed that a dvisory boards g ave students the opportunity to build relationships with mentors early and learn from successful adults. Work based learning connected the classroom t o the workplace and the workplace to the classroom. T his instructional strategy worked to provide students with a well rounded skill set that goes beyond academics and included the soft skills needed to succeed in college and the working ro ach to work based learning was centered on a continuum of work based learning experiences beginning with career awareness activities, progressing to career exploration activities, and culminating in career preparation activities, including internships. 13 Bu siness people guest speak in classrooms, host college and career skills workshops, and take part in mock interviews. Students ha d the opportunity to tour worksites, network with, and shadow business professionals. In addition work based learning culminate d in an internship that allows students to apply their classroom skills and learn more about what it takes to succeed. 14 Statement of the Problem High stake tests determining graduation were highly valued by administrators, teachers, parents and students du e to the implications and meaning of the scores. High 13 National Academy Founda tion. www.naf.org 14 Ibid.

PAGE 18

18 stakes tests were described as examinations necessary for graduation, to enter in a specific educational program or university, to gain a scholarship or proving a determined proficiency for an applicati on. 15 In the state of Florida, Geometry End of course exams me t this requirement as a required course with high stakes attached to the scores. This study focuse d on the correlation between student enrollment in National Academy Foundation (NAF) engineeri ng courses and their respective grades in a Geometry End of Course (EOC) exam over a two year period. Determining the cost effectiveness of the program was instrumental since school districts spen t a great deal of money in stipends to NAF teachers professional development, NAF program supplies as well as NAF engineering materials The Science, Technology, Engineering and Math (STEM) initiative and academy enrollment was heavily marketed to students and was included in district and school goals in o rder to recruit an unbiased population selection f or the NAF programs with regard to gender. End of course Geometry exam scores correlating to enrollment in NAF Engineering courses were analyzed to determine the success in those EOCs compared to those stu dents who were not enrolled in NAF engineering courses. Success among gender subgroups in each southwestern Florida high school was also studied to determine f uture recruitment initiatives. Purpose of the Study The purpose of this st udy was to determine if there were a positive relationship between enrollment in the National Academy Foundation Engineering program and 15 Hatice Kumandas and mer Kutlu (October 2015). High Stakes Tests Journal of Educational Sciences Research International E Journal. Vol 5, No.2.

PAGE 19

19 achievement on the Geometry End of Course test that was used as standards for graduation in Florida. The purpose of the study also served t o analyze the performance of the gender subgroups. Two years of data were analyzed to determine Geometry EOC performance and NAF enrollment. The cost effectiveness of the program w as analyzed to determine the significance of the EOC math exam results of NAF (Engineering program) enrolled students versus non enrolled students. Determining cost effective ness based on testing correlated to political motivations since test performance was a measure that was used for accountability purposes. Beyond controlli ng t eaching and learning, tests served as the accountability tool that ensured was allocated to valid expenditures. 16 Each high school in the so uthwestern district of study had an engineering academy. Geometry end of course exam scores correlating to NAF Engineering courses were analyzed and hypotheses were formed to determine areas of weakness in implementation and success among gender subgroups in each southwestern Florida hi gh school selected. Although test scores were only one measure of student achievement and school success, the results of these standardized tests were the most publicized. In Florida, End of Course (EOC) tests at the high school level were the standardize d tests used by the Florida Department of Education ( FL DOE) to measure school success annually during the period of the study The questions considered the performance Geometry EOC test results for all students enrolled in NAF and those not enrolled as well as consider ed the results for students by gender (enrolled and not enrolled in NAF) 16 George Madaus and Michael Russell. (2010 11). P aradoxes of High Journal of Education Vol. 190, Nos. 1 2.

PAGE 20

20 1. Did students who participate d in the NAF program have higher EOC scores than students who did not participate in NAF? 2. Did these results hold true for students regardl ess of their gender? 3. D id NAF enrollment demonstrate cost effectiveness of the program based on these results? All high schools in one south west Florida school district were selected as research sites. Student End of Course testing data w ere collected along with NAF participation data from each of the high schools. D ata were analyzed to determine if there w ere a statistically significant relationship between participation in NAF and achievement on the Geometry End of Course t est. The independent variable was in the NAF program or non participation, while the dependent variable s were the Geometry end of course exam scores of both gr oups of students, and gender was the covariate. The hy pothesis of this study indicated that enrollment in National Academy Foundation over two co nsecutive academic semesters had a statistical impact upon the end of course math achievement examination for 9 th and 10th grade students enrolled in Geometry The findings resulti ng from the research question was hypothesized to indicate a positive relationship between enrollment in NAF and math achievement; while non enrollment scores were deem ed lower or inconsistent. Organization of the Study All southwestern high schools in the district of varying socio economic populat ions were chosen. The design could be duplicated within different districts in different locations given demographic consideration. The study t ook anonymity precautions in the selectio n samples and references to schools.

PAGE 21

21 The high s chools in a Florida district were selected as research sites. Second, student End of Course testing data were collected along with NAF enrollment data from each of the schools involved. Third, the data w ere analyzed as a whole as well as disaggregated by g ender to determine if there w ere a statistically significant relationship between participation in the NAF academy and achievement on End of Course tests. Cost effectiveness was determined by the statistically significant results of the research questions : Research Questions 1. D id EOC Geometry scores differ by Enrollment in a NAF Engineering program or non Enrollment in a NAF Engineering program in 2014 15? Ho: EOC Geometry scores d id not differ by group NAF enrollment vs. Non NAF enrollment (Independent var iable Group 1 vs. Group 2) in 2014 15. Ha: EOC Geometry scores differ ed by (independent variable Group 1 vs. Group 2) NAF enrollment vs. Non NAF enrollment in 2014 15. 2. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or n ot Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2014 15? Ho: EOC Geometry scores d id not differ by gender when enrolled in NAF Engineering or not enrolled in NAF Engineering. (independent variable: group 1 vs. group 2) in 2014 15. Ha: EOC Geometry score s differed by gender (independent variable: g roup 1 vs. group 2) in 2014 15. 3. D id EOC Geometry scores differ by Enrollment in a NAF Engineering program or non Enrollment in a NAF Engineering program in 2015 16? Ho: EOC Geometry scores d id not differ by group NAF enrollment vs. Non NAF enrollment (Independent variable Group 1 vs. Group 2) in 2015 16. Ha: EOC Geometry scores differ ed by (independent variable Group 1 vs. Group 2) NAF enrollment vs. Non NAF enrollment in 201 5 16. 4. D id EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2015 16?

PAGE 22

22 Ho: EOC Geometry scores d id not differ by gender when enrolled in NAF Engineering or n ot enrolled in NAF Engineering (independent variable: group 1 vs. group 2) in 2015 16. Ha: EOC Geometry scores differ ed by gender (independent variable: group 1 vs. group 2) in 2015 16. 5. D id NAF enrollment demonstrate cost effectiveness of the program based on these results of the previous research questions? Ho: NAF Engineering enrollment was not significant on Geometry EOC score results thus not demonstrating cost effectiveness of the program. Ha: NAF Engineering enrollment was signif icant on Geometry EOC score results thus demonstrating cost effectiveness of the program. Data Analysis Plan To examine the research question, a two way analy sis of variance (ANOVA) was conducted to assess if mean differen ces exist ed The two way ANOVA was an appropriate statistical analysis when the purpose of research was to compare two or more discrete groups on a conti nuous dependent variable that was measured more than once. For this research, the co ntinuous dependent variables were the EOC Geomet ry te st; the independent variable had the following groups (NAF Engineering Enrollment vs. Non NAF Engineering Enrollment and Female NAF Engineering vs. Male NAF Engineering vs. Female Non NAF Engineering vs. Male Non NAF Engineering). The ANOVA used the F test to make the overall comparison on whether group me ans differ. If the obtained F w ere larger than the cr itical F, the null hypothesis was rejected. The result s of the mixed model ANOVA present ed findi ngs for the main effect and evaluate d the overall d ifferences by time (within subjects) and also separately by group (between subjects). The dependent variable was approximately normally distributed for each level of the independent variable. For the purpose of the rese arch, a level of

PAGE 23

23 significance of 0.05 was selected in order to determine whether to accept or re ject the null hypotheses E ight high schools were selected in a Southwest Florida district. Since the results of End of Course t est scores for NAF enrollees were compared against those for non enro l lees, there were two sets of data that were collected: a list of student scores on End of Course tests and a list of students who were participating in NAF enrolled academies at the time those scores were achieved. In order to compare the results by gende r, that data w ere compiled to differentiate those results Two years of NAF enrollment and EOC Geometry scores (2014 15 and 2015 16) were collected and analyzed separately due to the changing scale scores implemented by the Florida Department of Education Office of Assessments. Since numerous NAF academies were implemented in each schoo l and the Engineering Academy was consistent ac ross all schools, engineering was only selected. The 2014 2015 and 2015 20 16 End of Course test scores were utilized for the data collection. To ensure appropriate and meaningful data, only scores from the academic area that have served as graduation exit standards for hig h school students (Geometry) were utilized. After determining which data were needed and the schools involv ed the actual data collection was compiled through the district d ata warehouse site through written approval. 17 Once the necessary data w ere collected, students were anonymously compiled into a master spreadsheet with columns enrolled Engineering program NAF student (Y or N), test score, a nd the gender of each student. 17 See Appendix A.

PAGE 24

24 Assumptions and Limitations It is an assumption that i mproving EOC results were extremely important for high schools in Florida. Part of the state and federal annual measures for every high school were based on the results of EOC tests. These tests were also used as exit standards for students graduating high school 18 In 2013 Money Tax Cut Budget" that proposed to fund education at highest total and state funding levels in Florida history for Florida's K 12 public schools and the Florida college system. The funding listed $18.8 billion fo r Florida's K 12 public schools. Governor Rick Scott indicated, Education is one of the primary economic eng ines in this s tate." 19 It was recognized and assumed that Florida's students must be given the tools and resources they need to succeed in college, career and life. 20 For this reason, the investment in college and career readiness became a top discussion among district leaders and budgets and school improvement plans must address this goal. In 2015, the Florida Legislature approved a $20.15 bil lion K 12 budget that equated to $7,178 per student, reaching the total Governor Rick Scott outlined in his camp aign promise. 21 His campaign also recognized that a dditional funding and other highlights were needed and therefore, proposed : $8.4 million for professional 18 Assessments (February 2014). Florida Department of Education. Office of Assessme nt. www.fldoe.org/core/fileparse.php/7764/urlt/GradRequireFSA.pdf 19 Rick Scott. (June 2 News Releases. < www.flgov.com > 20 Ibid. 21 ). < http://www.rickscottforflorida.com/wp content/uploads/2014/11/Let%E2%80%99s Keep Florida Learning.pdf >

PAGE 25

25 development for principals and assistant principals; $5 million for teacher training and technical assistance related to the implementation of state standards; $2 million increase to expand the state portion of the dollar for dollar match of private donati ons from public school district educational foundations; and $10 million for performance funding ($5 million new funds) to Florida colleges based on an increase in students earning industry certifications in high skill/high wage Florida occupations as spec ified in the most recent Florida Stat ewide Targeted Occupations List 22 Although these proposals were slow to implement, the delivery outlined the focus on necessary educational funding to advance the quality of educational programs and training A limita tion of the study stemmed from the allocated grant monies, technology resources that were used for the NAF Engineering program as these dollars may have been allocated elsewhere if another program of interest existed for those students. The Nationa l Academy Foundation served as a tool that assisted the local school board to prioritize federal and state STEM (Science, Technology, Engineering and Math) initiatives It was a limitation that schools that may be STEM certified could possibly yield highe r performing students; however, the study did not focus on STEM certification nor identify STEM certified schools. Several academy and career focused organizations such as Project Lead the Way (PLTW) were also a tool that fueled the STEM initiatives and w ere fully implemented in each school of the study 22 tt Signs The News Releases < www.flgov.com >

PAGE 26

26 CHAPTER 2 REVIEW OF THE LITERATURE Performance tests have been tied to evaluation throughout history. In 200 BC, the Chinese used tests to help eliminate patronage and open access to the civil service. The Dead Sea scrolls describe the use of tests by the Qumran community to determine when a man was ready to become a formal member of the community. England, France, and Italy, among other nations, have used tests to ensure that students acquire certain s kills and establis h standards of performance. In Fifteenth C entury Italy, tests were used to hold teachers accountable for student learning. Since then, policy makers have used tests to hold students and schools accountable and allocate scarce resources. 1 However, these various testing policies were not meant to be punitive. Instead, these policies were, and continue to be, sincere attempts to address perceived problems in education. Policy makers were attracted to testing as a solution to problems in soci ety and education since test performance is a quantitative measure. Policy makers realize d they c ould not directly control instruction in classrooms, but they c ould indirectly influence instruction by attaching rewards or sanctions to the results of mandat ed tests. 2 The National Commission on Tes ting and Public Policy described how pressure to improve scor es on reading and math tests could narrow teaching to test preparation. The Commission warned that the high stakes attached to test use were 1 George Madaus and Michael Russell. (2010 11). P aradoxes of High Journal of Education Vol. 190, Nos. 1 2. 2 Ibid.

PAGE 27

27 schools and teachers away from instructional practices that would help to produce 3 This sentiment was quite important to evaluate the importance and cost effectiveness of academy enrollment. Applica ble skills and cross curricular instruction could drive learning back to producing those critical thinkers and active lear ners that the career fields needed History of Education Accountability Eamon DeValera, the Prime Minister of Ireland, expressed the logic behind using tests to hold teachers, students, and schools accountable. Proposing a system of certification examinations at the end of primary school, he argued successfully before Parliament in 1941: eached and we are paying the money, we have the right to see that something is secured for that money. The ordinary way to test it and try to help the whole educational system is by arranging our tests in such a way that they will work in a direction we wa nt. 4 evidence that determ well spent. This reason ing was evaluate the success of educational programs. This use of tests to measure the outcome of education reflected a larger belief in the use of metrics to determine the 5 3 George Madaus and Michael Russell. (2010 11). Paradoxes of High Sta Journal of Education Vol. 190, Nos. 1 2. 4 Ireland. Dail Eireann. (1941). Parliamentary Debates, Col 1119. 5 George Madaus and Michael Russell. (2010 11). P aradoxes of High Journal of Education Vol. 190, Nos. 1 2.

PAGE 28

28 Madaus and Russell claim ed that there were three predictable ways that a hi gh stakes test adversely affected teaching and learning as seen throughout history, in recent research, and in literature of all genres. First, teachers g ave greater attention to tested content and decrease d focus on non tested content that narrowed the c ontent and skills taught and learned within a course. Sec ond, a high stakes test prevented time and coverage from other non tested disciplines, thus narrowing the curriculum across subjects. Third, the content and skills covered on the high stakes tes ts at the upper grades displaced the content and skills of non t ested lower grades, which altered the curriculum across grade levels. 6 The debate over the use of tests in th e development of policy was a debate over what results were desired from schools. It was a debate over educational values and competing educational philosophies, and it was about a means and ends. It was not a debate on technical matters related to testing. Madaus and Russell indicated that the proper role of testing was in question. 7 Whethe r the test results indicate d success b eyond high school, the tests place d added stress on stakeholders. Infiltrating standards into career or academy coursework or providing an applicable course utilizing student ability and career pathways, tests could b egin to determine cost effectiveness in a greater capacity. Florida End of Course Testing The Florida End of Course (EOC) Assessments measure d student achievement of the Next Generation Sunshine State Standards (NGSSS), which specified the 6 George Madaus and Michael Russell. (2010 11). P aradoxes of High Journal of Education Vol. 190, Nos. 1 2. 7 Ibid.

PAGE 29

29 challenging con tent Florida students were expected to know and be able to do. On December 19, 2011, the State Board of Education approved the Achievement Levels for FCAT 2.0 Reading, Mathematics, and the Algebra 1 EOC Assessment. On December 12, 2012, the State Board app roved the Achievement Levels for FCAT 2.0 Science and the Biology 1 and Geometry EOC Assessments. On January 21, 2014, the State Board of Education approved Achievement Levels for the U.S. History EOC Assessment and established FCAT 2.0 and EOC assessment passing scores. The passing score on FCAT 2.0 Reading, Mathematics and Science an d each Florida EOC Assessment was the minimum score in Achievement Level 3, and the pass ing score on FCAT 2.0 Writing was a score point of 3.5. These passing standards were co nsistent with the 2014 2016 define d th e level of success a student had with the NGSSS on the FCAT 2.0 and Florida EOC Assessments. The Achievemen t Level Policy Definitions applied to all FCAT 2. 0 and EOC assessments and ranged from 1 (lowest) to 5 (highest). 8 See Table 3 2 through 3 5 for Geometry scores and range levels and Florida End of Course Assessments Achievement Level Policy Definitions. The Florida Stand ards in Mathematics and English Language Arts were approved by the Florida State Board of Education in February 2014 and were implemented in grades K 12 in the 2014 2015 school year. All Florida schools t aught the Florida Standards, and students were asses sed through the statewide Florida Standards Assessments (FSA). Data from the FSAs provide d information to parents, 8 FCA T 2.0 and Florida EOC Assessment. (February 2014). Florida Department of Education. Office of Assessment. < http://www.fldoe.org/core/fileparse.php/3/urlt/achlevel.pdf >

PAGE 30

30 teachers, policy makers, and the general public regarding how well students were learning the Florida Standards. 9 The Florida Standard Assess ment (FSA) End of Course exams (EOC) were computer based tests (CBTs). 10 Each assessment was administered in two 90 minute sessions with one session per day over two days. There were multiple forms of the assessment, with a maximum of sixty eight items on e ach test form. Six to ten of these items were field test items and were not used to calculate student scores. Students registered for Geometry, Geometry Honors, IB Middle Years Program Geometry Honors the study) and Pre AICE Mathematics 2 (not offered as of the study) 11 were those required to take the Geometry exam The Geometry EOC tested the following concepts: Congruence, Similarity, Right Triangles, and Trigonometry: Students understand congruenc e and similarity in terms of transformations. They prove and use geometric theorems. They demonstrate geometric constructions. They define trigonometric ratios. They solve problems involving right triangles. They use congruence and similarity criteria for triangles. Circles, Geometric Measurement, and Geometric Properties with Equations: Students prove and apply theorems about circles. They find arc lengths and areas of sectors. They derive the equation of a circle. They use coordinates to prove theorems a nd to solve problems algebraically. They explain and use volume formulas. 9 Understan ding Florida Standards Assessments Reports. (2015). Office of Assessment. Florida Department of Education. http://www.fldoe.org/core/fileparse.php/5663/urlt/Under standingFSAReports.pdf 10 Paper based versions (regular print, large print, braille, and one item per page) are provided for students with disabilities who cannot access assessments on the computer, as specified in their individual educational plans (IEPs) or Section 504 plans. CBT accommodations (e.g., text to speech) are available for students whose IEPs indicate these accommodations. 11 Florida Department of Education. (November 2014). Bureau of K 12 Student Assessment. 2014 15 FSA EOC Fact Sheet. < http://www.fldoe.org/core/fileparse.php/5423/urlt/FSAEOCFS2014 15.pdf >

PAGE 31

31 Modeling with Geometry: Students apply geometric concepts in modeling situations. 12 For the first administration, students receive d two performance indicators: a T score and a percen tile rank. The T score was a score on a scale of 20 80 with approximately 50 as the statewide average. Students also receive d a percentile rank, showing how they performed on each grade level/subject area test compared to all other students in Florida who took the same test. After achievement level cut scores were established, districts received 2014 2015 FSA test results retrofitted to reflect student performance on the new score scale. The success a student ha d achieved with the Florida Standards was assessed by FSA ELA and Mathematics assessments and was described by Achievement Levels that range from 1 (lowest) to 5 (highest). Level 3 indicates satisfactory performance. The level of performance required to score in each achievement level was establi shed prior to the spring 2016 administration. 13 Achievement level cut scores for FSA assessments were adopted in State Board of Education Rule 6A 1.09422, Florida Administrative Code, in January 2016. Level 3 was the passing score for each grade level and subject. The ranges included Level 3 (499 520) with 499 being the passing score; Level 4 (522 532) and Level 5 (544 575). Students who entered grade 9 in 2014 15 and beyond need to pass the FSA Algebra 1 EOC ass essment for graduation purposes, and students must earn passing scores on the Geometry and Algebra 2 EOC assessments to earn a standard diploma with a 12 Understanding Florida Standards Assessments Reports. (2015). Office of Assessment. Florida Dep artment of Education. http://www.fldoe.org/core/fileparse.php/5663/urlt/UnderstandingFSAReports.pdf 13 Florida Standards Assessments. (January 2016). 2015 16 Al gebra 1, Algebra 2, and Geometry End of Course Assessments Fact Sheet. http://www.fldoe.org/core/fileparse.php/5663/urlt/FSAEOC1516.pdf

PAGE 32

32 scholar designation. Scores for spring 2015 Geometry were reported on a T score scale. 14 Achievement level cut scores for FSA EOC assessments were establi shed in January 2016 Academy Schools, Charters and Imbedded Academy Programs It is widely acknowledged that there is a clear mismatch between the social and private returns to education; therefore, reform w ithin existing schools to include focused academies, US Charter schools, and Academy schools in England ha d taken shape. 15 Career academies, after more than four decades of development and three decades of evaluation, ha d been found by a conclusive random a ssignment study to be effective in improving outcomes for students during and after high school. 16 Career academies ha d therefore become the most durable and best tested component of a high school reform strategy to prepare students for both college and car eers in the United States. The number of career academies ha d expanded rapidly, in part because academies have been found to be effective, and in part because they embodied ideas promoted by several major high school reform movements. 17 14 Florida Standards Assessments. (Jan uary 2016). 2015 16 Algebra 1, Algebra 2, and Geometry End of Course Assessments Fact Sheet. < http://www.fldoe.org/core/fileparse.php/5663/urlt/FSAEOC1516.pdf > 15 Andrew Ey les and St Academy schools and the transformation of the English education system Oxford University Press. < https://blog.oup.com/2016/12/academy schools english education system/ > 16 David Stern, Charles Dayton, and Career Academies: A Proven Career Academy Support N etwork. University of California at Berkley. < https://casn.berkeley.edu/resource_files/Proven_Strategy_2 25 1010 03 12 04 27 01.pdf > 17 Ibid.

PAGE 33

33 In England, there was a recognized need to reform programs with the focus on academies. 18 A study by Eyles, Hupkau, and Machin used a quasi experimental research design and estimated the causal effect of attending one of ninety four sponsored academies that opened prior to September 2008 on pupil performance. To account for the fact that schools that became sponsored academies were below the national average in terms of test score performance prior to conversion, they compared outcomes for pupils attending academies that co nverted between September 2003 and September 2008 with those attending schools that would later convert between September 2009 and September 2010. 19 Looking at the characteristics of these two sets of schools before the start of the pro gram, Eyles, Hupkau, and Machin found that they were balanced in terms of test score performance and the demographic make up of their students. The result of this study suggested that on average, a student attending a sponsored academy between 2002 and 2009, scored 0.10 of a s tandard deviation higher in their end of school (Key Stage 4 exams) than a comparable pupil attending a school that would later gain academy status. The results were more pronounced the longer a student spent in an academy where those who spent four years in an academy gained 0.3 standard deviations on their peers who attended similar schools. T he key stage 4 achievement gap between girls and boys was approximately 0.2 of a standard deviation while the gap between those eligible for free school meals and those who were not was approximately 0.75 of a standard deviation. In addition to short term 18 Andrew Eyles and St English Education S ystem https://blog.oup.com/2016/1 2/academy schools english education system/ > 19 Andrew Eyles, Do New School Types Deliver Better O utcomes? Economic Policy Volume 31, Iss ue 87, Pages 453 501, < https://doi .org/10.1093/epolic/eiw006 >

PAGE 34

34 gains, positive effects were found in academy attendance on the probabil ity of staying on in education an d entering a degree program. Since 2010, the academies program in England ha d expanded at a rapid rate with well over 60 percent of secondary schools having gained academy status. 20 The freedoms afforded to academy schools also surrounded around curriculum changes that could also offer career based training. Fifty seven percent of 720 academies indicated that changes in the curriculum were linked to the improved attainment and 29 percent indicated that this was the most important change made. Other conditi ons were noted with only a change in leadership ( 31 percent) as the other most important change and 55 percent indicated that the change in leadership attributed to improved attainment. 21 To bridge the academy school in England to those in the United State s they operate similarly to charter school s in the United States. Historically, educatio n in England had been a centralized process: local educational authorities (LEAs) responsible for state education received funding from the central government, which t hey then distributed to schools under government control, typically according to school size. In the early 2000s this changed when the Labor government began the academies program that gave control of some secondary schools, which were perceived to be fai state funded schools, but had little restr ictions on how they spent money; furthermore, 20 Andrew Eyles, Do New School Types Deliver Better O utcomes? Economic Policy Volume 31, Iss ue 87, Pages 453 501, < https://doi.org/10.1093/epolic/eiw006 > 21 Ibid.

PAGE 35

35 had much greater autonomy from local authority control than the governing bodies of other state funded schools. By January 2010 there were 203 sponsored academies in existence, all of which were free to exercise a large number of freedoms these include hiring and firing of staff, being able to diverge from the national curriculum, set per formance management system for teachers, and outsource servi ces previously provided by the LEA. 22 English academies, charter schools, or career academies embedded into public or private s chools were seeking reform to improve schools. Specifically, career academies were one of the very few educational models that Standards of Practice" were adop ted by many existing career academies and led to improvements across the field. 23 As funding for education bec ame tighter and funding partners demand greater effectiveness and efficiency, these standards of practice should be used to guide the development a nd implementation of programs and promote functions such as: mission and goals; professional development; curriculum and leadership; employer, postsecondary, and community involvement; and student assessment. Standards provide specific guidance to programs. 24 For example, they ensure that career academies were open to any student. They specify that curriculum 22 Andrew Eyles, Do New School Types Deliver Better O utcomes? Economic Policy Volume 31, Iss ue 87, Pages 453 501, < https://doi.org/10.1093/epolic/eiw006 > 23 Ibid. 24 Ibid.

PAGE 36

36 was sequenced, integrated, relevant, and me t college entrance requ irements. Standards provide d assurances that students have options for dual credit with two and four year colleges, as well as work and community based service learning options. They also require d that teachers were credentialed and provided with ongoing training in the academy structure, curricular integration, student support, and employer involvement. 25 STEM, CTE Initiatives and Equity in Programs The question of what defines STEM (Science, Technolog y, Engineering, and Math) existed because of the many different approaches of research and education initiatives that have been a focus to compete globally. As the US federal government made STEM a top priority in funding multiple agencies carrying the STEM initiative h a d been competing for these dollars. 26 STEM programs were established as joint ventures between government, businesses, institutions of higher education, parents, and existing K 12 school systems esp ecially as the government forged ahead to reform science a nd math education. 27 To accomplish these collaborations, centers and programs with emphasis on STEM were formed to tackle this initiative of transforming the current educational paradigm toward a STEM education perspective. 28 Project Lead the Way 25 Betsy Brand. (November 2009). High School Career Academies: A 40 Year Proven Model for Improving College and Career Readiness. The Nat ional Career Acad < http://www.aypf.org/documents/092409CareerAcademiesPolicyPaper.pdf > 26 Jo Handelsman Barack Obama. https://obamawhitehouse.archives.gov/blog/2016/02/11/stem all 27 Jonathan M. Breiner, Shelly Sheats, Carla C. Johnson, and Catherine M. Koehler. (January 2012). What Is STEM? A Discussion About Conceptions of STEM in Education and Partnerships. School Science and Mathematics Volume 112, Issue 1, pages 3 11, http://onlinelibrary.wiley.com/doi/10.1111/j.1949 8594.2011.00109.x/epdf 28 Ibid.

PAGE 37

37 (PLTW), N ational Academy Foundation (NAF), Ford Next Generation Learning, National Career Academy Coalition were among the copious organizations formed to establish programs and resources to answer to the government STEM reform. Project Lead the Way (PLTW ) specific ally was analyzed in several studies to establish the impact of project based learning and the impact on math test scores. One study by Bottoms and Anthony 29 of the Southern Regional Education Board (SREB), the academic outcomes of 274 high school students who took two or more PLTW courses were compared with a control group consisting of students who were selected based on a stratified random sample of 274 students (using gender, ethnicity, and parental education level) who took career/technical education ( CTE) courses. Both groups of students participated in compared a snapshot of results for an assessment that was aligned with the National Assessment of Educational Proficiency (NAEP) and that had su btests for mathematics, science, and reading, and found that PLTW students scored significantly higher on all three subtests than the CTE control group. This study also took into consideration that higher performing students may have made up the PLTW grou p and enrolled in more science and math courses than the CTE control group 30 In spite of the fact that traditionally lower performing students were enr olled in PLTW courses in higher numbers, students who participated in PLTW were more prepared for higher education. Specifically and in comparison to matched, non PLTW students, PLTW students scored 29 Gene Bottoms and K imberly Anthony. (2005 e Way: A Pre Engineering Curriculum that W South ern Regional Education Board http://publications.sreb.org/2005/05V08_Research_PLTW.pdf 30 Ibid.

PAGE 38

38 minimum Mathematics standard, and a higher percentage met the college ready Mat hematics standard 31 In a subsequent study by Bottoms and Uhn, where PLTW students were defined as those who took three or more PLTW courses, they observed a similar pattern of results: 292 PLTW students scored significantly higher, on average, on the mathematics and science subtests (but not reading) than their CTE peers. 32 A study by Tran and Nathan used a very small sample of seventy PLTW students, students who took one or more PLTW courses during the 2007 08 school year, and compared their academic performance on a state assessment in mathematics and science to 70 students in a matched control group. Students in the control group were hand matched with the PLTW students based on Grade 8 mathematics and science performance, gender, a nd economic disad vantage status. Using a regression model, they found that PLTW students scored significantly lower in mathematics than their non PLTW peers but similarly to their peers in science. Tran and Nathan could not determine a definitive conclusion on the effec ts of enriched integration and the amount of exposure to the PLTW curriculum because the sample size for this analysis was relatively small 33 31 Higher American Journal of Engineering Education Vol. 4, No. 1. http://files.eric.ed.gov/fulltext/EJ1057109.pdf 32 Gene Bottoms and John Uhn N ew Type of Career and Technical P Southern Regional Education Board Atlanta, GA. 33 Natalie Tran & College Engineering Studies: An Investigation of the R elation ship Between Pr e College Engineering Studies and Student Achievement in Science and M Journal of Engineering Education 92(2), 143 157.

PAGE 39

39 The Department of Chemical and Biochemical Engineering at the University of Iowa implemented a study of the impa ct of PLTW on achievement outcomes in Iowa. PLTW participants and nonparticipants from 8 th grade into secondary education. The findings indi cated that PLTW participan ts were more likely to be white, male, and perform in the upper quartile in mathematics and science prior to PLTW enrollment. This study also found statisti cal evidence that PLTW increased mathematics or science scores on the Iowa Test of Educational Development by 5 points which corresponds to roughly half a grade level. 34 Van Overscheld e gathered student level data about students who participated in PLTW courses in secondary grades, as w ell as matched non PLTW students, obtained from the Texas Education Research Center. The study indicated the cohort group was significant with a higher percentage of PLTW students meeting the minimum state st andard ( 91.2 percent ) tha n the matched control group ( 90.2 percent ) 35 Project Lead the Way was one program that stressed industry based learning; thereby focusing on STEM programs to increase performance and college ready students. Legislatures allocated billions of dollars into this enterprise while teachers in the K 12 system were expected t o teach STEM to their students. Businesses were invest ing in the f uture employment pipeline where the students were the product of 34 David Rethwisch, Melissa Chapman Haynes, Soko S. Starobin, Frankie Santos Laanan and Tom American Society for Engineering Education BePress. Access to the study available: http://ir.uiowa.ed u/cgi/viewcontent.cgi?article=1033&context=aseenmw2014 35 American Journal of Engineering Education Texas State University, Vol. 4, No. 1. < http://files.eric.ed.gov/fulltext/EJ1057109.pdf >

PAGE 40

40 these efforts. 36 Project Lead the Way (P LTW) add ed rigor to traditional technical and academic programs through project and problem based learning. PLTW began in the 1997 1998 academic year, affiliated with the High Schools That Work (HSTW) 37 project in 1999, and served over 1,300 schools in forty five s tates in 2015 38 PLTW courses were designed to provide students with opportunities to understand the scientific process, engineering problem solving, and the application of technology; understand how technological systems work with other systems; use mathe matics knowledge and skills in solving problems; communicate effectively through reading, writing, listening and speaking; and work effectively with others. 39 PLTW aimed its programs for the top 80 percent of students. Students enrolled in the PLTW program were required to be enrolled in a college preparatory math sequence as well. 40 They were also required to take end of course examinations in all but the capstone course. 41 Several affiliated universities offer ed college credit for adequate end of course 36 Jonathan M. Breiner, Shelly Sheats, Carla C. Johnson, and Catherine M. Koehler. (January 2012). What Is STEM? A Discussion About Conceptions of STEM in Education and Partnerships. School Science and Mathematics. V olume 112, Issue 1, pages 3 11, http://onlinelibrary.wiley.com/doi/10.1111/j.1949 8594.2011.00109.x/epdf 37 See www.sreb.org/high schools work 38 Pam Newberry, T. Richard Grimsley, Lead th e Way Curricula, Pedagogy, and Professional Development: Act ivities Regarding Increasing E ngineeri ng and Technological Literacy of K12 students in the PLTW N Annual Conference of the American Society for Engineering Education. Chicago, IL. < https://peer.asee.org/648 > 39 Ibid. 40 Ibid. 41 Ibid.

PAGE 41

41 exa mination scores and cumulative averages. 42 When PLTW compared its students to those enrolled in the HSTW schools with similar career/technical fields, PLTW students out performed non PLTW students They completed more science and math classes and scored hig her on NAEP tests. 43 A longitudinal study reviewed by the National Cente r for Education Statistics looked at various characteristics of beginning postsecondary students that entered STEM fields between 1995 and 2001. 44 The percentage of men who entered STEM fields was higher than that of women (33 percent vs. 14 percent), especially in the fields of mathematics, engineering/engineering technologies, and computer/information sciences. 45 In the referenced study, it reported 15.1 male to 2.7 female enterin g the field of engineering. 46 The same study also reported that 20.8 and 27.8 left the engineering STEM field to a non STEM field or left post secondary schooling without a 42 Pam Newberry, T. Richard Grimsley, Lead th e Way Curricula, Pedagogy, and Professional Development: Activities Regarding Increasing E ngineeri ng and Technological Literacy of K12 students in the PLTW N Annual Conference of the American Society for Engineering Education. Chicago, IL. < https://peer.asee.org/648 > 43 Ibid. 44 This Longitudinal Study (BPS:96/01). This survey began in 1995 96 with a nationally representative sample of approximately 12,000 first time students who enrolled in postse condary education in 1995 96. These students were interviewed again in 1998 and, for the last time, in 2001, about 6 years after their initial college entry. The longitudinal design of BPS permits examination of student entrance, persistence, and attainmen who participated in the initial survey in 1996 and the two follow up surveys in 1998 and 2001 and who 45 Xianglei Chen. (July 2009). Stats in Brief: Students Who Study Science, Technology, Engineering, and Mathematics (STEM) in Po stsecondary Education. U.S. Department of Education. National Center for Education Statistics. http://files.eric.ed.gov/fulltext/ED506035.pdf 46 Ibid.

PAGE 42

42 degree altogether. 47 Studies indicate d that while young women were as competent as young men in the STEM disciplines they often tend ed to believe that science and technology were not relevant to their future career goals or they d id not find the learning contexts inviting. 48 For this reason, recruiting efforts of females to the field was increasing. History and Demand for Career Academies Focusing more precisely on future employer dema nd illuminated p a rt of the challenge, but there was also a problem at the supply end of the equation. Increasingly, U.S. employers complain ed that youn g potential employees were not equipped with the skills they need ed t o succeed in the 21st C entury workforce. 49 The Partnership for 21 st Century Skills, whose members include d such companies as Microsof t, Apple, Cisco and Pearson, had been equally critical of what it sees as obsolete and outmoded approaches to education, and was calling for more focus on the development of such (sic) communication. 50 This demand for prepared student s specifically in the STEM fields led to the increase and overwhelming support of career academies. Educational institutions have been criticized for being focused too exclusively on a few narrow pathways to 47 Xianglei Chen. (July 2009). Stats in Brief: Students Who Study Science, Technology, Engineering, and Mathematics (STEM) in Postsecondary Education. U.S. Department of Education. National Center for Education Statistics. http://files.eric.ed.gov/fulltext/ED506035.pdf 48 Sean Brophy Stacy Klein Merrideth Portsmore, and Advancing Engineering Education in P 12 Classrooms. Journal of Engineering Education 97: p.369 doi:10.1002/j.2168 9830.2008.tb0098 5.x 49 Pathways to Prosperity Project. (Februa Meeting the Challenge of Preparing Young Americans for the 21 st Harvard University College of Education. 50 Ibid.

PAGE 43

43 success. A path that is richl y diversified to a lign with the needs and interests of young people and better designed to meet the needs of a 21st C entury economy 51 Several organizations emerged to bridge the career and skill gap that occurred in the educational system. 52 Gr owth in the number of academies accelerated since 1990, and reached about 7,000 in 2010. 53 Before 2004, accurate counts of career academies were available only from three organized networks. In Philadelphia, the nonprofit Philadelphia Academies, Inc. had s upported career academ ies since 1969. After two nonprofit academies were established in 1981 in California academies were then fully funded in 1985. 54 The nonprofit National Academy Foundation (NAF) sponsored academies since 1982, and in 2014 16 supported academies in forty different states. 55 The number of academies in these three networks together grew to about a hundred in 1990, expanded to more than 700 in 2000, and exceeded 1,000 in 2010 56 At inception, career academies were especially focused on addr essing the needs of youth of urban populations with high risk drop out rates, but these academies ha d a much broader role in all types of institutions including high performing schools and non 51 Pathways to Prosperity Project. (February 2011). Meeting the Challenge of Preparing Young Americans for the 21 st Century. Harvard University College of Education. 52 Project Lead the Way (PLTW), National Academy Foundation (NAF), Ford Next Generation Learning, National Career Academy C oalition were among the organizations fo rmed to establish STEM programs and resources to answer to the government STEM reform and prepare young adults for the 21 st C entury workforce 53 Pathways to Prosp Meeting the Challenge of Preparing Young Americans for the 21 st Century. Harvard University College of Education. 54 55 National Academ y Foundation. www.naf.org 56

PAGE 44

44 traditional or specialty schools in 2014 16 57 Career academies expanded to the est performing school districts and schools of all types. Innovative career d students with advanced opportunities. 58 In this program, Advanced Placement level high school seniors t ook their senior year courses at work sites such as hospitals, environmental centers and law firms. 59 Branching from these three organizations, the National Career Academy Coalition partnered with organizations pledging to invest in career education. One organization driven by the Ford automobile company encompasse d the comprehensive system and package that schools implemented. 60 Career and interest themed academies serve d as the Ford NGL prac tice model for transforming the secondary school experience. 61 As a strong advocate on behalf o f academies, Ford Next Generation Learning ( NGL ) recognized that most skilled employment required a foundation of academic, Twenty F irst C entury, and technical kn owledge and skills that must be mastered in high school, as well as additional education b eyond high school. Ford believed that the most successful approach for high schools was one that infused 57 The Role of Career Academies in Education Impr ovement. (2009). Association for Career and Technical Education. < http://www.ncacinc.com/sites/default/files/media/research/ACTE%20Issue%2 0Brief%20Career_academi es.pdf > 58 Ibid. 59 The Role of Career Academies in Education Improvement. (2009). Association for Career and Technical Education. http://www.ncacinc.com/sites/default/files/media/research/ACTE%20Issue%20Brief%20Career_academie s.pdf 60 Ford Next Generation Learning. (2017). < https://fordngl.com/about > 61 Ibid.

PAGE 45

45 the high expectations and academic rigor of college preparato ry academic programs with the real world relevance and rigor of Career and Technical Education (CTE). 62 Similar to this approach, the National Academy Foundation (NAF) created a curriculum to market to secondary schools referenced in chapter one. NAF inc orporated an industry validated career themed curriculum in multiple disciplines that i ncorporated current industry standards and practices, literacy strategies, and STEM integration. 63 T his instructional strategy worked to provide students with a well roun ded skill set that went beyond academics and included the soft skills needed to succeed in based learning was centered on a continuum of work based learning experiences beginning with career awareness a ctivities, progressing to career exploration activities, and culminating in career preparation activities, including internships. 64 Industry certifications were the central component of the implementation of the Florida Career and Professional Education A ct passed in 2007. 65 The purpose of the 62 Ford Next Generation Learning. (2017). < https://fordngl.com/about > 63 National Academy Foundation (2015). www.naf.org 64 Ibid. 65 Refer to Florida State Statute Section 1003.4203 Digital materials, CAPE Digital Tool certificates, and technical assistance; Section 1003.491 Florida Career and Professional Education Act; Section 1003.492 Industry certified career education progra ms; Section 1003.493 Career and professional academies and career themed courses; Section 1003.4935 Middle grades career and professional academy courses and career themed courses; Section 1008.44 CAPE Industry Certification Funding List and CAPE Pos tsecondary Industry Certification Funding List; and Section 1011.62(1)(o) Calculation of additional full time equivalent membership based on successful completion of a career themed course pursuant to ss. 1003.491, 1003.492, and 1003.493, or courses with embedded CAPE industry certifications or CAPE Digital Tool certificates, and issuance of industry certification identified on the CAPE Industry Certification Funding List pursuant to rules adopted by the State Board of Education or CAPE Digital Tool certi ficates pursuant to s. 1003.4203. The Florida Statutes can be accessed at the following web link: http://leg.state.fl.us

PAGE 46

46 Act was to provide a statewide planning partnership between the business and education communities in order to attract, expand, and retain targeted, high value industry and to sustain a strong, knowledge based economy 66 Cost Estimation and Funding Grants were a large source of the funds that a school district used to generate funds to operate career and technical education, STEM programs, and purchase supplies. A total of 1 535 schools in 634 districts received federal Smaller Learning Community (SLC) grants from 2000 through 2007 according to the US Department of 67 The FDOE received federal funding from the U.S. Department of Education for Career and Tech nical Education (CTE) under the Carl D. Perkins Career and Technical Education Act of 2006 68 and for adult education (AE) and family literacy under the Adult Education and Family Literacy Act of 1998. 69 FDOE awarded sub grants to eligible providers to admini ster local programs. FDOE monitor ed providers to ensure and adult education/family literacy. 70 Each state had procedures for reviewing and approving applications for sub grants prov iding technical assistance, evaluating 66 Florida Department of Education. (2015). Secondary Industry Certification. http://www.fldoe.org/academics/career adult edu/industry certification/secondary.stml 67 U.S. Department of Education. Programs: Smaller Learning Communities Program. < https://www2.ed.gov/programs/slcp/index.html > 68 See https://www2.ed.gov/policy/sectech/leg/perkins/index.html 69 See https://www2.ed.gov/policy/adulted/leg/legis.html?exp=0 70 Career and Adult Education: Florida Department of Education. Quality Assurance and Compliance Onsite Monitoring Visit Report: (20 15). < http://www.fldoe.org/core/fileparse.php/7526/urlt/1415Collier.pdf >

PAGE 47

47 projects, and for performing other administrative responsibili ties the state determined were necessary to ensure compliance with applicable statutes and regulations. The division was also required to overs ee the performance of sub grantees in the enf orcement of all laws and rules 71 FTE (Full time Enrollment) state funding per student was spread out among the resources and course enrollment in each grade level. 72 To outfit an Engineering lab, the start up to tal cost for one high school lab with 20 student s tations equipped for all five Engineering courses is approximately $95,500. The cost will be less if the school already has equipment and/or computers that meet PLTW specifications. The Introduction to Engi neering course total cost is $40,465 with computers ($24,117), equipment ( $8,943 ), supplies ($167), furniture ($5,420), software requirements ($1,251) and consumables ($567) included in the total. 73 Summary Organizations such as National Academy Foundation (NAF) emerged to bridge the career and skill gap occurring in the educational system. The focus on STEM (Science, Technology, Engineering, and Math) integration in school programs also utilized programs such as Project Lead the Way (PLTW) to increase innovative thinking, career centered learning, and industry based educ ation. The literature suggested the implementation o f career themed academies coupled with a STEM focus within schools attributed to positive gains to mold career ready students. The impact of NAF 71 Sections 1001.03(8 ) and 1008.32, Florida Statutes 72 See http://www.fldoe.org/finance/fl edu finance program fefp/fte info/ 73 High Sc hools that Work Presents a Pre E ngineering Program of Study. Pre Engineering and Project Lead the Way. Southern Regional Education Board. < https://www.sreb.org/sites/main/files/file attachments/01v56_engineering_web1.pdf >

PAGE 48

48 enrollment on high stakes testing had yet to be isolated as the literature focused on career ready indicators. Recruitment of students to connect traditional education subject s to an applicable skill remaine d an initiative in public, charter, and academy schools alike.

PAGE 49

49 CHAPTER 3 METHODS For the research, eight high schools were selected in a Southwest Florida district. Since the results of End of Course test scores for NAF enrollees were compared a gainst those for non enrollees, there were two sets of data that were collected: a list of student scores on End of Course tests and a list of students who were participating in NAF enrolled academies at the time those scores were achieved. In order to com pare the results by ge nder, data were collected Two years of NAF enrollment and EOC Geometry scores (2014 15 and 2015 16) were collected and analyzed separately due to the changing scale scores implemented by the Florida Department of Education Office of Assessments. Two years of data were used to gather a trend of results and provide additional reliability of findings over time. Since numerous NAF academies were implemented in each school and the Engineering Academy was consistent across all schools, en gineering was selected. The engineering course, Introduction to Engineering, was the corresponding course to the students taking the Geometry EOC. Research Design The 2014 2015 and 2015 2016 End of Course test scores were utilized for the data collection. To ensure appropriate and meaningful data, only scores from the academic area that have served as exit standards for high school s tudents (Geometry) were utilized (Table 3 1) After determining what data w ere needed and the schools that would be involved, the actual data collection was compiled through the district Data warehouse site through written approval and permission for research 1 Once the necessary data 1 See Appendix A.

PAGE 50

50 w ere collected, students were anonymously compil ed into a master spreadsheet with columns enrolled Engineering program NAF student (Y or N), test score, and the gender of each student. For the analysis of these data, the information was uploaded into SPSS for the statistical analysis. The population sel ection included all students district wide who took the Geometry EOC in the 2014 15 and 2015 16 school years. The Geometry exam was chosen due to the course enrollments coinciding with NAF membership across the school district. Also, two years of data we re selected to provide a two year scope of performance scores since the NAF programs were newly implemented in 2014 15 and were increasing enrollment in the 2 015 16 school year. This increased the sample size of the NAF enrolled students compare d with the Geometry EOC performance. Geometry score ranges and levels for 2014 15 were shifted to new score ranges per level in 2015 16. Comparison of the two score sets for each year were calculated where level 3 still remained the proficient level ; however these data were not combined since th e test score ranges and corresponding levels changed Table 3 2 and Table 3 3 show these score ranges for 2014 15 while Table 3 4 and 3 5 show the ranges for 2015 16. To examine the research question, a two way analysis of v ariance (ANOVA) was conducted to assess if mean differences exist. The two way ANOVA was an appropriate statistical analysis when the purpose of research was to compare two or more discrete groups on a continuous dependent variable that were measured more than once. For this research, the continuous dependent variables were the EOC Geometry test; the independent variable has the following groups (NAF Engineering

PAGE 51

51 Enrollment vs. Non NAF Engineering Enrollment and Female NAF Engineering vs. Male NAF Enginee ring vs. Female Non NAF Engineering vs. Male Non NAF Engineering). Data Analysis Plan The ANOVA used the F test, which research reflects the overall comparison on whether group means differ. If the obtained F were larger than the critical F, the null hypo thesis was rejected. The results of the two way ANOVA present ed findings for the main effect and evaluate d the overall differences by time (within subjects) and also separately by group (between subjects). The dependent variable should be approximately normally distributed for each level of the independent variable. For the purpose of the research, a level of significance or 0.05 was selected in order to determine whether to accept or reject the null hypotheses as well. 2 The P valu e, or calculated probability, was the probability of finding the observed, or more extreme, results when the null hypothesis (H 0 ) of a study question was true was being tested. The null hypothesis was a hypothesis of "no difference" e.g. no difference between EOC test scores and groups (NAF, Non NAF, male or female). The t erm significance level (alpha) was used to refer to a pre chosen probability and the term "P value" was used to indicate a probability calculated following the study. The alternative hypothesis (H a ) was the opposite of the null hypothesis; which was the hy pothesis set out to investigate. For the purpose of this study, H a : EOC Geometry scores differ by (independent variable Group 1 vs. Group 2) NAF enrollment vs. Non NAF enrollment; and H a : EOC Geometry scores differ by gender (independent 2 Statistics Solutions. (2016). Data analysis plan: One Within, One Between ANOVA [WWW Document]. Retrieved from ht tp://www.statisticssolutions.com/data analysis pla n one within one between anova/

PAGE 52

52 variable: group 1 vs. group 2 including NAF enrollment analysis by gender). This test was run using a two way ANOVA to calculate mean, standard deviation, and significance. If the P value w ere less than the chosen significance level, then the null hypothesis would be rejec ted. For this study, the significance level at which the H 0 would be rejected was set at .05 or less than 5 percent or less than 5 five in one hundred chance of being wrong. Research Questions 1. Did EOC Geometry scores differ by Enrollment in a NAF Engine ering program or non Enrollment in a NAF Engineering program in 2014 15? Ho: EOC Geometry scores did not differ by group NAF enrollment vs. Non NAF enrollment (Independent variable Group 1 vs. Group 2) in 2014 15. Ha: EOC Geometry scores differ ed by (indep endent variable Group 1 vs. Group 2) NAF enrollment vs. Non NAF enrollment in 2014 15. 2. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2014 15? Ho: EOC Geometry scores did not differ by gender when enrolled in NAF Engineering or not enrolled in NAF Engineering. (independent variable: group 1 vs. group 2) in 2014 15. Ha: EOC Geometry scores differ ed by gender (independent variable: group 1 vs. group 2) in 2014 15. 3. Did EOC Geometry scores differ by Enrollment in a NAF Engineering program or non Enrollment in a NAF Engineering program in 2015 16? Ho: EOC Geometry scores did not differ by group NAF enrollment vs. Non NAF enrollment (Ind ependent variable Group 1 vs. Group 2) in 2015 16. Ha: EOC Geometry scores differed by (independent variable Group 1 vs. Group 2) NAF enrollment vs. Non NAF enrollment in 2015 16. 4. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2015 16?

PAGE 53

53 Ho: EOC Geometry scores did not differ by gender when enrolled in NAF Engi neering or n ot enrolled in NAF Engineering (independent variable: group 1 vs. group 2) in 2015 16. Ha: EOC Geometry scores differ ed by gender (independent variable: group 1 vs. group 2) in 2015 16. 5. Was NAF Engineering enrollment significant in the Geometry EOC score results and d id this significance demonstrate cost effectiveness of the NAF Engineering program? Ho: NAF Engineering enrollment was not significant on Geometry EOC score results thus not demonstrating cost effectiveness of the program. Ha: NAF Engineering enrollment was significant on Geometry EOC score results thus demonstrating cost effectiveness of the program. Summary The research methods included population selection, research design, data analysis pl an, and outline of research questions including hypotheses. Population included 9 th and 10 th grade students in the selected school district who were enrolled in Geometry or Geometry Honors courses in the 2014 15 and 2015 16 school years. The selection in cluded 2,513 students (1,312 males and 1,286 females) in 2014 15 (Table 7) and 2,264 students (1,288 males and 1,185 females) in 2015 16 (Table 12). Also identified were NAF Engineering enrolled students from the population selection. In 2014 1 5, there w ere eight y five NAF engineering students (seventy two male s and thirteen females) concurrently enrolled in Geometry courses. In 2015 16, there were 210 NAF engineering students (169 males and 41 females) concurrently enrolled in Geometry courses. Data w e re gathered with no names present and at no time were students identi fied by any means. All data were centrally compiled by the district Career and Technical office and collected by the researcher. The research questions guide d

PAGE 54

5 4 the direction of the study to determine any significance difference in performance on the Geometry EOC in the subgroups. Table 3 1. FSA EOC Corresponding Course List 2014 15 and 2015 16 Geometry EOC Course Geometry 1206310 Geometry Honors 1206320 IB Middle Years Program Geometry Honors 1206810 Pre AICE Mathematics 2 1209820 Table 3 2 Geometry Score Ranges and Level 2014 15 3 Level 1 Level 2 Level 3 (Passing) Level 4 Level 5 325 369 370 395 396 417 418 433 434 475 Table 3 3 2014 15 Florida End of Course Assessments Achievement Level Policy Definitions 4 3 FCAT 2.0 and Florida EOC Assessments Achievement Levels (February 2014). Florida Department of Education/Office of Assessment < http://www.fldoe.org/core/fileparse.php/3/urlt/achlevel.pdf > 4 Ibid. Level Description L e v e l 5 S t ud e n ts a t t hi s l e vel d e m o ns t ra te m a s te r y o f t h e m o s t c hall e ngin g c o n te n t o f t h e N ext G e n e r a t io n S un s hin e St a te St anda r d s L e v e l 4 S t ud e n ts a t t hi s l e vel d e m o ns t ra te a n ab o ve sa t isfa c t o r y l e vel o f s u cce s s w i th t h e c hall e ngin g c o n te n t o f t h e N ext G e n e r a t io n S un s hin e S t a te St anda r d s L e v e l 3 S t ud e n ts a t t hi s l e vel d e m o ns t ra te a sa t isf a c t o r y l e v el o f su c c e s s w i th t h e c hall e ngin g c o n te n t o f t h e N ext G e n e r a t io n S un s hin e S t a te St anda r d s L e v e l 2 S t ud e n ts a t t hi s l e vel d e m o ns t ra te a b e l o w s a t isfa c t o r y l e vel o f s u cce s s w i th t h e c hall e ngin g c o n te n t o f t h e N ext G e n e r a t io n S un s hin e St a te St anda r d s L e v e l 1 S t ud e n ts a t t hi s l e vel d e m o ns t ra te a n inad e qua te l e v el o f su c ce s s w i th t h e c hall e ngin g c o n te n t o f t h e N ext G e n e r a t io n S un s hin e St a te St anda r d s

PAGE 55

55 Table 3 4 2015 16 Florida End of Course Assessments Achie vement Level Policy Definitions 5 Level 1 Level 2 Level 3 Level 4 Level 5 Inadequate: Highly likely to need substantial support for the next grade Below Satisfactory: Likely to need substantial support for the next grade Satisfactory: May need additional support for the next grade Proficient: Likely to excel in the next grade Mastery: Highly likel y to excel in the next grade Table 3 5 2015 16 FSA EOC Scale Scores for Each Achievement Level 6 Assessment Level 1 Level 2 Level 3 Level 4 Level 5 EOC Scale Scores 425 485 486 498 499 520 521 532 533 575 5 Florida Standards Assessments: 2015 16 FSA Algebra 1, Algebra 2, and Geometry End of Course Assessments Fact Sheet. (January 2016). Bureau of K 12 Student Assessment. < http://www.fldoe.org/core/fileparse.php/5663/urlt/FSAEOC1516.pdf > 6 Ibid.

PAGE 56

56 CHAPTER 4 PRESENTATION OF RESULTS The results of the EOC Geometry performance with NAF vs. Non NAF enrollment and Gender correlations were outlined As presented in chapter one, the resear ch question addressed was: 1. Did EOC Geometry scores differ by Enrollment in a NAF Engineerin g program or non Enrollment in a NAF Engine ering program in 2014 15?; 2. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2014 15 ?; 3. Did EOC Geometry scores differ by Enrollment in a NAF Engineering program or non Enrollment in a NAF Engineerin g program in 2015 16?; 4. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2015 16? ; and 5. Was NAF Engineering enrollment significant in the Geometry EOC sco re results and did this significance demonstrate cost effectiveness of the NAF Engineering program? This chapter presented the results derived by calculating the variance of Geometry EOC performance by NAF, non NAF, and male, female subgroups by using ANO VA as presented in the methodology in chapter three. The results were presented by school year. Performance results from the 2014 2015 school year were presented first, followed by the results fro m the 2015 2016 school year (Tables 3 3 and 3 4) Variance of Results 2014 15 A two way ANOVA was conducted that examined the effect of gender and NAF enrollment on EOC Geometry assessment performance for 2014 15. The descriptive

PAGE 57

57 statistics of NAF and non NAF students are listed (Table 4 1) and NAF/Non NAF by gen der (Table 4 2) There was a not a significant interaction between the effects of gender and NAF enrollment on EOC Geometry performance (p=.558), but there was significant difference in the mean significance (p=.000) on EOC Geometry scores by NAF enrollm ent. F(1, 2594) = .343; with N=2598 (Table 4 3 ). The null hypothesis was accepted that EOC Geometry scor es did not differ by gender when enrolled in NAF Engineering or not enrolled i n NAF Engineering (independent variable: group 1 vs. group 2) in 2014 1 5 (Table 4 4 ). The 2014 15 mean scores of NAF (N=85) versus non NAF (N= 2513) were 428.56 (std. dev. 18.333) and 410.78 (23.806) respectively (Table 4 5). The estimated marginal means of scale for the 2014 15 Geometry scores for NAF and non NAF students were illustrated. The graph demonstrates the disparity betw 1). The null hypothesis for the second research question was reje cted that EOC Geometry scores did not differ by group NAF enrollment vs. Non NAF enrollment (Independent variable Group 1 vs. Group 2) in 2014 15 as outlined in Table 4 6. Variance of Results 2015 16 A two way ANOVA was conducted that examined the effect of gender and NAF enrollment on EO C Geometry assessment perfor mance for 2015 16. There was not a significant interaction between the effects of gender and NAF enrollment on EOC Geometry performance (p=.166), but there was significant difference in the mean significance demonstrated by p < .05 on EOC Geometry scores by NAF enrollment. F(1, 2469) = 1.915; with N=2474 ( Table 4 7 ).

PAGE 58

58 was rejec ted that EOC Geometry scores did not differ by group NAF enrollment vs. Non NAF enrollment (Independent va riable Group 1 vs. Group 2) in 2015 16. The estimated marginal means of scale for the 2015 16 Geometry score s for NAF and non NAF students we re illus trated. The graph demonstrated (Figure 4 2) The 2015 1 6 mean scores of NAF (N=210) versus non NAF (N= 2264) were 526.55 (std. dev. 18.756) and 503.21 (24.190) respectively listed in Tables 4 8 and 4 9 Although no significant difference was found in gender and NAF enrollment in 2015 16, NAF enrollment alone demonstrated significance over non NAF students (p < .05) on the Geometry EOC exam. F(1, 2469) = 106.510 as indicated in Table 4 10 The fourth res was acce pted that EOC Geometry scores did not differ by gender when enrolled in NAF Engineering or not enrolled in NAF Engineering (independent variable: group 1 vs. group 2) in 2015 16. ypothesis was rejected that NAF Engineering enrollment was significant on Geometry EOC score results thus demonstrating cost effectiveness of the program. Limitations in Results The two years of Geometry EOC scores were scaled differently per changes by th e Florida Department of Education Office of Assessment with 396 as a passing score in 2014 15 and 499 as a passing score in 2015 2016 with the levels of proficiency levels also adjusted. For this reason, this study did not compare mean scores between the two years but instead analyzed the standard deviation between subgroups and significance of NAF/non NAF enrollment and gender on EOC Geometry performance.

PAGE 59

59 As indicated in Chapter T wo, the 2015 16 Geometry test underwent significant changes in scoring an d the levels that indicated performance proficiency. In 2014 15, the scores ranged from 325 to 475. In 2015 16, the state migrated to the FSA platform for the Geometry EOC exam that changed the scoring ranges and matching proficiency levels. This is con sidered a limitation as an ideal comparison would not exist since the scales and exams are different. In 2014 15, thirteen females and seventy two males were NAF Engineering students who took the Geometry EOC in 2014 15. In 2015 2016, 41 females and 169 m ales were NAF Engineering students who took the Geometry EOC in 2015 16. Six of the eight high schools in this study were in the second or third full year of NAF Engineering implementation that demonstrated increasing enrollment who took the Geometry exam from 2014 15 to 2015 16 school year. In 2014 15, students who were enrolled in the NAF engineering program scored a mean score of 428 (males) and 431.69 (females) that equated to a l evel 4 proficiency. Students not enrolled in the engineering program sc ored means of 410.50 (females) and 411.07 (males) that was a l evel 3 proficiency according to Florida State assessment pe rformance scales as indicated in Table 4 6 In 2015 16, students who were enrolled in the NAF engineering program scored a mean score of 527.45 (males) and 522.85 (females) that equated to Level 4 proficiency. Students not enrolled in the NAF engineering program scored a mean of 502.54 (males) and 503.83 (females ) that was Level 3 proficiency according to the scales. The 2015 16 estimated marginal means for gender and NAF and non NAF performance were indicated in Tables 4 11 and 4 12

PAGE 60

60 While gender did not show significant differences in the marginal means NAF enrollment yield ed a higher mean Geometry EOC sco re among males and fem ales in the study. L imitation s in the study were that the scaled scores per the two years were scaled differently not creating a scenario of comparison and that NAF enrollment might attract a higher performing student in the recruitm ent process. Cost Effectiveness Analysis Based on the performance and higher mean test scores in the two years reported (2014 15 and 2015 16 school years), the expenditures could be justified that National Academy Foundation Engineering membership may in crease scores on the Geometry EOC exam. This demonstrated that there was some level of significance on EOC Geometry exam when enrolled in the NAF Engineering program to validate the minimal cost per student and Carl D. Perkins grant illustrated in this analysis The compilation of grants, FTE dollars, technology budget, and budgetary items in human capital for professional development of the academy were attributed as dollars receiving some return on investment Specifically, the supply and materials cost per twenty Introductory of Engineering enrolled student s w ere $929.73 1 In 2014 15, there were eighty five students enrolled in the Introductory to Engineering course in the selected school district yielding the cost per student at $46.49 and a total supply and materials cost of $3,951.65 In 2015 16, there were 210 students enrolled in the Introductory to Engineering course yielding the cost per student at $46.49 and a total supply and material cost of $9, 762.90. 1 R eferencing the CTE workbook of supply costs for 2014 15 and 2015 16. (District name removed for anonymity).

PAGE 61

61 Training for the Project Lead the Way (PLTW) Engineering Pathways was an estimated $4,5 00 to $5 000 per teacher depending on location of the training 2 In 2014 15, ten teachers in the district were trained and in 2015 16, an additional three teachers were trained in 2015 16 in the Engineering Pathways workshop that was used for the Introduction to Engineering NAF course Depending on the location, the amount spent in 2014 15 and 2015 16 totaled $58,500 to $65,000. This equated to an expense of $198.31 to $220.34 cost per s tudent enrolled (295) in the program over a two year period. In 2014 15, grants totaling $435,051.00 for secondary Career and Technical education programs were received. Carl D. Perkins grant money (Perkins Vocational and Technical Secondary grant $410,3 55) 3 was also allocated and during a review of the district via a state quality assurance review, purchase orders, local grants, and technology purchases with grant money were tagged The district furnished a five year information technology plan that incl uded a specialized plan for equipment with a $1,000 threshold for equipment specifically identified. All equipment purchased with grant funds were tagged and were located during the review. 4 In 2015 16, the Carl D. Perkins grant funded Career and Technica l educat ion in the allocated amount of 2 Referenc ing the Project Lead the Way (PLTW) Teacher Training Excel spreadsheet outlined in the Appendix. Also, the cost was $4,700 $5,000 depending on the location of the training but this was not specified by the district. 3 District Budget Book. Perkins Vocatio nal and Technical Secondary Grant. (2014 15). Project no. 154206 4 Career and Adult Education: Florida Department of Education. Quality Assurance and Compliance Onsite Monitoring Visit Report: (2015).

PAGE 62

62 $ 423,887 to the school district a 2013 14 roll over of $ 24,156 totaling $ 448,043 for the selected school district 5 With the allocated funds, resources such as supplies recommended for the Engineering course by Pro ject Lead the Way and NAF Engineering guidelines were purchased for each school. Technology needs were generated per the engineering course guidelines and were shared through the district technology department as many computers and resources were allocated from those resources and funds. 6 The analysis of funding and spending in relation to performance specifically on the Geometry EOC was important to validate the program on some level. Although many variables exist to determine cost effectiven ess, one variable of NAF Engineering enrollment positively impact ed Geometry test performance that can justify expenses to some degree in 2014 15 and 2015 16 For the two year study, the supply cost per student was $46.49 and the teacher training cost was estimated at $198.31 to $220.34 per student, totaling the cost per student of supplies plus teacher training to $244.80 to $266.83 per student. The increased Geometry EOC scores per this enrollment coupled with the expense per student showed that the pro gram was a good investment based on these factors. Table 4 1. 2014 15 Geometry EOC scores of NAF/Non NAF Students Descriptive Statistics Dependent Variable Mean Std. Deviation N Non NAF 410.78 23.806 2513 NAF 428.56 18.333 85 Total 411.36 23.855 2598 5 Carl D. Perkins Career and Technical Education Act of 2006. (2015 16). http://www.fldoe.org/core/fileparse.php/13147/urlt/2015 16 Secondary Roll Forward. pdf 6 Career and Technical Education Director (2015). Career and Technical Education Department. District School of Study. (Name and school district removed for anonymity).

PAGE 63

63 Table 4 2. 2014 15 Geometry EOC scores of NAF/Non NAF Students by Gender Descriptive Statistics Dependent Variable Gender M=1, F=0 Mean Std. Deviation N Non NAF Female 410.50 23.745 1273 Male 411.07 23.874 1240 Total 410.78 23.806 2513 NAF Female 431.69 22.537 13 Male 428.00 17.598 72 Total 428.56 18.333 85 Total Female 410.72 23.820 1286 Male 412.00 23.882 1312 Total 411.36 23.855 2598 Table 4 3. Univariate Analysis of Variance Tests of Between Subjects Effects 2014 15 Geometry EOC vs. NAF vs. Gender Source Type III Sum of Squares df Mean Square F Sig. Corrected Model 26318.554 a 3 8772.851 15.678 .000 Intercept 30590141.360 1 30590141.360 54666.902 .000 Gender 108.800 1 108.800 .194 .659 NAF 15726.235 1 15726.235 28.104 .000 GENDER*NAF 192.165 1 192.165 .343 .558 Error 1451533.253 2594 559.573 Total 441113389.000 2598 Corrected Total 1477851.808 2597 a. R Squared = .018 (Adjusted R Squared = .017 Table 4 4. 2014 15 Estimated Marginal Means: Gender Scale 95% Confidence Interval Gender M=1 F=0 Mean Std. Error Lower Bound Upper Bound Female 421.109 3.297 414.643 427.574 Male 419.523 1.434 416.712 422.335 Table 4 5. 2014 15 Estimated Marginal Means : NAF Scale 95% Confidence Interval NAF Mean Std. Error Lower Bound Upper Bound Non NAF 410.786 .472 409.861 411.711 NAF 429.846 3.564 422.857 436.835

PAGE 64

64 Figure 4 1. 2014 15 Estimated Marginal Means of Scale Graph Comparison Table 4 6. 2014 15 Estimated Marginal Means: Gender and NAF Scale 95% Confidence Interval Gender M=1 F=0 NAF Mean Std. Error Lower Bound Upper Bound Female Non NAF 410. 525 663 409. 225 411. 826 NAF 431.692 6.561 418.827 444.557 Male Non NAF 411.047 .671 409.730 412.363 NAF 428.000 2.788 422.533 433.467 Table 4 7. Univariate Analysis of Variance Tests of Between Subjects Effects 2015 16 Geometry EOC vs. NAF vs. Gender Source Type III Sum of Squares df Mean Square F Sig. Corrected Model 107773.717 a 4 26943.429 47.699 .000 Intercept 4037059.084 1 4037059.084 7147.032 .000 Gender 1768.972 2 884.486 1.566 .209 NAF 60163.018 1 60163.018 106.510 .000 GENDER*NAF 1081.979 1 1081.979 1.915 .166 Error 1394634.700 2469 564.858 Total 632915110.000 2474 Corrected Total 1502408.417 2473 a. R Squared = .072 (Adjusted R Squared = .070

PAGE 65

65 Figure 4 2. 2015 16 Estimated Marginal Means of Scale Graph Comparison Gender NAF identifier Table 4 8. Descriptive Statistics: 2015 16 Mean Geometry EOC Scores for NAF/Non NAF and Gender Comparisons Ge nder M=1 F=0 NAF Identifier Mean Std. Deviation N Female Non NAF 503.83 23. 589 1 144 NAF 522.85 23. 205 41 Total 504.49 23. 789 1185 Male Non NAF 502.54 24.767 1119 NAF 527.45 17. 778 169 Total 505.81 25.395 1288 Total Non NAF 503.21 24.190 2264 NAF 526.55 18.756 210 Total 505.19 24.648 2474 Table 4 9. 2015 16 Estimated Marginal Means: NAF Scale 95% Confidence Interval NAF Mean Std. Error Lower Bound Upper Bound Non NAF 515.792 7.929 500.243 531.340 NAF 525.152 2.069 521.095 529.208

PAGE 66

66 Table 4 10. Univariate Analysis of Variance Tests of Between Subjects Effects 2015 16 Geometry EOC vs. NAF vs. Gender Source Type III Sum of Squares df Mean Square F Sig. Corrected Model 107773.717 a 4 26943.429 47.699 .000 Intercept 4037059.084 1 4037059.084 7147.032 .000 Gender 1768.972 2 884.486 1.566 .209 NAF 60163.018 1 60163.018 106.510 .000 GENDER*NAF 1081.979 1 1081.979 1.915 .166 Error 1394634.700 2469 564.858 Total 632915110.000 2474 Corrected Total 1502408.417 2473 a. R Squared = .072 (Adjusted R Squared = .070 Table 4 11. 2015 16 Estimated Marginal Means: Gender Scale 95% Confidence Interval Gender Mean Std. Error Lower Bound Upper Bound Female 513.344 1.889 509.640 517.048 Male 514.995 .981 513.072 516.918 Table 4 12. 2015 16 E stimated Marginal Means: Gender and NAF Scale 95% Confidence Interval Gender M=1 F=0 NAF Mean Std. Error Lower Bound Upper Bound Female Non NAF 503.835 .703 502.457 505.213 NAF 522.854 3.712 515.575 530.132 Male Non NAF 502.540 .710 501.147 503.933 NAF 527.450 1.828 523.865 531.035

PAGE 67

67 CHAPTER 5 CONCLUSIONS AND IMPLICATIONS The demands of high stake tests not only affect ed what is and what is not taught within courses it is valid to recognize the importance of career readiness to students, parents and industries The greater need for career ready students in the STEM fields led to the increased funding and focus on Career and Technical Education (CTE). Geometry End of Course test minimum scores that were required for graduation in 2014 15 and 2015 16 also fueled the infiltration of applicable real world learning experiences with the anticipated increase of pass rate and score averages with enrollment in programs such as the National Academy Foundation Engineering program. This study allowed the researcher to examine the following research questions: 1. Did EOC Geometry scores differ by Enrollment in a NAF Engineering program or non Enrollment in a NAF Eng ineering program in 2014 15? 2. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: G roup 1 vs. Group 2) in 2014 15? 3. Did EOC Geometry scores differ by Enrollme nt in a NAF Engineering program or non Enrollment in a NAF Engineering program in 2015 16? 4. Did EOC Geometry scores differ by Gender when enrolled in a NAF Engineering program or not Enrolled in a NAF Engineering program (independent variable: Group 1 vs. Group 2) in 2015 16? 5. Was NAF Engineering enrollment significant in the Geometry EOC score resul ts and d id this significance demonstrate cost effectiveness of the NAF Engineering program? Significance of the Study The significance of this study was two fold. First, the study determined the performance of National Academy Foundation Engineering studen ts on the Geometry

PAGE 68

68 End of Course assessment compared to Non NAF Engineering students. The study also analyzed the performance of males and females in comparison to each other while considering NAF enrollment. The study was conducted over a two year period to gather a significant amount of data to identify any trends. Second, the study was conducted to identify cost effectiveness of the NAF program based on this performance. Supply costs as well as teacher training expenses were utilized to gather a per c ost student amount that would be spent above and beyond the FTE allowance. The two year study showed that NAF Engineering enrollment played a significant positive impact on Geometry EOC test score averages. The NAF Engineering program was also determined to be cost effective based on the increased test score averages and the relatively small cost per student allocated using the Carl D. Perkins grants and FTE expenditures. This study supports the work of Bottoms and Anthony, Bottoms and Uhn, Rethwisch, H aynes ,Starobin, Laanan and Schenk as well as the Texas study by Van Overschelde outlined in Chapter 2. The studies indicated higher performing levels on exams achieved by PLTW enrolled students. Implications of the Study The major implications of thi s study were that it illustrated how performance on the Geometry EOC was impacted by NAF Engineering enrollment over a two year period. Gender performance also was determined to provide an analysis of whether NAF enrollment showed any impact. Using a two way analysis of variance, ANOVA, the similarities and differences within the groups were compared within the NAF membership itself, and then compared across all groups in the ensuing explanations.

PAGE 69

69 The implications of the study allow for justification and p urpose for the NAF engineering program especially due to the increased test scores that could be attributed to real world application of math concepts and critical thinking skills utilized in the course. Validating cost of the program illustrated the purp oseful spending or dollar allocation of the grants as well as the local and state expenses. Limitations and Assumptions The potential for human error in the data collection due to the lack of an effic ient way to collect and mark data indicating NAF students and gender identifiers was present No current means existed that w ould allow test data to easily be separated by NAF enrollment status coupled with gender. This information may be uploaded into the Data warehouse system for greater accuracy and c omparison purpose s NAF enrollment was housed in a separate program provided by NAF software and exported make it efficient to cross reference for future studies. The Flo rida Education Finance Program (FEFP) determined funding including programs, instructional materials at $7,742 per student. 1 Local officials must make the complex and difficult decisions to fund programs from this allocated amount. The amount of FTE doll ars used for NAF and non NAF students was not used to determine cost effectiveness since these dollars would be allocated elsewhere if the NAF Engin eering program did not exist. Also, t eacher salary was not included as the 1 See Appendix B.

PAGE 70

70 teacher s employed to teach the E ngineering program would be teaching another discipline if t he NAF program did not exist. The supplemental figures located in Appendix B demonstrate the monies geared toward the instructional programs, resources, and personnel requirements to operate ; 2 however, this figure is not included to determine cost effectiveness of the NAF Engineering program since these dollars would be allocated to other areas if the engineering program did not exist. The district of study budge t expense allocations that 63.6 p ercent of the budget is allocated for direct instruc tion; technology services of .7 percent an d instructional support at 8.2 percent These dollars are also not specific to any extra costs need ed to operate the NAF Engineering program. Local officials are tasked with allocating these dollars to fund programs. Conclusions N four essential elements of practice: academy development & structure, curriculum & instructi on, advisory board, and work based learning. These elements buil t on each other to engage students, support school and district priorities, and give businesses the opportunity to connect with students and offer internships while a member of the program. 3 Students who immerse d themselves in career and technical education ha d a connection to the real world which may validate traditional math studies. As reported in this study, the positive impact that NAF engineering enrollment may play on performance of the Geometry EOC could be a precursor to market the measurable impact to students. Specifically, the students 2 Ibid. 3 National Academy Foundation (NAF ). < www.naf.org >

PAGE 71

71 enrolled in NAF (as reported in Chapter four) scored at means one level of proficiency higher than non NAF engineering students (Level 4 to Level 3 in comparison). Immersio n into STEM focused curricula was tied to critical thinking, decision making, and connection to real world applicable learning. Basic concepts learned in this traditional Geometry course then applied to the engineering courses mat erial may conclude higher performance on standardized tests. This may be a broad statement since there are multiple variables affecting student performance: attendance, parental support, learning abilities, intelligence, socio economic limitations, outsid e resources or factors to name a f ew. Although gender did not have major influence on performance, the enrollment of females into engineering cours es from 2014 15 to 2015 16 showed growing interest in the program If programs within public schools were funded to provide students career ready skills, students may gravitate to those programs rather than seeking other alternatives. The study approached a static model employing a conservative measure of cost effectiveness of the Introduction to Engineering program. Other programs may have and PLTW programs. Grant money received may also have been allocated elsewhere if the NAF Engineering program did not exist. Cost of these other programs were not compared in this study to determine the measure of effectiveness; however, the cost of the Intro to Engineering program was analyzed for effectiveness based on Geometry exam performance to gain insight on expenditures specifi c to that course. The funds and required program analysis reviewed by the state and local board ensure parents

PAGE 72

72 that there were quantitative benefits over and above the career ready skil ls they seek for their children It is suggested that research in the area for future consideration should explore the possibility of an association between math and science achievement level and a student's desire to continue a college or career path in engineering or similar math and science related paths and the correlati on between their other areas of achievement. In addition, STEM certification may need to be evaluated as this certification may yield higher performance. Further investigation into the growth of students over time may also yield insight into the strengt h of the NAF and PLTW programs over time for cohort students. Also, further research needs to be evaluated on the cost effectiveness of additional NAF engineering courses or other NAF academic tract programs and benefits that may serve the students other th an math and science test score s

PAGE 73

73 APPENDIX A RESEARCH REQUEST

PAGE 74

74 APPENDIX B SUPPLEMENTAL FIGURES Figure B 1 Florida Education Finance Program (FEFP) Equation 4 Figure B 2 2015 District of Study Budget Expense Allocations 5 4 Board Implementation PowerPoint. (2015) District Study Report. The school district name is removed for anonymity. 5 Ibid.

PAGE 75

75 LIST OF REFERENCES Andrew Eyles and Stephen Machin. (Dec. 14, 2016). Academy Schools and the Transformation of the English Education S ystem Oxford University Press. < https://blog.oup.com/2016/12/academy schools english education system/ > Economic Policy Volume 31, Issue 87, Pag es 453 501, Betsy Brand. (November 2009). High School Career Academies: A 40 Year Proven Model for Improving College and Career Readiness. The National Career Academy Coalition. < http://www.aypf.org/documents/092409CareerAcademiesPolicyPaper.pdf > Board Implementation Powe rPoint. (2016) District Study Report. (Name and District Removed for Anonymity). Footprints for Healt hy Youth Development. (2016). University of Colorado at Boulder. Institute of Behavioral Science. < http://www.blueprintsprograms.com/program costs/career academies > Career and Adult Education: Florida Department of Education. Quality Assurance a nd Compliance < http://www.fldoe.org/core/fileparse.php/7526/urlt/1415Collier.pdf > Career and Technical Education (2017). Advance CTE. < https://www.careertech.org/cte > Carl D. Perkins Career and Technical Education Act of 2006. (2015 16). http://www.fldoe.org/core/fileparse.php/13147/urlt/2015 16 Secondary Roll Forward.pdf David Rethwisch. (2014). on Achievement Outcomes in Iowa. f Chemical and Biochemical Engineering University of Iowa. David Stern, Charles Dayton, and Marilyn Raby. (February 2010). Career Academies: A Proven Strategy to Prepare High School Students for College and Careers. Career Academy Support Network. University of California at Berkley. < https://casn.berkeley.edu/resource_files /Proven_Strategy_2 25 1010 03 12 04 27 01.pdf > Director of Career and Technical Education (2015). Career and Technical Education Department. District School of Study. (Name and school district removed for anonymity).

PAGE 76

76 District Budget Book. Perkins Vocatio nal and Technica l Secondary Grant. (2014 15). Project no. 154206 (Name and school district removed for anonymity). Engineering, and Mathematics (STEM) in Postsecondary Education. U.S. Department of Education. National Center for Education Statistics. < ht tp://files.eric.ed.gov/fulltext/ED506035.pdf > Florida Department of Education. (November 2014). Bureau of K 12 Student Assessment. 2014 15 FSA EOC Fact Sheet. http://www.fldoe.org/core/fileparse.php /5423/urlt/FSAEOCFS2014 15.pdf Florida Department of Edu cation. (2015). Secondary Industry Certification. < http://www.fldoe.org/academics/career adult edu/industry certification/secondary.stml > Florida Standards Assessments. (2016). Florida Department of Education. < http://www.fldoe.org/accountability/assessm ents/k 12 student assessment/fsa.stml > Florida Standards Assessments. (January 2016). 2015 16 Algebra 1, Algebra 2, and Geometry End of Course Assessments Fact Sheet. < http://www.fldoe.org/core/fileparse.php/5663/urlt/FSAEOC1516.pdf > Ford Next Generatio n Learning. (2017). < https://fordngl.com/about > Southern Regional Education Board Atlanta, GA. Gene Bottoms and K imberly Anthony. (2005). Project Lead the Way: A Pre E ng ineering Curriculum that W Southern Regional Education Board. < http://publications.sreb.org/2005/05V08_Research_PLTW.pdf > George Madaus and Micha el Russell. (2010 11). Paradoxes of High Stakes Testing. Journal of E ducation Vol. 190, Nos. 1 2. Ireland. Dail Eireann. (1941). P arliamentary Debates, Col 1119. Hatice Kumandas and mer Kutlu. ( October 2015). High Stakes Tests. Journal of Education al Sciences Research. Intern ational E Journal Vol 5, No.2. Jonathan M. Breiner, Shelly Sheats, Carla C. Johnson, and Catherine M. Koehler. (January 2012). What Is STEM? A Dis cussion a bout Conceptions of STEM in Education and Partnerships. School Science and Mathematics Volume 112, Issue 1, pages 3 11, < http://onlinelibrary.wiley.com/doi/10.1111/j.1949 8594.2011.00109.x/epdf >

PAGE 77

77 Natalie Tran & Mitchel college Engineering Studies: An Investigation of the Relationship Between Pre College Engineering Studies and Student Achievement in Science and M Journal of Engineering Education 92(2), 143 157. National Academy Foundation (2015). < www.naf.org > Pam New berry, T. R ichard Grimsley, John Hansen, and Anne Spence. ( 2006 ) Re search of Project Lead the Way Curricula, Pedagogy, and P rofessional Development: Activities Regarding Increasing E ngineering and Technological Literacy of K 12 students in the PLTW network. In Proceedings of the Annual Conference of the American Society for Eng ineering Education. Chicago, IL. Pathways to Prosperity Project. (February 2011). Meeting the Challenge of Preparing Young Americans for the 21st Century. Harvard Un iversity College of Education. Scott Signs Florida Government News Releases. < www.flgov.com > Sean Brophy, Stacy Klein, Merrideth Ports Advancing Engineering Education in P 12 Classrooms. Journal of Engineering Edu cation 97: 369 387. doi:10. 1002/j.2168 9830.2008.tb00985.x Statistics Solutions. (2016). Data analysis plan: One Within, One Between ANOV A [WWW Document]. < http://www.statisticssolutions.com/data analysis plan one within one between anova/ > The Role of Career Academies in Education Improvement. (2009). Association for Career and Technical Education. < http://www.ncacinc.com/sites/default/files/media/research/ACTE%20Issue%20B rief%20Career_academies.pdf > Understanding Florida Standards Assessments Reports. (2015). Office of Assessment. Florida Department of Education. < www.fldoe.org/core/fileparse.php/566 3/urlt/UnderstandingFSAReports.pdf > U.S. Department of Education. Programs: Smaller Learning Communities Program. < https://www2.ed.gov/programs/slcp/index.html > Xianglei Chen. (July 2009). Stats in Brief: Students Who Study Science, Technology, Engineering, and Mathematics (STEM) in Postsecondary Education. U.S. Department of Education. National Center for Education Statistics. < http://files.eric.ed.gov/fulltext/ED506035.pdf >

PAGE 78

78 BIOGRAPHICAL SKETCH Kimberly Ann Stalcup was born in Somers Point, New Jersey. She moved to Florida with her family in 1983 and graduated from Barron Collier High School in 1995 She earned her B.A. in history and specialized in secondary education in 2000 from the University of Florida. S he earned her M.Ed. in educational l eadership in 2011 from American College of Education in Chicago, Illinois. Kimberly obtained her Ed.D in e ducational l eadership from the University of Florida in 2017 with the completion of this dissertation Kimberly is a high school Assistant Principal with six years as an administrator and eleven years as a high school teacher, swim coach and journalism and student government advisor. She currently holds an active Professional Educators Educator Certificate in the State of Florida and a Professional Educator Certificate in the State of Colorado. In Florida, she is certified in Educational Leadership K 12 an d School Principal K 12 History 6 12, Social Science 6 12, English 6 12, Middle G rades Integrated Curriculum 5 9 She is certified as a School Principal: All Levels in Colorado. Kimberly was a Teacher of Distinction in 2008 by the Collier County Educatio n Foundation, received numerous honors from Columbia University for outstanding yearbook publications and presented at the Learning Sciences International Robert Marzano National conference in 2016 Kimberly is a member of the Delta Epsilon Iota Academic Honor Society and the Golden Key International Honour Society. Kimberly has been married to J.R. Stalcup since 2001 and they have two sons: Brayden (born 2005) and Bryen, (born 2009). They reside in Erie, Colorado where they enjoy camping, skiing, snowboarding, whitewater rafting, paddle boarding, and anything outdoors.