This item is only available as the following downloads:
1 A MANAGEMENT BASED CIPP EVALUATION OF A NORTHERN NEW JERSEY DIGITAL BACKPACK PROGRAM B y BARRY A. BACHENHEIMER A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION UNIVERSITY OF FLORIDA 2011
2 2011 Barry A. Bachenheimer
3 To my mom, Letty, my dad, Allen, my wife Stacey, and my children Lea and Brett ; people who have always supported my learning
4 ACKNOWLEDGMENTS I did not get to this point in my educational career on my own. Along the way there have been many people who have helped, encouraged, and directed me. First, thank you to my doctoral cohort at the Univers ity of Florida. Throughout our journey I have enjoyed and appreciated your support and camaraderie. Next, thank you to my committee; Dr. Wendy Drexler, Dr. Ruth Lowery, and Dr. Ana Puig for their time and guidance. A special thank you goes to my advisor, Dr. Kara Dawson for her continual inspiration, support, and direction. I am grateful to the teachers in this study and their school district for their interest and participation. Thank you to my father, mother in law, and father in law for giving me time to work and encouragement while I worked To my children, thank you for understanding that I had to work when you wanted to play. To my wife Stacey, thank you for your enduring love, support, and encouragement throughout the entire proces s. I could not have done this without any of you. For that, I am eternally grateful.
5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURE S ................................ ................................ ................................ .......... 9 LIST OF ABBREVIATIONS ................................ ................................ ........................... 10 ABSTRACT ................................ ................................ ................................ ................... 11 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 13 Capstone Description ................................ ................................ .............................. 13 Professional Background ................................ ................................ ........................ 14 Cont extual Data ................................ ................................ ................................ ...... 15 Purpose and Guiding Questions of the Capstone Component ............................... 16 Statement of Subjectivity Bias ................................ ................................ ................ 17 2 LITERATURE REVIEW ................................ ................................ .......................... 19 Digital Backpacks ................................ ................................ ................................ ... 19 Professional Development ................................ ................................ ...................... 21 Integration of Technology ................................ ................................ ....................... 25 Student Engagement ................................ ................................ .............................. 30 3 METHODS ................................ ................................ ................................ .............. 35 Context ................................ ................................ ................................ ................... 41 Input ................................ ................................ ................................ ........................ 42 Description of the Professional Development Provided ................................ .......... 44 Prior to Training ................................ ................................ ................................ 44 First Release Day: January ................................ ................................ ............. 45 Online Collaboration Between Sessions ................................ ........................... 48 Second Release Day: February ................................ ................................ ...... 48 Between Sessions ................................ ................................ ............................ 50 Third Release Day: March ................................ ................................ ............... 50 Reflections on the Professional Development Provided ................................ ......... 51 Time ................................ ................................ ................................ ................. 52 Follow Up ................................ ................................ ................................ ......... 53 Access to New Technologies ................................ ................................ ........... 53 Cr eating a Community ................................ ................................ ...................... 54 Pedagogy Skills ................................ ................................ ................................ 55
6 Process ................................ ................................ ................................ ............ 56 Survey ................................ ................................ ................................ .............. 57 Focus Group ................................ ................................ ................................ ..... 57 Product ................................ ................................ ................................ ................... 58 Impact ................................ ................................ ................................ ............... 59 Effectiveness ................................ ................................ ................................ .... 60 Limitations ................................ ................................ ................................ ........ 62 Summary ................................ ................................ ................................ ................ 63 4 RESULTS ................................ ................................ ................................ ............... 64 Process: Survey ................................ ................................ ................................ ..... 64 Process: Focus Group ................................ ................................ ........................... 64 Attitude towards Technology Proficiency ................................ .......................... 67 Attitude towards the Professional Development ................................ ............... 68 Positive Perceptions ................................ ................................ ......................... 69 Negative Perceptions ................................ ................................ ....................... 70 SWOT Analysis ................................ ................................ ................................ 71 Strengths ................................ ................................ ................................ .......... 72 Weaknesses ................................ ................................ ................................ ..... 73 Opportunities ................................ ................................ ................................ .... 73 Threats ................................ ................................ ................................ ............. 74 Summary: Process ................................ ................................ ................................ .. 74 Impact: Technology Integration ................................ ................................ ............... 75 Use of Digital Video and Photography ................................ .............................. 75 Audio Recording ................................ ................................ ............................... 79 Singular A pproaches ................................ ................................ ........................ 79 Technology Integration Summary ................................ ................................ ........... 80 Effectiveness: Student Engagement ................................ ................................ ....... 81 Student Engagement Summary ................................ ................................ .............. 83 5 CONCLUSIONS ................................ ................................ ................................ ..... 84 Evaluation Summary ................................ ................................ ............................... 84 Context ................................ ................................ ................................ ............. 84 Input ................................ ................................ ................................ ................. 85 Process ................................ ................................ ................................ ............ 85 Product ................................ ................................ ................................ ............. 85 Discussion of Results ................................ ................................ ....................... 86 Implications ................................ ................................ ................................ ........... 100 Personnel Should be a Small Group of Volunteer Teachers in a Cohort Model ................................ ................................ ................................ .......... 100 An in House Facilitator Is Needed to Provide Professional De velopment and Support ................................ ................................ ................................ 102 Portability and Useful Tools Lead to Technology Integration .......................... 103 Professional Development Focused on Pedagogy More Than Technology Tools. ................................ ................................ ................................ .......... 105
7 Give Teachers Freedom, Support, and Encouragement to Take Instructional Risks ................................ ................................ ....................... 107 Summary ................................ ................................ ................................ ........ 109 Future Directions ................................ ................................ ............................ 110 Recomm endations for Future Research ................................ ......................... 112 Summary of Conclusions ................................ ................................ ................ 113 APPENDIX A SAMPLE ICOT CLASSROOM OBSERVATION TOOL ................................ ........ 115 B TECHNOLOGY INTEGRATION MATRIX ................................ ............................. 120 C DIGITAL BACKPACK APPLICATION RUBRIC ................................ .................... 122 D PROFESSIONAL DEVELOPMENT SURVEY ................................ ...................... 123 E FOCUS GROUP INTERVIEW QUESTIONS ................................ ........................ 125 F THREE STAGE OBSERVATION PROCESS ................................ ....................... 126 G SAMPLE OF TRANSCRIBED TEACHER OBSERVATION NOTES ..................... 127 LIST OF REFERENCES ................................ ................................ ............................. 141 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 149
8 LIST OF TABLES Table page 3 1. Explanation of the CIPP evaluation components ................................ ................ 40 3 2. Explanation of the context evaluation components ................................ ............. 41 3 3. Explanation of the input evaluation components ................................ ................ 43 3 4. Objectives of day 1 training ................................ ................................ ................ 46 3 5. Active learning strategies ................................ ................................ ................... 47 3 6. Objectives of day 2 training ................................ ................................ ................ 49 3 7 Objectives of day 3 training ................................ ................................ ................ 50 3 8. Explanation of the process evaluation components ................................ ............ 56 3 9. Explanation of the impact evaluation components ................................ .............. 59 3 10. Explanation of the effectiveness evaluation components ................................ ... 61 4 1. Professional development survey results n=6 ................................ ............... 65 4 2. Focus group attitudes and perceptions results n=6 ................................ ....... 67 4 3. Teacher lesson objectives ................................ ................................ .................. 76 4 4. Teacher observation summary ................................ ................................ ........... 77 4 5. Observed student engagement while using Digital Backpack tools .................... 81 4 6. Observed evidence of engagement ................................ ................................ .... 82 5 1. CIPP model summary ................................ ................................ ......................... 84
9 LIST OF FIGURES Figure page 4 1. Focus group coding sample ................................ ................................ ................ 66 4 2. SWOT Analysis ................................ ................................ ................................ .. 72
10 LIST OF ABBREVIATION S CIPP Context, Input, Process, Product Evaluation Model ICOT ISTE Classroom Observation Tool ISTE International Society for Technology Education IWB Interactive White Board NETS National Educational Technology Standards PD Professional Development SETDA State Educational Technology Directors Association SWOT Strengths, Weaknesses, Opportunities, Threats TIM Technology Integration Matrix TPACK Technology, Pedagogy, Content Knowledge
11 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education A MANAGEMENT BASED CIPP EVALUATION OF A NORTHERN NEW JERSEY DIGITAL BACKPACK PROGRAM By Barry A. Bachenheimer August 2011 Chair: Kara Dawson Major: Curriculum and Instruction The purpose of this study wa s to evaluate the Digital Backpack program in a Northern New Jersey School District using the CIPP Management Based Evaluation model as a framework. The Stufflebeam (1971) CIPP model is an acronym for Context, Input, Process, and Product Evaluation. Digital Backpack given to K 12 classroom teachers that contains portable digital tools that were used as in conjunction with intensive professional development Using a mixed methods approach this evaluation posed and answered key questions that addressed professional development, technology integration and student engagement as rela ted to the Digital Backpacks. A cohort of six teachers participated in the program As evidenced by their responses to a pre and post survey and focus group interviews, teachers indicated overall positive attitudes and perceptions toward s thei r professional development. Classroom observations showed participating teachers integrated technology into the classroom in a variety of ways; many of whic h contributed to high levels of student engagement. Opportunities were created by the teachers for students to use technology in meaningful ways that allowed for creativity,
12 collaboration, and reflection. Implications and recommendations for others wishin g to implement similar programs are provided.
13 CHAPTER 1 INTRODUCTION This capstone document consists of several components: an introduction, a literature review, evaluation methods, results and conclusions. The firs t component is the introduction which provides a brie f description of the capstone as well as my personal and professional background The introduction also provides contextual dat a on the school district that was evaluated as part of the study. The literature review chapter is a synthesis of my understanding of the research and theory in this area of educational technology and its connection and significance to the capstone project Areas of review include the history of Digital Backpack s, professional development, technology integration, and student engagement. Included in the evaluation methods chapter is the problem and purpose of the capstone components, data collection and da ta analysis. Lastl y, the conclusion include s a summary of the findings, i mplications o f this work and future directions and significance. Capstone Description The purpose of this capstone project is to evaluate the Digital Backpack program in a Northern New Jersey School District using the CIPP Management Based Evaluation model as a framework. The Stufflebeam (1971) CIPP model is an acronym for Context, Input, Process, and Product Evaluation. Digital Backpack computer bag given to K 12 classroom teachers that contains among other items, two netbook computers, a document camera, a portable scanner, an LCD projector, a digital camera, a Flip video camera, a digital voice recorder, microphones, headsets, and other assorted peripherals. T instructional practice. Digital Backpack
14 Amirian (2007) who described having a package of portable technology tool s that teachers could use in university classroom s Over the past seven years, the targeted school district has spent millions of dollars on technology projects, such as the Digital Backpack s. Bennet (2002) found that despite increased spending on technology, the schools they researched sa w no enhancement of academic literacy. To avoid a similar result, I conduc ted a m anagement based CIPP evaluation study of the Digital Backpack s, training in their use, and subsequent teacher effectiveness in engaging students Despite all of the spending of technology to date, the targeted school district has not formally evaluated the effects of technology on teachers or students. This CIPP evaluation gives teachers and administrators data and a sense of the effects of these digital tools befo re committing to a further investment in this technology. Professional Background To be successful, I believe that teachers need to be well prepared to do their job through effective, relevant, and sustained professional development in both pedagogy and te chnology. Once they have these skills, teachers need to be provided with the correct tools to implement their curriculum including digital hardware, software, and a network that allows them to broaden their student opportunities for collaboration, creat ion, and communication and to engage them in the learning process. I have a strong belief that effective professional development can improve technology in instruction can resul t in increased engagement for students. My niche is to design and lead educational change through effective professional development to transform teacher educational practices by modeling instructional designs that
15 effectively unite learning and technol ogy. I also believe that research based evaluation of practices is necessary in order to reflect, revise, and improve instructional practices. Contextual Data The Capstone study took place in a Northern New Jersey public school d istrict The district is comprised of two separate but closely related suburban upper middle class neighboring towns. The two towns share a school system the New Jersey A J (poor to wealthy) socio economic scale. The district has approximately 2,800 stu dents and a faculty of about 275 The district is governed by a five member elected Board of Education and led by a group of sixteen administrators including central office personnel and principals. Every year more than 99% of the student body gradua tes high school and about 90% of the graduates attend college. The district tries to set their students up for success by providing outstanding facilities and resources as well as emphasizing a college preparatory curriculum. The district wants to provid e its students with the best modern educational resources available to facilitate learning. To help accomplish this goal, over the past seven years the district has invested millions of dollars to provide technology hardware, software, network infrastruct ure, support, and training for teachers. Technology has become an integral part of the district and it is used for management tasks, student databases, communication, public relations, teaching and remediation, scheduling, facilities management, crisis pl anning and response, professional development, curriculum design, and more. The district moves forward towards a goal of continual improvement for its staff, facilities, resources, and most importantly, its students.
16 Purpose and Guiding Questions of the C apstone Component The purpose of this capstone project is to evaluate the Digital Backpack program using the CIPP Management Based Evaluation model as a framework. The Stufflebeam (1971) CIPP model is an acronym for Context, Inp ut, Process, and Product Evaluation. Most evaluations serve multiple audiences and the data is used to answer a variety of questions and to facilitate decision making. The most comprehensive evaluation models collect and report data from multiple perspect ives. The CIPP model reports from four parts which ask the following questions : What needs to be done? How should it be done? Is it being done? Did it succeed? (Stufflebeam, 1971) These four questions are answered through the CIPP model where c ontex t evaluations help prioritize goals, input evaluations assess different approaches, process evaluations assess the implementation of plans and product evaluations and assess the outcomes (both intended and not intended). The CIPP Model offers a comprehens ive way to gather and report evaluation data. This model has been used in countless educational and non educational settings with recognizable results for over 30 years (Stufflebeam, 1971, 2000). It is a very organized framework, evaluates a variety of el ements, and suits the needs of active practitioners. Many researchers have effectively used the CIPP Evaluation for their work from library evaluations was called The Sprit of Consuelo (2007), where an evaluation of a values based, self help housing and community development project for low income families was completed. With the CIPP Evaluation, Stufflebeam used multiple methods to gather data including observations, case stud ies, interviews, analysis, and synthesis to
17 generate a final report. These methods are similar to the ones used as part of th e present study. Matthews and Hudson (2001) evaluated parent training programs in Bundoora, Australia and found evidence of an eff ective program based on direct measures of parent and child behavior outcomes. They felt that the CIPP model best suited their role as practitioners. Another example is a study by Hodge and Jones (1999) that looked at the efficacy of using waivers to der egulate and improve public education in Florida. An analysis of waiver request forms from 67 sampled school districts revealed that the waiver process can facilitate school reform by assuring adequate planning and evaluation. They concluded that a need exi sts for state policies and procedures that assure the use of comprehensive evaluation practices. The CIPP model was a good fit looks at a specific situation, not society at large. With the CIPP evaluation, Stufflebeam (1971) used multiple methods to gather data including observations, case studies, interviews, analysis, and synthesis to develop a final report. Stufflebeam (1971) advocated that the purpose of the CIPP mo del is not to prove, but to improve This fits well with the goal to evaluate the effects of the Digital Backpack Program in the targeted school district and to use the results to determine the future of the program. Statement of Subjectivity Bias Many qualitative researchers consider that researcher bias and subjectivity are inevitable. Fitzpatrick, Sanders, & w hile several evaluation (p. 416) In this research evaluation study, I acknowledge that readers might think a positive
18 correlational outcome would be a foregone conclusion because a single person (the study author) serve d as the researcher, the designer and facilitator of the professional development, the observer of teachers, the reviewer of the data, and an administrator in the district being evaluated the potential for organizational pressure is greater when the evaluator is employed by the organi zation whose program is being evaluated than when the evaluator is employed by an outside (p. 421) Since this evaluation began on my initiative and not as a work assignment, the evaluation was less vulnerable to bias. Bias subjectivity was redu ced or eliminated to the fullest extent possible To complete this evaluation in an ethical way, I used strategies suggested by Lincoln and Guba (1981) including evaluator self reflection. One tacit goal of this study was an increased understanding of qu alitative research as a process of self discovery. Stufflebeam (2003) emphasizes that stakeholders play an integral role in CIPP evaluations. Evaluators are encouraged to be participants during the evaluation process so they understand the foundation and b ackground of a program, help define program goals and outcomes, and clarify criteria. Evaluators and stakeholders may also be called upon to assess evaluation reports. Stufflebeam (2003) believes that local representatives from the site of the implementation should conduct the evaluations. This approach builds the capacity of leaders and institutionalizes the work of evaluation. This process is consistent with the roots of the CIPP model.
19 CHAPTER 2 LITERATURE REVIEW The literature review provide s background research related to the evaluation of the Digital Backpack program. The development of this study considered the literature regarding Digital Backpack s, professional development, integration of technology, student engagement, and the CIPP Evaluation model. These five areas are pertinent to the development of a successful structure for creating a research based evaluation. T his literature review moves beyond summarizing the research Instead the research is synthes ized and applied to this capstone as suggested by the work of Beile & Boote (2005). Digital Backpacks The concept of a Digital Backpack as a collection of digital tools to be used in the practice of pedagogy is a relatively new idea. The first major study of Digital Backpack s was undertaken by Susan Amirian in 2007. Her focus was on college professors who were given a set of portable digital tools and a transporting case in an effort to facilitate learning and instructional design in a variety of location s within a university environment. Designed as a case study, her research showed a shift in the professional development for the professors from technology classes on a single topic or software program to an emphasis on teaching, learning, and technology integration. H not to be an expert computer user, but a good an d reflective pedagogical (p. 12). Building upon the work of Tiene & Ingram (2001) Amirian found better learning results for the professors and their Digital Backpack tools through hands on (p. 35). Using the work of Healy (1998) the study gave professors guided practice and
20 training using the technology to plan classroom les sons. Professional development was a key component, but the result of her study focused mostly on future recommendations and lessons learned. She hypothesized that there is a great potential to effect change at the classroom level with this model but que stioned if the results were transferable out of the college environment. This study transfers the Digital Backpack model to a K 12 integration practices and levels of student e ngagement when the backpacks are used. In 2010, Basham, Meyer, and Perry used a Design Based Research Model to look at the concept of Digital Backpack s in the elementary school environment Less about the backpacks and more about the instructional design their study showed that using Understanding by Design (1997) yielded better pedagogical planning than other formats. Using participant observation, video observation, field notes, and surveys, they ha d a group of elementary teachers pose essential questions for their classes and used the Digital Backpack s to help students create digital portfolios of their responses to these questions. They found that by using the tools in the backpacks, students were able to exercise stronger higher order thinking skills than if they did not use the tools. The key was not the tools themselves, but they explaine d that the important component wa s the use of the tools to explore essential questions. Pedagogy and techn ology must go hand in hand, as without it, technology simply provides access but not the pedagogy. These studies served as an exploratory study to the idea of Digital Backpack s, but greater depth is needed. The studies lacked a connection between the tools, professional development, and the integration of these tools towards the goal of
21 increasing student engagement. One common result outlined by the studies was a need for effective professional development that would provide practitioners the pedago gical skills to use in concert with the backpack tools. Professional Development teachers who to ok part in the Federal G overnment sponsored Eisenhower Professional Development programs and found five factors that had great potential for ach ieving results in schools. These factors include d long duration, collective participation, pertinent content, a ctive learning among participants, and coherence among pedagogy and curriculum. Though their study found that some schools offer high quality staff development with these characteristics, most respondent experiences tended to be a mixture of both high a nd low quality core components. The median number of professional development hours was fifteen and activities lasted a week or less. Generally, the process was not sustained for any significant length of time. Despite this research, it remains a commo n practice in schools to provide one time professional development activities with either a consultant or by sending teachers to single day workshops. The reason tends to be an issue of cost to school districts. Birman et al reflected that schools feel a responsibility to reach a large number of of the quality of the experience p. 32). They suggest that to be effective, schools either need to focus on better programs with less staff or increase the resources and time available while training a larger number of teachers. Certainly with a small cohort of teachers, resources, time, and impact could be effectively leveraged to provide a strong
22 professional development pr ogram, especially in the area of technology. A lot depends Pritchard and Marshall (2002) looked at professional development in what they required for professional development that leads to reform. They completed a four ye ar study in urban, suburban, and rural districts of the practices evident in teacher led activities offered in response to surveys of individual teachers or to demands fro m outside the school system, such as from state government, school improvement improve operations, broadly influence new directives, to remediate teachers or to fix isolated pr (p. 115). Good professional development has the power to bring about change to instructional practices or activities in a district, but according to Birman et al. (2000) it needs to be done with smaller groups in order to be effective. This create s a quandary in terms of large scale implementation of an initiative in techno logy or other areas. Based on over 400 hours of interviews with teachers and administrators, Pritchard and Marshall (2002) see professional development as the cornerstone of any reform effort, but found that most teachers are subjected to (and subsequently dislike) one day workshops taught by outside consultants. Their research showed that only 10% of teachers surveyed find these workshops useful. Despite this, many distri cts, including the targeted district continue the practice because it is cost and schedule effective. Instead of these so by workshops, Pritchard and Marshall (2002) found
23 117) in community groups. This point certainly presents a mandate to construct cohort or professional learning community groups that can be de veloped and sustained over a lon ger period of time. According to Pritchard and Marshall (2002) there is a strong correlation between effective professional development practices and positive reform activities by integrating them into a system of bel ief of vision that moves the district forward activities p. 137). The research shows a need to change some professional development components in the targeted district in order to move the district forward. An important outcome lacking in many of these professional development studies is a look at student engagement resulting in achievement. Martin et al. (2010) looked at the strategies of high quality professional development and found that greater professional development satisfaction by teachers was associated with higher quality student achievement in those classes. They defined high quality professional t in relevant activities, access to new technologies, collaboration and community building among strong implications for the instructional design phase as a relationship must be established between characteristics and outcomes, such as teacher instructional adjustment and resulting student outcomes. Specific tasks related to pedagogy model instruction, build community in the classroom, utilize technology, connect to their content areas, and establish inquiry
24 based learning p. 60). Ev professional development is an important component to measure impact From a managemen t perspective, the CIPP Evaluation (Stufflebeam 1971) seems well suited learned. Further, it is important to maintain a connection between professional development and resulting student engagement. Many educational studies focus on professional development in general, but not specifically with regard to technology. Culp, Honey, and M andinach (2005) completed a study that reviewed twenty years of key policy reports that addressed the challenges of implementing effective educational technology through professional development activities. They noted that even though 77% of classrooms i n the United States at the time of their study had I nternet penetration, very few cl assrooms had what they termed professional development for teachers in the area of tech nology integration. In a related study, Plair (2008) noted that despite national and state level technology plans, technology was not a significant component of professional development i nitiatives in K 12 classrooms. Plair felt that technology, pedagogy and content knowledge ( TPA CK ) st century skills needed to prepare for the new information or knowledge stricts spend on technology, most do not provide pedagogy based spec ific professional development for its use. To help mitigate this concern, Plair (2008) suggests that teachers need what person who serves as a resource to
25 help teachers sort through the plethora of information abou t tools, programs and resources. They then model these resources so teachers can see how to use them to support student learning. This turn key approach to training seems to be both collaborative and n on A teacher needs live professional development in a small group of like minded peers, not an online help desk or tutorial, nor a one da y workshop with a consultant. This article reinforces the idea previously mentioned by Mar t in et al. (2002) that effective professional development is in small groups, sustained over time, and focused on a particular set of skills or content. Martin et a l. (2010) further reinforced this when they found success in programs that had ongoing support from facilitators during a sustained professional development activity taking two years or more. All of these components form the basis of a plan for effective Dig i tal Backpack professional development with technology that includes a small group, Integration of Technology A commonality among the peer reviewed articles that I have read regarding technology and professional development is that the authors noted the focus of most technology professional development activities is on the technology itself, not learning his approach is that content and pedagogy are often considered a lower or even nonexistent priority tha n the use of technology tool s. have a need in this area, as virtually all classes in the established after school Professional Development Series focus on learning how to use tools such as a SMART Board or how to create a Web page, not the pedagogy needed to embed the tools
26 within the teaching environment. Matzen and Edmunds (2007) no researchers have argued that technology can cause a shift to more constructivist instruction, others have suggested that technology can only facilitate that transition p. 419). Citing Becker (2001) they noted t compatible instructional practices and beliefs were associated with increased technology use for higher order thinking p. 423). Quite obviously, a shift needs to occur where the pedagogy is the priority with a goal of greater student engagement and hi gher order thinking skills, while the technology is viewed as a means to that end. Professional development designed with this goal seems to be the paradigm mover. Lawless and Pellegrino (2007) questioned what kind of professional development is best when integrating technology into teaching and learning. Their work reviewed several examples of existing educational research on the topic and found in general that the int egration of technology into instruction. Their study found that most professional p. 576 ) They felt that to be effective, te chnology had to be incorporated into the daily routines of schools and be of long duration, including contact hours and appropriate follow up. Further, the professional development needs to be meaningful, collaborative, build community, and create a comm on vision for student achievement with the technology. Like Marin (2002), they also felt that single session workshops or bringing a consultant into the school for a continu e this practice. Darling
27 implementation of professional development p. 577). By t his she meant that the need for sustained investment in the people receiving the training was needed, not necessarily the ones providing the training as has been the model thus far. This can be changed by surveying and evaluating the participants in the program to see what their needs are and if they have been met. At the conclusion of professional development activit ies in the targeted district teachers evaluate the course or class in terms of how valuable they felt the course was to their practice. What has not been taken into account is i f the students are more engaged in their learning as a result of the teachers professional development. This information would be essential to truly gauge the effects of a professional development activity, especially with technology, in the area of student engagement. Lawless and Pellegrino (2007 ) define an effective professional development program as one where teachers do differen tly in their classrooms as a result of professional development? How (p. 597) T hese q uestions are a component of the CIPP based evaluation of the targeted district Digital B ackpack program. Additional research was done to discover factors that can increase the effectiveness of a technology based professional development activity. Zhao and Bryant (2007) noted that in most professional development workshops, integrated technology occurs at wha but does not lead to higher levels of integration on its own. As part of their study in Georgia of teachers who went through In Tech training consisting of 50 hours of sustained professional development, the ones
28 who were most successful at integration were those who had 1:1 help after the initial consultants will not cause sustained change. Instead, continual activities and availa ble help to assist individuals are needed. Working in a cohort with a mentor seems to be one solution that could be helpful Plair (2008) echoes this conclusion in her study s sort information towards the goal of engaging in new pedagogical and technological skills. To facilitate learner center ed, technology rich instruction, Polly and Hannafin understan development programs. Teachers need time to process and reflect, and time of this extent is hard to come by in most American school s To add to the definition of a successful and meaningful professional development program a focus on student learning, teacher ownership, developing knowledge of content as well as instructional strategies, sustained time intensive development, time for reflection, and most importantly, examining student outcomes such as engagement must be included Polly Zone of Proximal Developmen t, teacher learning opportunities are most effective when a more knowledgeable individual provides support p. 567). This conclusion also runs these factors need to be combined to create an effective professional development program with technology. Digital Backpacks were designed for teachers to provide themselves and their
29 In order for the tools to be effective, the training must be effective as well. In two studies by Zhao (2007) and Shriner et al. successful learning experiences with technology. If teachers receive extensive training focused not only on technology sk ills, but its application to curriculum and classroom activities, several studies including Duru (2005) and Hill (2004) demonstrated that teacher confidence will increase and students will benefit from a wider range of tools and appropriate activities. As important as professional development is, evaluation is also critical. An important piece of the Lawless and Pellegrino (2007) study is their literature review and analysis of professional development that focused on technology integration. First, they noted that of the scores of studies they looked at, very few used an evaluator to gather and analyze data. They believed that more objective studies occur when this is part of the process The use of an evaluator adds credibility to the CIPP model. Lawless and Pellegrino (2007) also positively noted that all the studies they reviewed used a mixed methodological approach a pplying both qualitative and quantitative data. This same mixed methodological approach is also used in the Digital Backpack evaluation. One limiting factor when evaluating professional development is that according to Martin et al. I t is i mpossible for us to know which, if any, specific strategies from the professional development teachers integrated into their instruction [to make positive (p. 70) Another limiting factor was noted by Archambault Williams, & Foulger ( 2010) i n a study they conducted involving professional development in Arizona with Web 2.0 tools. They found that 15 % of participants in their study did not see a pos i tive
30 impact on student achievement after professional developm ent with technology (p. 9). The participants had a positive perception of the training, but there were no marked student gains on teacher created assessments as a result. Student Engagement Hu and Kuh personal development during college is student engagement or the quality of effort students themselves devote to educationally purposeful activities that contribute dire ctly to the desir ed outcomes p.555). Referring to cognitive engagement, Fredericks et al. (2004) saw students who were engaged as having greater ability to problem solve, work harder, and cope well when faced with discouragement all as a result of being involved in an interactive environment. Hake (1998 ) defines "Interactive Engagement" (IE) methods as those designed at least in part to promote conceptual understanding through interactive engagement of students in heads on and hands on activities which yield im mediate feedback through discussion wi th peers and/or instructors. He defines "Traditional as those reported by instructors that make little or no use of IE methods They rely instead on teacher directed lectures without evidence of student feedback. Student engagement is critical to student motivation during the learning process. The more students are motivated to learn, the more likely it is that they will be successful in their efforts. Numerous factors influence student motiva tion including parental involvement, teacher motivation and skills, and effective use of technology. Technology can be utilized to create a motivating classroom environment where students are engaged in learning. An environment where technology is used in innovative ways leads to improved learning and teaching ( Wishart & Blease, 1999).
31 Several studies have shown a positive correlation between engagement and showed significantly larger learning gains for IE type courses relative to traditional courses In this same vein, Meltzer & Manivannan (2002) found that learning gains were noted among students in college physics courses using the IE method in lecture halls. Professors using a 1:1 discussion technique engage d students in dialogue. Finn Additio nally, Boekaerts, Pintrich, & Zeidner (2000) found t hat children of all ages who were actively engaged by activities in class and relate d new information to existing knowledg e better on various indictors of academic achievement. Furthe r, Walker and Greene (2009) found in a study of 249 high school students that when they found class work stimulating through goals of mastery learning and engagement, students had greater results o n tests of achievement than students who did not report the se elements of engagement. These studies further reinforce the connection between student engagement and subsequent student academic gains and make student engagement a good predictor of student academic success. According to a study done by Laird (2004) and data from the 2003 administration of the National Survey for Student Engagement results suggested that students who are engaged with online activities are most likely to engage in other effective educational practices. T here can be no question that t echnology is part of the learning cycle for students in schools in the 21 st century. However, Laird (2004) noted that using technology in general and using technology for academically engaging work are
32 two different matters (p. 1). Citing Hu and Kuh (20 01), Laird shared that students use technology primarily to communicate with each other and to get assistance with schoolwork. Further, there were self reported gains in general education and intellectual development in students who frequently used e duca tional technology. Based on the 60,000 respondents to the survey, Laird concluded that students who use technology for educational practices were more engaged and subsequently achieved higher academic attainment than students who did not use technology fo r academic purposes. While student gains as a result of technology use might be hard to measure, increased levels of student engagement have been shown to produce higher academic gains for students. As a result, greater student engagement should be one of the goals this idea in his research that project based work with computers is highly engaging for students attention effort, and engagement in academic tasks are a critical intervening variable in determining whether outcomes are attained p. 1). His research involving case studies of over 1,00 0 schools containing intensive computing using classes found that the highest levels of student engagement occurred empowered students to accomplish specific learning t (p. 38). Roschelle, Pea, Hoadley, Gordin, & Means (2000) identified four fundamental characteristics of how technology can enhance both what and how children learn in the classroom: (1) active engagement, (2) participati on in groups, (3) frequent interaction
33 and feedba ck, and (4) connections to real world contexts. When technology is embedded in teacher training it becomes more significant as a learning tool. This shows that the technology itself does not cause an increase in student engagement. Instead, when teachers are effectively trained in pedagogy that leads to engagement in addition to the technology tools there can be benefits for students. Beeland (2002) took this concept a step further by looking at a specific tool the interactive whiteboard (IWB). He used surveys as part of an action research study to measure student engagement while using an IWB He found that when text, graphics, and sound were use d to reinforce concepts, student engagement was h igh Surprisingly, student engagement was not as high when they got to use the whiteboard and touch it, as opposed to observing a visually and auditory stimulating activity. This was probably because limited numbers of students could use the IWB at a t ime. Though an IWB is not part of the Digital Backpack, this study is pertinent to cohort learning. For example, with only two laptop computers teachers will not have the abil ity to have all students use the tools at the same time. However, they may still encourage engagement for many students by using multi sensory stimulation through the use of audio and visual reinforcement. Not all research studies lauded the use of techno logy in instruction. A study by Peck, Cuban, and Kirkpatrick (2002) observed that despite studying schools where rarely observed anything resembling the type of student centered, construc tivist learning that many techno promoters hope computers and other machines will on their study, they concluded that schools they looked at with high access to technology
34 had little effect on student achievement because teachers u se d it incorrectly. This seems to be an obvious result of poor or missing professional development. Further, they felt that the isolation and structure of a modern day school prevents teacher learning and collab oration. As a result, they felt that many opportunities to eng age students were lost. This research demonstrates that effective professional development practices include opportunities for ongoing teacher collaboration, small cohorts of staff, and a focus on pedagogy. Further, when working with teachers to integrate technology, the emphasis should be on the pedagogy and ways that technology can support it, rather than the tools themselves. Also, rather than use of technology being the final goal, student engagement should be the targeted result of constructivist teaching with technology.
35 CHAPTER 3 METHODS Over the last seven years the targeted school district has increased the amount of technology hardware and software available to teachers and students from about 100 desktop computers to over 1500 desktop and laptop computers, mobile labs, interactive white boards, cameras, projectors, and scores of peripherals and software packages. The purpose of these purchases was to increase the amount and types of modern educational resources avail able to staff and students for learning. To give teachers the skills to effectively use these tools, an enormous professional development initiative in the form of an after school Professional Development Series was started. The training took place over the course of many years and was very well attended by teachers. The problem was that initially, the focus was not on the pedagogy but instead on the technology and learning how to use the tools, a process that Papert (1987) calls became the goal, rather than a means to the goal of increased engagement and learning for students. Another problem related to professional development was that many teachers shared classroo ms or teaching spaces and they did not have the time or opportunity to stay in the classrooms to gain independent practice. Most of these technology resources were placed in classroom or instructional spaces, not teacher work areas. Further, as a securi ty and virus safeguard, district computer users are prevented from downloading software programs onto their machines. While this does protect the computers from viruses and inappropriate software use, it also prevents teachers from downloading useful reso urces for their instructional programs.
36 Lacking access and collaboration, teachers did not have the tools available to use during their preparation periods or to take home to effectively learn the hardware, software and peripherals. This mirrored what La wless and Pellegrino (2007) found, that To be effective, they felt technology had to be incorporated into the daily routines of schools and had to be of long duration. Training would include sufficient contact hours and contain appropriate follow up. In the targeted school district, there was a need for access to digital tools by teachers a nd students not just in the classroom, but in other areas including hallways, science labs, non traditional instructional spaces, and on field trips. Lastly, despite millions of dollars being spent on digital technology by the district, no formal evaluatio n of its impact was undertaken. The exploration of portable technology and a supporting professional development component became necessary due to (1) i nadequate or ineffective professional development (2) a need for teachers to be able to downl oad useful academic software (3) the need for an evaluation of technology use in instruction The initial idea for a Digital Backpack came from the work of Amirian (2007) who described having a group of portable tools that teachers could use in the classroom. The Verizon Corporation in New Jersey investigated this study, liked the idea, and in 2007 gave a grant to Montclair State University and the East Orange, New Jersey School District that supported eight Digital Backpacks to gauge their effect on the technological literacy of inner city middle school students. After reading about the
37 Verizon Digital Backpack program in a Montclair State University magazine, in 2008 09 the targeted school district decided to launch a pilot program where a cohort of six teachers would each receive a Digital Backpack a portable computer bag that contained among other items, two laptop computers, a document camera, an LCD projecto r, a Flip video camera, a digital voice recorder, microphones, headsets, a pocket scanner, and other assorted peripherals. The supplies were paid for from the addition to the Digital Backpacks, the cohort was supplied with professional development that focused on the goal of helping students increase their understanding in the classrooms of teachers using digital tools. The teachers were also given support to set up a pe rsonal learning community both in person and in Google Groups to use as a discussion area, problem solving location, and reflective space. Throughout the district, teachers had desktop computers and televisions in their classroom. Students had access to one or two student desktop computers in the classroom and lab access when it could be reserved. With the Digital Backpacks, teachers now had a laptop computer for their use (in addition to their desktop computer), a laptop for student use, and a complete ly portable ensemble of A second cohort of seven teachers was solicited via an application process in 2009 10 and they too were each given a Digital Backpack and subsequent professional development. All teachers in the district were e mailed a description of the program, an invi tation to apply, and an application. The application asked why teachers were applying to be part of the program, what they would do with the Digital Backpack tools
38 that they currently could not do, how they hoped to expand student learning opportunities, what type of professional development support they anticipated they would need, and how they would measure their success. The teachers signed the application and it was co signed by their principal indicating endorsement prior to the submission of the app lication. The applications were then reviewed by a committee of administrators and parents using a rubric ( Appendix C). The highest scoring teacher Basham et al. (2010) put together a research based model to evaluate the effectiveness of professional development with Digital Backpacks among college professors. They found that exemplary pedagogy needs to be embedded and modeled in their professional development. A similar model was identified as a goal of the Digital Backpack program for K 12 teachers. A third cohort was solicited by the targeted district via the previous application process for 2010 11 school year. The third cohort included an elementary school resource room teache r, an elementary school Spanish teacher, a middle school science teacher, a high school chemistry teacher, a special education teacher, and a third grade teacher. The cohort was all female and had one Hispanic and five white teachers. This third cohort w as the subject of the capstone evaluation study. Each section of the CIPP evaluation has a description, guiding question, methods, timeline, and a connection to the capstone component. The Context component provides needs assessment and contextual informa tion. The guiding question for the
39 question of the Process component is ception and attitudes three subsections. For the Impact subsection the guiding question is Digital Backpacks by the teacher lead to greater technology int egration in the infusion of both digital tools and pedagogy skills lead to greater engagement for the ion of the aforementioned components and includes a Sustainability section. For the Sustainability section the guiding question is pedagogy skills lead to greater engagement for the students in that classroom Table 3 1 illustrates a brief description of the CIPP sections of the evaluation, guiding questions for each section, methodology to be used, a timeline for implementation, and the corresponding section of the capstone component. The overarching appro ach for the CIPP evaluation process has four sections that outline context, input, process and product.
40 Table 3 1. Explanation of the CIPP e valuation c omponents Description Guiding Questions Methods Timeline Capstone Component Context The context component assesses needs, problems, assets and opportunities within a specific setting Needs Analysis Application and selection process for members of Cohort III. Contextual background January 2011 Input Professional Development (PD) What is effective professional development for teachers to implement technology integration that engages students? Research effective PD strategies Design PD based upon best practices Implement PD January 2011 March 2011 Teaching Process Teacher satisfaction with the content and quality of the professional development provided and teacher perception of their readiness for Digital Backpack implementation. What were teacher perception and attitudes towards the Digital Backpack professional development? Pre and Post PD survey Interview participants using a protocol based on (SWOT [Strength, Weakness, Opportunities, and Threats] and SETDA) January 2011 March 2011 Scholarship Product Impact: Technology integration Practices In what way does the use of Digital Backpacks by the teacher lead to technology integration in the classroom? ICOT/Technology Integration Matrix [TIM] Format) to observed levels of technology integration. January 2011 March 2011; One observation per teacher Scholarship Effectiveness: Student Engagement In what ways does the infusion of both digital tools and pedagogy skills lead to engagement for the students in that classroom? ICOT/TIM Format Observations of Student Engagement January 2011 March 2011; One observation per teacher Scholarship
41 Context As shown in T able 3 2, t he context component reviews needs, problems, assets and opportunities within a specific setting This includes the Digital Backpack background that includes student population, socio economic data, levels of technology integration, and student achievement data. When the targeted school district considers purchasing technology, administrators and staff first identify an issue they hope technology mig ht improve or enhance. District philosophy is that technology use is not the goal or the end; it is one of many means Table 3 2 Explanation of the c ontext e valuation c omponents Description Methods Timeline The context component assesses needs, problems, assets and opportunities within a specific setting Needs Analysis Application and selection process for members of Cohort III. Contextual background January 2011 available to help improve student achievement. Despite all of the spending on technology, to date, the targeted school has not formally evaluated the effects of technology on teachers or students. All teachers in the targeted school district were invited to apply to be in Cohort III of the Digital Backpack program using the pre established application. Over fifteen applications were received. A cohort of six teachers from the targeted school district was selected from the applicant pool by a committee of administrators and parents using a selection rubric ( Appendix C). Cohort members received a fully stocked Digital Backpack and intensive professional
42 development training. The tools w ere then used in conjunction with ongoing sustained training as part of their instructional program. effects of the Digital Backpacks before committing to a further investmen t in this other local districts to provide data that other schools and districts may use to build and expand upon for their own technology purchases. Input T he guid ing question for the Input As part of ess that is shown in T able 3 3, the goal was to design, formulate, and i mplement a professi onal development program that would be beneficial to the teachers involved by utilizing best practices in professional development, particularly with the use of pedagogy and educational technology. The professional development provided many hours of training, was consistent with the No Child Left mandate for sustained, intensive and classroom focused professional development The research lite rature makes it clear that sustained professional development has a greater impact on teacher practice than one day or short term workshops. In particular, according to the U.S Department of Education (2000), teachers with more hours of technology related professional development report ed being more prepared to use technology in the classroom.
43 Table 3 3 Explanation of the i nput e valuation c omponents Description Guiding Questions Methods Timeline Input Professional Development (PD) What is effective professional development for teachers to implement technology integration that engages students? Research effective PD strategies Design PD based upon best practices Implement PD high quality professional development to the cohort members. Martin et al. (2010) defined high quality professional development support, active engagement in relevant activities, access to new technologies, collaboration and community building among participants, and a shared underst anding (p. 53). Based on this idea, the professional development for the cohort was planned to take place over a period of several months encompassing over 25 hours of contact time, would include continual follow up support from both peers and administration, provide access to new technologies, create a community among the cohort, and pr ovide the cohort with pedagogy skills to assist their instructional practices. According to Pritchard and Marshall (2002) there is a strong correlation between effective professional development practices and positive reform efforts. Good professional de velopment has the power to bring about change to instructional practices or activities in a district, but according to Birman et al. (2000) it needs to be done with smaller groups in order to be effective. To this end, the cohort was established with only six teachers. Pritchard and Marshall (2002) found that most educators want to
44 Throughout the professional development p rovided, the emphasis was on community building and helping peers through in person meeting, online communities, and frequent communication. Description of the Professional Development Provided The professional development provided to the Digital Backpack Cohort III consisted of several components. It included three full release days as a cohort for facilitator led training, an online network using Google Groups and a wiki to share information and files, informal discussions between cohort members via ind ependent meetings and e mail, and one on one classroom visits with each teacher by the facilitator in a role that suggestions regarding pedagogical practices, technology u se, and technology troubleshooting. Prior to Training Prior to the release days, a needs assessment took place and cohort members were asked to reflect and share areas of pedagogy and technology where they wanted specific development. This was accomplishe d via individual e mails. Areas of development requested by the teachers included working with digital video and audio media, designing effective student independent learning stations with technology, and digital copyright lessons for students. Based on research of best practices with professional development by Birman et al. (2000) and Lawless and Pellegrino (2007), a purposeful choice was made to not have the professional development session be simply about teaching the digital equipment, software, or websites. Instead, the focus was on pedagogy and how it could be enhanced with technology.
45 First Release Day: January On the first release day in January, cohort members were introduced to each other and then issued their Digital Backpacks and equipment. Several objectives for the day were shared. ( Table 3 4) A discussion then ensued about technology and how it can be infused into instruction. Teachers shared their visions for using technology in the classroom which included providing authentic assessments, engaging students with technology, creating problem b ased learning, and creating better data collection in the science lab. The teachers were very eager to open their Digital Backpacks and get started learning how to use the tools. I then raised the point that using technology is not the goal of this init iative. Instead, the goal is to engage students in the learning process with technology helping to reach that goal. To begin, I explained that the group needed to frame the professional development in terms of effective pedagogical practices, rather than just learning about the technology tools. Using the active lecture strategy proposed by Vella (1992), I modeled a variety of engaged learner strategies from the work of John Saphier (2009). These ideas ( Table 3 5) included having students work with images to illustrate key concepts, decreasing teacher talk and increasing student interaction, breaking class activities into smaller chunks, using visual and kinesthetic representations during instruction, providing frequent feedback to students and allow ing for self correction, using color, selecting high interest material, using flexible grouping and classroom set up, allowing students the opportunity to move about the room, and allowing students to converse with each other or work in small groups prior to a response in order to share ideas (p. 54 92).
46 Table 3 4 Objectives of d ay 1 t raining Knowledge and Skill Objectives Assessment Methods Teachers will discuss ways technology can be infused into instruction Teachers will become familiar with the ISTE ICOT model Teachers will discuss teacher roles while using technology for instruction Teachers will discuss and model essential questions Teachers will discuss differentiation of instruction with technology Teachers will be able to employ engage d learning strategies Teachers will be able to activate and use hardware and peripheral items in the Digital Backpack Teachers will be able to set up an account and log into Google Groups and the group wiki. Observation of cohort participation and engagement Inclusion of concepts and skills in observed lessons Observation of cohort engagement in the established Google Groups and wiki discussions Observation of cohort proficiency in using hardware and software After modeling many of these strategies, the group worked on ways to connect their new tools to learning activities, using the International Society for Technology Education (ISTE) Instructional Computing Observation Template (ICOT) as a model. (See Appen dix A) Using this template, the group discussed the various roles the teacher can take when using technology for instruction, such as lecturer, coach, modeler, discussion facilitator, and interactive direction giver. Next, the group learned about variou s student groupings that can be used with technology such as individuals, pairs, small groups, and whole classes (Gottfried, J., & McFeely, M.G. 1997)
47 Table 3 5 Active l earning s trategies Examples of Active Learning Strategies Working with images to illustrate key concepts Less teacher talk Increased student interaction and collaboration Use of multiple intelligence theory Frequent feedback Use of Color Movement Flexible grouping After taking time for the teachers to become more familiar with the tools in their backpacks, the group discussed ways to frame the use of technology using what Wiggins and McTighe (1997) call essential questions Essential questions are broad queries tha t help frame the learning. For example, should teachers have students learn from computers or with computers? Should students use technology collaboratively or individually? Should students be working on goal directed tasks or independently directed ta sks? Using a white board for notes, the issues were discussed and practical examples were cited and listed. Time was also spent discussing differentiation of instruction by readiness, interest, or learning profile. Differentiation can occur in the teaching process, the learning product, or the content being taught. Tomlinson (2001 ) states that this approach is responsive teaching rather than planning instruction that reflects a size methodology for st udents. Differentiation is delivered to help all students learn as efficiently as possible (Tomlinson, 2003). The cohort discussed how using technology can increase the choice of both learning styles and product for students. Tomlinson (2003) reports that student motivation and student attention to tasks increases when the topic of study reflects the personal interests of students. This also appears to assist with student engagement.
48 The remainder of the day was spent practicing with and using the new eq uipment and planning instructional units with both technology and guided pedagogy. Online Collaboration Between Sessions I wanted the group to be able to collaborate beyond the release days. To accomplish this, the group was introduced to the cohort wiki and Google Groups that were set up by to access resources, collaborate, and communicate. Accounts were established in both Google and Wikispaces and teachers had a chance to practice sharing questions, files, and helping each other online. Between in time to offer advice and to be a resource in the role of what Plair (2008) calls a mail or the Google Groups to assist each other with technical questions or to collaboratively plan. Second Release Day: February On February 14, the group met for its second release day. Objectives for the day were shared. ( Table 3 6) The me eting began with teachers sharing the different ways that the backpacks were being used and the successes and pitfalls they were encountering. Comments and helpful suggestions were offered by the group. The main focus of the day was designed around using audio and video media as teaching tools, based upon their completed needs assessments. Bruce and Levin (1997) developed the idea of four different ways that media can be used as part of technology : (1) media for inquiry (such as data modeling, spreadsheets, access to online databases, access to online observatories and microscopes, and hypertext), (2) media for communication (such as word processing, e mail, synchronous conferencing, graphics software,
49 simulations, and t utorials), (3) media for construction (such as computer aided design and control systems), and (4) media for expression (such as audio, video, animation and photographs ). With these four uses of media the group was shown how to use various media tools an d time was spent discussing ways the tools and subsequent media they produced or accessed could be used to as part of their instructional program. Examples of these tools included screen capture software, audio recording, and Google Docs and Spreadsheets Table 3 6 Objectives of d ay 2 t raining Knowledge and Skill Objectives Assessment Methods Teachers will share individual instructional practices with the Digital Backpack that occurred since the last release day. Teachers will learn and practice va rious ways media can be used to supplement instruction. Teachers will learn how to use video and screen capture software. Teachers will discuss effective feedback and how to use it with students. Teachers will work collaboratively to plan lessons that integrate technology. Observation of cohort participation and engagement Inclusion of concepts and skills in observed lessons Observation of cohort engagement in the established Google Groups and wiki discussions Observation of cohort proficiency in using hardware and software According to Hattie (1992), one of the most important practices used to improve student performance is feedback. When feedback denotes where and why students made errors, it seems significant increases in stude nt learning result (Tennenbaum & Goldring, 1989). Feedback examples that were modeled for cohort teachers included formative assessment strategies like exit cards, using digital recorders for oral metacognition, one question quizzes, and individual confe rencing. As with the previous release day, more time was spent in pairs and small groups creating lessons, troubleshooting, and collaborating. Teachers normally do not receive
50 extended time to collaborate during the school day, especially in the area of technology and certainly not with teachers from other schools. Between Sessions Again between in and to be a resource. The teachers continued to assist each other with technical questions or to troubleshoot technology issues. They continued to network and collaboratively problem solve, share best practices, and plan lessons. This was accomplished via e mail, the wiki, and the Google Groups Third Release Day: March Table 3 7: Objectives of d ay 3 t raining Knowledge and Skill Objectives Assessment Methods Teachers will share individual instructional practices with the Digital Backpack developed since the last release day. Teachers will learn and practice various ways audio can be used to supplement instruction. Teachers will learn how to effectively use search engines and model this skill for students. Teachers will discuss fair use and copyright rules and model them with students. Teachers will learn how to utilize videoconference software to communicate with people and groups outside their classroom. Teachers will work collaboratively to plan lessons that integrate technology. Observation of cohort participation and engagement Inclusion of concepts and skills in observed lessons Observation of cohort engagement in the established Google Groups and wiki discussions Observation of cohort proficiency in using hardware and software The third in service day was March 14. Teachers again shared their best practices since the last session. One teacher shared how she had started using a concept called flipping with her students. She shared that she was using the video camera to record a brief lecture which she had student view at night from her website with a guided reading. Then when students came to class the next day, they would be
51 placed into differentiated groups and assigned homework to complete in class based upon the content they learned. The teac her would then circulate, assist, and monitor the work by the students. Next, the group reviewed the National Educational Technology Standards (NETS) for students and teachers and discussed how well they were meeting the standards in class. I then facili tated a discussion on turning standards into classroom practices and reviewed teacher modes and student grouping with technology as well as ways to effectively engage students. Using targeted websites, the group discussed effective searching, using sound, and creating audio and visual slideshows. Other areas of exploration and review included video conferencing with Skype, promoting digital citizenship with students, and time to collaborate. The day following the third professional development release day, the cohort gave a 90 Backpacks and how they were being employed in their classes. A subsequent news story about the presentation was reported in the local paper ( http://caldwells. patch.com/articles/digital backpacks are changing teaching learning as your kids know it ). Reflections on the Professional Development Provided Martin et al (2010) defined high quality professional development duration, follow up support, active engagement in relevant activities, access to new technologies, collaboration and community building among participants, and a shared Lauro (1995) summari zed several models for developing teachers' technology expertise that included ongoing, systematic professional development, having attendees communicate their learning to each other,
52 and having an in house professional development expert in technology and curriculum available to provide ongoing training and support. Based on these research based ideas, the professional development for the cohort took place over a period of several months encompassing over 25 hours of contact time, had continual follow up s upport from both peers and administration, provided access to new technologies, created a community among the cohort, and provided the cohort with pedagogy skills to assist their instructional practice. Time The cohort was provided with three full day release days for professional development. Coverage for classes was provided by substitute teachers. Funds for the substitutes w ere provid ed from money allocated from the No Child Left Behind Grant, Title IIa. Title IIa provides funds for districts to p Child Left Behind Act, 2001) The professional development provided many hours of training, and was consistent with the No Child Left B intensive and classroom focused professional development No Child Left Behind Act, 2001 ). The research literature makes it clear that sustained professional development has a greater impact on teacher practice than o ne day or short term workshops. In addition to the release days, the cohort was set up with accounts on Google Groups in order to continue collegial conversations about how the Digital Backpacks were being used. District e mail was used to communicate as well. A wiki was set up to house collections of tutorials, resource links, and documents to continue the conversati on beyond the release days in order to provide what Birman et al (2000) calls
53 Follow Up classroom. This was done in order to provide feedbac k, offer suggestions, troubleshoot technology issues, and act in terms of technology and pedagogy expertise. Teachers seemed to appreciate these visits and more often than not, the time was spent discussing both classroom structure and technology related troubleshooting. Access to New Technologies As part of their participation in the cohort, each teacher received a Digital 4 GB t humb drive, and other assorted peripherals. Teachers selected were allowed to retain all the equipment as long as they are employed by the district. While every teacher in the district has access to a computer and many have LCD projectors, no teachers have access to as much portable digital technology as the Digital Backpack cohort. The Digital Backpacks were very well received by the teachers. One teacher In ad dition to the hardware, teachers received the full district software image on their creation software. Further, unlike the majority of the computers in the district, th ese
54 additional software at their discretion. Teachers seemed very appreciative of this privilege and many downloaded updated web browsers as well as content specific programs avai lable on the Internet. Creating a Community According to Pritchard and Marshall (2002), there is a strong correlation between effective professional development practices and positive reform efforts. Good professional development has the power to bring abo ut change to instructional practices or activities in a district, but according to Birman et al (2000) it needs to be done with smaller groups in order to be effective. To this end, the cohort was established with only six teachers. Pritchard and Marsha ll (2002) further found that most educators want Throughout the professional development provided, the emphasis was on community building and helping peers through in person meeting s online communities, and frequent communication. The cohort formed their own peer "community of learners" (Brown & Campione 1990). In other words, they provided each other with mentoring, support, and information that enabled them to imple ment the technology. One of the most common ways that cohort members communicated with each other was through e mail and on the Google Groups board. Most communication consisted of technology troubleshooting queries, suggestions for student collaboration, and ways to convert video formats. Additionally, Pritchard and Marshall (2002) noted in their research that teachers do not prefer their professional development to come from outside consultants. Subsequently, a decision was made for me to facilitate the professional development rather than bring in someone from outside the district. Since I had facilitated many
55 workshops for the district over the past seven years, all the cohort members know me and have worked with me in a professional dev elopment role prior to their membership in the cohort. This level of familiarity and comfort was one of the reasons for successful work sessions. Pedagogy Skills develop key technology training opportunities for teachers ha ve been simply that; training with technology. Emphasis was on learning and using the technology hardware and software. With the Digital Backpack cohort the majority of training was pedagogical and instruction. Once the foundation of pedagogy was laid, the group discussed ways the technology could be used to su pport the learning theories. of Future Technology Workshop This is a method they devised where series of structured workshop sessions, participants collaborate to like the professional development that the Digital Backpack participants went through shared the common goal of preparing participants to work with pedagog y and tools that helped to engage students. The most important indicator that the professional development provided was effective for promoting technology integration was a change in teacher practices. Sandholtz, Ringstaff, & Dwyer (2002) describe this ch ange as a shift in instructional style, from traditional to constructivist. They believe this shift takes place as teachers become expert technology users, leading to new levels of confidence and willingness to
56 experiment with instruction. This shift took place with the teachers in the cohort as evidenced by the practices they shared, the lessons observed, and their increased attitudes and perceptions of readiness to use the technology. The extended time that they spent in the release days might have been the most important aspect, as they had a chance to share with and learn from colleagues. In their study, Sandholtz et al. (2002) concurred with this as they noted: Teachers need increased and varied opportunities to see other teachers, to confront their actions and examine their motives, and to reflect critically on the consequences of their choices, decisions, and actions. They need opportunities for ongoing dialogue about their experiences and for continuous development of their abilities to imagine and discover more powerful learning experiences for their students (p. 51) These opportunities and more were effectively provided for teachers as part of the p rofessional d evelopment provided to the Digital Backpack Cohort. Table 3 8 Explanation of the pro cess e valuation c omponents Description Guiding Questions Methods Timeline Teacher satisfaction with the content and quality of the professional development provided Teacher perception of their readiness for Digital Backpack implementation What were teacher perception and attitudes towards the Digital Backpack professional development? Pre and Post PD survey Interview participants using a protocol based on (SWOT [Strength, Weakness, Opportunities, and Threats] and SETDA) January 2011 March 2011 P rocess What were teacher perceptions evaluate d teacher satisfaction with the content and quality of the professional development opportunities provided as well as teacher perception of their readiness for Digital Backpack implementation.
57 Survey Survey m et hods included pre and post professional development surveys completed by teachers and teacher interviews. The survey wa s a 10 question survey that utilized a Likert (1 5) scale which wa s designed to have respondents rate their perceived readiness to implement digital technology into their classroom teaching and assessment. The survey was designed based on the Intel Teach to the Future professional development survey (2004) and is aligned with the ISTE National Education Technology Standards (NETS) for teacher s It was administered to the cohort prior to the designed professional development with the Digital Backpack s and at the end of the professional development. A paired t test was used to calculate change in cohort member responses to questions prior to professional development training and at the conclusion of the training to measure statistical significance The purpose was to measure teacher satisfaction with the content and quality of the professional development provided, as well as readiness for technology implementation. A sample of the survey appears in Appendix D. Focus Group To determine teacher perception and attitudes in regards to the professional development, a focus group interview was conducted at the conclusion of the professional development components. The interview was conducted using a mixed method proto col comprised of elements from SWOT ( Strength, Weakn ess, Opportunities, and Threats) and The State Educational Technology Direct or Association (SEDTA) interview protocol The SWOT model was developed by Learned et al. ( 1965 ) to evaluate intern al and external organizational factors affecting a business or process
58 Strengths and Weaknesses refer to internal organizational forces an d Opportunit i es and Threats refer to external organizational forces. Dyson (2002) noted that once these factors are identified strategies are developed which may build on the strengths, eliminate the weaknesses exploit the op portunities or counter the (p. 632). In October 2002, SETDA, with a grant from the U.S. Department of Education, hired the Metiri Group to develop the interview protocol that was used as part of this evaluation. The focus of most of the The protocol was administered as a focus group interview with all six c ohort members. I act ed as the facilitator. The focus group was structured and recorded based upon the work of Glesne (2006), where she wrote that useful in evaluation research where participants can express multiple perspectives on a simila r experience The interview was digitally recorded and transcribed. Checking and analysis of tra nscriptions from the audio recordings was reviewed by the facilitator in order to ensure the reliability of the data (McMillan & Wergin, 2006). Responses were coded for analysis of common themes in the areas of teacher perception and attitude towards the p rofessional development received Questions used with the focus group interview are located in Appendix E. The results of the focus group were used to address the Process Product The product element i s s ubdivided into three subsections: impact, effectiveness, and sustainability. The Impact and Effectiveness subsections answer specific guiding questions while the Sustainability subsection is an analysis of the evaluation located in Chapter 5.
59 Impact: This section examine d teacher technology integration pract ices. The impact of included observations of teacher s while using the Danielson (1998) observation framew ork and utilizing the Technology Integration Matrix (TIM) to classify the levels of technology integration Table 3 9 Explanation of the i mpact e valuation c omponents Description Guiding Questions Methods Timeline Impact: Technology integration Practices In what ways does the use of Digital Backpack technology by the teacher lead to technology integration in the classroom? ICOT/Technology Integration Matrix [TIM] Format) to observe levels of technology integration. January 2011 March 2011; One observation per teacher The Technology Integration Matrix (TIM) illustrates how teachers can use technology to enhance learning fo r K 12 students. The TIM incorporates five interdependent characteristics of meaningful learning environments: active, co nstructive, goal directed (i.e., reflective), authentic, and collaborative (Jonassen, Howland, Moore, & Marra, 2003). The TIM associate s five levels of technology integration (i.e., entry, adoption, adaptation, infusion, and transformation) with each of the five characteristics of meaningful learning environments. Together, the five levels of technology integration and the five characteri stics of meaningful learning environments Educational Technology Department. Each teacher was observed one time scheduled in conjunction with the sustained professio nal development. Each observation followed the protocols utilized by Danielson & McGreal (2000) and consisted of a pre observation conference, the observation, and a post observation conference ( Appendix F.) Danielson states that,
60 and usefulness of classroom observation is directly related to the amount teachers discussed their goals for the lesson, student characteristics, how the selected goals connected to district curriculum and standards, and how they planned to engage students in the content and with the tools. We also discussed instructional materials, especially technology. During the observation, field notes and the TIM were used to gat her data and information that were used for the study as well as a source for post observation conference discussions and reflections on the teaching and engagement were During the observation, field notes were taken. According to Bogdan and Biklen (1992), f ield notes are critical to participant observation studies. They contend that in order for (p. 107). Post observation conferences took place no later than three days aft er the observation in order to ensure that the meeting was a useful reflection activity for the teacher. During the post observation conference teachers shared their impression of the lesson, exhibited evidence to show if objectives were met, and provided thoughts on how to improve for the future. Effectiveness: The guiding question for this section is pedagogy skills lead to greater engagement for t Effectiveness subsection examine d student engagement utilizing the ICOT tool with a focus on student e ngagement using technology. The ISTE ICOT was developed by
61 Barr et al (1998) as a way to assess the implementation of NETS standards in the classroom. It is describ ed by ISTE o n their website as: A computer based form designed to make classroom observations more efficient and useful. Observations are an essential part of evaluating and improving educational practice. However, they are time intensive and complex and the informat ion can be difficult to organize and interpret. The ICOT focuses on elements of the classroom that are related to effective technology use It automates the archiving and collating of data, and lets observers create simple reports and export data sets. The ICOT will help teachers, school administrators, and educational researchers look at classroom practice and use the information to make decisions. It was developed with support from the Hewlett Packard Company. Table 3 10 Explanation of the e ffectivenes s e valuation c omponents Description Guiding Questions Methods Timeline Effectiveness: Student Engagement In what ways does the infusion of both digital tools and pedagogy skills lead to engagement for the students in that classroom? ICOT/TIM Format Observations of Student Engagement January 2011 March 2011; One observation per teacher Hu and Kuh (2002) noted that personal development during college is student engagement, or the quality of e ffort students themselves devote to educationally purposeful activities that contribute directly to the desired outco mes p.555) The wor k of Hake (1998) and Meltzer & Manivannan (2002) showed that when students are engaged, they tend to have higher academic achievement than those who are not engaged. The work of Laird (2004) and Becker (2000) demonstrated that using technology during instruction is more engaging to students than not using technology. I conducted the observations and eac h observation was approximately 30 40 minutes in length. These observations generate d data for evaluating the overall effectiveness of engaging the students in purposeful learning while using techn ology. A reas the ICOT noted included setting, room description, student characteristics,
62 student grouping teacher role, percentage of students engaged, technology integration, learning activities, a rating of how essential technology was to instruction, technology used, NETS standards, and time on task If there were ways in which the technology contributed to the engagement level of the students, it was noted in the narrative comment section of question 6 of the ICOT. Further, during analysis of the data, I use d the ICOT data in the three minute chart to identify the amount of class time where technology was used and the amount of time that technology was utilized to increase student engagement. Limitations There were several limitations associated with this evaluation. Stufflebeam (2000) suggested that the point of entry and perspective of the evaluator implementing the evaluation process could influence the effectiveness of the evaluation. In some cases, s uch as this one, the evaluation process started with Cohort III when the Digital Backpacks had been in service for two prior years. Further, as the evaluator I was also the catalyst and trainer for the program and subsequently heavily invested in the res ults. Fitzpatrick, Sanders, & Worthen (2004) felt that was developed on my own initiative and not as a work assignment, the eva luation was less vulnerable to bias in that regard. Another limiting factor was the inability to fully evaluate the impact of the professional development. While surveys and focus groups can give the evaluator data regarding teacher perceptions and attitu des towards the professional development, according to Martin et al it is impossible for us to know which, if any, specific
63 strategies from the professional development teachers integrated into their instruction 70). Another limitation to the evaluation was the setting. This evaluation was specific to the program that has been established in the targeted district. As such, it might not have full transferability to other school districts or locations as a result of basic differences in demographics, composition of the tools in the D igital B ackpacks, resources available, expertise and skill levels of the facilitator and teachers, funding, time, and student abilities. Nonetheless, the results are valuable to the ta rget district and components might be useful to any district that is considering adapting a similar model. Summary The CIPP Evaluation model was used as a way for the targeted school district to decide if specific technology hardware and software were valuable before investing time and money in large scale roll outs. The overarching approach for the CIPP evaluation process had four sections that outlined context, input, process and product. These four sections combined to provide an evaluation of the Digital Backpack program in the areas of staff development, technology integration, and student engagement.
64 CHAPTER 4 RESULTS This chapter contains the results from the Process and Product sections of the evaluation. The Process section shares the survey and focus group results. In the Product section, the Impact subsection includes observations and evaluations of technology integration practices. The Effectiveness subsection includes observations and evaluations of student engagement with technology. Process: Survey The guidin g Process component question of this teacher perceptions and attitudes towards the Digital Bac kpack professional a mixed methods approach was used consisting of a pre and post professional development survey and a focus group interview with cohort members. ( T able 4 1) In every category, there was an increase in the mean response showing an overall improvement in teacher perception of readiness. However, of the ten questions, five were considered statistically significant and five were not statistically significant The areas considered significant asked about student evaluation, fair use and copyright, online communication, using portable tools for instruction, and using portable tools for student learning. Process: Focus Group In addition to the survey, t o help determine teacher per ception and attitudes towards the professional development experiences with the Digital Backpacks a focus group interview was conducted at the conclusion of three months of intensive professional development activities
65 Table 4 1. Professional d evelopment s urvey r esults n=6 Question Mean Pre PD Mean Post PD P Value CI (95%) (1) I feel prepared to integrate educational technology into the grade or subject I teach. 4.17 4.50 0.1747 0.33 (2) I feel prepared to support my students to use technology in their schoolwork. 4.00 4.50 0.0756 0.50 (3) I feel prepared to align my teaching with state and national learning standards. 4.50 4.83 0.1747 0.33 (4) I feel prepared to evaluate technology based work my students produce. 3.50 4.33 0.0422 0.83 (5) I feel I have an understanding of applying Fair Use and copyright law for students. 2.83 4.17 0.0103 1.33 (6) I feel I have an understanding of how to locate and evaluate resources for my instructional units. 4.00 4.50 0.2031 0.50 (7) I feel I have an understanding of how to facilitate the creation of student multimedia presentations. 3.50 4.17 0.1747 0.67 (8) I feel I have an understanding of how to create environments for student online communication and collaboration. 3.00 4.33 0.0250 1.33 (9) I feel I have an understanding of how to use portable digital tools in my instruction. 3.50 4.50 0.0409 1.00 (10) I feel I have an understanding of how to have students use portable digital tools for their learning and demonstration of learning. 3.17 4.33 0.0127 1.17 The interview was coded and then extrapolated for four specific coded themes: were then noted and subsequently triangulated as being positive, neutral, or negative.
66 Figure 4 1 is an example of a coding sample where attitudes were coded above the horizontal line and perceptions below. Figure 4 1 F ocus group coding sample For the purpose of this focus group interview, I defined attitude as a mental state involving feelings and dispositions to act in certain ways Perception was defined as using input to create meaning or a conclusion. Overall, the teachers reported a strong positive attitude towards using technology as well as a positive attitude towards the professional development that was provided. There was a positive perception of the quality of the professional development activities that were provided for them and a
67 corresponding strong positive perception of their individual readiness to implement technology as part of their instructional program. Table 4 2. Fo cus group attitudes and perceptions results n=6 Teacher Comments Positive (%) Negative (%) Neutral (%) Overall Attitude Overall Perception Attitude Towards Technology Attitude Towards PD Perception of Quality of PD Perception of Readiness to Implement 56.25 79.7 55 62.5 75 82.5 29.1 15.3 27.5 37.5 15 15 14.65 5 17.5 0 10 2.5 As noted in Table 4 2, in terms of attitudes, 56.25% of their comments were deemed as being positive, compared to 29.1% as negative. Neutral comments accounted for 14.65% of the responses. In terms of perception, 79.7% of responses were deemed as being positive, 15.3% as negative, and 5% as neutral. In a sub code towards the use attitude towards the provided professional development was 62.5% positive and 37.5% negative. In terms of perception of the quality of the professional development, 75% of omments were positive, 10% were neutral, and 15% were negative. For their perception of readiness to implement technology as part of their instructional practice, 82.5% of the responses were positive, 2.5% were neutral, and 15% were negative. Attitude towards Technology Proficiency Though most of the cohort had a good level of technology proficiency prior to the Digital Backpack professional development, many of the respondents felt that their perception of their own proficiency with using technology increased as a result of the professional development provided.
68 Attitude towards the Professional Development All the teachers indicated that a parti cularly effective part of the professional development was the opportunity to hear what other colleagues were doing with their Digital Backpack tools and sharing resources and ideas. They further indicated that the combination of modeling, sharing resourc es, discussing best practices, and discussions gave them confidence for the future that they could also engage in individual practice and independent research to improve their skills. Several noted that the small group size allowed for almost custom made professional development. One teacher noted: The digital backpack grant as a professional development program prov ides very personal, customized professional d evelopment. I was able to pick and choose what I wanted to do and created a project that benefi ted my students Teachers shared that they would seek out further opportunities for professional development, such as in and out of district workshops related to technology. One I would say that this [professional development] really gave me the confidence to explore technology on a whole new level. I am more motivated to research new tools and use this as a driving force for learning, as well as assessment, when it can enhance the lesson Another teacher commented : I would say that I was advanced proficient in using many of the actual items located in the backpack, such as a digital camera and a laptop. However, I would say I was only partially proficient in utilizing some of the services through websites and programs that would enha nce the technology in the backpack. These sessions were valuable because I had time to play.
69 Teachers shared several comments that showed positive attitudes from outcomes they experienced as a result of the Digital Backpacks in the areas of student grouping, saving student work, and increasing student access as outlined below. Student grouping: My recent lesson consisted of two computer stations with interactive components and one station with a laptop looping through an educational slideshow. I was also video taping the lesson for my website. I now feel quite prepared to integrate technology with the curriculum and standards based instruction because of this training. Saving student work: I used the portable digital scanner to scan the student work. It is easier for me because it saves it to my computer and I can take a closer look at the students work during my prep t ime. I sort the students work b y saving it I also add ed some of the students work to my P owerPoint presentations so that the students know th at their work is appreciated. Increasing Student Access: I am now in the process of adding video to my media and adding them to my school website for students to access when they are home due to illness o r away from class for TAG, music, or family vacations that take place when school is in session. P ositive Perceptions To a person, the respondents all felt that they had improved in their ability to use the New Jersey S tate S tandards for technology as well as the ISTE NETS standards for students and teachers in their teaching practice. One of the areas that were discussed in the professional development arena that most respondents found useful was the use of rubrics to aid in the differentiation of instruc tion. One teacher noted now feel extremely confident in using technology to differentiate. Often times, I find the technology i s most useful with students with special needs, while in retrospect also engages my high learners in a very intrinsic manner In many cases, the rubrics that
70 they produced were then posted on their website for student, parent, and colleague reference. Teachers also perceived an increased sense of confidence in their own skills as a result of the professional development. One teacher noted: I would say that this [the professional development] really gave me the confidence to explore technology on a whole new level. I am more motivated to research new tools and use this as a driving force for learning, as well as assessment when it can enhance the lesson. Other areas that they developed as part of the professional development were their understanding of teacher roles and class set up for use of technology, such as centers and learning stations. Another set of positive outc omes occurred when all the teachers joined collaborative online networks such as Google Groups and a wiki to engage in sharing, obtaining resources, gaining data for use in evaluating students, and altering their teaching style from a giver of information to that of a facilitator or coach One respondent noted that they had always used technology, but as a result of the Digital Backpack s t Negative Perceptions Few negatives were noted, but these centered on the reliability of the technology. past few months during the Digital Backpack deployment. The respondents expressed concern over the uncertainty about the technology working with the wireless network when they needed it. One teacher shared : T here is always the prospect that the equipment wo The Digital Backpack equipment is great, but some of our schools PCs just function
71 some of the time, and I get very frustrated. There always has to be a non digital plan B. This highlighted the importance of a maintained and reliable computer net work . The comment also indicated the need for teachers to have a non technical instructional plan in case the networks were not working as they were needed. SWOT Analysis The teachers were asked about strengths, weaknesses, opportunities, and threats in terms of the professional development provided for the Digital Backpack cohort. A summary of this analysis is in Figure 4 2.
72 Figure 4 2 SWOT Analysis Strengths Many strengths were noted, including that teachers could work collaboratively to learn new ideas, yet also were able to explore areas of individual interest and had time to work independently with support readily available. A teacher summarized this when Since each instructor has different needs, one of the greatest strengths was th e opportunity to work independently with supervision to concentrate on our areas of
73 One of the strengths of the Digital Backpack professional development was seeing how the other recipients of the Digital Backpack have us Teachers also noted the knowledge of the facilitator as a strength. One teacher shared: Another strength of this mode of training is all of the web sites and suggestions [ from the facilitator] He has a variety of resources that can be used in the classroom. It is also nice to come in with questions and I have to say that all of the troubles or problems I had were resolved through the collaboration with the other teachers that have the Digita l Backpack. Teachers further noted that the small size of the cohort afforded a way to have questions and problems resolved quickly. Weaknesses felt that there was too much professional development time was provided to the surveyed cohort than to previous cohorts, virtually all the teachers felt that even more time would have been useful to collaborate, share experiences and best practices, trouble shoot areas of confusion, share more resources, and work independently on individual projects. One teacher said : There are no weaknesses beyond the fact that there is never enough Opportunities Many opportunities were noted. The most common opportunity expressed was that professional development afforded teachers a specific time to practice and learn hands on with the technology while expert and collegial support was readil y available.
74 Further, all the teachers noted that they liked the collegial structure of the cohort. In I think it would be great to have yearly Threats Few threats were noted by the respondents. Among the m inority of teachers who perceived a threat, lack of time and lack of money were seen as the obvious potential threats. One teacher noted that being out of the classroom could be perceived as a threat. S ome people may not see the necessity in the teacher being absent from the classroom to learn how use these tools during school hours. There was a voiced hope that going forward, there would be more time for future professional development allotted as well as budgetary funds to sustain the program. One t eacher noted the threat of financial limitations: substantial amount of money, and some people may not see the ir need. I think this would be a very unfortunate circumstance. Summary : Process Based on the survey results and the overall increased mean response after professional development, the teachers felt a higher level of readiness to implement the technology from the Digital Backpack in their instructional practices compared to the ir feelings at the start of the professional development. Though there was an overall increase in the mean, only five of the ten questions demonstrated statistical significance. Based on the frequency of the codes and the actual responses during the focu s group interview, the majority of the teachers indicated that as a result of the professional development with the Digital Backpacks, they had a positive attitude towards both the professional development and towards using technology for instruction. Fur ther, their responses also demonstrated a positive perception of the
75 quality of the professional development and a subsequent positive perception of their readiness to implement the Digital Backpack technology as part of their teaching practice. Impact: Te chnology Integration Question 2: In what ways does the use of Digital Backpacks lead to technology integration in the classroom? To determine the levels of technology integration in h teacher was conducted utilizing the Danielson (2000) structure. Each observati on lasted 30 to 45 minutes. A combination of the field notes and cross referencing with the coded themes contained in the TIM help to organize the observation data. A summary of the lesson ob jectives are listed in Table 4 3 and a summary of the results are listed in Table 4 4 During the observations, several areas of commonality among teachers were noted in terms of technology integration. They included use of digit al video and photography and audio recording. Other singular approaches are noted as well. Use of Digital Video and Photography During the observations, several teachers utilized similar strategies for technology integration. One item that all teachers except one had in common was the use of either digital video or digital photography as part of the lesson for either teacher or student use.
76 Table 4 3 Teacher l esson o bjectives Teacher Grade Level Subject Area Lesson Objective LF 3 Language Arts Students will be able to recall the definitions of plot, conflict, and resolution; students will be able to decipher important climactic events from non important events in the novel; and students will be able to utilize various forms of technology to des ign and organize a plot sequence for the novel. RK 11 SPED English Students will be able to compare and contrast quotes from a novel for conflicting perspectives by the characters. MR 12 AP Chemistry Students will be able to prepare for the Chemistry Olympiad AG 4 Spanish Students will be able to create audio and visual postcards using Spanish language that describe an overview of an assigned Latin American country. RS K SPED Math Students will be able to realize that common shapes are prevalent in the world and to have the students identify known shapes (circle, square, rectangle, triangle) in and about the school. JK 7 Life Science Students will be able to demonstrate the concept of adaptive radiation by observing how different types of bird beaks are best suited for gathering types of food. LF had pr e instructions for the group, and a brief tutorial about how to use the equipment that students viewed in stations at the beginning of the lesson. In other lessons, students were t he ones using the cameras, most often in small take digital photographs to assemble a sequence of events in the chapter. A second mera to create and then film an alternate function on the digital camera, two groups of three were using the webcam on the
77 Table 4 4 Teacher o bservation s ummary Teacher Grade Level Subject Area Technology Used Overall TIM Classification ICOT Classification LF 3 Lang. Arts LCD Projector Laptop Computers Digital Camera Desktop Computers Interactive Web Sites Adaptation Useful RK 11 SPED English Document Cam. LCD Projector iPod Laptop Entry/Adoption Somewhat Useful MR 12 AP Chemistry LCD Projector Laptop Computers Digital Camera Web sites ( Gliffy, Diigo ) Audio Recorder Infusion Useful AG 4 Spanish Laptop Computers Document Camera Digital Camera Audio Recorder websites Scanner Flash Drive Adaptation Useful RS K SPED Math Laptop Digital Camera LCD Projector Adoption Somewhat Useful JK 7 Life Science LCD Projector Laptop Computers Digital Camera Excel PowerPoint Audio Recorder Website Adaptation Useful laptops to film their Spanish country postcards. Groups generally did one practice take and then a final take for recording. The groups of students worked collaboratively and took on roles such as camera operator, holding cue cards, or recording the video postc ard. As students completed their recording, they brought the camera over to the laptop and downloaded their film onto the laptop.
78 they went for a walk around the school inc luding the hallway, the library, and the stairwell. Students took approximately a dozen photos of shapes they identified including a fire extinguisher case (rectangle), a clock (circle), and a truss roof marker (triangle). On the secondary level, came and spent 30 minutes filming their set up, observations, reactions, and brainstorming of their use of a polyurethane auger. After the experiment, they reviewed the video and discussed what they saw and areas that they needed to better develop in order to prepare for the Chemistry Olympics. In the same class, another pair of students used the digital camera to snap photos of t he various stages of their experiment in which they injected carbon dioxide into carbonic acid. They then used the photos to create a flow chart using Gliffy a web based flow chart program to help them remember the steps in the sequence. school science class, there were several stations where students could work in groups of four to see how much birdseed they could capture using several video camera to sh oot 10 second clips of the most successful birdseed gathering beaks. In all cases when the students were using the equipment, the teacher circulated about the room, coaching and monitoring student progress. In all of these classes high levels of technol ogy integration were observed due to the use of the digital video or digital cameras.
79 Audio Recording class, students used the audio used the audio recorder for their Spanish country postca rd in situations where their pair of students used the audio recorder to narrate an experiment. As one student set up and ran the experiment that created ozone under t he chemical hood, the other student recorded an ongoing narrative about what was being done his observations, and ideas for the future. At the conclusion of the experiment, the pair listened to the audio and took notes on areas they wanted to remember or improve upon. In all situations the students used the audio to both reflect on and record their actions. Singular Approaches Several of the teachers took novel (but singular) approa ches to integrating technology with the Digital Backpack tools that they alone used. For example, LF had a group of her third grade students use the desktop computers and the website to assemble and describe the plot and climax activi ties from their chapters. Further, as another group in the same class was making a video, LF set up the LCD projector and the document camera as a make shift teleprompter to allow the students to film without having to hold a script. l Special Education English classroom she was observed using the document camera in conj unction with a class reading a chapter in a novel. She placed the text under the document camera and had students follow along in their
80 indiv id ual copies of the text. They were identifying quotes that would compare and contrast character perspectives. As they found quotes, they would be hand written by RK on a Venn diagram which she then also placed under the document camera. During the next part of the lesson, the students listened to an audio vers ion of the novel which RK played on her iPod As the students listened, they took some notes in a pre prepared active reading guide. As students were doing this, RK modeled the task using the document camera These were modifications she put in place to support her After students completed their lab, their documentation, and their data capture, they open for them to enter their data. As each group entered the data on the laptop and returned to their seats, the teacher shared that she would post on her website the medi ans and mode of the experiment data. This would allow students to complete the conclusions of their lab reports to test their hypothesis about adaptive radiation. In each of these observations, teachers creatively used the tools in the Digital Backpack i n order to integrate technology into their instructional practice. Technology Integration Summary As part of the post utilizing the categories and themes present in the Technology Integration Ma trix (TIM). indicators for level of technology integration in the classroom. F ield notes taken during
81 the observations showed evidence of technology integration in all the classrooms, but the level of integration varied from teacher to teacher. Effectiveness: Student Engagement Question 3: In what ways does the infusion of both digital tools and pedagogy Six teachers were observed using the ICOT tool. The lessons varied from 30 to 45 minutes with a total aggregate lesson time of 221 minutes. The mean t ime of the lessons was 36.8 minutes. During the lessons, 182 (82.3%) of the 221 minutes had the students utilizing technology from the Digital Backpack. The mean amount of minutes was 30.3 and the standard deviation was 9.37. The total amount of minutes of technology use by students for engagement was 163 minutes, or 89.5% of the class. The mean time of technology use per class was 27.1 minutes. A 95% confidence interval was computed to be 14.72 to 39.61, showing that the mean is statistically signific ant. Individual teacher data is noted in Table 4 5 and Table 4 6 notes the evidence of engagement from each observation. Table 4 5. Observed s tudent e ngagement w hile u sing D igital Backpack t ools Teacher Lesson time (minutes) Student use of technology (minutes) Technology use for student engagement (minutes) LF 45 36 (80%) 35 (97.2%) RK 41 31 (75.6%) 20 (64.5%) MR 30 30 (100%) 30 (100%) AG 30 27 (90%) 26 (96%) RS 30 15 (50%) 9 (60%) JK 45 43 (95.5%) 43 (100%) Totals 221 (Mean=36.8) 182 (82.3%) (Mean=30.3) 163 (89.5%) (Mean=27.1)
82 Table 4 6 Observed e vidence of e ngagement Teacher Grade Level Subject Area Evidence of Engagement LF RK MR AG RS JK 3 11 12 4 K 7 Language Arts SPED English Chemistry Spanish SPED Math Science A group of students worked independently to take digital photographs around the room and hallway to create and assemble a sequence of events in the chapter they read, document camera, and LCD projector to create and film an alternative plot and climax to the chapter, and a third to assemble and describe plot and conflict scenarios while using graphic organizers in the program. Only some techn ology was used hands on by the students. Technology from the Digital Backpack was mostly used by the teacher as an audio and visual support for the students as they read. It benefitted the students and they appeared to be engaged because of it, but they One student who was diagnosed with dysgraphia used the netbook to highlight quotes and color code them. That student in particular seemed engaged as evidenced by frequent interaction with the teacher and his peers in regard to the chapter being read. The Digital Backpack appeared to aid and supplement their work but did not lead to added engagement because that engagement was already present because of student motivation. Students were actively engaged in creating video and audio postcards utilizing video and audio recording devices. During the lesson, students actively collaborated and created a product that allowed them to practice written and spoken conversation in Spanish. While viewing the sh apes projected on the screen, students did not appear to be especially more engaged than if they were to see the shapes on a poster. While using the digital cameras in the hallway, students were observed as visibly excited when they found a shape and While viewing the PowerPoint on the screen, students did not appear to be more engaged than if they were viewing notes handwritten on a whiteboard. However, during the lab por tion of the lesson when students were using the hands on portable technology, the engagement among students was palpable. This was evidenced by students who were standing on chairs to get a better film angle, students who role played reporters to elic it feelings from others during the lab, and technology sharing between and among team members.
83 Student Engagement Summary Based upon the data obtained from the ICOT 3 minute chart, t he total amount of minutes of technology use by students for engagement was 163 minutes, or 89.5% of the time The mean time of technology use per class was 27.1 minutes. A 95% confidence interval was computed to be 14.72 to 39.61, showing that the mean is statistically significant. The classes that had extre mely high levels of engagement (>70%) were taught from a student centered perspective. In those classes, students worked as creators of content and meaning. As evidenced by this data and the observed use of technology by students, use of the Digital Back pack by students led to engagement in the classroom.
84 CHAPTER 5 CONCLUSIONS This chapter begins with a brief summary of the CIPP based evaluation followed by an analysis of the results. Implications and recommendations are presented for the school distri ct, participant teachers, and future direction. Suggestions are also made for further research on the use of Digital Backpacks. Evaluation Summary stages grounded with four research q uestions. Table 5 1. CIPP m odel s ummary CIPP Component Digital Backpack Component/Research Question Context Description of the Digital Backpack program as well as assessing needs, problems, assets, and opportunities within the educational technology area of the targeted school district. Input What is effective professional development for implementing technology integration to engage students? Process What were teacher percepti ons and attitudes towards the Digital Backpack professional development? Product Impact: In what ways does the use of Digital Backpacks by the teacher lead to technology integration in the classroom? Effectiveness : In what ways does the infusion of both digital tools and pedagogy skills lead to engagement for students in that classroom? Context The targeted school district is evaluating the Digital Backpack Program in order to assess the effects of digital backpacks on technology integration in the classroom and
85 on student engagement. Despite a large financial investment in technology, to date t he targeted district has not formally evaluated the effects of technology on teachers or the effects of these digital tools before committing to a further investment in this technology. Input The professional development provided to the six members of Digital Backpack Cohort III consisted of several components. It included three full day release days for facilitator led training, access to an online network using Google Groups and a wiki to share information and files, informal discussions between cohort members via independent meetings and email, and one on one classroom visits with each teacher by the facilitator. During visits t he facilitator gave sugg estions regarding pedagogical practices, technology use, and troubleshooting skills. The structure of the professional development was based on researched practices. Process This section evaluated teacher satisfaction with the content and quality of the p rofessional development opportunities provided as well as teacher perception of their readiness for D igital B ackpack implementation. Further, program activities were observed, monitored, documented and assessed. Methods included pre and post profession al development surveys completed by teachers and teacher interviews via a focus group. Product The product element i s subdivided into three sections: impact, effectiveness, and sustainability. The impact section examine d teacher technology integration p ra ctices.
86 Methods included classroom observations of teachers using the Danielson (1998) observation model and utilizing the Technology Integration Matrix (TIM) to classify the levels of integration. The effectiveness section examine d student engagement ut ilizing the same classroom observation s but instead utilized the ISTE ICOT tool in order to focus on student e ngagement using technology. This section looks to guide actions for future decision making in regards to the Digital Backpack program. Methods f or this section of the C apstone consist of synthesis and interpretation of the data in a n evaluat ion format. Discussion of Results In every category of the survey, there was an increase in the mean response showing an overall improvement in teacher perception of readiness. However, of the ten questions, five were considered to be statistically significant and five were not statistica lly significant. Two significance levels were used in determining the relative significance of the different statistical analyses. The p value identifies the likelihood that a particular outcome occurs by chance. It has generally been accepted that if the p value is less than .05, the result is statistically significant. Another accepted agreement is that when the significance level falls between .05 and .10, the result is considered marginally significant (George & Mallery, 2007). The smaller the sample s ize, the larger an observed difference would have to be in order to be statistically significant (Babbie, 2007). This study had a small sample size (n=6) and yielded several results in the range between .01 and .20. Therefore, in this study significance le vels less than .10 were considered statistically significant. A confidence interval at 95% was also computed.
87 Based upon the focus group interview, the coding of salient themes indicated positive perceptions and attitudes, particularly in the areas of time, individual needs, and collaboration. The first area noted as statistically significant was technology based work my students produce professional development mean ponse. This is likely due to the fact that most teachers much experience in evaluating technology based student work because it simply was not created. While the y probably were prepared to do so conceptually, they did not have any practice in doing so. As part of the professional development process, a great deal of time was spent discussing rubric creation and summative assessment. The use of rubrics to clarify teacher expectations helped students meet educational goals. Teachers also had some practical experience in their classes evaluating student work created with the Digital Backpack tools. T he post professional development survey The second area noted as significant was applying Fair Use professional de velopment mean was 2.83, indicating Neither the school district nor other regional training consortia, including area universities, have provided professional development in this area. As part of their needs assessment completed p rior to the professional development, most teachers indicated they wanted some training in this area. Utilizing an online training module from WestEd Unfair Use of e of
88 technology and looked at special situations including Creative Commons, use of music, use of images, and educational use. As a group, teachers shared that they needed to be more diligent in the way they modeled the ethical use of technology. For exa mple, o ne teacher noted that she had been copying pictures from Google Images for years and did not realize she was breaking the law. The post professional development t he largest change (tied with another question) of the ten questions. The next area noted as significant was create environments for student online communication and collaboration The pre professional developmen because most well known online communication tools are blocked by the district for teacher and student use including Skype chat rooms, and social media sites. As part of the professional deve lopment the cohort discussed ways that students could communicate with the technology within the parameters of available district tools. First, Skype was installed on their netbooks as the cohort was permitted to download outside software a privilege not afforded to other staff on district computers. Using this tool, teachers were able to set up video conferencing with other teachers and some local university professors to enable students to communicate with outside experts. Teachers were also shown how to use Google Docs and Presentations to set up inter class chat rooms to create discussions. One teacher had her students create video postcards which they filmed and saved to a thumb drive. These postcards were then erent class where the students had a chance to view and respond. Teachers were given the freedom to take instructional risks and to use
89 the technology in creative ways. As a result, several engaging projects with technology integration were implemented. The post professional development mean for this This response was tied with another question f or the largest change. The next area noted as significant was use por table digital tools in my instruction The pre professional development mean computers are desktops and most peripherals such as LCD projectors are permanently mounte d, eliminating the chance for teachers to be creative in their learning spaces while using technology. Amir i s ( 2007) focus with the first D igital backpacks was with college professors who were given a set of portable digital tools and a transporting case in an effort to facilitate learning and instructional design in a variety of locations within a university environment. Usi ng her work as a case study example teachers were encouraged to use the technology to create learning environment s in non traditional locations as mentioned in previous chapters. The post professional development mean The resp onse rate to t his question was similar to the next item that asked have an understanding of how to use portable digital tools in my instruction The pre Looking at the inf luences of Dewey (1916) and his idea of hands on learning, the cohort examined ways to use the technology to help students move through an inquiry process that stimulated their thinking, engaged them in authentic tasks, and demanded demonstrati on of mastery. In many cases, teachers would try out a project and then
90 discuss what they did with the group for feedback. Cohort members provided praise, suggestions, and modifications to help each other use the portable tools for learning projects. Ex amples included having chemistry students create multimedia projects of experiments, having elementary students create math videos to explain problem solving skills, and having middle school science students take the tools on a field trip to the Bronx Zoo to obtain media that was later used to create public service announcements that encouraged saving endangered species. As a result of the discussions and the feedback, the mean post professional development response was increased to 4.33, or response about using portable digital tools to demonstrate learning. In the five areas that did not have statistical significance, the responses were most likely due to teachers already having self perceived readiness to implement in those areas This was evidenced by the fact that teachers consistently rated themselves in statistically significant areas dealt with integrating educati onal technology, supporting students, aligning teaching with standards, locating resources, and facilitating the creati on of multi media presentations. They were most likely rated as such for two reasons. First, none of the six teachers noted these categ ories as an area needing development in their needs assessment This was because many of them had indicated prior experience with these tasks in their classrooms prior to the Digital Backpack cohort. Second, the district and local organizations ran sever al workshops related to these topics in the past, and many teachers had prior training in these areas.
91 At the conclusion of training, all teachers agreed they felt prepared to utilize the D igital B ackpack tools in their instruction. The results from the s urvey and the focus group showed an increase in teacher perceived readiness and positive attitudes and perceptions towards the professional development that was provided. In what ways does using the Digital Backpack lead to technology tors for level of technology integration in the classroom. During the observations there was evidence of technology integration in all the classrooms, but the level of integration varied fro m teacher to teacher for a variety of reasons. As part of the post observation conferences, the lessons were evaluated utilizing the categories and themes present in the Technology Integration Matrix (TIM). Allsopp, Hohlfeld, and Kemker (2007) initially describe d development of the TIM and according to an updated version (FCIT, 2010), the TIM should be used in classes to note learner choice and tool use as the key construct of learning. Zhao and Bryant (2007) found that teachers who only use technology for facilitating access to content had lower levels of integration compared to those who use technology for constructivist purposes. Of the TIM, both were using the Digital Backpack tools to help students learn specific content, such as the summary or plot of a chapter in a novel.
92 were formerly traditional activities. Teachers who were rated i category were observed as having students use the technology to be creators of content rather than consumers of content. These teachers encouraged student use of handheld peripherals and associated software fr om the Digital Backpack to create projects that accomplish ed learning objectives. In these cases, according to Pellegrino & Quellmalz (2010) integration of technology could offer ways of creating, over time, a complex stream of data about how students think and reason independently and included students filming video postcards from Latin American countries in the Spanish languag e, recording audio that narrated lab results, and making digital photo montages that described the rising plot of a book. It was interesting to note that the two teachers who were on the lower level scale of technology integration were Special Education t eachers with smaller class sizes. While they used a great deal of technology in their classes, it was more often in the hands of the teacher, rather than the students. For example, RK used a document camera to assist students in following parts of a nov el. In the past, she might have used an overhead projector or simply read aloud with similar educational results. While it was not observed, these teachers shared that as they became more comfortable with integrating technology from the Digital Backpack they slowly allowed students to use those peripherals for content creation later in the year. Clearly, greater professional development should be given to the special education teachers in the area of integrating technology with their student populations.
93 the offer of the facilitator to visit classes and also were more involved in th e online communities. It appears that one of the reasons for their higher levels of integration was a combination of live and online support. Plair (2008) reinforced this idea in her share the merits or weaknesses of a technology further noted that an unfortunate result of most professional development workshops is that integrated technology occurs at a basic level. According to their stu dy, successful technology integration takes places when there is strong 1:1 support for the teacher from a mentor or a knowledgeable colleague. Based upon this support, teachers had a model to work from, and they in turn provided a strong model of technol ogy use for problem solving for their students in several cases. technology was useful and oth er non technological methods would not have been as effective. For example, JK had used the observed lab in the past with good results, but the use of technology allowed for richer responses and data from the students that exceeded her expectations. AG f elt that technology provided by the Digital Backpack enhanced her lesson by adding real world authentic tasks and allowing students to practice public speaking skills in addition to the written and artistic component. By having students integrate various forms of multimedia in addition to the traditional
94 engagement at a greater level than if technology were not integrated. The integrated technology provided by the D igital Backpack coupled with the extensive professional development enhanced several lessons by giving teachers the opportunity to provide students with real world authentic tasks. If teachers received extensive training focused not only on technology ski lls, but its application to curriculum and classroom activities, several studies including Duru (2005) and Hill (2004) demonstrated that teacher confidence would increase and students would benefit from a wider range of tools and appropriate activities, as was the case in several regular education classrooms that were observed. some might be discourage d by this, it appears to be a good start. With limitations such as short amounts of preparation time and a computer network that had unpredictable outages, teachers still managed to integrate technology into the lessons with positive outcomes. In no case s did the technology hinder or distract from classroom learning objectives. As the teachers gain more experience and confidence from integrating technology with the Digital Backpack, one would assume that their level of technology integration would increa se over time. According to Dwyer, Ringstaff, & Sandholtz (1990), teachers progress through several stages of instructional and technological evolution as they become better at integrating technology. Teachers were allowed to keep the backpack for as lon g as they are employed by the district in order to allow
95 them to continue their practice and development when school was not in session. Technology support continued to be provided by district technicians. Most of the teachers did not use technology from the Digital Backpack to integrate technology simply for the sake of using technology. While in some cases they could have been nearly as effective if they had not integrated technology, the use of technology made the lessons more powerful because students had greater ownership, were more reflective, and used real world skills. In what ways does using the Digital Backpacks lead to student engagement Laird & Kuh (2004) concluded that students who use technology for academic purposes were more engaged and subsequently achieved higher academic attainment tha n students who did not use technology for academic purposes. While student gains as a result of technology use might be hard to measure, levels of student engagement are s hown to produce higher academic gains for students. To this end the goal for teaching with technology is engagement. Martin et al (2002) noted that specific tasks related to pedagogy instruction included showing teachers how to model instruction, buil d community in the classroom, (p. 60). Through the professional development I modeled, teachers learned a variety of engaged learner strategies from the work of John Saphier (2009). These strategies were designed to have teachers focus on producing active engagement for students. Hake (1998) defines "Interactive Engagement" (IE) methods as those designed at least in part to promote conceptual understanding through in teractive engagement of students in heads on and hands on activities which yield immediate feedback through
96 discussion with peers and/or instructor s. This professional development led to a shift in classroom practices where teachers developed real world a uthentic tasks for students. For example, in the observed classes, students practiced public speaking skills as they delivered presentations, communicated with experts and peers in other classes when they made Spanish postcards, created audio, video, and photographic content, and reflected on their work. These observations were a prime example of what Kearsley & Shneiderman (1999) call engagement theory This theory is based on the idea of creating successful collaborative teams that work on projects tha t are meaningful to someone outside the classroom. These three components of engagement theory imply that learning activities occur in collaborative teams, are project based, and have an authentic focus. As noted by MR, her students were engaged in the pa st because of these three components that are an intrinsic part of the Chemistry Olympiad. Technology seemed to increase this engagement as Kersey & Shneiderman (1999) note : The role of technology in the theory is to facilitate all aspects of engagement. The use of email, online conferencing, web databases, groupware, and audio/video significantly increases the extent and ease of interaction amongst all participants, as well as access to information. (p. 23) ith each other and engaged in their work while working on their preparation for the Chemistry Olympiad. Schlechty This wa s particularly evident in the observed Special Education classes where the Digital Backpack items were used by the teachers as audio and visual support for the students who have challenges such as learning disabilities. The special education teachers noted that when using traditional
97 T e achers noted that the students seem to like the technology, such as the cameras and iPod because they perceive d mark the location she stop ped at in each class and found it far easier to use than the cassette tape s used in the past. By using technology as a support for struggling learners, students experienced less distr action time while the teacher was tr ying to find a specific passage. It was interesting to note that w ith the exception of the netbooks in one class and the digital cameras in the other, very little technology was used hands on by the s pecial education students during observed lessons. The Digital Backpack equipment still benefitted the creating with the technology. Even though this seems to be in shar p contrast to the work of Hake (1998), this appeared to demonstrate that engagement can take place even without technology fully being in the hands of the students. Engagement with the Digital Backpack technology in the classrooms was measured through obse rvations utilizing the ICOT. During the observed lessons most students appeared to be highly engaged when using the technology from the Digital Backpack, especially the elementary students. In a study of elementary students by Skinner & Belmont (1993), t hey noted that student engagement increases when 3 rd grade classes for example, all students were active, and by placing them into small groups that were differentiate d by student ability all students were able to be involved with both the lesson and the use of technology tools. In many cases, while using the
98 technology students worked autonomously, but most of the teachers were in constant circulation among the grou ps to provide guidance, support, and clarification In line with the work of Skinner & Belmont (1993), to have students become more engaged they should receive all three teacher behaviors of involvement, structure, and autonomy support as well as hands on use of the technology throughout the lesson. In most of the observed lessons with high levels of engagement, these behaviors were observed. Creating hands on lessons and adding a digital component heightened student awareness of what they were doing an d provided an opportunity for hands on of a student taking place on her lab table and directing her teammates to get the best action. I saw a student using the audio recorder to assume the role of a sportscaster doing a play by play of the lab activity. Another student who acted as audio recorder introduced his team and then passed the recorder aro also saw one of the digital camera users t ake photo after photo from the surface of the that were arranged on the lab of the technology for recording raised their level of awareness of the lab and provided not only engagement th roughout the lab, but the potential for richer data and results. The potential for richer data and results also seemed to come about due to an increase in the opportunity for student reflection. For example, during the observations, it was noted on severa l occasions that students shot a practice take with the video and
99 then a final take. This method allowed students to practice, self evaluate, and make adjustments or corrections before completing their final work. An unintended result of the use of recor ding technology like video led to a higher level of reflection and editing on the part of students without teacher intervention. S everal teachers noted that the technology gave students the opportunity to be reflective in ways they could not, prior to emp loying the D igital B ackpack. In most cases, allowing students to capture images, sound, speech, or video with the D igital B ackpack tools gave them the opportunity to record and review more than they could have with traditional means. Whether word process ed essays on the netbook, digital photos of shapes, or videos of science experiments, students were able to save their work not only in greater multi sensory detail, but in a way that allow ed for review or reflection at a later time. E ngagement took pl ace on the part of students while using the tec hnology, especially when it was used in an active and collaborative way. In many of the classes observed, the teachers established a structure where students engag ed in project based constructivist learning. Using the tools, students made meaning of the content and applied that meaning to a formal project such as a video postcard, an alternate plot to a novel chapter, or a play by a science experiment. When changing from teacher to student centered instruction, compatible instructional practices and beliefs were associated with increased technology use f its own might lead to greater engagement, but student comfort in using the digital tools
100 as well as their innate interest and excitement led to greater engagement for students while us ing the Digital Backpack tools. Implications There are a number of critical components in the preparation and planning for a Digital Backpack program. These critical components became evident as a result of the of Digital Backpacks by the teacher lead to The following list represents sugges tions based upon key findings of this evaluation: 1. Personnel should be a small group of volunteer teachers in a cohort model. 2. An in house facilitator is needed to provide professional development and support. 3. Portability and useful tools lead to greater technology integration. 4. Professional development should be focused on pedagogy more than technology tools. 5. Give teachers freedom, support, and encouragement to take instructional risks. Personnel Should be a Small Group o f Volunteer Teachers i n a Cohort Mo del One of the reasons I believe the cohort model was successful was its small size. Good professional development has the power to bring about change to instructional practices or activities in a district, but according to Birman et al (2000) it needs to be done with smaller groups in order to be effective. To this end, the cohort was established with only six teachers. By only having six teachers at a time, each was given support and guidance without overwhelming the facilitator. Further, the ability t o communicate and interact was better initiated than if there had been a larger group.
101 Also, the smaller number lent itself to forming a learning community both in person and in the online groups. Pritchard and Marshall (2002) found that most educators w professional development provided, the emphasis was on community building and helping peers through in person meeting s online communities, and freq uent communication. The cohort formed their own peer "community of learners" (Brown & Campione 1990). In other words, they provided each other with mentoring, support, and information that enabled them to implement the technology. Individual teacher needs were able to be addressed. This was likely due to the small size of the cohort. With only six teachers, there was ample time to assess individual needs and for the facilitator to work one on one with teachers who required more intense assistance or consu ltation. All teachers received common training and took part in facilitated discussions, but there was ample time for individual exploration and pursuits. This was an unintended goal: to have teachers become a community of self directed learners. It se emed this goal was accomplished, as some teachers noted increased confidence in exploring technology on their own instead of taking an external workshop or class. Another factor leading to the success of the cohort model was that the initiative ed on teachers. As volunteers were solicited to apply, teachers who were awarded backpacks received them because they wanted greater technology access as well as the subsequent professional development. This intrinsic motivation seemed to make teachers m ore receptive to trying new things as well as taking instructional risks.
102 Since one teacher was selected per school building in the district, the resources were also divided equitably. For the future, it is recommended that another small cohort of teache rs be solicited and trained each year. Expansion should continue to be slow and calculated to provide for intensive and sustained professional development. An in House Facilitator Is Needed to Provide Professional Development and Support Plair (2008) foun d that effective learning communities need what she calls a pedagogical practices, technology use, and troubleshooting skills. Pritchard and Marshall (2002) noted in thei r research that teachers do not prefer their professional development to come from outside consultants. To this end, having an in house of the professional development. There was a high level of familiarity between the facilitator and the cohort and the facilitator had the ability to easily access the cohort through classroom visits and other forms of communication. The facilitator also had a great deal of experience as a staff developer and technology practitioner. Polly & Han n afin (2010) noted that teacher learning opportunities are most effective when a more knowledgeable individual provides in district, they had a greater stake in program success. While an individual from outside the district might have experience and technology skills, they would not have had as high a level of access, trust, or been fully vested in a positive outcome for t he program than an in house facilitator.
103 Portability and Useful Tools Lead to Technology Integration The Digital Backpack itself had many advantages. First was its relative low cost and ease to implement. Because the components were not hard wired to the network, less technology support and installation was needed as compared to other district hardware. The cost of each backpack made the purchase reasonable within the re were no reported breakages or breakdowns of any of the Digital Backpack equipment. from place to place, could be used in non traditional instructional spaces, and unlike desktop computers or hard wired peripherals, could be taken home by teachers for professional use and practice. For example, MR noted that she could easily move the Di gital Backpack between chemistry labs which allowed for greater student access and technology integration than she was able to provide in the past using desktop when col lege professors were given a set of portable digital tools and a transporting case in an effort to facilitate learning and instructional design in a variety of locations within a university environment. Very few teachers in the district have their own perm anent classroom. As a result, the more portable tools that the district can provide to teachers, such as the Digital Backpack, the more likely technology will be employed in the classroom or the instructional spaces they are using. For example, Digital B ackpack components were observed being used by students in classrooms, hallways and labs. The desktop computers that teachers have in their classrooms are permanently mounted and
104 students must go to them. The Digital Backpack components were seen used in a variety of places where the students brought them, including the floor, hallways, and various desks about the room. One of the most frequently used pieces of Digital Backpack for technology integration was the video camera and the digital camera. For e xample, use of recorded be reflective in ways she had not seen before. By simply putting a camera in their hands, they saw themselves as researchers. MR anticipates t hat as a result, the team will be more thoughtful and have better overall outcomes. During the observed team session, technology from the Digital Backpack was fully integrated and all students were using the digital backpack components the entire time. Recorded media also allows students and teachers to return to their work at a later date for reference. Outcomes cited on the website of Learning and Teaching Scotland: Early Years Online (n.d.) described results when young children used digital cameras. They found that students developed greater quality language and provided teachers students could load their shape photographs when reviewing later in the week. Furt her, the teacher was able to get evidence that she could look at and reuse to assess the Additionally, students provided more detailed insights as a result of the recording devices since they could view and review what they did. In a common theme noted among other teachers, this appeared to enable greater student reflection on the work.
105 Teachers also used recorded media in novel ways. In one instance LF pre recorded herself on a video for each of the g roups explaining their task. This gave students the opportunity to play and replay the directions (or parts thereof) if they required clarification during class. This effective strategy allowed the teacher more time to circulate among the groups and use t he time for guiding and clarifying, rather than repeating directions. While this lesson could have taken place and been effective without the technology, the use of the technology was most beneficial in allowing students a means to create more dynamic rep resentations of understanding than they could have with pencils and paper, such as videos, digital photos, and graphic organizers. Teachers using the portable tools shared concerns about network reliability. In recent months, the district network had seve ral outages. This led to a feeling of uncertainly and a loss of confidence in the technology. To this end, the district continues to work to improve the network backbone and at the time of this evaluation, more technicians have been hired to address the repairs and needed upgrades. This highlighted the importance of a well maintained network backbone, not just the need for peripherals. Teachers expressed a desire to prepare a non technical instructional plan in case the network was not working as they n eeded it to. Professional Development Focused on Pedagogy More Than Technology Tools. Professional Development for the cohort was effective because several components of successful research based professional development elements were employed. Martin et al (2010) defined high quality professional development as having new technologies, collaboration and community building among participants, and a
106 shared understanding It also included the establishment of a learning community model among the cohort both in person during the release days and in the online conversation areas that were created. Release days supported by the district with s ubstitutes were crucial. By giving teachers sustained time during the school day, this allowed them to focus on their learning and development without distractions. It is interesting to note that the teachers repeatedly described time to individually pr indicates enjoyment as well as the opportunity to explore. When teachers had time, they appeared to enjoy the discovery and learning process. The theme of time was repeated and several teachers noted that they had little time during the standard school day to experiment with the new technologies. The professional development release days afforded them that time. An emphasis of the training was not how to use the Digital Backpack tools. Instead, the re was a deliberate focus on pedagogy skills as the foundation and lens for technology use in the classroom. Martin et al. (2002) noted that specific tasks related to model instruction, build communi ty in the classroom, utilize technology, connect to their content areas, and The most important pedagogy skills that were emphasized on the release days included differentiating instruction based upon student read iness and interest, use of visual and auditory reinforcement, and student centered teaching that focused on creation and metacognition. Polly and Hannafin (2010) found s
107 There were observed successes in the classrooms in terms of technology integration and student engagement. I believe this occurred because this professional development led to a shift in classroom practices where teachers created real wor ld authentic tasks for students. For example, students practiced public speaking skills, communicated with experts and peers in other classes, created audio, video, and photographic content, and reflected on their work. On several occasions during classro om observations, it was noted that students shot a practice take with the video camera and then a final take. This method allowed students to practice, self evaluate, and make adjustments or corrections before completing their final work. An unintended r esult of use of recording technology like video led to a higher level of reflection and editing on the part of students without teacher intervention. This program utilized a paradigm shift in professional development for teachers from what Papert (1987) ca software program, to an emphasis on teaching, learning, and technology integration. Give Teachers Freedom, Support, and Encouragement to Take Instructional Risks One key observation made during the classroom visits was that when teachers were given support and approval to experiment with technology in order to increase student engagement, they took more instructional risks than they had in the past. During the pre assessments, teachers noted areas where they wanted further develop ment. During all the release days as they learned new skills they were encouraged to employ them towards the goal of technology integration and student engagement. They were told there would not be any evaluative consequences from a job performance perspective for taking an instructional risk.
108 To accentuate this level of trust, teachers were allowed to download and install any software or programs they wanted to their laptop computers. This wa s a privilege that was not afforded to any other district teachers due to virus and security concerns. As a result of opening access to the network, teachers were able to download and utilize a host of tools for student engagement including Skype Gliffy iTunes and Kindle software. No security or virus breaches were noted from the cohort. Based upon th e success of this group, this will be an area of further exploration for the district to allow for increased network access. The most important indicator that the professional development provided was effective for promoting technology integration was a ch ange in teacher practices. Sandholtz, Ringstaff, & Dwyer (2002) describe this change as a shift in instructional style, from traditional to constructivist. They believe this shift takes place as teachers become expert technology users leading to new level s of confidence and willingness to experiment with instruction. This shift took place with the teachers in the cohort as evidenced by the practices they shared, the lessons observed, and their increased attitudes and perceptions of readiness to use the te chnology. Teachers shared that this took place as a result of the support and encouragement that was provided to them by the facilitator and other district administrators. The support also comes from colleagues in the cohort who worked with one another du ring release days, online, and in each their study, Sandholtz et al. (2002) concurred with this as they noted: Teachers need increased and varied opportunities to see ot her teachers, to confront their actions and examine their motives, and to reflect critically on the consequences of their choices, decisions, and actions. They need opportunities for ongoing dialogue about their experiences and for
109 continuous development o f their abilities to imagine and discover more powerful learning experiences for their students. (p. 51) There were some limitations noted. While the cohort was repeatedly told that the Digital Backpack technology should be integrated when it made sense p edagogically, some teachers felt compelled to use and show off their newly acquired equipment in most of their lessons. It was noted that some lesson objectives could just as easily n could have been completed with a marker and poster paper without using any technology. In this case, it appeared to supplement the lesson, but the lesson could have accomplished its objectives without its use. It is possible that some teachers felt tha t because they had the Digital Backpack, the expectation was to use it at all times. The professional development emphasized that the pedagogy should drive the tools used, not the other way around. While in some cases they could have been nearly as effec tive if they had not integrated technology, the use of technology made the lessons more powerful because students had greater ownership, were more reflective, and used real world skills. As evidenced by the direct observations and the placement of instruc tional experiences on the TIM, the data indicates that using the Digital Backpacks led to increased technology integration in the observed classrooms. Summary As evidenced by the direct observations and the placement of instructional experiences on the T IM and ICOT, the data indicated that using the Digital Backpacks led to increased technology integration in the observed classrooms as well as increased levels of engagement. There were several critical components noted for these successes. Professional Development for the cohort was effective because several
110 components of successful research based professional development elements were employed. They included following the recommendations of Birman (2000) to have a small sized cohort; what Martin et al. (2010) recommended, ongoing sustained time to learn and practice; and use of internal personnel as facilitators. It also included the among the cohort and what Plair (2 008) suggested as a necessity to focus on pedagogy skills as the foundation and lens for technology use in the classroom. Going forward, it is suggested that these practices continue to be used and expanded upon. This will result in what Pritchard and M initial Digital Backpack program, this program utilized a paradigm shift in the professional development for the teachers from technology classes on a single topic or software program to an emphasis on teaching, learning, and technology integration. Future Directions As a result of the findings from this evaluation, several recommendations were made regarding future sustainability of the Digital Backpack Program. The areas included cohort membership, facilitators, hardware components, and budgeting. In the future, cohorts should remain at six teachers. Since one teacher was selected per school building in the district, the resources wer e divided equitably. Further, six Digital Backpack purchases per year are manageable within the budget. Volunteers should continue to be solicited for subsequent cohorts. Those who apply are doing so because they want the tools, experience, and training to make them better educators. This intrinsic motivation seems to make teachers more receptive to trying
111 new things as well as taking instructional risks. For the future, it is recommended that another cohort of six teachers be solicited and trained eac h year. There will be eighteen members at the end of this school year in the targeted district. One future direction is the gradual phase out of the district administrator as the facilitator. Going forward, it is recommended to move to a train the train er model where teachers who have been using the backpacks become the future facilitators, coaches, and trainers for the program. This will lead to greater teacher ownership of the program. To expedite this process, I will solicit volunteers from the exis ting cohort selection. These trainers will continue in the support role formerly occupied by an administrator. T eacher learning opportunities will continue to be effec tive when a knowledgeable individual provides support. Current cohort members will be surveyed to evaluate their perceptions on what tools they use most and least often. Based upon their responses, future Digital Backpacks should have some slightly differ ent items than the past three years. Emerging portable technology will be monitored for possible inclusion in future Digital Backpacks. Examples might include tablets, clicker response systems, or smart phones as educational uses become clearer for these items. New Jersey public schools are mandated to remain under a two percent budget cap each year. As a result, technology and other expenditures have been reduced or cut completely in many areas. As each Digital Backpack costs approximately $1,200.00, t he purchase of six Digital Backpacks can be factored into the budget each year, as the cost is not prohibitive. For the future, sponsorship of the Digital Backpacks
112 by the local parent association or educational foundation might be explored. If other dis tricts were looking to implement this program, they could choose to configure the Digital Backpacks with different tools to either reduce costs or meet particular district needs. Recommendations for Future Research The work of Amir ia n (2007) and Basham et al (2010) served as an effective introduction to the idea of digital backpacks, but greater depth is needed. Their work with Digital Backpacks lacked a connection between the tools, professional development, and the integration of these tools towards the goal of student engagement. While this evaluation addresses some of those areas, further research is recommended in the use of portable digital tools in the classroom. Further exploration is needed to examine the transfer of knowledge and pedagogical sk ills acquired during professional development with the Digital Backpacks into non technology instructional areas. As this is the third cohort of the Digital Backpack project, there will be eighteen members at the end of this school year in the targeted d istrict. Going forward, the district could look to move to a train the trainer model where teachers who have been using the backpacks could become the future facilitators, coaches, and trainers for the program. Would the program be as successful, more suc cessful, or less successful with cohort members as facilitators? One area to monitor is teacher and student use of the individual tools provided in the backpacks. If items are not being used, they should not be placed in future Digital Backpacks. If certa appear to be getting, purchase of more should be considered. Further, emerging
113 portable technology should be monitored for possible inclusion in future Digital Backpacks. Examples might include tablets, clicker response systems, or smart phones as educational uses and supporting research become clearer for these items. In the targeted district it would be useful to evaluate the Digital Backpack program on a yearly basis. When there are m ore cohorts of teachers and greater numbers of Digital Backpacks in use, it would be interesting to see what effects these have on the professional development program as a whole, overall district technology integration, and the effects of increased studen t engagement. The data could be collected for five years and then analyzed longitudinally. A limitation of evaluation is the setting. This evaluation is specific to the program that has been established in the targeted district. As such, it might not ha ve full transferability to other school districts or locations as a result of basic differences in demographics, composition of the tools in the digital backpacks, resources available, expertise and skill levels of the facilitator and teachers, funding, ti me, and student abilities. Nonetheless, the results were valuable to the target district and components might be useful to any district that is considering adapting a similar model. It is suggested that any district who pilots a Digital Backpack program conduct their own CIPP based evaluation. Further, districts that are considering starting a Digital Backpack program need to have a person serve as the administrator of the project in order to take care of cohort needs, such as tech support, tool procurem ent, and organizing release time for professional development. Summary of Conclusions is Did it Work ? This CIPP evaluation has shown that the professional development
114 provided w as effective in facilitating the integration of the Digital Backpack program to Cohort III due to a focus on pedagogy and instruction, a small cohort, ongoing and sustained development, a knowledgeable facilitator, and repeated opportunities for the cohort to collaborate and learn with each other. As evidenced by their responses to the pre and post survey and the focus group interview, teachers indicated overall positive attitudes and perceptions towards this professional development. The Digital Backpac ks led to technology integration in the classrooms as well as contributing to engagement among students as proven by classroom observations utilizing the TIM and the ICOT tools This was due to opportunities created by the teachers for students to use technology in meaningful ways that allowed for creativity, collaboration, and reflection. This CIPP evaluation provided the targeted district a sense of the many positive effects of the Dig ital Backpack on technology integration and student engagement so they can optimistically commit to a further investment and expansion of this technology program.
115 APPENDIX A SAMPLE ICOT CLASSROO M OBSERVATION TOOL Classroom Observation Tool The ISTE Classroom Observation Tool (ICOT) is a free online tool that provides a set of questions to guide classroom observations of a number of key components of technology integration. ICOT was developed by staff and consulta nts in the Education Leadership Department at the International Society for Technology i n Education (ISTE) with support from the Hewlett Packard Company. For free access to the ICOT software and online tools, visit http://www.iste.org/icot 1. Setting Date: _________________________ School: ___________________ Project/Program: _________________________ Site Code ___________________ Observer: _________________________ Teacher: ___________________ Grade: _________________________ Subject: ___________________ #Students: __________ Observation Start time: ________ End time: ________ (You can track technology use by three minute intervals throughout the observation using the three minute chart at the end of this form.) 2. Room description and student characteristics: 3. Student groupings (check all observed during the period): -----Individual student work -----Small groups -----Student pairs -----Whole class -----Other (please comment):
116 4. Teacher roles (check all observed during the period): -----Lecturing -----Facilitating/Coaching -----Interactive direction -----Modeling -----Discussion -----Other (please comment) 5. Learning activities (check all observed during the period): ___ Creating presentations ____ Test taking ___ Research ____ Drill and practice ___ Information analysis ____ Simulations ___ Writing ____ Hands on skill training ____ Other (please comment): 6. How essential was technology to the teaching and learning activities? ____ 1. Not needed; other approaches would be better ____ 2 Somewhat useful; other approaches would be as effective ____ 3. Useful; other approaches would not be as effective ____ 4. Essential; the lesson could not be done without it. Comment:
117 7. Technologies used by teacher (check all observed du ring the period): ____ Calculator ____ Presentation ____ CD ROM ____ Science Probe ____ Database ____ Shared Editor (wiki) ____ Desktop Computer ____ Simulation ____ Digital Camera ____ Spreadsheets ____ Drill/Practice ____ Tablet Computer ____ E mail ____ Video Camera ____ Graphics ____ Videoconferencing ____ Handheld Computer ____ Web Authoring ____ Laptop Computer ____ Web Browser ____ Library Database ____ Web Log ____ Outliner ____ Word Processing ____ Podcast ____ Other (please comment): 8. Technologies used by students (check all observed during the period): ____ Calculator ____ Presentation ____ CD ROM ____ Science Probe ____ Database ____ Shared Editor (wiki) ____ Desktop Computer ____ Simulation ____ Digital Camera ____ Spreadsheets ____ Drill/Practice ____ Tablet Computer ____ E mail ____ Video Camera ____ Graphics ____ Videoconferencing ____ Handheld Computer ____ Web Authoring ____ Laptop Computer ____ Web Browser ____ Library Database ____ Web Log ____ Outliner ____ Word Processing ____ Podcast ____ Other (please comment):
118 9. NETS Teacher Standards Addressed: ___ 1A.1. operating system procedures ___ 3C.2. teacher applies technology to develop students' creativity ___ 1A.2. routine hardware and software problems ___ 1A.3. content specific tools ___ 3D.1. class management facilitates engagement with technology ___ 1A.4. productivity tools ___ 1A.5. multimedia tools ____ 3D.2. technology integrated as a teacher tool ___ 1A.6. interactive communication tools ___ 3D.3. technology integrated as a student tool ___ 1A.7. curriculum based ___ 3D.4. student grouping varied as needed to facilitate learning presentations/publications ___ 1A.8. curriculum based collaborations ___ 4A.1. student learning of subject matter ___ 1A.9. appropriate technology selected assessed with technology ___ 2A.1. developmentally appropriate learning activities ___ 4A.2. teacher assesses student technology skills ___ 4A.3. teacher employs a variety of assessment strategies ___ 2A.2. technology enhanced instructional strategies ____ 6A.1. teacher models legal and ethical technology practices ____ 3A.1. learning experiences address content standards ____ 6A.2. teacher explicitly teaches legal and ethical technology practices ____ 3A.2. learning experiences address student technology standards ___ 6B.1. diverse learners enabled and empowered. ___ 3B.1. technology supports learner centered strategies ___ 6D.1. safe and healthy use of technology promoted ___ 3C.1. technology applied to develop students' higher order skills ___ 6E.1. equitable access to technology for all students Comments:
119 10. Three Minute Chart. During each 3 minute period, was technology in use by students and/or teachers, and was the time spent with technology used for teaching and learning (as opposed to recreation or routine tasks such as boot up and log on)? Technology :00 :03 :06 :09 :12 :15 :18 :21 :24 :27 :30 :33 :36 :39 :42 :45 :48 :51 :03 :06 :09 :12 :15 :18 :21 :24 :27 :30 :33 :36 :39 :42 :45 :48 :51 :54 is: In use by students Used for learning In use by teacher Used for learning 11. Estimated time technology used (if 3 minute chart is not used) Total minutes technology used by students ____________________ Minutes students used for learning ____________________ Total minutes technology used by teachers ____________________ Minutes teachers used for learning ____________________
120 APPENDIX B TECHNOLOGY INTEGRATI ON MATRIX
122 APPENDIX C DIGITAL BACKPACK APPLICATION RUBRIC Teacher Name_________________________________________ Rationale for Applying for the Grant 5 4 3 2 1 Clearly Defined Vaguely Defined Educational Uses for the Grant 5 4 3 2 1 Creative Non creative Inventive Routine Elaborate Small in scale Student Access 5 4 3 2 1 Accessible by Accessible by few Many students Students Measurable Outcomes/Plan 5 4 3 2 1 Highly Measurable Not Measurable Overall Impression of the Grant Proposal 5 4 3 2 1 Sophis ticated Nave Total Score out of 25: _________ Evaluator Initials: _________
123 APPENDIX D PROFESSIONAL DEVELOP MENT SURVEY Question Strongly Agree Agree Neutral Disagree Strongly Disagree I feel prepared to Integrate educational technology into the grade or subject I teach 5 4 3 2 1 I feel prepared to Support my students in using technology in their schoolwork 5 4 3 2 1 I feel prepared to Align my teaching with state and national learning standards 5 4 3 2 1 I feel prepared to Evaluate technology based work my students produce 5 4 3 2 1 I feel I have an Understanding & applying Fair Use & copyright law 5 4 3 2 1 I feel I have an Understanding of Locating & evaluating resources for my units 5 4 3 2 1 I feel I have an understanding of creating student multimedia presentations 5 4 3 2 1
124 Question Strongly Agree Agree Neutral Disagree Strongly Disagree I feel I have an understanding of creating environments for student online communication & collaboration 5 4 3 2 1 I feel I have an understanding using portable digital tools in my instruction 5 4 3 2 1 I feel I have an understanding of how to have students use portable digital tools for their learning and demonstration of learning. 5 4 3 2 1
125 APPENDIX E FOCUS GROUP INTERVIEW QUESTIONS SWOT: 1. 2. 3. development? 4. What perceived threats do you see in terms of professional development? (time, funding) Educator Proficiency: 5. Describe ways that you have furthered your own knowledge in the area of technology outside of school 6. Please describe any ways you u se technology to collect or manage student assessment data or processes. 7. Please describe any specific tools and strategies you use to assess student products created with technology. Effective Practice: 8. Give one example of an effective use of education technology in your classes. 9. How do you typically organize student work on computers? For example, do they work in centers, individually, collaboratively, etc.? 10. your students? 11. Describe any mechanisms in your department/grade, school, or district that regularly allow teachers to share ideas about powerful ways they have used technology with other educators.
126 APPENDIX F THREE STAGE OBSERVATION PR OCESS Pre Observation Teacher and observer discuss the lesson including content and methodology to be covered and used. Learning objectives are shared Connections to standards are noted Concerns are addressed and assistance is offered by the observer Observation The observer d ocuments the lesson including content and methodology used, the learning process, and if the learning objective was both noted and achieved. Post Observation Teacher reflects on instructional practice and the lesson Teacher demonstrates evidence of effec tiveness in addressing objectives Observer gives commendations and suggestions for growth
127 APPENDIX G SAMPLE OF TRANSCRIBE D TEACHER OBSERVATIO N NOTES LF: third grade teacher. The first teacher observed was LF. During the pre observation with LF, she indicated that she had three objectives for the lesson. They were: that students will be able to recall the definitions of plot, conflict, and resolution; students will be able to decipher important climactic events from non important events in the novel; and that students will be able to utilize various forms of technology to design and organize a plot sequence for the novel. Her lesson plan had objectives, an anticipatory set, pr ocedures, materials, essential questions, and an assessment in the form of an exit card to measure if the objectives were met. LF is a second year teacher of third grade. The observation of the language arts lesson with sixteen students began with the tea cher using circle time to review previous material and to explain the objectives and tasks for the day. Students were placed into three differentiated ability groups based asks to accomplish in order to demonstrate understanding of vocabulary terms and to create products that would show plot sequences from the novels the class was reading. Each group gathered around one of the laptops or desktops computers in the room as th eir explained the objectives, the instructions for the group, and a brief tutorial about how to use the equipment. One group was asked to take digital photographs to assemble a camera to create and then film an alternate plot and climax from the chapter they read. The third group used the desktop computers and the website to
128 assemble and describe the plot and climax activities from their chapters. The students worked independently for the majority of the class period and LF circulated from group to group, monitoring, clarifying, and answering questions as needed. While the video group was working, LF set up the LCD projector and the document camera as a make shift teleprompter to allow the students to film without having the hold a script. Effective public speaking skills were modeled and emphasized by LF for the students i n that group. As students completed their tasks, LF had them return to their seats where she resolution in their chapters along with any additional questions they mig ht have. The teacher used the laptop computer and the video camera to produce a video prior to the class to give students directions and the objectives. The students used a variety of technology including the desktop computer, laptop computer, digital c amera, a website, video camera, digital camera, and an LCD projector. Most of the items in the Digital Backpack were used in this lesson. In addition to the writing, thinking, analysis, and public speaking skills, the student groups that were working wit h cameras and websites practiced digital visual literacy skills. Spalter and van Dam (2008) define 94). In the observed lesson, the students accomplished this task as part of their group tasks. During the post observation it was noted that students were prompted by LF to go to the novel chapter for support or evidence as they produced their projects. The exit card at the conclusion of the class provided data for the teacher to monitor her effectiveness.
129 technology tools is integral to building an unders tanding of a concept. The teacher gives the students access to technology tools and guides them to appropriate rovided by the Digital Backpack enhanced the lesson by adding real world authentic tasks. RK: High School Special Education English teacher. The second teacher observed was RK. During her pre observation conference, RK noted that her objective was to have students compare and contrast quotes from a novel for conflicting perspectives by the characters. She teaches a tenth grade resource room English class to seven students and they are all special education classified. Her lesson plan had objectives, an anticipatory set, procedures, materials, essential questions, and an assessment. RK has been teaching for 18 years. During the lesson, RK was observed using the document camera in conj unction with a class reading a chapter in a novel. She placed the text under the document camera and had students follow along in their indiv id ual copies of the text. They were identifying quotes that would compare and contrast character perspectives. As they found quotes, they would be hand written by RK on a Venn dia gram which she then
130 also placed under the document camera. During the next part of the lesson, the students listened to an audio vers ion of the novel which RK played on her iPod. As the students listened, they took some notes in a pre prepared active rea ding guide. As students were doing this, RK modeled the task using the document camera. During the last ten minutes of the lesson, each student was given a netbook to continue work on an essay that analyzes a chosen character in the novel and uses the selected quotes as support for their thesis. As students worked on the netbooks, RK conferenced with each student to check for understanding and progress. During the lesson, the teacher used the laptop computer, an iPod (with speakers), and the document camera. The students viewed the visual reinforcers on the document camera, listened to the audio book on the iPod, and used netbooks to word process an essay they were writing for class. During the post observation it was noted that activities were purposeful and conducted with mutual respect between RK and the students. Structures that assisted students included two or more instructional modalities for each segment of the lesson. At any given time, students were given visual, auditory, or tac tile reinforcement to assist them in understanding content. The individual conferencing between RK and the students at the conclusion of the class provided data for the teacher to monitor her effectiveness and levels of student understanding. Using the Te chnology Integration Matrix, the lesson fell in an even split between in the con ventional use of tools (FCIT, 2010). In the learning environment
131 ICO provide multiple learning modalities for students to access the content of the curriculum. Primarily the students were not creating with the technology, except for word processing. Instead, they were the passive recipients of audio and visual content which they then classified. MR: High School Advanced Placement Chemistry teacher. The third teacher observed was MR. \ During the pre observation conference, it was noted that the lesson was not going to be a traditional classroom observation. Instead, eight st udents in her chemistry class had voluntarily elected to compete in the Chemistry Olympiad an academic competition between schools involving chemistry knowledge. The group meets after school to prepare for the competition. MR gave her students full acce ss to the Digital Backpack with the objective that they would use the tools to record their thoughts and prepare for the competition. Her planned objective for the group was that iad. eight students were working in pairs and each duo was responsible for different elements that they would have to present and respond to questions. They were performing sa mple experiments with these elements to become more familiar with the
132 reactions. The students prepared the elemental experiments themselves with the and spent 30 min utes filming their set up, observations, reactions, and brainstorming of their use of a polyurethane auger. After the experiment, they reviewed the video and discussed what they saw and areas that they needed to better develop. The second pair of studen ts used the audio recorder. As one of the students set up and ran the experiment that created ozone under the chemical hood, the other student recorded an ongoing narrative about what was going on, his observations and ideas for the future. At the conclu sion of the experiment, the pair listened to the audio and took notes on areas they wanted to remember or improve upon. The third pair used the digital camera to snap photos of the various stages of their experiment that injected carbon dioxide into carbo nic acid. They then used the photos to create a flow chart using Gliffy ; a web based flow chart program to help them remember the steps in the sequence. The last group was using one of the netbooks to research about methane safety. They took notes and b ookmarked websites using Diigo a web based reference and bookmarking program, as they worked on their task. Throughout the observation, the teacher circulated about the room, coaching and monitoring student progress. During the post observation conferenc e, the teacher compared this group to previous students who participated in the Chemistry Olympiad. She noted that while each year students are motivated, this year as a result of being able to use recording devices, she felt they were more reflective abo ut the process.
133 understanding of a concept. The teacher provides a context in which technology tools category as the technology enabled students to make connections to the world outside How essential was technology to previously mentioned, the Chemistry Olympiad teams had practiced in the past without Digital Backpack tools. However, the use of the tools this year is allowing them to be more reflective and to employ metacognition strategies where they review and think about their own learning. AG: Elementary school Spanish teacher. The fourth teacher observed was AG. She is a third year teacher assigned to four elementary schools. During the pre observation conference, she indicated that her objective was to have students create audio and vi sual postcards using Spanish language that describe an overview of an assigned Latin American country. She shared that since she meets with each of her 4 th grade classes only two times a week, this has been an ongoing project over a three week period. S tudents were previously given a sample post card with sentence stems
134 to complete using the Spanish phrases they were learning. Each student was also given a fact sheet on their country from the teacher that came from the CIA World Factbook located at https://www.cia.gov/library/publications/the world factbook/ to use for individual research on their assigned country. Previously in class they had completed an illustrated paper co py of their postcard. During the final class they would create a video or audio postcard explaining highlights of the country and encouraging students to visit that would be recorded in Spanish. The postcard would then be saved to a 4 th grade student in another class to view and then reply back to in either electronic or traditional writing. I observed the culminating class of the unit when students were expected to complete and turn in their work. At the beginning of the obse rvation, the objective and a flow chart of the tasks were projected via the document camera on the screen for students to view and refer to. The students quickly went to work in small groups. One group of three students was another three were using the movie function on the digital camera, two groups of three were using the webcam on the laptops, and another group was using the digital audio recorder. Groups generally did one practice take and then a final take for recordin g. The groups of students worked collaboratively and took on roles such as camera operator, holding cue cards, or recording the video/audio post card. As students completed their recording, they brought the camera over to the laptop and downloaded their film onto the laptop. AG then moved the files to a flash drive. The video/audio files on the flash drive would then be given and viewed by students in another fourth grade class and the observed class would receive videos from another class. Students ap peared excited by the work and AG circulated about the class and
135 hallway throughout the lesson, assisting, coaching, and clarifying for students. If worksheet to have the m learn and practice additional phrases in Spanish. At the end of the class, AG had the students orally share one thing they learned and one thing they had a question about. Most students shared facts about their countries and did not have many questions She asked the class if they enjoyed using the digital tools and During the post observation conference, the teacher indicated that she felt like she met the objectives, and estimated that 90% of the students went above and beyond her expectations. Her evidence was that students were able to correctly fill in the correct responses to a sample postcard that was completely in S panish. She shared that she had used the portable scanner from the Digital Backpack postcards which she then posted on her website for students and parents to view. Videos of students who have parent permission for their images to be on the website were also posted. Using the TIM, the majority of the lesson technology tools but the students used the tools on their own. Part six of the ICOT asks
136 provided by the Digital Backpack enhanced the lesson by adding real world authentic tasks and allowing students to practice public speaking skills in addition to the written and artistic component. RS: primary grade resource room special education teacher. The fifth teacher observed was RS. She is based at an elementary school. During the pre observation conference she s hared that she instructs very small classes of two to four students, mainly as pull out students in kindergarten or first grade. All of her students have Individual Education Plans (IEP) and in many cases the students have been placed into her less restri ctive environment as they were previously in the self contained Pre School Disabled program. She shared that they have been working on a math unit about shapes and her objective was to have students realize that common shapes are prevalent in the world and to have the students identify known shapes (circle, square, rectangle, triangle) in and about the school. She was planning on using multiple learning modalities to accomplish this goal. Her lesson plan had objectives, an anticipatory set, procedures, materials, essential questions, and an assessment. RS has been teaching for 17 years. The lesson began with the teacher placing a block of clay in front of her two students. They were each given small word blocks in the shape of a circle, square, rectan gle, and triangle. They were encouraged to press the wood block into the clay to embed a shape. They then compared the shapes in the clay to colored cards that the teacher projected on the screen using the document camera. Next, the students were told t
137 around the school to identify and then take a digital photo of the shapes they had learned. The teacher gave each student a digital camera and they went for a walk arou nd the school including the hallway, the library, and the stairwell. Students took approximately a dozen photos of shapes they identified including a fire extinguisher case (rectangle), a clock (circle), and a truss roof marker (triangle). After fifteen minutes, the students returned to the room and the cameras were connected to the computer to download the photos. The students showed the pictures they had taken to each other and the teacher facilitated the discussion to have them identify the shapes the others took. During the post observation conference, the teacher felt very positive about the lesson and the use of technology. She admitted that the lesson could have worked with the field trip and not taking pictures, but that the use of the camera pro vided not only a tactile component to the lesson, but also gives a permanent record for the students to access in subsequent classes. nts in the conventional use of tools (FCIT, 2010). The technology tools used in the lesson were the digital cameras, the document camera, the LCD projector, and the laptop computer. In the learning environment allowed the students to use it while they constructed meaning. Part six of the ICOT
138 provide some visual reinforcement as well as a way to capture the viewed shapes for further viewing. JK: Middle School life science teacher. The sixth observation was of JK. Dur ing the pre observation conference she shared that this week her students were looking at the importance of Charles D observed class would be a lab where students will be able to demonstrate the concept of adaptive radiation by observing how different types of bird beaks are best suited for gathering types of food. JK no ted that she has been using technology in the classroom for several years, but mostly for information dissemination through PowerPoint projections. In addition to the Digital Backpack, she shared that this year she was a paperless classroom by making all worksheets and information available on .pdf files on her website and by having students submit all work to her electronically through email. She said that as of March, she has been successful in her goal to that end. In the lesson that was going to be observed, she said that as part of the lab she was going to have many lab groups utilize much of the handheld recording devices that were part of the backpack to document results. At the beginning of class, PowerPoint not es were projected on the front board using the LCD projector. The teacher went over the daily plan and reviewed the Understanding by Design about natural selection which the students copied into their notebooks. After completing
139 the lecture, students were given the opportunity to ask the teacher questions. After clarifying several queries, the teacher explained the di rections for the lab. There were several stations where students would work in groups of four to see how much birdseed they could capture using several variations of simulated beaks. As students worked, clips of the most successful on their emotions including frustrations, excitement, and adaptations. They used the digital camera to document the comparison between using tweezers to gather birdseed and using hands. Stop watches were used by the students to keep track of time intervals. Data was recorded using small, dry erase boards. After students completed the lab, their documentation, and their data capture, th ey sent a group representative to As each group entered their data on the laptop and returned to their seats, the teacher shared that she would post on her website the medians and mode of the experiment data. This would allow students to complete the conclusions of their lab reports to test their hypothesis about adaptive radiation. During the post observation conference, the teacher felt that her students had met the objectives for the lesson. Her evidence of this was that she observed and noted students modeling the use of various types of bird beaks using everyday items such as hairpins, clothespins, and tweezers. She heard students document in their videos that s ome bird beaks are better for picking up insects and some are better for picking up seeds.
140 o the teacher provided opportunities for students to use the technology to work with others. not be past with good results, but the use of technology allowed for richer responses and data from the students that exceeded her expectations.
141 LIST OF REFERENCES Ahlfeld, K. (2010, February). Hands On learning with a hands off approach for professional development School Library Monthly 26 (1) 16 19. Amirian, S. (2007). D igital b ackpacks. Computers in the Schools 24 (1), 5 14. Archambault, L., Williams, M., & Foulger, T. (2010). Professional d evelopment 2.0: transforming teacher education pedagogy with 21st century tools Journal of Digital Learning in Teacher Education 27 (1), 4 12. Babbie, E.(2007) The Practice of social r esearch Belmont, CA: Wadsworth Barron, B. (2009). Advancing technological fluency of underrepresented youth and their teachers through project based learning opportunities Brigid B arron Retrieved June 6, 2010, from www stanford edu /~ barronbj / documents / Barron NSFcareer pdf Basham, J., Meyer, H., & Perry, E. (2010). The Design and application of the digital backpack Journal of Research on Technology in Education 42 (4), 339 359. Bebell, D., & O'Dwyer, L. (2010). E ducational outcomes and research from 1:1 computer settings Journal of Technology, Learning, and Assessment 9 (5), 1 43. Becker, H. (2000). Pedagogical motivations for student computer use that lead to student engagement Educational Technology 1 (1), 1 1 7. Becker, J. (2009) "K12 online conference 2009 | prove it technology integration: a synthesis of the research K12 Online Conference 2009 21 Oct. 2008. Web. 20 June 2009.
142 Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2009). Teacher preparation and student achievement Educational Evaluation & Policy Analysis 31 (4), 416 440 Brown, A.L. & C ampione, J.C. (1990). Communities of learning and thinking, or a context by any other name. Human Development, 21 108 125. Brown, K., Cummins, J., & Sayers, D. (2006). Literacy, technology, and diversity: teaching for success in changing times Boston : Allyn & Bacon. Chong, T. S. Impact Evaluation of the Intel Teach to the Future Program in Malaysia. unescobkk.org Retrieved October 6, 2010, from www.unescobkk.org/fileadmin/user_upload/apeid/Conference/13th_Conference/P apers/3.B.2._Impact_Evaluation_of_the_Intel_R__Teach_to_the_Future__Chong_ Seong_Toh_.pdf Clark, R. (1994). Media and m ethod. Educational Technology Research and Development 42 (3), 7 10 Cochran Smith, M., & Lytle, S. (1992).Communities for teacher r esearch. American Journal of Education 100 (3), 298 324. Cole, K., Simkins, M. & Penuel, W.R. (2002). Learning to teach with technology: strategies for inservice professional development Journal of Technology and Teacher Education, 10 (3), 431 455. Norfolk, VA: AACE. Considine, D., Horton, J., & Moorman, G. (2009). Teaching and reading the millennial gener ation through media literacy Journal of Adolescent and Adult Literacy 52 (6), 471 481. Cuban, L. (2003). Oversold and underused: computers in the classroom Cambridge: Harvard University Press. Cuban, L., Peck, C., & Kirkpatrick, H. (2002). Techno promote r d reams, student realities Phi Delta Kappan 83 (6), 472 480. Culp, K., Honey, M., & Mandinach, E. (2005). A retrospective on twenty years of educational technology policy Educational Computing Research 32 (3), 279 307. Dana, N. F., & Yendol Hoppey, D. (2008). Facilitator's guide to classroom research : Learning to teach and teaching to learn through practitioner inquiry (2 nd ed ition ). Thousand Oaks, CA: Corwin Press. Danielson, C., & McGreal, T. L. (2000). Teacher evaluation to enhance professional practice Alexandria, VA.: Association for S upervision and Curriculum Development.
143 Darling Hammond, L. (2007). Preparing teachers for a changing world : What teachers should learn and be able to do San Francisco: Jossey Bass. Deluna, J. (2006). Infusing technology into the k 12 classroom . New York: Cambria Press. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education New York: WLC Books. Dixson, M. D. (2010). Cr eating effective student engagement in online courses : What do students find engaging ?. Journal of the Scholarship of Teaching and Learning 10 (2), 1 13. Doering, A., Veletsianos, G., Scharber, C., & Miller, C. (2009). Using the technological, pedagogical, and content knowledge framework to design online learning environments and professional development Educational Computing Research 41 (3), 319 346. Duncan Howell, J. (2009). Teachers making connections: Online c ommunities as a source of personal learning. British Journal of Technology 41 ( 2), 324 340. Dwyer, D. C., Ringstaf, C., & Sandholtz, J. H. (1990). Teacher Beliefs and Practices Cuppertino, CA: Apple, Inc. Dyson, R. (2004); Strategic development and SWOT analysis at the U niversity of W arwick European Journal of Operation Research 152 pp. 631 640. Early Years Supporting pre birth to three, nursery and early primary care and education in Scotland. (n.d.). Supporting Curriculum for Excellence (CfE), assessment, community and lifelong learning Learning and Teaching Scotland Retrieved April 6, 2011, from http://www.ltscotland.org.uk/earlyyears English: A theoretical framework for research on learning to teach American Journal of Education 108 ( 1 ) 1 29 Ertmer, P., & Ottenbein Leftwich, A. (2010). Teacher technology change Journal of Research on Technology in Education 42 ( 3), 255 284. Ferdig R. (2006). Assessing technologies for teaching and learning: understanding the importance of technological pedagogical content knowledge. British Journal of Educational Technology 37 (5), 749 760. Finn, J. D., & Rock, D. A. (1997). Academic success among students at risk for school failure. Journal of Applied Psychology 82 (1 ), 221 34. Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2010). Program evaluation: alternative approaches and practical guidelines (4th Revised edition ). Boston : Ally n & Bacon.
144 Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74 (1), 59 109. George, D. S. (2010, September 15). U test scores, behavior. Washington Post pp. 1 3. Retrieved September 15, 2010, from www.washingtonpost.com/wp dyn/content/article/2010/09/14/AR2010091 407044_pf.html George, D & Mallery, P (2007). SPSS for w indows s tep by s tep: A s imple g uide and r eference, (8th Edition). Boston Allyn & Bacon. Glesne, C. (2006). Becoming qualitative researchers: an introduction (3rd edition). Boston:Allyn & Bacon. Gottfried, J., & McFeely, M.G. (1997). Learning a ll o ver the p lace: Integrating l aptop c omputers into the c lassroom. Learning & Leading with Technology, 25 (4), 6 11. Hake, R. (1998). Interactive engagement versus traditional methods: A six thousand student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66 (1), 64 74. H odge, W. & Jones J. (1999) Evaluating the role of waivers in systemic school reform. Evaluation and Program Planning. 23 (3), 267 286 Hu, S., & Kuh, G. (2002). Being ( d is)engaged in e ducationally p urposeful a ctivities: The i nfluence of s tudent a nd i nstitutional c haracteristics. Research in Higher Education 43 (5), 555 575. I nternational Society for Technology in Education | NETS for Students 2007. ISTE | Home Retrieved June 17, 2009, from http :// www iste org / Content / NavigationMenu / NETS / ForStudents /2007 Standar ds / NETS for Studen ts _2007. htm I nternational Society for Technology in Education | NETS for Teachers 2000. ISTE | Home Retrieved June 17, 2009, from http :// www iste org / Content / NavigationMenu / NETS / ForTeachers /2000 Standar ds / NETS for Teachers _2000. htm Kappan. (1995). Practices t hat s upport teacher development. Phi Delta Kappan 76 (1), 642 644. Kearsley, G & Shneiderman, B. (1999) Engagement t heory: A f ramework for t echnology b ased t eaching and l earning. E ducational Technology, 38 (5), 20 23 Knobel, M., & Lankshear, C. (2008). Digital l iteracies: c oncepts, p olicies and p ractices (New l iteracies and d igital e pistemologies) New York: Peter Lang Publishing.
1 45 Laird, T. (2004).Surfin' w ith a p urpose: e xamining h ow s pending t ime o nline is r elated to s tudent e ngagement. Student Affairs Online 5 (3), 1 5. Retrieved October 18, 2010, from www.studentaffairs.com/ejournal/Summer_2004 Laird, T. & Kuh, G (2004, May 30). Student e xperiences w ith i nformation t echnology and their r elationship to e ngagement. Annual Meeting of the Associa tion for Institutional Research Paper Presentation for the Association for Institutional Research, MA. Lance, W. (2005) Increasing v isual l iteracy s kills with d igital i magery: Successful m odels for u sing a s et of d igital c ameras in a c ollege of e ducation. T.H.E. Journal 32 (7), 24 26. Lauro, D.R. (1995). Five approaches to professional development compared. T.H.E. Journal, 22 (10), 61 65 Lawless, K & Pellegrino, J. (2007) P rofessional d evelopment in i ntegrating t echnology i nto t eaching and l earning: Knowns, u nknowns, and w ays to p ursue b etter q uestions and a nswers Review of Educational Research 77: 575 614 Learned, E.P., Christensen, C.R., Andrews, K.E., Guth, W.D.,(1965). Business p olicy: t ext and c ases Irwin, Homewood, IL. Maclean, M. S & Mohr, M. M. ), (1999). Teacher r esearchers at w ork New York: National Writing Project. Martin, W., Strother, S., Beglau, M., Bates, L., Reitzes, T., & Culp, K. (2010). Connection i nstructional t echnology p rofessional d evelopment to t eacher and s tudent o utcomes. JRTE 43 (1), 53 74. Mathews, J & Hudson, A (2001) Guidelines for e valuating p arent t raining p rograms. National Council on Family Relations 50 (1), 77 86. Retrieved December 2, 2010 from http://www.jstor.org/stable/585777 Matzen, N., & Edmunds, J. (2007). Technology as a c atalyst for c hange: t he r ole of p rofessional d evelopment. J ournal of Research on Technology in Education 39 (7), 417 430. Retrieved October 15, 2010, from the ERIC database. McMillan, J. H., & Wergin, J. F. (2006). Understanding and evaluating educational research (3rd ed ition ). Upper Saddle River, NJ: Pearson Prentice Hall. Meltzer, D. & Manivannan, K. (2002). Transforming the lecture hall environment: The fully interactive physics lecture. American Journal of Physics, 70(6), 639 654. Murray, T. (2006). XIV. Online assessmen t, measurement, and evaluation: emerging practices (pp. 238 259). Hershey, PA: Idea Group, Inc. No Child Left Behind Act of 2001, 20 U.S.C. § 6319 (2011).
146 Papert, S (1987, May ). A c ritique of t echnocentrism in t hinking a bout the s chool of the f uture. Opportunities for Creativity, Innovation, and New Activities Lecture conducted in Sofia, Bulgaria. Part I:Patterns of change. Cupertino, CA: Apple Computer, Inc. Retrieved April 16, 2011, from http://www.apple.com/education/k12/leadership/acot/library.html Pea, R., Roschelle, J., Hoadley, C., Gordin, D., & Means, B. (2000). Changing h ow and w hat c hildren l earn in s chool with c omputer b ased t echnologies. Children and Computer Technology 10 (2), 76 101. Pellegrino, J. W., & Quellmalz, E. S. (2010). Perspectives on the i ntegration of t echnology and a ssessment. Journal of Research on Technology in Education 43 (2), 119 134. Plair, S. (2008). R evamping p rofessional d evelopment for t echnology i ntegration and f luency. Clearing House: A Journal of Educational Strategie s, Issues and Ideas 82 (2), 70 74. Polly, D., & centered professional development. Educational Technology Research and Development 58 (5), 1042 1629. Pritchard, R., & Marshall, J. (2002). Profes sional d evelopment in h ealthy vs. u nhealthy d istricts. School Leadership and Management` 22 (2), 113 141. Rickey, D & Stacy, A. (2000) The r ole of m etacognition in l earning c hemistry. Journal of Chemical Education 77 (7), 915 Robert, K. (1991). The t echnological r evolution c omes to the c lassroom. Change 23 (1), 10 20. Salomon, G. (1977). Reexamining the m ethodology of r esearch on m edia and t echnology in e ducation. Review of Educational Research 47 (1), 99 120. Retrieved May 25, 2010, from the ERIC database. Sandholtz, J. H., Ringstaff, C., & Dwyer, D. C. (1997). Teaching with technology: creating student centered classrooms New York: Teachers College Press. Saphier, J. (2009) The s killful t eacher New York: Research for Better Teach ing. Schacter, J. (1999). The i mpact of e ducational t echnology on s tudent a chievement. Milken Foundation Retrieved June 25, 2009, from http :// www mff org / pubs / ME 161. pdf Shapley, K., Sheehan, D., Maloney, C., & Caranikas Walker, F. (2010). Evaluating the i mplementation f idelity of t echnology i mmersion and its r elationship with s tudent a chievement. Journal of Technology, Learning, and Assessment 9 (4), 5 68.
147 Shinkfield, A. J., & Stufflebeam, D. L. (2007). Evaluation t heory, m odels, and a pplications ( r esearch m ethods for the s ocial s ciences) San Francisco: Jossey Bass. Schlechty, P. (1994) Increasing Student Engagement Missouri Leadership Academy Shriner, M., Clark, D., Nail, M., & Schlee, B. (2010, Spring). Social s tudies i nstruction: Changing t eacher c onfidence in c lassrooms e nhanced b y t echnology. The Social Studies 101 37 45. Skinner, E.& Belmont, M. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology 85 (4), 571 581 Spalter, A. M., & Dam, A. v. (2008). Digital v isual l iteracy. Theory Into Practice 47 93 101. Stake, R. (1995).The art of case study research. Thousand Oaks, CA: Sage Publications. Sternberg, R. (1998) Metacognition, abilities, and developing expertise: What makes an expert student. Instructional Science 26 (1 2 ), 127 140 Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications. San Francisco: Jossey Bass. Stufflebeam, D. (200 0). CIPP. Evaluation m odels v iewpoints on e ducational and h uman s ervices e valuation s econd e dition (2nd ed ition ). New York: Springer. Stufflebeam, D. CIPP Checklists. Evaluation Center Retrieved October 15, 2010, from www.umich.edu/evalctr/checklists Tan, L., Guo, L., & Bophy, J. (2009). Portraits of n ew l itera cies in t wo s ingapore c lassrooms. RELC Journal 41 (1), 5 17 Technology Integration Matrix. (2007). Florida Center for Instructional Technology Retrieved November 5, 2010, from http://fcit.usf.edu/matrix/ Tenenbaum, G., & Goldring, E. (Spring,, 1989). A m eta a nalysis of the e ffect of e nhanced i nstruction: c ues, p articipation, r einforcement and f eedback, and c orrectives on m otor s kill l earning." Journal of Research and Development in Education, 22 (3), 53 64. Tiene, D and Ingram, A (2001).Exploring c urrent i ssues in e ducational t echnology. McGraw Hill. Trilling, B. (n.d.). The Partnership for 21st Century Skills Home. The p artnership for 21st c entury s kills h ome Retrieved March 16, 2011, from http://www.p21.org
148 Using Technology to Raise the Achievement of ALL Students. (n.d.). Using t echnology to r aise the a chievement of all s tudents Retrieved July 8, 2009, from http :// www accessible tech 4 all org / Vavoula, G., & Sharples, M. (2007). Future t echnology w orkshop: A collaborative method for the design of new learning technologies and activities. Computer Supported Collaborative Learning 2 393 419. Vella, F. (1992) Medical education: c apitalizing on the lecture method. FASEB Journal, 6 (3), 811 812 Walke r C. & Green B (2009). The relations between student motivational beliefs and cognitiv e engagement in high schools. Journal of Educational Research 102 (6), 463 471. Walters, M., & Fehring, H. (2009).An i nvestigation of the i ncorporation of i nformation and c ommunication t echnology and t hinking s kills with y ear 1 and 2 s tudents. Australian Journal of Language and Literacy 32 (3), 258 272. "WestEd: Fair and Unfair Use of Copyrighted Materials." WestEd WestEd, n.d. Web. 4 Jan. 2011. < http://www.wested.org/cs/we/print/docs/we/permission3.htm>. Wilbur, D. (2010). Using b logs, w ikis, and d igital s tories in the e nglish c lassroom New York: Heinemann Wishart, J., & Blease, D. (1999). Theories underlying perceived changes in teaching and learning after installing a computer network in a secondary school. British Journal of Educational Technology, 30(1), 25 42. Wolfson, R., & Amirian, S. (2003). Digital backpacks. Proceedings of Society for Information Technology & Teacher Education International Conference 2003, 837 839. Chesapeake, VA: AACE. Retrieved March 17, 2011, from http://www.editlib.org/p/18031 Worthen, B. R., Sanders, J. R., & Fitzpatrick, J. L. (2004). Program evaluation: alternative approaches and practical guidelines (3rd ed.). Boston: Allyn and Bacon. Zhao, Y., & Bryant, F. (2007). Can t eacher t echnology i ntegration t raining a lone l ead to h igh l evels of t echnology i ntegration Electronic Journal for the Integration of Technology 6 53 61. Retrieved October 11, 2010, from http://ejite.isu.edu/Archive.html
149 BIOGRAPHICAL SKETCH Barry A. Bachenheimer obtained his BA in political science from the State University of New York at Albany, his Ed.M in socia l studies education from Rutgers University, and his MA in administration and supervision from Montclair State University. He received his Ed.D from the University of Florida in August, 2011. He has worked in public school education as a teacher, supervi sor, staff developer, principal, and curriculum director in New Jersey. He is also an adjunct professor at Montclair State University.