An Evaluation of Professional Development on Using Student Response Systems and Interactive Whiteboards for Formative As...

MISSING IMAGE

Material Information

Title:
An Evaluation of Professional Development on Using Student Response Systems and Interactive Whiteboards for Formative Assessment in the Middle Schools of a Southeastern School District
Physical Description:
1 online resource (156 p.)
Language:
english
Creator:
Fuller,Julia Susanne
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ed.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Curriculum and Instruction (CCD), Teaching and Learning
Committee Chair:
Dawson, Kara M
Committee Members:
Adams, Alyson
Griffin, Cynthia C
Drexler, Wendy

Subjects

Subjects / Keywords:
adult -- assessment -- clickers -- design -- development -- education -- evaluation -- formative -- instruction -- srs -- teacher -- technology -- whiteboard
Teaching and Learning -- Dissertations, Academic -- UF
Genre:
Curriculum and Instruction (CCD) thesis, Ed.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
The purchase of 21st-century technologies for each middle school teacher in my school system coinciding with a historic lack of significant professional development in technology integration provided the impetus for the study. To address the problem, professional development focused on helping teachers use student response systems and mobile interactive whiteboards for formative assessment. The professional development incorporated adult learning theory, professional development literature, and instructional systems design. This study examined the design, development, and implementation of the technology-based professional development. This study used a combination of qualitative and quantitative methods for collecting and analyzing data within the framework of Guskey's Five Levels of Professional Development Evaluation (1998) to assess and improve the effectiveness of the professional development. Data-collection methods included Likert-Scale questionnaires about perceptions and organization support, rubrics for evaluating the learning and its application, and field notes from an observation cycle for examining use of new knowledge and student engagement. The findings revealed that the professional development was effective and provided information for developing plans to improve my professional practice. Additional outcomes point to future research about implementing an instructional coaching model to serve teachers in their technology-integration needs. This work is significant in that it demonstrates using a systematic framework for evaluating professional development. Few professional development evaluations assess effectiveness beyond the participants' perceptions of the experience. Completing comprehensive evaluations is important for continuous improvement of professional development. Recommendations for coordinating a professional development evaluation in other organizations are provided.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Julia Susanne Fuller.
Thesis:
Thesis (Ed.D.)--University of Florida, 2011.
Local:
Adviser: Dawson, Kara M.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043254:00001


This item is only available as the following downloads:


Full Text

PAGE 1

1 AN EVALUATION OF PROFESSIONAL DEVELOPMENT ON USING STUDENT RESPONSE SYSTEMS AND INTERACTIVE WHITEBOARDS FOR FORMATIVE ASSESSMENT IN THE MIDDLE SCHOOLS OF A SOUTHEASTERN SCHOOL DISTRICT By JULIA SUSANNE FULLER A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION UNIVERSITY OF FLORIDA 2011

PAGE 2

2 2011 Julia Susanne Fuller

PAGE 3

3 To my husband, Chet, and our children Co rene and Jenelle

PAGE 4

4 ACKNOWLEDGMENTS I thank my husband, Chet, for his love and endurance throughout this degree program and dissertation process without which it would have been most difficult to persevere. I thank our two children, Corene and Jenelle, for their love, patience, and understanding as well. The encouragement of my family was an essential component in my co mpletion of this dissertation. I thank Dr. Kara M. Dawson, chair of my committee, for her guidance and expertise through this entire pro gram and especially during this dissertation process. helped me to achieve what previously seemed impossible. I am grateful for her mentorship and for helping me improve as a reflective practitioner. This dissertation would also not have been possible without the support of my committee members. I thank them all Drs. Alyson Adams, Wendy Drexler, and Cynthia Griffin for their dedication, encouragement, and expertise during th e planning and implementa tion of this doctoral project. I acknowledge the teachers at my schools for their participation in this project. I appreciate their feedback and willingness to help me grow in my profession during this reflection process. I also admire their interest in implementing technologies for formative assessment to influence student achievement. I also thank my colleagues Mary Risner, Wendy Athens, Julia Carpenter, and Barry Bachenheimer for their encouragement and insight throughout the d octoral journey.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURES ................................ ................................ ................................ .......... 9 LIST OF ABBREVIATIONS ................................ ................................ ........................... 10 ABSTRACT ................................ ................................ ................................ ................... 11 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 13 A Need for Professional Development Evaluation ................................ .................. 13 Role in the Field of Educational Technology ................................ ........................... 15 Impetus for the Study ................................ ................................ .............................. 16 Purpose of the Study ................................ ................................ .............................. 18 Components of the Study ................................ ................................ ....................... 20 Summary ................................ ................................ ................................ ................ 21 2 LITERATURE REVIEW ................................ ................................ .......................... 23 Adult Learning Theory ................................ ................................ ............................. 23 Professional Development ................................ ................................ ...................... 27 Technology Based Professional Development ................................ ....................... 31 Instructional D esign of Professional Development ................................ .................. 33 A Framework for Technology Based Professional Development ............................ 35 3 METHODOLOGY ................................ ................................ ................................ ... 41 Guiding Questions ................................ ................................ ................................ .. 42 Context of the Study ................................ ................................ ............................... 42 Participants ................................ ................................ ................................ ............. 43 Description of the Professional Development ................................ ......................... 44 Overview ................................ ................................ ................................ .......... 45 Lesson Development ................................ ................................ ........................ 45 Lesson Goals and Objectives ................................ ................................ ........... 47 Lesson Activities ................................ ................................ ............................... 48 Data Collec tion ................................ ................................ ................................ ....... 54 Guiding Question for Level 1: ................................ ........... 55 Guiding Question for Level 2: ................................ ............. 56 Artifact ................................ ................................ ........... 57 ning Artifact Rubric ................................ ............................... 57

PAGE 6

6 Post Instruction Reflective Journal ................................ ................................ ... 58 Guiding Question for Level 3: Organization Support and Change ......................... 59 Guiding Questions for Levels 4 and 5: Skills and Student Learning Outcomes ................................ ................................ 60 Pre Observation Interview ................................ ................................ ................ 61 Observation ................................ ................................ ................................ ...... 62 Post Observation Interview ................................ ................................ .............. 65 Data Analysis ................................ ................................ ................................ .......... 66 Quantitative Data Analysis ................................ ................................ ............... 66 Qualitative Data Analysis ................................ ................................ .................. 70 Information for Triangulation ................................ ................................ ............. 71 Timeline ................................ ................................ ................................ .................. 72 Trustworthiness and Limitations of the S tudy ................................ ......................... 73 Summary ................................ ................................ ................................ ................ 74 4 RESULTS ................................ ................................ ................................ ............... 85 Participation in the Study ................................ ................................ ........................ 85 Guiding Questions ................................ ................................ ................................ .. 86 Guiding Question for Level 1: ................................ ........... 87 Guiding Question for Level 2: ................................ ............. 88 Artifact Rubric ................................ ............................... 89 Post Instruction Reflective Journal ................................ ................................ ... 90 Guiding Question for Level 3: Organization Support and Change ......................... 90 Skills and Student Learning Outcomes ................................ ................................ 91 Pre Observation Intervi ew ................................ ................................ ................ 92 Observation ................................ ................................ ................................ ...... 94 Post Observation Interview ................................ ................................ .............. 97 Summary ................................ ................................ ................................ ................ 98 5 DISCUSSION AND IMPLICATIONS ................................ ................................ ..... 107 Summary of the Study ................................ ................................ .......................... 107 Finding s ................................ ................................ ................................ ................ 110 Conclusions ................................ ................................ ................................ .......... 112 Guiding Question for Level 1: Participant Reactions ................................ ......... 112 Guiding Question for Level 2: ................................ ........... 113 Guiding Question for Level 3: Organization Support and Change ....................... 115 Guiding Question for Level 4: .... 117 Guiding Question for Level 5: Stude nt Learning Outcomes ................................ 122 Additional Outcomes ................................ ................................ ............................. 124 Implications for Future Work and Research ................................ .......................... 125 Formative Assessment to Meet Individual Needs ................................ ........... 126 More Time to Work with the Technologies ................................ ..................... 127 Ins tructional Coaching ................................ ................................ .................... 127 ................................ ................................ 129

PAGE 7

7 ............................ 130 APPENDIX A SESSION HANDOUT ................................ ................................ ........................... 134 B POST INSTRUCTION REFLECTIVE JOURNAL ................................ .................. 135 C PERCEPTIONS QUESTIONNAIRE ................................ ................................ ...... 136 D ARTIFACT RUBRIC ................................ .............. 137 E ORGANIZATI ON SUPPORT QUESTIONNAIRE ................................ .................. 138 F PRE OBSERVATION INTERVIEW ................................ ................................ ...... 139 G OBSERVATION FIELD NOTES RECORD ................................ ........................... 140 H OBSERVATION RUBRIC ................................ ................................ ..................... 141 I POST OBSERVATION INTERVIEW ................................ ................................ .... 142 J INTERVIEW CODING PROTOCOL FORM ................................ .......................... 143 K POST INSTRUCTION REFLECTIVE JOURNAL EXCERPTS ............................. 144 L EXAMPLE OF CODING QUALITATIVE DATA ................................ ..................... 148 LIST OF REFERENCES ................................ ................................ ............................. 149 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 156

PAGE 8

8 LIST OF TABLES Table page 2 1 Application of Adult Learning Assumptions during the Professional Development ................................ ................................ ................................ ...... 38 2 2 Alignment of Professional Development Core Features to the Technology Based Professiona l Development ................................ ................................ ....... 38 3 1 Alignment of Evaluation Levels and Guiding Questions ................................ ..... 76 3 2 School Facts for Middle Schools A & B ................................ .............................. 76 3 3 Description of the Professional Development ................................ ..................... 77 3 4 Clicker Strategy Components ................................ ................................ ............. 81 3 5 Alignment of the Five Evaluation Levels, Guiding Questions, and Data Collection Methods ................................ ................................ ............................. 82 3 6 Implementation Timeline ................................ ................................ .................... 84 4 1 Participation in the Study ................................ ................................ .................... 99 4 2 Means and Standard Deviations for the Perceptions Questionnaire ................ 100 4 3 Mean s and Standard Deviations for the Artifact Rubric 100 4 4 Means and Standard Deviations for the Organization Support Questionnaire 101 4 5 Frequency Data for the Observation Rubric ................................ ..................... 101 4 6 Pre Observation Interview Taxonomy ................................ .............................. 102 4 7 Classroom Observation Taxonomy ................................ ................................ .. 103 4 8 Post Observation Interview Taxonomy ................................ ............................. 105

PAGE 9

9 LIST OF FIGURES Figure page 1 1 Framework for the Components of the Study ................................ ..................... 22 2 1 Relationship between the Literature and Purpose ................................ .............. 39 2 2 Proposed Framework for Designing Technology Based Professional Development ................................ ................................ ................................ ...... 40 5 1 Venn Diagram Highlighting S imilarities A mong the Pre Observation Interview and Observa tion Data ................................ ................................ ....... 133 5 2 Connections Among the Data From the Observation Field Notes and the Post Observation Interviews ................................ ................................ ............. 133 L 1 Exampl ................................ ... 148

PAGE 10

10 LIST OF ABBREVIATION S ADDIE Components of Instructional Systems Design: Analysis, Design, Development, Implementation, and Evaluation. ISD Instructional Systems Design : A lso referred to as instructional design. ITS Instructional Technology Specialist. IWB Interactive Whiteboard: A device controlled by a stylus that provides an interactive display on the computer for projection. The Interactive Whiteboards used in th e study were mobile and contained a screen for displaying clicker data. SRS Student Response Systems: A lso referred to as clickers. These devices allow learners to submit multiple choice and numeric answers to a database f or data collection.

PAGE 11

11 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education AN EVALUATION OF PROFESSIONAL DEVELOPMENT ON USING S TUDENT RESPONSE SYSTEMS AND INTERACTIVE WHITEBOARDS FOR FORMATIVE ASSESSMENT IN THE MIDDLE SCHOOLS OF A SOUTHEASTERN SCHOOL DISTRICT By Julia Susanne Fuller August 2011 Chair: Kara Marie Dawson Major: Curriculum and Instruction The purchase of 21st ce ntury technologies for each middle school teacher in my school system coinciding with a historic lack of significant professional development in technology integration provided the impetus for th e study To address the problem, professional development fo cused on helping teachers use student response systems and mobile interactive whiteboards for formative assessment. The professional development incorporated adult learning the ory professional development literature, and instructional systems design. Th is study examined the design, development, and implementation of the technology based professional development Th is study use d a combination of qualitative and quantitative methods for vels of Professional Development Evaluation (1998) to assess and improve the effectiveness of the professional development Data collection methods included Likert Scale questionnaires about perceptions and organization support r ubrics for evaluating t he learning and its application and field notes from an observation cycle for examining use of new knowledge and student engagement The findings revealed that the

PAGE 12

12 profess ional development was effective and provided information for developing plans to im prove my professional practice. Additional outcomes point to future research about implementing an instructional coaching model to serve teachers in their technology integration needs. This work is significant in that it demonstrates using a systematic framework for evaluating professional development. Few professional development evaluations Completing comprehensive evaluation s is important for continuous improvement of profe ssional development. R ecommendations for coordinating a professional development evalu ation in other organizations are provided

PAGE 13

13 CHAPTER 1 INTRODUCTION This project consists of an introduction, a literature review, methodology, results, and conclusion. The first component is the introduction, which details the impetus for the study and an overview of the framework for the study. The literature review is a synthesis of my understanding regarding the research and theory concerning my role in educational technology. The next chapter explains the methodology for data collection and analysis. The results section presents the findings from the study I discuss and interpret the data in the last chapter which details the overall experience and its contributions to my learning, including implications for future work and research. A Need for Professional Development Evaluation Appropriate professional development is necessary in a variety of settings and particularly in education for instructional improvement ( Desimone 2009; Guskey, 2002b ). To provide adequate professional develop ment and support, it is essential to address adult learning assumptions professional development research, and instructional design issues when implementing professiona l development (Desimone, 2009; Guskey, 2002b; Hill, 2007; Knowles, 1980; Knowles et al 1998 ; Reiser & Dempsey, 200 2 ). development, and implementation of professional development opportuni ties are important to ensure successful outcomes. The challenge becomes knowing whether the professional development is indeed effective Evaluating professional development is vital for determining the outcome s regarding its purpose affecting teacher lea rning and student achievement (Guskey, 2002a). Literature in the field confirms the importance of evaluating professional

PAGE 14

14 development for improving practice (Kirkpatrick et al., 1998; Guskey, 1998, 2000, 2002a) ; however, few professional development evalu ations assess effectiveness Implementing a systematic and comprehensive evaluation plan is important for continuous improvement of professional development. Just as evaluation is significant during instructional design, evaluating professional development at levels beyond the participants' initial impressions ensures that the impact of the professional development is evident in the classroom (Guskey, 2002a) This study examines the effectiveness of a technology based professional development series for helping teachers use the Student Response Systems ( SRS ) and mobile Interactive Whiteboard (IWB) data for formative assessment. Implementing formative assessment to affect learning can lead to improve ment in student achievement (Guskey, 2003a; Stiggins & Chappuis, 2005; Wiggins & McTighe, 1998). Incorporating SRS throughout instruction coinciding with clicker strategies provides feedback for earning needs (Bruff, 209b). The foundation for effectively accomplishing such strategies is appropriate professional development for helping teachers use the technologies for formative assessment that influences instruction (Guskey, 2002b; Hill, 2007). Evaluation is essential to ensure that professional development achieves its purpose (Guskey 2002a) This study is significant because it demonstrates the role of literature in designing and implementing professional development, and it uses a comprehensi ve approach to

PAGE 15

15 determine the value of the professional development for supporting teachers in their technology integration and affecting student engagement. This study detail s important considerations for the instructional design of professional development and demonstrate s the value of evaluating professional development in educational settings. Role in the Field of Educational Technology In my Instructional Technology Speci alist ( ITS ) role I help teachers implement technology as an effective instructional tool for collecting formative data and engaging students. As a leader practitioner I reflect on and evaluate my application of the research literature related to professio nal development and instructional design to refine my practice for the benefit of teachers and students My professional goal as an ITS is to foster among educators an understanding of technology as an enabler for teaching and learning. As technology is frequently changing and research based instructional practices are continually developing, my goal entails using technologically innovative resources to increase instructional effectiveness, academic achievement, and engagement Accordingly, I am cognizan t of the leadership role that comes with my position, and I work toward improving my skills, which include designing and implementing helping teachers integrate new technologies into their curriculum As an ITS I facilitate technology based professional development in a variety of century technologies. I work to use professional development practices supported by research in the field (Bradshaw, 20 02; Desimone, 2009; Garet et al., 2001; Guskey, 2002 b 2003; Hill, 2007, 2009; Mouza, 2003; National Staff Development Council, 2001; Sparks, 2002; Wilson & Ball, 1996). I also integrate adult learning principles (Knowles, 1980; Knowles et al., 1998) and

PAGE 16

16 instructional design methods (Alessi & Trollip, 2001; Brown & Green, 2006; Mager, 1997; Morrison, Ross, & Kemp, 2007; Reiser & Dempsey, 2002; Schnackenberg et al., 2001) in my work. As a part of my work, I attempt to identify teachers' limitations in the area of technology integration and help them with educational application of the technologies by designing and developing sessions that address their needs (Morrison, Ross, & Kemp, 2007; Schnackenberg et al., 2001) and promote a community of practice. I r eflect on my practice and evaluate its merit based on my improvement as a tool that positively affects students in the classroom (Guskey, 2002; Hill, 2007; National Staff Development Council, 2001). Impetus for the Study The impetus for the study stems from a problem in my school system. Widespread integration of 21st century learning technologies among the teachers in my school system requires adequate professional development and support so that teachers may use the technologies for the benefit of the students. There is a historic school teachers, which hinders them from independen tly using these new technologies during instruction. Not only is it difficult to learn a new technology, but application may not occur without appropriate support (Williams & Kingham, 2003). A lack of technology supported pedagogical knowledge and skills is an identified hindrance to technology integration, and conducting professional development is a strategy to overcome this barrier (Hew & Brush, 2006). Thus, there is a need for professional development and support to integrate these technologies into the curriculum. This

PAGE 17

17 section details the problem and related issues that give substance to the purpose. A contributing factor to the need for technology based professional development is the recent purchase by the school system of 21 st century lea rning technologies. During school year 2009 2010, my school system purchased a SRS and mobile IWB for every secondary core academic classroom. Classroom implementation of 21 st century technologies can be daunting for teachers. A study by Williams & King ham (2003) suggests that of preparation for integrating technology into their lessons. It takes time to learn to use and integrate new technologies for the benefit of the students. Middle school teachers in my school system have an average of 14.59 years of teaching experience (Georgia Department of Education, 2010 b ). Many teachers did not receive technology integration training while in college due to a lack of nee d for such courses at the time. Currently, a local state university offers only one course Teaching, Learning and Technology Integration as a part of the Bachelor of Science degree in middle school education (Georgia State University, 2010). As a result, there is a compelling need in my schools for professional development focused on technology integration. In my school system the implementation of technology based professional learning started with the recent hiring of several ITS s Previously, the only option for teachers desiring to gain skills in technology integration has been to take a college course or pay for a class at the regional Educational Technology Center. In each of these situations, past strategies for professional development did not co nsistently employ research

PAGE 18

18 based best practices to meet the needs of the teachers, as some classes were one day sessions located off campus and lacked a content focus. Analysis of survey data from middle school teachers in the district has indicated a nee d for appropriately designed professional development (Fuller, 2010) that focuses st century technologies (Fuller, 2009). At the start of school year 2010, I conducted a needs assessment to help define the proble m for the study and purpose of the study. This included reviewing district technology goals and school improvement plans, as well as meeting with the principals and gaining teacher feedback from school based leadership teams. Based on a need to improve u se of the available technologies for collecting formative data, I developed four sessions for this study focus ing on using technology for formative assessment to promote increased student engagement. Because there is not a strategic method in place i n my school system for evaluating the technology professional development conducted by the ITSs, this study established an opportunity for developing a plan for continuous improvement of the technology based professional development in which teachers parti cipate. Purpose of the Study The purpose of this mixed method research study was to evaluate the design, development, and implementation of the technology based professional development I provide d to teachers at two Georgia middle schools regarding integra tion of the SRS Development Evaluation (1998 2000, 2002a ) serve d as a framework for the data collection components of the study as outlined in Figure 1 1.

PAGE 19

19 This study is signifi cant in that it caused me to consider ways to improve the professional development I facilitate. This study suggests ideas to share with other ITSs and professional development coordinators. The components demonstrate my understanding of theory, research, and instructional design and my ability to implement an academic inquiry that will benefit the teachers with whom I work as well as other practitioners in educational technology. Accordingly, during this experience I studied the design of the p rofessional development I conducted related to integrating the SRS and mobile IWB into instruction for formative assessment at the middle schools in my school system. This study provided information concerning the effect of the professional development I conducted and the ways in which the teachers used the tools for formative assessment to elicit engagement among their students, which is an effective instructional strategy (Hake, 1998). Professional Development E valuation (1998 2 000, 2002a ) provided data for informing and improving my practice and implementing future professional development in the school district. The model also offered information es the technologies into instruction. Additionally, the literature review is the foundation of knowledge I gained in the field and outlines the importance of applying research based strategies to my technology based professional development. The study dem onstrates my ability to apply the evaluation component of instructional systems design ( ISD ) to my professional practice. The evaluation component of ISD bridges the teaching and scholarship components of the study Specifically, the study examined the p rofessional development I provided regarding the

PAGE 20

20 needs of the teachers, as well as data for improving my future instruction. In addition, Evaluation (1998 2000, 2002a ) helped me determine if I have adequately applied adult learning pri nciples and professional development concepts during the design and implementation of the instruction I facilitate Components of the Study The central focus of the study was to assess the technology based professional development that I provided. The Professional Development Evaluation (1998 2000, 2002a ), addressed teaching, scholarship, and leadership components of the study with the scholarship component of the study spanning all five levels of the e valuation protocol as noted in Figure 1 1. Guiding questions for the Evaluation (1998 2000, 2002a ). The data collected to answer these questions helped me reflect deeply on my tea ching. For this study I examined the effectiveness of the instruction I provided the middle school teachers. Research strategies discussed in the literature review guided the instructional design of the professional development I conducted for supporting teachers with technology integration. Adult learning principles and professional development literature helped me deliver instruction for helping teachers implement technology tools for student engagement, specifically during formative data col lection. Goals for my instruction address the elements pinpointed as areas of need (Mager, 1997; and developed my lessons based on the established goals and revised as needed based on formative evaluations (Reiser & Dempsey, 2002 ). As indicated in Figure 1 1, 2000, 2002a )

PAGE 21

21 guided the data collection process for the components of this study as researc h questions are aligned to each Level 2000, 2002a ) research based model for the study helped facilitate thoughts about future improvement in the instructional design of the professional development I conducted. Summary The impetus f or the study stems from a problem in the middle schools at which I work. All of the core academic teachers received 21 st century technologies, a SRS and mobile I W B, for their classrooms. There is a need for adequate professional development and supp ort so that teachers can use the technologies for formative assessment to benefit the ir students. Due to a lack of training and support in previous years, the teachers have difficulty using new technologies independently during instruction. Additionally, no strategic method was in place for evaluating the technology Levels of Professional Development Evaluation (1998 2000, 2002a ) provided an opportunity for evaluating and imp roving the technology based professional development in the schools. The following four chapters include a literature review, methodology, results, and discussion and implications. The literature review is a synthesis of my understanding regarding the res earch and theory concerning my role in educational technology. The next section explains the methodology for data collection and analysis. The results section presents the findings. In the last section I discuss and interpret the data, which details the future work and research.

PAGE 22

22 Figure 1 1 Framework for the Components of the Study

PAGE 23

23 CHAPTER 2 LITERATURE REVIEW Each of the elements described i n this literature review was essential for this study which examined the design, development, and implementation of the technology based professional development I provided. The development of this study considered literature from adult learning theory, professional development, technology based professional development, instructional design, and evaluation of professional development. Figure 2 1 shows the areas of literature that contributed to the evaluation process for providing instruction that met t technology, and refined my professional practice. In addition, as Beile & Boote (2005) suggest, this literature review moves beyond summarizing the research into a synthesis of the various literature bases and the ir application to the foundation of this study. Adult Learning Theory Andragogy is a set of methods and techniques regarding the characteristics of adult learners (Knowles, 1980) and is essential to consider when designing professional development for prac ticing teachers. Using these methods and techniques to design instruction helps facilitate learning for adult learners. The following section describes the assumptions and gives examples in educational practice. According to Knowles et al. (1998) learnin g should be oriented to an apparent learning need so the learner understands why the learning is important. Traditional learners may not consider why a particular concept is valuable for them to learn, whereas with adult learners instruction should includ e information about why it is necessary to learn about a particular topic (Knowles et al., 1998). Adult learners need to understand why and how things are important (Knowles et al., 1998). A facilitator

PAGE 24

24 may accomplish this by creating a learning environm ent that focuses on practicality and relevance of the content. Examples that illustrate the concept provide a meaningful context to help the adult learner understand the significance of the learning. With the development of self concept, learners move bey ond depending on the instructor (Knowles, 1980); therefore, instruction must facilitate self directedness in the learner (Knowles et al., 1998). Traditional learners need direction from their instructors, y for making decisions about their learning stimulates respect and acknowledges the need for self directedness in the learning design. Involving the participants in planning and evaluating their instruction develops self concept and self directedness. Ad ditionally, a learner centered course that is somewhat open ended helps adult learners decide the direction and strategy that they will use in their learning. An open ended environment may help learners choose what to learn or how to learn it. Although tr learner increasingly uses experiences as a resource (Knowles, 1980). Even errors made during le arning are experiences that provide the foundation for learning activities. Since adult learners bring experiences and knowledge to their various learning al., 1998). Offering learning opportunities that draw on experiences, such as the use of collaborative activities or open ended questioning, can allow learners to reflect, broaden their perspective of the content, and learn from each other.

PAGE 25

25 earn relates to the tasks and expectations in his or her everyday life (Knowles, 1980). Traditional learners are typically motivated to learn content regardless of its relationship to their personal goals, whereas topics having immediate relevance to work and/or personal situations are of most interest to adult learners. Adults have a practical outlook; therefore, instruction should focus on a real world task (Knowles et al., 1998). Opportunities for teachers to develop technology integrated lesson plans or work in an active learning environment to learn a new technology will help make the learning authentic. perspective of immediate application of knowledge for problem solving ( Knowles, 1980). Whereas postponed application of knowledge was once acceptable, the adult learner does not realize the significance of learning unless it has immediate application to his or her job and/or personal situation. This goal oriented nature of adults suggests that instruction should focus on solving a current problem or learner need (Knowles et al., 1998). This may be accomplished through a variety of strategies when considering how to integrate a new technology, including asking learners to de scribe an instructional problem that may be solved by using the tool and developing a plan for implementation. A task such as this will help to keep the focus on applying the knowledge to a relevant problem. n to shift from external to internal (Knowles et al., 1998). Adults tend to respond better to internal motivators than to external ones. Since adults are internally motivated, intrinsic motivational factors are important to the development of adult instr uction (Knowles et al., 1998). For example,

PAGE 26

26 learners, the sense of self fulfillment gained from being successful personally and professionally motivates an adult learner. An instructor offering the sort of professional esteem regarding their ability to integrate technology effectively by giving positive feedback to them about their plans for technology integration, respecting their input during discussions about technology integrated lessons, and offering time for the teachers to collaborate to develop innovative lessons In considering how to apply adult learning assumptions to instruction on technology integration, facilitators should provide opportunities for self directedness and solving an instructional problem, and the structure of the professional development should give tea chers the opportunity to consider ways to apply the content to a classroom situation. Additionally, instruction should address intrinsic motivational factors, such as how the content will help the teachers improve student learning. Accordingly, the profes sional development I provided addressed the adult learning assumptions as displayed in Table 2 1. During my instruction a focus on the importance of formative assessment and student engagement helped the teachers understand the value of using the technolo gies to meet these instructional needs. As noted by Fredricks et al. (2004), low engagement in the classroom tends to correspond with low achievement levels. A focus on formative assessment during instruction can improve learning (Black & William, 2010). Questionnaires administered throughout the professional development I conducted gave participants an opportunity to offer

PAGE 27

27 feedback for making decisions regarding future instruction. The teachers had occasions to incorporate their background kno wledge during the instruction, as I posed open ended questions regarding formative assessment to the group. Additionally, the instruction I facilitated on the use of the SRS and mobile IWB provided technology savvy users with opportunit ies to experiment a nd create while less experienced users work ed at learning stations and ask ed questions. During the sessions, the teachers worked collaboratively or independently on developing a multimedia lesson with embedded questions to collect formative data for enhan cing student engagement. Each of the professional development sessions I conducted provided the teachers with a chance to discuss how the use of the tools may benefit the students in their content area to help them immediately apply their new knowledge. Last, I addressed the need for internal motivation during the instruction as teachers shared with each other the benefits they experienced from using the technology in their classrooms. This study focused on evaluating the professional development and det ermining further ways to improve my practice. Professional Development Student learning can increase because of teacher improvement via research based professional development strategies. Emphasis on appropriately designed and implemented instruction for 2007) as teachers use instructional technologies to maximize their instruction and facilitate student success. As noted in learning theory, Guskey (2002 b ) suggests that successful professional dev elopment addresses the needs of teachers as learners, which enhances their effectiveness with students. It is important to consider the context, process, and content standards established by the National Staff Development

PAGE 28

28 Council (2001), which focus on pr oviding research based professional development that will improve the learning of all students through data driven decision making. In the two middle schools in which I work, as teachers collected data to help with instructional decisions, their efforts t o continue implementing strategies learned through the professional development were reinforced (Guskey, 2002 b ). This gives relevance to the instruction I facilitated on the use of SRS as a tool for formative data collection. The emphasis on improving st udent achievement is a common thread throughout the professional development literature. The literature on professional development pinpoints specific valuable components for adult learners. A review of empirical studies by Desimone (2009) describes a cor e set of features for effective professional development in a variety of contexts. The framework components include (a) content focus, (b) active learning, (c) coherence, (d) duration, and (e) collective participation (Desimone, 2009). Desimone (2009) su ggests a relationship between increased teacher knowledge, instructional changes, student improvement, and the core professional development features. Professional development with a content focus connects content related activities ing of the content ( Desimone et al., 2002; Desimone, 2009; Garet et al., 2001). A focus on content is essential to providing effective professional development for educators (Guskey, 2003 b ; Hill, 2007 2009; National Staff Development Council, 2001). Add itionally, professional development for educators should emphasize content related strategies that teachers may use to improve student outcomes (Hirsh and Killion, 2009; Sparks, 2002).

PAGE 29

29 Active learning provides teachers with an opportunity for engaging in t he learning process (Desimone, 2009). This instructional method results in better retention of knowledge as well as a greater level of involvement among learners (Bonwell & Eison, 1991). The strategy of active learning is related to effective professiona l development (Garet et al., 2001) and allows teachers to learn techniques for supporting student achievement ( Desimone et al., 2002; Hirsh & Killion, 2009), as is also suggested by the focus on content. When the content that teachers learn is consistent w ith their own goals as well as aligned with reform policies, the design of professional development is coherent ( Desimone et al., 2002; Desimone, 2009; Garet et al., 2001). Coherence, consistency between learning and beliefs, gives the professional develo pment meaning to teachers by providing relevance to what they are learning. Similarly, professional development aligned to instructional goals and curriculum can enhance teaching and learning (Hill, 2007). In the school system in which I work, the distri ct level leaders have determined that there is a need for engaging students in the learning process, including during the collection of formative data. Collaboration among (1) the district leadership and principals and (2) the principals and teachers in d etermining this need helped establish coherence (Desimone, 2009) in the design of the technology based professional development sessions I conducted. Research supports professional development that lasts several days or longer and many hours ( Desimone et a l., 2002; Garet et al., 2001; Hill, 2007), preferably more than 20 hours of contact time (Desimone, 2009). Additionally, it is important for professional development to include time for reflective practice over sustained blocks of time

PAGE 30

30 (National Staff Dev elopment Council, 2001). In depth study over an extended number of days can provide teachers with time to complete an authentic task or work in a self directed manner to solve an instructional problem. llective participation, is a successful strategy for teacher learning (Ball, 1996; Georgia Department of Education, 2008; Hill, 2009; National Staff Development Council, 2001; Wilson & Ball, 1996). Collective participation provides an opportunity for coll aboration among participants from within the same school and department ( Desimone et al., 2002; Desimone, 2009; Garet et al., 2001). For collaboration to be effective, a focus on improving student achievement must guide it (Guskey, 2003 b ; Desimone, 2009). As noted in Table 2 2, professional development literature is applicable to the development and implementation of technology based instruction for educators. Table 2 2 describes how I incorporated the core professional development elements into the profe ssional development I facilitated. The sessions I conducted focused on considering instructional strategies and resources for affecting student achievement. Through a focus on content, I attempted to help teachers understand how to use technology as a to ol to support the learning of content. In addition, I worked to provide teachers with the opportunity to gather collectively and collaborate on a goal related topic in an active learning environment tak ing place over an extended period. In profess ional development sessions in which teachers are learning about a technology and considering its uses during instruction, implementation of active learning may occur via occasions to work with the technology or collaborate on a technology integrated lesson plan. Implementing these professional development core features benefited the

PAGE 31

31 teachers as learners and influenced their instruction in the classroom. This study helped me to consider ways to implement these core features more effectively. Technology Bas ed Professional Development A review of the literature related to technology based professional development revealed similarities to the professional development literature. For each of the components described by Desimone (2009) as core features of profe ssional development there is supporting literature in educational technology. This connection among the two literature bases is important to consider when applying the professional development literature to technology training. It is important for facilit ators of professional development to provide teachers with the opportunity to consider how they may implement particular content in their classroom (Hirsh & Killion, 2009; Sparks, 2002). A focus on content during technology inquiry groups is effective (Hu ghes & Ooms, 2004). A content focus is also influential because of the impact it has on teacher knowledge and practice as well as on student learning (Desimone, 2009). An active learning environment in which teachers are involved in hands on instruction r egarding technology integration is essential. The inclusion of practice in professional development opportunities helps teachers implement technology in the classroom (Bradshaw, 2002). Providing time for practice is an important component that contribute s to technology integration (Mouza, 2003). As Hooper and Rieber (1995) suggest, to move beyond integration in their adaption of technology teachers must actively construct knowledge regarding instructional technology. Essential to coherent professional development is an in depth focus on a limited number of topics (Firestone et al., 2005). The traditional approach to professional

PAGE 32

32 development in which several topics are addressed in a short time is not effective for helping teachers maximize their learni ng and implement instructional changes (Firestone et al., 2005). As recent research suggests, coherence as described by Desimone (2009) is important for improving implementation of professional development (Penuel et al., 2008). For teachers to integrate technology consistently and effectively, they must have adequate access and support. Teachers may evaluate the coherence of the professional development based on the association between instructional expectations and available technological resources (Pe nuel et al., 2008). Considering technology use within the context of practice promotes teachers to use the technology in the way in which it was demonstrated (Matzen & Edmunds, 2007). In a study in one district implementing technology professional develo pment (Firestone et al., 2005), the teachers struggled to connect content knowledge with standards and the details of teaching strategies due to lack of coherence, as the professional development was fragmented with no systematic connection to state educat ional goals. Firestone et al. (2005) suggest that focusing on specific content areas helps build cohesion. For intellectual and pedagogical change to occur, professional development requires sufficient duration ( Desimone et al., 2002; Garet et al., 2001; Hill, 2007; National Staff Development Council, 2001). Besides practice, theory, and demonstration, Bradshaw (2002) found follow up to professional development vital to technology implementation by teachers. Additionally, follow up on the application of the new skill or strategy is a fundamental component to coherent professional development (Firestone et al., 2005). Follow up adds to the duration of the learning experience.

PAGE 33

33 Collective participation can assist with discussion, collaboration, and reflecti on. As indicated in the professional development literature, collaboration among participants is an essential component of technology based professional development (Hur & Brush, 2009). Mouza (2003) notes that in addition to addressing instructional rele vance and time for practice, it is essential to discuss and reflect on the use of the technology in instruction. Especially when teachers are learning new technologies, administrative support, student needs, teacher collaboration, and technological resour ces affect the One on one opportunities for collaboration, as well as goal setting and reflection, are critical for successful professional development (Orrill, 2001). The professional development literature contributes to the knowledge of technology based professional development. It is important to consider this base of literature in addition to that specifically related to technology instruction. Designing, developing, implementing, and evaluating technology based professional development that incorporates elements described in the literature is essential for maximizing the integration of technology during instruction. Instructional Design of Professional Development The generic ADDIE model (Analysis, Design, Development, Implementation, and Evaluation) of ISD is the foundation of many modern models (Reiser & Dempsey, 2002). Use of these essential components in the development of technology based instruction occurs in sy stematic linear formats such as the Dick and Carey Systems Approach Model (2004), as well as a systematic flexible application of the Morrison, Ross, and Kemp model (2001). In many of the models, the preferred approach is that

PAGE 34

34 of implementing evaluation throughout instructional design. The essential components of ISD models assist in the effective design of professional development. During the analysis phase of instructional design, goal setting for the professional development focuses on meeting the lea Schnackenberg et al., 2001). A needs assessment helps determine goals and objectives for the instruction (Mager, 1997; Schnackenberg et al., 2001). During the analysis phase it is important to consider any fact ors relevant to the professional development and the learners. To ensure a thorough analysis, the data are evaluated after this phase. One formative evaluation activity is to share the data from the needs analysis with someone in the target group for f ee dback (Brown & Green, 2006). Learning objectives are specified during the design phase. Also during this phase the scope and sequence of instruction and the methods of delivery are decided (Brown & Green, 2006; Morrison, Ross, & Kemp, 2007). The content of the professional development session is determined and plans are detailed. Revisions resulting from formative evaluations take p lace (Reiser & Dempsey, 2002). Development entails creating materials decided on during the design to use during the profess ional development (Morrison, Ross, & Kemp, 2007). Creating instructional activities results from the needs analysis and development of goals and objectives (Brown & Green, 2006) for developing content to correct the instructional problem (Morrison, Ross, & Kemp, 2007). An evaluation of developed materials may indicate a need for revision based on their alignment with instr uctional goals and objectives. The implementation phase involves providing the instruction to the learners. An effective presentation facilitates an instructional situation created to accommodate a

PAGE 35

35 variety of learners (Brown & Green, 2006) in the setting for which the instruction was designed (Reiser & Dempsey, 2002). Formative evaluations assess the overall worth of the instruction dur ing implementation (Reis er & Dempsey, 2002). The evaluation phase emphasizes measurement of the objectives (Alessi & Trollip, 2001) for each phase of the instructional design. Conducting formative and summative evaluations will enhance programs (Morrison, Ross, & Kemp, 2007). Dick and Carey (2004) indicate that formative evaluation should take place throughout the design of indicated by the ADDIE model. The summative evalu ation provides information about expected program outcomes and evidence of program achievement of the outcomes (Morrison, Ross, & Kemp, 2007). Evaluating professional development throughout the instructional design process is essential for continuous impr ovement. It is important to consider the fundamental components of ISD when planning professional development. The ISD components help develop appropriate instruction to meet the needs of learners. Applying data from the analysis phase to the design, dev elopment, and implementation phases of professional development helps meet instructional needs. In addition, the evaluation phase of ISD provides data for modifying instruction to mak e it appropriate for learners. A Framework for Technology Based Professi onal Development The principles described in this study establish a cohesive approach to professional development upon which I ground my practice. The literature revealed an inherent connection between adult learning assumptions professional development literature, technology based training literature, and ISD. It is because of the importance of incorporating systematically all of these elements into my own technology based

PAGE 36

36 professional development sessions that I propose the framework in Figure 2 2, whi ch demonstrates the relationship among adult learning assumptions, findings in the profes sional development literature, technology based professional development, and instructional desi gn of professional development. Technology Based Professional 2 begins with an examination of adult learning assumptions An analysis of the professional literature in the areas of professional development and technology based professional development revealed that adult learning theory is the foundation, as identified in the framework (Figure 2 2). The professional development literature is founded on adult learning methods and techniques as is apparent in the core elements described by Desimone (2009 ). In turn, the professional development literature provides the groundwork for the literature on technology based training. Additionally, as noted in Figure 2 2, each of these affects the instructional design. It is necessary to consider adult learning assumptions and the related professional literature during the analysis, design, development, and implementation phases of instructional design. As displayed in Figure 2 2, each of the ISD phases informs the others during the design process. Last, the fr amework (Figure 2 2) emphasizes that the other six components each influences the evaluation of the professional development. In conclusion, implementing technology based professional development with teachers requires several considerations. Adult learni ng assumptions, which include concept, experience, readiness to learn, orientation to learning, and motivation to learn (Knowles et al., 1998), not only affect learning

PAGE 37

37 outcomes as discussed in the literature, but also infl uence instructional design. The literature related to professional development, including that conducted in technology integration settings, incorporates particular effective elements with a focus on improving student achievement, including (a) content fo cus, (b) active learning, (c) coherence, (d) duration, and (e) collective participation as described by Desimone (2009). It is important to consider each of these components during instructional design, as without such considerations the professional deve lopment will not include the core aspects to meet the needs of practitioners. Therefore, I work to incorporate all of the elements in Figure 2 2 as a foundation for consistently providing the best professional development possible and continually evaluate and reflect on my professional practice to achieve that end.

PAGE 38

38 Table 2 1. Application of Adult Learning Assumptions during the Professional Development Adult Learning Assumptions Application of the Assumptions Need t o Know Focus on related supporting research. Self Concept Elicit feedback from learners, as well as provide feedback. Experience D ifferentiate instruction by process and product, consider learner backgrounds, provide online instructions for self paced le arning, and work with individuals as needed. Readiness to Learn Focus on a real world task such as developing a technology integrated lesson. Orientation to Learning Facilitate discussion among teachers regarding how the tools may solve an instructional problem. Motivation to Learn Facilitate discussion among teachers regarding beneficial classroom exp eriences with the technologies. Table 2 2. Alignment of Professional Development Core Features to the Technology Based Professional Development Professi onal Development Research Based Core Features Application of the Core Features during Technology Based Professional Development Content Focus Focus on using technologies for formative assessment. Active Learning Opportunities for hands on experimentation with technologies and strategies. Coherence Topics support instructional goals of the system and schools. Duration Sessions on similar topics offered weekly with time to complete an authentic task such as a multimedia lesson that incorporates strategies for formative assessment. Collective participation Teachers attend during th eir grade level planning times. Opportunities for collaboration with a focus on improving student achievement.

PAGE 39

39 Figure 2 1. Relationship between the Literature and Purpose

PAGE 40

40 Figure 2 2. P roposed Framework for Designing Technology Based Professional Development

PAGE 41

41 CHAPTER 3 METHODOLOGY This study used a combination of qualitative and quantitative methods for ve Levels of Professional Development Evaluation (1998 2000, 2002a ), a model designed to provide formative and summative information for improving and assessing the effectiveness of professional development (Guskey, 2006). Additionally, the leve ls are designed to build upon one another to measure improvement (Guskey, 1998 2006). This model (Guskey, 1998 2000, 2002a ) helped me address the problem of the study, which is to design, develop, and implement professional development to help teachers use the SRS and mobile IWB for formative assessment and evaluate this professional development for continuous improvement. (1998, 2000, 2002a) , helped evaluate the basi c requirements of the professional development such as its usefulness to the teachers. Analyzing data at this level provided information to judge development. The second l evel, helped gauge the level of knowledge the teachers had acquired during the sessions. The third level, Organization Support and Change, helped determine whether the professional development and organizational factors gave the teachers enough support to implement their new identify how the teachers were applying their learning from the sessions during classroom ins truction. The fifth and final level of the model Student Learning Outcomes, helped me assess the impact of the Professional Development on

PAGE 42

42 Levels of Profession al Development Evaluation (1998 2000, 2002a ) provided information for helping me improve the technology based professional development I conducted. Guiding Questions The guiding questions for the study helped pinpoint the potential learning resulting from the professional development and application of that learning in the classroom. Evaluation (1998 2000, 2002a ) as displayed in Table 3 1. The development of each question Evaluation (1998 2000, 2002a ). Accordingly, the following questions guided the data collection for studying the professional development: bout the professional development? In what ways did the teachers acquire the intended knowledge? In what ways does the organization help teachers implement the technologies? In what ways are the teachers using the mobile interactive whiteboard and student response system? What effect did the professional development have on student engagement? Context of the Study I serve two middle schools (School A and School B) in a rural school district in Georgia as their ITS. As noted in Table 3 2, the Georgia Depart ment of Education rated percent of economically disadvantaged students, neither of the schools held Title I

PAGE 43

43 status for the school year 2008 2009. Additionally, the numbe r of students with disabilities and English language learners at each of the schools does not currently meet the minimum requirement to qualify as a sub group for determinin g annual yearly progress. There are two computer labs at each school used by the Ca reer, Technical, and Agricultural Education D epartments. Each school also has two computer labs for the teachers to use for instructional purposes. The media center at School A has six networked computers for student use, and the media center at School B has 28 netbook computers that access the Internet via a wireless network hub in addition to 14 desktop computers. All of the computers at the two schools provide Internet access, a home drive for storage, a student and teacher shared drive, and a variety of software for instructional and productivity purposes. Each core academic teacher also has a classroom set of SRSs and a mobile IWB. Participants The population, or potential target audience for the study, consisted of teachers from the two mid dle schools in which I serve who attended the professional development related to using the SRS and mobile IWB for formative assessment (N=40). The sample consisted of the teachers who attended the professional development sessions and consented to participate in one or more aspects of the study participate in the professional development sessions. Each participant in th e study signed a consent form approved by t he Institutional Review Board A strategy that Patton (1987) refers to as convenience sampling designated participants in the evaluation, as the teachers are naturally present in the setting of the

PAGE 44

44 study. This sampling strategy is appropriate and typica l in evaluating professional development. Since the middle school teachers are a natural part of the school environment and the professional development that I conduct there, I approached the teachers before the first professional development session to a sk if they would like to participate in the sessions and be a part of the study. When meeting with the teachers I explained the main parts of the study including the purpose of studying the professional development I offer to improve future sessions I a lso in my writing and discussions of the data, which helped to increase participation in the study. Face to in the interviews and observations. The data I collected from the participants provided me with information for improving the future profession al development that I conduct. Of the thirty six (N=36) participants 72.2% were female (n=26) and 27.8% were male (n=10). Of the participants 22.2% 77.8% hold a graduate degree (n=28) including 17 masters, 10 specialists, and 1 doctorate. The mean years experience teaching in Georgia is 14.6 with the frequency distribution as follows: 6 teachers with 1 5 years, 6 teachers with 6 10 years, 7 teachers with 11 15 years, 12 teachers with 16 20 years, and 5 teachers with 21 or more years. The primary subject areas of the participants are as follows: 1 teaches technology education, 8 teach language arts, 9 teach math, 10 teach scienc e, and 8 teach social science. Description of the Professional Development As described in the problem and purpose sections of this paper, the professional development design included input from the building principals and a rev iew of system and school goals in the initial needs assessment. Development of the professional

PAGE 45

45 development sessions considered research based best practices, and the goals and objectives align with the Georgia Department of Education (2008, 2010 a ) evalua tion standards. The sessions emphasized using the SRS and a corresponding mobile IWB screen along with clicker strategies for formative assessment. Additionally, the professional development sessions included four main parts described below. Overview The initial needs assessment identified the need for technology integration sessions focusing on formative assessment leading to engagement of students. In contemplating how the SRS and mobile IWB technologies could support this content focus during learning I developed a list of possible session descriptions for the professional development. I shared the draft session list with the leadership teams at each of the schools during the initial needs assessment. This study examines four professional development sessions, which occurred hour each and occurred weekly at each of the schools. Additionally, the sessions incorporated the SRS and mobile IWB for demonstration and partic ipant experimentation to help the teachers learn to use the technologies for formative assessment. I placed materials online for teachers to access at their convenience for self paced learning and helped teachers individually as requested durin g and outsi de of the sessions. Lesson Development Tables 2 1 and 2 2 in the Literature Review detail how the professional development addressed adult learning assumptions and the core professional development elements. Because of my recent research (Fuller, 2010), I worked to

PAGE 46

46 design my instruction with a focus on adult learning assumptions (Knowles et al., 1998) and the following elements found in the professional development literature: (a) content focus, (b) active learning, (c) coherence, (d) duration, and (e) co llective participation (Desimone, 2009). Each of these components contributes to effec tive professional development. In keeping with adult learning assumptions (Knowles et al., 1998), I designed the professional development with a focus on formative asses data collection methods in the classroom and to help the teachers value the instruction. The teachers had opportunities to work collaboratively or independently with the SRS and mobile IWB, which helped address their levels o f experience with technology. Working collaboratively on a real world task, a multimedia lesson for collecting formative tion to learning as they collaborated on ideas for applying their knowledge and skills regarding instructional use of the technologies. Finally, opportunities to share their experiences with each other about past trials with the technologies or plans for future implementation motivated the teachers. Using adult learning assumptions in the design of the instruction helped me formative assessment. The lesson activities incorporate d the key aspects of professional development as well (Desimone, 2009). Each lesson focused on how the SRS and mobile IWB technologies could affect student achievement. A focus on formative assessment strategies helped teachers consider ways to use the t echnologies to examine student

PAGE 47

47 the sessions the teachers had opportunities to participate actively by experimenting with the SRS and mobile IWB at stations to develop t heir skill s and confidence using the technologies. Alignment of the instructional goals for the sessions with the needs of the teachers based on discussions with the principals helped establish coherence during the initial needs assessment. Teachers felt more comfortable with the technologies, as four lessons taking place over five weeks gave them time to learn and practice their new skills. As the sessions were during planning periods, the teachers also had an opportunity for collective participation, w hich helped them gain ideas for using the clickers and mobile IWB for formative assessment. Lesson Goals and Objectives The Georgia Department of Education (2008 2010 a 2011 ) considers Formative Assessment to be a tool for guiding and monitoring the progr ess of student learning during instruction and for adjusting instruction to maximize student achievement on the Geo rgia Performance Standards An overarching goal is to provide teachers with professional development that will help them integrate their new 21 st century technologies into their instruction. The following learning goals were adapted from the School Keys (Georgia Department of Education, 2008) and designed to incorporate the needs of the learners as based on the discussions with school princip als: The teachers will be able to design lessons guided by the instructional technology literature that integrate technology for collecting formative data. The teachers will be able to use the technology collected formative data for monitoring student lear ning during instruction. The teachers will explain ways to adjust instruction based on the technology collected formative data.

PAGE 48

48 I developed the objectives, learning activities, and an assessment based on the instructional goals as detailed in Table 3 3 I intentionally aligned the objectives to the learning goals. I designed the learning activities and assessment to address and measure mastery of the objectives respectively. The objectives for each of the lessons ( Table 3 3 ) focused on three main issues: Using the technologies with a clicker strategy for formative assessment. Experimenting with using the technologies to collect formative data. Creating a multimedia lesson for formative assessment. The following sections describe the technologies I address ed during the lessons, the SRS and mobile IWB, and the lesson development process. Additionally, I explain the activities I used during the lessons to help achieve the instructional goals and objectives. Lesson Activities Each lesson focused on using the SRS and a mobile IWB for formative assessment. When considering use of the technologies, I focused on using the verbal question feature and corresponding charting components of the clickers along with the mobile IWB data screen. The verbal question allow s a teacher to ask a question instantaneously or use one typed in any digital file and display the results from the clicker data in a charting format. Simultaneously, individualized student results from the SRS display on the mobile IWB screen, providing the teacher with data for immedia te modification of instruction. knowledge ( Table 3 3 ). The initial activity for this lesson included a review of the

PAGE 49

49 ons and modeling of use. For this lesson I also incorporated a modified K W L activity in which the teachers expressed what they kn e w and what they wanted to learn from each other about the clicker charting feature and the mobile IWB report screen as formative assessment tools. To gather this information I asked clicker questions such as (1) Are you comfortable with what you know about verbal questions and charts? and (2) Are you comfortable with what you know about the mobile interactive whiteboard s creen? Open ended question such as the following I recorded using the mobile IWB for reference a s the sessions were proceeding: What do you want to learn about verbal q uestions and charts? What do you want to learn about the mobile interactive whiteboard screen? How would you best le arn about these technologies? Additionally, in this lesson I included a cooperative learning activity in which the teachers discussed how t hey would use the technologies to monitor the progress of student learning and ways to adjust instruction to maximize student achievement. This initial lesson helped guide the development of my instruction during the following sessions on using the techno logies and considering formative assessment strategies. During the remainder of the session, the teachers began to work on a multimedia lesson for use with their students. Similar to the first lesson, the general instructional sequence of activities ( Tabl e 3 3 ) for Less ons 2 through 4 was as follows: An introduction to the technology and a strategy for using it formatively. A technology integrated formative assessment discussion facilitated by cooperative learning activities.

PAGE 50

50 An opportunity for teachers to experiment with the technology and strategy at their discretion. Time to work on a multimedia lesson that incorporates formative assessment strategies. The following sections explain in detail each lesson component described in Table 3 3 according to the instructional sequence Technology integrated formative assessment strategies. Each of the lessons used a similar approach for the introduction to the technology and formative assessment strategies. During the professional development I facilitated discu ssions to help the teachers consider instructional strategies for using their SRS and mobile IWB report screen to collect formative data about their students. I incorporated three literature based strategies into Lessons 2 through 4 for this purpose ( Tabl e 3 4 ). The strategies I used are Contingent/Agile Teaching clicker strategy (Bruff, 2009b; Beatty et al., 2006; Draper & Brown, 2004), Discussion Warm up/Think Vote Share clicker strategy (Bruff, 2009a, 2009b; Lyman, 1981), and Peer Instruction clicker s trategy (Bruff, 2009a, 2009b; Mazur, 1997). Each of these strategies incorporates SRS, providing an avenue to collect data and an approach for adjusting instruction. For ex ample, by using the Contingent/Agile Teaching clicker strategy (Bruff, 2009b; Beatty et al., 2006; Draper & Brown, 2004) as the teacher instructs he or she can collect real time clicker data to monitor student achievement and then use the data to modify in struction immediately. The Discussion Warm up clicker strategy (Bruff, 2009a, 2009b; Lyman, 1981) also helps a teacher monitor the student progress. The students thinking about a question posed by the teacher and voting on an answer using the SRS prom

PAGE 51

51 understandings as well as misinterpretations. The Peer Instruction strategy (Bruff, 2009a, 2009b; Mazur, 1997) discussed during the fourth session helps students learn content that is more d ifficult. For this strategy a teacher would involve the students in an instructional cycle of questioning and peer interaction until the g roup understood the material. During the sessions (Table 3 3) we also discussed that during use of clickers for forma tive assessment, the literature suggests between three and six clicker questions for a 50 minute lesson (Bruff, 2009a; Beatty & Gerace, 2009). Pacing the questions appropriately between segments of content allows a teacher to create an opportunity for for mative data collection. Additionally, pacing the questions rather than asking them all at once keeps the students attentive and engaged throughout the lesson. During the professional development the teachers considered this research in their discussions about formative assessment. I presented the clicker research and formative assessment strategies concisely ( Table 3 4 ) and facilitated a cooperative activity to engage the teachers in considering use of the strategies in the classroom. Cooperative activit ies Following introduction to the technology and one of the clicker strategies discussed above, a cooperative activity helped the teachers consider and discuss how they may use the strategy during instruction for data collection. During the sessions (Ta ble 3 3) the teachers received a handout (Appendix A ) containing the formative assessment and technology integration components of the Class Keys teacher evaluation rubric (Georgia Department of Education, 2010 a ). The teachers referred to this handout dur ing the cooperative activities when considering how they might incorporate the formative assessment strategies into their instruction to meet the

PAGE 52

52 expectations of the State. The handout also helped spark ideas for using the technologies formatively. Durin g the cooperative activity I incorporated questions such discussion among the teachers. Cooperative learning activities during the profes sional development also facilitated discussions regarding how to integrate technology for data collection, monitor student learning using the data, and adjust instruction as a result of the data to meet the needs ormative Assessment Class Keys strands, in what way(s) can the strategy help you to monitor the progress of student learning as screen data, what strategies might you imple were questions that cooperative groups discussed. To facilitate their thinking about the Exemplary level on the evaluation rubric, during the third session I had the teachers could you use the technologies to facilitate archived the information from these discussions using the mobile IWB to record the thoughts that teachers shared with the group. I referred to this type of information from the sessions each week to help decide which concepts to review in future sessions. Additionally, I recorded my observations in a Reflection Journal (Appendix B) for reference throughout the study. Durin g the final lesson (Table 3 3) I incorporated a cooperative activity to review assessment. Before the session I set up two mobile IWBs for the teachers to record

PAGE 53

53 their thoughts fo r displaying on the screen in addition to a computer with presentation software for recording notes. At each of these three technology stations, there was one question for each teacher group to answer. At the mobile IWB stations, the teachers wrote on th e mobile IWBs, and their responses were projected. At the computer station the teachers typed on a shared online presentation document. The teacher groups rotated to each station and answered the following questions during this coope rative activity: How might you use the clickers and/or mobile interactive whiteboard to monitor the progress of student learning? Based on formative data, if re teaching is needed what strategies would you use to adjust instruction immediately? How might you use the technologi es to involve students in decisions about re teaching? These questions were a culmination of the objectives for the sessions ( Table 3 3 ) and the items on the O bservation R ubric (Appendix H ). I reviewed the answers to these questions to provide information regardin and recorded my reflections in the journal Experimentation with the technologies The teachers had an opportunity to experiment with the technologies and the associated formative assessment clicker strategies (Table 3 4 ) during each of the second through fourth sessions (Table 3 3). Two stations were set up in the computer lab with a SRS and mobile IWB. During this part of the sessions, teachers gathered around the stations to work with the technologies. I circula ted and helped the teachers as needed answering any questions they had about the technologies or their use. Between five and 15 teachers participated at the stations each week while the other teachers at the session either experimented

PAGE 54

54 with the technologi es independently or began to work on a multimedia lesson designed to integrate the fo rmative assessment strategies. Data Collection 2000, 2002a (1998 2000, 2002a ), there is alignment among the guiding questions and data collection methods (Table 3 5 ). The guiding question for each level helped me study the effectiveness of four professional development s essions I conducted regarding collection instruments (Table 3 5 ) include a Perception Questionnaire for Level 1, a Artifact Rubric fo r Level 2, an Organization Support Questionnaire for Level 3, and an Observation Cycle for Levels 4 and 5 including Pre an d Post Observation Interviews. At the end of the second professional development session, I asked teachers in attendanc e to complete the Level 1 Perceptions Questionnaire (Appendix C ). Although some respondents to this questionnaire had not attended the first session, I still needs and improve future sessions. I also asked teachers attending at least two of the sessions to complete this survey later using this same rationale. Teachers worked on a multimedia lesson throughout the four sessions. If teachers attended all four sessions an d displayed their multimedia lesson, then I reviewed it using the Learning Artifact Rubric (Appendix D) I asked the teachers who attended all of the sessions to complete the Level 3 Organization Support Que stionnaire (Appendix E ) following the fourth session. The prerequisite for participation in the Observation Cycle

PAGE 55

55 was attendance at all four of the professional development sessions and completion of the Level 1 and Level 3 questionnaires, which ensured t hat my cases provided the information I required. Guiding Question for Level 1: As noted in Table 3 5 I administered the Perceptions Questionnaire ( Appendix C) Evaluation (1998 2000, 2002a What Questionnaires focusing on teacher perceptions and reflections are an appropriate method for gathering data (Guskey, 1998). I administered ques tionnaires to collect data for Level 1 and Level 3 of the model During the design of both questionnaires, I focused on collecting data aligned with adult learning assumptions and the professional development literature. I modified the questionna ires as appropriate to ensure their alignment with the goals of the evaluation level and the lesson objectives. Additionally, to assist with face and content validity in the questionnaires, a panel of education experts, including a technology director, IT S, and three professional development directors, reviewed the questionnaires. For face validity each expert agreed that the questionnaires appeared to measure what I had intended. For content validity the experts determined each item on the questionnaire s to be esse ntial to the data collection. I used several strategies found in the literature to help with data collection. The format for the majority of the questionnaire items is a matrix that uses space efficiently that they can complete the questionnaire quickly (Babbie, 2007). Additionally, I considered the importance of question order and placed the more interesting questions at the beginning of the questionnaires (Babbie, 2007).

PAGE 56

56 To increase the response rate, I also used a bimodal method that include d web based Method (2007) suggests making multiple contacts for increasing participation in questionnaires. I asked the teacher s to complete the Perceptions Questionnaire at the end of the second session; however, based on initial response rates I sent out the link to the web based survey to gather sufficient data for analysis. The questionnaire for Level 1 elicited information from the teachers regarding the usefulness of the professional development. Items on this four point Likert Scale questionnaire range from Strongly Disagree (1) to Strongly Agree (4). The Perceptions Questionnaire (Appendix C) includes items such as (1) The professional learning my prior knowledge. I administered this questionnaire to all participants during the second week of the evaluation process, as not ed in Table 3 6 to gather formative data and make needed adjustments as the sessions were proceeding. Additionally, I asked teachers attending at least two of the professional development sessions to complete the survey following the session sequence. I dentification numbers on the questionnaires allowed me to consider perceptions and ideas of individual teachers while allowing for anonymity. Guiding Question for Level 2: I used two instruments to collect data f 2000, 2002a ) model One instrument is a Artifact Rubric (Appendix D) for rating a teacher deve loped multimedia lesson. The other instrument is a Reflective Journal (Appendix B) I completed following each lesson I conducted. The Reflective Journal

PAGE 57

57 for triangulation (Denzin, 2006) during the professional development to hel p me meet the ; however, it did not provide data for the study Artifact The Artifact for Level 2 helped ga because of the professional development. The artifact is a teacher developed multimedia lesson that demonstrated evidence of its intended use for formative assessment. The participants worked on their multimedia lesson during e ach of the sessions Since Guskey (1998) notes that the preceding levels must be successful for the following levels to be successful, I offered teachers suggestions and answered questions during the professional development sessions regarding use of formative assessment strate Artifact Rubric To evaluate the teacher developed multimedia 2000, 2002a ) Level 2, earning I developed a rubric ( Appendix D ). I aligned the rubric to the literature on using clicker questions formatively during instruction (Bruff, 2009a), as well as to the standards for implementing formative assessment strategies, which are outlined i n the Class Keys (Georgia Department of Education, 2010 a ). I used facilitate (1) using the technologies for formative assessment and (2) using formative data to change instruction in response to learning needs. I reviewed the artifacts the The completion of the Artifact Rubric provide

PAGE 58

58 learning as a result of the professional development and for me to use to improve my future instruction. Post Instruction Reflective Journal After each lesson I reflected on my teaching using the focus questions on the Post Instruct ion Reflective Journal (Appendix B ) designed to gather information to answer the guiding question for 2000, 2002a what ways Following each session I kept a Reflection Jo urnal activity helped me review the strategies that I used during the instruction and consider ng meaning from and reflecting on experiences (Killion, 2008). The reflections in the journal provided valuable information during triangulation (Denzin, 2006) of the results for Level 2 but were not considered data for the study During the lessons I ob understanding of how they could use the clickers to monitor student progress and adjust their instruction to maximiz e student understanding. I recorded a part of this observation process during the sessions as I used the mobile IWB to record the the cooperative activities in whic h the teachers answered questions to demonstrate their understanding of using the technologies for formative assessment. I recorded my observations and thoughts about the cooperative activit ies in the Reflective Journal. Each week after the session I revi ewed the previous written reflections in the Reflective Journal to formatively review the professional development sessions related

PAGE 59

59 to the items this tool addresses, which are (1) how the teachers responded to the various parts of the sessions (2) how I monitored their learning, and (3) considerations for modifying the sessions. My observations during the sessions provided me with important information to help consider the methods that I use during instruction and to standing. My reflections helped me learn more about my practice and ways I may improve it (Dewey, 1938). Guiding Question for Level 3: Organization Support and Change As noted in Table 3 5 I administered the Organization Supp ort Questionnaire (Appendix E) 2000, 2002a ) model In what ways does the organization help teachers implement the For face and content validity, the panel of ex perts who reviewed the Level 1 Perceptions Questionnaire (Appendix C) also reviewed this questionnaire. Similar to the Level 1 Perceptions Questionnaire, I designed this questionnaire in a matrix format considering the research on increasing response rate (Babbie, 2007), and I administered it using a bimodal method of distribution (Dillman, 2007). During Week 5 of the study, I administered the questionnaire for Level 3 to gather data regarding the support structure provided by the professional development. Items on this four point Likert Scale range from Strongly Disagree (1) to Strongly Agree (4). This instrument includes items such as (1) I identified strategies for using the technologies for formative assessment and (2) I am able to use th e technologies for formative assessment. I administered this questionnaire following the intervention as designated on the timeline (Table 3 6 ).

PAGE 60

60 Guiding Questions for Levels 4 and 5: Skills and Stude nt Learning Outcomes Evaluation (1998 2000, 2002a ) I systematically employed a three part Observation Cycle (Danielson, 2007) consisting of a Pre Observation Interview, a classroom observation, and a Post Observation I nterview. I implemented the Observation Cycle to address these guiding qu model (1998, 2000, 2002a) : Level 4 In what ways are the teachers using the mobile interactive whiteboard and student response system? Level 5 What e ffect did the professional development have on student engagement? (1998 2000, 2002a ) model The instrument for the Pre Observation Interview (Appendix F) the fir st component of the Observation Cycle, consisted of interview questions that I asked the participants for collecting data for Level 4. For the second component of the Observation Cycle, the Level s 4 and 5 classroom observation, I used two instruments the Observation Field Notes Record (Appendix G ) and the Observation Rubric (Appendix H ) to record data from the observation of the teacher implementing the SRS and IWB for formative assessment during instruction and observations regarding student engagement For the last part of the Observation Cycle, which I implemented for the Level 5 guiding question, I asked the participants a set of questions for the Post Observation Interview (Appendix I) The Pre Observation Interview for Level 4 helped me clarify the t plans for the observed lesson and determine if they needed assistance before I observed the lesson. The Observation Rubric for Level 4 and the Observation Field

PAGE 61

61 Notes Record for Level s 4 and 5 use of the technologies for formative assessment and gave me an opportunity to reflect deeply on how my instruction contributed to their use as well as student engagement The Post Observation Interview for Level 5 gave me further insight into how the tea chers used the technologies for formative assessment; teacher input into how the technologies engaged their students, if at all; and their thoughts regarding the professional development and its effect. The following sections describe how each of the comp onents of the Observation Cycle addressed the guiding questions for Levels 4 and 5 of 2000, 2002a ) model Pre Observation Interview I designed the Pre (1998 2000, 2002a ) L In what ways are the teachers using the mobile interactive whiteboard and The interviews helped me confirm or invalidate the data gained through the observations (Fraenkel & Wallen, 2006). Interviewing also helped elic it information that was not directly observable (Patton, 1987). For example, I designed the Pre Observation Interview (Appendix F) lesson. The interview questions als for using the technologies for formative assessment, which gave me insight into each To facilitate comparison of responses for data collection and to avoid leading questions, I c reated the Pre open interviewee reduced the potential bias that can result from analyzing data from different

PAGE 62

62 interviews (Patto experience and for their opinion (Patton, 1987) to help me study the effects of the professional development I provide d The probing questions helped gather particular details not explained by t he i actual words of the interviewee and avoid misinterpretat ions (Patton, 1987). Recording additional probing questions if needed. To help devise follow up questions and later find quotations within the recording (Patton, 1987), I took some notes on the pre observation form ( Appendix F ) during the interview. Since I have been working with the teachers for two years, I have developed a rapport with the teachers that helped keep a conversational tone throughout the interview; howeve r, I remained neutral to the Observation guiding question s for Level s 4 and 5 2000, 2002a ) model In what ways are the teachers using the mobile interactive whiteboard and student response I developed a protocol that includes a Field Notes Record (Appendix G ) and an Observation Rubric (Appendix H ) for the second component of the Observation Cycle, the classroom observation, to help collect data This observation protocol helped n their use of SRS

PAGE 63

63 and a mobile IWB. This protocol guided my observations for determining how using the technologies affec and engagement I conducted one observation for each of the participating middle school teach ers (N=12) as a part of the Level s 4 and 5 data collection. I observed the participants while he observation was the length of the lesson. On average the time spent in each classroom was an hour. Observation Field Notes Record. I created the Observation Field Notes Record (Appendix G ) to facilitate organization during the observation and collec tion of data. technology and observations regarding student engagement The format includes focus points at the top of the record as a constant reminder of what to observe in the classroom, which helped direct the data collection. There is a place on the record for a description of the environment and reflections (Patton, 1987). Additionally, I documented reflective thoughts noted by brackets around the text to dif ferentiate from observation notes. The format of this F ield N ote R ecord also provides space for In my role of observer as participant (Glesne, 2006), I recorded my observations on t h e Field Notes Record 1987). The field notes are a detailed description of what I observed in the classroom (Patton, 1987) and include an environmental description, direct quotes, specif ic

PAGE 64

64 behaviors, and reflections. My recorded observations focused on data regarding how my instruction has affected cl assroom use of the technology: In what ways is the teacher using the technologies? In what ways is the teacher collecting formative data? I n what ways has the teacher directed the students to use the technologies? Observation Rubric. I developed the indicators on the Observation Rubric (Appendix H ) considering the Class K eys (Georgia Department of Education, 2010 a ) teacher evaluation standards for formative assessment and technology integration. In the rubric I also addressed the specific features of the clickers and mobile IWB relating to their potential formative assess ment use. I carefully aligned each indicator on the rubric with the teacher evaluation standards (Georgia Department of Education, 2010 a ) to help obtain an accurate measurement of teacher proficiency. The format of this O bservation R ubric is modeled afte r the Rubric for Evaluating North Carolina Teachers (North Carolina State Board of Education, 2007), which is us ed for classroom observations. Following each observation I completed the Observation Rubric considering how the teacher used the technologies f or formative assessment during the lesson. Additionally, for accuracy I reviewed each rubric following the Post Observation Interview The observations generated data for studying the effectiveness of the professional development as it relate formative data using the technologies and the effect of the instruction on student

PAGE 65

65 engagement. On the F ield N otes Record I collected data for Level 4, and regarding student eng agement I described the behaviors of the class as a whole to indicate the active participation level of the students and their use of the technologies, which provided additional insight during the analysis of Level 5 data. The Observation Rubric data ormative assessment for Level 4 Post Observation Interview I administered the Post Observation Interviews (Appendix I ) to collect data for Level 5 8 2000, 2002a ) model : What effect did the professional I designed the Post Observation Interview questions similar to the Pre Observation Interview questions to confirm or invalidate data g ained through the observations (Fraenkel & Wallen, 2006). Additionally, and in a and to he lp reduce bias (Patton, 1987). Just as with the Pre Observation Interviews, I c onducted the interviews in the observation form as needed to avoid any misinterpretations. The method and its rationale during the Post Observation Interview process are the same as during the Pre Observation Interview. The Post Observation Interview s technologies during instruction for formative assessment and their thoughts about the technology on student engagement. Additionally, sinc e it was the last component of the study I asked the teachers questions related to improving the professional development. The questions related to the professional development provided valuable information

PAGE 66

66 regarding the strategies that teachers found mos t useful, as well as strategies they would like me to use to meet their needs. Data Analysis The data for Guiding Questions 1, 2, 3, and part of 4 are represented quantitatively, as the instrument for the data collection was a Likert Scale questionnaire fo r 1 and 3 and a rubric for 2 and 4. Additionally, I reported qualitative data from the Pre Observation Interviews for Guiding Question 4, Observation Field Notes Record for Guiding Questions 4 and 5, as well as the Post Observation Inte rviews for Guiding Question 5. Additionally, I used the information from the Reflection Journal for triangulation. Quantitative Data Analysis I collected quantitative data during Weeks 2 11 of the study using the questionnaires and Weeks 5 12 using the rubrics. I analyzed the quantitative data continually through Weeks 3 15 of the study (Table 3 6 ). This section describes the analysis of the ques tionnaire and rubric data. Questionnaires 2000, 2002a ) Level 1 and Level 3 question naires using descriptive statistics. For each Likert Scale item on the questionnaires, table data displays the mean and standard deviation. I calculated these statistical measures for the Likert Scale items using the formulas in Microsoft Excel. These d ata helped determine areas of strength or needs improvement. perceptions about the professional development strategies for the Level 1 Perceptions Questionnaire (Append ix C ) and perceptions about how the organization supported the

PAGE 67

67 Level 3 Organization Supp ort Questionnaire (Appendix E ). earning Artifact Rubric The Artifact 2000, 2002a ) Level 2 (Appendix D ) incorporates two components: (1) Plan for use of technology for formative assessment and (2) Use of questions. U individually. I selected observed indicators on the rubric and determined the rating for each component based on the category with the highest frequency of indicators recorded. For each of the two components the ratings were as follows: 1 = Emerging, 2 = Proficient, and 3 = Exemplary. The sum of the two scores designated the total score for the rubric, which was applied to the following scale: 6 points = Exemplary, 4 5 points = Pr oficie nt, and 2 3 points = Emerging. An Emerging rating essentially means that a teacher only minimally considers using the technologies for formative assessment. The lesson incorporates very few clicker questions for collecting data. Additionally, the q uestions in the lesson lack cle ar alignment to the standards. For a teacher to receive a Proficient rating, the lesson plan includes charting for displaying the clicker data, the mobile IWB screen data, or another form of data to monitor the progress of in dividual students. The formative assessment plan also strategies for this rating may include more than the recommended three to six clicker questions for collecting formati ve data (Bruff, 2009a; Beatty & Gerace, 2009); however,

PAGE 68

68 the questions have appropriate alignment to the standards and grouping in the lesson is unobtrusive. In addition to the elements found in an artifact scoring Proficient, an Exemplary multimedia lesson includes use of a variety of clicker strategies to gain formative data as well as adjust instruction. A lesson of this caliber includes ideas for using clicker ts in instructional decisions. Due to the calculation of the two rubric components, a teacher scoring Exemplary in only one section and Proficient in the other would still receive an overall rating of Proficient. I compiled the data for the Artifact Rubric by placing the overall rating data set into a frequency distribution before calculating descriptive statistics. For each item on the rubric, I reported the mean and standard deviation. I calculated each of these sta tistical measures using th e formulas in Microsoft Excel. Observation Rubric 2000, 2002a ) Level 4 (Appendix H ) combines elements from the Class Keys (Georgia Department of Education, 2010 a ) technology integration and formative assessment teacher standards. SRS and mobile IWB for formative assessment, (2) using formative data to monitor instruction in response to individua l learning needs, and (3) involving students in decisions about adjustments to instruction to enhance their learning. Based on the dent, Emerging, Proficient, or Exemplary. I selected observed indicators on the rubric and determined the overall rating based on

PAGE 69

69 the category with the highest frequency of indicators recorded. I reported the data from the Observation Rubric during Level 2000, 2002a ) Evaluation in the form of a frequency distribution for comparative analysis of the various levels of proficiency regarding use of the SRS and IWB for formative assessment. A teacher scoring Emerging on the Observation R ubric used the technologies primarily for drill and practice such as a series of questions with no discussion. This score also results from lack of descriptive feedback or other methods for adjusting instruction. For this rating a teacher ma y use formative assessment strategies to help adjust whole class instruction, but may not be consistent and rarely uses it at the individual level. The teacher inconsistently addresses student misconceptions during the lesson. For example, if the majorit y of the class does not understand the concept based on the data, the teacher makes plans to alter the teaching method and re teach the concept later. For a teacher to receive a Proficient rating, the lesson must align with the Georgia Performance Standard s (Georgia Department of Education, 2011) use technology to enhance student learning, and substantiate formative assessment strategies used at the individual level, which teachers may accomplish via use of the SRS and mobile IWB. The teacher may accompli sh this through two methods for this observation based on the rubric. One method is for the teacher to use the charting results from the clickers to monitor and adjust instruction at the individual level. This is observable when the teacher re teaches a concept in a way that meets the instructional needs of students that have not understood the concept based on the data.

PAGE 70

70 To earn a score of Exemplary on the rubric, the lesson must go beyond using the technologies for formative assessment at the individual level. The lesson demonstrates evidence of a comprehensive approach for technology use that enhances the achievement of all students. A variety of activities and technologies may monitor student progress and adjust instruction to maximize achievement for all learners. A teacher may achieve the Exemplary rating in a variety of ways, including implementing the clicker strategies ( Table 3 4 ) discussed during the professional development. For example, a teacher may adjust instruction using techniques such a s the Peer Instruction clicker strategy (Bruff, 2009a, 2009b; Mazur, 1997), which involves pairing students based on the mobile IWB screen data and then having them answer the question again after their discussion to check for increased understanding. Add itionally, in an exemplary lesson students are involved in instructional decisions preferably via use of the technologies for data collection regarding needed ins tructional modifications. Qualitative Data Analysis D uring Weeks 6 12 of the study I collected qualitative data using the Observation Cycle. I analyzed the qualitative data continually through Weeks 1 15 of the study (Table 3 6 ). This section describes the analysis of the qualitative Observation Cycle data for Levels 4 and 5. Interviews and Obser vations A qualitative analysis (Glesne, 2006) of the interviews and the observations provided data for evaluating the professional development (Glesne, 2006; Patten, 1987). I transcribed the audio from interviews accurately for the Pre and Post Observa tion Interviews onto the Interview Coding Protocol Form (Appendix J ). I recorded notes and observations from the classroom observations on the Observation Field Notes Record (Appendix G ). I used a process

PAGE 71

71 nd dividing the data by commonalities for further analysis and description (Glesne, 2006). I continually review ed the data to locate codes and sub codes and categoriz ed the codes (Glesne, 2006; Patten, 1987). pared and contrasted the data as I collected it for categorizing into codes in order to understand how the teachers applied their learning regarding using the SRS and mobile IWB reports for formative assessment (Glaser, 1965). As codes became saturated th e categories developed (Glaser, 1965). As I reviewed the codes, I refined them until they were no longer repetitive, and grouped the codes based on patterns in the data to create an organizational framework (Glesne, 2006). Categorizing the codes lead to the development of themes for analyzing the data into the common areas (Fraenkel & Wallen, 2006; Glesne, 2006; Patten, 1987). I also reviewed the open ended questionnaire items for common themes during triangulation (Denzin, 2006). software to designate codes on the interview and observation records and grouping them in taxonomy, I created an organized visual representation of the data for understanding the relationships among the codes and locating themes in the data (Glesne, 2006; Spradley, 1979). The themes represented the most pervasive thoughts among the coded items. I t is important to find meaningful connections when interpreting the data, which involves transforming the data through description, analysis, and interpretation (Wolcott, 1994). I organized the data, reflections, coding, and analysis in a digital fieldwork fo lder to help analyze the data. Information for Triangulation I collected information for triangulation during Weeks 1 5 using the Reflection Journal These reflections were not coded data, but were a source of valuable

PAGE 72

72 information for Level 2 This section describes the analysis of the information from the Reflection Journal for triangulation Reflection Journal The information from the Reflect ion Journal (Appendix B) helped with Methodological Triangulation (Denzin, 2006), allowing me to compare findings across multiple data types. This cross examination helped me to be more n the Reflection Journal, I reflected on the instruction for this study to determine if it met the basic responses during the sessions and used this information form atively to help me determine any adjustments required for future professional development. Answering quest ions in the journal about the introductory material, cooperative activities, and discussions provided information about the structure of the lessons. Other questions in the journal helped me consider the effectiveness of the technology stations and other for each of the lessons. This Reflection Journal was a so urce of reference for me throughout the study, providing an opportunity to contemplate ways to improve the professional development I provide. Timeline The timeline for the study (Table 3 6 ) focuses on implementing each part of the evaluation in a timely m anner. I taught each of the four lessons during the first five weeks of the study. I initially distributed the Perceptions Questionnaire during Week 2, but I continually worked to gather further data from this survey through Week 10. For efficiency I be gan tabulating quantitative survey data after administering this first questionnaire. The teachers worked on developing their multimedia lessons during the

PAGE 73

73 first five weeks of the study and I completed the Reflection Journal as well. I administered the Organization Support Questionnaire during Week 5; however, as with the first questionnaire I continued to collect data throughout the remainder of the study, as some participants completed the paper version and others completed the online vers completing interviews and observations. The components of the Observation Cycle took place during Weeks 6 through 12. The timeline was fluid to facilitate maximum data collection. Tr ustworthiness and Limitations of the Study To increase my objectivity, I assessed my interpretations of the data (Glesne, trustworthiness of the study and limit researcher b ias (Glesne, 2006). To increase validity I collected data at both of the middle schools. In addition to multiple sites for data collection, I addressed possible validity issues and increased the trustworthiness of the evaluation by being objective (Glens e, 2006). For face and content validity a panel of expert s agreed that the questionnaires appeared t o measure what I had intended and determined each item to be esse ntial to the data collection. I developed a set of questions for each interviewee using a and to reduce the potential bias that can result from analyzing data from different interviews (Patton, 1987). I recorded the interviews to avoid misinterpretations (Patton, 1987 ). I used the i nterview questions to confirm or invalidate data gained through the observations (Fraenkel & Wallen, 2006). Similarly, t he observation data helped to confirm or invalidate data gained through the questionnaires and rubrics. I validated the

PAGE 74

74 observations by observing the teachers before drawing conclusions about the effectiveness of the professional development. Additionally, I achieved validity of the study through triangulation (Denzin, 2006); I carefully inspected and interpreted multiple sources of da ta. Surveys, assessment of teacher created lessons, interviews, and observations, all provided data that contributed to determining the effectiveness of the professional development and decisions regarding future modifications. L imitation s of the study in clude the focus on using the technologies for formative assessment. This limited focus may have caused me to overlook other valuable uses of the technologies or other instructional strategies used during classroom instruction. Self reported data obtained from questionnaires and interviews pose a potential bias as participants may not remember experiences or recall events correctly. Self reported data may also attribute negative events to other people or situations for which he or she is not responsible. Another limitation of the study is the sample size because it may not accurately reflect the thoughts or abilities of all teachers in the school. Additionally, in technologi es such as previous difficulty with formative assessment strategies limited use of the technology or expertise Because the participants volunteered their motivation or prior knowledge and instructional practice may vary from non participants in the sch ools, which may have resulted in selection bias (Wayne et al., 2008) Summary The participants for the study were teachers who attended the professional development I conducted. I designed four professional development sessions to incorporate adult learni ng assumptions and professional development literature for

PAGE 75

75 mobile IWB for formative assessment. 2000, 2002a ) model as the framework for developing guiding questions for the study. I developed a data collection strategy for each guiding question and its corresponding level of the model (Table 3 5 ). I used quantitative and qualitative analys is for coll ecting data for the evaluation.

PAGE 76

76 Table 3 1 Alignment of Evaluation Levels and Guiding Questions Professional Development Evaluation (1998 2000, 2002a ) Guiding Questions Level 1: Reactions professional development? Level 2: In what ways did the teachers acquire the intended knowledge? Level 3: Organizati on Support and Change In what ways does the organization help teachers implement the technologies? Level 4: New Knowledge and Skills In what ways are the teachers using the mobile interactive whiteboard and student response system? Level 5: Student Learning Outcomes What effect did the professional development have on student engagement? Table 3 2 School Facts for Middle Schools A & B School Facts for 2008 2009 School Year School A Schoo l B Number of Students in 2009 752 775 Economically Disadvantaged 19.00% 20.00% Students with Disabilities 10.00% 10.00% English Language Learners 3.00% 1.00% Did this School make Adequate Yearly Progress in 2009? Yes Yes Improvement Status in 2009 D IST DIST Note: Data retrieved from the G eorgia Department of Education ( 2010 b )

PAGE 77

77 Table 3 3. Description of the Professional Development Lesson Objectives Activities to Meet the Objectives Lesson 1: Formative Assessment using the Charting Feature and Mobile IWB Screen Considering the charting feature, teachers will explain how they could use the student response system to monitor the progress of student learning and idea(s) for adjusting instruction to maximize student achievement. Considering the method of using the mobile interactive whiteboard screen reports, teachers will explain how they could use the student response system to monitor the progress of student learning and idea(s) for adjusting instruction to maximize student achievement. Consid ering the charting feature, the teachers will begin creating a multimedia lesson that incorporates questions for collecting formative data via student response system and is aligned with the Georgia Performance Standards. Use a modified KWL chart to help f acilitate the lesson content related the technologies. Model how to modify software settings and the process for engaging a clicker question to display the data chart in verbal and standard modes. Model how to navigate through the various mobile interactive whiteboard screen reports. Use a cooperative learning strategy to consider, collaborate, and explain how to use the student response system and/or mobile interactive whiteboard report screen to monitor the progress o f student learning and discuss ways to adjust instruction based on formative data to maximize student achievement. Ask the teachers to share their ideas with the group. Circulate and pose the question to teachers while they are working on their multimedi a lesson. Explain that they may select which software to use for developing their lesson. Ask the teachers to select the standard(s) and work on the lesson content to develop a multimedia lesson for use in their classroom. Have the teachers work collab oratively or independently on the lesson. Ask them to share the lesson with others upon completion.

PAGE 78

78 Table 3 3. Continued Lesson Objectives Activities to Meet the Objectives Lesson 2: Formative Assessment using the Contingent/Agile Teaching Clicker S trategy (Bruff, 2009b; Beatty et al., 2006; Draper & Brown, 2004) Considering the Contingent/Agile Teaching clicker strategy, teachers will explain how they could use the student response system to monitor the progress of student learning and idea(s) for a djusting instruction to maximize student achievement. Given 21 st century technology tools/software and considering Contingent/Agile Teaching, teachers will experiment with and consider how they could use the 21 st century technologies to ask questions on th e fly to guide instruction. Considering Contingent/Agile Teaching, the teachers will continue to create a multimedia lesson that incorporates questions for collecting formative data via student response system and is aligned with the Georgia Performance St andards. Model content based application of the Contingent/Agile Teaching clicker strategy, including asking questions on the fly to guide instruction. Share that the literature suggests between 3 6 clicker questions for a 50 minute lesson. Have the teach ers use a cooperative learning strategy to consider, collaborate, and explain how they could use the student response system, specifically the Contingent/Agile Teaching clicker strategy, to monitor the progress of student learning and discuss ways to adjus t instruction based on formative data to maximize student achievement. Circulate and pose the question to teachers while they work on their multimedia lesson. Have the teachers modify software settings so that the charting feature will display after each e ngaged clicker question. Have the teachers experiment with engaging a clicker question in verbal mode, using the chalkboard function for writing the question, viewing the chart with class data, and identifying individual student achievement using the mobil e interactive whiteboard reports. There is flexibility for the time teachers elect to spend on experimentation before working on their multimedia lesson. Have the teachers continue to work on their multimedia lesson and consider including an idea(s) from t his session.

PAGE 79

79 Table 3 3. Continued Lesson Objectives Activities to Meet the Objectives Lesson 3: Formative Assessment using t he Discussion Warm up/Think Vote Share Clicker Strategy (Bruff, 2009a, 2009b; Lyman, 1981) Considering the Discussion Warm up/Think Vote Share clicker strategy, teachers will explain how they could use the student response system to monitor the progress of student learning and idea(s) for adjusting instruction to maximize student achievement. Given 21 st century technology tools/software and considering the clicker strategies, the teachers will continue to experiment with using the technologies for formative assessment. Considering Discussion Warm up/Think Vote Share, the teachers will continue to create a multimedia lesson that incorporates questions for collecting formative data via student response system and is aligned with the Georgia Performance Standar ds. Model content based application of the Discussion Warm up/Think Vote Share clicker strategy. Ask the teachers to use a cooperative learning strategy to consider, collaborate, and explain how they could use the student response system, specifically the Discussion Warm up/Think Vote Share strategy, to monitor the progress of student learning and discuss ways to adjust instruction based on formative data t o maximize student achievement. Ask the teachers to discuss ideas for using clicker data to facilitat e learning and involve them in instructional decisions to address exemplary use of formative assessment according to the Georgia Class Keys teacher evaluation instrument (2010 a ). Circulate and pose the question to teach ers while they are working on their multimedia lesson. Have the teachers experiment with using the verbal mode for asking a question, the mobile interactive whiteboard software for writing the question for students, or any of the other previous strategies. There is flexibility for the time teachers elect to spend on experimentation before working on their multimedia lessons. The teachers continue to work on their multimedia lesson and consider including an idea(s) from this session.

PAGE 80

80 Table 3 3. Continued Lesson Objectives Activities to Meet the Objectives Lesson 4: Formative Assessment using the Peer Instruction Clicker Strategy (Bruff, 2009a, 2009b; Mazur, 1997) Considering the Peer Instruction clicker strategy, teachers will explain how they could use the student response system to monitor the progress of student learning and idea(s) for adjusting instruction to maximize student achievement. Given 21 st century technology tools/software and considering clicker strategies, the teachers will continue to e xperiment with using the technologies for formative assessment. Considering Peer Instruction, the teachers will continue to create a multimedia lesson that incorporates questions for collecting formative data via student response system and is aligned with the Georgia Performance Standards. Model content based application of the Peer Instruction clicker strategy. Discussion includes ideas for using the Peer Instruction clicker strategy for formative assessment. Review with IWB in the group mode. Using a co operative learning strategy each group uses the IWB and shared document to share how they could use the student response system and/or mobile interactive whiteboard to monitor the progress of student learning and ways to adjust instruction, including stude nt involved decisions to maximize student achievement. Discussion of ideas allows groups to share. Circulate and pose the question to teachers while they are working on their multimedia lesson. Teachers may experiment with any of the technologies with whi ch we have worked. There is flexibility for the time teachers elect to spend on experimentation before working on their multimedia lessons. The teachers continue to work on their multimedia lesson and consider including an idea(s) from this session.

PAGE 81

81 Tabl e 3 4 Clicker Strategy Components Clicker Strategy Main Components for using the SRS and IWB Formative Assessment Contingent/Agile Teaching Clicker Strategy (Bruff, 2009b; Beatty et al., 2006; Draper & Brown, 2004) Teacher asks clicker questions spread throughout conte nt and collects real time data. Students use charting feedback to monitor learnin g. Teacher uses mobile IWB screen reports to identify needs of individual students. learning needs (pa cing, probing, questions on new topic, etc.). Discussion Warm up/Think Vote Share Clicker Strategy (Bruff, 2009a, 2009b; Lyman, 1981) Teacher asks a clicker question, students think and record answers v ia the student response system. Teacher displays char ting results for class wide discussion. Teacher uses mobile IWB screen reports to identify needs of individual students. Students think about (Think) and commit to an answer (Vote), setting the stage for greater di scussion participation (Share). Option: S tudents respond twice to difficult questions, once right after they read the question and then again after the discussion. Peer Instruction Clicker Strategy (Bruff, 2009a, 2009b; Mazur, 1997) Teacher asks a clicker question following a content segment, st udents vote and review charting data. Teacher uses mobile IWB screen reports to identify needs of individual students. If needed students discuss with partner (Peer Instruction) and revote, which often leads to convergence to the correct answer. (Teacher uses IWB reports to help assign partners.) Students may listen to mini lecture or engage in class wide discussion and respond/vote again if class average is still low. Students may find that, for particularly challenging questions, this can be an effective technique for discovering and exploring course material.

PAGE 82

82 Table 3 5 Alignment of the Five Evaluation Levels, Guiding Questions, and Data Collection Methods Levels of Evaluation (1998 2000, 2002a ) Guiding Questions Data Collection Method Data Collection Details ( W hat is measured and how the information is used ) Level 1: Reactions What are the perceptions about the professional development? Questionnaire Perceptions Questionnaire ( Appendix C ) administered during implementation of professional development to gather formative data regarding needs and program improvement. Level 2: Learning In what ways did the teachers acquire the intended knowledge? Rubric The artifact is a teacher developed multimedia lesson to gauge the improving future professional development. The Ar tifact R ubric (Appendix D ) assesses its effectiveness. Level 3: Organi zation Support and Change In what ways does the organization help teachers implement the technologies? Questionnaire Organization Support Questionnaire (Appendix E ) gathered data regarding the support structure provided by the pr ofessional development to improve organizational support and f uture professional development.

PAGE 83

83 Table 3 5 Continued Levels of Evaluation (1998 2000, 2002a ) Guiding Questions Data Collection Method Data Collection Details ( W hat is measured and how the information is used ) Level 4: Use of New Knowledge and Skills In what ways are the teachers using the mobile interactive whiteboard and student response system? Observation Cycle: Pre Observation Interview and Cl assroom Observation Pre Observation Interview (Appendix F ) and the Direct Observation provide d data for evaluating the effectiveness of the professional development as it relate d to the knowledge gained for techno logy integration and effects in the classroom. Data from the observation s were recorded on the O bservation F ield N otes R ecord (Appendix G) and the O bservation R ubric (Appendix H) Level 5: Student Learning Outcomes What effect did the professional devel opment have on student engagement? Observation Cycle: Classroom Observation and Post Observation Interview The O bservation F ield N otes R ecord (Appendix G) and Post Observation Interview (Appendix I ) help ed gather data about the effect the professional development had on engagement.

PAGE 84

84 Table 3 6 Implementation Timeline Study Component Timeline (Week of Evaluation Process) Design and Development of Instruction Pre Evaluation Lesson 1 : Formative Assessment using the Charting Feature and Mobile I nteractive W hiteboard Screen 1 Lesson 2 : Formative Assessment using the Contingent/Agile Teaching Clicker Strategy (Bruff, 2009b; Beatty et al., 2006; Draper & Brown, 2004) 2 Level 1 : Perceptions Questionnaire 2 10 Lesson 3 : Formative Assessment using the Discussion Warm up/Think Vote Share Clicker Strategy (Bruff, 2009a, 2009b; Lyman, 1981) 4 Lesson 4 : Formative Assessment using the Peer Instruction Clicker Strategy (Bruff, 20 09a, 2009b; Mazur, 1997) 5 Level 2 : Teacher Developed Multimedia Lesson and Post Instruction Reflective Journal 1 5 Level 2 : Teacher Developed Multimedia Lesson Artif act Rubric 5 12 Level 3 : Organization Support Questionnaire 5 11 Level 4 : Pre Observation Interview and Observation Rubric 6 12 Levels 4 & 5: Observation Field Notes Record 6 12 Level 5 : Post Observation Interv iew 6 12 Analysis of Data Collected 1 15

PAGE 85

85 CHAPTER 4 RESULTS Five Levels of Professional Development Evaluation (1998 2000, 2002a ) during an evaluation of technology based professional development conducted at two middle schools. This chapter describe s the participation in the study and the findings for each L 2000, 2002a ) regarding the effectiveness of the professional developm ent. This chapter presents the data associated with each guiding question for the study 2000, 2002a ) The guiding question is stated explicitly under each subheading in the chapter follow ed by a concise description of the data. The data for Guiding Questions 1, 2, 3, and part of 4 are represented quantitatively, as the instrumentation for the data collection was a Likert Scale questionnaire for 1 and 3 and a rubric for 2 and 4. Additiona lly, I report qualitative data from the Pre Observation Interview for Level 4, O bservation Field Notes Record for Level s 4 and 5 as well as the Post Observation Interview for Level 5 Participation in the S tudy I asked the 35 teachers who attended the second professional development the end of the session as only attendance at some of the professional development was requir ed for participation Two teachers left a couple of minutes early; one completed an online version that I emailed following the session, but the other did not. One teacher who stayed for the whole session did not complete the questionnaire during the ses sion and did not complete the online version. Teachers attending at least

PAGE 86

86 two of the sessions but who were not present at the second session and who signed a consent form (n=5) were emailed an online version of the questionnaire during the last week of th e professional development. Overall, 36 teachers completed the Perceptions Questionnaire (N=36). There were more participants for the Level 1 Perceptions Questionnaire (N=36) than for the Organization Support Questionnaire (N=16) for Level 3 as the Organization Support Questionnaire was reserved for participants attending all four of the professional development sessions (Table 4 1). I administered this questionnaire via email and 11 participants completed it during the first two weeks. I emailed both of the questionnaires weekly until the end of data collection to the participants who had not completed them. Additionally, I delivered the paper version of the Organization Support Questionnaire to four teachers participating in the Observation Cycle. Two teachers who attended all of the professional development did not complete the Organization Support Questionnaire. Twelve teachers out of the 16 who attended all four of the professional developme nt sessions and completed the two questionnaires which was required for participation in the Observation Cycle volunteered (N=12). Three other teachers outside of the 12 participating in the Observation Cycle submitted an artifact for Level 2 (N=15). T able 4 1 displays the percentage of teachers who comp leted each part of the study and the requirements for participation in each component. Guiding Questions The guiding questions for the study each correspond to one of Professional Deve lopment Evaluation (1998 2000, 2002a ) This alignment of questions to the evaluation l evels helped to ensure that the data collected at each l evel informed

PAGE 87

87 the study appropriately. The following questions guided the data collection for studying the prof essional development I provided regarding use of the SRS and mobile IWB for formative assessment : Level 1: about the professional development? Level 2: P In what ways did the teachers acquire the intended knowledge? Level 3: Organization Support and Change In what ways does the organization help teachers implement the technologies? Level 4: Knowledge and Skills In what ways are the teachers using the mobile interactive whiteboard and student response system? Level 5: Student Learning Outcomes What effect did the professional development have on student engagement? Guiding Question for Le vel 1: 2000, 2002a ) Level 1 to collect data for this guiding question : perceptions about the professional This questionnaire consisted of 12 items using a Likert Scale format with 1 = Strongly Disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly Agree. Number 13, an open any, would you like the profession There was also a Comments section at the end of the questionnaire. Of the 36 participants completing this questionnaire, one did not answer Item 9 and another did not answer Item 10. Ten participants co mpleted Question 13 and seven c ompleted the Comments section. The results from this questionnaire indicate that the participants had a positive reaction to the professional development. As noted in Table 4 2, I used descriptive

PAGE 88

88 statistics to analyze the d ata from the Perceptions Questionnaire. The survey item with the highest mean and a rating of Strongly Agree was Item 11, The Professional Learning was Facilitated in a Professional Manner, with a mean of 3.81and standard deviation of .40. Item 10, The P rofessional Development Included Discussion and/or Collaboration, had the lowest mean on the questionnaire with a mean of 3.60. The standard deviation of .55 for this item indicates that a rating of 4 (n=22) is within one s tandard deviation of the mean. F or Question 13, five comments indicated no need for modifications to the professional development. These comments included statements regarding the effectiveness of the professional development such as the following: Thanks for spending encouraging time with me, meeting me at my instructional level and easing me into what seemed scary. Five other comments for Question 13 provided suggestions for modification or items to continue during future sessions. These comments were helpful in modifying the ses following two sessions. For the Comments section of the questionnaire, seven participants included comments. Six of these comments included positive remarks about the sessions Guiding Question for Level 2: For this level I used a Artifact Rubric and Post Instruction Reflective Journal to address this guiding question : acquire the intended indicate that the teachers got ideas for developing an artifact to use during instruc tion for formative assessment.

PAGE 89

89 Artifact Rubric The Artifact Rubric used to rate the teacher developed multimedia lesson (N=15) for 2000, 2002a ) Level 2 provided data to answer this question : The teacher developed multimedia artifacts were scored using this rubric aligned to the formative assessment and technology integration Georgia Class Keys (Georgia Department of Education, 2010 a ) for teacher evaluations. For scoring the two individual items on this rubric, 1 = Emerging, 2 = Proficient, and 3 = Exemplary. For determining the overall score on this rubric, which was the sum of the two items on the rubric, 2 3 = Emerging, 4 5 = Proficient, and 6 = Exemplary. The data from the Artifact Rubric on Table 4 3 indicate that the participants are proficient in their ability to design a technology integrated lesson for collecting formative data. Regarding the individualized participant data, two teachers scored a 3 in their use of technology for formative assessment and four teachers scored a 3 in their use of questions to demonstrate their exemplary ability in those areas. Only one of the teachers scored a 3 in both areas with a rating of Exemplary overall. The rest of the teachers were rated a 2 on the components of the rubric, which gave them a Proficient rating overall. None of the teachers scored Emerging in either area for this multimedia lesson rubric. With the exception of the one teacher scoring a 6 ove rall, each overall rating was Proficient with a score of either 4 or 5. The descriptive statistics in Table 4 3 display the data for the individual components of the rubric as well as the total rubric score. The mean of 2.13 and standard dev iation of .35 for Planned use of Technology for Formative Assessment

PAGE 90

90 indicate an overall rating of Proficient within the sample. Similarly, the mean of 2.27 and standard deviation of .46 in the Use of Questions indicate that the sample population was proficient with this strategy. The mean for the total score on the rubric is 4.40 with a standard deviation of .63, indicating that scores of 4 and 5 are within one standard deviation of the mean. Post Instruction Reflective Journal In addition to the Artifact Rubric, I used the questions in the Post Instruction Reflective Journal to guide my writing following each of the professional 2000, 2002a ) Level 2 : excerpts (Appendix K ) compiled from the Reflective Journal about the lessons represent my thinking during this reflection process and focus on considering how to m odify the sessions to improve the effectiveness of the professional development. I grouped my learning, and modification considerations for future sessions. Although not u sed as data for the study, t he Post Instruction Reflective Journal provided supporting information for triangulation (Denzin, 2006) of the results for Level 2 regarding the rmatively during instruction. Guiding Question for Level 3: Organization Support and Change I administered the Organization Support Questionnaire to collect data to answer the following question : Scale format with 1 = Strongly Disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly Agree. All participants (N=16) answered every Likert Scale item on the questionnaire. There wa s

PAGE 91

91 also a Comments section at the end of the questionnaire. Of the participants completing this questionnaire, only one completed the Comments section Overall, the data from the Organization Support Questionnaire yielded positive results. Similar to the Perceptions Questionnaire, as noted in Table 4 4, I used descriptive statistics to analyze the data for the Organization Support Questionnaire. The means for the items on the Organization Support Questionnaire w ere between 2.63 and 3.81, designating Agree and Strongly Agree for the majority of outcomes. on the Technology Professional Development Topics, had the lowest mean. The mean for this item was 2.63 and the standard deviation was .72. For this item, ratings of 2 and 3 were within one standard deviation of the mean. Two items with the highest means (Table 4 4) were Item 11, Julia Helped Me When I Needed Assistance with the Techn ologies, and Item 12, I Felt Supported during Implementation of the Technologies. With means of 3.81 and 3.75 and standard deviations of .40 and .45 respectively, the responses to these items had little fluctuation from a rating of Strongly Agree, as a sc ore of 4 is within one standard deviation of the mean. Skills and Student Learning Outcomes I used the Observation Cycle, consisting of a Pre Observation Intervie w, classroom observation of a technology integrated lesson, and a Post Observation In what ways are the teachers using the mobile interactive whiteboard and

PAGE 92

92 Observation Interview addressed the first question the classroom observation addressed both questions, and the Post Observation Interview addressed the latter question for Gusk (1998 2000, 2002a ) Levels 4 and 5 respectively. The following sections describe the results of the Observation Cycle for answering these two guiding use of the technologies for formative assessment during instruction. Pre Observation Interview The Pre Observation Interview data helped answer the question for Level 4: In what ways are the teachers using the mobile interactive whiteboard and student response sys The Pre Observation Interview data consisted of interview transcripts that I transcribed by typing the content of the audio recording for each of the Pre Observation Interviews. I coded the transcripts as described in Chapter 3 of this document. T his process included developing codes and sub codes and refining codes as needed to represent the interviewees accurately (Glesne, 2006; Patten, 1987) Figure L 1 displays an example of the coding process using HyperText software. This software helped me refine the codes and determine themes across the Pre O bservation Interview data. I considered the relationship among the prevalent codes to develop the Pre Observation Interview Taxonomy ( Table 4 6 ) that represents the themes and codes in the data. I eliminated codes unrelated to the focus of the study during development of the taxonomy. Due to the focus of the questions during the Pre Observation Interview, several main themes developed during the coding process under which I categorized other data on the taxonomy: Planning to Use the Technologies during Instruction,

PAGE 93

93 Planning to Use Formative Assessment Strategies, and Stating Tho ughts about the Technologies. The most prevalent codes that represented the theme Planni ng to Use the Technologies during Instruction present material during the lesson and their plans for using the clickers to collect o present material or instructional content included use of video, digital presentations, and writing on the mobile IWB. Additionally, praise regarding the anonymity of the use of clickers to collect data was coded numerous times. The codes appearing most frequently indicated in the Pre Observation Interview transcripts related to the theme Planning to Use Formative Assessment Strategies. Three main codes overarched various sub codes: Planning to U se T echnology to M onitor S tudent P rogress, Planning to A djust I nstruction, and Planning for S tudent I nvolvement. The technologies that the participants addressed during the interviews for monitoring student progress included general clicker data, the clicker charting data, and the mobile IWB screen. As repre instruction included using technology based data for decisions about adjusting instruction. Additionally, during coding it was evident that several of the interviewees planned to have students monitor their own progress during the lesson by comparing their clicker response to the correct response signified in the charting data. The last overarching theme in the Pre Observation Interview data, Stating Thoughts about the Technologies, relates to the part technologies. In coding the data I found that the participants were interested in using

PAGE 94

94 the clickers for formative assessment. Additionally, the coded data show that at the time of the Pre Observation Interview participants b elieved the clickers would provide lesson. Observation The classroom observation data helped answer the question s for Levels 4 and 5: he mobile interactive whiteboard and student The observation data, the second component of the Observation Cycle, consist of F ield N ote s Record data and results of an Observation Rubric. The next two sections describe the data and the recurring themes. Observation Field Notes Record As displayed in Figure L 1 an example of the coding process, I coded the field notes from the ob servations using the method described in Chapter 3, which included developing codes and sub codes and refining codes throughout the data analysis for Levels 4 and 5 (Glesne, 2006; Patton, 1987). As I coded the Observation Field Note s Records, I developed a Classroom Observation Taxonomy of themes among the codes. I eliminated codes not related to the observation data from consideration and reworded codes to represent the observations accurately. Thinking about the relationship among the coded behaviors h elped me develop the taxonomy. Table 4 7 displays the Classroom Observation Taxonomy of the recurring observed behaviors during the classroom observations. Since there were focus points during the observations, the coding of the field notes had a distinct focus. While the observation field notes were collected and coded, themes emerged which are reflected

PAGE 95

95 in the taxonomy. I categorized the themes under the following descriptive elements: Using the Technologies, Using Formative Assessment Strategies, and Directing Use of the Technologies. The observed patterns in student behaviors related to their level of engagement; therefore, I indicated the category Engagin g in Learning on the taxonomy. The data indicate that the participants used the mobile IWB, clickers, and digital during instruction to display content or clicker data. Example common codes include : W riting on the T ablet to E xplain a C oncept, U sing the T ablet S creen to R eview I ndividual S tudent R eports, D isplaying C licker Q uestions, and U sing C licker D ata to C hart R esults. Additionally, the teachers asked for my assistance, positioning me in the role of participant observer (Glesne, 2006), most often to help them navigate the IWB screen reports and use th e charting feature of the SRS. The prevalent themes of the observation regarding formative assessment strategies were : M onitoring I ndividual S tudent P rogress, A djusting I nstruction B ased on D ata, and H elping S tudents M onitor T heir O wn P rogress. To monitor student progress, the teachers were displaying the chart with clicker question data and discussing the results with the class, as well as reviewing the mobile IWB screen reports of individual student data. For adjusting instruction, the teachers gave descriptive feedback in the form of rephrasing concepts, explaining correct or incorrect answers, writing on the tablet during re teaching, and giving additional examples of the concept. T he teachers often asked additional questions beyond the planned clicker questions to facilitate discussion among the students. As teachers helped the students monitor their own

PAGE 96

96 progress they asked the students to take notes of important concepts, display ed and discussed clicker data in chart format, and provided descriptive feedback as they discussed re asons for a particular answer. technologies. When posing a clicker ques tion, the teachers referred to the clickers as the method for answering the question. Teachers used various procedures for distributing the clickers at the sta rt of the class as well. During the observations I also recorded general student behaviors regar ding their engagement during the lesson for triangulation (Denzin, 2006) of Level 5 data Several of the codes relate to use of the technologies. The codes that indicate student engagement include : Answering the Questions with Clicke rs, Responding to the Charting D ata, and Discussi ng Clicker Question with Other S tudents. Observation Rubric. I completed the Observation Rubric following the classroom observation eachers The category with the most indicators observed during the classroom observation determined the score on the rubric for a particular participant. Whereas the rubric for the teac her technologies for formative assessment, this Observation Rubric helped me assess the entation of their lesson plan. Table 4 5 displays the data for the parti Observation Rubric. One teacher scored Emerging on the Observation Rubric. Of the 12 teachers, 10 scored Proficient on the rubric. One teacher scored Exemplary on the

PAGE 97

97 Observation Rubric. These frequencies indicate that 8.33%, 83.33%, and 8.33% scored Emerging, Proficient and Exemplary, respectively. Post Observation Interview The Post Observation Interview the third component of the Observation Cycle, helped answer the question for Level 5: he professional development s section focuses Evaluation (1998 2000, 2002a ) model Coding of the Post Observation Interview transcripts resulted in the Post Observation Interview Taxonomy ( Table 4 8 ). One port ion of the taxonomy emphasizes the points discussed by the teachers relating to student learning and engagement. All except one teacher stated that during the lesson I observed the technology helped engage or increase the focus of the students. The teach er who did not think the technologies helped engage the students mentioned that occasionally technical problems cause a slowdown in the lesson, which increases off task behavior s. The Post Observation Interview Taxonomy revealed outcomes regarding the enga gement level of the students and their learning to help answer the question for Level 5. The teachers mentioned that the technology helped the students to learn and reflect on their learning. During the Post Observation Interview s the teachers ta lked about the usefulness of the technology for providing instant feedback to the students. As noted on the Post Observation Interview Taxonomy, the teachers stated that the

PAGE 98

98 helped reveal whether the teacher needed to re teach the content. Teachers noted that student reflection and instant feedback helped to engage learners. Additionally, as shown on Table 4 8, the Post Observation Interview Taxonomy the teacher s indicated that the students remained engaged during the lesson. The teachers stated that the students were engaged in their use of the clickers and that the clickers helped to engage the students during learning due to the accountability involved with t he collection of data t ied to the individual student. Summary Evaluation (1998 2000, 2002a ) model helped evaluate the professional development focused on using the SRS and mo bile IWB for formative assessment. The outcomes indicate that the professional development was effective. In the final chapter of this study I discuss

PAGE 99

99 Table 4 1 Participation in the Study Study Component Criteria for Participation Number of Teachers Meeting Criteria for Participation (N) Number of Teacher Participa nts Who Met Criteria (N) Percentage of Teacher Participa nts Who Met Criteria Perceptions Questionnaire for Level 1 Attended some of the professional development. 40 36 90.00% Teacher Multimedia Artifact for Level 2 Attended all of the professional development. 18 15 83.33% Organization Support Questionnair e for Level 3 Attended all of the professional development. 18 16 88.89% Observation Cycle for Levels 4 and 5 Attended all of the professional development and completed the Perceptions and Organization Support Questionnaires. 16 12 75.00%

PAGE 100

100 Table 4 2 Means and Standard Deviations for the Perceptions Questionnaire Item Regarding the Professional Development N M SD 1. 36 3.72 0.45 2. Incorporated research based inst ructional strategies 36 3.72 0.45 3. Was a positive experience 36 3.72 0.45 4. Contributed to my learning 36 3.67 0.48 5. Met my needs as a learner 36 3.61 0.49 6. Connected to my prior knowledge 36 3.67 0.48 7. Included useful and meaningful content 36 3.64 0.49 8. A llowed me to consider use of the technologies in my content area 36 3.69 0.47 9. Included segments of time for reflection and/or lesson development 35 3.66 0.48 10. Included discussion and/or collaboration 35 3.60 0.55 11. Was facilitated in a professional manner 36 3.81 0.40 12. Was conducted at a convenient time 36 3.61 0.49 Note : The ratings on the Likert Scale were 4 = Strongly Agree 3 = Agree 2 = Disagree and 1 = Strongly Disagree. Table 4 3 Means and Standard Deviations for th e Artifact Rubric Rubric Item N M SD Plan for use of Technology for Formative Assessment 15 2.13 0.35 Use of Questions 15 2.27 0.46 Total 15 4.40 0.63 Note : For each of the two components the ratings were 1 = Emerging, 2 = Proficient, and 3 = Exemplary. The sum of the two scores designated the total score for the rubric, which was applied to the following scale: 6 points = Exemplary, 4 5 points = Proficie nt, and 2 3 points = Emerging.

PAGE 101

101 Table 4 4 Means and Standard Deviations for the Organization Support Questionnaire Item Regarding the Professional Development N M SD 1. I identified strategies for using the technologies for formative assessment. 16 3.50 0.52 2. I am able to us e the technologies for formative assessment. 16 3.44 0.51 3. When I use the technologies, my students are engaged in learning. 16 3.38 0.50 4. The professional development had a positive effect. 16 3.50 0.52 5. The teachers in my school view the professional dev elopment as positive. 16 2.94 0.44 6. My administrators view the professional development as important. 16 3.50 0.52 7. Other school district leaders view the professional development as important. 16 3.31 0.60 8. for input on the technology professional development topics. 16 2.63 0.72 9. The professional development helped me with my professional learning goals. 16 3.44 0.51 10. The content was connected to school improvement and student achievement. 16 3.63 0.50 11. Jul ia helped me when I needed assistance with the technologies. 16 3.81 0.40 12. I felt supported during implementation of the technologies. 16 3.75 0.45 13. during the sessions. 16 3.13 0.81 Note : The ratings on the Likert Scale were 4 = Strongly Agree 3 = Agree 2 = Disagree and 1 = Strongly Disagree. Table 4 5. Frequency Data for the Observation Rubric Rating Frequency (N=12) Percent of Observed Population Not Evident 0 0.00% Emerging 1 8.33% Proficient 10 83.33% Exemplary 1 8.33%

PAGE 102

102 Table 4 6 Pre Observation Interview Taxonomy Overarching Themes Codes and Sub Codes Planning to Use the Technologies during Instruction Planning to use technology to present material Planning to use the interactive whiteb oard to navigate the presentation Planning to use embedded clicker questions Planning for students to use clickers Planning to use the interactive whiteboard to write lesson notes Planning to use clickers to collect anonymous data Planning to Use Format ive Assessment Strategies Planning to use technology to monitor student progress Planning to use clicker data to monitor student progress Planning to use charting to monitor student progress Planning to use the interactive whiteboard screen to monitor stu dent progress Planning to adjust instruction Planning to use technology based data for decisions about adjusting instruction Planning to use the interactive whiteboard to provide descriptive feedback Planning to give explanation of correct or incorrect ans wers Planning for student involvement Planning to have students monitor their own progress Planning to use clickers to involve students in instructional decision making Stating Thoughts about the Technologies Stating interest in using clickers Stating the clickers provide immediate feedback Stating technology helps students learn Stating students are engaged with technologies less frequently used Stating students need more experience with technology Stating advantage of clickers to other forms of data c ollection Stating students are engaged when using clickers Stating the technology is new to them

PAGE 103

103 Table 4 7 Classroom Observation Taxonomy Overarching Themes Codes and Sub Codes Teacher Behavior : Using the Technologies Using the tablet to teac h Using tablet to solve problem Writing on tablet to explain a concept Using tablet to record student response Using tablet to engage clicker question via verbal bar Using tablet to navigate the lesson Using tablet screen to review individual student repor ts Using digital presentation medium to teach Displaying textual content Displaying images Displaying clicker questions Using the clickers to teach Asking a clicker question Using clicker data to produce charting of results Asking for my assistance Asking question related to the charting Asking how to return to the presentation Asking how to navigate the tablet reports Asking how to engage a different type question Asking how to navigate the verbal question bar with the interactive tablet Teacher Behavior : Using Formative Assessment Strategies Monitoring individual student progress Recording student responses with tablet Displaying chart of clicker data Discussing clicker data Looking at tablet report screen Reviewing cumulative percent correct Embeddin g assessment into instruction Adjusting instruction based on data Providing descriptive Feedback Rephrasing concepts Giving explanation of correct or incorrect answers Writing on tablet during re teaching Giving additional examples Asking additional questi ons to facilitate discussion

PAGE 104

104 Table 4 7. Continued Overarching Themes Codes and Sub Codes Pairing students to discuss content Facilitating student activity for understanding of concept Helping students monitor own progress Telling students to take notes Asking students if they understand/to express comfort level Asking students to explain answer choice Displaying chart of clicker data Giving descriptive Feedback Discussing reasons for a particular answer Discussing clicker data Discussing content with in dividual students based on tablet data Involving students in decision making Teacher Behavior: Directing use of the Technologies Telling students to get clickers Explaining use of clickers Telling students to use clickers Student Behavior: Engaging in Learning Answering questions with clickers Looking at presentation Responding to the charting data Looking at teacher Listening to teacher Looking at other students Listening to other students Discussing clicker question with other students Taking notes about content

PAGE 105

105 Table 4 8 Post Observation Interview Taxonomy Overarching Themes Codes and Sub Codes Student Learning Stating that students gained knowledge Stating technology helps students learn Stating technolo gy helped students to reflect on learning Stating that the technology provides instant feedback Stating that clicker data revealed students' learning Stating that data revealed students' progress throughout the lesson Stating that data helped to reveal w hether or not re teaching was needed Stating that re teaching occurred during lesson Comparing technology to other methods Student Engagement Stating that students were engaged/focused during the lesson Stating that students were engaged via interaction with clickers Stating that clickers helped engage students Stating that clickers helped make students accountable during lesson increasing their focus Stating that students enjoy using technology Stating that students were engaged during lesson via gro up activity Stating disagreement that clickers engage students due to technical problems Professional Development Session Design: Instructional Grouping Strategies Stating preference for working individually rather than with content team Stating pre ference to collaborate during sessions Stating that there are different levels of technology ability among the teachers Stating differentiation by level/process may be beneficial Stating preference for grouping by ability rather than content Stating it is difficult to work with slow learners during sessions

PAGE 106

106 Table 4 8 Continued Overarching Themes Codes and Sub Codes Professional Development Session Design: Learning Strategies Stating that professional development sessions were helpful with lear ning technology Stating that sessions helped with design and implementation of the observed lesson Stating usefulness of stations/hands on practice during sessions Stating preference for independent exploration Stating preference for help guides Statin g preference for modeling technology use Stating that learning technology is difficult Stating preference for le arning a small amount at a time Stating that the technology is new for them Stating need more practice with technology Suggestions for Pr ofessional Development Stating that applying in classroom is different than using technology during sessions Stating preference for personal feedback/support Stating preference for observing and providing feedback/support during implementation Stating preference for one on one assistance with learning technology Stating that having me come to the classroom would be beneficial Stating that individualized help may be difficult with a lot of teachers Stating that confidence increased following observatio n due to help given Suggesting an overview of upcoming sessions during meeting to spark interest

PAGE 107

107 CHAPTER 5 DISCUSSION AND IMPLI CATIONS This final chapter of the study includes a summary of the study, a review of the findings, and conclusions presented for each guiding question in the study. Additionally, I discuss additional outcomes and the implications for my work and future research. Summary of the Study focusin g on using newly purchased SRS and mobile IWB technologies for formative assessment during instruction. I designed professional development for this purpose based on the assumptions of adult learning (Knowles et al., 1998) and findings from professional d evelopment literature (Desimone, 2009). Additionally, I implemented the ADDIE instructional design model to help design, develop, and implement professional IWB for formati ve assessment. To determine the effectiveness of the technology based professional development, 2000, 2002a ) as a framework for designing the research questions and data col lection. 2000, 2002a ) model facilitated collection of data spanning learning during the following questions guided the data collection for studying the professional development:

PAGE 108

108 development? Level 2: In what ways did the teachers acquire the intended knowledge? Level 3: In what ways does the organization help teachers implement the tech nologies? Level 4: In what ways are the teachers using the mobile interactive whiteboard and student response system? Level 5: What effect did the professional development have on student engagement? 2000, 2002a ) L evels I collected data by various methods including Likert Scale questionnaires rubrics, and field notes from an observation cycle 2000, 2002a ) model helped evaluate the professional development o n using the SRS and mobile IWB for formative assessment. For Level 1 of the model , I collected reaction to the profes sional development I collected data for Level 2 Learning using the proficiency in their ability to design a tec hnology integrated lesson for collecting formative data. I collected the data for Level 3, Organization Support and Change, using the Organization teachers with t heir implementation of the SRS and mobile IWB during instruction. I for Level 4 using the Pre Observation Interviews and classroom observations to help determine the ity to effectively plan and implement the use of the SRS and mobile IWB for formative assessment. Last, I collected Level 5, Student Learning Outcomes, during

PAGE 109

109 the observations and Post Observation Interviews to determine if the professional development ha d an effect on student engagement. I conducted the study at two middle schools in Georgia. The teachers who participated in the study were from a convenience sample (Patton, 1987) as they worked at the schools that I serve and attended the professional development sessions (N=40). The response rate for the study components was as follows: Participant Perceptions Questionnaire (n=36), Teacher developed multimedia lesson (n=15), Organization Support Questionnaire (n=16), and Observation Cyc le to include the Pre and Post Observation Interviews and the classroom observation allowing for completion of the Observation Rubric (n=12). I used quantitative and qualitative methods for the study based on the type of data s (1998 2000, 2002a ) Levels. I designed questionnaires for the study align ed to assumptions of adult learning (Knowles et al., 1998) and findings from professional development literature (Desimone, 2009). I designed rubrics align ed to the Class Ke ys (Georgia Department of Education, 2010 a ) teacher evaluation instrument and modeled the design according to other methods and tools used for teacher evaluation ( Danielson 2007; North Carolina State Board of Education, 2007). Additionally, I used resear ch based techniques to collect (Dillman, 2007) and analyze the data ( Fraenkel & Wallen, 2006; Glesne, 2006; Patten, 1987; Spradley, 1979; Wolcott, 1994) Evaluation (1998 2000, 2002a ) model for this study helped develop a formali zed data collection plan for indicating the impact of my work and areas for further exploration. As noted by Dana and Yendol Hoppey (2009), formalized inquiry improves the teaching and learning process. Data from this

PAGE 110

110 evaluation of the professional devel opment provided information for developing plans to Findings Evaluation (1998 2000, 2002a ) model h elped evaluate the professional development on using the SRS and mobile IWB for formative assessment. The outcomes indicate that the professional development was effective. For Level 1 of the model , th e data from the Participant had a positive reaction to the professional development. Item means on this four item Likert Scale questionnaire ranged between 3.60 and 3.81. These outco mes indicate that the professional development met t The data for Level 2 of the model , from the Learning Artifact Rubric indicate that th e participants are proficient in their ability to design a technology integrated lesson for collecting data. None of the teachers earned a rating of Emerging on the rubric. All of the teachers, except one who earned an Exemplary rating, scored Proficient based on the rubric regarding their ability to design a lesson that used the technolo gies for formative assessment. The Reflective Journal although not used for data in the study, provided information 2000, 2002a ) that was helpful during triangulation (Denzin, 2006) of the results and gave me an opportunity to reflect on the components of my instruction, including what went well and what I could have done differently. The journal also helped me to keep notes thro ughout the process

PAGE 111

111 activities. The reflections in a written format were a useful reference throughout the sessions I facilitated, and the reflection process gave me an opportunity to use my reflections for improving the instruction. Data collection occurs naturally in the setting of a reflective practitioner (Dana and Yendol Hoppey, 2009). Likewise, this reflection on my teaching in written format helped me make conne ctions between my observations and the professional development literature and consider the implications for my work. Overall, the data from the Organization Support Questionnaire (N=16) regarding Level 3, Organization Support a nd Change, yielded positive results regarding the for formative assessment. Item means on this four item Likert Scale questionnaire ranged between 2.63 and 3.81. These outcomes indicate that overall the teachers felt supported during their learning about using the SRS and mobile IWB. addressed by Level 4 data from the Pre Observation Interviews, Observation Field Notes R ecords, and classroom Observation Rubrics indicate that the teachers are able to effectively plan and implement their learning. On the Observation Rubric (N=12), 8.33% of the participants (n=1) scored Emerging, 83.33% of the participants (n=10) scored Pro ficient, and 8.33% of the participants (n=1) scored Exemplary. The triangulation (Denzin, 2006) of these results implementing the technologies for formative assessment during their less on. The data for Level 4 indicate that the professional development helped the teachers plan and implement a lesson that incorporates the SRS and mobile IWB for formative assessment.

PAGE 112

112 Level 5 data regarding Student Learning Outcomes focused on the Observat ion Field Notes Records and Post Observation Interviews, which indicate that the professional development had a positive effect on student engagement. The data specify the use of the technologies helped students with their learning and engaged students du ring the lessons. Based on the data for this Level, student learning and engagement resulted from the student reflection stimulated by the instant feedback from the charting data, the individualized reports on the mobile IWB screen, and the interaction wi th the clickers. Conclusions I based the conclusions discussed in this section on the data from the guiding research questions for the study. To help with clarity I presented the conclusions in s Evaluation (1998 2000, 2002a ) model Guiding Question for Level 1: Participant Reactions development to be effective in the areas addressed by the Perceptions Questionnaire. The Perceptions Questionnaire items related to application of adult learning assumptions (Knowles et al., 1998) and professional development literature (Desimone, 2009) during imple mentation of the professional development sessions. Therefore, these data show an overall agreement by the participants that the professional development design and implementation addressed the supporting literature. Additionally, these data demonstrate that the design and facilitation method of the the adult learning assumptions (Knowles et al., 1998) and professional development

PAGE 113

113 strategies (Desimone, 2009) and work t o refine the instructional design of the sessions to maximize learning for the teachers. Because the data for Item 10, Included Discussion and/or Collaboration, included one Disagree rating, I reflected on whether the sessions effectively addressed this co mponent because the research on effective professional development indicates a need for collective participation ( Ball, 1996; Georgia Department of Education, 2008; Hill, 2009; National Staff Development Council, 2001; Wilson & Ball, 1996 ). I think there were sufficient opportunities for the participants to collaborate or discuss ideas for using the technologies for formative assessment or implementing strategies for adjusting instruction; however, this particular participant may have perceived the discuss ions as somewhat hypothetical and not truly collaborative. For this reason I considered ways to incorporate collaboration into the professional development ( Desimone et al., 2002; Desimone, 2009; Garet et al., 2001; Hur, & Brush, 2009; Mouza, 2003) I th ink more opportunities to collaborate on lesson planning would meet this need because it would help teachers complete a meaningful task for use with their students (Knowles et al. 1998 ). In support of this idea, when I reviewed the comments for Question 13 which In what ways, if any, would you like the professional development to be collaboration and time to c reate instructional materials. Guiding Question for Level 2 : the results indicate that the teachers are able to develop a lesson that uses the technologies for collecting formative assessme nt data, and they have ideas for as indicated in Figure 5 1

PAGE 114

114 Additionally, the results indicate that the teachers are proficient in their ability to incorporate use of clicker questions during instr uction (Bruff, 2009a; Beatty & Gerace, 2009). For example, one artifact I reviewed that met the proficiency criteria was a symmetry PowerPoint that had clicker questions directly aligned to the standards. The pacing of the questions and the planned use of technology was such that the teacher could immediately clarify misconceptions through re teaching (Bruff, 2009b; Beatty & Gerace, 2009). The multimedia lesson had slides with content prior to a clicker question, and the teacher planned to use the mobile IWB screen and charting to identify and address weakness among individual students before moving fo rward with the lesson. criteria for an Exemplary rating. This artifact iden tified plans for using the Peer Instruction clicker strategy (Bruff, 2009a, 2009b; Mazur, 1997) to assist struggling learners. Additionally, this lesson included a question at the end for gauging thoughts about the instructional strategies used in the lesson in order to involve them in future instructional decisions. Four components in the literature that I implemented in the design of the instruction contributed to the learning. First, development of the artifact provided an active le arning environment for effective professional development ( Bradshaw, 2002; Desimone et al., 2002; Garet et al., 2001; Hirsh & Killion, 2009; Mouza, 2003 ) allowing the teachers to consider how to implement the research based clicker strategies into a multim edia lesson. Second, the artifact gave the teachers an

PAGE 115

115 opportunity to focus on the content that they teach the students in addition to the focus on using the multimedia lesson they developed for formative data collection and assessment ( Desimone et al., 2 002; Desimone, 2009; Garet et al., 2001; Hirsh & Killion, 2009; Hughes & Ooms, 2004; Sparks, 2002 ). Third, the development of a multimedia lesson gave the teachers a real world task to accomplish that helped provide meaning to their learning (Knowles et a l., 1998). Last, the teachers had time during the four professional development sessions to collaborate on strategies for formative assessment (Desimone et al., 2002; Desimone, 2009; Garet et al., 2001; Guskey, 2003b) I will continue to use these litera ture based strategies in the design of professional development because of the effect on teacher learning. In addition, the evaluation of the professional development is important. As noted by Guskey and Yoon (2009), critical evaluation of professional d evelopment helps determine its effectiveness, as studies reveal a connection between student learning and professional development adequat ely designed and implemented Guiding Question for Level 3: Organization Support and Change For the qu In what ways does the organization help teachers implement the received from the organization. Two of the Organization Support Questionnaire items, Item 11 and Item 12, received significant approval. Item 8 had a lower rating on the scale than expected. The data pinpointed strategies to continue, a s well as some to reconsider. For example Item 11, Julia Helped Me When I Needed Assistance with the Techno logies, received the highest rating on this questionnaire. I reflected on the

PAGE 116

116 that the teachers felt supported because of certain strategies I implemented during th e sessions. I set up stations during the sessions and helped the teachers experiment with the technologies. This helped increase their comfort level with the technologies because they had hands on opportunities to use them and I spent time answering thei r questions. The stations and hands on opportunities provided the teachers with an active learning environment for engaging in the learning process (Bradshaw, 2002; Bonwell & Eison, 1991; Desimone, 2009; Mouza, 2003), which contributed to the rception of receiving support during their learning. Another opportunity I had to help the teachers was during the sessions when they were working on developing their multimedia lesson. While I circulated I answered the ting the clickers and mobile IWB into their lessons to collect formative data. If the teachers asked, I also helped with the design of their lesson and provided positive feedback, which is a motivational strategy in the literature regarding adult assum pti ons (Knowles et al., 1998). Item 12, I Felt Supported during Implementation of the Technologies, also received a high rating with a mean of 3.75. This indicates that not only did the teachers appreciate my help as they were developing their lesson materia ls as discussed above, but they also felt supported by their teammates and administrators. This support was evident to me as I observed the teachers helping each other during the sessions. Additionally, the administrators at each school made known their support of using the technologies through visits during the sessions, formation of technology committees, and discussions with the teachers. This support by the administrators was also evident in the data for Item 6, My Administrators View the Professiona l Development as

PAGE 117

117 Important, which had a mean of 3.5. The emphasis described here on collective participation and coherence is explained in the professional development literature as contributing to effective professional development ( Desimone et al., 2002 ; Desimone, 2009; Garet et al., 2001; Penuel et al., 2008 ). Collective participation and coherence contributed to the success of the professional development in this stud y as detailed in the findings. Item 8, ed for Input on the Technology Professional Development Topics, rated lower than I expected. Two ratings of Disagree and two Strongly Disagree ratings affected the of 2.63. In ed the teachers for their input; however, some participants indicated they did not have the opportunity to provide input on the professional development topics. I think that facilitating the communication differently could help give everyone an opportunit y to provide input on the professional development. There are several ways to accomplish this; however, one idea is having the leadership team administer a survey or set of group interview questions to their colleagues and bring the data back to the group for review. This would help to formalize the needs assessment and solidify beliefs regarding involvement in the decision making. Collaboration about the content and the process for professional development increases motivation and commitment to the lear ning ( Katzenmeyer & Moller, 2009). Guiding Question for Level 4: The Pre Observation Interview and the classroom o bservatio n helped me collect data for answering th is question : In what ways are th e teachers using the mobile

PAGE 118

118 teachers are using the technologies to collect data that will help them to monitor student progress and adjust instruction as needed to assist a ll learners. These results reflect 2000, 2002a ) Level 4 were valuable in helping me identify behaviors that were most common among the participants regarding their us e of the SRS and mobile IWB, as well as behaviors that were useful during instruction but not as prevalent. From these data I identified ways to improve the professional development I facilitate. As I triangulated the results from the taxonomies for the P re Observation Interview, the Classroom Observation, and the Observation Rubric, I thought about the implementing the SRS and mobile IWB for formative assessment. The tea strengths as identified from the Pre Observation Interview Taxonomy include their ability to plan for data collection using the technologies, plan for monitoring student progress using the technologies, plan for using the data to make decisions abou t adjusting instruction, and plan for having students monitor their own progress during the lesson based on the data. The overarching themes established by the Classroom Observation Taxonomy when coding the field notes Using the Technologies, Using Format ive Assessment Strategies, and Directing Use of the Technologies pinpointed the common behaviors of the teachers when using the technologies during instruction. The Using Formative Assessment Strategies section of the Classroom Observation Taxonomy alig ned with the data from the Pre Observation Interview Taxonomy because in general the teachers effectively implemented their plans. The data from the

PAGE 119

119 collecting formative data and adjusting their instruction based on the data to meet Exemplary levels of the Observation Rubric focuses not on the collection of data about student learning, but on ho w teachers use the findings to improve student achievement ( Garrison & Ehringhaus, 2007). Technologies can help the teachers immediately access student data and then teachers may use formative data to fine tune instruction in real time for improving stud ent comprehension (Miller, 2009). The data from the Observation Rubric triangulated with the data from the Pre Observation Interview Taxonomy and the Classroom Observation Taxonomy indicate that the majority of teachers are proficient in their implementat ion of the technologies for individualized and immediate formative assessment during instruction. These outcomes result from the collaboration among participants focusing on student achievement (Guskey, 2003 b ; Hur, & Brush, 2009; Mouza, 2003). These data helped me reflect on how the professional development mobile IWB for formative assessment. As an example of proficiency in a math classroom, I observed a teacher reviewin g the concept of symmetry and then asking application questions in which the students had to apply the concept to various sets of shapes. When students missed a question, the teacher facilitated a discussion about why each of the shapes did or did not hav e a line of symmetry and provided additional examples, which is an example of the Contingent/Agile Teaching clicker strategy (Bruff 2009b; Beatty et al., 2006; Draper & Brown, 2004). Similarly, another way identified by the rubric as Proficient is to use the

PAGE 120

120 mobile IWB screen to determine the individual students who lack conceptual understanding and adjust instruction to meet their needs. The teachers scoring Proficient demonstrated their ability to use the technologies for formative assessment according to t he guidelines of the rubric. Although the teachers were able to demonstrate their ability to adjust their of strategies for this purpose to achieve an Exemplary score on the rubric. Therefore, I reflected on how the Observation Rubric, may provide information to help teachers wit h use of these tools to enhance student learning. For instance, in a science classroom I observed a teacher facilitating a discussion about animal adaptations. Following a series of clicker questions, the teacher noted that several students were having d ifficulty with the concept by reviewing the mobile IWB screen data The teacher paired the students based on the data so that students who did not understand had a partner who did. Then the teacher asked the students to discuss the concept and answer wit h the clickers again. The students discussed the question with their partner and then answered the most recent clicker question again as described in the Peer Instruction clicker strategy (Bruff, 2009a, 2009b; Mazur, 1997) A follow up discussion gave st udents the opportunity to explain their answer choices. Additionally, in an exemplary lesson such as this one, students are involved in helping the teacher determine successful instructional strategies or modifications preferably via use of the technologi es for data collection. The science teacher exemplified this concept by

PAGE 121

121 asking the students to use their clickers to identify the effectiveness of a particular component of the lesson for increasing their understanding of the topic. This teacher went bey ond the Proficient rating by asking a question to involve the students in instructional decisions and using a variety of clicker strategies to maximize student achievement. Since one teacher scored Emerging on the Observation Rubric, I considered how I mig ht help that teacher and potentially others not involved in this study to use the technologies effectively for formative assessment. In reviewing the field notes for this one teacher, I noticed that there was no weakness in using the technologies. The ma in reason for the Emerging rating was inefficiency in using data to help individual students understand the concepts. This lack of ability to adjust instruction may stem from a lack of planning. Similarly, one outcome from the Pre Observation Interview d emonstrated specific strategies for adjusting instruction based on the data. In this study the teachers were able to adjust instruction effectively; however, a lack of planning could translat e to a lack of impleme ntation. Although modifying instruction in a timely manner is a skill that takes practice, I can facilitate the importance of planning strategies in advance during future professional development sessions through teacher collaboration. Additionally, outco mes reveal that the majority of participants are not using the technology to involve the students in making instructional decisions for achieving an Exemplary rating on the Class Keys teacher evaluation (Georgia Department of Education, 2010 a ). I will add ress this concept again in future sessions by providing opportunities for collaboration among content area teachers ( Desimone et al., 2002; Desimone, 2009; Garet et al., 2001).

PAGE 122

122 Teachers learn from each other through their collaborative interactions (Ball, 1996; Georgia Department of Education, 2008; Hill, 2009; National Staff Development Council, 2001; Wilson & Ball, 1996). Critique, analysis, and self challenge resulting from discussions with colleagues help teachers expand their knowledge about teaching (Freedman, 2001). Overall, the professional development was effective in that the teachers implemented the strategies for using the SRS and mobile IWB for formative assessment as indicated on the Classroom Observation Taxonomy. The literature on adult le arning assumptions (Knowles et al., 1998) and professional development (Desimone, 2009) was an integral part of the design and implementation of the professional development for this study. The use of these effective strategies contributed to the particip learning. Guiding Question for Level 5: Student Learning Outcomes The Observation and Post Observation Interview collected data for answering th is question : engagement? lessons helped engage the students as indicated in Figure 5 2, which highlights the connections among the data from the Observation Field Notes and the Post Observation Interviews These results reflect positively on the effect of the professional development, indicating that the teachers have learned ways to incorporate the tec hnology for engaging learners. perceptions during the Post Observation Interviews regardin g the stated that the clickers helped to engage the students by increasing their focus during

PAGE 123

123 the lesson as well as their interaction with the clickers. Using clickers th roughout a lesson provides opportunities for formative assessment and helps to engage learners (Bruff, 2009a; Beatty & Gerace, 2009). The Post Observation Interview Taxonomy revealed that teachers think the students enjoy using technology during instructi on. The teachers noted that the use of the technologies makes the student accountable during the lesson. For example, when a teacher asked a question the clicker system indicated which students had not answered the question. Similarly during the obser vations when the teacher reviewed the mobile IWB screen data it was apparent which students had answered the item correctly, as well as those needin g re teaching of the concept. I observed the teachers using this data to engage their learners. The teache rs used the data to monitor learning adjusted instruction by providing descriptive feedback asked additional clicker questions to facilitate discussion, and used the clicker strategies to promote engagement. Each of these techniques, as well as others i n the Observation Taxonomy, facilitated interaction among the students, teacher, technologies, and data for helping students learn. The reason that the professional development positively affected the engagement of the students during the lessons is the based clicker strategies during instruction which I observed during their lessons The participants learned the importance of integrating clicker questions throughout the lesson (Bruff, 2009a; Beatty & Gerace, 2009) as indic ated by the observation data. The teachers were also consistent in their use of the Contingent/Agile Teaching clicker strategy (Bruff, 2009b; Beatty et al., 2006; Draper & Brown, 2004), and several participants implemented the Discussion Warm up/Think Vo te Share clicker strategy (Bruff, 2009a,

PAGE 124

124 2009b; Lyman, 1981). The professional development sessions helped the teachers work cooperatively to focus on their content, accomplish authentic tasks, consider use of the clicker strategies during their instructi on, and implement the strategies successfully for the benefit of learners. I would like to help teachers increase their strategies for engaging learners. An exemplary teacher uses a variety of strategies for formative assessment (Georgia Department of Edu cation, 2010 a ). The participant who earned an Exemplary rating on the Observation Rubric implemented the Peer Instruction clicker strategy (Bruff, 2009a, 2009b; Mazur, 1997) during the lesson I observed in addition to the other clicker strategies Th e Pee r Instruction strategy is effective for engaging the students in discussions with each other and providing a method for adjusting instruction. Given the benefits of this strategy, I will help the teachers in the schools in which I work to implement it dur ing instruction. Teacher leaders can help make changes regarding student learning as they Capacity building with a focus on results that includes positive pressure from t eacher leaders can escalate the effectiveness of the group for increased student achievement (Fullan, 2006). Therefore, I will work with this teacher leader and others to help facilitate an increase in teacher knowledge th at effects student achievement. A dditional Outcomes The data collection revealed an ancillary finding not specifically related to the guiding questions in the study. During the Post Observation Interviews the teachers indicated a need for one on one professional development. This outcom e resulting from the data points to future research regarding implementation of an instructional

PAGE 125

125 coaching model to serve teachers with their technology integration needs. This finding (1998 2000, 2002a ), as the last interview questions addressed collecting further data to improve the professional development I provide. The following questions included in the Post ding the professional development: In what ways, if any, could I modify the professional development to help instructional needs? Are there other strategies that you would like me to use to support you in your technology integrati on? If so, please explain. One predominant theme arose in 11 of the 12 interviews. The teachers expressed a need for one on one support while learning and implementing technology. They discussed working with me during their instruction for increasing th eir comfort level with with you being in here, in the classroom, one on one. I was able to talk to you and ask you questions as they came up. In addition, once you le ft after the first core, I felt very throughout the Post Observation Interviews. Implications for Future Work and Research There are four concerns worth considering regarding their app lication to my work as an ITS: 1. A weakness was variety of strategies for adjusting their instruction immediately based on data to meet 2. he Participant Perceptions Questionnaire and interview transcripts indicate their preference to have more time to

PAGE 126

126 work with the technology incorporated into the prof essional development sessions. 3. The results of data analysis for the additional questions in the Post Observation Interview indicated a need for implementing an instructional coaching mode l of professional development. 4. No strategic method is in place for evaluating the technology professional development conducted by the ITSs in the school system Evaluation (1998 2000, 2002a ) provides a systematic method for continuous improvement in the technology based professional development in which teachers participate, which I can implement again and may be valuable to others as well. This section describes strategies regarding these concer ns and ways to address them. Formative Assessment to Meet Individual Needs adjust their instruction immediately based o n data to achieve an Exemplary score on the Observation Rubric. I will work with the teachers to develop ideas based on research for using the technology based formative assessment data for immediately adjusting instruction to accommodate the needs of ind ividual students. During the upcoming school year, the District has planned a continued focus on using the technologies for formative assessment. I will facilitate professional development sessions that address using a variety of strategies for adjusting instruction. During the sessions the teachers will discuss using the technologies and associated research based strategies. I will re address the following literature based strategies during the professional development for this purpose: Discussion Warm up/Think Vote Share clicker strategy (Bruff, 2009a, 2009b; Lyman, 1981) and Peer Instruction clicker strategy (Bruff, 2009a, 2009b; Mazur, 1997). I will not re address the Contingent/Agile

PAGE 127

127 Teaching clicker strategy (Bruff 2009b; Beatty et al., 2006; Drap er & Brown, 2004) understanding and develop ideas for implementation in their specific content areas. One option for helping the teachers conceptualize and implement these formative assessment clicker strategies is a strategy called Assessment as Professional Learning in which teachers collaborate to design and evaluate assessments (McTigh e & Emberger, 2001). More Time to Work with the Technologies to work with the technologies during the professional development sessions by incorporating work sessions into the instructional sequence. The work sessions will give the teachers additional time to explore the technologies, ask questions, and work on developing technology integrated lessons for use with their students. A work session will follow a series of sess ions on a particular topic, and the work session will focus on learning about and developing lessons that incorporate that particular topic and its related technologies. Adult learning assumptions (Knowles et al., 1998) and professional development litera ture (Desimone, 2009; Bradshaw, 2002; Mouza, 2003 ) support a focus on an authentic task in a n active learning environment. Instructional Coaching on one professional learning that includes support during implementation of tech nologies and feedback during observations aligns with a professional development strategy referred to as instructional coaching. This professional development model, initially discussed by Joyce & Showers (1980),

PAGE 128

128 suggests that for professional developme nt to be effective it should include a combination of theory, modeling, practice, feedback, and coaching. Technology integration areas suitable for instructional coaching are those that are a focus point for the teacher and observable during instruction. Content focused coaching described by Yendol Hoppy and Dana (2010) is a strategy that would facilitate the work of ITSs as instructional coaches as they worked with teachers to develop their technology based content knowledge. Instructional coaching is a component of professional development that is worth further exploration based on the additional findings in this study. A teacher involved in instructional coaching would meet with the coach and set goals related to his or her technology integration (Bark ley, 2010). The teacher would attend sessions that incorporate theory, modeling, and practice related to his or her goals (Joyce & Showers, 1980). The teacher would then participate in a coaching cycle that includes a pre observation conference, an obser vation, and a post observation conference. During the pre observation conference the teacher would share his or her lesson and review his or her technology integration goals to help define the focus of the observation (Barkley, 2010). During this confer ence the teacher and coach would define the level of participation/assistance available during the observation. The coach would document strategies related to the instructional goals that were useful, document questions about the strategies, and document ideas or suggestions for future lessons during the classroom observation. At the post observation conference the coach and teacher would discuss the focus point of the observation and the coach would provide feedback (Barkley, 2010). Additionally, a coac hing model would require an atmosphere

PAGE 129

129 in which the instructional coach is viewed as an expert, while continuing to maintain trust among the teachers (Mangin & Stoelinga, 2011). Downplaying expert status by an instructional coach causes difficulty in prov iding the type of feedback that facilitates instructional improvement (Mangin & Stoelinga, 2011). Instructional coaching would provide one on one opportunities for collaboration, as well as goal setting and reflection, which are critical for successful pr ofessional development (Orrill, 2001). It is important professional development includes follow up coaching and support (Katzenmeyer & Moller, 2009). Future Use of G Evaluation Evaluation (1998 2000, 2002a ) provides a framework for future implementation. Since it is flexible regarding the professional development topic that is evaluated this model is applicable in a variety of technology related professional development venues. This model provided valuable data in this study; therefore, I plan to use it again as a framework for evaluating the professional development I conduct. On the next occasion I will consider ways to streamline the data collection methods if the evaluation is conducted independently; if conducted by a team, I think it would be helpful to divide the data collection and analysis among the group to facilitate efficiency in completing the evaluation. For example, next time I will not code the data for the last question on the Pre Observation Interview that asks the participant about any help needed before the lesson because this question did not effe ctively contribute to the data collection.

PAGE 130

130 Recommendations It is apparent that professional development needs to incorporate adult learning assumptions, professional development and technology based literature, as well as ISD to be effective. Albeit consistent efforts to integrate the literature into the professional development may be apparent, indeterminate weaknesses in the design could affect its success This study demonstrates ofessional Development Evaluation (1998, 2000, 2002a) model is a comprehensive approach for identifying areas in the professional development strategies for enhancement and informing future initiatives for teacher growth and student improvement (G uskey, 2002a) For school systems or other organizations interested in evaluating the impact of t heir professional development, I recommend a segmented approach that includes E valuation (1998, 2000, 2002a) in phases over two or more years This method will provide an opportunity to address any apparent weaknesses in advance and confront the evaluation systematically to ensure the effectiveness of the professional development at each of the evaluation l evels. First, overview your current professional development plans for any apparent weaknesses in the design or implementation Us e the proposed framework in Figure 2 2 as a guide for assessing the design of the professional devel opment prior to the evaluation. As discussed herein, it is important to c onsider the literature during instructional design of professional development Rev iew the system and school improvement goals and appropriate needs assessments to ensure alignment with a ny

PAGE 131

131 long term professional development initiative s Make any needed modifications prior to the evaluation. Next implement Guskey s Evaluation (1998, 2000, 2002a) model in phases. Since most professional development initiatives take place over several years, it is feasible to ensure success at the early l evels of the evaluation prior to implementing the later l evels. Evaluation (1998, 2000, 2002a) for this study, I recommend completing Levels 1 3 consecutively during the first year of the evaluation. Avoid moving to the next level of the evaluation until effectiveness is established at the current level. This is important because Guskey (1998) notes that the preceding levels must be successfu l for the following levels to be successful Once the first three Levels Participant Reactions , and Organization Support and Change demonstrate that the professional development is eff ective proceed with Levels 4 and 5 of the evaluation in succeeding years to measure the teacher application of learning and student achievement In conclusion, Five Levels of Professiona l Development Evaluation (1998 2000, 2002a ) to be useful in assessing the value of the technology based professional development I provided to the middle school teachers in my school system on using the SRS and mobile IWB for formative assessment The re sults of the study support the literature regarding consideration of adult learning assumptions, core features for effective professional development, and ISD during the design and implementation of professional development. The data also indicated areas for growth to promote continuous improvement and identified a need for implementation of an

PAGE 132

132 instructional coaching model. Finally, a segmented approach to Evaluation (1998, 2000, 2002a) model can benefit organizations interested in evalu ating their professional development.

PAGE 133

133 Figure 5 1. Venn Diagram Highlighting S imilarities A mong the Pre Observation Interview and Observation Data Figure 5 2 C onnections Among the Data F rom the Observation Field Notes and the Post Observation In terviews

PAGE 134

134 APPENDIX A SESSION HANDOUT SBI. 1.5 The teacher uses accessible technology effectively to enhance student learning. AL 1.2 The teacher uses formative assessment strategies to monitor student progress and to adjust instruction in order to maximiz e student achievement on the Georgia Performance Standards (GPS). Ga DOE Class Keys (2010) Continuum of Improvement SBI 1.5 Not Evident Emerging Proficient Exemplary The teacher does not use accessible technology to enhance student learning. The t eacher uses accessible technology; however, technology is used primarily with the whole class, select students, or as a tool for tutorials and drill. The teacher routinely uses accessible technology to enhance student learning and support their achievemen t. The teacher develops, implements, and evaluates a comprehensive approach for using accessible technology to enhance learning and achievement for all students. AL 1.2 The teacher does not use formative assessment strategies either to monitor student pr ogress or to adjust instruction to meet student needs. The teacher uses some formative assessment tasks and tools to guide adjustments of whole class instruction; however, formative assessment is rarely used at the individual level or may be inconsistentl y implemented. The teacher consistently uses formative assessment tasks and tools to monitor student progress over the course of most units and to adjust instruction to meet learning needs relative to GPS. The teacher consistently us es a variety of formative assessment tasks and tools to monitor student progress over the course of all units and adjusts instruction to maximize student achievement relative to GPS for all learners. The teacher also involves students in decisions about ad justments to instruction to enhance their learning. Note: This session handout aligns two of the standards from the Class Keys teacher evaluation instrument ( G eorgia Department of Education, 2010 a )

PAGE 135

135 APPENDIX B POST INSTRUCTION REFLECTI VE JOURNAL Les son # ________ Participant ID #s ____________ 1. How did the teachers respond to the introduction material? 2. How did the teachers respond to the cooperative activities? 3. How did the teachers respond to the discussions? 4. How did the teachers res pond while experimenting with the technologies? 5. What indicated that the teachers learned during this lesson? 6. What could I have done differently during this lesson?

PAGE 136

136 APPENDIX C PERCEPTIONS QUESTION NAIRE ID#________ Please answer the followi ng questions regarding the professional development on the use of the interactive tablet and/or student response system for formative assessment. The professional learning provided by Julia Fuller on integrating the interactive whiteboard and clickers for formative assessment: Strongly Agree Agree Disagree Strongly Disagree 1. improvement plan 2. Incorporated research based instructional strategies 3. Was a positive experience 4. Contributed to my learning 5. Met my needs as a learner 6. Connected to my prior knowledge 7. Included useful and meaningful content 8. Allowed me to consider use of the technologies in my content area 9. Included segments of time for reflection and/or lesson development 10. Included discuss ion and/or collaboration 11. Was facilitated in a professional manner 12. Was conducted at a convenient time 13. In what ways, if any, would you like the professional development to be modified in future sessions? ______________________________________ ________________________________ ______________________________________________________________________ Comments: ______________________________________________________________________ _____________________________________________________________________

PAGE 137

137 APPENDIX D PARTICIPANT S LEARNING ARTIFACT RU BRIC ID#(s)________ Rubric for Evaluating the Teacher Developed Multimedia Lesson Emerging (1) Proficient (2) Exemplary (3) Points Plan for use of Technology for Formative Assessment o The multimedia lesso n includes a formative assessment plan. o The formative assessment plan includes some use of the charting feature or other data to monitor student progress and adjust instruction. o The description of the formative assessment plan includes use of the charti ng feature and other data to monitor individual student progress or use of the mobile interactive whiteboard screen to monitor learning of individual students and the class. o The plan addresses using the technologies to adjust instruction relative to the Georgia Performance Standards. o Addresses Proficient indicators. o The description of the formative assessment plan includes use of a variety of clicker strategies. o Includes an idea for using clicker data to facilitate student reflection about their own le arning and involve them in instructional decisions. Use of Questions o The multimedia lesson includes fewer than 3 questions. o The questions are not clearly aligned to lesson content. o The questions are grouped together rather than spread throughout the lesson. o The multimedia lesson includes more than 6 questions. o The questions are clearly aligned to lesson content. o The questions are unobtrusively grouped together. o The multimedia lesson includes 3 6 questions throughout the lesson content. o The qu estions are clearly aligned to lesson content. o The pacing of the questions is such that formative assessment data may be used immediately to clarify misconceptions. Total Points Rating Scale Exemplary 6 points Proficient 4 5 points Note: Aligned to Class Keys (Georgia Department Emerging 2 3 points of Education, 2010a)

PAGE 138

13 8 APPENDIX E ORGANIZATION SUPPORT QUESTIONNAIR E ID#________ Please answer the following questions regarding your use of the te chnologies (interactive tablet and/or clickers) for formative assessment and the associated professional development. Regarding the professional learning provided by Julia Fuller, please rate the following: Strongly Agree Agree Disagree Strongly Disagree 1. I identified strategies for using the technologies for formative assessment. 2. I am able to use the technologies for formative assessment. 3. When I use the technologies, my students are engaged in learning. 4. The professional development had a pos itive effect. 5. The teachers in my school view the professional development as positive. 6. My administrators view the professional development as important. 7. Other school district leaders view the professional development as important. 8. My sc representative asked for input on the technology professional development topics. 9. The professional development helped me with my professional learning goals. 10. The content was connected to school improvement and student achie vement. 11. Julia helped me when I needed assistance with the technologies. 12. I felt supported during implementation of the technologies. 13. recognized and shared during the sessions. Please list suggestions as wel l as strategies I used during the sessions that were most helpful to you: ___________________________________________________ ________________________________________________________________

PAGE 139

139 APPENDIX F PRE OBSERVATION INTERVIE W Date _____________________ __________________ ID# ____ ____________ Beginning Time _____________________ Ending Time ________________ Introduction: The purpose of this interview is to help me understand your plans for the lesson and to determine if you need any assistan ce before the lesson. 1. What is the sequ ence of events for this lesson? Probe: How do you plan to integrate technology into this lesson? Probe: How will you or the students use the technology? Probe: Why did you decide to use the technology in this way ? 2. What is your plan for monitoring student progress? so, please explain. Probe: Why did you decide to monitor student progress in this way? 3. Is there anything I can do to assi st you before this lesson? If so, how may I help? Ending Statement: I am looking forward to your lesson. The purpose of my observation is to study the effects of my technology integration sessions.

PAGE 140

140 APPENDIX G OBSERVATION FIELD NO TES RECORD Researcher Name: Julia Fuller Study Name: An Evaluation of Instructional Technology Professional Development in the Middle Schools of a Southeastern School District Protocol #: Observation Date: Beginning Time: Participant ID#: Protocol Completion Date: Endin g Time: Focus Points for the observation: focus on teacher technology use, student technology use, formative assessment use of technology, and student engagement : In what ways is the teacher using the technologies? In what ways is the teacher collecting formative data? In what ways has the teacher directed the students to use the technologies? Description of environment: Observations: 1 2 3 4 5 6 7 8 9 10 (Continue numbering until end of observation) Reflections/Insights: [Brackets will indicate reflections noted during the observation. Reflections made during the observation will be noted alongside the observation field notes.]

PAGE 141

141 APPENDIX H OBSERVATION RUBRIC SBI. 1.5 The teacher uses accessible technology effectively to enhance student learning. AL 1.2 The teacher uses formative assessment strategies to monitor student progress and to adjust instruction in order to maximize student achievement on the Georgia Performance S tandards (GPS). Ga DOE Class Keys (2010) Continuum of Improvement SBI 1.5 Not Evident Emerging Proficient Exemplary The teacher does not use accessible technology to enhance student learning. The teacher uses accessible technology; however, techno logy is used primarily with the whole class, select students, or as a tool for tutorials and drill. The teacher routinely uses accessible technology to enhance student learning and support their achievement. The teacher develops, implements, and evaluates a comprehensive approach for using accessible technology to enhance learning and achievement for all students. AL 1.2 The teacher does not use formative assessment strategies either to monitor student progress or to adjust instruction to meet student ne eds. The teacher uses some formative assessment tasks and tools to guide adjustments of whole class instruction; however, formative assessment is rarely used at the individual level or may be inconsistently implemented. The teacher consistently uses form ative assessment tasks and tools to monitor student progress over the course of most units and to adjust instruction to meet learning needs relative to GPS. The teacher consistently uses a variety of formative assessment tasks and too ls to monitor student progress over the course of all units and adjusts instruction to maximize student achievement relative to GPS for all learners. The teacher also involves students in decisions about adjustments to instruction to enhance their learnin g. Examples of Evidence Aligned to the Ga DOE Class Keys (2010) Indicators: o does not use the provided technologies. o uses clicker questions that are not guided by GPS aligned lessons. Indicators: o employs clickers for whole cla ss summative assessment (e.g. giving a test) or game. o delivers clicker questions during direct instruction. o uses some charts or other instant feedback to monitor student learning and adjust instruction. Indicators: o delivers GPS aligned cli cker questions during direct instruction. o consistently uses charts or other data to monitor individual student learning and adjust instruction. o or uses the mobile interactive whiteboard screen to monitor learning of individual students and the class and ad just instruction. Indicators: o consistently implements each of the proficient indicators. o uses a variety of questioning strategies with clickers. o uses clicker data to facilitate student reflection about their own learning and involves them in instructional decisions. Note: Modeled after the Rubric for Evaluating North Carolina Teachers (North Carolina State Board of Education, 2007)

PAGE 142

142 APPENDIX I POST OBSERVATION INTERVIE W Date _______________________________________ ID# ____ _________ ____ Beginning Time _____________________ Ending Time _________________ Introduction Statement: Thank you for the opportunity to observe your lesson. The engagement, as w ell as to gather information regarding the professional development I offer. 1. What Probe: and understanding? If so, please explain. Probe: How do you know if the students were learning during your lesson? on learning? Probe: Was there an opportunity for you to use data to adjust i nstruction during this lesson? If so, please explain. 2. In what ways, if any, could I modify the professional development to help you 3. Are there other strategies that you would like me to use to support you in your technology integration? If so, please explain. 4. In what ways, if any, did you use the information from my sessions to design and/or implement this lesson? Ending Statement: Thank you so much for your participation in this study. Your insight will help me to improve my practice.

PAGE 143

143 APPENDIX J INTERVIEW CODING PRO TOCOL FORM Researcher Name: Julia Fuller Study Name: An Evaluation of Instructional Technology Professional Development in the Middle Schools of a Southeastern School District Protocol #: Inter view Date: Beginning Time: Participant ID#: Protocol Completion Date: Ending Time: Transcript Codes and Themes I completed this process using Hy Note: Modeled after Fraenkel & Wallen ( 2006, p. 489) Data Collection Form

PAGE 144

144 APPENDIX K POST INSTRUCTION REFLECTI VE JOURNAL EXCERPTS The various presentation tools that I used during each lesson Some teachers were very excited to learn about the tools and eager to create a lesson with the tools to use with their students. Additionally, in modeling the technologies, there was a general interest in learn ing more about the mobile IWB screen. One teacher expressed interest in just focusing on th e interactive screen for now. During the needs assessment portion of the first session, the teachers were honest with their responses regarding their use and comfor t level of the technologies. I collected information to help me know what they needed to practice over the next few weeks. They perceived themselves as less skilled than I thought they would. This activity helped me think about how to support them in th eir learning of the technology. I think that last year when I was teaching the basic uses of the clickers, there were so many possibilities that some of the teachers were overwhelmed. I designed these four lessons to focus on only a few strategies for us ing th e tools formatively, which helped the teacher to not only feel more comfortable with this particular aspect of the tools but also provide d some strategies for using them. After the first session I was not really su re that the teachers understood the general idea of using the technologies for formative assessment, at least in the way that I was planning to emphasize during these sessions which was to use the charting and interactive screen data to immediately modify instruction. Based on their discussions, it seemed that they were used to referring to data after the ir lesson s I planned to facilitate their consideration of how they could

PAGE 145

145 differentiate based on the individual mobile IWB screen reports during the ir l esson s I designed questions for the cooperative activities for the following sessions to meet this need. For each cooperative activity, the majority of the teachers talked to each other about the tools and discussed ideas regarding how they were using th em. They talked about possible lessons they could create to collect formative data. For one group this discussion did not specifically address the question posed to the group, but actually took it a step further to thoughts about application. One time I was not sure that they all understood the question at first, but then a participant responded in a way that I think focused on figuring out how they could design a lesson f or gathering formative data and application of formative assessment strategies During one discussion the teachers expressed an interest in the Peer Instruction (Bruff, 2009a, 2009b; Mazur, 1997) clicker strategy. The groups were actively coming up with i deas that made sense for effectively adjusting instruction and involving the students in reflection and instructional decisions. All of the groups were willing to share, and I think they learned from each other. I valued their ideas for using the technol ogies in their classrooms. One group was a little timid about sharing, but the other groups shared multiple ideas, which I think helped the others to conceptualize use of the tools for formative assessment. During the group discussions they also discusse d their thoughts about formative assessment and shared ideas about how they might be able to use the tools. They shared their ideas willingly, and I recorded the ideas using the mobile IWB. The teachers seemed to value their discussions.

PAGE 146

146 During the experi mentation portion of the session, we practiced with the mobile IWB screen and navigated through the various reports. Many of the teachers grouped around the stations while others worked on lessons. One group of teachers did not use the stations, but inst ead work ed collaboratively on lesson development. While teachers were at the stations, I worked with them on the verbal questions and the mobile IWB screen report s. During the fourth session th e cooperative activity gave the teachers an idea of how they c ould use multiple mobile IWBs during instruction. Additionally, their written and verbal answers to the questions demonstrated their understanding of formative assessment and student decision making using the technologies. During the activity they genera ted new ideas to improve their instruction. They also shared ideas about using the technologies to do things that they normally do in another way. This activity information to validate the learning that had taken place during the sessions. Modification considerations for future sessions. It seemed that I had a bit too much planned for the first session. The review of the tools functions that are useful for gathering data, the modified KWL, and the cooperat ive activity were enough for one day. There was not time to start on the multimedia lesson but rather just time to think about it. After the first session I wondered whether my focus question for the cooperative activity was too broad or not clear enoug h. It could be due to the ; however, I carefully phrased the cooperative discussion question for the next lesson I planned to monitor this during the next session to see if their understanding improved b ased on the ir discussions.

PAGE 147

147 During the first session I gathered information from the teachers to help me understand their needs regarding their use of the technologies. I planned to use this information to help design the upcoming sessions. They also expr essed interest in some modeling and some hands on opportunities to help them understand the concepts with which they were unfamiliar. I decided it might be helpful to have centers set up so that they could practice the technical concepts that they relayed to me as being areas of concern. caused me to consider changing the way I presented the clicker strategy. I reflected on f the strategy without delivering the information myself. For the next session I gave the teachers a strip of paper with the clicker strategy and facilitated a cooperative activity so the teachers could develop and share ideas for using the strategy in th eir content area. This seemed to be a more effective way to help the teach ers comprehend and contemplate the information. Over the next few sessions, certain circumstances made me consider how I could modify the strategies to better facilitate the session To address some lack of discussion among certain groups I asked them to partner up with their content area colleagues for the discussion rather than with someone near them. This helped certain groups to communicate more during the cooperative activitie s. I also decided to set the digital timer for some cooperative activities so that each teacher would have an equal discussion on the mobile IWB, which I think gave value to their ideas.

PAGE 148

148 APPENDIX L EXAMPLE OF CODING QUALITATIVE DATA Figure L 1. Example of Coding Qualitative Data Using Hypertext

PAGE 149

149 LIST OF REFERENCES Alessi, S. M., & Trollip, S. R. (2001). Multimedia for learning: Methods and developme nt (3rd ed.). Boston, MA: Allyn & Bacon. Babbie, E. (2007). The practice of social research (11th ed.). Belmont, CA: Thomson Learning, Inc. Ball, D. L. (1996). Teacher learning and the mathematics reforms: What we think we know and what we need to learn. P hi Delta Kappan, 77 (7), 500 508. Barkley, S. G. (2010). Quality teaching in a culture of coaching (2nd ed.). Lanham, MD: Rowman & Littlefield Education. Beatty, I., Gerace, W., Leonard, W., & Dufresne, R. (2006). Designing effective questions for classroom response system teaching. American Journal of Physics 74 (1), 31 39. Beatty, I. D., & Gerace, W. J. (2009). Technology enhanced formative assessment: A research based pedagogy for teaching science with classroom response technology. Journal of Science Edu cation & Technology 18 (2), 146 162. Beile, P., & Boote, D. N. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34 ( 3), 3 15. Black, P., & William, D. (2010). Insid e the black box: Raising standards through classroom assessment. Phi Delta Kappan, 92 (1), 81 90. Bonwell, C. C., & Eison, J. A. (1991). Active learning: Creating excitement in the classroom. Washington, D. C.: George Washington University. Bradshaw, L. (20 02). Technology for teaching and learning: Strategies for staff development and follow up support. Journal of Technology and Teacher Education, 10 ( 1), 131 150. Brown, A., & Green, T. D. (2006). The essentials of instructional design: Connecting fundamental principles with process and practice. Upper Saddle River, NJ: Pearson Education, Inc. Bruff, D. (2009a). Teaching with classroom response systems: Creating active learning environments San Francisco, CA: Jossey Bass. Bruff, D. (2009b ). Vanderbilt Univers ity Center for Teaching. Retrieved October 1, 2009, from http://www.vanderbilt.edu/cft/resources/teaching_resources/technology/crs.htm

PAGE 150

150 Dana, N. F., & Yendol Hop pey, D. (2009). The reflective educator's guide to classroom research: Learning to teach and teaching to learn through practitioner inquiry (2nd ed.). Thousand Oaks, CA: Corwin Press. Danielson, C. (2007). Enhancing professional practice: A framework for t eaching (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. Denzin, N. (2006). Sociological methods: A sourcebook (5th ed.). Piscataway NJ: Aldine Transaction. Desimone, L. M., Porter, A. C., Garet M. S., Yoon, K. S., & Bir man, B. F. (2002). Effects of professional development on teacher instruction: Results from a three year longitudinal study. Educational Evaluation and Policy Analysis 24 (2), 81 112. Desimone, L. M. (2009). al development: Toward better conceptualizations and measures. Educational Researcher, 38 (3), 181 199. Dewey, J. (1938). Experience and education New York, NY: Kappa Delta Pi. Dick, W. O., Carey, L., & Carey, J. O. (2004). Systematic design of instruction (6th ed.). Boston, MA: Allyn & Bacon. Dillman, D. A. (2007). Mail and Internet surveys: The tailored design method (2nd ed.). Hoboken, NJ: John Wiley & Sons, Inc. Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an elect ronic voting system. Journal of Computer Assisted Learning 20 (2), 81 94. Firestone, W. A., Mangin, M. M., Martinez, M. C., & Polovsky, T. (2005). Content a nd coherence in district professional development: Three case studies. Educational Administration Qu arterly 41 (3) Retrieved September 11, 2010, from http://www.temple.edu/lss/pdf/publications/pubs2004 3.pdf Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate rese arch in education (6th ed.). Boston, MA: McGraw Hill. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74 (1), 59 109. Freedman, S. W. (2001). Tea cher research and professional development: Purposeful planning or serendipity. In A. Lieberman & L. Miller (Eds.), Teachers caught in the action: Professional development that matters (pp. 188 208). New York, NY: Teachers College Press.

PAGE 151

151 Fullan, M. (2006). Change theory: A force for school improvement. Centre for Strategic Education: Seminar Series Paper, 157, 1 15. Retrieved October 2, 2010, from http://www.michaelfullan.ca/Article s_06/06_change_theory.pdf Fuller, J. (2009). Teacher participation rates in technology based professional learning. Unpublished manuscript. Fuller, J. (2010). Professional learning for assisting middle school teachers with using student response systems effectively. Unpublished manuscript. Garet, M. S., Porter, A. C., Desimone, L. M., Birman, B., & Yoon, K. S. (2001). What makes professional development effective? Analysis of a national sample of teachers. American Educational Research Journal, 38 (3), 915 945. Garrison, C., & Ehringhaus, M. (2007). Formative and summative assessments in the classroom. Retrieved January 24, 2011, from http://www.nmsa.org/Publicat ions/WebExclusive/Assessment/tabid/1120/Default.a spx Georgia Department of Education. (2008). School keys: Unlocking excellence through the Georgia school standards Retrieved January 8, 2009, from http://www.doe.k12.ga.us/DMGetDocument.aspx/SCHOOL%20KEYS%20FINAL% 205 29 07.pdf?p=6CC6799F8C1371F6175E5B6E474BB7C617F852E1ADE57E7942B6D 677375D A861&Type=D Georgia Department of Education. (2010 a ). Class keys classroom analysis of state standards: The Georgia teacher evaluation system. Retrieved October 1, 2010, from http://www.doe.k12.ga.us/DMGetDocument.aspx/CK%20Standards%204 30 09.pdf?p=6CC6799F8C1371F60C8684DFDC96C1C9E173A927D7D04E1B1E862 FC762CCF7F9&Type=D Georgia Department of E ducation. (2010 b ). Report card Retrieved September 26, 2010, from http://www.gadoe.org/ReportingFW.aspx?PageReq=211&PID=61&PTID=212&Sta teId=ALL&T=0&FY =2009 Georgia Department of Education. (2011). Georgia performance s tandards Retrieved January 5, 2011, from https://www.georgiastandards.org Georgia State University. (2010). College of education Retri eved September 19, 2010, from http://education.gsu.edu/main/ Glaser, B. G. (1965). The constant comparative method of qualitative analysis. Social Problems, 12 (4), 436 445. Glesne, C. (2006). Becoming qualita tive researchers: An introduction (3rd ed.). Boston, MA: Pearson Education, I nc.

PAGE 152

152 Guskey, T. R. (1998). The age of our accountability: Evaluation must become an integral part of staff development. Journal of Staff Development 19 (4) 36 44. Guskey, T. R. (2 000). Evaluating professional development. Thousand Oaks, CA: Corwin Press. Guskey, T. R. (2002a). Does it make a difference? Evaluating Professional Development. Educational Leadership, 59 (6), 45 51. Guskey, T. R. (2002 b ). Professional development and tea cher change. Teachers & Teaching: Theory and Practice, 8 380 391. Guskey, T.R. (2003a). How classroom assessments improve learning. Educational Leadership, 60 (5), 6 11. Guskey, T. R. (2003 b ). What makes professional development effective? Phi Delta Kappan 84 (10), 748 750. Guskey, T. R. (2006). A conversation with Thomas R. Guskey The Evaluation Exchange, 11 (4). Retrieved October 1, 2010, from http://www.hfrp.org/evaluation/the evaluation exchange/issue archive/professional development/a conversation with thomas r. guskey Guskey, T. R. & Yoon, K. S. (2009). What works in professional development? Phi Delta Kap pan, 90 (7), 495 500. Hake, R. R. (1998). Interactive engagement versus traditional methods: A six thousand student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66 (1). Retrieved September 26, 2010, from http://web.mit.edu/rsi/www/2005/misc/minipaper/papers/Hake.pdf Hew, K. F. & Brush, T. (2007). Integrating technology into K 12 teaching and learning: Current knowledge gaps and re commendations for future research. Educational Technology Research & Development, 55 (3), 223 252. Hill, H. (2007). Learning in the teaching workforce. Excellence in the Classroom, 17 (1), 111 127. Retrieved July 5, 2010, from http://www.futureofchildren.org/futureofchildren/publications/journals/ Hill, H. (2009). Fixing teacher professional development. Phi Delta Kappan 90 (7), 470 476. Hirsh, S., & Killion, J. (2009). W hen educators learn, students learn. Phi Delta Kappan 90 ( 7), 464 469. Hooper, S., & Rieber, L. P. (1995). Teaching with technology. In A. C. Ornstein (Ed.), Teaching: Theory into practice (pp. 154 170). Needham Heights, MA: Allyn & Bacon.

PAGE 153

153 Hughes, J. E., & Ooms, A. (2004). Content focused technology inquiry groups: Preparing urban teachers to integrate technology to transform student learning. Journal of Research on Technology in Education, 36 (4), 397 411. Hur, J. W., & Brush, T. (2009). Teacher participati on in online communities: Why do teachers want to participate in self generated online communities of K 12 teachers? Journal of Research on Technology in Education 41 (3), 279 303. Joyce, B., & Showers, B. (1980). Improving inservice training: The messages of research. Educational Leadership 37 (5), 379 385. Katzenmeyer, M., & Moller, G. (2009). Awakening the sleeping giant: Helping teachers develop as leaders (3rd ed .). Thousand Oaks, CA: Corwin Press Killion, J. (2008). Journaling. In L. B. Easton (Ed.) Powerful designs for professional learning (2nd ed.). (pp. 163 169). Oxford, OH: National Staff Development Council. Kirkpatrick, D., Basarab, D., & Freitag, E. (1998). Evaluating training programs: The four levels (2nd ed.). San Francisco, CA: Berrett K oehler Publishers. Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to andragogy. Englewood Cliffs, NJ: Prentice Hall/Cambridge. Knowles, M. S., Holton, E. F., & Swanson, R. A. (1998). The adult learner Houston, TX: Gulf Publis hing. Lyman, F. (1981). The responsive classroom discussion. In A. S. Anderson (Ed.), Mainstreaming D igest (pp. 109 113) College Park, MD: University of Maryland College of Education. Mager, R. (1997). Goal analysis: How to clarify your goals so you can a ctually achieve them (3rd ed.). Atlanta, GA: Center for Effective Performance. Mangin, M. & Stoelinga, S. R. (2011). Peer? Expert? Teacher leaders struggle to gain trust while establishing their expertise. Journal of Staff Development 32 (3), 48 51. Matze n, N. J., & Edmunds, J. A. (2007). Technology as a catalyst for change: The role of professional development. Journal of Research on Technology in Education, 39 (4), 417 430. Mazur, E. (1997). Peer instruction: A manual. Upper Saddle River, NJ: Pren tice Hall. McTighe, J. & Emberger, M. (2008). Assessment as professional learning. In L. B. Easton (Ed.). Powerful designs for professional learning (2nd ed.). (pp. 71 83). Oxford, OH: Nation al Staff Development Council.

PAGE 154

154 Miller, M. (2009). Achieving a weal th of riches: Delivering on the promise of data to transform teaching and learning Washington, DC: Alliance for Excellent Education. Retrieved April 27, 2011, from http://www.all4ed.or g/files/AchievingWealthOfRiches.pdf Morrison, G., Ross, S., & Kemp, J. (2007). Designing effective instruction (5th ed.). Hoboken, NJ: John Wiley & Sons, Inc. Mouza, C. (2003). Learning to teach with new technology: Implications for professional developm ent. Journal of Research on Technology in Education, 35 (2), 272 289. National Staff Development Council. (2001). NSDC's Standards for Staff Development Retrieved March 16, 2009, from http://www.nsdc.o rg/standards/index.cfm North Carolina State Board of Education. (2007). Rubric for evaluating North Carolina teachers Retrieved January 24, 2011, from http://www.necollabo rative.org/docs/ncteacherevaluationrubric.pdf Orrill, C. (2001). Building technology based, learner centered classrooms: The evolution of a professional development model Educational Technology Research and Development, 49 (1), 15 34. Patton, M. Q. (1987 ). How to use qualitative methods in evaluation. Newbury Park, CA: Sage Publications, Inc. Penuel, W., Fishman, B. J., Gallagher, L. P., Korbak, C., Lopez Prado, B. (2008). The mediating role of coherence in curriculum implementation. Proceedings of the 8 th International Conference for the Learning Sciences. Utrecht, The Netherlands. 2, 180 187. Retrieved September 11, 2010, from http://www.fi.uu.nl/en/icls2008/195/paper195.pdf Reiser, R. A. & Dempsey, J. V. (2002). Trends and issues in instructional design and technology. Upper Saddle River, NJ: Pearson Education, Inc. Schnackenberg, H., Luik, K., Nisan, Y., & Servant, C. (2001). A case study of needs assessment in teacher in service develop ment. Educational Research & Evaluation 7 (2/3), 137 160. Sparks, D. (2002). Designing powerful professional development for teachers and principals. Retrieved March 16, 2009, from http://w ww.nsdc.org/news/sparksbook/sparksbook.pdf Spradley, J.P. (1979). The ethnographic interview. Belmont, California: Wadsworth Group.

PAGE 155

155 Stiggins, R., & Chappuis, J. ( 2005 ). Using student involved classroom assessment to close achievement gaps Theory Into Pr actice, 44 (1), 11 18. Retrieved July 18, 2011, from http://www.assessmentinst.com/wp content/uploads/2009/05/tip pub.pdf Wayne, A. J., Yoon, K. S., Zhu, P., Cronen, S., & Garet, M. S. (2008). Experimenting with teacher professional development: Motives and methods. Educational Researcher, 37 (8), 469 479. Wiggins, G., & McTighe, J. (1998). Understanding by design Alexandria, VA: Association for Supervision and Curriculum D evelopment. Williams, H., & Kingham, M. (2003). Infusion of technology into the curriculum. Journal of Instructional Psychology 30 (3), 178 184. Wilson, S. M., & Ball, D. L. (1996). Helping teachers meet the standards: New challenges for teacher educators [Special issue] The Elementary School Journal, 97 (2), 121 138. Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA: Sage Publications. Yendol Hoppey, D. & Dana, N. F. (2010). Powerful professi onal development: Building expertise within the four walls of your school. Thousand Oaks, CA: Corwin Press Yildirim, S. (2000). Effects of an educational computing course on preservice and inservice teachers: A discussion and analysis of attitudes and use Journal of Research on Computing in Education 32 (4), 479.

PAGE 156

156 BIOGRAPHICAL SKETCH Julia Susanne Fuller is a career educator, and her experiences include teaching at the elementary level and in a K 5 science lab setting. As district level I nstructional Technology Specialist she facilitates face to face and online trainings and works with teachers to integrate technology for improving student performance. Julia uses instructional design methods and adult learning theology to guide her effor ts with technology based professional development. In addition to teacher professional development, her research interests include blended learning environments, technologically innovative approaches, and data driven instruction. Julia is recognized as a n instructional technology leader in the state as she has been selected to speak at the Georgia Educational Technology Consortium conference yearly since 2006.