Teachers' Concerns Regarding the Implementation and Continued Use of an Evidence-Based Educational Practice

MISSING IMAGE

Material Information

Title:
Teachers' Concerns Regarding the Implementation and Continued Use of an Evidence-Based Educational Practice
Physical Description:
1 online resource (147 p.)
Language:
english
Creator:
Becker,Mallory Kelly
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Special Education, Special Education, School Psychology and Early Childhood Studies
Committee Chair:
Crockett, Jean
Committee Members:
Smith, Stephen W
Daunic, Ann
Clark, Mary Ann

Subjects

Subjects / Keywords:
adoption -- based -- concerns -- model -- teachers
Special Education, School Psychology and Early Childhood Studies -- Dissertations, Academic -- UF
Genre:
Special Education thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Teachers need practices that can be implemented in the classroom to teach children how to address conflict and solve problems proactively. There are curricula available for teachers to use in promoting improvement in the social behavior of students and for further enhancing their ability to self-regulate their behavior (Smith, Lochman, & Daunic, 2005). The focus of this study was two-fold. One focus was to identify teachers? concerns regarding the implementation of a social problem-solving curriculum for fourth and fifth grade students. The second focus was to examine how teachers who have been trained to implement a promising practice continue to use that practice over time. The present investigation used a mixed-method design using the Concerns-Based Adoption Model framework (Hall & Hord, 2006) to examine how teachers? concerns contribute to or impede the adoption and sustainability of an intervention. The Stages of Concern Questionnaire was used to identify teachers? concerns with implementing the intervention; interviews were conducted to determine the levels at which they continued to use the intervention; and an Innovation Configuration Map was developed to assess fidelity of implementation and guide observations. The findings suggest that teachers were unconcerned about implementing the innovation and most were no longer using it. Although the teachers? concerns were unrelated to current use, their responses indicate they chose to discontinue the practice primarily because of personal concerns and management issues. The results of this study suggest that sustainability of a promising practice can be influenced by the concerns of those prepared to use it. Those looking to establish the implementation and sustainability of a promising or proven practice first need to provide clear demonstration of how its use can be responsive to a range of teachers? concerns that could be unrelated to the practice such as teacher?s sense of self, the feasibility of the task, or the impact of the innovation on student outcomes. Addressing the individual concerns of teachers could likely lead to a more sustained use of the desirable practice, especially if they are asked about their concerns with implementation early and often in the process.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Mallory Kelly Becker.
Thesis:
Thesis (Ph.D.)--University of Florida, 2011.
Local:
Adviser: Crockett, Jean.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0042880:00001


This item is only available as the following downloads:


Full Text

PAGE 1

1 USE OF AN EVIDENCE BASED EDUCATIONAL PRACTICE By MALLORY K. BECKER A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FUL FILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2011

PAGE 2

2 2011 Mallory K. Becker

PAGE 3

3 To my husband, the most amazing man I have ever met, my incredible children, my loving family, and the graduate faculty who have supported me throughout the past five years. I could not have accomplished this goal without you.

PAGE 4

4 ACKNOWLEDGMENTS I would like to acknowledge several people for their help throughout my educational career. First and foremost, I would like to tha nk my graduate advisor, Dr. Jean Crockett, for her endless support and facilitation of my professional development. From the first year in my graduate program, she encouraged me to engage in additional research interests above and beyond the requirements o f the program, and greatly assisted me from start to finish on multiple projects. Additionally, she always reminded me to reward myself for hard work by enjoying what life has to offer. I am also grateful to have such an incredible doctoral committee and t o have worked with several inspiring faculty. I would like to thank Drs. Stephen Smith and Ann Daunic for helping me further develop my interest in students with emotional and behavioral difficulties, and for consistently motivating and reinforcing me. In addition, I would like to thank them for offering me four years of employment on the most wonderful graduate assistantship a student could ask for. They helped enhance my research and manuscript writing skills, and thank you for providing some of the data for this study. Finally, I would like to thank my loving and encouraging family. I would not have been able to achieve this milestone without your endless support. I am especially lucky to have my husband and mother as such a wonderful support system. Tha nk you to my children, Rori and Garrison, for being understanding throughout this entire process.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURES ................................ ................................ ................................ .......... 9 ABSTRACT ................................ ................................ ................................ ................... 10 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 12 Introduction to the Study ................................ ................................ ......................... 12 Statement of the Problem ................................ ................................ ....................... 12 Changing Teac ................................ ................................ .......... 13 ................................ ................................ ........... 14 Implementing Problem Solving Interventions ................................ ................... 15 Moving Toward the Use of Effective Practices ................................ ................. 17 Purpose of the Study ................................ ................................ .............................. 18 Overview of Methods ................................ ................................ .............................. 19 Definitions of Key Terms ................................ ................................ ......................... 20 Delimitations and Limitations ................................ ................................ .................. 21 2 LIT ERATURE REVIEW ................................ ................................ .......................... 23 Evidence Based Instructional Practice ................................ ................................ .... 2 4 Implementing and Sustaining the Use of an Evidence Based Instructiona l Practice ................................ ................................ ................................ ................ 27 Overview ................................ ................................ ................................ .......... 27 Examining Implementation ................................ ................................ ............... 28 Summary of the implementation of evidence based practices ......................... 31 What Should Teachers Know about Implementing Evidence Based Practices? ................................ ................................ ................................ ..... 32 Con ceptual Models for Linking Research to Practice ................................ ....... 33 Based Practices? ......................... 36 Cognitive Behavio ral Intervention ................................ ................................ ........... 37 Managing Conflicts Effectively ................................ ................................ ................ 38 Problem Solving Cognitive Behavioral Interventions ................................ ........ 40 Role of Adults in Cognitive Behavioral Interventions ................................ ........ 41 Implications of CBIs ................................ ................................ .......................... 43 Conc erns Based Adoption Model ................................ ................................ ........... 45 Tools of CBAM ................................ ................................ ................................ 46 Utilizing CBAM ................................ ................................ ................................ 49 Discussion of the Literature ................................ ................................ .................... 51

PAGE 6

6 3 METHODOLOGY ................................ ................................ ................................ ... 58 Overview of the Methods ................................ ................................ ........................ 58 Building on Previous Research ................................ ................................ ............... 58 The Perspective of the Present Study ................................ ................................ .... 61 Case Study Methods ................................ ................................ .............................. 62 ................................ ................................ ............................ 63 Site of the Study ................................ ................................ ............................... 64 Participants ................................ ................................ ................................ ....... 64 Accessible Population Demographic ................................ ................................ ...... 65 Instrumentation ................................ ................................ ................................ ....... 66 Stages of Concern (SoC) (Questionna ire) ................................ ........................ 66 Levels of Use (LoU) (Interview) ................................ ................................ ........ 66 Innovation Configuration Map (IC) (Observations) ................................ ........... 67 Data Collection ................................ ................................ ................................ ....... 67 Questionnaire Procedures ................................ ................................ ................ 68 Interview Procedures and Protocols ................................ ................................ 69 Direct Observation Procedures ................................ ................................ ........ 70 Data Analysis ................................ ................................ ................................ .......... 71 Assurances of Quality ................................ ................................ ............................. 73 Issues of Entry and Ethics ................................ ................................ ................ 73 Assurance of Confidentiality ................................ ................................ ............. 73 Ass essing the Cultural Context ................................ ................................ ........ 73 Addressing Quality Indicators ................................ ................................ ........... 74 Credibility ................................ ................................ ................................ ... 74 Transferability ................................ ................................ ............................ 74 Dependability ................................ ................................ ............................. 74 4 RESULTS ................................ ................................ ................................ ............... 80 Findings ................................ ................................ ................................ .................. 80 and Promising Practice? ................................ ................................ ...................... 81 Unrelated ................................ ................................ ................................ .......... 83 Unconcerned Stage. ................................ ................................ ......................... 83 Self ................................ ................................ ................................ ................... 84 Informational stage. ................................ ................................ ................... 84 Unconcerned, informational, and personal stages. ................................ .... 84 Personal stage. ................................ ................................ .......................... 85 Task ................................ ................................ ................................ ................. 85 Unconcerned and management stages. ................................ .................... 86 Management stage. ................................ ................................ ................... 87 I mpact ................................ ................................ ................................ ............... 87 Question Two: How do These Concerns Influence the Level to Which Teachers Report Implementing the Practice? ................................ ................................ ..... 88 Levels o f Use Interview ................................ ................................ .................... 89 NonUsers ................................ ................................ ................................ ......... 89

PAGE 7

7 Level of Use 0: Nonuse. ................................ ................................ ............. 90 Level of Use I: Orientation. ................................ ................................ ......... 91 Level of Use II: Preparation. ................................ ................................ ....... 92 Users ................................ ................................ ................................ ................ 93 Level of Use IVA: Routine. ................................ ................................ ............... 94 Patterns of Practice ................................ ................................ .......................... 94 Effectiveness. ................................ ................................ ............................. 95 Feasibility. ................................ ................................ ................................ .. 96 Outside Requirements. ................................ ................................ .............. 97 Partial Use. ................................ ................................ ................................ 98 Question Three: How Faithfully do Teachers Sustain These Practices? ................ 99 Discussion of the Findings ................................ ................................ .................... 102 Finding # 1: Teachers were unconcerned about implementing this innovative and promising practice. ................................ .............................. 102 Lack of Implementation Support ................................ .............................. 103 Questioning the Feasibility of Implementation ................................ .......... 105 report implementing the practice. ................................ ................................ 106 Finding # 3: The teacher who sustained this practice did so with a high level of fidelity. ................................ ................................ ................................ ..... 107 Summary ................................ ................................ ................................ .............. 108 5 CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH ........ 119 Limitations of the Study ................................ ................................ ......................... 120 Conclusions ................................ ................................ ................................ .......... 121 The Importance of People in the Change Process ................................ ......... 121 The Necessity of Professional Development ................................ .................. 122 Implications for Practice ................................ ................................ ........................ 123 Recommendations for Future Research ................................ ............................... 126 APPENDIX A STAGES OF CONCERN QUESTIONNAIRE ................................ ........................ 128 B LEVELS OF USE BRANCHING INTERVIEW FORMAT ................................ ....... 132 C SOCQ QUICK SCORING DEVICE ................................ ................................ ....... 134 D INNOVATION CONFIGURATION MAP TOOLS FOR GETTING ALONG CURRICULUM ................................ ................................ ................................ ...... 137 LIST OF REFERENCES ................................ ................................ ............................. 140 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 147

PAGE 8

8 LIST OF TABLES Table page 3 1 Number of Elementary Schools Across Five Districts in Follow Up Study .......... 76 3 2 Florida Comprehensive Assessment Test (FCAT) 2009 Scores for Five Districts in Follow Up Study ................................ ................................ ................ 77 3 3 Student Ethnic and Gender Information across the Five Districts ....................... 78 3 4 Percentage of Students Receiving Free or Reduced Price Lunch in the Five Districts in the Follow Up Study ................................ ................................ .......... 78 3 5 Participant demographic information gathered during effectiveness study ......... 79 4 1 ................................ ................................ ................... 118 4 2 g observations ................................ ................. 118

PAGE 9

9 LIST OF FIGURES Figure page 2 1 Illustration of the Phases of Implementation framework (Fixsen, Naoom, Blas, Friedman, & Wallac e, 2005) ................................ ................................ .... 54 2 2 Factors that support sustained use of research based practices (Gersten, Vaughn, Deshler, & Schiller, 1997) ................................ ................................ ..... 55 2 3 Stages of Concern (Hall & Hord, 2006) ................................ .............................. 56 2 4 ..... 57 2 5 Levels of Use (Hall & Hord, 2006) ................................ ................................ ...... 57 4 1 Percentages of Concern by stages for all respondents ................................ .... 110 4 2 le ................................ ................................ ............. 111 4 3 ................................ ................................ ............. 112 4 4 ................................ ................................ ............. 113 4 5 ................................ ................................ ............. 114 4 6 ................................ ................................ ............. 115 4 7 ................................ ................................ ............. 116 4 8 ................................ ................................ ............ 117

PAGE 10

10 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy USE OF AN EVIDENCE BASED EDUCATIONAL PRACTICE By Mallory K. Becker August 2011 Chair: Jean Crockett Major: Special Education Teac hers need practices that can be implemented in the classroom to teach children how to address conflict and solve problems proactively. There are curricula available for teachers to use in promoting improvement in the social behavior of students and for fur ther enhancing their ability to self regulate their behavior (Smith, Lochman, & Daunic, 2005). The focus of this study was two fold. One focus was to solving curriculum for fourth and fifth grade students. The second focus was to examine how teachers who have been trained to implement a promising practice continue to use that practice over time. The present investigation used a mixed method design using the Concerns Based Adoption Model framework (Hall & Hord, 2006) to examine how concerns with implementing the intervention ; interviews were conducted to determine the levels at which they continued to use the intervention; and an Innovation Configuration Map was developed to assess fidelity of implementation and guide observations.

PAGE 11

11 The findings suggest that teachers were unco ncerned about implementing the unrelated to current use, their responses indicate they chose to discontinue the practice primarily because of personal concerns and management issues. The results of this study suggest that sustainability of a promising practice can be influenced by the concerns of those prepared to use it. Those looking to establish the implementation and sustainability of a promising or proven practice first n eed to provide clear demonstration the impact of the innovation on student outcomes. Addressing the individual concerns of teachers could likely lead to a more sustained use of the desirable practice, especially if they are asked about their concerns with implementation early and often in the process.

PAGE 12

12 CHAPTER 1 INTRODUCTION Introduction to the Study Effective and efficient e vidence based practices shown to improv e academic behavior al, and social skills are becoming increasingly available ; however, encouraging teachers to use these practices, particularly with fidelity and conv iction, has not been an easy task (Cook, B., Landrum, Cook, & Tankersley, 2008; Tankersley, Harjusola Webb, & Landrum, 2008) Evidence based practice is a term that is often used in the literature without a clear definition. Odom et al. (2005) advanced th e term evidence based practice to refer to teaching practices that have been shown to be based practices are established by systematic reviews of research, ensuring that a practice is deemed evidence based only whe n it is supported by multiple high quality research studies designed to determine whether a 2008, p. 67). Some practices have a proven record but have yet to be replicated e nough to be considered evidence based practices, while some practices are supported with promising yet preliminary data of their effectiveness. Statement of the Problem based practices has become a major foc us of federal education policies including the Individuals with Disabilities Educational Improvement Act (IDEA, 2004), and a priority for funding agencies including the National Science Foundation. The effectiveness of proven or promising practices is docu mented in peer efforts to implement them (e.g., Cook, B., Landrum et al., 2008; Tankersley et al., 2008).

PAGE 13

13 Once the support of research teams is removed, however, teachers often fail to impleme nt the practices faithfully or for an appropriate length of time (Cook, Tankersley, & Harjusola Webb, 2008). As a result, teachers may miss the opportunity to recognize se practices are unfulfilled. Although the adoption and use of evidence based practices is a national priority, continue (or discontinue) their use, especially once exte rnal support is no longer available (Cook, B., Cook, Landrum, & Tankersley, 2008). Q uestions remain as to why teachers discontinue their use, or why they continue to implement the practice, but not with fidelity. An additional question is whether teachers are more inclined to ward practices that are not evidence based but easier to use Implementing promising and proven practices supported by research is complex and the process of changing teaching practices must be addresse d Im plementing change at the school and classroom levels is a major undertaking for any educator. For administrators, the usefulness and feasibility of implementing particular curricula must be considered (Carnine, 1997). E thical constraints, cultural issues, and the effectiveness of the practice are also important (Fixsen, Naoom, Blas, Friedman, & Wallace, 2005) major gaps exist between what is known as effective practices (i.e., theory and science) and w hat is actually done (i.e., policy and p. 2). Change is often difficult and may involve loss, anxiety, and struggle (Fullan, 2005). Fullan argue d that regardless of whether change is imposed on schools and teachers, or if it is v oluntary,

PAGE 14

14 initial disco rd is often the norm. I f care is taken to incorporate change in a systematic fashion and in advance, change can lead to a sense of mastery, increased self esteem, and professional growth by all involved (Fullan). Change, however, is not something that can happen quickly or on demand. Unfortunately due to high stakes accountability where teachers are held accountable for student academic growth, this is exactly what is expected. The practice might be abandoned for yet another practic e if no immediate results are seen (Fixsen et al.). U nderstanding why, when, and how teachers adopt and sustain evidence based practices requires a breakdown of complex components of the intervention including understanding the attributes of the teacher a nd the school ( Vaughn, Kling n er, & Hughes 2000). For example, it is important to understand the motives of individual teachers to sustain practices (e.g., increased pay and improved classroom behavior) ( Vaughn et al.). Comprehensive Assessment Test (FCAT) and receiving additional money allotted from the school district could be garding the implementation of the new promising or proven practice and how to provide professional development to address these individual concerns so both the teacher and student will be successful. In the current climate of accountability, academic learning is emphasized but teachers also have to help students learn and hone social skills as an important secondary goal. It is imperative that s tudents acquire positive problem solving skills to be successful in the classroom a nd in society I f students are unable to resolve interpersonal conflicts in appropriate ways t eachers may spend more time dealing with

PAGE 15

15 conflicts and less time on academics. As teachers are increasingly held accountable for avioral interventions are becoming more necessary to keep the students on task and orderly. As teachers experience disruptive student behavior with increasing frequency and severity, they report that their classrooms are increasingly more difficult to mana important for them to recognize that problem solving interventions are vital to the learning environment. One category of evidence based practices available to teachers that incorporates this approach is cognitive behavioral interventions. When implemented with fidelity, cognitive behavioral interventions have been shown to improve social and academic outcomes for students (Johnson, D., Johnson, Dudley, & Acikgoz, 2001). An important question for educatio nal research is: Why are these evidence based practices (i.e. cognitive behavioral interventions) not being implemented more broadly and faithfully sustained? This topic is accountability. Probl em solving interventions, using effective practices, and sustaining the use of these evidence based practices are three areas of interest. Implementing Problem Solving Interventions To help reduce negative conflicts in the classroom between students or bet ween students and teachers, problem solving training has become a necessary part of the school curriculum. Greater awareness of problems like bullying and discrimination by school leaders has helped make schools safer (Sugai & Horner, 2002). There are now programs available to schools that help students negotiate the issues of conflict resolution, problem solving, and social skills training, which ultimately could help protect students from aggression and confli cts. S chool administrators also need to remain

PAGE 16

16 vigilant and take precautionary measures to prevent aggressive acts on school campuses. For example, data from the Centers for Disease Control and Prevention (2006) indicated that in 2004 an estimated 30% of sixth to tenth graders in the United States wer e involved in bullying either as a bully, a target of bullying, or both. The results of this nationwide survey of high school students indicated that about 6% reported not going to school on one or more days in the 30 days preceding the survey because they felt unsafe at school or on their way to and from school. Some responsibilit y lies with school personnel for teaching students to become more socially acceptable and for assisting them in acquiring the skills necessary to deal with conflicts as they ari se. The most comprehensive study of conflict management engaged in by children was conducted by DeCecco and Richards (1974), who noted that o ver 90% of the conflicts reported by students were perceived to be unresolved or to be resolved in destructive ways DeCecco and Richards concluded that within schools, individuals are trying either to shun conflict or to crush the opposition. Conflicts play a particularly important role for adolescents. Not only do conflicts pervade offer opportunities for such positive outcomes as clarifying personal identity and values, increasing social status, promoting personal growth, generating interpersonal insight, and creating heroic drama (Opotow, 1989). Most of the time, however, potential positive outcomes are never realized because adolescents tend to manage their conflicts in destructive ways. In recent years, researchers have demonstrated that cognitive behavioral problem solving strategies can decrease student misbehavior and incre ase their pro social behavior (Daunic, Smith, Brank, & Penfield, 2006). Due to the increased pressure

PAGE 17

17 placed on schools to improve academ ic achievement there has been a push to teach students to be better problem solvers and to be able to self regulate. I f students are not conf lict resolution and less on academics. Therefore, implementing preventive interventions such as cognitive behavioral problem solving strategies cou ld be a way to enhance the academic performance of students, especially those at risk for academic failure and conduct problems. Moving Toward the Use of Effective Practices Implementation is another factor that affects the sustainability of effective prac tices. Implementation is defined as a specified set of activities or strategies designed to put into practice a program of known proportions (Fixsen et al., 2005). O nly when effective practices are implemented with full fidelity can positive outcomes be ex pected Sustainability of the intervention once the researchers have extracted their support continues to be a concern for researchers and administrators alike and is seen as a separate concern from implementation. Implementing a promising or proven prac tice often brings up concerns for teachers. Teachers must be open to change and be willing to address their concerns and how these concerns affect engagement in using an innovation. How teachers feel about and perceive change will be one determinant of whe ther or not change actually occurs in classrooms (Hall & Hord, 1987). Researchers and administrators who are concerns with the teachers proactively to ensure implementation of the innovation with fidelity.

PAGE 18

18 Purpose of the Study The purpose of this study was to examine how teachers who have been trained to implement a promising practice continue to use that practice over time. This study of sustainability was a follow up inves tigation to a previous research project funded by the Institute of Education Sciences. Specifically, the follow up study explores these questions: 1) practice? 2) How do these concerns influence the level to which teachers report implementing the practice? 3) How faithfully do teachers sustain these practices? This study was designed to complement the original project conducted in several elementary schools in a large southeastern state (S mith & Daunic, 2006). The initial investigation determined whether a universally applied, classroom based, problem solving curriculum (Tools For Getting Along) was effective in reducing disruptive/aggressive behavior among fourth and fifth grade students, particularly those with high behavioral risk profiles. The curriculum, Tools For Getting Along is a cognitive behavioral intervention (CBI) that incorporates modeling, feedback, students manage frustrating or anger provoking situations that often lead to aggression (Daunic et al., 2006). use of the TFGA intervention, those teachers who parti cipated in the effectiveness research project were asked to participate in the follow

PAGE 19

19 is characterized by the increasing use of scientific methods and standardized measures for determining efficacy. This follow up study is relevant because it links scientifically based practices to their useability by practitioners and identifies which factors aide or inhibit the sustainability of a promisin g practice. Overview of Methods The design was a mixed method explanatory case study using the Concerns Based Adoption Model (CBAM) framework (Hall & Hord, 2006). All teachers who participated in the treatment group of the effectiveness study, and who wer e taught to implement the intervention over the course of a school year, were asked to participate in this follow up study. An explanatory case study was an appropriate method for this t a promising practice. In addition, the case study approach is preferred in examining contemporary events, but when the relevant behaviors cannot be manipulated (Yin, 2003). According to Yin, case studies add two sources of evidence, which are direct obse rvation of the events being studied and interviews of the persons involved in the events. These are two components of the CBAM framework used in this analysis. The CBAM framework was created to assist those involved in the change process and to identify th e needs of the individual so that change facilitation can be personalized (Hall & Hord, 2006). People change at different rates and for a variety of reasons. The CBAM model accounts for these differences. The CBAM framework includes three components that w ere used in conducting this investigation: (a) The Stages of Concern Questionnaire which was mailed to all potential participants to ascertain their concerns regarding the implementation and sustainability of TFGA; (b) The Levels of Use Interview which w as used with teachers from all stages of concern;

PAGE 20

20 and (c) observations of the self reported high users of the practice using an Innovation Configuration Map which was developed with researchers from the effectiveness study and used to assess fidelity of u se of Tools For Getting Along Definitions of Key Terms For the purposes of this follow up study, key terms are defined in the following ways: A CCESSIBILITY Accessibility practitioners can obtain research findings and extract the necessary information related to a certain goal. If it is difficult and time consuming for knowledge consumers to locate C OGNITIVE PROBLEM SO LVING useful responses to a challenging situation, and its use increases the likelihood of (Smith & Daunic, 2006, p. 1) Therefore, cognitive behavioral problem solving traini ng would entail teaching others how to solve problems as they arise and help implement problem solving strategies, which are specific steps that can lead to problem solving in an appropriate way. Cognitive strategies are viewed as techniques that help stud ents decrease destructive behaviors and increase socially appropriate behaviors (Daunic et al., 2006). E VIDENCE BASED PRACTICE Refers to teaching practices that have been shown to based p ractices are established by systematic reviews of research, ensuring that a practice is deemed evidence based only when it is supported by multiple high quality research studies designed to determine whether a practice causes meaningful changes in student I MPLEMENTATION An explicit group of activities designed to put into practice an activity or program of known scope (Fixsen et al., 2005). P ROMISING PRACTICES Some practices have a proven record but ha ve yet to be replicated enough to be considered evidence based practices, while some practices are supported with promising yet preliminary data of their effectiveness (Cook, B., Landrum et al., 2008). S USTAINABILITY The extent to which a practice is ado pted and used over time ( Vaughn, Kling n er, & Hughes 2004) T RUSTWORTHINESS Trustworthiness determines the confidence with which a given set of findings can be acted upon by knowledge consumers (Carnine, 1995a). U SEABILITY T he practicality of the res earch based practices for those who attempt to put them into practice (Carnine, 1995a) Useability also addresses the

PAGE 21

21 likelihood that those who have the responsibility for making decisions that impact students, and those who actually teach or provide servi ces to them, will do so Furthermore, research findings must be clearly written and address topics of importance to practitioners in order to be deemed useable. Delimitations and Limitations The present study is explanatory and descriptive. It is intende d to identify cognitive behavioral intervention implemented in fourth and fifth grade general education classrooms. Due to the limited application of the intervention in these g rade levels, generalizations of any findings should not be attributed to teachers of younger or older students. In addition, the results of this study are based on a small number of elementary school teachers in general education classrooms. Further resear ch is needed to investigate the sustainability of such a curriculum in a variety of settings such as self contained classrooms for students with disabilities, and other alternative placements. Factors identified in this study as improving the sustainabilit y of Tools For Getting Along should not be viewed as confirmation of the only factors that are related to the sustainability of cognitive behavioral interventions. Further research is needed to investigate identified external factors across a greater range of teachers at varying age levels and settings. A key limitation of this follow up study includes a small sample size due to the small number of teachers that sustained the problem solving intervention after the effectiveness research study concluded. Including teachers at every stage of concern in the levels of use interview process addressed this limitation. Given the information gained from the interviews and observations of their classroom practice, a more in

PAGE 22

22 depth knowledge was ascertained as to bo th how and why teachers sustain an evidence based practice.

PAGE 23

23 CHAPTER 2 LITERATURE REVIEW Evidence based practices are necessary for educators to use in teaching children to address conflict and solve problems proactively. Cognitive behavi oral interventions that are promising or proven practices represent feasible evidence based strategies for promoting improvement in the social behavior of students (Smith et al., 2005). The present study builds on an earlier investigation to determine the effectiveness of a promising practice over a four year period. Of specific interest are a promising instructional practice and how teachers, who have been trained to implement this promising practice, con tinue to use it over time. The intent of this chapter is to review the literature written over the past 20 years on evidence based instructional practices, cognitive behavioral interventions in schools, and the Concerns Based Adoption Model (CBAM). First a discussion about evidence sustainability of such practices is provided. A conceptual model linking research to evidence based instructional practices. The second component of this review addresses the use of cognitive behavioral interventions. The foundations of cognitive behavioral theory are reviewed along with empirical studies supporting the effectiveness of in terventions grounded in this approach. The final component of this review is focused on the theoretical rationale and practical application of the Concerns Based Adoption Model. In conducting this literature re view a systematic search was undertaken of r elevant electronic databases from 1970 present, although more recent publications

PAGE 24

24 were targeted. The databases searched were Academic Search Premier (EBSCOhost) and Expanded Academic ASAP (Gale) accessed through the University of Florida library system. An ancestral search was conducted from the articles retrieved electronically. Several references found through the ancestral search process proved to be relevant and applicable to the present study. The following keywords were used in various combinations: evidence based practices, cognitive behavioral intervention, conflict resolution, problem solving, school s education, implementation, sustainability, research to practice gap, Concerns Based and e lementary schools. The combination of evidence based practices, cognitive behavioral intervention and elementary schools was the most useful and numerous resources were identified this way. Evidence Based Instructional Practice Evidence based practice is a term that is often used without a clear definition. The phrase evidence based has made its way into the field of education via the field of medicine. Evidence based medicine and the first comprehensive description of its tenets appeared almost two decad es ago (Guyatt, Cook, & Haynes, 2004). Odom et al. (2005) advanced the term evidence based practice in the field of education to refer to teaching practices that have been shown to be effective by credible research. Strain and Dunlap (2006) noted that evid ence based practices are established by systematic reviews of research, ensuring that a practice is deemed evidence based only when it is supported by multiple high quality

PAGE 25

25 research studies designed to determine whether a practice causes meaningful changes Recent legislation has made it clear to teachers that the use of eviden ce based practices is important in both general and special education. For example, the No Child classroom (NCLB, 2002). In addition, the Individuals with Disabilities Education Improvement Act (IDEA, 2004) acknowledges the need for teachers to be trained in scientifically based (i.e., evidence based) instructional practices to improve the academic and functional performance of students with disabilities. The importance of utiliz ing evidence based practices in both general and special education is reflected in these pieces of legislation, which could guide how teachers and administrators determine which curricula to implement in their classrooms. Criteria and standards for dete rmining if a practice is evidence based have been developed in fields such as school psychology (Kratochwill & Stoiber, 2000) and medicine (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996). In the field of education, researchers have begun to establis h criteria for determining if an instructional practice is evidence based (Cook, B. et al., 2008). First, there must be high quality, experimental research supporting the effectiveness of the practice. The experimental design must include adequate control s and the results must demonstrate that the instructional practice improved student outcomes. Second, there must be sufficient quantities of high quality research demonstrating the effectiveness of the practice. The type of research design used to study an instructional practice determines the standard for how many high quality studies are necessary to provide sufficient

PAGE 26

26 evidence of its effectiveness. For example, Gersten et al. (2005) suggested that for group experimental designs, two high quality studies are necessary to consider a practice evidence based. For single subject designs, however, Horner et al. (2005) argued that the practice must be supported by a minimum of five high quality studies. In addition, Horner et al. argued for a minimum of three different independent groups of researchers in several different locations (i.e., schools and settings) and at least 20 participants. Although these criteria are not all inclusive, they provide a blueprint for researchers and practitioners to follow when e valuating evidence based practices. Once researchers have determined a practice as evidence based, the next step is to identify and address concerns regarding the implementation and sustainability of these practices. Scholarship over the past four decade s supports specific practices and principles linked to helping teachers sustain the use of evidence based practices. The body of research on sustainability makes significant contributions by providing researchers, policymakers, and school administrators wi th ways to help teachers enhance their repertoires with research validated practices (Gersten, Chard, & Baker, 2000). One principle with a record of projecting success and long term sustainability is active commitment by district and school leaders to the use of promising or proven practices (Gersten et al.). A second principle is that sustained use of an instructional practice is directly related to practices teachers see as actually helping difficult to teach students (Berman & McLaughlin, 1976). Kinder, Gersten, and Kelly (1989) noted that practice mastery was an essential component for sustained use of an evidence based practice. Knowing this information could be a starting point for improving professional

PAGE 27

27 development efforts to enhance classroom instruc tion by implementing evidence based practices and sustaining these practices rather than having them fade out only to be replaced by yet another practice. Implementing and Sustaining the Use of an Evidence Based Instructional Practice Overview Within the c ontext of evidence based instruction, implementation refers to an explicit group of activities necessary to put into practice an activity or program of known scope (Fixsen et al., 2005). Implementation processes must be purposeful and described in sufficie nt detail such that independent observers can detect the presence The purposes and outcomes of implementation across numerous fields can be categorized in three ways ( Goggin, 1986). First, paper implementation pertains to putting into place new policies and procedures with the adoption of an evidence based practice as the rationale for the policies and procedures. Paper implementation is prevalent when outside groups ar e monitoring compliance (e.g., for accreditation) and much of the monitoring focuses on the paper trail (Goggin). It appears that this form of strategies. Second process i mplementation pertains to putting new operating procedures in place to conduct training workshops and provide supervision with the adoption of an evidence based practice as the rationale for the procedures. However, process implementation is not necessaril y productive. Not much of what is taking place is related to the new practice; teachers are learning about the evidence based practice and how to implement it, but that is where the process ends. Third, performance implementation pertains to putting proced ures and processes in place in such a way

PAGE 28

28 that the identified functional components of change are used with good effect for teachers. Performance implementation is what researchers strive for when teachers are implementing evidence based practices in the c lassroom. Examining Implementation For teachers to implement evidence based instructional practices, several factors must be considered. For example, teachers should be informed about and motivated to implement the evidence based practice and prepar e the learning environment for implementation. Adelman and Taylor (2003) identified some early stages of preparation for adopting evidence based practices in an educational setting. These stages include the following : (a) develop an understanding of the local big picture context for all relevant interventions; develop an understanding of the current status of efforts; delineate how the innovation can contribute with respect to the larger agenda; articulate cost effective strategies; (b) mobilize interest, conse nsus, and support among key stakeholders; identify champions and other individuals who are committed to the innovation; plan and implement a social marketing strategy to mobilize a critical mass of support; plan and implement strategies to obtain support o f key policymakers; and (c) clarify feasibility; clarify how the functions can be institutionalized through existing, modified, or new infrastructure and operational mechanisms; clarify how necessary changes can be accomplished; formulate a long range stra tegic plan. (p. 12) Although Adelman and Taylor (2003) were most concerned with preparing school wide contexts for implementation, m aking sure th at a classroom environment is conducive for implementation of an evidence based practice is an important factor that researchers need to consider. As previously mentioned, this includes school administrator support and teacher willingness to change existing routines. Teachers must be open to change and well prepared to implement evidence based practices; however, access alone to cognitive behavioral interventions through

PAGE 29

29 in service training sessions is inadequate for changing existing patterns of teaching ( Bickel & Hattrup 1995) Ongoing, interactive forms of professional development have shown to be most effectiv e, and least effective are practices with minimal ongoing professional engagement (Gersten, Vaughn, Deshler, & Schiller, 1997). The Dissemination Working Group (1999) identified effective implementation as having some common elements: (1) Clear philosophy beliefs, and values that provide guidance, are fully integrated, an d promote consistency; (2) Specific treatment components that promote consistency of implementation; (3) Treatment decision making with built in accountability systems; (4) Structured s ervice delivery components; (5) Continuous improvement components that ascertain if a program is beneficial over a period of time. These elements, some of which reflect concerns about implementing new practices, pertain to professional development for teachers addressing effective behavioral interventions. For example, t o implement behavioral strategies teachers must understand principles of human behavior. However, teachers are frequently ill prepared in this regard (Gerber & Solari, 2005). To implemen t relatively sophisticated evidence based behavioral practices, school administrators must be willing to invest resources to promote, sustain, and increase teaching skills for all teachers. Arguing that implementation is a process and not an event, Fixsen et al. (2005) developed a framework for the process of implementation (Figure 2 1 ) that could be used in schools to support investments in on going professional development Phase 1:Exploration and adoption In this first phase, r esearchers address the nee ds of the school, teachers, and students ( Fixsen et al., 2005). The evidence based

PAGE 30

30 practice is matched to a school site where the needs addressed in the practice match the needs of a particular school or classroom. At the end of the adoption phase, a n acti on plan should be developed with clear tasks and timelines A ll stakeholders, such as school administrators and teachers, must be in agreement to facilitate the installation and initial implementation of the evidence based practice. Phase 2:Installation o f the intervention At this phase, it is important to determine if the new practice is implemented school wide or at the classroom level. During this phase, structural supports necessary to initiate the practice are put into place. This includes acquiring staff (i.e., teachers and guidance counselors) that will not only implement the practice but also those who will be used as support staff. School wide, t his may also include a host of school staff (i.e., bus driver, cafeteria worker, media teacher) that st udents encounter throughout the school day that w ould support the practice by using the same vocabulary and procedures that the student learn s during the practice. Phase 3:Initial implementation At this point in the process, participating school staff is provided with professional development that is both meaningful and productive. Researchers will need to assist teachers in moving beyond their fears of change to embrace the evidence based practice with their full commitment. Teachers must be made aware t hat change does not occur simultaneously or evenly in all parts of a practice or an organization. Therefore, they need to learn how to assess student learning quantitatively and not just rely on observable student data (Fixsen et al., 2005). As in other su bject areas, not all students learn at the same rate. Therefore, teachers

PAGE 31

31 need to use a variety of data to ascertain what each student has learned and where they still need to focus. Phase 4:Full operation The phase occur s once the new learning becomes i ntegrated into practitioner, organizational, and community practices, policies, and procedures. During this phase, teachers become more skillful with the practice and the procedures and processes of the evidence based practice become routinized. The effec tiveness of the evidence based practice should become evident once teachers are implementing the practice with fidelity. During the full operation phase, researchers and teachers are able to identify practices or procedures that need to be modified to inv oke a positive change. Phase 5:Innovation Once the practice is fully operational, changes are incorporated into the evidence based practice routine to ensure its effectiveness. Winter and Szulanski (2001) noted that changes made after evidence based prac tices were implemented with fidelity were more successful than adaptations made before full implementation. Phase 6:Sustainability This is the final phase in the Fixsen et al. (2005) model. term survival and contin ued effectiveness of the needs to be realized despite changes in teachers, school administrators, and involvement of researchers. Summary of the implementation of evi dence based practices when deciding to implement an evidence based instructional practice. This framework emphasizes that implementation is a process with phases one must progres s through in

PAGE 32

32 order to have successful implementation of an evidence based practice. Using this framework can inform teachers and guide them through the process. What Should Teachers Know about Implementing Evidence Based Practices? Implementation of an ev idence based practice is multi faceted. Prior to the implementation of the practice, teachers need to be adequately trained. This training should be an ongoing, interactive form of professional development where teachers are provided the opportunity to dis cuss issues regarding the practice with both researchers and colleagues. Teachers who participated in professional development workshops, implemented the practice with fidelity and who had more opportunities to discuss implementation concerns had greater s uccess in implementing and sustaining the newly acquired practice (Fixsen et al., 2005). leadership, sufficient investment, and strong internal and external support, very few innova tions reach the institutionalization stage where they become a routine and sustain an evidence based practice in the classroom, a teacher must actually put it into practice. Unfortunately, there is a research to practice gap in the field of education (Fixsen et al., 2005). Many have speculated as to why this gap exists when it is known certain practices are capable of producing positive effects for students. Greenwood and Abb ott (2001) identified four reasons for the lack of impact of research on practice: (a) research and practice communities are too secluded from one another; (b) the relevance of educational research to practice is not viewed as important to teachers; (c) la ck of useability of evidence based practices in the classroom; and (d) limited opportunities for researchers and teachers to collaborate and participate in professional

PAGE 33

33 development. These ideas will be explored in the following section by reviewing concept ual models for thinking about the research to practice gap. Conceptual Models for Linking Research to Practice There are a number of conceptual models regarding the sustainability of an evidence based practice in the classroom (Greenwood & Abbott, 2001; L ittle, 1993; Schaughency & Ervin, 2006). Gersten et al. (1997) developed a framework that includes elements of these multiple models depict ing the varying factors that are crucial for sustained use of evidence based practice s ( Figure 2 2 ). This graphic dep iction highlights different factors that must be taken into account when attempting to implement an evidence based practice, such as a cognitive behavioral intervention, in a classroom. The reality principle ( Figure 2 2 A ) in this model pertains to th e feasibility and fit of the practice. The practices that researchers expect teachers to use in the classroom must be practical and concrete. F or sustained use of an evidence based practice, teachers must perceive the practice as being effective for most s tudents including those with disabilities. Gersten and Dimino ( 2001) found that c lassroom teachers were willing to make accommodations for students with disabilities if the practice enhanced learning for the typically functioning students as well Given t his, it is important for researchers to discover how to communicate research findings so that teachers will find the required change both manageable and effective for the entire class (Gers ten et al., 1997; 2000). It is also important for r esearchers to he lp teachers fit the practices into daily routines of classroom instruction. It is doubtful that most teachers are willing to revamp their entire daily routine in order to implement an evidence based practice ; h owever, if researchers can assist teachers by developing practices that fit

PAGE 34

34 neatly into the current classroom routine, teachers are more apt to adopt the intervention (Gersten et al.). Another factor discussed in the Gersten et al. (1997) framework is the scope ( Figure 2 2 B ) of intended change required by evidence based practices. McLaughlin emphasiz ed that researcher s do not need to meddle with small, trivial aspects of teaching behavior, but rather focus on changes that are going to have the most substantial effects. Conversely overly detailed practices did not capture interest (Gersten et al., 2000). Con sequently, when establishing the scope and specificity of proposed practices it is necessary to strike a middle ground. In addition, active commitment by the school and district leadership is a key element to project success and long term sustainability. Furthermore, researchers must have realistic timeframes for substantial changes to happen. Change should be mapped out much like an action plan with an appropriate timeline in place. One challenge for researchers is to discover how to disseminate research findings in such a way that teachers will find the required change both manageable and beneficial. Referring to C and D of Figure 2 2, t he technical and conceptual aspects of change in this framework refer to ive interaction is necessary so that teachers have regular opportunities to discuss the impact of new practices on s 1995, p. 54). These interactions could be conducted with a change or linking agent such a s a member of the research team or a staff member specifically assigned to assist with the

PAGE 35

35 implementation of the evidence based practice (Gersten & Dimino, 2001). This person must be truly knowledgeable about the evidence based practice and be able to prov ide specific feedback to teachers on implementation issues. Equally important is that this person is easily accessible and helps with the logistics and practical problems typically encountered during implementation. In order for teachers to truly grasp the conceptual meanings of new practices, they must have opportunities to discuss the practice with their colleagues, apply the evidence based practice long enough to realize student change, and be provided sufficient time to adjust to the new change in their day to day ideas, understand how these ideas differ from those they have held in the past, and be persuaded that these ideas are better than the ideas they had in the past Linking changes in teaching to student learning is another component of Gersten ( Figure 2 2 E ) The need to continually link the proposed changes in instruction to specific gauges and measures of student learning is critical, because intensive collaboration -planning, exchanging materials, and regulating pupil performance -does not automatically translate into observable changes in classroom practice and may, if pushed too hard, actually eat into time for ongoing inst ructional work in class. (Gersten et al., p. 471) In order to help teachers sustain the evidence based practice, researchers need to instruct teachers in how to collect and use student performance data to fine tune their teaching. Unfortunately, most teac hers rely more on observable student behavior than on quantitative assessment data (Gersten & Dimino, 2001). collegial support networks ( Figure 2 2 F ), pertains to substantive professional intera ctions with colleagues. P rofessional interactions can affect teachers abilit ies to adopt, implement, and sustain

PAGE 36

36 evidence based practice s in the classroom. Researchers need to find a way to interact with teachers regarding practices in ways that make the teachers feel like they are productive and responsible members of a broader professional community. D irect input from teachers is beneficial for both teacher s and researcher s Teachers have the opportunity to interact with professional colleagues and resea rchers have the opportunity to hear exactly what teachers want, need, and are willing to do in the classroom environment. Little (1993) cited at least one benefit of having collegial networks is n teaching (i.e., opportunities to experiment with new techniques, evaluate their impact, and then increase teacher capacity to implement an evidence based practice by allowing teams of teachers to profit from collaborative expertise. Schaughency and Ervin (2006) argued that there is a reciprocal influence between research and practice in that researchers inform teachers of evidence based practices and in return, ar e informed of the necessity of context through collaborative partnership with teachers in the field In short, both parties learn from each other by the giving and receiving of first hand knowledge. Based Practices? Sustaining an evidence based practice in the classroom is not an easy feat. Researchers struggle with ways to have their proven or promising practices implemented and sustained with fidelity in classrooms (Fixsen et al., 2005). Evidence based practices sh ould be effective for most students including those with disabilities for an increased chance that teachers will implement and sustain the practice (Gersten et al., 1997). Teachers want tools that are practically significant, and not old practices presente

PAGE 37

37 collect and use student performance data rather than rely on subjective observations of student behavior (Gersten & Dimino, 2001). These data will help them assess the strengths and wea knesses of each student within the framework of the evidence based practice. Although numerous interventions that are evidence based are available to teachers, one type will be the focus in this study: cognitive behavioral interventions (CBI). Cognitive b ehavioral inter ventions were developed based on the premises set forth in cognitive behavioral theory, which was developed out of two separate well known theories: behavioral theory and cognitive theory. Cognitive Behavioral Intervention In the 1960s, beh aviorists and cognitivists began to blend ideas (Mayer, Lochman, & Van Acker, 2005) Researchers began using a hybrid conceptual model to develop cognitive behavioral strategies. Cognitive Behavioral Theory guides the use of behavioral strategies that incr ease activities so that the student feels a sense of proficiency and gratification. According to Craighead L. et al. (1994 ), cognitive procedures which can be done in the classroom include (1) identification of dysfunctional and distorted cognitions and r ealization that they produce negative feelings and maladaptive behaviors; (2) self monitoring of negative thoughts, or self talk; (3) identification of the relationships of thoughts to underlying beliefs and to feelings; (4) identification of alternative ( functional and nondistorted) thinking patterns; assumptions about self, world, and future. (p. 43) to think C ognitive B ehavioral T heory problem solving strategies teach individuals to behav ior (Polsgrove & Smith, 2004). Thus, in a classroom setting, problem solving is

PAGE 38

38 student driven, which purportedly aids in the generalization of newly learned skills to other setti ngs. Tools For Getting Along is one example of an instructional evidence based practice grounded in Cognitive Behavioral Theory. Within this practice, Smith and Daunic (2006) utilize six problem solving steps to help students manage behavior that directly to I think about the cause; (d) I think of possible solutions; (e) I try a solution; and (f) I evaluate how it turned out. These six steps are addressed throughout a 27 lesson play. Students who learn and use these problem solving skills demonstrate less disruption and aggression in the classroom, contributing to a more positive classroom en vironment (Polsgrove & Smith). Managing Conflicts Effectively There are many reasons why children should be taught effective practices for managing conflicts. First, failure to learn how to resolve conflicts can result in unhealthy and destructive interpe rsonal patterns of communication and this pattern can reach far affected by his or her ability to problem solve in a constructive manner, which could lead to greater p eer acceptance (Sugai & Horner, 2002) Cognitive Behavioral Theory provides the foundation for a particularly effective approach to teaching conflict management to school aged children (Durant, Barkin, & Krowchuk, 2001). There is a significant body of rese arch demonstrating the efficacy of these practices to address anger/aggression, anxiety, and depression, which constitute significant needs areas for students with emotional and behavioral disorders (Mayer et

PAGE 39

39 al., 2005). In addition, older children who are in the cognitive stage of formal operations may benefit more from Cognitive Behavioral Theory (Durlak, Furhman, & Lampman, 1991; Sukhodolsky, Kassinove, & Gorman, 2004). When conducting a meta analysis of the effectiveness of cognitive behavioral interven tions for children, Durlak et al. found that children entering treatment at a higher level of cognitive functioning benefit more from cognitive behavioral interventions than do children at less advanced levels. For example, the effect size (.92) for childr en in the cognitive stage of formal operations (ages 11 to 13) was almost twice the magnitude of the effect obtained for children in less advanced cognitive stages (ages 5 to 7, .57; or ages 8 to 11, .55). The results support the notion that age may be a f actor in the success of cognitive behavioral interventions. Cognitive processes are critical in predicting responsiveness to treatments, and it appears that the ability to think abstractly is a potentially important it from cognitive behavioral interventions. This could suggest that younger children are less likely to benefit from a cognitive behavioral intervention than older children who have more developed cognitive abilities. The literature addressing the effect iveness of cognitive behavioral interventions is vast and has been studied from a variety of perspectives (Smith, Daunic, Miller, & Robinson, 2002) For the purposes of this study, the discussion will be limited to two key areas: (a) Problem solving cognit ive behavioral interventions, and (b) the role of adults in cognitive and behaviors regarding the implementation of a problem solving curriculum that is an example of a cognitive behavioral intervention. The reasons teachers should use cognitive behavioral interventions will be highlighted.

PAGE 40

40 Problem S olving Cognitive Behavioral Interventions Teachers spend a great deal of time during the school day attempting to mediate problems be tween their students (Larrivee, 2009) Teachers could use an evidence based problem solving cognitive behavioral intervention to teach children how to address their problems in appropriate ways. This could reduce the amount of time teachers have to spend o n problem solving issues and could increase their time for academics. A cognitive behavioral intervention that demonstrates problem solving strategies can help with everyday, low level classroom conflicts, but can it reduce the number of aggressive acts i n the classroom? Second Step is a universal primary prevention program designed to reduce the development of social, emotional, and behavioral problems and promote the development of core competencies (Frey, Hirschstein, & Guzzo, 2000). This cognitive prob lem solving model teaches children how to solve social problems rather than teaching them specific behavioral responses. Edwards, Hunt, Meyers, Grogg, and Jarrett (2005) investigated the effectiveness of the Second Step curriculum on all fourth and fifth g rade students (N = 455) in one small urban school district. Based on several pre and post measures, significant gains were obtained in the areas of empathy, anger management, impulse control, and bully proofing, which is consistent with previous research ( Edwards et al.) Through teacher interviews, researchers noted that teachers were confident in the effectiveness of the believe it to be effective. Another group of rese archers examined the impact of implementing Second Step using a comprehensive, city wide approach (Cooke et al., 2007). The foci of this study

PAGE 41

41 w ere : (a) high implementation fidelity; (b) strong teacher and administrator buy in and support; (c) high levels of community involvement and support; and (d) intensive professional development and technical support. The study included 741 third, fourth, and fifth graders in six schools, and student surveys, observations and discipline referrals were used to assess a nti and pro social behaviors. Cooke et al. found significant improvements in positive approach coping, caring cooperative behavior, suppression of aggression, and consideration of others. The program was implemented with high fidelity as evident by the171 teachers who responded to a year end teacher survey, 80.7% reported delivering all the Second Step lessons for their grade and 68.3% reported delivering additional, non required booster activities. Administrators need to be cognizant of the roles of schoo l personnel when implementing a cognitive behavioral intervention, because implementation greatly depends on the cooperation and motivation of the school staff. Role o f A dults i n Cognitive Behavioral Interventions Teachers play an integral role in the lev el of success of the implementation of a cognitive behavioral intervention (Daunic et al., 2006) School administrators and teachers must understand how the evidence based cognitive behavioral intervention works, must be dedicated to implementing the cogni tive behavioral intervention with fidelity, and must know the cognitive ability level of their students in order to gain success when using a cognitive behavioral intervention. When using this type of intervention in the classroom, teachers frequently serv e as role models. For example, teachers use role play to demonstrate how to go through the steps of problem solving effectively They use self talks, which is where they verbally work through a problem in front of the class. This is a good way for students to see that adults also have to work

PAGE 42

42 through problems and sometimes struggle with their decisions. It is important for them to be cognizant of this and be open to sharing their thoughts about why they performed a certain skill with their students. This wi ll help the students to not only see the skill being performed but will allow them the ability to hear what the teacher was thinking at the time they used the skill. Implementing cognitive behavioral interventions requires the support and the cooperation o f the adults surrounding the students. Smith (2002) argued that it is important for teachers to model the behavioral and especially the cognitive skills they are teaching (i.e., role plays, self talks). Teachers and administrators must support the students in learning the cognitive numerous opportunities for them to practice what they are learning in a safe student behavior, the classroom environment may become more conducive to the promotion of prosocial skills (Sutherland & Oswald, 2005). This suggests that as teachers witness student behavioral improvements in the classroom while implementing an evidence based practice, the cla ssroom environment may take on a more caring and nurturing feel, which would in turn foster the use of better social skills. Bickmore (2001) studied elements of citizenship education in a peer conflict mediation program model at six elementary schools in one big city school district in the northern United States. The full study involved a range of qualitative and quantitative information, collected between 1997 and 1999, examining the character and effectiveness of Center for Conflict Resolution peer media tion training and program

PAGE 43

43 self determination of various students in conflict mediation re lated activities at each Bickmore (2001) found that schools where the school administrator was an active supporter of the cognitive behavioral intervention had much greater success. As students spend a majority of their day at school, faculty need to be positive role models of constructive ways of solving problems. As they teach the students these skills, they need to provide opportunities to practice. Then the faculty needs to allow the students to use these skills when real life conflicts arise. Monitoring their progress and providing feedback are also essential components of any problem solving intervention. Implications of CBIs Students can learn and use problem solving skills and cognitive behavioral intervention curricula can decrease disrup tive or aggressive behaviors in school settings and can have positive influences in classroom environments (Polsgrove & Smith, 2004). Cognitive behavioral interventions can be effective for students as young as first grade through high school although the effectiveness of the intervention (Johnson, D. et al., 2001). Teachers and other school faculty play a major role in the success of the cognitive behavioral interventions. They need to implement the interve ntion with fidelity, be positive role models of constructive ways of solving problems, allow students adequate opportunities to practice newly learned skills, and provide constructive feedback. For teachers to successfully implement cognitive behavioral in terventions in the classroom, however, there must be system wide (i.e., school wide) acceptance of the particular behavioral program, quality teachers who can implement the intervention with fidelity, teacher training, and

PAGE 44

44 reinforcement (e.g., recognition by administrators, positive response from peers, and positive student behavior changes) to motivate teachers to continue with the evidence based practice (Gerber & Solari, 2005). Teachers who work in schools that have adopted a school wide approach called Response to Intervention (RTI) have more success with intervening early with evidence based practices and monitoring the progress of how students respond to an array of increasingly intensive interventions across multiple tiers of support (Gersten & Dimin o, 2006). RTI, a practice widely implemented for students who are at risk for school failure, strengths and weaknesses, and for making data based instructional decisions that st rengthen intensive early intervention. Cognitive behavioral interventions can be used within the RTI framework across universal, secondary, and tertiary levels to support Although teachers may be motivated to continue with a proven or promising practice even within a structured framework like RTI, often times many of their other academic requirements or concerns regarding the practice get in the way (Gerber & Solari, 2005) Researchers must determine how to keep these practices in use in the concerns about the practice, their ability to implement the practice with fidelity, and how they will need to adapt are all teacher level variables that change facilitators should address. The Concerns Based Adoption Model contains several tools that can be utilized by change facilitators to individualize the change process and address each

PAGE 45

45 Concerns Based Adoption Model A concep tual framework for the dissemination of an innovation known as the Concerns Based Adoption Model (CBAM) (Hall & Hord, 1987, 2001, 2006) guided the present study. Hall and Hord (1987) contributed to the body of literature regarding implementation by changin g the emphasis from the innovation as an object to the individual users as subjects. Hall and Hord noted that there are several important assumptions and assertions that underlie the CBAM work. First, understanding the point of view of the participants in the change process is critical. It is imperative that change facilitators understand the perceptions of the clients and are cognizant of where in the change process each user currently resides. Secondly, change is a process, not an event. Administrators, t eachers, and change facilitators must be aware that there is a process involved in implementing innovations in the school setting and this process requires time. One way to guide the planning and pacing of change would be for change facilitators to break t he process into phases and steps. Thirdly, it is possible to anticipate issues that will occur during a change process. By incorporating an action plan that breaks the process up into phases and steps, the change facilitator can anticipate certain events a nd be better prepared to handle and approach unexpected occurrences. Fourth, innovations vary in number of components and focus. An innovation can be a product innovation, such as a new textbook, or a process innovation, such as a different behavior manage ment approach. Therefore, not all innovations can be implemented in the same fashion. An action plan for each specific innovation needs to be developed by the change facilitator. Fifth, innovation and implementation have parallel sets of steps and procedur es. In order to implement an innovation, a change facilitator needs to view the change process as consisting of

PAGE 46

46 innovation development plus implementation. Sixth, to change something, someone has to change first. This assumption cuts to the heart of the CB AM framework. Attention must be on the individuals who are involved in the change process and how they experience the change process. Finally, change facilitation should be a shared responsibility that involves everyone at the school (i.e., teachers, admin istrators, & students) at one time or another. Tools of CBAM implementing a new problem solving practice. The first diagnostic dimension of the CBAM is the Stages of Concern Questionnaire (SoCQ). Hall and Hord (2006) identified a set of seven specific categories of concerns about implementing an innovation that they have called Stages of Concern. The stages of concern regarding implementation of an innovation escalate through a continuum ( 2 3 ). Stages and levels of concern. Hall and Hord (2006) grounded the CBAM in the work of Fuller (1969), who proposed that concerns moved through four levels: (a) unrelated, (b) self, (c) task, and (d) impact. H all and Hord noted that these four levels of concerns are found in people involved with all types of innovations and change processes (Figure 2 4 f concern. For example: change being proposed, is not interested, or actively resists learning about it. This level corresponds with Stage 0: Basic Awareness.

PAGE 47

47 ncerns move beyond basic awareness and correspond with the next two stages of concern. Stage 1: Informational concerns, would describe a person who is aware of the innovation and interested in learning more details. Stage 2: Personal concerns, would descri be an individual who is uncertain of the demands of the innovation and how to meet those demands. would describe someone who is concerned with efficiency, organizing, managing, s cheduling, and time demands. own use of the innovation. Stage 5: Collaboration concerns f ocus on collaborating with colleagues to make the innovation more effective and efficient. Stage 6: Refocusing concerns indicate that the person has ideas about a more effective alternative to the proposed innovation. Levels of use. A second diagnostic dim ension of the CBAM model emphasizes the levels of use of the innovation. This interview tool could be used to focus on behaviors of teachers and portrays how they act in response to a specified change. According to Hall and Hord (2006), the individual teac her passes from nonuse through eight levels of the use of an innovation (Figure 2 5 ). Hall and Hord have identified three nonuse and five use levels within this framework. Levels 0, I, and II are considered nonusers of the innovation. Level of Use 0 Nonu se is when a person knows very little about an innovation and exhibits no behavior related to it.

PAGE 48

48 Level of Use I Orientation is when a person takes action to learn about an innovation or display an interest in a particular innovation. Level of Use II Pre paration for Use is when a person has decided to use the innovation, but has not started yet. The remaining five levels represent users to some degree of the innovation. Level of Use III Mechanical Use is when the user is actively engaged with the innova tion in the workplace. This user is beginning to change but there is a short term day to day focus on planning the implementation of the innovation. Level of Use IVA Routine is when the implementer has mastered the innovation and implements it with fideli ty. Level of Use IVB Refinement the user is beginning to assess the effectiveness of the innovation for students. This user begins to adapt the innovation for greater student benefit. Level of Use V Integration the implementer begins collaborating wi th other users to adapt the innovation for greater student outcomes. Level of Use VI Renewal the implementer modifies the innovation in major ways or replaces it altogether. Innovation Configuration Map. The final diagnostic dimension of the CBAM is t he Innovation Configuration Map. According to Hall and Hord (2006), the main purpose of the map is to present a developed description of different ways of doing the innovation. Observations can be conducted using the Innovation Configuration Map to assess the level of fidelity of the implementation. The map is developed by the creators of the innovation and is used to address the presence and variation of key components

PAGE 49

49 informs both the observer and the user as to the level of fidelity. Utilizing CB AM The CBAM model has been used in empirical studies wherein educational innovations have been implemented in school settings. For example, the CBAM was employed in Australia in structuring professional development activities for 12 teachers incorporating newly acquired computer hardware and software skills into their teaching practice (McKinnon & Nolan, 1989). McKinnon and Nolan administered both the Stages of Concern Questionnaire and conducted Levels of Use interviews. The questionnaire was administered four times over a 21 month time period, and interviews were conducted three times during an 18 month period of time. The researchers and program designers used the information from the first SoCQ to redesign the technology. The information gained from the remaining three administrations of the SoCQ was used to conduct professional development activities designed to address the concerns of the teachers. During that time, participant concerns shifted from Stage 2 (personal) concerns to Stage 1 (information) c oncerns. The researchers attributed intense early focus on personal and refocusing (Stage 6) concerns to problems with technology during the first term of the study. Following the first round of interviews, the researchers found that 58% of the participant s remained at levels below routine use. Again the researchers and program designers used the information gained from the interviews to focus professional development activities to assist teachers to reach a routine level of

PAGE 50

50 use. By the end of the study, se ven of the teachers (58%) reported levels of use at or greater than routine. In researching studies that utilize the CBAM model, it was discovered that some studies only use one of the diagnostic tools provided. The SoCQ appears to be used by itself in a majority of the studies (Dass, 1997; Kember & Mezger, 1990; Ward, West, & Isaak, 2002) possibly due to its ease of use. Hope (1995) used all three diagnostic dimensions (i.e., Stages of Concern, Levels of Use, and Innovation Configuration Map) of CBAM to investigate the effect of technology on 18 classroom teachers at an elementary school. As the principal of the school, Hope was a participant investigator who assessed the use of desktop computers to support teacher administrative tasks. This study investi gated the relationship between the degree of implementation of computers for learning in primary schools and selected characteristics of those schools. The findings indicated: (a) Principals are critical figures in successful implementation of new programs ; (b) they appear to be influential in determining the degree of implementation in their buildings and district; and, (c) implementation success was as opposed to a s In these previous studies CBAM has been used to inform ongoing professional development for teachers, as well as the implementation of innovative practices in schools. In the present study all three diagnostic dimensions of CBAM were put to use to develop an explanation for why teachers adopted or discontinued the use of a promising practice.

PAGE 51

51 Discussion of the Literature There are evidence based practices available for teachers to implement in the classroom to aid in teaching problem solving behaviors. Promising or proven cognitive behavioral interventions are one type of practice that teachers can use in the classroom setting. Cognitive behavioral interventions represent a viable approach for promoting improvem ent in the social behavior of students and for further enhancing their ability in self regulating their behavior (Smith et al., 2005). Teachers can benefit from evidence based practices that can be implemented in the classroom to teach students to solve co nflicts with their peers and teachers in an appropriate manner. Some cognitive behavioral interventions are evidence based practices that help students make positive changes in their behavior (Smith & Daunic, 2006). Cognitive therapy seeks to identify and change distorted or unrealistic ways of thinking and replace them with appropriate, alternative concepts (Smith & Daunic). Cognitive therapy involves the use of behavioral strategies to increase activities where the student feels a sense of proficiency an d gratification. Using cognitive behavioral interventions in the classroom to assist students in improving their ability to resolve conflicts can help produce problem solving skills that are productive and healthy. The Concerns Based Adoption Model can be used to assist change facilitators when encouraging teachers to implement an evidence based cognitive behavioral behaviors when implementing a new innovation. By using these di agnostic tools, the change facilitator can address individual teacher s and provide ongoing professional development to address these individual concerns. CBAM can be used to assess how teachers utilized a cognitive beha vioral problem

PAGE 52

52 solving curriculum (e.g., Tools for Getting Along) after the researchers were no longer involved with the implementation process. Gersten et al. (1997) developed a framework of the varying external factors considered crucial for sustain ed use of an evidence based practice. Researchers need to continue to move from focusing on the effectiveness of a practice to the sustained use of an evidence based practice (i.e., CBI). The sustainability of an evidence based cognitive behavioral interve ntion could be studied in a variety of ways. For example, This would determine which school practices assist or hinder the use of a particular evidence based practice. Res earchers could also investigate how certain teacher characteristics such as ethnicity and gender affect the sustained use of an intervention. The purpose of this study is to examine how teachers who have been trained to implement a promising practice conti nue to use that practice over time. Specifically, How do these concerns influence the level to which teachers report implementing the practice? How faithfully do teachers sustain these practices? For an increased chance that teachers will implement and sustain the practice, researchers must ensure that the evidence based practice is effective for most students including those with disabilities (Gersten et al., 1997). Teach ers would benefit from being taught how to collect and use data that will help them ascertain the strengths and weaknesses of each student within the framework of the evidence based practice. These skills could be taught in teacher preparation programs, so that beginning teachers already have these skills and are able to use them as needed. In addition,

PAGE 53

53 ongoing, interactive professional development where teachers are provided the opportunity to discuss issues regarding the practice with both researchers and colleagues will assist the teacher in using the evidence based practice successfully. CBAM could assist administrators and change facilitators in developing appropriate professional development for teachers based on their concerns regarding the implementa tion of an evidence based cognitive behavioral intervention. An in depth look at the three diagnostic dimensions of the CBAM framework (Stages of Concern Questionnaire, Levels of Use Interview, and Innovation Configuration Map) concluded chapter two. Empi rical research was provided to support the use of the CBAM for the present study. The CBAM framework was used in this practice, to observe the behaviors of teachers who reported ly continued to implement this practice, and to assess their level of fidelity of use of this cognitive behavioral intervention.

PAGE 54

54 Figure 2 1. Illustration of the Phases of Implementation framework (Fixsen, Na oom, Blas, Friedman, & Wallace, 2005) I. Exploration and Adoption II. Program Installation III. Initial Implementation IV. Full Operation V. Innovation VI. Sustaina bility

PAGE 55

55 Figure 2 2. Factors that support sustained use of research based practices (Gersten, Vaughn, Deshler, & Schiller, 1997) B. Scope Neither too broad/vague nor too narrow A. Reality Principle Sustained Use of Research Based Practices E. Link Changes in teaching to student learning C. Technical opportunities to practice/experiment with feedback F. Collegial Support Networks Structures to support risk taking D. Conceptual Linking research ideas to classroom situations Joint problem solving

PAGE 56

56 Score Type of Concern 6 Refocusing Concerns 5 Collaboration Concerns 4 Consequence Concerns 3 Management Concerns 2 Personal Concerns 1 Informational Concerns 0 Basic Awareness Figure 2 3. Stages of Concern (Hall & Hord, 2006)

PAGE 57

57 IMPACT 6 Refocusing 5 Collaboration 4 Consequence TASK 3 Management SELF 2 Personal 1 Informational UNRELATED 0 Awareness Figure 2 SCORE LEVELS OF USE USERS VI Renewal V Integration IVB Refinement IVA Routine III Mechanical Use NON USERS II Preparation for Use I Orientation 0 Nonuse Figure 2 5. Levels of Use (Hall & Hord, 2006)

PAGE 58

58 CHAPTER 3 METHODOLOGY O verview of the Methods This study was designed to address a serious problem in schools: Teachers are expected to use promising and/or proven instructional practices; however, research suggests there is a gap between what teachers know they should use, and what they actually put into practice. In this chapter, methods are described for conducting an explanatory single concerns about implementing an innovative intervention, and how do these conc erns affect their implementation of this promising practice? Building on Previous Research The present study was a follow up investigation to a Goal 3 study funded by the United States Department of Educational Research IES, assessing the effectiveness of the problem solving curriculum, Tools For Getting Along. The research group, based at the University of Florida, conducted a Goal 3 four year research study on the effectiveness of this problem solving curriculum. Professional development was provided to all treatment teachers that participated in this effectiveness study. Each teacher attended a 2 day training prior to using the Tools For Getting Along curriculum and ongoing professional development addressing implementation and fidelity was provided thro ughout the school year. Tools For Getting Along is a cognitive behavioral problem solving curriculum that incorporates modeling, feedback, reinforcement, and cognitive mediation to help students manage frustrating or anger provoking situations that often l ead to aggression (Smith & Daunic, 2006). This curriculum is a universally applied classroom based curriculum that has been shown to reduce

PAGE 59

59 disruptive/aggressive behavior among fourth and fifth grade students. School personnel, teachers, and guidance couns elors implement the intervention within the normal routines of the school setting. Tools For Getting Along is a 27 lesson curriculum. The first 21 (i.e., core) lessons are taught at the rate of two per week, each requiring approximately 30 minutes. Followi ng an overview of social problem solving, lessons focus on six problem solving steps. The six steps are covered in 16 content lessons and five strategically placed role play lessons provide opportunities to practice steps as they are learned. Following the core lessons, six booster lessons are implemented once a week, providing opportunities for review and generalization. Although the Tools For Getting Along study is still in progress, some initial promising results have been obtained. If students were in t he treatment group and had high risk at pre test, they had relatively lower risk at post test than students in the control group with comparable pre scores (condition by pre score interaction). This means that Tools For Getting Along has a positive effect on high risk students in the treatment group as compared to high risk students in the control group. While Tools For Getting Along shows promise for high risk groups, any promising or proven practice must be delivered consistently for an appropriate length of time to produce desired student outcomes. An in depth discussion of the Tools For Getting Along effectiveness study was presented in Chapter 2 and the participant information was outlined t here. Three schools participated in the 2006 2007 school year, four schools participated in the 2007 2008 school year, and three schools participated in the 2008 2009 school year. The five districts located within a 50 mile radius to the University of Florida had numerous schools that could have participated in the e ffectiveness study (Table 3 1). The majority

PAGE 60

60 of the elementary schools in the five school districts were designated as Title 1 schools (high risk). The students in the five districts had comparable scores on the Spring 2009 FCAT (Table 3 2). A majority of the schools also had minority populations of 70 95% and a large proportion of the students received free or reduced price lunch (Table 3 3 and Table 3 4). The approximately 91 elementary schools within the five districts were rank ordered based on SES (pe rcent of students who qualified for free or reduced price lunch) from highest to lowest. Ten elementary schools (grades K 5 th ) were chosen to participate in the study through a process using specific selection criteria. Only those schools were included in which 75% or more of the fourth and fifth grade classroom teachers agreed to participate in the Tools For Getting Along study. Once enough schools were enlisted to participate, the schools were matched based on SES and school size, and matched schools were randomly assigned to either the treatment group or control group. The teachers who participated in the present study were drawn from those in the treatment group. Intensive professional development in the utilization of TFGA was provided in the effective ness study. All teachers in the previous study attended a two day training that these teachers met on a weekly basis with a research assistant assigned to their school. The purpose of this meeting was to provide general support to teachers in their implementation efforts; insure teacher commitment, focus, and adherence to the implementation schedule; enhance teacher attempts to encourage generalization of

PAGE 61

61 learned skills; address implementation/generalization concerns and issues; and provide opportunities to observe and document treatment fidelity. Ideally, these teachers have continued to me et with one another to collaborate on the implementation/generalization of the promising practice. The present study was predicated on the hypothesis that teachers trained in an evidence based or promising innovation have continued to implement the innova tion with fidelity after the research team has withdrawn its support. This follow up study explored the concerns of these treatment teachers about the innovation, and their actions related to its continued use. The Perspective of the Present Study The p resent investigation used a mixed method design using the Concerns Based Adoption Model (CBAM) framework (Hall & Hord, 2006). A questionnaire, developed as part of this model, was used to derive descriptive statistics related to lementing innovative educational practices. An interview protocol, also developed as part of the CBAM framework, was used to determine the extent to which teachers have continued to use the curriculum in the classroom. An additional CBAM strategy, an Innov ation Configuration Map, was established to determine the level of fidelity with which the curriculum was being delivered. empirical inquiry that investigates a contemporary phen omenon within its real life context, especially when the boundaries between phenomenon and context are not case study was an appropriate approach to use in ascertaining if teachers have sustained the curriculum and to wha t extent they have done so ( Yin, 2008). Branching interviews were

PAGE 62

62 conducted along with observations of one teacher implementing the curriculum in their classrooms. A branching interview was constructed so that, t hrough a series of related behaviors (Hall & Hord, 2006) (Appendix B). The goal of the interview was to stimulate the participant to describe and provide examples of behaviors that he o r she had taken in relation to the innovation. This case study embodied both quantitative and qualitative properties in providing a follow up analysis to the previously conducted effectiveness study. The focus of this inquiry was on the instructional behav iors of teachers, specifically investigating whether classroom teachers, trained in the implementation of a promising practice, continued to use this practice faithfully after several years. Case Study Methods In general terms, a case study is used as a questions are being posed. A more technical definition of a case study begins with the contemporary phenomenon within its real life conte xt, especially when (b) the study inquiry (a) copes with the technically distinctive sit uation in which there will be many more variables of interest than data points, and as one result (b) relies on multiple sources of evidence, with data needing to converge in a triangulating fashion, and as another result (c) benefits from the prior develo pment of theoretical propositions to 14). In conducting this case study, research methods were guided by the CBAM framework, a model that is based on 20 years of research in a variety of organizational

PAGE 63

63 settings (H all & Hord, 1987; 2006). CBAM is a research based approach also used in multiple studies regarding the implementation of educational innovations. An in depth discussion of the CBAM model was presented in Chapter 2 and components are described in subsequent methodological sections of this chapter. case study researcher (Yin, 2009), and that was my intention throughout this study. I served as a graduate research a ssistant on the effectiveness study for the past four years. It was my responsibility to assist teachers at both treatment and control schools to collect pre and post data on their fourth and fifth grade students. In addition, I met on a weekly basis with treatment teachers to discuss their progress with delivering the curriculum, and I observed many of the Tools For Getting Along lessons being delivered. I have continued to be a member of the effectiveness study team but with new responsibilities. Although we are no longer implementing the curriculum in any classrooms as part of the research study, I was responsible for collecting follow up data on students in the treatment group, which entailed collecting measures completed by their current homeroom teache rs and also completed by the student. Through this follow up process, I had minimal contact with treatment teachers because the focus of the research has shifted from teacher behaviors to student outcomes. The potential for conflict of interest in the pres ent study was diminished because the professional development component of the effectiveness study was completed more than 12 months prior to the start of data collection. A possible compromising situation was that while conducting follow up visits for the effectiveness study I could possibly have encountered the original participating teachers. I was initially concerned that if they saw

PAGE 64

64 me in their school again, they might have felt obligated to continue to use Tools For Getting Along. Such proved not to b e the case. Further, potential participants in the present study were informed that they could refuse to participate without fear of repercussions. One ethical issue that was anticipated was how I would handle my subjective reaction to the observation of teachers using non evidence based practices or neglecting to address social development aspects in their classroom practice. Where I bracket my own response, collect my i mpressions at the time, and to inform my subsequent analysis of the findings. Site of the Study This study was carried out in elementary schools across five school districts in North Florida. Selected teachers from these districts who participated in the earlier effectiveness study were chosen through a process using purposeful selection criteria. All of the original teachers who agreed to participate were given a Stages of Concern Questionnaire. Based upon data in the returned questionnaires, teachers at various stages of concern were interviewed using the Levels Of Use branching interview. Following the interviews, an Innovation Configuration Map was used to observe the one self reported high user of the practice she was previously trained to use. Permiss ion to gain access to these schools was obtained through district representatives and principals at each school. Participants The participants in this study were fourth grade and fifth grade elementary teachers who were taught a promising intervention (To ols For Getting Along) and who

PAGE 65

65 were provide professional development in implementing this innovative curriculum in their classrooms as part of a previously conducted effectiveness study (Smith & Daunic, 2006). The participants selected for the present stud y were comprised of those teachers who are presently teaching children within the age range for which the intervention was developed, (9 years to11years of age), and were teaching at one of the original treatment schools. The target population comprised 71 previously trained teachers; 48 teachers were eligible to participate. Twenty three teachers were ineligible to participate because no current contact information was available for 12 teachers and 11 teachers no longer were assigned to elementary schools. After considerable effort was made to participants. Accessible Population Demographic One hundred percent of the Stages of Concern Questionnaires returned by the 21 teachers were deemed usable for assessing the demographic characteristics of these participants. The ages of the 21 teachers who returned a completed questionnaire ranged from 23 years 60 years with an average age of 41years. Of these 21 participants, onl y 3 were male and only 1 teacher was African American; no other teachers were members of ethnic minority groups. Their ye ars of experience ranged from 2 20 years with an average of 8 years of experience (Table 3 5). Although this study had possible partici pants in 6 different counties of similar size within the same geographical region, teachers only responded from 5 of those counties. None of the four teachers in the sixth county could be located for this study.

PAGE 66

66 Instrumentation Instrumentation used in thi s study was guided by the CBAM framework, a 3 fold sequential, cumulative approach, which provided tools for the researcher to use to gain about using a new practice. The thre e components included the Stages of Concern Questionnaire, the Levels of Use Interview, and the Innovation Configuration Map. Stages of Concern (SoC) (Questionnaire) Hall and Hord (2006) identified and confirmed a set of seven specific categories of concer ns about an innovation called Stages of Concern (ranging from Awareness to Refocusing). This part of the framework addressed the affective side of change innovation). The S tages of Concern Questionnaire was a 35 item measure related to solving curriculum, TFGA. All teachers who had previously participat ed in the effectiveness study across those 10 schools were asked to complete the questionnaire. The theoretical test/retest reliability ratings for the Stages of Concern Questionnaire ranged from .65 to .86 and internal consistency alpha coefficients range d from .64 to .83 (Hall & Hord, 2001). Levels of Use (LoU) (Interview) An interview protocol developed by Hall and Hord (2006) to determine a level of use of an innovation addressed eight possible levels (ranging from Nonuse to Renewal) was used with teac hers from each Stage of Concern. Early questions in the protocol were designed to categorize participants as either users or nonusers of the innovation. Subsequent questions were designed to look more closely at three possible levels of

PAGE 67

67 nonuse. Subsequent questions were also designed to assess five possible levels of use three years after initial training in the earlier effectiveness study. Participants for the Levels of Use interview were selected purposefully based on their scores from the Stages of Concern Questionnaire. Ideally, teachers from varying Stages of Concern were interviewed using the branching interview technique. Innovation Configuration Map (IC) (Obser vations) The final tool was an Innovation Configuration Map, which outlined the degree of fidelity that was unacceptable, acceptable, and ideal according to experts familiar with the innovation. Hall and Hord (2006) recommended using a team of two to seven experts familiar with the development and the intended use of the innovation to create an Innovation Configuration Map. This was completed with members from the original research team of the effectiveness study. The purpose of this map was to present a ca refully developed rubric of different ways of implementing the innovation. This provided the implementers with a picture of what the innovation should look like across time. The actions of the teacher who identified herself as a high user w as observed and the fidelity of her use of the innovation was plotted on the configuration map. Data Collection Once IRB approval was obtained, the appropriate personnel at all five school districts were contacted to explain the present study and to solicit approval for c ontacting the teachers who had participated in the previous effectiveness study. After district representatives gave approval additional district personnel provided assistance in locating potential participants in their current school and grade level.

PAGE 68

68 Que stionnaire Procedures In the first phase of data collection, a mailed questionnaire was sent to potential participants to assess their concerns and perceptions toward using Tools For Getting Along as a preventative problem solving tool. The teachers partic ipated by completing a Stages of Concern questionnaire (Hall & Hord, 2006), which included 35 questions regarding their concerns about implementing the innovation (Appendix A). Prior to this mailing, an email was sent to each potential participant informin g them of the impending study and alerting them that the questionnaire would be arriving shortly. Follow up phone calls, mailings, and emails should have insured a high return rate. A response rate of 70% or more is considered very good for analysis and re porting, so diligent follow up measures were taken to reach this response rate (Babbie, 1990). A second and third mailing took place 2 and 4 weeks, respectively, after the initial mailing. From returned questionnaires, a chart was created to categorize res ponses by stages of concern. Twenty one teachers completed and returned the questionnaire out of 48 possible participants, which was a return rate of 43.8%, which Babbie has noted is an adequate response rate. Teachers from each targeted district and schoo l responded. The sample for the Levels of Use interview was derived from participants at different Stages of Concern. Although a guiding semi structured interview protocol was utilized, respondents were asked for further information when appropriate. All i nterviews were audio taped and transcribed, and member checks were conducted. Observations of self reported high users were conducted to determine fidelity using an Innovation Configuration Map developed to ascertain the fidelity with which Tools For Gett ing Along continued to be used. Additional data were triangulated from the

PAGE 69

69 effectiveness study including treatment fidelity forms, observational data, and school, teacher, and student demographic data. The Stages of Concern questionnaire was sent to 48 el ementary teachers who participated in the previous study. As discussed in the previous chapter, each completed questionnaire was analyzed in accordance with Hall, George, and (Ap pendix C). Once the data were entered into section F of the scoring guide, a concern profile was produced. This profile provided a snapshot of which stage of concern the teacher was in with regard to changing their instructional practice. Interview Procedu res and Protocols Based on completed questionnaires, the answers were scored and a stage of concern with the implementation was determined for each teacher. Teachers were purposefully selected to participate in a Levels of Use interview based on their stag e of concern. Interviews were conducted with teachers whose scores reflected different levels of concern. These interviews were conducted before or after school, or during the ool districts had a shortened school day on Wednesdays, and this time was used to behavioral problem solving inter vention, Tools For Getting Along. Branching interview questions were used to explore the use or non use and the exte nt of use of the innova tion ( Appendix B). These interviews lasted approximately one hour and were audio taped with consent. Following th e completion of the interview, the interview notes were transcribed and a first level member check was conducted for accuracy. Once all interviews were transcribed, data

PAGE 70

70 were analyzed to determine their level of use of Tools For Getting Along. A coding sh eet was developed using descriptors for the Levels Of Use to assist in determining the most appropriate level for each participant. A table was created classifying the teachers by their continued level of use of the innovation. The self reported high user was then observed using an Innovation Configuration Map to determine if Tools For Getting Along was being implemented with fidelity. Direct Observation Procedures The teacher selected via the Level of Use interview as a high user was observed on four separ ate occasions implementing the Tools For Getting Along curriculum. Each observation lasted approximately 30 60 minutes. A second observer, trained in Tools For Getting Along, provided inter rater reliability by simultaneously observing the lessons. Suffici ent time at each site occurred to gain saturation of information regarding the fidelity of implementation. The first observation was conducted to understand the context of the learning environment and to establish a rapport with the teacher and students (M erriam, 1998). A brief talk occurred after each observation to obtain the the teacher was asked if the current context differed from the effectiveness study context A treatment fidelity form, developed specifically for the effectiveness study, was used to gauge the extent of implementation. Data gathered from the treatment fidelity forms were plotted on the Innovation Configuration Map, which was created with guida nce from members of the original effectiveness study. This map was used to determine if the self

PAGE 71

71 s looked to see if curriculum overheads and student workbooks were being used. Throughout this follow up study, periodic credibility checks were conducted to insure unexplored bias did not affect this study (Anfara, Brown, & Mangione, 2002). Member checks with participants were conducted following the transcriptions of interview data so that I could verify that I accurately understood their concerns and behaviors regarding the implementation of Tools For Getting Along. Phone calls were placed to the partici pants to determine accuracy of my thoughts and analysis of the data. Sufficient time in the field was spent to observe fidelity of the participant who reported high levels of use. The entire lesson was observed along with spending approximately 15 minutes prior to and after the lesson in order to get a feel for the learning environment. In addition, the data from the interview and observation were triangulated with fidelity data collected during the effectiveness study. For example, the profile of the parti cipant in the original study was reviewed to determine if she was a strong implementer during the effectiveness study. Data Analysis Data were analyzed across three dimensions. First, the Stages of Concern data were analyzed in accordance with Hall, Georg recommendations for scoring the SoC Questionnaire (Appendix C). Five statements of the questionnaire represented one of each of the seven stages of concern. This information provided descriptive statistics about the parti implementing the Tools For Getting Along innovation. Second, a branching technique was used to a nalyze interview responses ( Hall & Hord, 2006). A preliminary level of use classification was the outcome of this analysis, based on individual responses to the

PAGE 72

72 series of questions in the Levels of Use Interview protocol. Third, data from the Fidelity Observation Form were analyzed and the one participant s behavior was plotted on the Innovation Configuration Map to assess the fidelity with which the self described high user was sustaining the practice. The interview transcripts were read carefully to identify levels of use categories as described earlier. These categories were finalized and the transcripts were coded. Peer reviewers as sisted with categorizing participant statements into levels of use categories. Insuring inter rater reliability and screening for disconfirming evidence prevented the creation of inconsistent categories. Narrative text and tables were used to report the d ata of this case study. The narrative was presented in a linear analytic structure (Yin, 2003). Within this structure, the overall findings were presented. Next, a discussion followed that divided into three subtopics. The first subtopic included informati on obtained from the Stages of Concern Questionnaire. The second subtopic included information obtained from the Levels of Use Interview. The final subtopic reported observation data an d the observed participant s continued use on the Innovation Configurat ion Map. The observed various components of Tools For Getting Along. The observed teacher was plotted on the Innovation Configuration Map (Appendix D). Information cont ained in each subtopic covered the techniques used, the results from the data collected and analyzed, and the conclusions and implications for further research from the findings. In summation, the CBAM model was used as a systematic approach to determine h ow faithfully an innovation was continuing to be used.

PAGE 73

73 Assurances of Quality This inquiry was conducted as an explanatory, mixed method case study using the CBAM as a guiding framework. The procedures in this study were held to a high standard and were con stantly assessed for quality and rigor. Issues of Entry and Ethics Before beginning the study, approval was obtained from Drs. Smith and Daunic, principal investigators for the effectiveness study, to conduct a follow up sustainability study. Next, Univers survey, interview, and observe selected teachers that participated previously in the effectiveness study. Assurance of Confidentiality All participants signed an informed consent form that outlined the study details and informed them of their right to withdraw from the follow up study without repercussions. Participants were assured that demographic data and answers to any questions would remain confidential. Assessing the Cultural Context To assess th e cultural context, I toured the schools in which the participants taught and where I subsequently conducted observations. Notes were taken regarding student teacher interactions witnessed and general thoughts about the school. While interpreti ng the data obtained through the various CBAM dimensions, I used these data obvious behavior management system was in place schoolwide and to note if there was an instruct ional climate that supported the use of positive student behavior.

PAGE 74

74 Addressing Quality Indicators Brantlinger, Jimenez, Klingner, Pugach, and Richardson (2005) outlined several quality indicators for researchers to use when engaged in Qualitative Research. To insure that the qualitative components of this study were held to high standards, many of these quality indicators were used and documented throughout this study. These quality indicators (i.e., data triangulation and member checks) were also used to h elp Credibility Throughout the study, credibility checks were conducted to insure that un explored bias did not affect this study (Anfara et al., 2002). Within this follow up study, member checks were conducted following the transcription of every Levels of Use in person interview. The time in the field was sufficient inasmuch as all teachers that reported they were high users of the innovation were observed implementing Tools For Get ting Along. In addition, the data from the interviews and observations were triangulated with fidelity data gained during the effectiveness study. Transferability The effectiveness study was focused on a specific cognitive behavioral problem solving inter vention. This follow up study focused on the continued use of that specific intervention. Sufficient descriptions of the districts, schools, and teachers were provided to assist readers in determining the transferability of these findings to their own circ umstances. Dependability This follow up study could be deemed dependable, as a code recode strategy was used to condense the data. Inter rater reliability was established by having another

PAGE 75

75 person review the data and establish Levels of Use categories. In addition, inter rater reliability was established by having another person conduct observations in the classroom during Tools For Getting Along lessons. The second observer was part of the previous effectiveness study and was familiar with the Tools For Getting Along Curriculum. Using the Innovation Configuration Map and TFGA: Treatment Integrity Checklists developed for the effectiveness study, both observers matched with 100% reliability. Given these significant scores, no further observations were dee med necessary. I was alert to and cognizant of prior knowledge as a member of the research team for the previous effectiveness study. Internal and external value constraints were recognized and addressed so as to minimize their influence in the follow up study. My continued to implement a promising practice once the researchers have concluded their study.

PAGE 76

76 Table 3 1 Number of Elementary Schools Across Five Districts in Follo w Up Study District A B C D E # of Elementary Schools 31 32 12 7 9

PAGE 77

77 Table 3 2. Florida Comprehensive Assessment Test (FCAT) 2009 Scores for Five Districts in Follow Up Study Grade 3 4 5 6 District A Reading # T ested 2,137 1,992 1,999 1,931 Total Score (Average) 317 325 309 313 Mathematics # Tested 2,139 1,994 2,000 1,930 Total Score (Average) 343 330 332 314 District B Reading # Tested 3,277 3,289 3,120 3,095 Total Score (Average) 314 320 308 311 Mathematics # Tested 3,275 3,290 3,118 3,099 Total Score (Average) 339 329 332 309 District C Reading # Tested 881 856 840 850 Total Score (Average) 311 319 300 301 Mathematics # Tested 881 855 840 849 Total Score (Average) 333 327 327 297 District D Reading # Tested 472 453 427 438 Total Score (Average) 303 320 302 305 Mathematics # Tested 473 453 424 438 Total Score (Average) 330 320 326 313 District E Reading # Tested 839 767 724 735 Total Score (Average) 320 325 314 310 Mathematics Grade 3 4 5 6 # Tested 839 769 726 734 Total Score (Average) 338 326 334 309 Passing scores for the Reading FCAT are as follows: third grade 284 and up; fourth grade 299 and up; fifth grade 286 an d up; and sixth grade and up. Passing scores for the Math FCAT are as follows: third grade 294 and up; fourth grade 298 and up; fifth grade 326 and up; and sixth grade 315 and up.

PAGE 78

78 Table 3 3. Student Ethnic and Gender Information across the Five Districts White Non Hisp Blk Non Hisp Hisp Asian/ Pac. Isldr Am. Ind/ Alska Native Multi racial Total Fem. Total Male District A 27,111 13,562 2,508 1,095 107 1,479 48.6% 51.4% District B 41,600 9,939 5,126 440 316 1,462 48.4% 51.6% District C 11,747 4,135 1,713 93 73 325 47% 53% District D 6,556 1,233 469 31 42 143 48.4% 51.6% District E 10,874 2,874 491 81 88 294 47.8% 52.2% Table 3 4. Percentage of Students Receiving Free or Reduced Price Lunch in the Five Districts in the Follow Up Study District A District B District C District D District E 44.4% 53.9% 67.6% 61.5% 53.9%

PAGE 79

79 Table 3 5. Participant demographic information gathered during effectiveness study Teacher Age Gender Ethnicity Years Experience A 27 F W 4 B 54 F W 4 C 45 F W 13 D 34 F W 7 E 43 F W 2 F 32 F W 3 G 43 M B 18 H 50 F W 19 I 50 F W 3 J 48 F W 19 K 32 M W 3 L 41 F W 12 M 27 F W 3 N 48 F W 7 O 28 F W 3 P 60 F W 20 Q 39 F W 4 R 44 M W 2 S 23 F W 2 T 32 F W 7 U 52 F W 13

PAGE 80

80 CHAPTER 4 RESULTS This st udy was designed to address a serious problem in schools: Teachers are expected to use promising and/or proven instructional practices, however, research suggests there is a gap between what teachers know they should use, and what they actually put into pr actice. This study used a mixed method design using the Concerns Based Adoption Model (CBAM) framework (Hall & Hord, 2006). The CBAM framework was used within an explanatory case study design. The focus of this inquiry was on the instructional behaviors of teachers, specifically investigating whether teachers, trained in the implementation of a promising practice, continued to use this practice faithfully after several years. Findings ubsequent behavior that should inform the translation of research to practice. In this study, teachers were minimally concerned about this intervention. Participants were aware of the innovation, but unconcerned about implementing it because most had not s ustained implementation past the time frame of the initial study. Most teachers felt informed about the intervention, but expressed doubts about its feasibility in their current classrooms. Several teachers were still using concepts and vocabulary from Too ls For Getting Along, but only one teacher was fully implementing this promising practice. It is important to note that this teacher was using the intervention in an alternate setting. She was using it in after school tutoring sessions with approximately 1 1 fourth and fifth grade girls versus a classroom with a full complement of students. Based on observations, she has sustained the use of Tools For Getting Along with high fidelity.

PAGE 81

81 nnovativ e a nd Promising P ractice? Stages of Concern profiles were developed from the questionnaire data provided by the 21 participating teachers. The participants were representative of the original group of teachers who were trained to implement Tools For Gettin g Along in the original study. The data were scored and plotted to illustrate a concern profile for each participant. All 21 teachers expressed that they were aware of the innovation but unconcerned with issues surrounding its implementation. Although they were primarily unconcerned with implementation issues, they varied in their concerns about the innovation itself. Fourteen teachers registered responses that indicated concerns unrelated to the innovation Of the remaining 7 teachers, 3 responded with con cerns primar ily at the Informational, Personal, and Management levels; the remaining 4 teachers registered responses with high scores indicating concerns across a number of dimensions. Using percentile scores from the SoCQ Quick Scoring Device, results sho wed the Unconcerned stage (m = 77.33%) emerged as the highest rated area; concerns with the Consequences of using the innovation was the lowest rated area (m = 19.62%). Three other areas of concern were rated at a medium to high level range: Information co ncerns (m = 50.62%), Management concerns (m = 47.81%), and Per sonal concerns (m = 36.86%) ( Figure 4 1). Participants were grouped by levels of Unrelated, Self, Task, these four levels. Participants in the Unrelated level had concerns that did not center on teaching or teaching

PAGE 82

82 eaching but with an egocentric frame of reference in terms of their own personal experience and whether articipants with Task concerns were focused on the management of the innovation. A typical response was: Few participants responded at the Impact level where concerns would be on their students and what they could do to be more effective in improving student outcomes. At this level, My adding some new interest centers that might attract those chil implementing the innovation. They expressed personal concerns about information pertaining to the innovation and concerns with managing the task. Conc erns about the impact of the innovation were much lower. Based on these questionnaire scores, six teachers became the focus of the next part of the analysis. These teachers were chosen because their scores fell in different Stages of Concerns as illustrat ed in the following concern profiles. A seventh profile is included in this section because it illustrates a different and commonly occurring that the Stages of Concern determined during the Levels of Use interview.

PAGE 83

83 Unrelated escribe a person who is not aware of the change being proposed, is not interested, or actively resists learning about it (Hall & Hord, 2006). Analysis of the concerns profiles indicated that 77% of the respondents were unconcerned about the implementation of the practice. This pattern of concern suggests that a majority of teachers were aware of the promising practice but were not actively interested in it. The fact that more than three quarters of the respondents fell into this stage of concern is not surp rising. All of the teachers involved in this study were previously trained to implement the innovation, and had implemented it for a full school year under the guidance of a research team. These teachers had multiple opportunities during their participatio n in the effectiveness study to increase their awareness by discussing concerns about the technical aspects of utilizing the practice during the implementation process. Teacher A provides an example of a participant who is aware of the innovation, but prim arily unconcerned about its implementation. Unconcerned S tage. Teacher A is a 31 year old Caucasian female with 8 years of teaching experience. Based on the Stages of Concern Questionnaire, Teacher A was in the Unconcerned stage. Teachers at this stage gen erally have little concern about the innovation. Her scores on the SOCQ reflect concerns about the Information and Management of the innovation, which means she had some concerns regarding the general characteristics, effects, requirements, and time demand s of the curriculum, as illustrated by her peak scores at the Information and Management stages of concern in her concern profile (Figure 4 2).

PAGE 84

84 Self Within this level, teache rs are aware of the innovation and interested in learning more about the promising practice. In addition, they are uncertain about the demands of the innovation and their ability to adequately meet those demands. Several participants expressed concerns at this level. Informational stage. Teacher B is a 57 year old Caucasian female with 7 years of teaching experience. concern was Informational. Teachers at this stage typically expressed an awareness of the innovation and an interest in learning more about the innovation (Hall & Hord, 2006). ranging from a high of 40 to a low of 5 unlike Teache r A who had concerns ranging from a high of 84 to a low of 26, which means that Teacher B had very few concerns regarding Tools For Getting Along (Figure 4 demonstrate that Teacher B was no longer devoting much thought or energy to this promising practice. Unconcerned, informational, and personal stages. Teacher C is a 49 year old Caucasian female with 17 years of teaching experience. Based on the Stages of Concern Questionnaire, Teacher C had multiple concerns: Unconcern ed, Informational, and Personal. Teachers at these stages generally have little concern about the innovation, have a general interest in learning more about the innovation, and are uncertain about the demands of the innovation, their ability to meet those demands, and their role within the innovation (Hall & Hord, 2006).

PAGE 85

85 As illustrated in her concerns profile, Teacher C had high concerns ranging from a high of 95 to a low of 38, which demonstrates that she was very concerned about some aspects of implementi ng TFGA (i.e., how much time and effort she would have to devote to the implementation process) while not concerned about other areas (i.e., student outcomes) (Figure 4 4). Personal stage. Teacher F is a 33 year old Caucasian female with 4 years of teachin g experience. concerns were Personal. Teachers at this stage are uncertain about the demands of the innovation, their ability to meet those demands, and how hard they might ha ve to work where her concerns lie with a range from a high of 78 to a low of 43, which shows that she was more concerned about how this would affect her ver sus student outcomes (Figure 4 7 ). Task stage of concern, which is when teachers are concerned about the management of the innovation (i.e., organizing, scheduling, and time demands). At tention is focused on the processes and tasks of using the innovation and the best use of information and resources. At the conclusion of the Stages of Concern Questionnaire, teachers were asked to address one open ended question regarding additional con cerns pertaining to the implementation of Tools For Getting Along as an innovative and promising practice. Twelve of 21 teachers answered this question, and one overarching theme emerged.

PAGE 86

86 Teachers expressed concerns with the time necessary to implement Too ls For Getting Along with the competing priority of meeting extensive academic requirements. These concerns coincide with the Management concerns of efficiency, organizing, managing, scheduling, and time demands (Hall & Hord, 2006). One would think that a majority of the teachers would have Management concerns instead of being Unconcerned. This could be due to the fact that 95% of the teachers are not currently implementing this promising practice. If they were currently involved with implementing the innov ation, their main concerns might be centered on the management of using Tools For Getting Along. One teacher stated: It was very time consuming. We are held to a very tight schedule in teaching academics (Teacher T) Another participant stated: When using t he Toolbox approach, I enjoyed it. However, time is not available to do it effectively. A school wide approach could be more beneficial (Teacher E) Eleven of 21 teachers made similar comments in voicing their concerns over the amount of time it takes to im plement Tools For Getting Along. Although they stated the impact of the innovation was effective for their students based on reduced numbers of referrals, classroom distractions, and increased time on task, the time constraints were a major factor in their discontinued use of the practice. Unconcerned and management stages. Teacher D is a 35 year old Caucasian female with 8 years of tea ching experience. Based on the results of her Stages of Concern Questionnaire, Teacher D is in two stages: Unconcerned an d Management. Teachers at these stages had little concern or involvement with the innovation and their attention was focused on the potential

PAGE 87

87 concern profile illustrates a wi de range of concerns (Figure 4 5 ). Management stage. Teacher E is a 45 year old Caucasian female with 4 years of teaching experience. Based on the Stages of Concern Questionnaire, Teacher E had Management concerns. Teachers at this stage are focused on t he task of implementing the innovation. Teachers in this stage typically pay close attention to organizing, managing, scheduling, profile illustrates her high Awareness co ncerns but also highlights major concerns pertaining to Informational, Personal, and Collaboration, and these concerns ranged from a high of 92 to a low of 48 (Figure 4 6 ). This range of concerns illustrates that Teacher E was concerned not only with her a bility to implement the innovation effectively but having time to collaborate with her colleagues who were implementing the same promising practice. Impact Consequences, Collaboration, and Refocusing had a noticeable drop in concern for most teachers with Consequences being the lowest overall (19.62%). In the own use of the innovation. This was not a concern for most of the teachers who responded to the questionnaire, as a maj ority w ere not implementing the practice. These teachers are not coordinating or cooperating with others regarding the use of Tools For Getting Along, nor are they focused on the exploration of more universal benefits from the innovation.

PAGE 88

88 It was sometimes difficult to ascertain one specific stage of concern for respondents, in accord with Hall and Hord (2011). At times, it appeared that two or three stages of concern could be highlighted for a respondent based on the Stages of Concern Questionnaire. Hall a Profile due to the peaks and valleys in the configuration. The profile of Teacher G is a Unconcerned issues, he also had peaks in ot her stages with concerns ranging from a high of 96 to a low of 4 and peaks at Awareness, Management, and Refocusing (Figure 4 8). in the next phase of the study. Hall and H ord (2006) pointed out that school administrators could use information from this type of uneven profile to ascertain a Overall, teachers noted being aware of the innova tion with additional concerns focused on managing the innovation. When given the opportunity to discuss their concerns, only management concerns specifically with time constraints were expressed. Teachers felt they did not have enough time during their sch ool day to implement Tools For Getting Along appropriately and simultaneously cover required academics. Question Two: How do These Concerns Influence the Level to Which T e achers Report Implementing the P ractice? This question of how these concerns influe nce the level to which teachers report implementing the practice is addressed by looking at the Patterns of Practice of the participants. These patterns include patterns of concern and levels of use reported by these teachers.

PAGE 89

89 Levels of Use Interview Acco rding to Hall and Hord (2006) there were eight levels of use of an innovation that teachers passed through. Unlike the Stages of Concern Questionnaire, which probes thoughts and feelings, the Levels of Use Interview is looking at the behavior of the teache r. In this study, individual interviews were conducted with Teachers A F to ascertain their current level of use of Tools For Getting Along. As previously discussed, there were three nonuse levels (Nonuse, Orientation, and Preparation) and five use levels (Mechanical, Routine, Refinement, Integration, and Renewal) within the CBAM of use was determined for these six teachers. This protocol enabled one to quickly determine i f the teacher was a user of an innovation. Additional questions were asked during the interview to gain clarity regarding other concerns or thoughts about the innovation that the teacher had yet to discuss. The level at which the six teachers were using th e innovation was determined based on an analysis of the transcripts from their 1). For the purpose of identifying patterns of implementation, the following discussion will patterns of practice. Nonusers are discussed first followed by users. NonUsers According to Hall and Hord (2006), there are three different types of nonusers. It is important for change facilitators and school administrators to understand what level of nonuse a teacher is in, so they can provide the appropriate support and assistance to help each teacher move through the change process.

PAGE 90

90 Level of Use 0: Nonuse. At this level of non use, teachers exhibit no behavior related to the innovation. No currently in the Nonuse level. Although she was familiar with the innovation and was trained 3 years ago, she had no current involvement with the innovation and had no plans of using it in the future. She was no longer using Tools For Getting Along, because her school district was mandating the use of another intervention. As we continued the interview and di scussed how the school was using PBS, the discussion turned to the school wide use of multiple tiers of behavioral support within an RTI framework and the interventions being used with struggling students at Tier 2 and Tier 3. When asked if she would consi der using Tools For Getting Along with these students, she stated: I hope that they would use something like Tools For Getting Along, because trouble all the time, and they obvious ly need more strategies in place. I think it would be great. I feel like those kids who already found their way, innovation at of the words and the different strategies that the other kids have already learned. I would like to see it more as a Tier 2 intervention. In summary, this teacher is aware of Tools For Getting Along, but unconcerned with using it. She is a nonuser who is not interested in continuing to implement this practice with her entire class. She was unaware t hat she could use Tools For Getting Along as a Tier 2 or 3 intervention with struggling students.

PAGE 91

91 Level of Use I: Orientation. Three of the six teachers interviewed discussed behaviors that categorized them as nonusers in the Orientation level, which is the state in which the user has explored the value and demands of the innovation. When asked if she currently used Tools For interview, she revealed that her school district had a district wide mandatory curriculum in place, which was similar to Tools For Getting Along. Teacher B made it clear throughout the interview that she would have continued to use Tools For Getting Along if she were not required by her district to implement the curriculum prescribed by her the students really enjoyed the role plays in the Tools For Getting Along curricul um. Similar to Teacher B, Teacher E was using a mandatory district wide program instead of Tools For Getting Along. Although Teacher E was also a nonuser, she was classified in the Orientation level because she, like Teacher B, stated that she continued to mainly implementing the district wide program. When asked which pieces of the innovation she still used, Teacher E discussed the impact of Tools For Getting Along on her st udents: then calming down in a structured way to figure what is going on with yourself. In addition, I t each them the tools to calm down. Similar to Teacher E, Teacher F was classified as a nonuser in the Orientation level based on her responses during the branching interview. Teacher F was not using the curriculum per se, but was using the concepts.

PAGE 92

92 Teache r F clarified her choice not to use the Tools For Getting Along curriculum this year. She felt that the behavior problems in her class were not as severe as with the previous class. She did not have to spend as much academic time addressing behavioral conc erns. When asked what she liked the most about the Tools For Getting Along curriculum, Teacher F stated: Students have to be involved, and they have to give their honest opinions. They need to write things down. It really seemed to help the students to phy sically write things down, and stop and go back and think about what they did and how to do it differently. Level of Use II: Preparation. The statements that Teacher D shared during her interview helped classify her as a nonuser of the curriculum in what Hall and Hord (2006) describe as the Preparation school as when she was a participant in the previous effectiveness study. She continued to teach fifth grade in the same district, but at a newly opened school. When asked why she discontinued the use of Tools For Getting Along in its more in depth throughout the remainder of the intervie w. When it was explained that she could use Tools For Getting Along once a week and would not have to complete all of the student and teacher measures that were involved in the previous study, she was very open to implementing the program once again. I co uld definitely do Tools For Getting Along once a week, because I would fit it in during my RTI time. Generally, we are supposed to meet with the kids who are in RTI groups at least three times a week for 30 minutes at a time. It definitely would be benefic ial, and one day a week would be doable. (Teacher D)

PAGE 93

93 Following the completion of the audio taped interview, Teacher D stated that she would be interested in implementing Tools For Getting Along at her new school. However, she commented that she left all of her curriculum supplies (i.e., scripted lessons, overheads, etc.) at her previous school when she left. She followed up with requests for the Tools For Getting Along curriculum, student workbook, and overheads. It was discussed how she could secure the cu rriculum for use in her classroom. After receiving permission from Dr. Daunic, I was able to send her an electronic version of all of the Tools For Getting Along materials. She was classified as a nonuser in the Preparation level, because she was preparing to use the innovation once again. In addition, she mentioned that she would recommend Tools For Getting Along to her RTI task force and would contact the previous researchers if they were interested in implementing the promising practice at this new site. All of these teachers, who were nonusers, noted that Tools For Getting Along was effective at improving student behavior, yet discontinued the use within one or two years of being trained to use it in the effectiveness study. It is important to note that all of them misunderstood that the student and teacher measures used during the effectiveness study were not part of the curriculum. Once this issue was clarified, they stated that this curriculum was not as time consuming as they once thought, and it wou ld be manageable in their already rigorous academic schedules. Users Hall and Hord (2006) identify five different types of users, which is important to identify during the change process. The key to making the distinction among the five different users is the type of adaptations that are being made by the user in relation to his or her use of the innovation or in the innovation itself. Of the 6 teachers interviewed,

PAGE 94

94 only 1 teacher continued to implement Tools For Getting Along after the end of the previous study. Level of Use IVA: Routine. Teacher A stood out from the other 5 teachers in that she was the only teacher interviewed who reported currently using the curriculum as intended. Teacher A, whose Stage of Concern was Awareness, was a user in the Rout ine level. Therefore, she was a user of the innovation, had mastered the use of the innovation, and was using it on a regular basis. At the Routine level, users are not making any changes or adaptations to the innovation and tend to use it as was predeterm ined by the researchers. Teacher A reported implementing this innovation once a week with fourth and fifth grade girls that she tutored in an after school program. She described how she covered one lesson on the first session and then reinforced the concep ts on the second day with the students. From her perspective, having a discussion with them about what they covered during the previous session provided reinforcement and an opportunity for the girls to generalize the concepts to other settings. During the se discussions, the students discussed their personal problems at school, home, etc., and were given the opportunity to problem solve ways to address those problems in different settings in the future. She allowed them to discuss their own problems and sol utions, and tied their experiences back to what they just learned in the Tools For Getting Along curriculum. Patterns of Practice Of the six teachers interviewed using the Levels of Use branching protocol, only one, Teacher A, was classified as a user of the Tools For Getting Along Curriculum. In the course of conducting these six interviews, several overarching patterns became

PAGE 95

95 (1997) framework, which depicts vary ing factors that are necessary for sustained use of evidence based practices. Effectiveness. One factor Gersten et al. highlighted was linking changes in teaching to student learning. All six teachers expressed that they felt Tools For Getting Along was a n effective curriculum. They noted that they could see noticeable positive changes in their solving steps outlined in the curriculum and reduce the number of inappropriate behaviors. Several teachers explicitly supported the value of Tools For Getting Along for their students in a brief written response on the Stages of Concern Questionnaire or in the program that yielded Teacher E discussed how effective she tho ught Tools For Getting Along was for her students due to the fact that they were able to generalize the concepts and strategies to other settings. I think it was definitely effective for some of my students. It made them stop and think. They were using it (Teacher E) Teacher D noted that Tools For Getting Along helped build a positive classroom community. Along was that it gave the kids a chance to work through problems when

PAGE 96

9 6 they get angry and discuss possible solutions as a grou p. I felt like it was a huge component to my team building that year. (Teacher D) Feasibility. related to the reality principle. This factor pertains to the feasibility and fit of the practice. Teachers must be able to fit the practice into daily routines of classroom instruction. The biggest complaint voiced by the teachers was that the Tools For Getting Along curriculum was too time consuming. They stated that they could not impl ement the curriculum as intended and complete all of their other required academic activities. According to her brief written response on the Stages of Concern Questionnaire, Teacher E focused on the time management of Tools For Getting Along. It was diff icult to find the time during the day to do it (TFGA), and I wished I to be effective. I struggled with figuring out when to fit Tools For Getting Along into my hectic schedule (Teacher E) Time management was a concern voiced by many of the teachers. For example, ring The one teacher who continues to implement this innovation voiced some time management concerns. I hav e a lot of things that I want to do with this group, and I know I really want them to complete this program, too. I just want to make sure I can get everything done and still meet all of my goals with them. (Teacher A) When asked if she would recommend thi s innovation to other teachers at her school, Teacher A stated:

PAGE 97

97 Yes. Other teachers have the same concerns of trying to have time to set something up into their busy academic schedule. However, I would recommend it to them especially if they have students who are at risk. Outside Requirements. Throughout the course of this study, other patterns began to emerge. District constraint was a pattern voiced by some teachers. The teachers no longer had the choice to implement Tools For Getting Along. Instead, the y were mandated to implement a different curriculum throughout their entire school district. After the research team from the effectiveness study completed the previous study at Teacher ool wide intervention program. This is in direct conflict to what researchers have found to be necessary to sustain promising practices. One principle that has been shown to support success and long term sustainability is active commitment by district and school leaders to the use of promising or proven practices (Gersten et al., 2000). All too often, administrators forget that change is a process not an event (Fullan, 2005). Administrators must be willing to support the implementation of evidence based pra ctices for long enough to witness positive change. When asked if she planned on using it in the future or if she is using something in place of Tools For Getting Along, Teacher C stated: am other programs come along, but we just kind of pull chunks from them. We have a similar innovation at this school and in all the schools in this ginning of the year, it was sort of similar to the program we did with University of

PAGE 98

98 something very similar to that now. It reinforces everything that we did with the Tools For Gett ing Along project. (Teacher B) Partial Use. A final pattern that emerged was that although most teachers no longer implemented Tools For Getting Along as intended, a number of them continued to use concepts from the curriculum. Although they felt they did not have enough time to implement the curriculum, there were concepts that they integrated into their daily teachings. concepts such as having the students write things down an d go back and review their choices. That was one of the most important ideas I learned from Tools For Getting Along. (Teacher F) When asked if she continued to use Tools For Getting Along after the research program ended at her former school, Teacher D st ated: I did actually. There were several things about it that I liked that I continued to use. We did some of the role playing, which the students really enjoyed. I had posters I put up to keep the concepts fresh for the students and myself. This helped th e students monitor the steps involved in Tools For Getting Along. (Teacher D) Although this is n o t a pattern, it is notable that how long ago the teachers were trained did not make a difference with regard to sustainability. Teacher A was trained 4 years p reviously during the first year of the effectiveness study and is the only teacher still implementing the curriculum. In order for evidence based practices to be sustained in the classroom, these patterns need to be explored. Recommendations for how resear chers might bridge this research to practice gap will be discussed in the following chapter.

PAGE 99

99 Question Three: How Faithfully do Teachers Sustain These P ractices? To determine how faithfully teachers sustain ed these practices, members of the research team of the effectiveness study developed the Innovation Configuration Map. This map was used during the four observations of Teacher A to determine the level of fidelity with which she continued to use the innovation. The Innovation Configuration Map consisted of seven components (Appendix D). Not every component was represented in every lesson, but was present in several lessons throughout the Tools For Getting Along curriculum. Teacher A was observed for this study as she tutored a group of fourth and fifth grade girls involved in a 21 st Century after school program. The teacher met with this st Century Community Learning Centers program was a component of th e No Child Left Behind Act (NCLB) authorized under Title IV, Part B, of the Elementary and Secondary Education Act (ESEA)(NCLB, 2001). The primary focus of this program was to provide additional academic enrichment opportunities for children attending scho ols eligible to receive Title 1 funding. This program offered these opportunities a broad array of activities that could be included, such as tutoring services, mathemati cs and science education activities, telecommunications and technology education programs, and character education programs. Teacher A used Tools For Getting Along as part of the character education curriculum. ipated in the effectiveness study 4 years ago. During the time of that study, she implemented the curriculum as a full time fifth grade teacher. Now Teacher A is a full time media specialist in the same school,

PAGE 100

100 but she is implementing the curriculum in an after school group made up of fourth and fifth grade girls. Teacher A was observed on four separate occasions implementing Tools For Getting Along lessons 7 10 over a four week span. Each observation lasted between 30 minutes to an hour. During these obser vations, two separate instruments were used to collect treatment fidelity data. One instrument was the Innovation Configuration Map, and the second was the TFGA: Treatment Integrity Checklist, created for the previous effectiveness study. During the previo us study, treatment teachers were observed implementing Tools For Getting Along lessons, and the research assistants used this form to gauge treatment fidelity. The researcher conducted two observations alone; two additional observations were conducted wit h a research assistant for inter rater reliability purposes. During the first observation (Lesson 7) according to the Innovation Configuration Map, Teacher A implemented the curriculum as intended scoring at an Ideal level on Components 2, 3, 4, and 7. She scored at an Acceptable level for Component 1: Curriculum Overheads, as she did not display all overheads as intended. Components 5 and 6 (Role Plays and Self Talks) were Not Applicable as they were not included in this lesson. During the second observati on (Lesson 8), many of the components were not included in the lesson. Of the two components included (Role Plays and Student Participation), Teacher A scored at the Ideal level. The remaining components (1, 2, 3, 4, and 6) were scored as Not Applicable. A lthough only two components were included in the lesson, Teacher A implemented this lesson as intended and allowed the students

PAGE 101

101 to do the role play on numerous occasions. She wanted each of the girls to have an opportunity to participate in the role play. During the third observation (Lesson 9) conducted by both the researcher and research assistant, the teacher scored at Ideal level for Components 1, 2, 3, 4, 6, and 7. Role plays were not included in this lesson, so this component was scored Not Applicabl e. On the final observation (Lesson 10) scored by the researcher and research assistant, Teacher A scored Ideal level for Components 1, 2, and 7. Components 3, 4, 5, and 6 were not included in this lesson, so they were scored as Not Applicable. Informatio n for all four observations was charted to make it clear how Teacher A performed during each observation for each component (Table 4 2). During the four observations, Teacher A was also observed with the TFGA: Treatment Integrity Checklist. According to t his checklist, she implemented Lesson 7 with 95% fidelity. There were 21 possible tasks in this lesson, and she completed all but board or overhead projector While implemen ting Lesson 8, Teacher A demonstrated 100% fidelity (i.e. there were seven tasks for the teacher to complete in this lesson, and she completed all seven). Both the researcher and research assistant observed Lessons 9 and 10. During lesson 9, students were taught a strategy for defining a problem. This was the only lesson observed that included teacher self talk, an important component of this cognitive behavioral intervention. This lesson had 32 tasks for the teacher to complete. During lesson 10, students were taught a strategy for generating

PAGE 102

102 alternative solutions. This lesson included 21 tasks for the teacher to complete. Both observers scored Teacher A with 100% fidelity for both lessons. During the previous effectiveness study, Teacher A was observed im plementing Tools For Getting Along on 5 occasions. At that time, she demonstrated a wide range of fidelity during these lessons ranging from a low of 43% to a high of 95%. This was the first time Teacher A had implemented this curriculum, which could expla in the large range of fidelity scores. This variable suggests that there is a learning curve when implementing this promising practice. Teachers need time to understand how to implement the various components effectively. In addition, teachers benefit from being able to dialogue with researchers and colleagues about the curriculum in order to implement TFGA appropriately. During the present study, Teacher A was free to choose a tool to implement in an after school tutoring program; had practiced using Tools For Getting Along over the past 4 years; and deeply understood the method, the implementation process, and the meaning of a cognitive behavioral intervention. Discussion of the Findings This study investigat ed whether teachers trained in the implementatio n of a promising practice continued to use this practice faithfully after several years. The content of the following discussion examines how the findings from this study comport with current literature, draws conclusions, and offers recommendations for fu ture research. Finding # 1: Teachers were unconcerned about implementing this innovative and promising practice. Teachers in this study were not concerned about implementing the practice. In fact, all but one participant reported discontinuing its use aft er the initial study was

PAGE 103

103 course of this study, and although some reasons were anticipated, others were more unexpected. For example, it should be noted that the practice ha d not been initiated as their satisfaction with its results for their students might have helped sustain its use. Lack of Implementation Support Firstly, teachers were not supported in the ongoing implementation of this innovation because effectiveness of the intervention was the focus of the first study, rather than the sustainability of the practice as part of school wide change. According to Goggin (1986) there really was no genuine implementation regarding Tools For Getting Along. The intention was not to put the practice into place or to adopt it, but rather to gauge its effectiveness. There was no paper implementation, because compliance was not being monitored exce pt within the study. There was a focus on process implementation, but not with the rationale for changing practice immediately. More explicitly, performance implementation was a goal of the initial study, but only within the frame of that inquiry. During t he effectiveness study, the only participants were teachers who were expected to implement the curriculum after attending a two day training. There was no shared responsibility with administrators and thus no real buy in from others in the school. Principa ls had given permission for research to be conducted in their schools for the effectiveness study, but that was the extent of their participation. In addition, in order for a cognitive behavioral intervention to be fully implemented, Gerber and Solari (200 5) noted there needs to be school wide acceptance of the practice associated with the behavioral program. In the effectiveness study, the remaining

PAGE 104

104 faculty, staff, and administrators were not trained and were therefore uninformed about the practice. Give n the intent of the prior study, it was not surprising to find that use of the practice had not been sustained; however, the extent to which the participants were disconnected from the practice, and as a result unconcerned with this implementation, was sur that very few innovations reach the institutionalization stage where they become a This finding suggests that if researchers hope to have teachers adopt proven or promising practices, they have to be mindful of the realities of the change process in the organization of schools. Hall and Hord (2006) noted that attention must be on the individuals who are involved in the change p rocess and how they experience that process. In addition, change facilitation should be a shared responsibility that involves all the stakeholders (i.e., teachers, administrators, students, as well as researchers) at ings pointed to the need for administrative support in order to implement a new curriculum appropriately and sustain the curriculum figures in successful implementatio n of new programs; (b) they appear to be influential in determining the degree of implementation in their buildings and district; and, (c) their influence on implementation is greatest when they demonstrate a change facilitator style as opposed to simply m anaging or responding to change. Gersten et al. (2000) noted that active commitment by the school administrators and district leadership is a key element to long

PAGE 105

105 to the present study. Sin ce a majority of the teachers stated that this promising practice effectively reduced the number of negative behaviors and increased the problem solving ability of their students, there is reason to believe that the sustainability of Tools For Getting Alon g would have been much greater had the principals at the various schools been active members of the change process. Questioning the Feasibility of Implementation Secondly, as a group, teachers were aware of the practice but took issue with the feasibility of potentially managing it in their classrooms. Teachers stated that if they were to use the curriculum, they would have concerns about time constraints. Researchers alike are concerned about the feasibility of innovations in the classroom setting. Instit ute of Educa tion Sciences (IES) Director John Easton (2011) stated recently in a speech given at the University o f North Carolina at Chapel Hill that his main priority is to work more collaboratively with practitioners and policy makers to build partnershi ps that engender relevant, useful research. In addition, the National Board of Education Sciences (NBES, 2010) endorsed a new set of research priorities linking research to practice for IES that highlight this need. The overall mission of the Institute is to expand fundamental knowledge and understanding of education and to provide education leaders and practitioners, parents and students, researchers, and the general public with unbiased, reliable, and useful information about the condition and progress o f education in the United States; about education policies, programs, and practices that support learning and improve academic achievement and access to educational opportunities for all students; and about the effectiveness of Federal and other education programs. The Institute seeks to understand causal linkages to the greatest extent possible by conducting or sponsoring rigorous studies that support such inferences.

PAGE 106

106 Making research useful is the practical goal of federal policy; however, changing practic e remains easier said than done. Some scholars assert that the CBAM method can be utilized to facilitate the change process. implementing the practice. Although teachers in this study were generally unconcerned with the innovation, they had various explanations for why they were no longer implementing Tools for Getting Along. According to Fixsen et al. (2005), teachers who participate in professional development workshops, impleme nt the practice with fidelity, and who have more opportunities to discuss implementation concerns, have greater success in implementing and sustaining the newly acquired practice. However, in this study, 95% of the teacher respondents failed to sustain thi s promising practice after the research team vacated their school. Easton (2011) stated that good schools are learning organizations that value strong leadership; encourage and support innovation; use data for continuous improvement; hire quality teachers, support and develop them, and encourage their collaborative efforts; make good programmatic decisions and constantly change to improve student outcomes. Additional research would need to be conducted to determine what additional supports could have been p ut in place at each school in order for the implementation of this curriculum to continue after the conclusion of the effectiveness study. Six teachers were profiled because their responses were representative of the different Stages of Concern. However, a ccording to the Stages of Concern Awareness, Informational, and Personal concerns. After implementing this promising

PAGE 107

107 practice for an entire school year, one would think that th ese teachers had moved through self and task concerns and were now focused more on impact concerns (i.e., effectiveness of the practice). These results could be explained by the fact that these teachers were participants in the effectiveness study up to fo ur years ago. It is evident that their concerns did influence their low level of use of this promising practice; however, additional factors contributed greatly to the poor sustainability of Tools For Getting Along. In addition to the self concerns describ ed during the Levels of Use interview, most of the teachers discussed task concerns such as management of the promising practice. When the teachers discussed the level of difficulty of implementing Tools For Getting Along, they focused on the pre and post measures required during the effectiveness study. It was apparent from their concerns that they had not separated the curriculum itself from certain aspects required for rigorous data collection within the previous research study. During the interviews, t his fact was discussed with each participant, and it was as if they had a revelation. After this discussion, all six of the teachers stated that implementing this promising practice would be manageable and not as time consuming as they had originally thoug ht. Finding # 3: The teacher who sustained this practice did so with a high level of fidelity. As previously mentioned, the sustained use of an instructional practice is directly related to practices teachers see as actually helping difficult to teach stu dents (Berman & McLaughlin, 1976). Only one teacher in this study sustained this promising practice, and with a high level of fidelity, following her participation as a member of the effectiveness study. Teacher A demonstrated practice mastery, which Kinde r et al. (1989) noted is an essential component for sustained use of an evidence based

PAGE 108

108 an effective program especially for at effe ctiveness study, Teacher A realized that implementing this promising practice was having a positive effect in her classroom for all her students especially her more difficult to 1997) statement that teachers will implement and sustain a practice if it is effective for most students including those with disabilities. In addition, Teacher A made reference to the fact that Tools For Getting Along allows the students to bring in their own issues and develop possible solutions as a group using role plays and other tools. This comports with Polsgrove and Smith (2004) who stated that problem solving strategies that teach to to ts and assist with the generalization of newly learned skills to other settings. Summary This chapter presented the findings of this study based on data collected and analyzed from three components of the CBAM. Each component addressed a specific resea rch question. Using the Stages of Concern Questionnaire, 21 teachers were assigned to a specific stage of concern with most teachers being unconcerned about the use of the innovation. Six teachers representing different stages of concern were interviewed u sing the Levels of Use branching interview to determine if their concerns influenced their use of the innovation. Five of the six teachers were considered nonusers of Tools For Getting Along. Four patterns of practice were identified based on the responses from the questionnaire and from the interviews. The Innovation Configuration Map was the final component used while observing one teacher who continued to implement Tools For Getting Along. Triangulating data from the Innovation

PAGE 109

109 Configuration Map with add itional data sources revealed that Teacher A continues to implement Tools For Getting Along at what the developers of the innovation described as an ideal level. First, teachers in this study voiced concerns mainly around time constraints when implementing Tools For Getting Along. However, after conducting interviews with several teachers, it became clear that the time constraints were not related to the innovation itself. For the purposes of the previous effectiveness study, the teachers and students were required to complete pre and post measures that were quite time consuming. In the present study, the teachers did not realize that implementation of this promising practice in their current classrooms would not include these lengthy measures; they were un able to separate the curriculum from the research requirements. Second, most teachers did not realize that they could use Tools For Getting Along in other ways other than whole class implementation, as was the case in the effectiveness study. Once they rea lized its potential application, all agreed this curriculum could be very useful within an RTI framework as a Tier 2 or Tier 3 intervention for students who need extra assistance with problem solving in the classroom. The findings presented in this chapter have been discussed with regard to the literature I n Chapter 5 c onclusions, implications, and recommendations for both researchers and practitioners are presented.

PAGE 110

110 Figure 4 1. Percentages of Concern by stages for all respondents

PAGE 111

111 Figure 4

PAGE 112

112 Figure 4

PAGE 113

113 Figure 4

PAGE 114

114 Figure 4

PAGE 115

115 Figure 4

PAGE 116

116 Figure 4

PAGE 117

117 Figure 4

PAGE 118

118 Table 4 Levels of Use Teachers 0 Nonuse Teacher C I Orientation Teacher B Teacher E Teacher F II Preparation Teacher D III Mechanical Use IVA Routine Teacher A IVB Refinement V Integration VI Renewal Table 4 Component Ideal Acceptable Unacceptable Not Applicable Curriculum Overheads Observation 3 Observation 4 Observation 1 Observation 2 Scripted Lessons Observation 1 Obse rvation 3 Observation 4 Observation 2 Student Workbook Observation 1 Observation 3 Observation 2 Observation 4 Toolkits Observation 1 Observation 3 Observation 2 Observation 4 Role Plays Observation 2 Observation 1 Observation 3 Observation 4 S elf Talks Observation 3 Observation 1 Observation 2 Observation 4 Student Participation Observation 1 Observation 2 Observation 3 Observation 4

PAGE 119

119 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE RESEARCH vidence based practices has become a major focus of federal education policies including the Individuals with Disabilities Educational Improvement Act (IDEA, 2004), and a priority for funding agencies. The effectiveness of proven or promising practices is documented in peer reviewed studies, B., Landrum et al., 2008; Tankersley et al., 2008). Tools For Getting Along can be considered a promising practice since evidence is b eing gathered regarding its effectiveness. According to Cook, B., Landrum et al. (2008) some practices have a proven record but have yet to be replicated enough to be considered evidence based practices, while some practices are supported with promising ye t preliminary data of their effectiveness. For a practice to be evidence based, Cook et al. stated there must be high quality, experimental research supporting the effectiveness of the practice; the experimental design must include adequate controls; and r esults must demonstrate improved student outcomes. Once the support of research teams is removed, however, teachers often fail to implement practices faithfully or for an appropriate length of time (Cook, Tankersley, & Harjusola Webb, 2008). As a result, t eachers may miss the potential benefits of these practices are unfulfilled. The purpose of this study was to build upon the current literature by examining how teachers who have been trained to implement a promising practice continue to use that practice over time. Specifically, this study explored the following questions: (1.) practice?

PAGE 120

120 (2.) How d o these concerns influence the level to which teachers report implementing the practice? (3.) How faithfully do teachers sustain these practices? This study was designed to complement an effectiveness study conducted previously in elementary schools in a l arge southeastern state (Smith & Daunic, 2006). The Concerns Based Adoption Model (Hall & Hord, 2006) was used to assess Along, a cognitive behavioral intervention introduce d during the effectiveness study, and their continued use of this promising practice. A questionnaire was used to determine with selected participants were conducted to d etermine the current level of use of the innovation; and an Innovation Configuration Map was developed and utilized during observations to determine the level of fidelity with which the innovation was still being used. Limitations of the Study The intenti contribute to or impede the adoption and sustainability of a promising cognitive behavioral intervention designed to be used with fourth and fifth grade general education students. Due to the limited application of the intervention to students at these grade levels who typically range from 9 years to 11 years of age, transferability of the findings cannot be attributed to teachers of younger or older students. It should be noted that the f indings might have been different if Tools for Getting Along were already proven as an evidence based practice rather than a promising practice. In addition, the results of this study were based on a small number of elementary school teachers in

PAGE 121

121 general ed ucation settings. Factors identified as inhibiting or contributing to the sustainability of the specific practice in this study, Tools For Getting Along, cannot be viewed as confirmation of the only factors related to the sustainability of cognitive behavi oral interventions. Further research is needed to investigate various external factors across a greater number of teachers and settings. A key limitation of this study was the small number of participants Although 72 teachers participated in the previo us effectiveness study, only 48 of these teachers were eligible for the current study, and only 21 agreed to participate. In order to address this limitation, data collection strategies were expanded to include interviews with teachers at varying stages of concern, and observations in one case of sustained implementation. Given the information gained from the interviews and observations of classroom practice, a broader knowledge base was developed to explain and describe how and why teachers previously taug ht to implement a research based practice decided whether to sustain that practice over time. Conclusions Several conclusions can be drawn from the findings of this study. The first conclusion addresses the importance of recognizing the needs of individu als who are expected to implement an innovation. The second conclusion addresses the necessity of ongoing professional development responsive to those needs so that the process of change can occur. The Importance of People in the Change Process The results of this study indicate that sustainability of a promising practice after a research team concludes in a school is not guaranteed, and can be influenced by the concerns of those prepared to use it. Findings from th is study suggest that those looking

PAGE 122

122 to est ablish the implementation and sustainability of a promising or proven practice first need to provide clear demonstration of how its use can be responsive to a range of s ense of self, the feasibility of the task, or the impact of the innovation on student outcomes (Fuller, 1969). In addition, addressing the individual concerns of teachers will likely lead to a more sustained use of the desirable practice, especially if the y are asked about their concerns with implementation early and often in the process. The Necessity of Professional Development Given the findings of the present study, it can be concluded that a systemic, ongoing program of professional development is nec essary to assist teachers to help address their concerns and to understand appropriate ways to implement promising desired instructional practice. Interactive professional development activities are needed for teachers who lack knowledge (i.e., self concer ns) and vision in their ability to appropriately implement an innovative practice (Fixsen et al., 2005). Professional teachers effectively plan for, teach, and assess the ef fectiveness of the innovation. Researchers, school administrators, and change facilitators can use the CBAM to assist with the change process. Professional development should be formulated around the information obtained from the various components of the CBAM. The CBAM could be an effective tool that helps reduce the research to practice gap. Change facilitators (i.e., administrators or researchers) could use the CBAM (a) to gauge how their implementers of change (i.e., teachers) are moving through the ch ange process, and (b) to provide necessary professional development that is individualized to

PAGE 123

123 reverting to their old ways. In their study, McKinnon and Nolan (1989) administ ered both the Stages of Concern Questionnaire and conducted Levels of Use interviews. During a 2 year period they administered the questionnaire four times and conducted the interviews 3 times. Each time they used the information gathered to conduct timely professional development activities to address the concerns of the teachers. Throughout the course of their study, they saw participants move through the early stages of concern to the higher stages and move from being nonusers into becoming users at diff erent levels. These findings suggest the importance of assessing the needs of implementers and responding to them in a timely and ongoing process, whether implementation takes place during a research study or a school based initiative. The question is how to make this happen. Implications for Practice Understanding why teachers adopt and sustain evidence based practices requires a breakdown of complex components of the intervention, including understanding the attributes of the teacher and the school (Vaug hn et al., 2000). This explanatory case study of a problem solving innovation provided data about the sustainability of the implementation of an innovative curriculum based on a promising practice. This study might also explain curriculum implementation in other schools with fourth and fifth graders under similar conditions of curricular change and flux. It would be beneficial if researchers could take action to enhance the useability of any desired innovation. Carnine (1995a) defined useability as the pra cticality of the research based practice for those who attempt to put it into practice. A debriefing meeting following the completion of the study with the teachers and the researchers could increase the useability and rate of sustainability, because the t eachers would

PAGE 124

124 have an opportunity to voice concerns or questions for future use. Adelman and Taylor (2003) stated that researchers could clarify the different ways for teachers to use the curriculum once they are no longer involved in the study, which woul d improve the adoption of an evidence based practice in an educational setting. In addition, Greenwood and Abbott (2001) noted that increased opportunity for researchers and teachers to collaborate and participate in professional development would increase the impact of research on practice. Researchers could conduct follow up professional development workshops to provide additional opportunities for collaboration to help bridge the research to practice gap. For example, teachers previously in the effective ness study failed to see the link with the innovation and tiered instruction. Follow up professional development could have addressed this issue and researchers would have been able to express the useability of this promising practice. Gersten et al. (1997 ; 2000) noted the importance for researchers to communicate th eir research findings in such a way that teachers find the required change both feasible and effective for the entire class. A change agent might be required for this to happen. If researchers hope to have a promising or proven practice sustained, that vision has to be somehow built in as a follow through to foster and support the initial adoption. Based on the findings, an action plan specific to the particular innovation needs to be developed, by the change facilitator, at the conclusion or as a follow up to a research study. However, it is important to note the focus of the study. In testing the effectiveness of an innovation, the focus must be on the practice itself and the implementation und er the

PAGE 125

125 promising/proven outcomes into sustainable implementation. Finally, this explanatory case study contributes to the body of research on the use of the CBAM framework to explain the implementation process and the sustainability of curriculum implementations over time. A research based framework such as CBAM may call attention to what is happening at the grassroots level and can provide important perspectives for administrators on the impact of subtle challenges for teachers and the eventual achievement patterns of their students. As demonstrated by d be used to study innovations from research to practice and improve the degree of sustainability. While analyzing research to practice facilitators could use information ga rnished from the various CBAM tools to move teachers through the change process. The CBAM would be useful to use if implementing an innovation and sustaining it in practice were corresponding goals of a study. In addition, it would be useful to use during an effectiveness study. Researchers During the course of the study, they would be able to make changes to the curriculum by making it more feasible or by providing prof essional development to the teachers to insure their understanding of the components incorporated in the curriculum. One way that school districts and school administrators can address Awareness and Management concerns is the use of a concerns based tr aining model rather than a skills based training model. Hall and Hord (2006) have noted with appropriate training that focuses on individual concerns there will be a shift in concerns from lower area (i.e.,

PAGE 126

126 Awareness and Information) to higher areas (i.e., Management and Refocusing). Teachers would most likely benefit from trainings that are more individualized and are interactive where teachers can discuss concerns and strategies with colleagues, administrators, and researchers. Recommendations for Future Research Researchers and teachers must work together to find a way to reduce the research to practice gap. Significant gaps exist between what is known as effective practices (theory and science) and what is actually done (policy and practice) (Fixsen et al., 2005). Research that is more socially valid could help reduce this gap between what researchers know is effective and what practitioners implement in the classroom. Social validity is the extent to which all the consumers of an intervention like it (i .e., like its goals, effects, components, and procedures) (Baer, Wolf, & Risley, 1987). This could be accomplished by allowing participants to be active members at all phases of the research. This corresponds with one of the conclusions of this study: to r ecognize the needs of individuals who are expected to implement an innovation. In future rese arch studies, researchers might assess and address the needs of the participants in their study early and often to identify their concerns with implementation, and to potentially influence their adoption of evidence based practices Promising and/or proven practices continue to be discarded due to a variety of reasons such as time constraints, not benefitting all students, and no administrative support. Attention ne eds to be given to these various elements when developing curriculum for the classroom. Making the research more socially valid could address a few of these reasons. Receiving that important ongoing feedback from the teachers could guide researchers to dev elop a p roduct with high useability

PAGE 127

127 Findings from this study support the recommendation that an individual change agent be recruited, or retained from the research team, to assist with the adoption process of translating the research to practice. Who bec omes the change agent or facilitator might depend on the goal of the study and the funding source If a study has an implementation in practice goal, which Tools For Getting Along did not have, a school staff member might be a better choice than a member f rom the research team. Therefore, this staff member would still be at the school after the research team has concluded its study and might be personally invested in seeing the teachers and students be come successful. There has been scant research on the h igher stages and levels of the Concerns Based Adoption Model. In future rese arch studies, CBAM rubrics could also be used more frequently with teachers and administrators as a self assessment tool, and to increase both self awareness and administrative per ception of how change is bein g accommodated at the teacher level.

PAGE 128

128 A PPENDIX A STAGES OF CONCERN QU ESTIONNAIRE Stages of Concern Questionnaire Name__________________________________________________ Date Completed__________________________________________ T he purpose of this questionnaire is to determine what people who are using or thinking about using various programs are concerned about at various times during the innovation adoption process. The items were developed from typical responses of school and c ollege teachers who ranged from no knowledge at all about various programs to many years experience in using them. Therefore, a good part of the items on this questionnaire may appear to be of little relevance or irrelevant to you at this time. For the com represent those concerns you do have, in varying degrees of intensity, and should be marked higher on the scale, according to the explanation at the top of each of the following pag es. For example: This statement is very true of me at this time. 0 1 2 3 4 5 6 7 This statement is somewhat true of me now. 0 1 2 3 4 5 6 7 This statement is not at all true of me at this time. 0 1 2 3 4 5 6 7 This statement is i rrelevant to me. 0 1 2 3 4 5 6 7 Please respond to the items in terms of your present concerns, or how you feel about your involvement or potential involvement with Tools F or Getting Along We do not hold to any one definition of this program, so plea se think of it in terms of your own perceptions of what it involves. Since this questionnaire is used for a variety of innovations, the name Tools F or Getting Along never appears. However, phrases Tools For Getting Along Remember to respond to each item in terms of your present concerns about your involvement or potential involvement with Tools F or Getting Along Thank you for taking time to complete this task.

PAGE 129

129 0 1 2 3 4 5 6 7 Irrelevant Not true of me now Somewhat true of me now Very true of me now 0 1 2 3 4 5 6 7 2. I now know of some other approaches that might work better. 0 1 2 3 4 5 6 7 3. 0 1 2 3 4 5 6 7 4. I am concerned about not having enough time to organize 0 1 2 3 4 5 6 7 myself each day. 5. I would like to help other faculty in their use of the innovation. 0 1 2 3 4 5 6 7 6. I h ave a very limited knowledge about the innovation. 0 1 2 3 4 5 6 7 7. I would like to know the effect of this reorganization on my 0 1 2 3 4 5 6 7 professional status. 8. I am concerned about conflict between my interests and my 0 1 2 3 4 5 6 7 responsibilities. 9. I am concerned about revising my use of the innovation. 0 1 2 3 4 5 6 7 10. I would like to develop working relationships with both our 0 1 2 3 4 5 6 7 faculty and outside faculty using this innovation. 11. I am concern ed about how the innovation affects students. 0 1 2 3 4 5 6 7 12. I am not concerned about this innovation. 0 1 2 3 4 5 6 7 13. I would like to know who will make the decisions in the new 0 1 2 3 4 5 6 7 system. 14. I would like to discuss th e possibility of using the innovation. 0 1 2 3 4 5 6 7 15. I would like to know what resources are available if we decide 0 1 2 3 4 5 6 7 to adopt this innovation. 16. I am concerned about my inability to manage all the innovation 0 1 2 3 4 5 6 7 requires. 17. I would like to know how my teaching or administration is 0 1 2 3 4 5 6 7 supposed to change. 18. I would like to familiarize other departments or persons with the 0 1 2 3 4 5 6 7 progress of this new approach.

PAGE 130

130 0 1 2 3 4 5 6 7 Irrelevant Not true of me now Somewhat true of me now Very true of me now 19. I am concerned about evaluating my impact on students. 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 21. I am completely occupied with other things. 0 1 2 3 4 5 6 7 22. I would like to modify our use of the innovation based on the 0 1 2 3 4 5 6 7 experiences of our students. 0 1 2 3 4 5 6 7 about other things in this area. 24. I would like to excite my students about their part in this approach. 0 1 2 3 4 5 6 7 25. I am concerned about my time spent working with nonacademic 0 1 2 3 4 5 6 7 problems related to this innovat ion. 26. I would like to know what the use of the innovation will 0 1 2 3 4 5 6 7 require in the immediate future. 27. I would like to coordinate my efforts with others to maximize 0 1 2 3 4 5 6 7 28. I would like to have more information on time and energy 0 1 2 3 4 5 6 7 commitments required by this innovation. 29. I would like to know what other faculty are doing in this area. 0 1 2 3 4 5 6 7 30. At this time, I am not interested in learning about the innovation. 0 1 2 3 4 5 6 7 31. I would like to determine how to supplement, enhance, or replace 0 1 2 3 4 5 6 7 the innovation. 32. I would like to use feedback from students to change the program. 0 1 2 3 4 5 6 7 33. I would like to know how my role will change when I am using 0 1 2 3 4 5 6 7 the innovation. 34. Coordination of tasks and people is taking too much of my time. 0 1 2 3 4 5 6 7 35. I would like to know how this innovation is better than what 0 1 2 3 4 5 6 7 we have n ow.

PAGE 131

131 PLEASE COMPLETE THE FOLLOWING: 36. What other concerns, if any, do you have at this time? (Please describe them using complete sentences.) 37. Briefly describe your job function.

PAGE 132

132 APPENDIX B LEV ELS OF USE BRANCHING INTERVIEW FORMAT

PAGE 133

133

PAGE 134

134 APPENDIX C SOCQ QUICK SCORING D EVICE Directions for using the SOCQ Quick Scoring Devise The Stages of Concern Questionnaire (SoCQ) contains 35 items. The scoring of the SoCQ require s a series of operations that result in a SoCQ profile. The following steps should be carried out on the Quick Scoring Device: Step 1 In the box labeled A, fill in the identifying information taken from the cover sheet of the SoC Questionnaire. Step 2 Copy the numerical values of the circled responses to statements 1 through 35 in the numbered blanks in the Table labeled B. Note that the numbered blanks in Table B are not in consecutive order. Step 3 Box C contains the Raw Scale Total for each stage (0 6). For each of the Seven columns (0 6) in Table B, add the numbers within each column, and enter the sum for each column (0 6) in the appropriate blank in Box C. Each of these seven Raw Score Totals is a number between 0 and 35. Step 4 Table D conta ins the percentile scores for each Stage of Concern. Find the Raw Scale Score Total for Stage 0 from Box C; locate this number in the left hand column in Table D, then look in the Stage 0 column to the right in Table D and circle that percentile ranking. D o the same for Stages 1 through 6, only match the left hand column raw score with the corresponding stage. Step 5 Transcribe the circled percentile scores for each stage (0 6) from Table D to Box E. Box E now contains seven numbers between 0 and 99. Step 6 Box F contains the SoC graph. From Box E, take the percentile score for Stage 0 and mark that point with a dot on the Stage 0 vertical line on the SoC graph. Do the same for Stages 1 through 6. Connect the points to form the SoC profile. For interpret ation of the SoC profile, refer to Hall, George, and Rutherford (1979).

PAGE 135

135 SoCQ Quick Scoring Device A Date: _____________________________________________ Site: ____________________________ SS#: _______________________________ Innovation: Tools Fo r Getting Along B Stage 0 1 2 3 4 5 6 3 6 7 4 1 5 2 12 14 13 8 11 10 9 21 15 17 16 19 18 20 23 26 28 25 24 27 22 30 35 33 34 32 29 31 Raw Score Totals C _____ _____ _____ _____ ______ ______ ______ Percentile Scores E _____ _____ _____ _____ _____ _____ _____ D Five Item Raw Scale Score Total Stage 0 1 2 3 4 5 6 0 10 5 5 2 1 1 1 1 23 12 12 5 1 2 2 2 29 16 14 7 1 3 3 3 37 19 17 9 2 3 5 4 46 23 21 11 2 4 6 5 53 27 25 15 3 5 9 6 60 30 28 18 3 7 11 7 66 34 31 23 4 9 14 8 72 37 35 27 5 10 17 9 77 40 39 30 5 12 20 10 81 43 41 34 7 14 22 11 84 45 45 39 8 16 26 12 86 48 48 43 9 19 30 13 89 51 52 47 11 22 34 14 91 54 55 52 13 25 38 15 93 57 57 56 1 6 28 42

PAGE 136

136 16 94 60 59 60 19 31 47 17 95 63 63 65 21 36 52 18 96 66 67 69 24 40 57 19 97 69 70 73 27 44 60 20 98 72 72 77 30 48 65 21 98 75 76 80 33 52 69 22 99 80 78 83 38 55 73 23 99 84 80 85 43 59 77 24 99 88 83 88 48 64 81 25 99 90 85 90 54 68 8 4 26 99 91 87 92 59 72 87 27 99 93 89 94 63 76 90 28 99 95 91 95 66 80 92 29 99 96 92 97 71 84 94 30 99 97 94 97 76 88 96 31 99 98 95 98 82 91 97 32 99 99 96 98 86 93 98 33 99 99 96 99 90 95 99 34 99 99 97 99 92 97 99 35 99 99 99 99 96 98 99 F Relative Intensity AWARENESS INFORMATION PERSONAL MANAGEMENT CONSEQUENCE COLLABORATION REFOCUSING 100 90 80 70 60 50 40 30 20 10 0 0 1 2 3 4 5 6 SoC Stages

PAGE 137

137 APPENDIX D INNOVATION CONFIGURA TION MAP TOOLS FOR GETTING AL ONG CURRICULUM Component 1: Included in lessons 1 7, 9 16, 18 19, and 21 B1 Curriculum Overheads Ideal (I), Acceptable, (A), Unacceptable (U) Teacher always displays overheads when discu ssing them I Teacher refers to overheads but does not always display them when discussing A Teacher rarely displays overheads when discussing them U Component 2: Included in lessons 1 7, 9 12, 14 16, 18 19, and B1 Scripted Lessons Ideal (I), Acceptabl e (A), Unacceptable (U) Teacher follows the scripted lesson for the entire lesson I Teacher partially reads the scripted lesson but stays on task A scripted lesson at all. Teaches the topic using own vocabulary and opinions. U Component 3: Included in lessons 2, 4 7, 9, 12, 14 16, and 18 19. Student Workbook Ideal (I), Acceptable (A), Unacceptable (U) Students are provided ample time to complete corresponding worksheets individually or in small groups following lesson I S tudents occasionally are provided ample time to complete corresponding worksheets individually or in small groups following lesson A Students are rarely given time to complete corresponding worksheets U Component 4: Included in lessons 1, 5, 7, 9, 11, 1 6, 19, and B1. Toolkits Ideal (I), Acceptable (A), Unacceptable (U) Teacher introduces the toolkit, provides the rationale, and students are given time to complete toolkit individually, in small groups, or as a whole group I

PAGE 138

138 Teacher occasionally intr oduces the toolkit, provides the rationale, and students are given time to complete toolkit individually, in small groups, or as a whole group A Teacher rarely introduces the toolkit, provides the rationale, and students are given time to complete the to olkit U Component 5: Included in lessons 3, 8, 13, 17, 20 21, and B2 B4. Role Plays Ideal (I), Acceptable (A), Unacceptable (U) Teacher allows students to participate in role play (several times if time permits) and identifies a number of students to lo ok for a specific problem solving step during role play I Teacher occasionally allows students to participate in role plays (several times if time permits) and identifies a number of students to look for a specific problem solving step during role play A Teacher rarely allows students to participate in role plays U Component 6: Included in lessons 6, 9, 12, and 18. Self Talks Ideal (I), Acceptable (A), Unacceptable (U) Teacher discusses a personal problem with class when instructed by curriculum. Te acher discusses problem in concert with the problem solving steps in TFGA I Teacher occasionally discusses a personal problem with class when instructed by curriculum. Teacher occasionally discusses problem in concert with the problem solving steps in TF GA A Teacher rarely discusses a personal problem when instructed by curriculum. Teacher never discusses problem in concert with the problem solving steps in TFGA U

PAGE 139

139 Component 7: Included in all lessons and all Booster lessons. Student Participation Ide al (I), Acceptable (A), Unacceptable (U) Students are encouraged to participate by reviewing previously learned problem solving steps, by sharing personal experiences related to problem solving, and by being asked to read overheads, directions role plays etc. I Students are occasionally encouraged to participate by reviewing previously learned problem solving steps, by sharing personal experiences related to problem solving, and by being asked to read overheads, directions role plays, etc. A Students are rarely encouraged to participate by reviewing previously learned problem solving steps, by sharing personal experiences related to problem solving, and by being asked to read overheads, directions role plays, etc. U

PAGE 140

140 LIST OF REFERENCES Adelman, H., & Taylor, L. (2003). On sustainability of project innovations as systemic change. Journal of Educational and Psychological Consultation, 14, 1 25. Anfara, V., Brown, K., & Mangione, T. (2002). Qualitative analysis on stage: Making the research process more public. Educational Researcher, 31, 28 38. Babbie, E. (1990). Survey Research Methods. California: Wadsworth Publishing Company. Baer, D., Wolf, M., & Risley, T. (1987). Some still current dimensions of applied Behavior analysis. Journal of Applied Be havior Analysis, 20 (4), 313 327. Berman, P., & McLaughlin, M. (1976). Implementation of educational innovations. Educational Forum, 40, 345 370. Bickel, W., & Hattrup, R. (1995). Teachers and researchers in collaboration: Reflections on the process. Ameri can Educational Research Journal, 32, 35 62. Bickmore, K. (2001). Student conflict resolution, po citizenship education. Curriculum Inquiry, 31 137 162. Brantlinger, E., Jimenez, R., Klingner, J., Pugach, M., & Richardson, V (2005). Qualitative studies in special education. Exceptional Children 7 1 ( 2 ), 195 207 Carnine, D. (1995a). Becoming a better consumer of research. The School Administrator, 6, 10 16. Carnine, D. (1997). Bridging the research to practice gap. Exceptional Children, 63, 513 521. Carnine, D. (1999). Campaigns for moving research into practice. Remedial and Special Education, 20 2 6. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. Web based Injury Statistics Quer y and Reporting System (WISQARS)[online}. (2006) [cited 2006 Feb 8]. Available from URL: www.ced.gov/ncipc/wisqars Cook, B., Cook, L., Landrum, T., & Tankersley, M. (2008). Examining the role of group experime ntal research in establishing evidence based practices. Intervention in School and Clinic, 44, 76 82. Cook, B., Landrum, T., Cook, L., & Tankersley, M. (2008). In troduction to the special issue: Evidence based practices in Special Education. Intervention in Scho ol and Clinic, 44, 67 68.

PAGE 141

141 Cook, B., Tankersley, M., & Harjusola Webb, S. (2008). Evide nce based special education and professional wisdom: Putting it all together. Intervention in School and Clinic, 44, 105 111. Cooke, M. B., Ford, J., Levine, J., Bourke, C., Newell, L., & Lapidus, G. (2007). The effects of city wide implementation of Second Step on elementary school prosocial and aggressive behaviors. The Journal of Primary Prevention,28 (2) 93 115. Craighead, L., Craighead, W., Kazdin, A., & Mahoney, M. (1994). Cognitive and behavioral interventions. Boston: Paramount. Dass, P. (1997). District wide professional development of science teachers: Factors Influencing the implementation of the Iowa Chautauqua Model [Electronic version]. Pro ceedings from the National Association for Research in Science Teaching Annual Meeting, Oak Brook, IL. Daunic, A., Smith, S., Brank, E. & Penfield, R. (2006). Clas sroom based cognitive behavioral intervention to prevent aggression: Efficacy and social val idity. Journal of School Psychology, 44, 123 139. DeCecco, J., & Richards, A. (1974). Growing pains: Uses of school conflict. New York: Aberdeen Press. Dissemination Working Group. (1999). Common elements of developing and implementing program models, Calg ary, Alberta: FYI Consulting Ltd. DuRant, R., Barkin, S., & Krowchuk, D. (2001). Evaluation of a peaceful conflict resolution and violence prevention curriculum for sixth grade students. Journal of Adolescent Health, 26, 386 393. Durlak, J., Fuhrman, T., & Lampman, C. (1991). Effectiveness of cognitive behavior therapy for mal adapting children: A meta analysis. Psychological Bulletin, 110, 204 214. Easton, J. (2011) How relevant research can help build a science of education. Speech Given at The University of North Carolina at Chapel Hill and Frank Porter Graham Institute Retrieved from http://ies.ed.gov Edwards, D., Hunt, M., Meyers, J., Grogg, K., & Jarrett, O. (2005). Acceptability and student outcomes of a violence pr evention curriculum. The Journal of Primary Prevention, 26 (5), 401 418. Fixsen, D., Naoom, S., Blas, K., Friedman, R., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

PAGE 142

142 Frey, K., Hirschstein, M., & Guzzo, B. (2000). Second step: Preventing aggression by Promoting social competence. Journal of Emotional & Behavioral Disor ders, 8, 102 119. Fullan, M. (2005). Leadership and sustainability: System thinkers in action. Thousand Oaks: Corwin Press. Fuller, F. (1969). Concerns of teachers: A developmental conceptualization. American Educational Research Journal, 6 (2), 207 226. Ge rber, M., & Solari, E. (2005). Teaching effort and the future of cognitive behavioral interventions, Behavioral Disorders, 30, 289 299. Gersten, R., Chard, D., & Baker, S. (2000). Factors enhancing sustained use of research based instructional practices. J ournal of Learning Disabilities, 33 445 457. Gersten, R., Morvant, M., & Brengelman, S. (1995). Closer to the classroom is close to the bone: Coaching as a means to translate research into classroom practice. Exceptional Children, 62, 52 66. Gersten, R., Vaughn, S. Deshler, D., & Schiller, E. (1997). What we know about using research findings: Implications for improving special education practice. Journal of Learning Disabilities, 30, 466 476. Gersten, R., & Dimino, J. (2001). The realities of translating research into classroom practice. Learning Disabilities Research & Practice, 16, 120 130. Gersten, R., & Dimino, J. (2006). RTI (Response to Intervention): Rethinking special education for students with reading difficulties (yet again). Reading Research Qu arterly, 41, 99 108. Gresham, F. (2005). Methodological issues in evaluating cognitive behavioral treatments for students with behavioral disorders. Behavioral Disorders, 30, 213 226. lementation research. The Western Political Quarterly, 39, 328 347. Greenwood, C., & Abbott, M. (2001). The research to practice gap in special education. Teacher Education and Special Education, 24, 276 289. Guyatt, G., Cook, D., & Haynes, B. (2004). Evid ence based medicine has come a long way: The second decade will be as exciting as the first. BMI, 329, 990 991.

PAGE 143

143 Hall, G., George, A., & Rutherford, W. (1979). Measuring stages of concern about the Innovation: A manual for use of the SoC Questionnaire (Repo rt No. 3032). Austin: the University of Texas at Austin, Research and Development Center for Teacher Education. (ERIC document Reproduction Service No. ED 147 342). Hall, G., George, A., & Rutherford, W. (1998a). Measuring stages of concern about the Innov ation: A manual for use of the SoC Questionnaire. Austin, TX: Southwest Educational Development Laboratory. Hall, G. & Hord, S. (1987). Change in schools: Facilitating the process. Albany, NY:SUNY Press. Hall, G., & Hord, S. (2001). Implementing change: Patterns, principles, and potholes. Boston: Allyn & Bacon. Hall, G., & Hord, S. (2006). Implementing change: Patterns, principles, and potholes (2 nd ed ). Boston, MA: Pearson Education, Inc. Hall, G., & Hord, S. (2011). Implementing change: Patterns, pri nciples, and potholes (3 rd ed ). Boston, MA: Pearson Education, Inc. Hargreaves, A., & Goodson, I. (2006). Educational change over time? The sustainability and nonsustainability of three decades of secondary school change and continuity. Educational Admin istration Quarterly, 42, 3 41. Hope, W. (1995). Microcomputer technology: Its impact on teachers in an elementary school. Unpublished doctoral dissertation, Florida State University. Individuals with Disabilities Education Act, 20 U.S.C. 1400 et seq. (200 4). Johnson, D., Johnson, R., Dudley, B., & Acikgoz, K. (2001). Effects of conflict resolution training on elementary school students. The Journal of Social Psychology, 134, 803 817. Kember, D., & Mezger, R. (1990). The instructional designer as a staff de veloper: A course team approach consistent with the Concerns Based Adoption Model. Distance Education, 11 (1), 50 71. Kennedy, M. (1991). Implications for teaching. In E. A. Ramp & C. S. Pederson (eds.), Follow through: Program and policy issues (pp.57 71). Washington DC: Office of Education Research and Improvement, U.S. Department of Education. Kinder, D., Gersten, R., & Kelly, B. (1989, March). Expert math instruction for at risk students. Paper presented at the annual meeting of American Educational Rese arch Association, San Francisco.

PAGE 144

144 Kratochwill T. & Stoiber K. (2000). Uncovering critical researc h agendas for school psychology: Conceptual dimensions and future directions. School Psychology Review, 29, 591 603. Larrivee, B. (2009). Authentic Classroom Management: Creating a Learning Community And Building Reflective Practice. New Jersey: Pearson Education, Inc. Educational Evaluation and Policy Analysis, 15, 129 151. Mayer, M., Lochman, J., & Van Acker, R. (2005). Introduction to the special issue: Cognitive behavioral interventions with students with EBD. Behavioral Disorders, 30, 197 212. McKinnon, D., & Nolan, P. (1989). Using computers in education: A concerns based approach to professional development for teachers. Australian Journal of Educational Technology, 5 (2), 113 131. McLaughlin, M. (1990). The rand change agent study revisit ed: Macro perspectives and micro realities. Educational Research, 19, 11 16. Me rriam, S. (1998). Qualitative Research and Case Study Applications in Education. San Francisco : Jossey Bass Publishers. No Child Left Behind Act of 2001, 20 U.S.C. 16301 et seq. (2002). Odom, S., Bran t linger, E., Gersten, R. Horner, R., Thompson, B ., & Har ris, K. (2005). Research in special education: Scientific methods and evidence based practices. Exceptional Children, 71, 137 148. Opotow, S. (Aug., 1989). The risk of violence: Peer conflicts in the lives of adolescents. Paper presented at the annual meet ings of the American Psychological Association, New Orleans, LA. Polsgrove, L., & Smith, S. (2004). Informed practice in teachi ng self control to children with emotional and behavioral disorders (pp. 399 425). Handbook of research in emotional and behavior al disorders. New York: Guilford. Sackett, D., Rosenberg, W., Gray, J., Haynes, R., & Richardson, W. (1996). Evidence British Medical Journal, 312, 71 72. Schaughency, E., & Ervin, R. (2006). Building capacity to implement and sustain effective practices to better serve children. School Psychology Review, 35, 155 166. Smith, S. (2002). Applying cognitive behavioral techniques to social skills instruction.{online}. Available at URL: www.cec.sped.ort/AM/Template

PAGE 145

145 Smith, S., & Daunic, A. (2006). Managing difficult behaviors through problem solving instruction. New York: Pearson Education, Inc. Smith, S., Daunic, A., Miller, D., & Robinson, T. (2002). Conflict resolut ion and pee r mediation in middle schools: Extending the process and outcome knowledgebase. The Journal of Social Psychology, 142, 567 586. Smith, S., Lochman, J., & Daunic, A., (2005). Managing aggre ssion using cognitive behavioral interventions: State of the practice and future directions. Behavioral Disorders, 30, 227 240. Strain, P., & Dunlap, G. (2006). Recommended practiced: Being an evidence based practitioner. Retrieved December 3, 2006, from http://challengingbehavior.fmhi.usf.edu/handouts/Practitioner.pdf Sugai, G. & Horner, R. (2002). The evolution of discipline practices: School wide positive behavior supports. Child and Family Behavior Therapy, 24, 23 50. Sukhodolsky D., Kassinove, H., & Gorman, B. (2004). Cognitive behavioral therapy for anger in children and adolescents: A meta analysis. Aggression and Violent Behavior, 9, 247 269. Sutherland, K., & Oswald, D. (2005). The relationship between teacher and student be haviors in classrooms for students with emotional and behavioral disorders: Transactional processes. Journal of Child and Family Studies, 14, 1 14. Tankersley, M., Harjusola Webb, S., & Landrum, T. (2008). Using single subject research to establish the evi dence base of special education. Intervention in School and Clinic, 44, 83 90. Vaughn, S., Klingner, J., & Hughes, M. (2000). Sustainability of research based practices. Exceptional Children, 66, 163 173. Vaughn, S., Klingner, J., & Hughes, M. (2004). Sust ainability of research based practices. Critical issues in Special Education: Access, diversity, and Accountability. New York: Pearson Education, Inc. Ward, J., West, L., & Isaak, T. (2002). Mentoring: A strategy for change in teacher technology education. Journal of Technology and Teacher Education, 10 (4), 553 569. Winter, S., & Szulanski, G. (2001). Replication as strategy. Organization Science, 12, 730 743. Yin, R. (2003). Case study research: Design and methods. California: Sage Publications, Inc.

PAGE 146

146 Yin, R. (2008). Case study research: Design and methods (4 th ed ). California: Sage Publications, Inc.

PAGE 147

147 B IOGRAPHICAL SKETCH Mallory Becker was born in Hollywood, Florida, in 1969. She spent many wonderful years in the area and graduated from South Broward Hi gh School in 1987. She earned her B.S. in psychology from the University of Florida (UF) in 1991. She earned her M.Ed. in education from UF in 1993. Specializing in emotional and behavioral disorders (EBD), Mallory had the opportunity to complete many uniq ue practicum placements to enhance professional development including the public school system, a center day school for students with EBD, a Department of Juvenile Justice Residential Facility, and Shands Psychiatric Hospital. Mallory earned her MSW in soc ial work from California State, Long Beach in 1999. She completed a rewarding internship at a psychiatric facility for adolescents and at an outpatient mental health facility for an extremely diverse population in downtown Los Angeles. Mallory earned her T ier 1 plans include working collaboratively with school system professionals to enhance the lives of children and living a fulfilling life with family and friends.