The Future of Simulations in Allied Healthcare Education and Training

MISSING IMAGE

Material Information

Title:
The Future of Simulations in Allied Healthcare Education and Training A Modified Delphi Study Identifying Their Instructional and Technical Feasibility
Physical Description:
1 online resource (366 p.)
Language:
english
Creator:
Kenney, Johanna K
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ed.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Curriculum and Instruction, Teaching and Learning
Committee Chair:
RITZHAUPT,ALBERT D
Committee Co-Chair:
BLACK,ERIK WADE
Committee Members:
KENNEDY-LEWIS,BRIANNA L
OLIVER,BERNARD

Subjects

Subjects / Keywords:
allied -- delphi -- education -- healthcare -- simulation
Teaching and Learning -- Dissertations, Academic -- UF
Genre:
Curriculum and Instruction thesis, Ed.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Healthcare educators have an important responsibility in educating and training students so they have the knowledge and skills necessary to successfully and competently work in today's healthcare fields. The demand for allied healthcare professionals continues to grow, and many programs are struggling to accommodate the increase in applicants and the decrease in state and federal funding. In today's challenging environment it is crucial to find a way to effectively teach in a cost effective manner without risking patient safety. To do this, institutions must strategically plan for the future of healthcare education and training by analyzing the literature, gathering strong empirical data, and surveying expert opinions.Many experts who advocate for the use of simulation maintain that simulated training can successfully be used as a complement to didactic and clinical training by giving students the opportunity to demonstrate specific skills and confirm a high level of understanding before they begin working with patients. However, the use of simulations does have its critics and limitations. The effectiveness of simulation in allied healthcare education is often diminished due to a lack of knowledge about how to use the equipment, how to set up the scenarios, and how evaluate the simulation.This study used the modified Delphi method to explore the feasibility of successfully addressing the critical issues facing the future use of simulator technology and simulation methodology in allied healthcare education. There were a total of 85 items separately rated for their technical and instructional feasibility. The finding revealed that there are still many unknowns when it comes to the use of simulation in allied healthcare education. No item was ranked at either end of the feasibility scale, 1: not at all feasible or 5: extremely feasible, meaning that while they are possible to pursue, none is completely ready to pursue at this current time. The outcome of this research point to a strong need for more research concerning the need for each of these items as well as continued research which answers how and when simulation as a teaching method is as good as or better than other models.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Johanna K Kenney.
Thesis:
Thesis (Ed.D.)--University of Florida, 2014.
Local:
Adviser: RITZHAUPT,ALBERT D.
Local:
Co-adviser: BLACK,ERIK WADE.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2014
System ID:
UFE0046484:00001


This item is only available as the following downloads:


Full Text

PAGE 1

1 THE FUTURE OF SIMULATIONS IN ALLIED HEALTHCARE EDUCATION AND TRAINING: A MODIFIED DELPHI STUDY IDENTIFYING THEIR INSTRUCTIONAL AND TECHNICAL FEASIBILITY By JOHANNA K KENNEY A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL O F THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF E DUCATION UNIVERSITY OF FLORIDA 201 4

PAGE 2

2 201 4 Johanna Kenney

PAGE 3

3 To Mom and Dad and all those who literally helpe d me survive 2013

PAGE 4

4 ACKNOWLEDGMENTS I would like to thank my family whose support has been invaluable. Without you I never would have made it this far. My sincere gratitude goes to my advisor Dr Albert Ritzhaupt who was both supportive and encouraging t hrough a very difficult time. My thanks also goes to my dissertation committee and the rest of faculty involved in the online program at UF, this journey was made possible because you believe d in me. I would also like to thank all those at the START Center Baptist Breast Center and Hospital You believed that not only would I survive, I would succeed.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 8 LIST OF FIGURES ................................ ................................ ................................ ........ 10 LIST OF ABBREVIATIONS ................................ ................................ ........................... 11 ABSTRACT ................................ ................................ ................................ ................... 12 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 14 Background ................................ ................................ ................................ ............. 14 Allied Healthcare ................................ ................................ .............................. 14 Medical Simulations ................................ ................................ ......................... 17 Identification of Need ................................ ................................ .............................. 19 Research Question ................................ ................................ ................................ 23 Overview of Methodol ogy ................................ ................................ ....................... 25 Significance of the Study ................................ ................................ ........................ 27 Limitations and Delimitations ................................ ................................ .................. 28 Definition of Terms ................................ ................................ ................................ .. 29 2 LITERATURE REVIEW ................................ ................................ .......................... 32 History of Simulation Use in Healthcare Education ................................ ................. 32 Terminology ................................ ................................ ................................ ............ 35 Types of Simulation ................................ ................................ .......................... 38 Mannequin based patient simulators ................................ ......................... 38 Partial task trainers ................................ ................................ .................... 40 Actor patients ................................ ................................ ............................. 43 Screen based simulations ................................ ................................ .......... 45 Summary ................................ ................................ ................................ .......... 46 Current Simulation Use ................................ ................................ ........................... 47 Evaluating Simulations ................................ ................................ ............................ 50 Limitations of Simulation ................................ ................................ ......................... 53 The Cost of Simulation ................................ ................................ ..................... 56 Planning for Si mulation ................................ ................................ .................... 57 Research Challenges ................................ ................................ ....................... 58 Simulation in Allied Health Education ................................ ................................ ..... 60 Simulations and the Role of Patient Safety ................................ ...................... 61 The Changing Role of the Allied Healthcare Worker ................................ ........ 63 The Role of Debri efing ................................ ................................ ..................... 71

PAGE 6

6 Summary ................................ ................................ ................................ .......... 73 Theoretical Framework ................................ ................................ ........................... 74 Constructivism ................................ ................................ ................................ .. 75 Adult Learners ................................ ................................ ................................ .. 77 Limits of Theoretical Framework ................................ ................................ ...... 80 Delphi Metho d ................................ ................................ ................................ ......... 81 Types of Delphi ................................ ................................ ................................ 83 Classical Delphi method ................................ ................................ ............ 83 Policy Delphi ................................ ................................ .............................. 84 Decision Delphi ................................ ................................ .......................... 85 Modified Delphi method ................................ ................................ ............. 86 Strengths of th e Delphi Method ................................ ................................ ........ 87 Weaknesses of the Delphi Method ................................ ................................ ... 88 Prior Studies ................................ ................................ ................................ ..... 92 Summary ................................ ................................ ................................ ................ 94 3 METHODS AND PROCEDURES ................................ ................................ ........... 95 The Modified Delphi Method ................................ ................................ ................... 96 Rationale for Using the Delphi Method ................................ ............................. 98 Selection of the Expert Panel ................................ ................................ ........... 99 Instrument Development for Round One ................................ .............................. 105 Validation of Questionnaire ................................ ................................ ............ 106 Questionnaire Distribution ................................ ................................ .............. 108 Da ta Collection ................................ ................................ ................................ ..... 109 Data Analysis ................................ ................................ ................................ ........ 113 Reporting the Data ................................ ................................ ................................ 114 4 RES ULTS ................................ ................................ ................................ ............. 116 Delphi Survey Round 1 ................................ ................................ ......................... 118 Delphi Survey Round 2 ................................ ................................ ......................... 128 Del phi Survey Round 3 ................................ ................................ ......................... 144 Summary of Findings ................................ ................................ ............................ 162 5 DISCUSSION, RECOMMENDATIONS, AND FUTURE WORK ........................... 164 Discussion ................................ ................................ ................................ ............ 164 Research Issues ................................ ................................ ............................. 165 Curriculum Issues ................................ ................................ ........................... 167 Collaboration Issues ................................ ................................ ....................... 171 Tools and Simulator Technology Issues ................................ ......................... 172 Faculty and Staff Issues ................................ ................................ ................. 174 Resource Issues ................................ ................................ ............................. 176 Recommendations ................................ ................................ ................................ 178 Stakeholder Considerations ................................ ................................ ........... 178 General Recommendations ................................ ................................ ............ 180

PAGE 7

7 Summary ................................ ................................ ................................ ........ 186 Future Work ................................ ................................ ................................ .......... 187 Lessons Learned ................................ ................................ ................................ .. 1 89 Conclusion ................................ ................................ ................................ ............ 191 APPENDIX A INVITATION TO PARTICIPATE ................................ ................................ ........... 193 B REJECTION LETTER ................................ ................................ ........................... 196 C INFORMED CONSENT ................................ ................................ ........................ 197 D DEMOGRAPHICS SURVE Y ................................ ................................ ................. 199 E DEMOGRAPHICS SURVEY RESULTS ................................ ............................... 202 F ROUND ONE QUESTIONNAIRE ................................ ................................ ......... 216 G ROUND TWO QUESTIONNAIRE ................................ ................................ ......... 231 H ROUND THREE QUESTIONNAIRE ................................ ................................ ..... 255 I SURVEY RESULTS ................................ ................................ .............................. 285 Round 1 Results ................................ ................................ ................................ ... 286 Round 2 Results ................................ ................................ ................................ ... 303 Round 3 Results ................................ ................................ ................................ ... 321 LIST OF REFERENCES ................................ ................................ ............................. 343 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 366

PAGE 8

8 LIST OF TABLES Table page 3 1 Data collection and analysis timeline ................................ ................................ 110 4 1 Questionnaire items which reached consensus in round one ........................... 119 4 2 Round one, e merging areas of concern by topic area ................................ ...... 127 4 3 Questionnaire items which reached consensus in round two ........................... 128 4 4 Round two, emerg ing areas of concern by topic area ................................ ...... 142 4 5 Questionnaire items which reached consensus in round three ........................ 145 4 6 Round three, emerg ing areas of concern by topic area ................................ .... 161 A 1 Study timeline ................................ ................................ ................................ ... 194 E 1 What is your preferred contact method? ................................ ........................... 202 E 2 Age group ................................ ................................ ................................ ......... 202 E 3 Gender ................................ ................................ ................................ ............. 202 E 4 What is the highest level of education you hav e completed? ........................... 203 E 5 Degree(s) earned ................................ ................................ ............................. 203 E 6 Please list any relevant certifications ................................ ................................ 204 E 7 Years of experience ................................ ................................ .......................... 204 E 8 Have you had any simulation specific training? ................................ ................ 205 E 9 What is the job title for your current position? ................................ ................... 206 E 10 About how long have you been in your current position? ................................ 206 E 11 Have you previously h eld a simulation related position(s)? .............................. 207 E 12 What is the job title for your current position? ................................ ................... 207 E 13 Time in this position ................................ ................................ .......................... 207 E 14 Other relevant previous experience ................................ ................................ .. 208 E 15 On average, how many hours per week? ................................ ......................... 209

PAGE 9

9 E 16 Type(s) of simulator(s)/simulation used ................................ ............................ 210 E 17 Please list any simulation related publications ................................ .................. 211 E 18 Other relevant information ................................ ................................ ................ 211 E 19 Reason for interest in the study ................................ ................................ ........ 212 E 20 I am currently associated with a techn ical or community college ...................... 212 E 21 Institution Name ................................ ................................ ................................ 212 E 22 Allied health programs offered ................................ ................................ .......... 213 E 23 Carnegie Classification ................................ ................................ ..................... 214 E 24 Programs currently utilizing human patient simulators ................................ ..... 214 E 25 Programs currently utilizing patient actor simulation ................................ ........ 215 E 26 Programs currently utilizing screen based simulation ................................ ....... 215 I 1 Comments from round 3 ................................ ................................ ................... 339

PAGE 10

10 LIST OF FIGURES Figure page 1 1 C ardiopulmonary resuscitation on a mannequin ................................ ................. 39 1 2 E xamines the mouth of a mannequin head with a laryngoscope ........................ 41 1 3 A triage officer attaches triage tags to a Sailor with a simulated injury ............... 44 1 4 A simulated dental head used to train dental technicians ................................ ... 46

PAGE 11

11 LIST OF ABBREVIATIONS ADEA American Dental Education Association ASAHP Association of Schools of Allied Health Pro fessions CASE Comprehensive Anesthesia Simulation Environment CNA Certified Nursing Assistants CPR C ardiopulmonary Resuscitation DARPA Defense Advanced Research Projects Agency: Established in 1958 to research and create cutting edge technologies for the m ilitary. DoD Department of Defense EFI Educational Framework Initiative FMSTC Federal Medical Simulation and Training Consortium GAS Gainesville Anesthesia Simulator HPS Human Patient Simulator : A d evice or technology that presents a simulated patient I OM The Institute of Medicine IPSE Inter Professi onal Simulation Based Education LPN Licensed P ractical N urse : a generalist with one year of vocational training who works in various areas of health care METC Medical Education and Training Campus METI Medica l Education Technologies, Inc. OR Operating Room PII Personally I dentifiable I nformation RN Registered Nurse: provide and coordinate direct patient care in a variety of settings including hospitals, nursing homes, and in the home Degree. TATRC Telemedicine and Advanced Technology Research Center : this Department of Defense office supports medical research VR Virtual Reality

PAGE 12

12 Abstract of Dissertation Presented to the Graduate School of the University of Florida in P artial Fulfillment of the Requirements for the Degree of Doctor of Education THE FUTURE OF SIMULATIONS IN ALLIED HEALTHCARE EDUCATION AND TRAINING: A MODIFIED DELPHI STUDY IDENTIFYING THEIR INSTRUCTIONAL AND TECHNICAL FEASIBILITY By Johanna K Kenney Ma y 201 4 Chair: Albert Ritzhaupt Major: Curriculum and Instruction Healthcare educators have an important responsibility in educating and training students so they have the knowledge and skills necessary to successfully and are fields. The demand for allied health care professionals continues to grow, and many programs are struggling to accommodate the increase in applicants and the decrease in state and federal funding. challenging environment it is crucial to fin d a way to effectively teach in a cost effective manner without risking patient safety. To do this institutions must strategically plan for the future of healthcare education and training by analyzing the literature, gathering strong empirical data, and s urveying expert opinions. Many experts who advocate for the use of simulation maintain that simulated training can successfully be used as a complement to didactic and clinical training by giving students the opportunity to demonstrate specific skills and confirm a high level of understanding before they begin working with patients. However, the use of simulations does have its critics and limitations. The effectiveness of simulation in allied healthcare

PAGE 13

13 education is often diminished due to a lack of know ledge about how to use the equipment, how to set up the scenarios, and how e valuate the simulation. This study used the modified Delphi method to explore the feasibility of successfully addressing the critical issues facing the future use of simulator tech nology and simulation methodology in allied healthcare education There were a total of 85 items separately rated for their technical and instructional feasibility. The finding revealed that there are still many unknowns when it comes to the use of simul ation in allied healthcare education. No item was ranked at either end of the feasibility scale 1=as not at all feasible or 5=extremely feasible, meaning that while they are possible to pursue, none is completely ready to pursue at this current time. The outcome of this research point to a strong need for more research concerning the need for each of these items as well as continued research which a nswers how and when simulation a s a teaching method is as good as or better than other models

PAGE 14

14 CHAPTER 1 I NTRODUCTION This project is concerned with exploring and identifying issues imp acting the future of simulation in allied healthcare education and training It seeks to consolidate and assess expert opinions about the feasibility of successfully addressing the critical issues facing the future use of simulator technology and simulation methodology within allied health care education in order to help develop recommendations for strategic decision making This work consists of an introduction, a literature re view, methodology, results, and a conclusion The purpose of Chapter 1 is to provide the background and an overview of this study It establishes the need for a rigorous analysis concerning the future of simulation in allied health care education and intr oduces the methodology and context for this research. Background Allied Healthcare Allied health professionals currently comprise an estimated 60 % healthcare workforce and can be defined as medical practitioners with some formal education and clinical training who are credentialed through a certification, registration, and/or licensing process (Fitzgerald, 2006) The allied healthcare field encompasses over 200 health careers and can be broken into two board categories: technicians (assistants ) and therapists/technologists Technicians are often trained to perform specific procedures and work under the supervision of a therapist/technologist who has the addition al training needed to evaluate patients, diagnose conditions, and develop treatment plans (American Dental Education Association (ADEA), 2012) These

PAGE 15

15 individuals are responsible for a wide variety of tasks and are often directly responsible for patient care and safety. Healthcare educators have an important responsibility in educating a nd training students so they have the knowledge and skills necessary to successfully and As the demand for allied health professionals (e.g., dental hygienists, diagnostic sonographers, surgical technologists physical therapists, radiographers, and respiratory therapists) continues to grow, many education programs are struggling to accommodate the increase in applicants and the decrease in state and federal funding (McLaughlin, Starobin & Laanan, 2010) I t is projected that the U.S. will face a shortage of allied health professionals which will range from 1.6 million to 2.5 million by 2020 (Fitzgerald, 2006) placing more pressure on schools to produce larger numbers of qualified healthcare professionals To day patient care has shifted f rom being largely hospital based to being predominantly outpatient, limiting student exposure to live patients, and the accompanying opportunities to acquire problem solving and clinical skills (Morgan, Cleave Hogg, Desousa, & Lam Mc C ulloch, 2006) Due to time limitations, as well as legal, ethical, and cost issues it has become a challenge to meet the clinical component of these educational programs, the traditional method for providing students hands on experience In toda environment it is crucial to find a way to effectively teach in a cost effective manner without risking patient safety To do this, institutions must strategically plan for the future of allied healthcare education and training by analyzing the literature, gathering strong empirical da ta, and surveying expert opinions.

PAGE 16

16 A 2012 survey conducted by the Healthcare Association of New York State concluded that with the passing of the Affordable Care Act there is a high probability tha t more people will seek treatment in the near future and therefore the difficulty of recruiting qualified allied health professionals such as clinical lab technicians, medical coders, physical therapists, and occupational therapists is likely to worsen I ncreases in the cost and demand of healthcare are likely to impact the role of the allied health professional requiring them to take on more responsibilities The qualifications, and therefore the training and education of an allied health professional will also need to expand (Lizarondo, Kumar, Hyde, & Skidmore, 2010) With over 200 different professions each educational institution is unique in its programs and scope, and faculty shortages continue to impact the increasing number of rejected qualifi ed applicants (Fitzgerald, 2006) The changing needs of the healthcare industry require educational institutions to find new cost effective methods to educate more students th ese concerns through the formation of an allied healthcare core curriculum to help expand capacity through greater efficiency (2012) ; however, the program is new so its impact on the field is as yet unknown I t is important to note one of the essential id eas behind building a common core curriculum for allied healthcare education is that there are certain courses and skills that all allied health professionals need regardless of specialty: medical terminology, anatomy/physiology, first aid, and pharmacolog y to name a few This builds on a 1999 report by the U.S. Department of Health and Human therapeutic, diagnostic, and preventive areas of health care, their practices a nd

PAGE 17

17 education have common elements, and they share a commonality of purpose and p. 1) Utilizing simulations within these core courses may help ensure the greatest impact on the largest number of stud ents with the least amount of cost. Medical Simulations Healthcare simulations can seem like a blessing to healthcare educators curriculum developers, and program directors as they try to develop new methods to teach more students more information with gr eater efficiency Limited research shows that simulation can be beneficial in training diagnostic skills, communication skills, and team building while allowing for the systematic collection of archival data, and the deliberate practice of responding to rare scenarios (Gaba, Howard, Fish, Smith, & Sowb, 2001; McLaughlin, et al., 2010; Okuda et al., 2009; Ziv et al., 2006) Resear ch has successfully established outcome improvement for anesthesiologists who have used simulator training, but in surgical tra ining the research is in sufficient to prove that simulator training can result in error reduction (Dawson, 2002) Simulation is often used in an effort to improve patient safety as it provides students an opportunity to learn ves or making patients feel uncomfortable by being used as live training subjects ( Burgess, 2007; Okuda et al., 2009 ; Seropian, 2003) Many ex perts who advocate for the use of medical simulation maintain that simulated training can successfully be used as a complement to didactic and clinical training as students have the opportunity to demonstrate specific skills and confirm a high level of u nderstanding before they begin working with patients (Dawson, 2002; Gaba, 2004 b ; Haluck et al., 2007; Okuda et al., 2009) However, the use of simulations in healthcare education does have its critics and limitations (Bradley & Postlethwaite, 2003b; Cantrell

PAGE 18

18 & Deloney, 2007; Chiniara et al., 2012; Colliver, 2002 ; Gibbs, Durning, & Van Der Vleuten 2011 ) and the rese arch concerning its impact and outcomes on allied health care education is scarce. Healthcare simulations have been commonly used for more than twenty years, yet high ly quality rigorous research concerning quality assessment, learning outcomes, patient outc omes, and skill transfer capabilities of simulations is limited (Holzinger, Kickmeier Rust, Wassertheuer, & Hessinger, 2009; Kardong Edgren Adamson & Fitzgerald, 2010; McGaghie, Issenberg, Petrusa, & Scalese, 2010; Okuda et al., 2009) Simulation s allow for realistic, real world training and skills asse ssment in a controlled, non threatening environment, but the se emerging technologies are in need of a framework to guide their development, a supporting taxonomy for instructional design, a formulation of best practices for impleme ntation, and rigorous research designs for evaluating learning outcomes ( Chiniara et al., 2012; Jeffries, 2006) The practice of using healthcare simulation for educational purposes is based largely on anecdotal feedback from inf ormal user studies (Magee, 2003) and not upon validated methods In a 2004 survey concerning human patient simulators (HPS) in nursing education, it was shown that community college programs used more hours in almost all courses than universities and simu lation centers (Nehring & Lashley, 2004 ) Despite this, a literature search uncovers few research articles specifically addressing healthcare simulations in community and technical college settings for any degree of study So is there a future for health care simulations in allied health care education and training, and if so, what does it look like?

PAGE 19

19 Identification of Need The Medical Education & Training Campus (METC ) at Fort Sam Houston, Texas provides for the initial and advanced education and training of enlisted medical personnel throughout the military METC resulted from a Base Realignment and Closure (BRAC) initiative which in 2011 brought together students, fa culty, and staff from various military locations including San Diego, CA; Portsmouth, VA; Great Lakes, IL; and Wichita Falls, TX The majority of all initial medical education and training programs for enlisted personnel are now located at METC as single Service, consolidated (more than one Service component taking the same course), or colloc ated (same course but classes separate d by Service) This training campus consists of over 50 medical programs most of which fall into the allied healthcare category, with an estima ted annual graduation rate of 21 ,000 Army (A), Navy (N), Air Force (AF), and Coast Guard (CG) enlisted personnel Simulations have been assessed for use in higher education for nursing education (Jeffries, 2006; Leigh & Hurst, 2008), surgical training (Dawson, 2002; Haluck et al. 2007; Hammoud et al., 2008), and veterinary t raining (Scalese & Issenberg, 2005), but at METC many of the incoming students are recent high school graduates with only military basic training (boot camp) behind them For most, the level of training they receive is comparable to associate degrees or medical certifications, not at the level of surgeons or registered nurses Programs include Army healthcare specialist (combat medic), Air Force/Navy basic medical technician/corpsmen, dental assistant/specialist (A, N, AF), surgical technologist (A, N, A F), biomedical technician (A, N, AF), medical laboratory technician (A, N, AF), radiologic technologist (A, N, AF, CG), respiratory therapist (A, N), and behavioral health specialist (A, N, AF)

PAGE 20

20 Depending on their program, students receive a few weeks or m onths of Phase One classroom education and lab training, followed by a Phase Two clinical internshi p. After graduating from METC they are deployed or sent to field/specialized training with their specific Service component At METC it is imperative that the students become competent practitioners because of their training rather than despite of the highly compressed training (Morgan et al., 2006) Many of these students comprise the ne at the point of injury in combat, accidents, and humanitarian missions Others may be the only medical support readily available (e.g ., on some Navy deployments, a corpsman may be the only medical personnel available ) Many of the programs at METC util ize simulations during P hase One; h owever, there currently is no centralized strategic plan concerning the future integrati on, sustainment, and implementation of healthcare simulations within and across curriculums In my role as a Learning Systems Develo per, I compiled a survey to assess modeling and simulations capabilities and needs This was conducted in October 2 011 shortly after METC was at full operating capability The Academic Support division attempted to discover what medical models and simul ations were currently being used, how they were being used, what support structure existed for these technologies, and what methods and technologies the programs would like to use, as well as to identify barriers to the adoption of these methods and techno logies The results indicated that only nine of the twenty programs who responded were using modeling and simulation on a regular basis Nevertheless all but one program indicated that they were interested in using such technologies A Modeling &

PAGE 21

21 Simul ations (M & S) Working Group was subsequently formed and chaired by the Associate Dean of Academics to help design and support a strategic path forward The shortage of academic literature verifying that simulation training has a long term positive impact on learning outcomes and results in successful skills tran sfer to the clinical environment (Bradley, 2006; Cooper & Taqueti, 2008; Gaba, 2004 b ; Ziv et al., 2006) leaves some decision makers uncertain that increased use and curriculum integration is the mo st effective and cost efficient course of action Therefore, t his group has been working on a number of issues and concerns including how to develop a sho rt and long term plan for the sustainment and growth of modeling and simulation at METC To fully in tegrate healthcare simulations successfully into the curriculum at METC an assessment and reworking of hundreds of hours of curriculum may be required, along with intensive faculty training, and support from the Services, administration and staff (Bradle y, 2006; Moreno Ger et al., 2010) At METC the periods of instruction and curriculum are highly controlled, and significant changes to the curriculum, including the integration of medical simulation, require s approval from the Services before they can be implemented Before any significant change can be requested, let alone implemented, it is necessary to ensure that the continued use and integration of medi cal simulation is the proper course of action and that it will result in improved learning outcome s, more effective use of classroom/lab time, and/or cost savings Through t his modified Delphi study I attempt to identify technical and instruction al issues impacting the future of simulations in allied healthcare education and training by consolidat ing and assessing expert opinions in order to help build actionable recommendations and guidelines for strategic decision making

PAGE 22

22 Role of the researcher. As part of my work as a Learning Systems Developer, I assess and integrat e emerging and innovative techno logies for use in enlisted military medical education In this role, I strive to ensure that students and educators have the most appropriate technologies and methodologies in order to participate in effective and engaging learning It is my belief that the appropriate use of t echnology including simulators, can engage and support adult learners and those who educate them As a member of the Education Technology Department, I work to integrate student centered technology into a military classroom settin g which has a strong didactic tradition and a great emphasis on PowerPoint presentations Simulator t echnology and simulation methodology has the potential to give instructors another tool to use to enhance the social and collaborative aspect of learning increase retention, and assist in the practice of skills which are transferrable to clinical and field settings It is my assertion that simulator technology and simulation methodology can enhance and support the learning experience at METC but it is vi tal that these technologies and methods are based on relevant a nd sound research and validated practices The Department of Defense (DoD) has begun several initiatives to decrease the use of live tissue through the increased use of simulation technology but admit that based methods until simulation devices and measurable outcomes can be scientifically validated with training methods (Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, 2013 p. 2 ) While simulation is 2008) and standards for simulations, curricula, and simulation centers (Gaba, 2004 b )

PAGE 23

23 The exact future uses and effectiveness of healthcare simulation is in question, but few including myself, question that it will play an ever expanding role in healthcare education and training (McGaghie et al. 2010; Rosen, 2008) It is my belief that simulation will be and should be integrated into the military medical curriculum, but the process should be grounded in research based instructional practices Today technology is often purchased at METC without fully understanding how it will be utilized it is my goal to help ensure that simulation is carefully considered as curriculum is reviewed and technology is acquired to meet training needs. Gaba (2004) notes that it may be too soon for research to prove long term benefits a s the simulation studied needs to be applied consistently over long period s of time long term implementation without a definitive evidence base t ( p. i 7) Many healthcare educators and researchers argue that despite the lack of rigorous research supporting the use of simulations ; it is likely that the future of healthcare education and training will include their widespread use (Okuda et al., 2009 ; Rosen, 2008) I maintain that simulation has the potential to increase instructional effectiveness, student engagement, and the transfer of skills, but only if it is carefully and appropriately integrated into the learning environment But how do instit utions such as METC prepare for the future use of simulations when there are still many questions concerning their current application? It is my goal to help find answers to this question through a detailed study of the literature and the gathering of exp ert opinions. Research Question The purpose of this study was to explore and assess the issues facing the future use of simulation in allied health care education by examining and correlating the views

PAGE 24

24 of a panel of experts pertaining to specific issues in implementing, integrating, utilizing, and assessing simulation methodology and simulator technology in allied healthcare programs (Ziglio, 1996) Specifically, the questions which guide this research are: 1. In the opinion of a panel of experts, what technic al issues will impact the future use of simulators/ simulation in allied healthcare education? 2. In the opinion of a panel of experts, what instructional issues will impact the future use of simulators/ simulation in allied healthcare education? 3. What is the te chnical feasibility of each issue impacting the future use of simulators/ simulation in allied healthcare education? 4. What is the instructional feasibility of each issue impacting the future use of simulators/ simulation in allied healthcare education? Techni cal issues are concerned not only with the simulation technology itself, but can include issues with peripherals such as network support, simulator maintenance, technical difficulties in running the simulators, and the design and development of new technol ogies Instructional issues may include pedagogical issues such as the choosing of the most effective instructional method, policy issues such as accreditation requirements, curriculum issues such as assessment validation, and faculty issues such as train ing It is important to not only identify and define technical and instructional issues which may impact the future of simulations in allied healthcare education, but to develop a consensus concerning whether or not issues can be dealt with successfully. The purpose of this research was to develop a list of items ( specific issues) to be addressed based not only on the strengths and weakness es of the technology, but also on the strength s and weaknesses of the methodology The research could not be simply t If it is not actionable, then it is strictly educational and less likely to affect decision making in real time p. 318) By understanding the future opportunities and/or threats of each item, decision m akers can

PAGE 25

25 make informed decisions concerning the possible success of a specific plan to adopt, integrate, and/or use simulation It is proposed that by utilizing a panel of experts valid and actionable guidelines and recommendations can be developed that can help inform decision makers in their strategic planning initiatives As the tasks and skills needed vary greatly w ithin healthcare education, it was decided to limit this study to healthcare education taught at a comparable skill and learning level to that taught at METC It was established that this research would look only at allied healthcare programs (McLaughlin et al., 2010) For the purpose of this study allied healthcare fields include only those listed by the Commission on Accreditation of Al lied Health Education Programs (CAAHEP) W hile licensed practical nurse ( LPN ) programs are often taught at the institutions as allied healthcare programs, they are considered a separate field and are not included in this study (ADEA, 2012; Health Professi ons Network, 2010). Overview of Methodology judgments of experts using a series of data collection and analysis techniques Skulmoski, Hartman & Kr ahn, 2007 p. 1) in order to improve understanding of a problem, solution, or prediction This project utilized the modified Delphi method proposed by Custer, Scarcella, and Steward (1999) in that it began with a set of carefully selected items drawn from an extensive literature review concerning issues affecting the future of simulation in allied healthcare education The items were then subjected to a three round Delphi to ascertain expert agreement concerning the items and the technical and instruction al feasibility of addressing these issues The Delphi method was applied in order to facilitate an anonymous exchange of

PAGE 26

26 ideas and opinions among a panel of experts concerning the items derived from the literature (Fleuren, Wiefferink, & Paulssen, 2004) By utilizing the Delphi method experts from a variety of geographic locations with varied backgrounds and professions could be utilized. For this study the term expert was defined as someone who has designed or developed a medical simulation lab for alli ed health care programs or has a minimum of three experience in any of the following roles within allied healthcare programs : operating simulations ; assessing simulation based competencies ; using simulation as a teaching or assessment tool ; developin g new curriculum for the integration of simulation ; or having published peer reviewed research concerning simulations at the allied healt hcare level (Holden & Wedman, 1993 ; Pollard & Pollard, 2 004) Basic demographic information, including his/her backgro und in simulation, years of experience, qualifications, and the reason for his/her interest in the study was collected from each potential participant (Keeney, Hasson, & McKenna, 2011) via a demographic survey hosted on SurveyMonkey (Melnick et al., 2010; Schibler, 2008) Any individual not meeting the minimum standards was thanked for his/her time and received notice that he/she would not be invited to participate in the Delphi study Prior to the first round of the modified Delphi the initial round one questionnaire was developed by drawing on a synthesized literature review (Custer et al., 1999; Hsu & Sandford, 2007; Stewart et al., 1999) T he questionnaire and instructions for round one were then disseminated via email to the pre selected Delphi pane l (Holden & Wedman, 1993 ; Pollard & Pollard, 2004) All rounds were conducted so the responses were anonymous to the participants

PAGE 27

27 Qualitative analysis was used to synthesize and analyze the data as issues were modified or added ack Items were rated on a scale of one to five for their instructional feasibility and technical feasibility: 1= not at all feasible, 2= probably unfeasible, 3= feasible, 4= very feasible, 5= extremely feasible Quantitative analysis was conducted on th e items based on means (as a measure of central tendency), standard deviation (a measure of spread), and the interquartile range (IQR) to demonstrate the range of consensus within the panel (Wicklein, 1993) According to Linstone & Turoff (1975), the inte rquartile analysis is considered the best method for determining consensus because the spread shows a distribution of responses for a single item Defined as the absolute value of the difference between the first and third quartiles of the data set IQR w as calculated with the formula IQR = Q3 Q1 Following the IQR first established the first and third quartiles using the formula Q1 = (n+1)/4 and Q3 = 3(n+1)/4 The IQR show ed the level of agreement in responses by demonstrating if they are more varied (spread out) or tighter (more consensus) (Zeedick, 2010) This helps panel members understand their response in relation to the panel as a whole and limits the effects of outlier responses T he goal of the Delphi is to reach con s ensus among a group of experts, therefore the IQR is used to exclude extreme scores on either end (Kreber, 2002) The use of all these descriptive statistics allow ed the panel to work towards consensus concerning the feasibility of each item as it pertain ed to the future use of simulation in allied healthcare education Significance of the Study The significance of this study is that it fills an important gap in the literature as simulation is often unquestioningly accepted by healthcare educators even whe n they are uncertain how they should fully utilize the technology/methodology (Leigh & Hurst,

PAGE 28

28 2008) S ome literature concerning the future use of healthcare simulation in education and training exists (Bradley, 2008; Gaba, 2004 b ) ; however, like much of th e research on healthcare simulations the literature is limited and focused mainly o n surgical and applications A llied health care areas such as first aid, dental assistance, pharmacy technician emergency medical s ervices, and are often associate degrees or certifications with skill sets and learning levels comparable to those taught at METC While a few articles have been published regarding the use of simulation for allied healthcare and associate de gree nursing students (Kuznar, 2007; Maas & Flood, 2011; McLaughlin et al., 2010), there is a distinct lack of literature concerning simulations in certificate and associate degree allied health care programs. The Delphi method can be difficult to employ an d some question its valid ity and reliability (Sackman, 1975; Rie ger, 1986), but when used carefully and appropriately it can explore critical issues, help predict the future, equip leadership with vital information to use in decision making, or help impro ve the practice (Nworie, 2 011) By determining the future issues facing the feasibility of medical simulation in allied health care education and training, this study provide s valuable planning information for education and training facilities such as METC The consensus of opinions of this panel of experts should be of interest to those charged with planning and implementing future educational programs for allied healthcare education and training. Limitations and Delimitations This study is constrained by certain limitations which are inherent within the modified Delphi method These limitations include the fact that o utlying views can be repressed by the nature of consensus building ; therefore some insightful or important

PAGE 29

29 information may be lost through the iterations It is also important to note that the results are the collated opinions of a specific group; they may or may not be factual and the d ata may not be applicable to other medical education and training settings E ach member of the panel bri ngs his/her own perspective and context as their input is based on personal experience which may bias some of the input also impact the interpretation of the data as individual panel input was consolidated and correlated for re view The final limitation using the Delphi method is related to t he number of rounds and subsequent length of the survey instrument and study The very nature of its length may have lead some experts choose not to participate or resulted in significant attrition rates Only the responses included in all three rounds are included in the final results. Delimitations are linked to the narrow scope of the project These delimitations include the fact that participants were restricted to those who are curre ntly planning, using, or are published in the field of simulation used in allied healthcare settings ; a different demographic may have resulted in different responses Other delimitations are related to the scope of the project as there was a limited amou nt of time to conduct the research and a limited number of participants an expert pool of 20 experts Because of this small pool the e xperts may not be fully representative of all allied healthcare programs or adequately address the unique needs of METC Definition of Terms Allied health professionals The Association of Schools of Allied Health Professions (ASAHP) (2012 ) states that: the delivery of health or related services pertaining to the identificatio n, evaluation and prevention of diseases and disorders; dietary and nutrition services; rehabilitation and

PAGE 30

30 section ) Examples of occupations include athletic trainer, cytotechnologist, dental hygienists, dietitians, emerg ency medical technicians, medical technologists, medical laboratory technician, physical therapists, radiographers, respiratory therapists, surgical technologists, and ultrasound technologists Associate degree nursing (LPN) programs are typically not considered allied healthcare programs, but certified nursing assistants (CNA) are (ADEA, 2012) Delphi method The Delphi method is a method for soliciting expert opinions about real world knowledge ( Hsu & Sandford, 2 007) It often consists of a series of three rounds where expert input and analysis moves the group towards consensus It expert forecasts about the p. 267 ). Experts Individu als who are leaders in the field of inquiry, who have firsthand experience, or are primary stakeholders related to the target issue or research effort (Hsu & Sandford, 2007) In this study it was an individual who has had firsthand experience implementin g, using, or researching medical simulation for allied healthcare education and training Healthcare simulat or Device or technology that presents a simulated patient, or an anatomic part of a simulated patient, that appropriately interacts and reacts to the actions taken by the simulation participant(s) (Gaba, 2004 b ) Healthcare simulation is often divided into four areas of practice: standardized patient, computer based interactions, partial task trainer, and high fidelity simulators (Okuda et al., 2009 ).

PAGE 31

31 Modified Delphi The traditional Delphi begins with an open ended questionnaire In the modified model the items are preselected based on synthesized reviews of the literature that it (a ) typically improves the initial round response rate, and (b) provides a solid 6). Simulation A technique which reproduces, takes the place of or amplifies real exper iences, events, situations, or environments which is a fully interactive, and often immersiv e, guided experience (Lee, Lee, Wong, Tsang & Li 2010; Gaba, 2004 b ). Simulator A physical object, technology, or representation of a task, environment, or exper ience with which one or more targe ted tasks are carried out (Cooper & Taqueti, 2008).

PAGE 32

32 CHAPTER 2 LITERATURE REVIEW This study explores and assesses the technical and instructional feasibility of successfully addressing the issues facing the future use of s imulation in allied health care education Chapter 2 reviews the literature and theoretical framework related to the current practice of simulation in health care education in general and then in allied healthcare specifically Chapter 2 begins by reviewin g the history, strengths and weakness es and possible future uses of simulation in healthcare education It also identifies the gaps in the current literature and weaknesses in the current research T his literature review is the basis for the questionnai re given during the first round of this Delphi study This review also briefly looks at the history, evolution, strengths and weaknesses of the modified Delphi method Specifics on how and why the Delphi method was used in this study are discussed in C h apter 3 History of Simulation Use in Healthcare Education Healthcare simulations have humble beginnings such as using an orange to practice injections; however, modern technologies and methodologies are a relatively recent addition to education Today s imulations (the technique) and simulators (the technology) are used to train a wide variety of professions including surgeons, pilots, mechanics, soldiers, and scientists ; but t he modern use of simulations for educational purposes began in the aviation and aerospace industries and was then appropriated by other fields including healthcare education (Lynagh Burton, & Sanson Fisher, 2007) Anesthesiology was one of the first medical fields to rigorously research, integrate, and study the impact of the use o f simulation for education and training purposes In the 1980s anesthesiologists purposely studied how simulations were being used in military

PAGE 33

33 and aviation training as it pertained to team and individual performance during critical events They used what they learned to develop a simulated environment for anesthesia administration (Hovancsek, 2007) According to Shaffer et al. (2001) medical and aviation fields have similar needs: unstructu red problems, where a potentially unlimited number of features are related in unclear and complex ways Theorists argue that skill development in such domains requires practical experience, rather than develop finely tuned perceptual and motor skills, the ability to analyse (sic) complex situations quickly and accurately, based on limited information, and the ability to make sound decisions about how to proceed, based on their assessment of the tactical or clinical information ( p. 76 77) Both the healthcare field and the aviation field have a low threshold for failure, high costs for training, and strive to limit mistakes by ensuring skills are mastered before individuals apply them in real situations, but currently the aviation field has used simulation to a much greater extent than the healthcare education field (Ziv, W olpe, Small, & Glick, 2003) made, the human body is in many ways still a pla ce of great mystery. Simulation based medical education can be traced as far back as 17th century France where birthing mannequins were used (McGaghie et al., 2010), but it is only recently that simulations have become a common part of healthcare education and training Early simulators mimicked human anatomy and were often used to teach basic psychomotor skills (Lynagh et al., 2007) In 1960 Resusci Anne was developed as a simplistic mannequin and was widely used to train mouth to mouth resuscitation an d c ardiopulmonary resuscitation (CPR) This model was used until the mid 1990s when the Laedral Company began to develop a higher fidelity mannequin In the late 1960s, Sim One, often considered the first true computer controlled mannequin simulator, was

PAGE 34

34 developed to train anesthesiology residents in endotracheal intubation This was a significant advancement in technology; however, it was little more than a proof of concept as only one was ever made (Cooper & Taqueti, 2008) In contrast, the Harvey ma nnequin was developed in 1968 and was able to simulate 27 cardiac conditions and three different diseases This technology has continuously been improved upon, both as the Harvey model and as numerous smaller cardiology patient simulators (Cooper & Taquet i, 2008) The Harvey model is still used today to help students develop and improve cardiac care skills. The 1980s saw an expansion of medical related simulator technology as both Stanford University and the University of Florida (UF) started development on high fidelity simulators The group at Stanford developed the comprehensive anesthesia simulation environment (CASE), which was commercialized as Medsim, and the UF group developed the Gainesville anesthesia simulator (GAS), which evolved into Medical Education Technologies, Inc (METI) (Bradley, 2006) and is now CAE Healthcare In the 1990s the military, which had been using simulation for training pilots and soldiers in combat scenarios, utilized internal organizations such as Defense Advanced Resea rch Projects Agency (DARPA) and Telemedicine and Advanced Technology Research Center (TATRC), to develop medical simulators for use in training medical personnel (Kunkler, 2006) As the technology advanced the cost of simulation decreased, making it more accessible to more teaching institutions and more appealing to developers (Issenberg et al., 1999) Cooper and Taqueti (2008) note that over twenty types of skill training devices have been developed since the mid 1980s Today medical simulators include the highly complex da Vinci Skills Simulator computer

PAGE 35

35 based simulations such as those developed by Mad Scientist Software, and a variety of high fidelity mannequins in a variety of ages, races, shapes, and sizes Some of t hese mannequins can breathe, bl ink, and bleed, providing real time feedback in order for students to develop new skills practice existing ones, and be evaluated on their performance Terminology As with any specialty there are numerous terms used to describe and explain simulators and simulations Simulations can be described by technology type, methodology, functionality, or design, but even the word simulation can cause problems when a precise educational definition is sought for its implementation (Alinier, 2007) One of the terms that se ems to have little consensus concern s the patient simulators as the literature uses both mannequins and manikins In this study the term mannequin is used based on the recommendation of the Society for Simulation in Healthcare (Gaba, 2006) The S ociety for Simulatio n in Healthcare also recommends not using the term human patient simulator to refer to generic simulators representing human beings as the Human Patient Simulator is a specific model produced by METI (Gaba, 2006) Therefore, this stud y uses the term patient simulator or mannequin simulator when referring to generic human patient simulator models. The term fidelity is typically defined as how accurate a reproduction is to the original (Lampotang, 2008), but in the world or healthcare si mulation the level of fidelity is not always clear The same technology can be thought of differently depending on the objective and task being simulated Simulations are described along a low to high fidelity spectrum, which refers to the level of real ism that a simulation presents (Feinstein & Cannon, 2002; Tuoriniemi & Schott Baer, 2008) They range in variety,

PAGE 36

36 abilities, and platform and can be anything from scenario based patient simulations (Jeffries, 2006) to highly realistic surgical trainers, o r comprehensive multimedia curriculum (Issenberg et al., 1999) There are also many different kinds of fidelity, including educational, procedural, visual, physical, and functional (Lampotang, 2008) and some simulations may have high fidelity in one area and low fidelity in another There is also great variety concerning the role feedback plays when it comes to deciding the level of fidelity simulators is the ability to provide immediate feedback about clinical decision and Lane, Slavin, & Ziv 2001 p. 306), but this feedback mechanism is often not available in models with functional lower fidelity In this study the following definitions of fidelity were used. Low fidelity s imulations are those which focus on teaching or practicing a single skill (Yaeger et al., 2004 ) Low fidelity simulations can be bench models which assist with skills such as knot tying and dissection, simple mannequins such as Resusci Anne, or video box trainers which incorporate real surgical equipment and video monitors for practicing hand eye coordination, camera operation, suturing, and cutting (Hammoud et al., 2008; Seropian, Brown, Gavilanes, & Driggers, 2004) On the other end of the spectrum are high fidelity simulators that can utilize motion capture, augmented and virtual reality technology, and can respond both verbally and physiologically to interventions and omissions on the part of the student (Leigh & Hurst, 2008) On some models response s can be either automatic or given by the instructor depending on the educational need High disconnect between preparatory exercise and actual incident by combining consistent

PAGE 37

37 lifelike clinical scenarios with re al Gillett et al., 2008, p. 1145) making its use a growing trend in disaster medicine The term high fidelity can be used to describe two types of fidelity, cosmetic (does it look real) and response (does it react in a realistic manner) ( Seropian et al., 2004) Examples of high fidelity patient simulators are the SimMan the METIman TM and Noelle with Newborn Hal In between the low and high fidelity simulators is a large array of partial task trainers and mannequins that help student s perform specific psychomotor skills and sophisticated computer based simulations aimed at teaching students to problem solve, perform a task, or make diagnostic observations (Hovancsek, 2007) Patient care scenarios often use medium fidelity full body s imulators, such as the VitalSim TM Anne which simulates breath sounds, but does not simulate chest rises or eye movements (Hayden, 2010; Seropian et al., 2004) High fidelity simulators are often more attractive to students (Hammoud et al., 2008), but are more expensive than the lower fidelity models and may not be optimal for teaching basic life support (BLS) skills such as those needed by many allied healthcare professionals This wide variety of technology can be both exciting and daunting when trying to choose the best technology or methodology to address the needs of the students, curriculum, and institution so it is important that institutions closely examine their requirements and their learning objectives before making a costly purchase It has n ot been verified that more technology always equals a better learning outcome Lampotang (2008) argues that high fidelity simulations may actually overload the learner with too much information and details, detracting from its effectiveness and overshadow ing the learning objective Haptic technology ( tactile feedback technology

PAGE 38

38 which gives a feeling of resistance) and virtual reality (VR) present virtual objects to learners in a way that is identical to their natural environment (Jones & Sheppard, 2007), but currently these technologies are expensive and not well studied for their effectiveness While Seymour et al. (2002) argue that VR training significantly improved the operating room (OR) performance of residents, Seropian et al., (2004) maintain that when it comes to VR devices, there is no convincing evidence that skills developed with these devices translate to better clinical performance or results Types of Simulation Mannequin based patient simulators The mannequins and partial task trainers dis cussed in the previous section are probably the most well known types of simulation for healthcare education and training Much of the history and variety of mannequin based simulators has already been discussed, but it is important to note that these sim ulators are only one option for educators Mannequin simulators were developed to reproduce aspects of the human patient (Bradley, 2006) and are used in a variety of ways to teach and assess student skills A mannequin based simulator can vary widely in cost, fidelity, and function, but p. 7) This interactivity gives the educator options when designing scenari os to meet the learning objectives and skill level of the student These simulators are often used in conjunction with clinical equipment and are placed in labs designed to replicate clinical settings (Gaba, 2004a), giving students an opportunity to pract ice skills in a real world context.

PAGE 39

39 Figure 1 1 Elizabeth Stewart, left, a nurse, and Jennifer Barreiro, a health technician, perform cardiopulmonary resuscitation on a mannequin during a mock code blue drill in the pain mana gement clinic at Naval Medical Center San Diego, Calif., Oct. 13, 2010. Mannequins offer health care providers the opportunity to practice using simulated patients and sophisticated technology. (U.S. Navy photo by Mass Communication Specialist 1st Class An astasia Puscian/Released)

PAGE 40

40 Partial task trainers It is not always necessary in healthcare education to replicate the human body in its entirety Surgeons, nurses, allied healthcare students and practitioners often need to learn or practice specific skills which require the simulation of only a specific body part or the use of specific instruments Task trainers can be found to support a wide variety of objectives including abdominal ultrasounds, pelvic examinations, urinary catheterization, and central li ne placement The goal of a task trainer is to teach the specific steps associated with a particular procedure and allow repeated practice of desired skills in order to become competent Malekzadeh, Pfisterer, Wilson, Na, & Steehler, 2011 p. 532) The Life/form Airway Management Trainer is a simulated head which allows students to practice intubation, ventilation, and suction techniques whereas the Gaumard S402 Patient Training Arm is a replication of an arm which allows for the practice of intraveno us, infusion, and blood collection techniques

PAGE 41

41 Figure 1 2 Sgt. Marlen Lopez examines the mouth of a mannequin head with a laryngoscope at an intubation class during the 2011 Warrior Exercise on Fort McCoy, Wis., Aug. 22, 2 011. Lopez is from 7246th Installation Medical Support Unit, based out of Omaha, Neb. (U.S. Army photo by Spc. True Thao/Released) Bench models are inexpensive and portable, allowing students to practice skills on a static model Hammoud et al. (2008) lis ts several types of bench models : tying trainers, tissue models for practicing dissection and suturing, abdominal opening and closure trainers, episiotomy repair trainer, anal sphincter repair trainer, and urethral p. 339) There is no automatic feedback mechanism with these Box trainers are only slightly more complex and are literally a box with slits to insert real surgical instruments These models include a camera and video screens so the students can see what they are doing in a manner which replicates clinical practice

PAGE 42

42 Box trainers are an excellent mechanism to train for eye hand coordination, camera handling, suturing techniques, grasping mechanisms, point to point movement clip applying, and cutting These also require an instructor to directly observe in order to provide feedback. The use of a partial task trainer helps to control costs and there are even videos on YouTube demonstrating how to build your own laparoscopic box trainer While some box trainers can cost between $215,000 and $285,000 depending on the quality of the surgical and video equipment used, some institutions, such as the University of Texas Southwestern Medical Center, have developed their own at a cost of $270 per graduating resident In comparison, operating room time to train residents is estimated at around $48,000 per graduating resident (Scott et al., 2000) T ask trainers allow students to concentrate on a single objective as they use and become familiar with the function and operation of real equipment and practice specific skills such as knot tying and suturing ( Malekzadeh et al., 2011) An honest assessment of the requiremen ts and learning outcomes will help institutions decide if and when to use a task trainer as opposed to a full mannequin Box trainers have been shown to be effective in teaching and improving basic psychomotor skills that are transferable to real tasks ( Clevin & Grantcharov, 2008 ; Katx, 2006; Munz, Kumar, Moorthy, Bann, & Darzi, 2004 ; Scott et al., 2000) There are limitations to these studies however, as they are often convenience studies conducted on residents It is also unclear how long the effects of this training last, and therefore it is unclear how often a student or practitioner needs to use such mechanisms to keep proficient It is also difficult to compare results from training on partial task trainers to

PAGE 43

43 training which occurs in clinical sit uations as the simulations are controlled environments while there are numerous variables in operating room scenarios. A ctor p atients It is important to understand that the term healthcare simulation does not just refer to a physical task trainer or manneq uin simulator ; they can also be case studies and drills where students use personal and diagnostic skills to properly assess and treat actor patients who have been moulaged (made up with fake injuries) This technique is often used to teach triage and oth er mass casualty skills ( Gillett et al., 2008) Actor patients can play diverse roles in an effort to teach students how to deal with a variety of clinical situations such as giving a patient bad news or how to interact with family members after a death ( Sinz, 2004) These actors are trained to role play patients, giving students real experience and allowing instructors to assess their abilities to take patient histories, conduct physicals, and communicate with patients, families, and other medical person nel (Ziv et al., 2003) Simulated patients can be used in either summative or formative evaluation (Ladyshewsky, Baker, Jones, & Nelson 200 0) allowing students to both practice and be evaluated on their performance before interacting with real patients i n a clinical situation.

PAGE 44

44 Figure 1 3 U.S. Navy Lt. Graham Danyleyko acts as a triage officer and attaches triage tags to a Sailor with a simulated injury during a mass casualties drill in USS Abraham Lincoln's (CVN 72) hangar bay while under way in the Pacific Ocean Aug. 5, 2007. (U.S. Navy photo by Mass Communication Special ist Seaman Chantell J. Wilson/ Released ) One of the most significant factors in the use of live patients for educational purposes is that the actors must be well trained in both their clinical and inter personal interactions, but there is some evidence in the literature that when done effectively, these types of simulated scenarios have high validity and reliability (Ladyshwesky et al. 200 0; Swanson & Stillm an, 1990; Vleuten, & Swanson 1990 ) Simulated patients may seem like a cost effect ive alternative to human patient simulations, but besides the costs of the actors themselves, institutions must factor in the time and costs associated with the use and set up of the facilities needed to run the simulation, developing the

PAGE 45

45 scenarios and instrumentation, and training the actors, staff, and faculty (Ladyshwesky et al. 200 0) The time needed and the cost could range widely depending on the scale and objectives of the simulation. Screen based simulations There are also a number of computer or screen based simulations that are used alone or in conjunction with mannequins Screen based simulations use the computer to graphically present a clinical scenario where the student selects diagnostic and therapeutic options (Hovancsek, 2007; Schwid, 2004) They can also be used to simulate processes such as the distribution and pharmacodynamics effects of anesthetic drugs (Kalkman, 2012) These simulations can be part of the classroom curriculum, or they can be run by the student on their own time in support of their didactic learning Screen based simulations can be used to train and assess clinical knowledge concerning a variety of decision making skills including: p erioperative critical incident management; problem based learning; physical diagnosis in cardiology; and acute cardiac life support (Ziv et al., 2003) Similar to the state of research concerning patient actors, the research concerning the effectiveness o f screen simulation shows some effectiveness (Schwid, Rooke, Ross & Sivarajan, 1999; Nyssen, Larbuisson, Janssens, Penderville & Mayne, 2002; Ziv et al. 2003), but is in the early stages of validation (Reznek, 2004) Screen based simulations are relative ly new as this technology required the advent of easily accessible personal computers (Lane et al., 2001); however, this method may have great potential as more individuals and technologies are going mobile through the use of smartphones and tablets.

PAGE 46

46 Fig ure 1 4 A simulated dental head used to train dental technicians ( DoD Visual ID Number 111209 N ZZ999 014 /released ) Summary When it comes to simulation there is a wide variety of technologies and methodologies to choose from B r adley (2006) argues that different types of simulation can be used in combination when the learning objective combines different types of skills such as technical (task trainer) and interpersonal (actor patient) Girzadas et al. (2009) agree, noting th fidelity mannequins, standardized patients, and task trainers, may be able to more fully p. 430) It is rare that one mod e l or method would meet all of an instituti It is important that decision makers understand the differences in types of fidelity, feedback mechanisms, outcome validation, and technical capabilities available in order

PAGE 47

47 to mak e an informed decision concerning what, when, where, why, and how to use a specific type of simulation Current Simulation Use Students respond positively to the use of simulations and often identify an increase in learner confidence and motivation (Curtin & Dupuis, 2008; Jeffries, 2005; Kuznar, 2007 ; Shear, Greenberg, & Tokarczyk, 2013 ); and confidence is vital to the healthcare provider (Henrichs, Rule, Grady, & Ellis, 2002) Allied healthcare can be a high risk profession and confidence is an important trait; efficacy will not create competent professionals without the (Kuznar, 2009, p. 10) Rodgers (2007) argues that learners are more confident in their abil ities when the learning environment includes high fidelity patient simulation, but also admits that the peer reviewed literature on the subject is deficient In their 2009 study Wenk et al. found that simulation based medical education did not lead to sig nificantly higher test scores, but it did lead to students overrating their clinical abilities and knowledge improvement When it comes to healthcare education, a high level of self efficacy does not automatically translate to a high level of knowledge or skill mastery The ability to accurately self assess is an important skill, and there has been some research that shows that simulation can be used to help students improve self assessment skills (MacDonald, Williams, & Rogers 2003 ; Sadosty et al., 2011 ) but it is unclear how this impacts their clinical abilities. Skills labs and simulation centers have become widely used for programs such as orthopedics, anesthesiology, nursing, dentistry and veterinary medicine, but the level of integration and valida tion varies widely from program to program and school to school The literature make s it clear that to be effective ly measured as a learning

PAGE 48

48 methodology simulation based education must be tied to a clear objective that can be evaluated and not simply use d because it appears to be engaging Simulation should its ability to impart knowledge (medical knowledge), the translation of knowledge into action (therapeutic intervention) and perhaps most powerfully, the action and its impact on patient outcome (success or failed treatment of disease) (Shear et al., 2013, p 160) In 1910 the Flexner Report to the Carnegie Foundation established that healthcare education should contain two components; a post secondary based scientific curric ulum and a clinical practicum (Flexner, 1910) Since then medical educators have strived to find a balance between scientific knowledge and clinical practice to be a student and instructor friendly meth od to achieve this balance, teaching theoretical concepts and allowing students to actively apply them to various simulated p. 11) Some believe that w hen properly integrated into the curriculum simulation can help alle viate gap s between theory and practice (Morgan, Cleave Hogg, Desousa, & Lam Mcculloch, 2006) Both aspects are important but simulation needs to become a reliable and validated method to supplement limited clinical rounds before it can truly assist in achieving this balance Even when students are lucky enough to receive extensive time in a clinical setting, there is no guarantee that every student will receive an identical clinical experience, see a representative patient mix, and learn all the necessary skills (Lane et al., 2001; Reznek, 2004) The use of simulation allows students to experience uniform scenarios which cover a variety of issues giving students th e opportunity to evaluate

PAGE 49

49 and practice skills they may never have an opportunity to practice in clinical settings Clinical rotations, while an important part of the learning process, are problematic for developing communication, problem solving, and inte rpersonal skills ( McLaughlin et al., 2010) as patient care and safety is the priority The use of simulations helps to ensure students can practice and be evaluated on these skills without endangering lives or disrupting patient care Other factors affe cting the increased use and interest in simulation in healthcare education include issues with teaching basic anatomy and physiology Traditional dissection techniques are feeling the pressure from the decreasing availability of qualified anatomy instruct ors and the high cost and limited availability of cadavers There is also some concern with the ethical considerations of using human bodies, the potential of cadavers to transmit infectious diseases, as well as the possible negative health effects of for maldehyde exposure (Anastakis et al., 1999; Gabard, Lowe, & Chang, 2012; McLachlan, Bligh, Bradley, & Searle, 2004) Many medical schools have also traditionally used live and dead animals to train certain procedures Practicing on live animal tissue has numerous ethical implications as research now supports the idea that animals experience fear, stress, pain, and other emotions (Ferdowsian, 2011; Rollin, 2006) Live pigs have often been used in medical training as they display anatomical and physiologic al similarities to humans (Tanaka & Kobayashi, 2006), but no matter how close, animals have a different anatomy and they require special storage, feeding, and maintenance ( Liu, Tendick, Cleary, & Kaufmann, 2003 ) Technology such as simulation has slowly s tarted changing the practice of using live animals in medical education and experiments

PAGE 50

50 medical and veterinary students were forced to bleed animals out and watch the stages to death Today such labs, onc e ubiquitous, have been replaced by films or computer p. 299) Currently there is very little evidence based research on the effects of students learning concerning the use or non use of cadavers and/or live tissue; however, in some cases the decision to use simulation instead is based on other considerations. The healthcare educational field has been criticized for adopting and implementing innovations such as simulations without sound ev idence for their efficacy (Bradley, 2006) S imulation is now firmly established in healthcare education (Kneebone et al., 2006), but an overload of products and concepts not supported by validation or institutional expertise has often lead to the purchase of simulation equipment without the institution understanding how, when, and why to use it (Seropian, 2003) Research still has not adequately answered if the use of medical simulation improves clinical skills when compared to standard training technique s or if skills learned via simulation are retained over time (Greenberg, 2004; Lynagh et al., 2007) The future of simulation in healthcare education may be determined by the answers to these questions and the ability of educators to successfully develop a standard for when and how to use it. Evaluating Simulations While there are some evaluation tools designed to assess student learning outcomes, detailed studies related to assessing the simulations themselves are limited he level and type of simulati on will need to be adapted to the educational needs of the learner and the design and intended outcomes of the programme

PAGE 51

51 ion needs and intended program outcomes Will a simulator provide better and more effective training than other methods? Why are certain simulations used to teach certain actions or skills? When is a high fidelity simulator necessary and when will a low f idelity simulator work? Can simulators be used for multiple purposes? What is the best way to assess new simulators for purchase? Finding answers to these and similar questions are crucial if a best practice for acquisition, assessment, integration, and ap plication of healthcare simulations is to be established (McGaghie et al., 2010) In the future, educators may find that while there are some limitations with the simulation technology itself, the challenge facing healthcare education may be keeping up w ith the pace of changing technology (Hamilton, 2005), but today many of the challenges come from a lack of understanding concerning instructionally sound simulator technology and simulation methodology The success of the simulators also depends on the de velopers of the technology, who do not necessarily have the same agenda as those who use them (Kneebone, 2003; Wilson, Shepherd, Kelly, & Pitzner, 2005) The design, development, integration, and use of simulation are often two separate enterprises It i s important that those using the technology communicate with developers to push for technologies that support their learning objectives. Hammound et al. (2008) argue that the first step for evaluating any simulator based training (SBT) is defining the simu The second step is to define the relationship between the simulator and the simulation scenario Kneebone (2005) goes further, stating that there are four criteria which should be considered when

PAGE 52

52 designing, implementing, evaluating, or purchasing simulators for procedural skills The simulations should 1. a llow for a safe environment for deliberate practice within a defined curriculum, ensuring that recently acquired skills are reinforced; 2. p rovide scaffolded support and access to exper ts, ensuring new learners have greater access to feedback and support than advanced learners; 3. a ccurately reflect real world clinical experience, ensuring that learning supports actual practice ; 4. a fford a supportive, motivational, and learner centered enviro nment. When strategically evaluating the use of medical simulation, Bonnel and Smith (2010) recommend taking into account the following questions. Will a simulated experience help the student achieve the learning goals? Are there other techniques that co uld be used which could help the students achieve those goals? Which of these techniques best fits the learning objective? (Or if they are equal which is the least expensive, simplest, and easiest to implement?) What support structure is in place for th e simulator, including training, vendor support, and proper facilities? What options are there for presenting the material if the simulator is unavailable? These are all important questions to ask and issues to consider, but the decision maker(s) are giv en little guidance on how to find answers to these questions The answers and solutions will be unique and contextual based on the tasks to be taught, the level of learning being supported, and the outcome objectives as well as the financial and logistica l constraints of the institution Keeping all these issues in mind as institutions plan for the future is a difficult task, and one that is compounded by the limitations of simulations.

PAGE 53

53 Limitations of Simulation Many of the limitations of healthcare simul ations are tied to the simulation type (Gaba, 2004 b ; Hammoud e t al., 2008; Kunkler, 2006; Liu et al. 2003) However, some are inher ent in the methodology S imulations take time, money, and space including storage space Knowledgeable faculty and staff are required to develop scenarios, run the simulations, provide helpful feedback, and maintain the simulator and peripheral equipment (Moreno Ger et al., 2010) It is also important to keep in mind that healthcare simulators currently cannot simulate all scenarios under all circumstances ing its mental and behavioural aspects, a system that can not be fully or effectively replicated .209) Human behavior is not a constant, nor is human physiology . Our p. 51 ) It is possible to simulate an average or an expe cted scenario, but it is also important to teach students how to think critically and problem solve so they are prepared to deal with the unexpected Knowing when to seek help or ask questions can be as important as knowing how to perform a procedure. Eve n the best run simulations cannot fully recreate the chaos, emotions, and ambiance of an emergency room, field hospital, or first responder situation (Guillaume, 2007) No matter how detailed or realistic a simulation is, it is still just a simulation; it can help a student learn and practice skills and gain confidence, but until those skills are put into practice it is unknown just how well a person will perform Treating and caring for patients is a complex cognitive process no matter if the caregiver i s a laparoscopic surgeon, a clinical nurse, or a physical therapist Simulators are often

PAGE 54

54 used to develop and refine fine motor skills specifically needed for a procedure; n Determining what features are required for successful training in a simulator is thus as p. 76) Other concerns are with the lack of research and awareness concernin g curriculum integration, embedded feedback, human simulator interfaces, the seamless integration of simulation modalities, and data representation (Rosen, 2004) Educators have not developed pedagogical standards for how, why, or when to use which simula tion versus more traditional training or assessment methods. Fast, realistic mechanical modeling of soft tissues and organ properties is also a challenge for simulators (Liu et al., 2003) The elasticity, look, feel, and behavior when cut or bruised varie s greatly by simulator, but no simulator can fully replicate human tissue and its reaction to medical and surgical instruments (Kerdoka et al., 2003) Real time tissue and organ deformation is also limited to specific organs or simple structures such as a rteries and ducts (Liu et al., 2003 ; Rosen, 2004 ) These limitations are why a few medical educational programs, including some military programs, still use live tissue (e.g., live pigs or goats). Shaffer et al (2001) also note that tactile skills are on ly a part of what healthcare students need to learn They must also learn to make difficult and complex clinical decisions, which often involve medical, social, moral, and psychological factors some decision making ability can clearly be develope d through interaction with a simulation curriculum, true clinical wisdom is developed only through interactions with p. 77) There are certain aspects of healthcare

PAGE 55

55 practice that are learned through experience as every patient, environment, procedure, and circumstance varies While there are adult and infant simulators, there is currently no simulator designed specifically for geriatric patients, a growing population with illnesses that are often not curable (Bruhn & Ph ilips, 1985) Moral and ethical dilemmas are also a part of healthcare that students need to learn about and practice managing Simulated scenarios using standardized patients or ethical situations such as famous Heinz's dilemma can be used t o help students identify, understand, and mentally prepare for these potential problems ( Tysinger, Klonis, Sadler & Wagner, 1997) will need doctors they can trust; and the ne cessary qualities of humanity, compassion and professionalism can never be acquired from skill centres or the internet only by (Hamilton, 2005) Mentoring is still an important aspect of healthcare education. It is impractical to believe that a s imulated scenario, no matter how realistic or immersive, can truly prepare students for the complexity of dealing with live patients and their families The responsibility of patient care can only be understood when a caregiver is accountable for the life and health of another person (Jordan, 2003/1903) These are skills that come only from experience, practice, and repetition (Cantrell & Deloney, 2007) Personal interactions are required to ca re for the patient as a person as t echnology cannot substitu & Philips, 1985, p. 294) These limitations should be kept in mind when deciding which educational objectives will be taught through simulation.

PAGE 56

56 The Cost of Simulation Cost can also be an issue for those seeking to integrate simulation into healthcare education Moreno Ger et al. p. 460) A single simulator mannequin can cost between $5,000 and $200,000 depending on the level of technology used, number of accessories, and realism achieved (Issenberg et al., 1999) Some institutions are deterred w hen the costs of simulator equipment maintenan ce, ancillary equipment such as drug carts, patient monitors, and defibrillators, facility space maintenance for simulation labs, and salaries for support personnel is added to the budget ( Adamson, 2010; Lee et al. 2010 ; Okuda et al., 2009; Schwid, 2004 ) For fiscal year 2009 a national survey of medical schools and teaching hospitals conducted by the Association of American Medical Colleges (AAMC) reported that for the majority of respondents, their annual operating budget for their simulation centers w as under $500,000, but some reported budgets of over a million dollars (Passiment, Sacks, & Huang, 2011 ; Tuoriniemi & Schott Baer 2008 ) So how can institutions with limited resources justify the cost when research does not consistently show effectivenes s or successful transfer of training to clinical environments (Cooper & Taqueti, 2008 ; Lynagh et al., 2007; Rodgers, 2007 )? It is important to plan for the use of simulation in a way that w ill help increase the likelihood of successful integration and ensu re respo nsible use of education dollars Some innovative programs have successfully introduced simulations to institutions with limited budgets (Curtin & Dupuis, 2008), but they required extensive planning, research, and a deep understanding of the progr While

PAGE 57

57 these courses of action are recommended for any institution regardless of budget, few seem to have the expert knowledge or resources needed to accomplish this McLaughlin et al. (2010) built a matrix for simulation integration an d implementation at the community college level and breaks it down to five levels The levels range from multiple simulations to no simulation, but demonstrate how institutions can implement a wide range of options based on the type of equipment, staff tr aining, leadership buy in, institutional mission, and infrastructure Planning for Simulation According to Haluck et al. (2007) shopping for simulators is like shopping for any piece of electronic equipment; there is often a difference between what is de sired and what is needed and affordable With all that is involved in choosing, buying, using, and supporting simulations, it is crucial that educators also identify potential future issues associated with using simulations In order to plan and prepare for the future it is important that institutions accurately understand how these potential issues can impact them Haluck et al. (2007) observe d that for most situations (and it is certainly true at a place as diverse as METC ), one type of simulator will not provide all that is needed to address all training needs Some combination of mannequin based simulations, physical part task trainer, and high fidelity sim ulation will probably be needed, but there is little comparative data concerning simulator attr ibutes ( Bradley & Postlethwaite, 2003a ) making decisions about current needs difficult and forecasting future needs nearly impossible At the same time it is important to understand that just because a simulation is available it does not mean it is the b est option ( Lane et al., 2001) When assessing their future needs educational institutions should look at all options as simulation is not always the answer.

PAGE 58

58 Planning for the future is essential to ensure that institutions are capable of producing fully c ompetent healthcare practitioner s given their numerous time and budgetary constraints This planning can be biased if carried out by a small group of individuals who do not possess the knowledge and/or empirical data needed to formulate a plan capable of issues and concerns (Wicklein, 1993) Therefore, i t can be in the best interest of the institution to seek input from a wide range of experts from outside the institution echnologies there is an enormous amount that can be accomplished with simulation that is not being done because the institutional mechanisms for providing it p. 1) There is more to using simulation than simply purchasi ng the technology; a holistic approach allows educators to assess curriculum, match technology with learning outcomes, and integrate different methodologies where most appropriate. The effectiveness of simulation in healthcare education is often diminished due to a lack of knowledge about how to use the equipment, h ow to set up the scenarios, and how to evaluate the simulation Jeffries (2006) notes purchased either without a plan or faculty willing to implement the innovative teaching p. 163) It is not known how underutilized simulation technology is in healthcare education (Adamson, 2010; Huang et al., 2012), but the research suggests that the barriers to full implementation and integration have led to th ousands of dollars in misspent purchases. Research Challenges The limited number of methodologically sound, long term studies with large sample sizes which demonstrat e that simulation is more effective than traditional

PAGE 59

59 methods (Bradley, 2006 ; Bradley & Pos tlethwaite 2003b ) also makes deciding how and when to use simulations difficult Garden (2008) lists six consistent flaws found in simulation based research, including poor knowledge of the literature, lack of awareness of basic research design, poor att ention to measurement properties, and no attention to statistical power These flaws make the validity and applicability of the existing literature questionable There is also a question of bias in much of the simulation research as a quick search of the literature finds that the research funding often comes from an academic simulation center where the research was at least partially funded by the academy (Wenk et al., 2009) or by simulation companies such as Laerdal Medical and METI There is little kn own concerning the conditions for the successful implemen tation of innovations in health care ( Fleuren et al., 2004) and Rosen (2008) lists three reasons why there has been slow progress concerning the acceptance and innovation of healthcare simulations : s kepticism ; lack of communication ; and the burden of proof Bradley and Postlethwaite (2003 b ) add that much of the current simulation research validity and reliability ready when proponents can show that clinically useful learning results from simulation p. 16) This lack of empirical proof has not stopped many educational insti tutions from moving forward with simulation acquisition, but it has made successful integration difficult (Jeffries, 2006) Institutions often fail to effectively network with each other to share resources, best practices, analysis, or outcome data (Adams on, 2010) Bradley (2006) argues that some issues with small sample sizes and

PAGE 60

60 few class iterations could be addressed by using data from multiple sites, but currently many simulation initiatives are contained to educational and professional silos (Blackst ock & Jull, 2007; Huang et al., 2012; Jeffries & Battin, 2012) Collaboration and the exchange of ideas will be imperative to the future of simulations in healthcare education (Rosen, 2008) These collaborations should cross not only institutional bounda ries, but include input from clinicians, simulation developers, researchers, and educators. While simulation has become more common in healthcare education and training, it still has a long way to go before reaching its full potential and unquestionable ac ceptance (Liu et al., 2003) Kuznar (2007) argues that adopting an advanced methodology without critically evaluating and determining its effectiveness could be considered negligent with regard to student learning A few even hypothesize that this lack o f evidence will make simulation obsolete in the future (Greenberg, 2004), but most agree that there is still great potential for simulation to positively impact healthcare education if the proper educational and methodological steps are collectively taken to develop both the technology and the methodology Simulation in Allied Health Education This study assumes that clinical practice must be a significant compo nent of the allied health care providers The practice of healthcare is changing, an d simulation allows allied health professionals to better prepare for this changing environment The need to practice and prepare for a variety of common and rare situations is one of the basic arguments for using simulation in healthcare education ( McLau ghlin et al., 2010; Reznek, 2004); however, t he National Council of State Boards of Nursing survey found that 96% of all associate degree simulation labs (N=523) use

PAGE 61

61 simulation to practice routine results, whereas only 61% p ractice rare patient scenarios There is still a gap between what is theoretically possible with simulation and the practice of simulation. Simulations and the Role of Patient Safety In 1999 the Institute of Medicine (IOM) released To Err Is Human in an effort to make health care in Ame rica safer by shedding light on the number of preventable medical errors. The report stated that preventable hospital errors cause over one million injuries and the death of 44,000 to 98,000 people each year. To Err Is Human made public the concern that the decentralized and fragmented healthcare delivery system often gives little incentive for health care providers to improve safety and quality as breast cancer, mot T he issue was taken up by the media and through it the public became more conscious of the issue In 2006 a review of the patient safety literature revealed that it was extremely difficult to assess t he impact of To Err Is Human as there was no comprehensive nationwide monitoring system for patient safety (Stelfox, Plaminsani, Scurlock, Orav, & Bates, 2006). Many of the problems noted in this report remain over a decade later as 30 % of healthcare wor kers in industrialized nations still fail to wash their hands and failures in teamwork and communication are recognized as causing more than 75 % of medical errors and injuries (Gordon, Mendenhall The public may be more aware of the probl em, but the industry has yet to find a wide spread viable means to address the problems impacting patient safety. To Err Is Human (1999) was soon followed by Crossing the Quality Chasm (2001) which called for a complete redesign of the health care system i n the U.S.

PAGE 62

62 (Berwick, 2002). This report is broken down into six areas in need of improvement: safety; effectiveness; patient centeredness; timeliness; efficiency; and equity. Perhaps the most far reaching assertion this report makes is that these changes cannot be p. 4) the system of care has to be redesigned from the ground up if the healt hcare system is to provide safe high quality care It is in the chapter concerning the preparation of the workforce that there is a pronounced place for simulation to be a part of this healthcare redesign. Many of the issues needing to be addressed are i n areas that have been or are seeking to utilize simulation: team building; patient communication; identifying errors and hazards to care; and the use of decision support systems. Crossing the Quality Chasm (2001) calls for wide ranging changes in the cur riculum, and simulation should be seen as one tool that can be utilized to help improve healthcare education. The use of simulation in education and training may help make initial interactions with patients safer as experience can help decrease error rate s (Schmidt, Goldhaber Fiebert, Ho, & McDonald, 2013). The literature concerning the link between simulation practice and improved quality of delivered care is still sparse but it is beginning to show some positive relation (Shear et al 2013). Despite t he sparse evidence, p atient safety has become one of the most frequently cited reasons for using simulation as a teaching strategy ( Gough, Hellaby, Jones, & MacKinnon, 2012; McLaughlin et al., 2010; Seropian et al., 2004; Shear et al., 2013; Tuoriniemi & S chott Baer 2008) as making a simulated mistake both decreases

PAGE 63

63 working with live patients. One of the IOM s recommendations was shorter work hours for residents. The inte nt was to improve training while limiting the harm to patients due to the traditionally long hours of residency. This resulted in many institutions looking at simulation for alter native ways to ensure residents experienced and practiced various scenarios The recommendation that simulation and team training be used to improve patient care and patient safety helped increase funding for simulation research, improve simulation technologies and increase the number of simulation centers to support the adoptio n of best practices and technologies (Nishisaki, Keren, & Nadkarni, 2007; Shear et al., 2013) Its overall impact on patient safety is still being assessed and work still needs to be done to ensure simulation can address the issues noted in both reports. In the aviation field, simulation has been used not to just in conjunction with checklists and protocols to help increase safety, it has brought about a major cultural and behavioral shift, one still lacking in healthcare (Gordon et al., 2012). Despite th e growing use of simulation there is little research which quantifiably ties its use to improved patient safety. Nishisaki et al. (2007) noted that there is no evidence that simulation improves patient outcomes as studies on this topic are difficult due t o time, resources, large numbers of trainees needed, and the possibility of contamination. The Changing Role of the Allied Healthcare Worker The population in the United States is aging, presenting healthcare providers with an increasing patient load of in dividuals facing chronic and difficult illnesses (Lizarondo et al., 2010) Another growing trend is home health care services (Lipton, 2009; Mase & Wattenbarger, 1980) an area where allied healthcare workers are often the primary care giver In this env ironment,

PAGE 64

64 for bringing a theoretically sound and evidence based approach to problem solving in p. 161) Patients today te nd to be active participants in their care as the Internet has made it easier for patients to research problems and potential solutions (Bradley & Postlethwaite, 2003a ; Lizarondo et al., 2010; Masys 2002). It is important that allied healthcare workers l earn a wide variety of medical, technical, and social skills as well as the ability to think critically and act ethically in the best interest of their patients. In order to curtail costs, many of the routine tasks previously handled by doctors are being a llocated to allied health care workers or other medical support staff (Cooper, 2001; Lizarondo et al., 2010) Patient care in many cases has shifted from being largely based in a hospital to being predominantly outpatient, limiting student exposure to prob lem solving and clinical skills acquisition (Morgan et al., 2006) O ne method managed care is using to help mitigate costs is to put more emphasis on patient education and prevention (Robinson, 1993) where many allied healthcare workers interact with pat ients to help mitigate or prevent problems, rather than simply clinically treat them A few innovative programs target patients at high risk for hospital to face consultations from allied health professionals fo llowing discharge; and allied health professionals worked in close p. 1946) This role requires allied healthcare professionals to have both a wide understanding of health problems and treatments a s well as strong interpersonal skills Allied health professionals are an high risk patients and their families, patient self management support, consultations

PAGE 65

6 5 with o ther health professionals, maintenance of close relationships with the patients' primary care physician, and intense, individualized follow up with patient and family p. 1946) This increase in responsibilities has led to an increasing need for skilled allied healthcare professionals and places new demands on community and technical colleges to find creative solutions for educating ever larger numbers of students under increasingly restrictive budgets ( McLaughlin et al., 2010) These solutions must be cost effective and produce successful learning outcomes to positively impact patient care. The changing needs and practice of the industry is prompting changes in how practitioners are educated, but these changes are slow t o be constructed, verified, and adapted confirm that there is unjustifiable slowness of change and incomplete implementation of Masys 2002, p. 35) Addressing the issues of cost, acquisition, training, integration, and validation are essential if allied healthcare education is to produce qualified professionals ready to take on these new challenges The cost of simulators can be partially contained by carefully choosing the lowest fidelity model needed to meet the largest number of learning objectives It may be cost effective for novice trainees to utilize low fidelity devices during the first stages of learni ng (Feinstein & Cannon, 2002 ) Some argue the low fidelity models are effective for learning new skills ( B radley & Postlethwaite, 2003 a; Wilson et al., 2005), while others argue that these low fidelity simulations do not produce skills that will transfer to real life situations (Seropian et al., 2004; Tuoriniemi & Schott Baer 2008) So while

PAGE 66

66 there is no definitive answer as to the effectiveness of cheaper, low fidelity models, institutions wishing to utilize simulation need to look at creative ways to ge t access to the technology they need The consortia model has been in effect between community colleges and four year institutions as it pertains to degree plans since the late 1960s ( Hendee, 1971; Mase & Wattenbarger, 1980) and can be expanded to suppor t simulation programs, thereby helping to divide responsibilities and costs (Jeffries & Battin, 2012 ; Sportsman et al., 2009 ) Another possible way to control costs is to utilize mobile simulation labs such as those used in Idaho, Florida, and Indiana T hese labs give multiple sites access to simulation while sharing the cost for the equipment, space, and technical personnel. Even if institutions find a way to distribute costs and responsibilities, the question remains, what are the hallmarks of a good si mulation for allied healthcare education ? Research by Grober et al. (2004) found that surgical skills training on low fidelity simulators is as effective as training on high fidelity model training for the acquisition of technical skills among novice surge ons but it is unclear if this is applicable to other skills and tasks Jeffries (2007) argues that the complexity of the simulation should be individually suited to the requirements of the learning objective as well as to the level However, t here is little research at any level that shows educators which teaching and learning process need s to be incorporated by the simulator (Jeffries, 2006 ) making it difficult to determine what the technical requirements are for the simulat or

PAGE 67

67 In addition, when it comes to institutions with allied health programs, they often have several certificate or degree programs, (e.g., San Diego Mesa College offers Associate degrees or certificates in Animal Health Technology, Dental Assisting, Health Information Technology, Medical Assisting, Physical Therapist Assistant, and Radiologic Technology) Many of these programs require similar training in anatomy and life saving techniques, but require different training when it comes to clinical applicati on That does not mean that the same simulation cannot sometimes be used to teach different clinical application s In clinical settings multi disciplined teams work together towards a common goal, and while each member of the team has their own tasks and skill sets, simulation can be used as an educational tool to build these teams. Buelow et al. morbidity rates, fewer hospitalizations, decreased costs and improved function by patients amon p. e110) Teamwork, coordination, and collaboration are central parts of the allied healthcare communicate and w ork with both patients and their fellow healthcare professionals Buelow et al. (2008) conducted an inter discipline live clinical case simulation at the University of South Dakota and found that many students had positive experiences, but that there were challenges to working in teams Student comments noted that it was difficult to prioritize ideas; there was a tenden cy towards ethnocentric thinking; and opinions values, and emotions A few studies conducted on inter professional simulated activities found that s tudent satisfaction and perception of learning were high

PAGE 68

68 ( Gough et al., 2012 ; Zhang Thompson, & Miller, 2011 ), but were inconclusive concerning learning outcomes Those studies that do exist concerning i nter professional educational interventions often lack rigor in terms of design and the use of validated investigator developed questionnaires (Zhang et al., 2011 ) Like other simulation methodologies th e common conception is that inter professional education is beneficial, but it needs a reliable methodology, educationally sound curriculum integration, and rigorous research to support its use. There are still many unanswered questions when it comes to th e use of simulation in allied healthcare education presents educators with the challenge of deciding how to apply this technology to achieve the most effective simulation al., 2008, p. 340) The literature does little to shed light on how a simulator or simulation can be utilized across programs as there is a distinct lack of cross disciplinary or even general medical practice research concerning the effectiveness of simu lators (Lynagh et al., 2007) So while one simulator type may work for multiple programs with similar courses there is no roadmap to assist institutions in their decision making concerning acquisition or integration reflectively embark on a cautious and thoughtful integration of simulation This should be done with an eye towards outcomes, best practices, and what is best for the students and the healthcare et al., 2010, p. 472) This approach requires an inter disciplinary team, which takes into account the student population, the learning objectives, institutional capabilities, and the clinical skills needing to be taught.

PAGE 69

69 There is a limited amount of evidence based research conc erning allied health professions in general which leads to an absence of theoretical and scientific based research concerning evidence based practice, optimal training methods, and program evaluation (Arena et al., 2011) Research concerning simulation us e in allied health care education faces the same problem as other research areas in allied health care based, attempting to translate the experiences of clinicians and the science of other fie lds (medicine, basic science, education) into practice without discipline 2011, p. 163) L imited research exists which studies simulation use in specific allied healthcare disciplines One study was found concerning e ndotracheal intubation (ETI) which showed that paramedic students trained to intubate on a human patient simulator are as effective as students trained on human subjects (Hall et al., 2005) However, this study demonstrates many of the weaknesses found in simulation studies: while it was randomized, it was done at a single site; with a small sample (n=36), and outcomes were tested in a single setting. There have been several studies concerning simulation and physiotherapy education (Blackstock & Jull, 2007 ; Ladyshwesky et al. 200 0) Blackstock and Jull (2007) support the use of high fidelity simulation in physiotherapy (physical therapy), but simulation into the curricu lum, or evaluating the outcomes of such an education medium p. 4) They also argue for the need to develop physiotherapy specific simulation equipment On the other hand, Ladyshewsky et al. (2000) studied the

PAGE 70

70 reliability and validity of s imulated patients in physiotherapy education They argue that simulated patient scenarios can result in high reliability and validity if the actors are properly trained, the checklist validated, and the simulated procedures specific They also note that their study was limited and more research needs to be done concerning other disciplines and types of scenarios Miles Tapping Dyck, Brunham, Simpson, and Barber (1990) note that in the field of physical therapy there is some frustration on the part of cl inicians in that the available research rarely applies to practice. In 2007, Jones and Sheppard attempted to review the literature concerning physiotherapy education and medium fidelity simulation, but due to the limited number of studies available they ha d to expand their search to other allied healthcare and medical fields While 6% of the studies they found reported positive results in favor of using simulators as a training method, 28% found no difference between simulation and other methods Perhaps studies which found no difference between methods of training were scored at high quality on critical appraisal Only five of the twelve studies with positive findings were appraised as high qualit p. 2) As simulation use becomes wider spread in allied healthcare education more research should be conducted to assess its effectiveness. Because little research has been done on allied healthcare education it is necessary to extrapolate data from s ome studies done on associate degree nursing programs The issues and concerns of associate degree nursing programs and their use of simulations are a place to start when addressing potential issues for allied healthcare education and are arguably closer in type and scope than those issues faced in surgical or graduate level programs A study conducted by the National Council of

PAGE 71

71 State Boards of Nursing found that in 420 associate degree programs, the top barriers to increasing simulation use were: more fa culty need to be trained in facilitating simulations (78%); and not enough staff to run the simulation controls and oversee the students (72%) Two other issues, more faculty need to be trained in writing scenarios and more faculty need to be trained in d ebriefing simulation, were at 57% (Hayden, 2010) These issues and concerns should be noted and addressed by allied health programs looking to integrate simulation into their curriculum. While many medical schools have reported spending $500,000 or more o ver a 12 month period on simulation activities (Huang et al., 2012), community and technical colleges typically have smaller budgets Adamson (2010) noted in her study of simulation use in associate degree nursing programs, schools only allocated between $2,000 and $5,000 annually for maintenance and training for an average of 1 7% of their initial investment Among other things, she found that the lack of ongoing support meant that skills were lost over time, machines fell into disrepair, and scenarios w here not updated to match changes in the curriculum A long term strategy which includes simulation support mechanisms should be in place before acquisition and integration. The Role of Debriefing The debriefing session should be the last step in the simu lation activity as it which allows the participant to link theory to practice and research, think critically, and discuss how to intervene professionally in very complex p. 101) The debriefing session follows a constructivist framework and implements the principles of adult learning theory, offering a safe space for self assessment and interactive reflection While feedback is a continuous process, the debriefing period

PAGE 72

72 between the learner and facilitator, setting the stage for the learner to self correct p. 546) Through the reflective and interactive nature of the debriefing session the student has the opportunity to further reflect on the actions they took and the outcomes of those actions They also have the opportunity to learn from the actions of othe rs. The debriefing session is facilitated by one or more instructors and is often supported by video of the simulation event During the session the instructor honestly reviews individual and team performance in a non threatening manner and cr e ates an env ironment of trust (Fanning & Gaba, 2007) This does not mean that there is no constructive criticism; this session needs to be a time to accentuate, reinforce, correct, and when appropriate, challenge the student (Mort & Donahue, 2004) T he re is signific ant research that states that the debriefing session is an aspect of simulation use that is pedagogically vital (Bonnel & Smith, 2010; Childress, Jeffries & Dixon, 2007; Leigh & Hurst, 2008) and some of the literature maintains that the debrief is the mos t important part of the learning experience (Fanning & Gaba, 2007; Kalkman, 2012; Mort & Donahue, 2004) It is this period of guided reflection, correction, and analysis that supports the building of knowledge and skills through the simulation modality. T he debriefing session is a vital but often overlooked step in healthcare simulation due to the time, training, and cost involved in conducting a quality session It is important that the debriefing session is conducted by clinically knowledgeable and flex ible faculty with a working theoretical understanding of facilitated learning (Mort & Donahue, 2004; Siddall, 2008) This often requires instructors to add new

PAGE 73

73 responsibilities and learn new skills It can also mean adding audio and video support staff a nd equipment The cost to obtain and support high quality audio video equipment and the staff to run and support these technologies is an often overlooked aspect of simulation ( Bradley & Postlethwaite, 2003a ) Despite the extra cost and effort involved i n planning and delivering quality debriefing sessions it is a valuable tool which aids in the effectiveness of simulation use. Summary Many healthcare programs are investing in simulation but fail to adequately plan or allocate resources for educating pers onnel on how to effectively use the equipment (Adamson, 2010) Simulations are used in allied healthcare education to help students interventions, prioritizing and carrying out interventions efficiently, cooperating with other members of the staff, and managing complexity ( Tuoriniemi, & Schott Baer 2008) Caring for patients is a collaborative practice especially as it pertains to disciplines such as geriatrics, rehabilitat ion, and primary care disciplines (Robinson, 1993) and simulation can be used to build effective teams The few studies conducted on inter professional simulated education (IPSE) found that s tudent satisfaction and perception of learning were high ( Gough et al., 2012 ; Zhang et al., 2011 ) but fail to prove that the skills learned successfully transfer to clinical settings or are retained over time Gaba (2004b) eloquently summed up a possible way forward for healthcare simulations : The simulation community must educate the public and the implementing agencies on the vision of improved patient safety using the tool of simulation The simulation community must also provide the core leadership in developing standards for simulators, curricula, and simulation centres

PAGE 74

74 benefits equivalent to multiple randomised clinical trials, we should assemble the evidence where we can, and be forthright in our drive to move forward where possible without ironclad proof ( p. i9) There are still many unanswered questions as pertaining to the use of simulation in allied healthcare education It is recommended that educators, practitioners, and researchers work together to gather reliable and valid data that supports the use of simulation to supply community and technical colleges with best practice s for the acquisition, integration, and support of simulation in allied healthcare education. Theoretical Framework There are several theories and methodologies that are us ed to support the use of simulation in allied healthcare education Some studies suggest that simulation may be an effective teaching method because it employs four key facets of healthcare education: developing technical proficiency through practice of p sychomotor skills and context; and incorporation of the affective (emotional) component of learning (Kneebone, 2005) Thus, simulations allow for a learning experience that involves the entire person learning is an active process, where mental representations of learning material are constructed and integrated into existing knowledge structures This theory allows for a student centered active learning approach consistent with healthcare simulation methodologies Other education principles that are discussed in relation to the use of healthcare al learning theory, active learning, collaboration, diverse ways of learning (Jeffries, 2006), situated learning, reflective practice, activity theory (Bradley & Postlethwaite, 2003 b ) brain based

PAGE 75

75 learning, social cognitive theory, and the novice to expert continuum (Rogers, 2007) This Delphi study relies mainly on constructivist and adult learning theories. Constructivism Constructivism holds that well designed instruction moves the teacher from a leader to a facilitator of personal meaning making ( Jonas sen, Davidson, Collins, Campbell & Haag, 1995 ) This requires the learner to be an active participant in his/her own learning by interacting with the information, the environment, the teacher, and each other (Liu & Matthews, 2005) Instead of relying on didactic instruction where students they already know about a topic, search for new information, and collaborate with others to solve realistic problems and derive new p. 38) (Tam, 2000, p. 52) is well situated for the hands on scenario based training simulation can provide Within the context of simulation the instructor creates a receptive atmosphere, providing constructive feedback during and/or after the scenario, and uses methods such as video feedback and debriefing sessions to assist with the learning process (Ziv et al., 2003) Simulations are not meant to take the place of qualified educators, but educators should utilize the technology and the methodology to support their teaching (Lampotang, 2008) wh en and where appropriate. Rodgers (2007) notes that the constructivist idea of situated learning is especially relevant to the use of simulation in allied healthcare education: First, in simulation based education, the knowledge or skills are presented in context as opposed to being presented in an environment that may not have a real world implication Second, simulation based education

PAGE 76

76 emphasizes the function of debriefing after a simulation This provides the opportunity to review the situation and exa mine what other contexts the knowledge and skills may be applied Lastly, through the reflective process of debriefing, simulation based education instills a critical thought process in the learner that better prepares the learner to transition the knowle dge and skills into new situations ( p. 74) In this context well designed instruction should move the teacher from a class leader to a facilitator of personal meaning making The learner actively participant s in his/her own learning through simulation by interacting with the simulator, the teacher, and each other (Jonassen et al., 1995) The simulation is one tool that can be used to help students build applicable skills from theoretical concepts support processes of actively d (Holzinger et al., 2009, p. 299); ideas central to constructivist theories Learning is not an abstraction, but is tied directly into real world situations where the learner applies his/her learning to solve a variety of problems p. 52) It is the teach instruction, moving the learner through a journey from novice to expert, a role supported by the proper integration and utilization of simulation This social view of learning involves the whole person, treating learning as a proces s of constructing practice, meaning, and identity in relation to a community of practice (Barab, MaKinster & Scheckler, 2004) The ability of simulation to be used to build and support teamwork both as a learning modality and in practice supports the idea of social learning This social constructivism (Vygosky, 1978) is not completely egalitarian; it is still the

PAGE 77

77 novice to expert This constructivist model can be ap plied to a variety of simulated scenarios, which include facilitated real time feedback and after action debriefing sessions the opportunity to push learners past thei r current knowledge level and see new areas p. v) Constructivism also supports the idea of a safe learning environment, an environment that allows students to take risks without the fear of failure or ridicule, and in the case of healthcare, without the fear of harming the patient ( Bradley & Postlethwaite 2003 b ) Safe is a central part of simulation education and one reason why the post scenario debriefing sessions are such an im portant aspect. Adult Learners The theory, or practice, of adult learning is a concept built upon constructivist ideas of facilitated learning and emphasiz ing the self directed nature of adult students However, there is no specific age at which someone b ecomes an adult learner; it is more of a social and psychological state of being (Blondy, 2007) Knowles asserts that an individual is psychologically an adult when they reach a need for self direction ( Cavanagh, 1990) whereas Lindman characterized adulth ood as a growing self awareness (Brookfield, 1984) However, not all individuals become self directed or self aware in respect to their educational needs In their study of university students, Delahaye, Limerick and Hearn ( 1994 ) discovered that adult l earners tend to utilize both pedagogical and andragogical principles dependent on context and learning goal Within the context of this study, allied healthcare students can range from young adults

PAGE 78

78 directly out of high school to more mature adults looking for a career change This wide demographic requires a teaching methodology that can support a diverse range of students In this context it is important that instructors are not only subject matter experts (SMEs) with knowledge of the educational proces s, but that they also recognize the needs and expectations of their adult learners ( Cavanagh, 1990). The practice of andragogy, a learning strategy focused on adult learners, is rooted in the idea that learning is contextual and the teacher should facilita te a self directed, project centered learning process In the United States the term andragogy is closely associated with the work of Malcolm Knowles ; while the term dates back as far as the German educator Alexander Kapp in 1833, the concept remained ill defined as researchers attempted to address the theoretical differences between accepted pedagogical practice and the needs of adult learners (Knowles, 1988) Knowles work leans heavily on John Dewey (1916) and Eduard Lindeman (1926) who remarked on the social nature and purpose of education ( Brookfield, 1984; Rodgers 2007) In 1970 Knowles wrote The Modern Practice of Adult Education: Andragogy Versus Pedagogy where he explained andragogy as the art and science of helping adults learn In it he makes the argument that a dult education is a separate area of study and practice from primary and secondary education In this view of adult education, educators move from educating people (pedagogy) to helping them learn (Smith, 2002) Knowles (1988) later ch anged the subtitle to From Pedagogy to Andragogy to emphasize education as a continuing spectrum, not a dichotomy (Davenport & Davenport, 1985), as he noted that adults have a wide variety of needs, and sometimes those needs are best addressed through peda gogical approaches.

PAGE 79

79 According to Knowles (1984) the learning environment should be collaborative, welcoming, and one of mutual respect and trust Knowles emphasizes the self directed nature of more mature students where learning can be directly applied to real world situations Andragogy places the teacher in the role of a facilitator who utilized a self directed, project centered learning process The learning environment should be collaborative and welcoming where it is safe to fail, ask questions, and experiment analyze their own actions (or failure to act), reflect on their own skill sets, and critique p. 5) This is one way simulat ions are thought to be superior to clinical rotations; clinical settings are not designed for teaching and learning, but to deliver patient care and ensure patient safety Supervisors are obligated to intervene before mistakes occur, so students typically do not have the opportunity to learn from their mistakes or have first hand experience of negative outcomes (Reznek, 2004) This limitation of the learning experience is necessary in clinical settings, but learning through mistakes is a valuable teaching method supported by simulation. Andragogy strives for a differential world unique and relevant (Davenport & Davenport, 1985) It holds that people become more self directed as they mature; their motivation to learn is immediate and contextual In healthcare the concept of being self directed is accompanied by the concept of self assessment (Baxter & Norman, 2011; Sadosty et al., 201 1 ) but as previously noted, many students and practitioners are inaccurate when it comes to self assessment It is important that educators allow self directed learning to occur, but under conditions which can be accurately and honestly assessed

PAGE 80

80 Adults often expect to be active participants in their learning (Fanning & Gaba, 2007) a nd in the development of their learning environment as they typically have real world experiences which they use as a resource for their learning These experiences can influence their learning, and can include a preference to work in collaborative groups Adult learners not only rely on their experiences to build new knowledge, but expect to integrate their learning with their daily lives on a concrete level through immediate application Lindeman 19 26, p. 37) which means that not only do adult students learn at their own pace, but that each student has his/her own experiences that they integrate into their learning process Healthcare simulations consistent with constructivism and adult learning t heory, allow students to actively participate in creating their own knowledge for immediate application (Bonnel & Smith, 2010) Both the constructivist and adult learning perspectives uphold that learning is social and contextual where each learner brings his/her own experience and prior knowledge to the process, but more research needs to be done in the areas of best practices and roles required by students to achieve optimal learning outcomes when using simulation (Jeffries, 2005) Student needs should be balanced with validated teaching methodologies and outcome assessments Limits of Theoretical Framework The constructivist approach to learning is not without its limitations and it has been criticized as being inefficient and costly (Dick, 1992; Moor e & Kearsley, 2005) It has also been argued that constructivism i gnores that passive perception, memorization, and didactic lectures have demonstrated positive teaching effectiveness (Liu & Mathews, 2005) and should not be simply abandoned In addition, Hardr and

PAGE 81

81 Miller (2006) assert that motivation is not an assumable characteristic of adult learners; it is an individual and mutable trait The amount, quality, and variety of interactions can also vary greatly between adult learners, as personal abilit ies and incentives impact the quality of interactions Woods and Baker (2004) warn that high levels of learner learner interaction can lead toward intellectually shallow dialogue and groupthink, indicating that a teacher or qualified peer facilitator is f undamental to ensuring that students engage in a deeper level of interaction Questioning and reflective opposition should be fundamental; therefore, it is the role of the teacher/facilitator to ask for differing opinions and require a justification for i ndividual conclusions. While adult learning theory can be useful there is no unifying theory which describes the motivations and needs of every adult learner Brookfield (1992) notes that for some theorists, the very idea of trying to develop a formal the p. 92) Thus simulations are not seen as being a substitute for didactic or clinical methods of training, but another tool to support and facilitate student skill acquisition, comprehension, retention, and confidence. Delphi Method T he term Delphi conjures up images of the Ancient Greek oracle renowned for her ability to foretell the future through the less then scientific method of inhaling sulfur fumes Today the term Delphi is used to des solicitation and collation of judgments on a particular topic through a set of carefully designed sequential questionnaires interspersed with summarized information and (Delbecq, Van de Ven, & Gustafson, 1975) Predicting the future is difficult and inexact, and sometimes the best way to forecast an uncertain future is to use the experience of experts (Hall, 2009) By

PAGE 82

82 calling on their experience and knowledge a group o f experts can work collaboratively to reflectively develop input and ideas at a level of accuracy and trustworthiness that could probably not be reached by an individual, no matter how learned or experienced. Hanafin (2004) notes that as a research methodo logy the Delphi has ta ken a number of different paths and has been variously presented as a survey ( Cohen, Harle, Woll, Despa & Munsell, 200 4; Parker & Taylor, 198 0 ), procedure ( Meijer, Ihnenfeldt, Vermeulen, De Haan, & Van Limbeek 2003; Rogers and Lopez, 2002), method (Crisp Pelletier, Duffield, Adams, & Nagy 1997 ; Linstone & Turoff, 1975; Weaver, 1971) and a technique (Broomfield & Humphries, 2001; Sharkey & Sharples, 2001; Snyder Halpern, Thompson, & Schaffer 2002) In this research the definition of Delphi used is the one put forth by Lindstone and communication process so that the process is effective in allowing a group of ( p. 3 ) The Delph i can be a qualitative or a quantitative study tool, but often it is conducted as a mixed method analysis where the issues are identified with qualitative questions and then the results are quantified into actionable information (Hall, 2009) Analysis of the qualitative data requires a certain amount of inductive reasoning on the part of the researcher, as he/she interprets and derives meaning from the data It is both an intellectual and conceptualizing process of transformation (Thorne, 2000) The Delp hi method is an iterative process which requires a collaborative approach where all members work towards finding the best answers, even if they do not always agree on those answers. The Delphi method was initially developed in the 1950s by the RAND Corpora tion as a method for forecasting future military events such as nuclear

PAGE 83

83 armament (Clayton, 1997; Dalkey & Helmer, 1963; Hall, 2009; Van Zolingen & Klassen, 2002), but over the years the Delphi has changed in application and scope as technology has helped s treamline the data gathering process and more disciplines have adopted the method There have been numerous modifications to the data collection process and the types of issues researched Some of these will be explored in the next section Types of Delp hi Classical Delphi method The classic al Delphi is defined by four elements: anonymity, iteration, controlled feedback, and statistical group response ( Dalkey, 1969a; Gracht, 2012; Landeta, 2005 ) Some researchers add a fifth element, stability in respons es among those with expertise on a specific issue These researchers argue that the strength of stability in responses between rounds is necessary to demonstrate consensus ( Hanafin, 2004 ; Van Zolingen & Klaassen, 2003 ) In the classical Delphi the first round consists of open questions used to generate ideas around a specific problem or issue (Keeney et al., 2011) The panel participates by giving anonymous qualitative responses Anonymous and iterative group responses have been found to be slightly mo re accurate than face to face groups (Dalkey, 1969b), and it is believed that the anonymity of the process helps guard from cooperation and coordination of the panelists (Rauch, 1979) thereby giving more reliable and original responses than would be gather ed in a face to face meeting Controlled feedback is handled through the communication of results during each subsequent round that the opinion of every member of the group is represented in the final re (Dalkey, 1969b, p. 16) During rounds two and three the quantitative analysis of the

PAGE 84

84 group responses (usually given as the mean or median) is reported along with the This gives panelists whose responses lay outside the accepted mean an opportunity to give a reason for their score Dalkey (1969b) noted that it is important that panelists receive not just the statistical feedback, but that the feedback includes the reasons supplied by any individual supporting their diverse answer responses, but they may increase reflection Delphi studies are often done with slight modification in order to accommodate the needs of the situation (Murray & Hammons, 1995) These modifications include the electronic Delphi where email and/or electronic surveys are administered through sites such as SurveyMonkey (Melnick et al 2010) Technology has allowed for some variations, which do not inherently impact the des ign of the classical Delphi, but have impacted the method of communication and delivery of the questionnaires These modified Delphi studies have become very popular and some are used for specific purposes. Policy Delphi The policy Delphi is not specifica lly concerned with building consensus or making decisions but aims to use expert opinions to generate policy alternatives In the classic Delphi a homogeneous group is often used, but the policy Delphi requires a heterogeneous response group in order to generate as many divergent ideas about the issues as possible ( Brill, Bishop, & Walker 2006 ; Van Zolingen & Klassen, 2003) Like the classic Delphi it also relies on iteration and controlled feedback; however unlike the classic Delphi, it may actually s eek to polarize group response and promote structured conflict in order to gather the greatest number of alternatives (Hanafin, 2004) Personal

PAGE 85

85 perspectives are crucial (Van Zolingen & Klaassen, 2003) as it is important to include controversial and minori ty opinions in order to stimulate exploration of all sides of an issue (Gracht, 2012) This structured approach often seeks conflict as the rounds are meant to gather information from a diverse group in order to define or clarify an issue. In this type of Delphi t he participants may be anonymous during early rounds, but then come together in a group setting to consider and discuss the alternative ideas (Van Zolingen & Klassen 2002) Other times anonymity throughout the process is important as it makes i t possible for members to safely propose extreme or unpopular views (Rauch, 1979) This Delphi method allows policy input to be gathered from individuals need and desired outcome. Decision Delphi The decision Delphi has been successfully applied to the social science arena and has become popular for researching issues concerning healthcare (Walker & Selfe, 1996) This process is similar to a traditional Delphi except the f ocus is not on building consensus but to prepare, assist, and make decisions (Rauch, 1979) The target panelists are principal decision makers as the purpose is to create a realistic roadmap in order to not simply predict the future, but to create the fut ure Through the iterative rounds feedback is used to brainstorm ideas and encourage panelists to delve deeper into their own statements by requiring supporting comments Rauch (1979) notes that: For the practical application of such a decision Delphi it is not only important to include a large absolute number of participants (as in the case of the classical Delphi) or to touch all relevant areas (as in the case of the policy Delphi) but to include in the panel a high percentage of all the actual decision makers in the field under consideration ( p. 164)

PAGE 86

86 The amount of influence the decision Delphi has is partially dependent on the field being discussed and the amount of interest decision makers have in the outcome As the decision Delphi aims to influenc e and direct future decision those with the power and authority to make changes are not just the primary participants, but also the primary audience (Clayton, 1997) anonymous approach, where the members of the panel are known to each other but the responses are anonymous (Van Zolingen & Klassen 2002) This lends the study the authority needed to promote change while helping to assure open dialogue between the panelists. Modified Delphi method Modifications of the classical Delphi can take many forms These include using interviews or focus groups for the first round ( Cusick, McIntosh, & Santiago, 2004 ; Keeney et al 2011) or conducting interviews after the rounds to clarify issues (Zunker, 2009) The classi cal Delphi uses three rounds, but some modified Delphi use only two rounds (Duffield, 1993) while others have used four in an effort to reach consensus on all items (Okoli & Pawlowski, 2004) While many Delphi studies are mixed method, others are strictly qualitative or quantitative in nature ( Donohoe, Stellefson, & Tennant, 2012 ) Other modifications occur in c ases where the framework is non existent or unfamiliar ; therefore, researchers chose to formulate the initial Delphi items themselves from an exte nsive literature review ( Brill et al. 2003; Custer et al 1999; Kreber, 2002; Steurer, 2011; Wicklein, 1993) Part of the strength of the Delphi is its flexibility as many of the aspects of the Delphi are capable of being modified to meet the specific n eeds of the study There is no correct version of a Delphi study, only a best version for a particular Delphi study.

PAGE 87

87 The traditional mail method is often abandoned in current Delphi studies as researchers choose to communicate solely through email or some times use a combination of email and follow up calls to participants Some recent studies have used online survey methods to collect questionnaire data (Brill et al., 2006; Cabaniss, 2001; Schmidt, 1995) Sometimes referred to as an electronic or an e De lphi, these Delphi studies use Internet 2012 p. 40) One of the primary reasons to utilize the Delphi method is that it is designed to use a panel of experts that are geographically dispersed; traditional mail methods could be slow, with no guarantee that the participant received the invite or the questionnaires By utilizing email communication the time between soliciting p articipants and the rounds can be greatly shortened, which may increase the willingness and ability of panel members to participate. Strengths of the Delphi Method Unlike other methods for consensus building, such as the Nominal Group Technique (NGT), the Delphi method does not require face to face meetings This means that the experts can be geographically dispersed and the ability to participate is not constrained by travel time and budgets In committee meetings, strong personalities or ingrained hiera rchies (Reid, 1988) can cause inhibitions or fear of reprisal leading to unequal participation or acquiescence to the status quo These social and professional pressures are avoided by the anonymity of the Delphi method In most professions, there is a h ierarchy based on traits such as years in the profession, rank, professional degrees, or performance, and within that hierarchy those with higher status can manipulate or influence group decisions (Murphy et al 1998) and in the healthcare

PAGE 88

88 field concern s about status can lead people to devalue the important information of those with perceived lower status (Gordon, et al., 2012) Within the Delphi, all members are equal and are considered experts based on the criteria set out by the researcher While th e Delphi method often seeks consensus, group pressure to conform is avoided in that minority opinions are heard and weighed by the group (Clayton, 1997) Members of the panel can take their time to consider new ideas and weigh their own rankings based on the input of others without feeling embarrassed or pressured to change their minds and conform. The flexibility of the Delphi method allows the researcher to contextually structure the data collection process and the design of the data collection tool Ea ch researcher decides what information is important, such as if the issues should be ranked, the level of consensus needed, how many rounds, and the number or experts needed Donohoe et al. (2012) state that the Delphi method is both flexible and reflexiv e, allowing for a research design which can result in th e collection of a rich and varied data set Flexibility and reflexivity also allows participants to scrupulously think through the problem between rounds so that the validity of the data and the out comes are enhanced p. 40) By allowing both the panel members and the researcher time to think over and analyze the issues and items it is possible to receive deeper reflection than would occur under the constraints of a face to face meeting. Weaknesses of the Delphi Method The purpose of the Delphi method is to elicit expert opinion for a specific purpose 2011, p. 9) The Delphi does not produce hard facts or unquestionable truths, but valid expert

PAGE 89

89 opinions and insight the usefulness of the method While there is extensive literature concerning the Delphi method, there are no set guidelines to follow and over 20 different variations are found in the literature A quick search for Delphi will include terms such as real time, online, modified, argument, technique, study, survey, and method (Keeney et al., 2011) This wide variety has led to confusion and a lack of conformity when it comes to the study design, analytical analysis, and reporting methods. Committee meetings can happen in quick succession while it can be weeks between the Delphi rounds depending on the size of the panel, the number of items, and the amount of qualitative data gathered Many of the weakness es with the Delphi method can be avoided by paying st rong attention to the planning, execution, and analysis of the process (Landeta, 2005; Walker & Selfe, 1996) The purpose and execution must be clearly stated and explained to the expert panel in order for the responses to reach a high level of validity There are many aspects of the Delphi which remain unclear (Hasson & Keeney, 2011; Walker & Selfe, 1996), including proper sample size (Williams & Web, 1994), the degree of expertise required by the panel (Duffield, 1993; Sackman, 1975; Walker, 1994), and the method of analysis (Hanafin, 2004; Okoli & Pawlowski, 2004) In addition, there are no guidelines for acceptable response rates, but several researchers agree that a 70% return rate is acceptable (Keeney et al., 2011; Walker & Selfe, 2003), therefore the researcher should build in the possibility of less than 100% response rate into their decision of how many panel members to include.

PAGE 90

90 There is no consensus as to the required size of the expert panel Okoli and Pawlowski (2004 ) recommended 10 to 18 me mbers, Turoff (1970) recommended less than 15, and Murphy et al. seems likely that below about six participants, reliability will decline quite rapidly, while above about 12, improvements in reliability will be subject to diminishing ( p. 37) Others have recommended groups as large as 50 100 (Rowe & Wright, 1999) or even greater ( Bck Pettersson, Hermansson, Sernert, & Bjrkelund 2008) The target panel size for this study is between 20 and 30 as there is no evidence from th e research that larger groups 30 are difficult to manage Delbecq et al. (1975) argues that 30 is frequently used as an upper bound due to the fact that few new ide as have been demonstrated to be the result of groups larger than 30, whereas three or four people is probably too few, so between ten and twenty five is a reasonable number of panelists The group should be large enough to overcome the issue of attrition as some participants may drop out during the process due to time constraints or loss of interest (Sackman, 1975) There is also no consensus concerning how the data is to be analyzed (Keeney et al., 2011) There is also some debate whether it is consensu s or stability that is the goal of the Delphi ( Dajani, Sincoff, & Talley, 1979 ; Gracht, 2012) Measures of central tendency tend to be analyzed in three ways: mode; median; and mean Measures of dispersion are sometimes analyzed in conjunction with measu res of central tendency These measures are typically analyzed using range; standard deviation; interquartile range; and coefficient of variation Several studies (Rogers & Lopez, 2002; West & Cannon, 1988) utilize both the mean and 1.64 standard deviat ion as their consensus

PAGE 91

91 criterion, whereas Wicklein (2004) utilized mean, standard deviation, and interquartile range to analyze degree of consensus in order to rank the critical issues and problems identified The Delphi process is often used to rank issu W may be used to determine consensus as standard deviation does not apply to ordinal data W in order to rank the importance of the issues (Brancheau & Wetherbe, 1987; Cougar, 1988), but it is no t always necessary to rank responses in the Delphi. Another area of contention is the threshold for consensus, which for most Delphi is related to the stopping criteria In other words, the number of rounds is sometimes tied to a specific level of consens us on all or the majority of items Boyce, Gowland, Russell, and Goldsmith (1993) set consensus at 66% while Keeney et al (2011), and McKenna, Hasson and Smith (2002 ) required a 70% consensus level for their research Several studies set consensus even higher at 80% (Gabard et al 2012; Raine 2006) Determining the level of consensus needed is up to the individual researcher and partially dependent on the goal of the study Whatever level is set by the researcher, much of the literature contends tha t this level should be determined prior to the start of the Delphi (Williams & Webb, 1994) This consensus can take two forms, the extent to which each panelist agrees with the issues (items) under consideration, and the extent to which panelists agree wi th each other (Jones & Hunter, 1995) The type of consensus being analyzed will impact the type of statistical analysis used. The disadvantages of the Delphi method must be considered and addressed, but the disadvantages of the Delphi do not outweigh the fact that in many instances there is no other established method for gathering group opinions and building consensus

PAGE 92

92 across geographically distributed experts In fact, it is in many cases that these same undefined characteristics make the Delphi method a flexible tool for a variety of purposes (Linstone & Turoff, 1975) The use of the Delphi method must be carefully considered by the researcher and decisions concerning the panel size, response rates, methods of analysis, consensus level, and stability sh ould be made before panelists receive the first questionnaire. Prior Studies In the field of education, the Delphi has been used to identify competencies (Rogers & Lopez, 2002; West & Cannon, 1988), qualifications (Moercke & Eika, 2002 ; Van Zolingen & Klas sen, 2002), critical issues and problems ( Schibler, 2008; Wicklein, 1993), the role of technology ( O'Neill, Scott, &, Conboy, 2011 ; Pollard & Pollard, 2004), define terminology (Kreber, 2002), and improve curriculum ( Bell, Daly, & Chang 2008; Blair & Uhl, 1993) According to Landeta (2005) the use of the Delphi method in educational research has increased over the last two decades and based on the number of research articles and dissertations being produced, it is expected that it will continue to be popu lar Weaver (1971) alleged that: Although Delphi was originally intended as a forecasting tool, its more promising educational application seems to be in the following areas: a) a method for studying the process of thinking about the future, (b) a pedago gical tool or teaching tool which forces people to think about the future in a more complex way than they ordinarily would, and (c) a planning tool which may aid in probing priorities held by members and constituencies of an organization ( p. 270) E ducati on al Delphi studies have used classical, policy, decision, and modified versions. dissertation research investigat ed critical issues facing higher education This Delphi study was used to help determine future goal planning for university presidents Using a modified three round Delphi, Stillwell collected

PAGE 93

93 information from executive leaders of the International Association of University Presidents (IAUP) to develop 46 categories of issues focused primarily on society and students Forty two executive officers were approached for the study but only 18 completed the third round Issues reaching consensus included global and local issues such as the need for increased access, funding, technology, and research. Delphi studies are also popula r in the field of medical research and healthcare education (Holey, Feeley, Dixon, & Whittaker, 2007) A Delphi study on the knowledge and skills required of allied health care graduates identified critical thinking and reasoning skills as highly important (Elder & Nick, 1997) Other studies in the field of healthcare have included the setting of research priorities (Annells, DeRoche, Koch, Lewin, & Lucke, 2005; Bck Pettersson et al., 2008; Bell et al., 2008 ; Bond & Bond, 1982; Daniels & Ascough, 1999; Li ndeman, 1975; Misener, Watkins, & Ossege, 1994 ) and education priorities ( Broomfield & Humphris, 2001 ; Cusick et al., 2004 ; Keyes, Wilson, & Becker, 1975 ; Moercke, & Eika, 2002 ) The researcher was unable to find a Delphi study specifically related to the future use of simulations in healthcare education at any level (nursing, surgical, or allied health) Thompson, Repko, and Staggers (2003) used a Delphi study to help develop nursing competencies for peacetime and deployment preparation settings for Air Force Medical Surgical Nurses (46N3) Using a web based questionnaire 109 nurses participated in three rounds to achieve consensus on 83% of the items concerning the importance statements and 67% of the items concerning the practice statements From th is study the researchers were able to determine valid competencies to be considered for further development or research.

PAGE 94

94 Burnett, Kumar and Grimmer ( 2005 ) attempted to address the lack of research concerning evidence based practice in allied health profes sions through a Delphi survey the essential criteria for clinical appraisal and whether a generic critical appraisal tool could be constructed for allied health use ( p. 1 ) Fifteen allied health professionals throughout Austra lia participated either through an electronic questionnaire or by phone interviews Themes and criteria were identified, but there was no consensus concerning the feasibility of a generic critical appraisal tool. The Delphi method is a flexible approach t o research that can be used when it is important to use structured communication with a group of experts This gathering of collective human intelligence (Linstone & Turoff, 1975) can assist researchers in finding answers or solutions which otherwise may be too complex to fully explore The Delphi method has been utilized by numerous studies in order to research issues in both the education and healthcare fields, but it is important to keep in mind the limitations and weaknesses of the method in order to design a study with usable outcomes. Summary It is clear from the literature that there are still many questions when it comes to simulations in allied healthcare education and how best to assess and analyze their current and future use The articles conc erning both simulation and the Delphi method note the pitfalls related to the lack of rigorous design and research methodology By conducting a thorough literature review this study seeks to avoid many of the problems noted in the current research and add to the body of knowledge concerning the technical and instructional feasibility of successfully addressing the issues facing the future use of simulations in allied health care education

PAGE 95

95 CHAPTER 3 METHODS AND PROCEDURES This modified Delphi study explore s the future of simulations in allied health education and training by consolidating expert opinions concerning the feasibility of successfully addressing the critical issues facing the future use of simulator technology and simulation methodology It see ks to find consensus concerning which items are feasible to pursue as solutions Chapter 3 contains information concerning the methods and procedures used by the researcher in the course of this study The methods presented in Chapter 3 were used to answ er the following research questions: 1. In the opinion of a panel of experts, what technical issues will impact the future use of simulators/ simulation in allied healthcare education? 2. In the opinion of a panel of experts, what instructional issues will impact the future use of simulators/ simulation in allied healthcare education? 3. What is the technical feasibility of each issue impacting the future use of simulators/ simulation in allied healthcare education? 4. What is the instructional feasibility of each issue i mpacting the future use of simulators/ simulation in allied healthcare education? This project utilized the modified Delphi method (Steurer, 2011) to work towards consensus concerning a list of items developed from a comprehensive literature review ( Brill e t al. 2003; Steurer, 2011; Wicklein, 1993) This literature review included scholarly articles covering topics such as the current use of healthcare simulations in education, specific examples of the use of simulation in allied healthcare, and the possib le future of simulation in healthcare education Each item was then rated by a panel of experts on its technical and instructional feasibility because for most issues, both are necessary if the item is to be strategically pursued The feasibility rating scale was based on one developed by Ziglio (1996) and a combination of qualitative and

PAGE 96

96 quantitative methods were used to analyze the expert input and the level of consensus on each item The research protocol was submitted and approved by the Institutio n Review Board at the University of Florida (protocol #2013 U 0575) The Modified Delphi Method The Delphi method was developed in the 1950s by the RAND Corporation to research national security issues (Gracht, 2012; Zolingen & Klassen, 2003) It is a sys tematic procedure for soliciting and collating opinions from experts on a particular topic of study through the use of carefully designed rounds of questionnaires and structured feedback t echnological events and to judge when the events might occur through the p. 267), it has more recently been used to help solve complex problems such as institutional planning (Uhl, 1983) The entire process is typically completed anonymously, and in this modified Delphi study done via email Although each expert interacts with the researcher, all expert interactions are done through the questionnaire (Zolingen & Klaasen, 2003) ; thereby obtaining diverse expe rt opinion without the social pressure that can occur within a focus grou p. The questions that a Delphi study investigates are those of high uncertainty and speculation ( Okoli & Pawlows commun ki 2004 p. 19) and require thoughtful exploration through sev eral iterations The flexibility of the Delphi allows the method to be used in a variety of ways to answer many different types of research questions, but this flexibility also makes it confusing to develop a methodology best suited for each situation. Th e first round of this study consisted of a questionnaire (Appendix F) built from carefully selected items concerning the use of simulations in allied healthcare education drawn from an extensive literature review This grounded the project in the literatu re

PAGE 97

97 and gave the panel the opportunity to focus on existing items in the first round The questionnaire was then subject ed to a three round Delphi The a rticles chosen for the literature review focused on several aspects of simulation in education and tra ining There is little research concerning the future of simulation in healthcare education or on the current use of simulation in allied healthcare education; therefore it was necessary to research articles which discussed current simulation issues, the ories, and practices among all levels of healthcare related education Original search terms included allied healthcare education; future of medical simulation; community college healthcare education; simulation research; critiques of simulation in medica l education; and validation of healthcare simulations Subsequent searches were for specific studies referenced in the literature or for studies concerning specific issues noted in the literature Searching for relevant information was completed remotely through the George A Smathers Library by searching through five main databases, Google Scholar, PubMed MedlinePlus Academic Search Premier (EBSCOHost) and Science Direct (Elsevier) Data mining techniques were also employed b y reviewing and utilizing the reference sections of crucial articles Each article was reviewed for relevance and verified in Ulrich's International Periodicals Directory for a cademic /s cholarly classification. During round one members of the expert panel rated the existing items and were also given the ability to add new items The researcher then quantitatively analyzed the ratings and qualitatively assessed, clarified, and was one new item added to the questionnaire During the second round individuals receive d the modified questionnaire with group ratings for existing items as well as new

PAGE 98

98 item (NTF and NIF) In the second and third rounds the questionnaire included individualized feedbac k, which compared individual expert ratings on each of the initial items with the panel scores (West & Cannon, 1988) Those whose ratings fell outside the established range of consensus were asked to reconsider their rating or give written justification f or their scoring of the item outside the range of consensus T hey were also asked to rate the new item added from round one All three rounds were concerned with analyzing the level of consensus between the items (Fleuren et al., 2004). Rationale for Usi ng the Delphi Method The purpose of using the Delphi method is to move towards group consensus through an iterative process of informed decision making (Duffield, 1993) The future utilization of simulation in allied healthcare education is an issue of im portance in the field, but currently lacks a rigorous research foundation upon which to make strategic decisions Introducing a new idea, practice, or object to healthcare is widely considered a complex process ( Fleuren et al., 2004), and there is a consi stent gap between evidence and practice (Grol & Grimshaw 2003) partially because there is often insufficient or contradictory information (Jones & Hunter, 1995) To make things more challenging, the decision making process for academic institutions can b e frustrating and hindered by divergences in opinion, strong allegiances, and varying vested interests (Uhl, 1983) Through the proper utilization of the Delphi method experts can collaboratively work to develop a way forward. The Delphi method used for t his study w as based on the guidelines fr om Linstone and Turoff (1975) T his research fits five of their seven criteria for choosing the Delphi method:

PAGE 99

99 1. The problem does not lend itself to any precise analytical technique 2. Those contributing to the examinati on of the complex problem represent diverse backgrounds with respect to experience or expertise 3. It is not effective to interact face to face due to the number of experts involved 4. The time and cost involved to conduct face to face meetings is prohibitive 5. Th e heterogeneity of the expert panel is needed to assure validity of the results (each individual needs to give honest input without feeling pressured by the group) Linstone and Turoff (1975) note that only one of these criteria needs to be present to make the Delphi an appropriate choice for the research method. This project also utilized the modified Delphi method proposed by Custer, Scarcella, and Steward (1999) in that it began with a set of carefully selected items drawn from an extensive literature rev iew concerning issues affecting the future of simulation in allied healthcare education The items were then subjected to a three round Delphi to ascertain expert agreement concerning the items and the technical and instructional feasibility of addressing these issues The Delphi method was applied in order to facilitate an anonymous exchange of ideas and opinions among a panel of experts concerning the items derived from the literature (Fleuren et al., 2004) By utilizing the Delphi method, experts from a variety of geographic locations with varied backgrounds and professions could help explore an area of significance in the allied healthcare field that has lacked rigorous research. Selection of the Expert Panel According to Adler and Ziglio (1996) the p anel members should meet four requirements: 1 ) knowledge and experience with th e issues under investigation; 2 ) capacity and willingness to participate; 3 ) sufficient time to participate in the Delphi; and, 4 ) effective communica tion skills The fact that experts willing to participate are often

PAGE 100

100 those that are most interested or even affected by the outcomes is a source of subject bias that should be acknowledged (Hasson, Keeney, & McKenna, 2000 ; Powell, 2003 ) For this study the term expert is defined a s someone who has run, designed or developed a simulation lab for allied health care program ( s ) or has a minimum of three experience in any of the following roles within allied healthcare programs : operating simulations ; assessing simulation based c ompetencies ; using simulation as a teaching or assessment tool ; developing new curriculum for the integration of simulation ; or having p ublished peer reviewed research concerning simulations at the allied healt hcare level (Holden & Wedman, 1993 ; Pollard & Pollard, 2 004) While most researchers agree that the reliability of the Delphi lies heavily on the proper selection of experts (Duffield, 1993) Sumison (1998) argues that in reality what is important is to topic and are willing to dedicate the time p. 154) Recruiting the best minds in the field is only effective if they are willing to fully engage in all rounds of the process Therefore, the definition of an expert is contex responsibility to defend their choice of experts The number of experts is also left up to the researcher as the numbers noted in the literature range anywhere from 12 to 1600 (Williams & Webb, 199 4a) The number of experts is contingent on several factors including the topic being researched, the time and resources available to the researcher(s), and the breadth of expertise required (Grundy & Ghazi, 2009; Sumison, 1998) Whether the sample can b e considered representative for statistical purposes is more a question of the quality of the expert panel than the number of experts on the panel (Powell, 2003). Based on the literature,

PAGE 101

101 it was determined that this research would be limited to a panel of no more than 30 experts The expert panel for this research effort consist ed of individuals who were purposively sampled because they are uniquely qualified to know about the problem or issues being researched due to their immediate knowledge or experienc es (Hall, 2009; Martino, 1972) The Delphi relies on a collective wisdom (Surowiecki, 2004), asserting that different voices are essential to the discussion Certain demographic information concerning the background and area of expertise of the participa nts is included in the results section so potential users of the study can themselves determine the validity and relevance of the findings (Hall, 2009) Information concerning school demographics is also included Some Delphi studies use randomized sampl es when various types of expertise are sought (Clayton, 1997), but this was not deemed to be appropriate for this research; however, some snowball sampling did occur as those experts initially contacted by the researcher were given the opportunity to nomin ate other panel members. The panel of experts was selected through the use of four methods: sending email invitations to participate to allied healthcare programs with known simulation centers or labs across the United States; through postings on national list servers and message boards for relevant organizations such as the Society for Simulation in Healthcare (SSH) and the Center for Medical Simulation ; utilizing the professional connections of METC faculty and staff; and soliciting authors published in t he field Many times experts in academic related Delphi studies are chosen based on the literature review ( Okoli & Pawlowski 2004 ), but due to the lack of literature concerning

PAGE 102

102 simulation use in allied healthcare education it was not possible to find eno ugh experts just from the literature review It was vitally important to find a panel that would be motivated to complete the three rounds of the Delphi by inviting members who were interested and engaged in the topic and the research findings All indiv iduals who participated and qualified in the demographic survey were asked to participate as it was expected that there would be some attrition and it was hoped to have over 2 0 panelists complete all three rounds The Delphi panel is a heterogeneous group as it is comprised of members who are experts on a particular topic but come from different social/professional stratifications, e.g., clinicians, teaching faculty, scholars, and administrators (Clayton, 1997) heterogeneous groups can greatly incr ease the complexity and difficulty of collecting data, reaching consensus, conducting analysis, and verifying results ( Skulmoski, 2007 p. 10 ) but it was deemed important to have a heterogeneous group as they all play essential roles in allied healthcare education The initial group consisted of 23 members, a number which gave enough data to analyze without being so large that data gathering and analysis is excessively time consuming and consensus is difficult to reach (Murphy et al., 1998; Okoli & Pawlo wski 2004 ) This is a slightly larger 10 panel members for a heterogeneous population, but it was decided that a larger group would give more diverse input Purposive and snowball sampling was done to gat her the panel of experts Introductory letters were emailed to nearly 40 potential panel members throughout the United States as there was no guarantee as to how many potential panelists would

PAGE 103

103 meet the criteria and be willing to participate (Appendix A) ( Schmidt, 1995) Due to the fact that the spring semester had ended many of these introductory letters received an out of office response Several others replied that they would not be able to participate but one forwarded the information to a colleague and two gave suggestions on other individuals to contact A message requesting help with dissertation research was also posted on the Society for Simulation in Healthcare (SSH) listserv and several simulation and allied healthcare related groups on Linked In Those potential participants showing interest were sent the introductory letter and were asked to complete the demographics survey Several contacted the res earcher for more information Participant fatigue is a challenge in Delphi studies, and it is important that the researcher kept the panelists interested and engaged throughout the process in order to ensure a 70% response rate across all rounds (Sumison, 1998) Therefore all responses were replied to and participants were kept up to date on an y changes in the timeline The initial contact letter to the potential panelists found in Appendix A, gave certain details about the study including the criteria for the expert panel, procedures, an estimate of the time expected to complete each round, an d the purpose of the study (Hasson et al., 2000) The letter of inquiry asking for Delphi participation was sent to allied healthcare programs with simulation labs or simulation centers and it was requested that the individual participating in the Delphi study be someone who has designed, developed, or supported a medical simulation lab for an allied healthcare program(s) or worked for a minimum of three years in any of the following roles: operating healthcare simulations assessing simulation based compe tencies, using

PAGE 104

104 simulation as a teaching or assessment tool, developing new curriculum for the integration of simulation, or having published peer reviewed research concerning simulations for teaching and training at the allied healthcare level (Holden & We dman, 1993, Pollard & Pollard, 2004) Contained in the initial solicitation was a link to a SurveyMonkey site which took the potential participant to a demographics questionnaire (Appendix D) The informed consent information (Appendix C) was contained a t the front of the demographic questionnaire, thereby the participants gave consent to participate by completing the survey This questionnaire was used to determine basic demographic and background information including gender, age education current po sition and use of medical simulation years of experience with simulations amount and type of training with simulations and the reason for their interest in the study (Keeney, Hasson, & McKenna, 2011) While gender and age information was gathered in th e initial qualification survey for descriptive reasons, these were not used to determine qualification for participation Panel members were also asked to include their preferred method of contact either mail or email Any individual not meeting the min imum standards received notice (Appendix B) that they would not be included in the Delphi panel and were thanked for their time. There were 26 individuals who responded to the demographic survey One individual did not complete the survey and did not leav e valid contact information, and therefore was not eligible for the study Two others did not meet the minimum requirements concerning years of experience These individuals were thanked for their time and interest leaving the total number of participan ts for r ound o ne at 2 3 Nearly

PAGE 105

105 45 % of the participants were 40 49 years of age with 24 % being 60 or older, 16 % being 30 39, and 16 % being 50 59 They were split almost evenly by gender with 48 % being male and 52 % being female Over 60 % of the participan ts had graduate degrees and the majority had between six and ten years of experience supporting, using, researching, and/or evaluating simulator technologies and or simulation modalities in allied healthcare education There was a big gap in time spent us ing or working with simulator/simulation with nearly 46 % answering one to ten hours and just over 29 % answering 21 or more hours Most (84 % ) responded that they had received simulation specific training The majority of respondents were from the United S tates with one from Australia and two from the United Kingdom A more complete breakdown of the demographics survey results can be found in Appendix E. Instrument Development for Round One There currently is a lack of a clear strategy when it comes to pla nning for the future of simulation in allied healthcare education It is also apparent from the literature that no consensus exists regarding the most important issues facing the use of simulation in allied healthcare education and training The initial questionnaire was developed by the researcher after an extensive literature review in order to ground the project in the research and give it greater focus (Brill et al., 2006 ; Tough, 2009) Over 85 different articles were reviewed from across medical fie lds including allied healthcare, nursing, surgical, and anesthesiology Most articles were concerned with the use of simulation in education in general but a few dealt specifically with continuing education and the retention of skills Fewer than a doze n dealt solely or in part with the use of simulation in allied healthcare education There were 8 4 items developed for the

PAGE 106

106 initial questionnaire with each to be rated on instructional and technical feasibility, making 168 separate item s to score Validati on of Questionnaire The act of developing a clear and unambiguous questionnaire is difficult and time consuming, requiring the author to think of multiple ways in which the items can be (mis)interpreted and answered Designing a clear and unambiguous ques tionnaire can assist in the collection of data by increas ing survey response and help ing ensure the data collected is the data wanted (Drennan, 2003) One method utilized by researc hers to validate a questionnaire is cognitive interviews There are two t ypes of cognitive interviews used by researchers to provide feedback on research instruments in order to identify and correct issues before the instrument is delivered to the target audience These two methods, think aloud and verbal probing, assist the r esearcher in using cognitive theory to understand how respondents may perceive and interpret questions (Drennan, 2003) Beatty and Willis (2007) define cognitive interviewing as the bal information about the survey responses, which is used to evaluate the quality of the response or to help determine whether the question is generating the information that its author p. 287) For this current research it was determined the th ink aloud protocol would provide the necessary feedback to improve the researcher developed questionnaire and was chosen partially due to the length of the survey (Dietrich & Ehrlenspiel, 2010) T hese t hink aloud sessions are a type of cognitive interview which allow s the researcher to determine if the participant answer s the question as intended

PAGE 107

107 Think aloud sessions are standardized, where each participant completes an identical task of verbalizing out loud while answering the questionnaire (Beatty & Willis, 2007; Priede & Farrall, 2010) The researcher is mainly a passive observer In contrast, v erbal probing relies on the interviewer taking on a more active role guiding the participant through the questionnaire while askin g specific questions concerning the cognitive process (Willis, Royston, & Bercini, 1991) Some encouragement or requests for clarification by the researcher are acceptable during a think aloud, but the researcher does not take as active a role as in the v erbal probing cognitive interviewing technique During cognitive interviewing, it is possible that the very act of knowing they are being listened to and analyzed may change the amount of time and attention a participant gives to answering a question (Will is, 2005) which may change the way a participant responds This may lead to some impact on the usefulness of the think aloud method and it is up to the interviewer to decide how to interpret the responses The responses may not always be useful as they m ay show that problems exist, without providing any insight as to what the problems are or how they may be fixed (Beatty & Willis, 2007; Priede & Farrall, 2010) The difficulty for the interviewer is knowing how and when to seek more information Accordin g to Drennan ( 2003) issues that are discovered often deal with lexical misunderstandings (e.g., what is often?), scope (e.g., is there any overlap between allied healthcare education and nursing education), temporal problems (e.g., fiscal vs calendar year ), and logic problems ( e.g., too many conjunctions/issues in a single item) During the think aloud the respondents are encouraged to talk through their thought processes as they read and answer the items

PAGE 108

108 After the interview, the researcher uses the tr understanding of the question and the information that they drew upon when answering The researcher can then redesign the questionnaire to remove as many potential 010, p. 272) The questionnaire is p. 54) in order to address issues that were expressed during the think aloud session In order to validate and help ensure the questionnaire used in this study was clear and concise, a think aloud session was conducted with three of coworkers with differing backgrounds in education and medical simulation during off duty hours Some participants found the process difficult as they were asked to verbalize their thoughts as they fill ed out the questionnaire, and there were times it was necessary to remind participants to speak out loud Sessions were conducted with the participants an d notes were recorded By listening to the participant s as they work ed through the questionnaire the researcher pinpointed some of these areas of confusion and rework ed the items for greater clarity. Questionnaire Distribution This Delphi study used email to communicate with panel members and an online survey application ( SurveyMonkey ) to collect data for the demographic survey Due to the complexity of individualiz ing surveys during rounds two and three it was decided not to use SurveyMonkey to distrib ute the questionnaires, but to use e mail to distribute the instructions and Adobe form questionnaire s to the chosen panel for all three rounds During rounds two and three each panelist received not just the group feedback and quantified group responses, but their own previous response to each item thus making

PAGE 109

109 each survey instrument unique A n Adobe form was developed and attached to an individualized email whose body included the instructions for each round thereby allowing easy export into Microsoft Ex cel for analysis (Appendixes F H ) Because the research from which to draw upon was sparse, participants were requested to add other items during round one as they deemed appropriate No cut off was given for number of items (Schmidt, 1997) Only two ne w items were requested and after being coded it was determined only one matched the intent of the study and was included for round two During round two participants were given the opportunity to verif y or challenge the new item (Schmidt, 1997) The add itional item only went through two rounds of the Delphi study but this was sufficient to reach a high level of consensus (Uhl, 1983). Data Collection A three round Delphi was used to collect qualitative and quantitative data from a panel of predetermined e xperts An explanation of the purpose of the study as well as instructions and a timeline accompanied each round The final results were disseminated to the panel members who completed all three rounds In order to protect anonymity, all participants we re given a random participant identification number This number was used to track individual responses as well as any justifications for outlier responses. The Delphi is an iterative process, requiring several phases to be completed before the data colle ction takes place Sumison (1998) lists seven distinct steps found in most Delphi studies: 1. Identification of the primary aim 2. Contacting possible panel participants 3. Round one 4. Tabulating and quantifying round o ne results 5. Round two (includes tabulation)

PAGE 110

110 6. Ro u nd t hree (includes tabulation) 7. Publication T able 3 1 uses these steps to form the initial data collection timeline for this research. Table 3 1 Data collection and analysis timeline Task Duration Identification of the primary aim create initial questionnaire items and Internet based survey 7 days Contact possible panel participants and send out demographic survey Review criteria and respond to potential panelists 7 days Round one panelists respond to first round 10 days Tabulating and quantifying round o ne results 1 7 days Round two (includes tabulation) 14 days Round t hree (includes tabulation) 14 days Final analysis and recording of findings 21 days Round 1 Round one consisted of emailing the panelists a fillable Adobe form containing the list of initial items developed by the researcher based on the literature review and presented by theme (e.g., research, curriculum, f aculty and staff issues ) The experts then reviewed the list and rated the items on a 5 point L ikert scale (1= not at all feasible, 2= probably un feasible, 3= feasible, 4= very feasible, 5= extremely feasible) for both their instructional and technical feasibility It was decided to include a not applicable ( NA ) response because the items were take n from broad a range of literature and not all items may be relevant to all allied healthcare fields. The purpose was to identify issues, but also to find consensus on those items that are feasible to pursue. Each theme included a space where panelists cou ld add items (Rogers & Lopez, 2002) or modify existing ones (West & Cannon, 1988) The r esearcher reviewed the input and coded them as new item, edit to existing item, or general comments Only one panelist suggested new items during round one, resulting in one new issue being

PAGE 111

111 added for round two Several panelists had comments concerning the existing items which resulted in those items being edited for clarity T hese were noted as edited for round two in the item number column and d isplayed strikeouts for deleted words and bold for additional words These documents, which were unique for each panelist as they included not the only group data, but individual responses to round one were then made into fillable Adobe form s for round two The day before t he first questionnaire was due back to the researcher only 11 responses had been received. A reminder letter was sent in hopes of increasing the response rate. Several responded that they needed more time and the timeline was extended. This resulted in 19 panelists (83 % ) responding to round one. Panelist ratings were analyzed for means, standard deviation, and range of consensus using IQR by importing the form information into Microsoft Excel Consensus had been set at an IQR score of less than 1.2 (Ale xander, 2008), where a high IQR score indicated a wide variance of opinion Those items which were within the range of consensus were marked as such with an asterisk ( ) on the round two questionnaire This process was repeated for the next two rounds as the panel worked towards building a consensus (Custer et al 1999) There were a number of items where some respondents answered NA The percentage of NA responses as well as any given reasons for giving the NA response was also on the round two questi onnaire. Round 2 The panelists who responded to round one were sent the ir personalized and updated questionnaire for round two There were 17 returned questionnaires for round two (74 % response rate). During this round the experts were provided with bot h their individual ranking and the analysis of the group rankings

PAGE 112

112 (Appendix G) During rounds two and three the panelists had three options: 1) maintain their previous position regardless if it fell within the acceptable range ; 2) attempt to sway others by voting strongly against the group rating; or 3) change their position based on group ratings (Walker & Selfe, 1996) This allowed the panelists an opportunity to think deeper about their response based on group responses and feedback. All items were in cluded for the round regardless if consensus had been reached, helping to control bias in the results (Grundy & Ghazi, 2009) While some Delphi studies remove items which reach consensus, it was decided to keep all items as opinions are not explored and therefore the resultant Greatorex & Dexter, 2000 p 1022 ) The Delphi is designed to elicit and explore minority opinions; therefore, all items were considered for all three rounds Panelists were asked to rate the new item developed from the expert feedback in round one as well as reconsider their ranking of the round one items based on the analyzed results All comments from round one were included but some were edi ted for clarity Panelists were asked to give written justification for any rankings outside the range of consensus Rankings for round two were done on the same 5 point Likert scale (1= not at all feasible to 5= extremely feasible) and included the NA o ption All items were analyzed for means, standard deviation, and IQR R ound two was not expected to reach consensus on all items (Hsu & Sandford, 2007), but those within the range were marked as such with an asterisk on the third round questionnaire O nly the responses from round two were included in the analysis.

PAGE 113

113 Round 3 Round three was sent to the 17 panel members who responded to round two and 16 returned the questionnaire (7 0 % response rate) It was constructed of all items developed from the exte responses from round two, the analyzed group responses for each item, and summaries of justifications for rankings outside the range of consensus as well as any NA responses Panel members were again aske d to re rate their outlier responses in light of the overall group response (Keeney, Hasson, & McKenna, 2011) or justify their continued rating outside the norm Round three r esponses were then analyzed for means, standard deviation, and range of consensu s It was not necessary that panel members reached full consensus on all items (Keeney et al., 2011; Hsu & Sandford, 2007; Steurer, 2011) as these demonstrate areas in need of further review and research Data Analysis During round one qualitative data wa s gathered in the form of new items and feedback on existing items New items had to be analyzed and considered for inclusion in the next round Memos were added to each new item in order to find themes By asking if the coded themes were relevant to th e research study the researcher came up with one new item which was included in round two There were 14 comments concerning the existing items These were coded to see if there was confusion on an existing item which required editing t he item for clari fication or if the feedback on the item dealt with why the panelists chose to score the item in a particular way Ten comments were included and five items were edited for round two It should be noted, the lack of independent raters during this step may have allowed for researcher bias to influence the second questionnaire (Jenkins & Smith, 1994)

PAGE 114

114 Descriptive statistics were used to analyze the data collected through the Likert scale The initial questionnaire was presented as a five point Likert scale (1= not at all feasible to 5= extremely feasible) broken down by topic area but given in no particular order During the rounds items were analyzed for means (as a measure of central tendency), standard deviation (a measure of spread) ( Greatorex & Dexte r 2000 ; Wicklein, 1993), and IQR (range of consensus) (Wicklein, 1993) Consensus was realized when an IQR score of less than 1.2 was achieved (Alexander, 2008) All items were analyzed for each round with the group results and individual responses pres ented to each panelist for rounds two and three. Reporting the Data As a method, the Delphi has questionable reliability and validity that can be partially addressed through rigorous design and honest reporting of the data Whittemore, Chase and Mandle ( 2001) note that all methods have l imitations, biases, and threats to validity ; therefore t r ying to create a perfect investigation is futile The important thing is to determine what validity ideals are important to a particular study, to employ the best t echnique, and to honestly and critically present the entire process in detail The reader can then decide, based on all relevant data, including the qualitative code, if the research is valid for their particular need Hence, the validity and applicabili ty of the results are partially contextual, for the reader must answer for themselves if the results are authentic and credible for the context of the research (Whittemore et al. 2001) Audit trails help to substantiate the rigor and trustworthiness of t he research ( Glass, 1997; Koch, 2006 ; Wolf, 2003 ) therefore all qualitative data is reported.

PAGE 115

115 There is no standard method for reporting the findings of a Delphi study, but it is recommended that the findings of each round are reported (Hasson et al., 2000 ; Thorne, 2000) It is important to be honest about researcher bias and note if any decisions are based out of the literature or personal experience ( Koch, 2006 ) In order to tell good numbers from bad numbers, we need to understand not only what was le arned, but also how the researchers collected their data We need to see the instr Skulmoski et al., 2005, p. 12) All relevant information concerning this study is available in the Appendices for this study to be replicated with the same approach but with a different pool of experts (Dalkey, 1969) Those interested in the results of this Delphi study might have greater confidence in the results should a second group of experts produce similar results. The purpose of this study was to explore and identify technical and instructional issues imp acting the future of si m ulation in allied healthcare education and training The modified Delphi method was utilized in order to facilitate the open distr ibution of ideas and input from a heterogeneous group that was geographically dispersed Qualitative and quantitative methods were used to analyze expert input and level of consensus Chapter 4 include s the results of this modified Delphi study following a logical progression of each Delphi round

PAGE 116

116 CHAPTER 4 RESULTS Chapter 4 reports the results of the data obtained through the three rounds of this modified Delphi study This section describes the results of the study as they relate to the questions: 1. I n the opinion of a panel of experts, what technical issues will impact the future use of simulators/ simulation in allied healthcare education? 2. In the opinion of a panel of experts, what instructional issues will impact the future use of simulators/ simulati on in allied healthcare education? 3. What is the technical feasibility of each issue impacting the future use of simulators/ simulation in allied healthcare education? 4. What is the instructional feasibility of each issue impacting the future use of simulators/ simulation in allied healthcare education? It was the intent of this research to provide valid data to assist in the develop ment of actionable guidelines and recommendations that can help decision makers in their strategic planning initiatives Both quali tative and quantitative data was collected and analyzed during each round Items were rated on a scale of one to five for their instructional feasibility and technical feasibility: 1=not at all feasible, 2=probably unfeasible, 3=feasible, 4=very feasible, 5=extremely feasible. For this study, feasibility is the likelihood of an item being accomplished or being dealt with successfull y Quantitative analysis was conducted on the items based on means (as a measure of central tendency), standard deviation (a measure of spread), and the interquartile range (IQR) to demonstrate the range of consensus within the panel (Wicklein, 1993) For the purpose of statistical analysis, item means and standard deviations were figured to the 100th decimal place, but when d iscussing where an item fit in the feasibility scale the mean was rounded to the nearest whole number For instance, an item with a mean of

PAGE 117

117 3.26 was rounded to 3 and is reported as feasible, while a 3.68 was rounded to a 4 and reported as being ranked as very feasible When analyzing the data it was important to decide what the numbers represented for the future practice of simulation in allied healthcare education and training When reporting statistical tests the reader must be informed of how to inter pret the results and how to digest the fi ndings in relation to the emphasis being placed upon them Hasson et al., 2000 p. 1013). In other words, what does it actually mean when an item has a mean rank ing of 3=feasible versus an item having a ranking o f 4= very feasible For example: T echnical feasibility=1 is interpreted as cannot be implemented as the cost is prohibitive or the technology does not exist Instructional feasibility=2 is interpreted as cannot be implemented under current conditions or the re is significant opposition to its implementation Technical feasibility =3 is interpreted as possible to implement but some challenges must first be addressed Instructional feasibility =4 is interpreted as there are some concerns about its implementation wh ich can be easily addressed or has minor obstacles Technical feasibility=5 is interpreted as having no barriers to im plementation development, resources, and training are available and adequate Those items with a feasibility score of a 1 or 2 are conside red unfeasible at this time and should not be pursued until the technology, research, and/or methodology is more complete. The full rating scale can be seen in Appendix F There were 23 Individuals who completed the demographic survey and met the basic re quirements for the expert panel Each panelist was given a random number in order to ensure anonymity This number was used on all correspondence including the

PAGE 118

118 questionnaires. Once the expert panel had been chosen the panel members were sent the first round questionnaire found in Appendix E and were asked to : 1. Review all the issues listed in the questionnaire 2. Make any comments or suggestions, give feedback, or request clarification on any item in the spaces provided If commenting on a current item t hey were asked to note the item number before the comment 3. Add new items as deem ed appropriate 4. Rate the technical (TF) and instructional feasibility (IF) of each i tem based on the included feasibility scale The first round consisted of 8 4 statements each to be ranked on their technical and instructional feasibility based on a scale of 1 5 : 1= n ot at all feasible, 2= p robably unfeasible, 3= f easible, 4= v ery f easible, 5= e xtremely f easible They also had the option of choosing NA The items were given in no particular order but were grouped by theme (e.g., research, curriculum, faculty and staff issues ) There was some loss of participants through the rounds thereby introducing some level of inconsistency; however 16 of the original 23 participants (70 % ) c ompleted all three rounds Delphi Survey Round 1 This section presents the results from the first round of the Delphi study Each item on the survey had two L ikert scales, one for technical feasibility and one for instructional feasibility An example of some issues at each level was sent with the instruction s A comment box was placed at the end of each section for feedback on existing items or for panelists to add new items As the items ha d been predetermined from an extensive literature review and n ot by the panelists themselves, it was decided that panelists would have the option to choose NA for the items These responses would then be analyzed against the panelists background and area of expertise to see

PAGE 119

119 if there were any patterns based on speci fic allied healthcare field or level of experience There were only two comments made in the first round concerning new items one of which resulted in the inclusion of a new item for round two The level of consensus had been predetermined as having an I QR of less than 1.2 (Alexander, 2008) Listed in Table 4 1 are those items that reached consensus in the first round It was not expected that the majority of items would reach consensus in round one. As instructional and technical feasibility were rate d independently, some items reached consensus on one but not both standards The full results of each round can be found in Appendix I. Table 4 1. Questionnaire items which reached consensus in round one Item Technical Feasibility Mean IQR Inst ructional Feasibility Mean IQR 1 Need to set up a clear research agenda for simulations in allied healthcare education 3.56 1 2 There is a need to research the impact of simulated learning as it impacts real world skills application 3.74 1 5 Evaluat ion of simulation needs to be done contextually and over time through long range studies 3.53 1 3.42 1 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual clinical p erformance 3.22 1 3.50 1 7 Need valid research concerning the impact of simulation on student learning outcomes for allied healthcare skills 3.56 1 3.73 1 8 Need for defensible research to identify methods and technologies that work best fo r specific learning outcomes 3.33 .75 3.44 1 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques 2.76 1

PAGE 120

120 Table 4 1. Continued Item Technical Feas ibility Mean IQR Instructional Feasibility Mean IQR 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time a vailable for session, and experience level of the participants 3.63 1 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) 3.39 1 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respiratory therapists) 2.58 1 16 Need validated method of measurem ent to verify transferability of simulation performance to the clinical setting 3.33 1 3.28 1 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too complex 3.50 1 3.83 1 18 Need to develop guide lines which ensure faculty are not overwhelmed with scenarios that are too complex 3.47 1 3.78 1 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare 3.53 1 21 Need for faculty ag reement on purpose and methodology for debriefing sessions 3.67 1 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient car e) 3.67 1 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs 3.60 1 2.44 1 25 Need to verify if commercially ava ilable scenarios address tasks and objectives at the proper level for allied healthcare education 3.56 1

PAGE 121

121 Table 4 1. Continued Item Technical Feas ibility Mean IQR Instructional Feasibility Mean IQR 26 Need to verify if pre written scenarios are appropriate for the average educational level of the allied healthcare student 3.61 1 3.61 1 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education 3.59 1 3.42 1 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum 3.25 1 3.32 1 33 Need to find ways to address institutional or accreditation pressures which can hamper the implementation of flexible methods of training 3.07 0.5 2.88 1 34 Need to identify core curriculum across allied healthcare programs where the use of s imulation could benefit the greatest number of students 3.42 1 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas 3.28 1 36 Need to establish a validated matrix concerning when to employ tr aditional teaching methods versus simulation 3.32 1 3.11 1 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) 3.28 1 38 Institutions need to give faculty the ability to network through observation of clinical simulation use in other programs/institutions 3.53 1 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediti ng bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations providing de facto regulation 2.71 1 2.41 1 41 Institutions should work to address the lack of a systematic or coordinated means to identify issues in si mulation based allied healthcare education 2.82 1 2.50 1 42 Institutions should network with other institution to optimize simulation research 3.42 1 3.35 1

PAGE 122

122 Table 4 1. Continued Item Technical Feas ibility Mean IQR Instructional Feasibility Mean IQR 43 Educational institutions should work to find ways to schedule inter professional simulation ba sed education (IPSE ) scenarios 3.42 1 3.39 1 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medical centers when using the consortium model 2.74 1 46 Need to find ways to ensure educational issues are considered during the early stages of simulator design (a framework) 3.74 1 47 Need to find ways to ensure simulators are designed with feedback features that are pedagogically sound 3.42 1 3.39 1 48 Develop w ays to mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technology 2.68 1 2.83 1 49 Need to find a way to ensure that the design, development, integration, and u se of simulator technology becomes an integrated enterprise with developers, clinicians, and educators working together towards the same goal 2.84 1 2.78 1 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex 3.56 1 3.72 1 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex 3.26 1 3.47 1 52 Need to address the fact that there is a lack of simulation technology designed specifically for alli ed healthcare curriculums 3.06 0.75 3.06 1 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives 3.53 1 3.37 1 54 Need to develop guidelines concerning the appropri ate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives 3.33 1 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs 3.24 1

PAGE 123

123 Table 4 1. Continued Item Technical Feas ibility Mean IQR Instructional Feasibility Mean IQR 56 Need to find ways to ensure that educators, simulation technicians, and clinical faculty work together, they can have a tendency of not communicating and integrating ideas, needs, methods, and resources 3.47 1 3.58 1 57 Need to address the fact that simulation is often vi ewed as an added burden by faculty members and is therefore not fully supported 3.33 1 3.32 1 58 Need to address the fact that simulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in education 3.5 3 1 59 Need to address the fact that faculty does not have the time to prepare complex simulation scenarios 3.31 1 3.61 1 60 Need to address the fact that faculty development needs to include the proper use of simulator technology 3.74 1 3.47 1 61 Need to address the fact that faculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology 3.72 1 3.68 1 62 Need to address the fact that faculty development should include th e use of the debriefing sessions; why, when, and how to conduct them 3.74 1 3.84 0 63 Need to address the fact that faculty development should include the integration of simulation into the curriculum which includes an understanding of the different modal ities and technologies available 3.68 1 3.68 1 65 Ne e d to address how best to provide simulation specialists continuing education opportunities to keep abreast of changes in the field 3.53 1 3.53 1 66 Need to create standardized certification opportuniti es for those teaching with simulation 3.58 1 2.89 .75 67 Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students 2.89 1 2.89 1 68 Need an acceptable continuing education curriculum for faculty and staff teaching with simulators/simulation 3.56 1 3.32 1

PAGE 124

124 Table 4 1. Continued Item Technical Feas ibility Mean IQR Instructional Feasibility Mean IQR 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios 3.24 1 3.21 1 72 Need to address the fact that a lack of time for orientation and mentori ng of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) 3.06 .75 73 Need to address the fact that there is a lack of support from administration, faculty, and technical staff due to concerns about the validi ty of simulation 3.06 1 3.13 1 74 Need to address the fact that individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own 3.17 1 3.16 1 75 Need to address the lack of time and resources to train incumbent faculty, meaning simulators that are purchased go unused or underused 3.06 .75 2.94 1.75 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to ach ieve a specific teaching method, and if the session should be a team or individual interaction 3.47 1 3.53 1 78 Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive 3.24 0 2.82 1 80 Need to address the fact that there is a lack of appropriate support equipment such as cameras, crash carts, catheters supplied by the institution 3.18 1 81 Need to address the fact administrators do not fully fund ongoing maintenance and training in their a nnual budgets 2.81 1 2.82 1 82 Need to address the fact that it is not known what the best mix of clinical and simulation is in order to meet cost effectiveness 3.05 0 3.06 0

PAGE 125

125 Table 4 1. Continued Item Technical Feas ibility Mean IQR Instructional Feasibility Mean IQR 83 Need to find solutions to the fact that individual institutions are hampered in their expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space 2.94 1 84 Need to educate stakeholders better so they have an understanding of when simulation may be an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load 3.12 0 Most consensus items fell in the mid range of the feasibility scale with the lowest mean score being Need to address the lac k of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs S tudents not being fully prepared for higher education classes in the h ealthcare fields may be an issue facing many institutions ( Allied Health Center of Excellence 2012) but this was seen as a general education issue, not a simulation issue. T he highest rated was Need to address the fact that faculty development sho uld include the use of the debriefing sessions; why, when, and how to conduct them 3.84 ( 0.83) Simulation literature is clear that feedback is essential to successful simulation based learning, and while not all simulation programs fully utilize th is process there exists research and guidelines concerning what works ( Fanning & Gaba, 2007; Mort & Donahue, 2004) The standard deviation for consensus items ranged from 0 78 to 1.57 with an average of 1.21, meaning there was some variation in responses Five items (62IF, 78TF, 82 IF and TF, 84 IF) reached absolute consensus (IQR=0). The only item that reached absolute consensus on both technical (3.05 0.78) and instructional feasibility

PAGE 126

126 (3.06 1.19) Need to address the fact that it is not known what the best mix of clinical and simulation is in order to meet cost effectiveness This is consistent with the literature which states that the use of simulation to enhance or replace clinical rotations is one of the major initiatives in allied healthc are education and training (Bradley & Postlethwaite 2003b) The average rounded mean for most items (n=147) was 3 which is consistent with the literature which states that simulation will continue to be used but that implementation will be challenging ( McGaghie et al. 2010 ; McLaughlin et al.; 2010 ; Rosen, 2008 ) There were 58 items that did not reach consensus during the first round and those items covered the entire range from research to resources. The areas of disagreement highlighted the complexit y of the issue and set the stage for deeper analysis by the panelists for their rankings in round two. It was hoped that the results of round one would lead to more insight in to the panelist rankings through the inclusion of the justifications or suppor ting comments in the round two questionnaire There were 109 items in round one with NA response s ; 37 items had NA on both technical and instructional feasibility, giving a total of 163 NA responses. Reasoning for those responses were provided during roun d two when given. It was decided due to the fact that items had been drawn from diverse literature and not from panelist input, that if an item reached consensus (70 % ) on an NA response during round two the item would be removed for round three. Panelis ts did comment on some items and these were cod ed for theme Table 4 2 displays all the input statements from round one that were coded as an issue or concern including those explaining an NA response.

PAGE 127

127 Table 4 2. Round one, e merging areas o f concern by topic area Theme Count Item # Associated Statement(s) Limitations of technology 1 30 Technology is not always adequate for replacing clinical experience (P 1428 30 ) Need versus feasibility 1 General, resource items Many things are feasible t o study, but are not needed, with comparatively few the opposite; there should be a distinction made between NEED and FEASIBILITY (P 1438 resources) Barriers to collaboration 1 General collabor ation items There may be different models available in the U S compared to experience in the UK. While there is interest in collaboration, it can be very difficult across sectors (P 13 73, collaboration) Currency of technology 2 12TF, 12IF Assuming we are optimizing the mix using current technology (P1428, 12TF & IF ) NA not needed 5 10TF, 10IF, 50 TF 50 IF 52 TF 52IF, 54 TF 54IF, 55 TF, 55IF Not needed. The same metrics should be used to evaluate educational outcomes for any method or technique, with many examples are wel l established in the literature (P1438, 10TF & IF) L ikely not needed or unnecessary, or the question is based on a false perception of need, e.g. simulations specifically for allied health (P1438, 50 TF & IF 52 TF & IF, 54TF & IF, 55 TF & IF ) NA not an issue specific to simulation 10 27TF, 27IF, 29 TF, 29IF, 30TF, 30IF, 33TF, 33IF, 35TF, 35IF Not needed, redundant, or not specifically applicable or attributable to the use of simulations in healthcare education specifically, but exist as educational issues in general (P1438, 27TF & IF, 29 TF & IF, 30 TF & IF, 33 TF & IF, 35 TF & IF) The panelist input was in several cases used to make clarifying edits to existing items (i.e., 12, 30, 79, 80, and 84). The edits to the round two questionnaire can be seen in Appendix G. There were only five panelist s (1367, 1373, 1399, 1428, and 1438) who made comments in round one. Panelist 1438 a Program Director of Cardiovascular Sonography made the most with four comments. These comments were used to explain an NA response and after reviewing these comments i t was

PAGE 128

128 decided that if the number of NA responses increased to 70 % in subsequent rounds the item would be removed The comments also highlighted areas that were seen as too broad in scope, too contextual, or general education issues as opposed to simulatio n issues. Delphi Sur vey Round 2 This section presents those items which reached consensus in the second round As expected, the number of consensus items did increase during this round. The level of consensus had been predetermined as having an IQR of le ss than 1.2 (Alexander, 2008). Listed in Table 4 3 are those items that reached consensus in the second round with their mean, IQR, and the change in the IQR from round one to help demonstrate the variation in consensus between rounds Because instructio nal and technical feasibility were rated independently, some items reached consensus on one but not both feasibility standards. The full results of each round can be found in Appendix I. There were 17 responses to the second round. Those items that wer e edited due to panelist input in round one have the word edit under their number and changes are marked with bold for additions or strikeouts for deletions. Results were analyzed only for those who responded to round two Table 4 3. Questionna ire items which reached consensus in round two Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 1 Need to set up a clear research agenda for simulations in allied healthcare education 3.65 1 0.75 3.5 1 0.00 2 There is a need to research the impact of simulated learning as it impacts real world skills application 3.88 1 0.00

PAGE 129

129 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning specific procedural skills 3.71 1 0.50 3.79 1 0.75 4 Research is needed to demonstrate if skills acquired through simulation training are retained ove r time 3.79 1 0.50 3.74 1 1.00 5 Evaluation of simulation needs to be done contextually and over time through long range studies 3.59 1 0.00 3.41 1 0.00 6 Need for verification or refutation of relationship between increased student confidence as a re sult of using simulation and actual clinical performance 3.29 1 0.00 3.53 1 0.00 7 Need valid research concerning the impact of simulation on student learning outcomes for allied healthcare skills 3.59 1 0.00 3.65 1 0.00 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes 3.29 1 0.25 3.29 1 0.00 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments 3.35 1 0.50 3.18 1 0.75 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques 3.31 .25 1.25 2.88 0 1.00

PAGE 130

130 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change New Conduct research to find best an dragogical strategies concerning when to use simulation with large class sizes versus condensed but personalized simulation scenarios (when to use team based, individual scenario, student observation, etc.) 3.41 1 NA 3.28 1 NA 11 Institutions need to dev elop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available for session, and experience level of the participants 3.71 1 0.00 1 2 edit Need to develop guidelines concerning the best mix of current clinical and simulation based training for optimal learning outcomes 3.55 1 1.00 3.68 1 0.50 14 Need to find consensus concerning curriculum content for simulation across allied healt h disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) 3.29 1 0.00 2.81 1 1.00 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across instituti ons (e.g., standard curriculum for all respiratory therapists) 2.68 1 0.00

PAGE 131

131 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 16 Need validated method of measurement to verify transferability of simulation performance to the clinical setting 3.24 1 0.00 3.29 1 0.00 17 Need to develop guidelines whic h ensure students are not overwhelmed with scenarios that are too complex 3.47 1 0.00 3.76 1 0.00 18 Need to develop guidelines which ensure faculty are not overwhelmed with scenarios that are too complex 3.41 1 0.00 3.71 1 0.00 19 Clear learning outcome s must be developed in a way that is easily communicated to and understood by students using simulation 4.24 1 0.75 4.18 1 0.50 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare 3.53 1 0.75 3.44 1 0.00 21 Need for faculty agreement on purpose and methodology for debriefing sessions 3.86 1 0.75 3.59 1 0.00 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulat ed patients cannot teach the responsibility of patient care) 3.75 1 0.00 3.62 1 0.50 23 Need for all stakeholders to create measurable (objective) benchmarks for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation 3.19 .25 1.75 3.12 0 1.50 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs 2.46 1 0.00 2. 27 1 0.00

PAGE 132

132 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare education 3.65 1 0.00 3.38 1 1.00 26 Need to verify if pre written scenarios are appropriate for the average education al level of the allied healthcare student 3.71 1 0.00 3.50 1 0.00 27 Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions 3.13 1 1.00 3.19 .25 1.5 0 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced educati on 3.81 1 0.00 3.41 1 0.00 29 Need to develop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher 2.85 1 0 .25 2.94 .25 1.25 30 edit Need to address the issues that accreditation, licensing, and certification do not always allow for required clinical experience to be substituted with simulated experience when it has been proven to be a sufficient substitute 3.07 1 1.00 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum 3.38 1 0.00 3.24 0 1.00

PAGE 133

133 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 32 Need to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system 3.66 1 0 .75 3.38 .25 1.75 33 Need to find ways to a ddress institutional or accreditation pressures which can hamper the implementation of flexible methods of training 3.21 .75 0 2 5 2.75 .88 0.13 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could benefi t the greatest number of students 3.65 1 0.00 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas 3.56 1 1.00 3.31 1 0.00 36 Need to establish a validated matrix concerning when to employ tr aditional teaching methods versus simulation 3.37 1 0.00 3.18 0 1.00 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium mod el) 3.13 1 0.00 38 Institutions need to give faculty the ability to network through observation of clinical simulation use in other programs/institutions 3.56 1 0.00 3.62 1 1.00 40 Need to address the disparate voluntary oversight of educational inst itutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations providing de facto regulation 2.63 1 0.00 2.33 .5 0.50

PAGE 134

134 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 41 Institutions should work to address the lack of a systematic or coordinated means to identify issues in simulation based allied healthcare education 2.75 1 0.00 2.56 1 0.00 42 Institutions should network with other institution to optimize simulation research 3.41 1 0.00 3.31 1 0.00 43 Educational institutions shou ld work to find ways to schedule inter professional simulation based education (IPSE ) scenarios 3.59 1 0.00 3.31 1 0.00 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources 3.58 1 0.75 4 5 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medical centers when using the consortium model 2.71 1 0.00 2.72 1 0.75 46 Need to find ways to ensure educational issues are considered during the early stages of simulator design (a framework) 3.46 1 0. 5 0 3.29 0 1.00 47 Need to find ways to ensure simulators are designed with feedback features that are pedagogically sound 3.32 1 0.00 3.34 1 0.00 48 Develop ways t o mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technology 2.59 1 0.00 2.75 1 0.00

PAGE 135

135 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 49 Need to find a way to ensure that the design, development, integration, and use of simulator technology becomes an integrated enterprise with developers, clinicians, and educators working together towards the same goal 2.79 .5 0.50 2.84 .13 0.88 50 Proper care needs to be taken to ensure students are not overwhelmed with te chnologies that are too complex 3.61 1 0.00 3.73 1 0.00 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex 3.21 1 0.00 3.44 1 0.00 52 Need to address the fact that there is a lack of simulation t echnology designed specifically for allied healthcare curriculums 2.88 0 0.75 3.00 0 1.00 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives 3.41 1 0.00 3.29 1 0.00 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives 3.31 1 0.00 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs 2.88 .25 1.75 3.00 .50 0.50 56 Need to find ways to ensure that educators, simulation technicians, and clinical faculty work together, they can have a tendency of not communicating and integrating ideas, needs, methods and resources 3.53 1 0.00 3.53 1 0.00

PAGE 136

136 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 57 Need to address the fact that simulation is often viewed as an added burden by faculty members and is therefore not fully supported 3.47 1 0.00 3.41 1 0.00 58 Need to address the fact that simulation is not supp orted because there is a lack of strong theoretical and philosophical basis for its use in education 3.50 1 0.25 3.57 1 0.00 59 Need to address the fact that faculty does not have the time to prepare complex simulation scenarios 3.47 1 0.00 3.62 1 0.00 60 Need to address the fact that faculty development needs to include the proper use of simulator technology 3.75 1 0.00 3.53 1 0.00 61 Need to address the fact that faculty development needs to include an understanding of simulation methodology and the u nderlying learning theories that support the methodology 3.81 1 0.00 3.65 1 0.00 62 Need to address the fact that faculty development should include the use of the debriefing sessions; why, when, and how to conduct them 3.71 1 0.00 3.82 0 0.00 63 Need to address the fact that faculty development should include the integration of simulation into the curriculum which includes an understanding of the different modalities and technologies available 3.71 1 0.00 3.74 1 0.00 64 Need to address the fact that oft en too much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop 3.24 1 0.50

PAGE 137

137 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 65 Ne e d to address how best to provide simulation specialists continuing education opportunities to keep abreast of chang es in the field 3.52 1 0.00 3.38 1 0.00 66 Need to create standardized certification opportunities for those teaching with simulation 3.53 1 0.00 3.59 1 0.00 67 Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students 2.94 0 0.75 2.88 0 1.00 68 Need an acceptable continuing education curriculum for faculty and staff teaching with simulators/simulation 3.70 1 0.00 3.44 1 0.00 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios 3.34 .63 0.38 3.32 1 0.00 70 Need to address the fact that an aging workforce will lead to critical faculty shortages in community and technical colleges 2.85 0 2.00 2.93 1 1.00 71 Need to address the fact that low salaries make retention of a qualified workforce difficult 2.71 1 1.25 2.63 1 2.00 72 Need to address the fact that a lack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledg e loss) 2.94 .25 1.75 3.00 0 0 .75 73 Need to address the fact that there is a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation 3.07 .50 0.50 3.07 .75 0.25 74 Need to address the fact t hat individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own 3.14 .06 0.94 3.18 0 1.00

PAGE 138

138 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 75 Need to address the lack of time and resources to train incumbent faculty, meaning simulators that are purchased go unused or underused 3.06 0 0 .75 2.91 .13 1.63 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to achieve a specific teaching method, and i f the session should be a team or individual interaction 3.59 1 0.00 3.65 1 0.00 77 Need to show return on investment to decrease pressure to reduce high cost programs 2.96 0 2.00 3.21 0 2.00 78 Need to address the fact that the cost associated with h iring dedicated experts for simulation programs is prohibitive 3.20 0 0.00 2.76 1 0.00 79 edit Need to address how to best equip and support facilities to properly run the appropriate scenarios Need to address the fact that facilities are inadequate to pr operly run the scenarios 3.18 0 2.00 2.94 .25 1.75 80 edit Need to understand the appropriate level of support equipment such as cameras, crash carts, catheters supplied by the institution Need to address the fact that there is a lack of appropriate sup port equipment such as cameras, crash carts, catheters supplied by the institution 3.12 0 1 .00 2.81 1 1 .00 81 Need to address the fact administrators do not fully fund ongoing maintenance and training in their annual budgets 2.81 1 0.00 2.70 1 0.00

PAGE 139

139 Table 4 3. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 82 Need to address the fact that it is not known what the best mix of clinical and simulation is in order to meet cost effectiveness 3.24 0 0.00 3.06 0 0.00 83 Need to find solutions to the fact that individual institutions are hampered in their expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space 2.73 1 0.00 2.93 .75 0.50 84 edit Need to find ways to educate stakeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load 3.26 1 1 .00 3.19 .25 0.25 in their IQR but five i tems had significant changes (2.00): Need to address the fact that an aging workforce will lead to critical faculty shortages in community and technical colleges Need to address the fact that low salaries make retention of a qualified workf orce difficult 77 Need to show return on investment to decrease pressure to reduce high cost programs Need to address how to best equip and support facilities to properly run the appropriate scenarios None of these items had comments in round one that specifically addressed why the item was given a specific rank, although there was one general comment concerning resource items (77 and 79). Item 79 had been edited for clarity in the round two

PAGE 140

140 questionnaire but for the most part panel ists decided to modify their ranking based solely on the statistical data from round one. Item 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes of 0.75 to 1, meaning there was more variance in the ratings in round two, but it remained Need to find ways to educate stakeholders better so they have an understanding if and when simulation is an appropriate so lution for dealing with institutional issues such as student learning, funding cutbacks, increased student load 0 .25, although the mean remained relatively stable going from 3.12 ( 1.32 ) to 3.19 ( 1. 1 7). These adjustments in scores can be explained in part by the lower return rate in round two; however when developing the round three questionnaire modifications were made to the previous response column for all but one panelist, demonstrating that pan elists did reexamine their round one score s and in some cases modified their choice s Standard devi ation ranged from a low of 0.53 on Need to address the fact that individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own Need to find ways to address institutional or accreditation pressures which can hamper the implementation of flexible methods of training and Need to address the fact that there is a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation The average standard deviation was 0.98, meaning there was nearly an entire point on the responses for most items ; however the average spread between

PAGE 141

141 it ems was less for round two than it was in round one demonstrating more consistency in responses between panel members. There were 105 total NA responses on 52 items, 27 of which had NAs for both technical and instructional feasibility. Sixteen items had two NA responses and five had three NA responses. Reasoning for those responses were provided during round three when available Seven participants had at least one NA response : 1387, 1399, 1419, 1428, 1438, 1453, and 1471. All of these individuals are over the age of 40, five have a BA or graduate degree, six hold at least one relevant certification, and six had at least six years of experience in the field. All these panelists reported having simulation specific training and are currently work ing with simulation. Three of the participants voting NA had similar positions, answering simulation coordinator or patient simulation lab coordinator. Demographically, while all the participants with an NA response had extensive experience, there was no discern able pattern to their having voted NA. Similarly, there was little commonality between the two participants with the largest number of comments. There were a total of 187 comments for round two significantly more than in round one, with eight participant s giving no comments. Participants 1419 and 1453 made the majority of the comments, 37(20 % ) and 110(58 % ) respectively. Participant 1419 has over seven years of experience in simulation and currently lectures in radiation therapy. Participant 1453 has ov er 11 years of experience in simulation and currently works as a simulation coordinator. I t appears that experience has a greater impact on the way a participant answered than their specific field or context although the small sample size makes compariso ns across participants difficult to validate

PAGE 142

142 Only five items had no comments: 49, 54, 63, 76, and 83. Comments were given no t only for NA responses or for ratings outside the range of consensus; they often gave ckground. Not all comments are listed in Table 4 4, only those themes coded more than once as being a concern, or coded as a justification for ranking outside the mean. The same item is sometimes listed twice due to the fact that more than one panelist m ade a comment on an item that was coded with the same theme. Some comments have been edited to include only the relevant text but were not edited for grammar or spelling A full list of comments can be seen in the questionnaire for round three in Append ix H. Table 4 4. Round two e merging areas of concern by topic area Theme Count Item # Example Associated Statement(s) Justification 1 1TF Changed from a 5 to a 4 to account for the lack of resources currently available (P1367 1TF ) Lack of co nsensus 2 1TF 26IF Many of these items are feasible If you could ever get consensus of a group of educator to agree within a few years. (P1414 1TF ) Complex issue 2 45TF, 80TF Depending on the t echnological means, complexity can be the challenge (P1453, 80TF) Curriculum challenges 2 28IF, 45IF Some modifications to curriculums may be required to determine if objectives can be met with simulation (P1453 28IF ) Questionable item 2 73TF, 73 IF not sure this is fact (P1419 73TF & IF ) Time consuming/ const raints 5 2TF, 4TF, 4IF, 59TF, 59IF This is also time consuming but has been implemented in some programs (P1453, 4TF & IF) Resources require time to develop (P 1419 59TF & IF ) Cost 6 15TF, 45TF, 58TF, 67IF, 70IF, 78IF curriculum content would require high fidelity trainers and not all institu tions can fund such equipment ( P1423 15TF) This is a budgetary concern and obstacle (P1453 70IF )

PAGE 143

143 Table 4 4. Continued Theme Count Item # Example Associated Statement(s) Not important/ needed 8 52TF, 52IF, 55TF, 55IF, 68TF, 68IF, 70TF, 70IF U nnecessary over kill (P1419 68TF & IF ) Faculty limitations 8 11IF, 29TF, 29IF, 32IF, 36IF, 58TF, 62IF, 73TF Instructional feasibility would be very instructor dependent but is seemi ngly not difficult to implement (P1453 1 1IF ) Important as not all teachers can get the best out of simulation (P1387 29IF ) More development/ R&D 14 4TF, 4IF, 18TF, 18IF, 37IF, 39IF, 47TF, 48TF, 65IF, 72IF, 77TF, 81TF, 81IF, 82TF More research and development needed (P1453 37IF ) Some R & D may be required (P1453 47TF ) Context specific/ contextual 16 3TF, 8TF, 8IF, 12TF, 14TF, 14IF, 15TF, 15IF, 17TF, 17IF, 20IF, 24IF, 36TF, 36IF, 69IF, 82TF This would be very student specific (P1453 8TF & IF ) I don t know how this would be possible with so m any different disciplines and contexts (P1419, 12TF) This is program specific and faculty may not unde rstand how to write scenarios. ( P1453 69IF) Difficult to accomplish/ study 16 4TF, 5TF, 5IF, 6TF, 7TF, 7IF, 9TF, 10TF, 17TF, 18TF, 37TF, 37IF, 50TF, 51I F, 77TF, 77IF I think it is difficult to isolate the impact of discrete variables (P1373 6TF ) I feel this is too big and could be to the detriment discreet disciplines (P1419 37TF, 37IF ) Not sur e how this could be assessed (P 1419 561TF )

PAGE 144

144 Table 4 4. Continued Theme Count Item # Example Associated Statement(s) Politics 29 10I F, 16IF, 23IF, 31IF, 35IF, 37IF, 38IF, 39IF, 40IF, 41IF, 42TF, 43IF, 44IF, 46IF, 58IF, 61IF, 64IF, 66TF, 66TF, 66IF, 67IF, 68IF, 71IF, 74IF, 75IF, 77IF, 78IF, 80IF, 82IF I foresee politics as the greatest hurdle here (P1453 10IF ) Institutional autonomy ca n be a challenge here. perhaps regional or national organisations could step (P 1373 66TF ) Politics ....ohh yea. Lots of politics in this one (P1453 71IF ) not sure this is fact (P1419 73TF & IF ) resulted in the only edit fo r the round three questionnaire. There were eight comments stating that the item was not needed and these were associated with items that had at least one NA response. Those concerns that were coded most often included cost, faculty and contextual constr aints, needing more research and development, too difficult to accomplish, and political constraints. Delphi Sur vey Round 3 This section presents the results of the third and final round There were 16 responses to this round Items having reaching an IQ R of less than 1.2 (Alexander, 2008) were considered to have reached consensus Listed in Table 4 5 are those items that reached consensus in the third round with their mean, IQR, and the change in the IQR from round two. As instructional and technical f easibility were rated independently, some items reached consensus on one but not both standards. Results were analyzed

PAGE 145

145 only for those who responded to round three. The full results for each round can be found in Appendix I. Table 4 5. Questio nnaire items which reached consensus in round three Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 1 Need to set up a clear research agenda for simulations in allied healthcare education 3.53 1 0.00 3.53 1 0. 00 2 There is a need to research the impact of simulated learning as it impacts real world skills application 3.75 1 0.00 4.13 1 1.00 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning sp ecific procedural skills 3.58 1 0.00 3.63 1 0.00 4 Research is needed to demonstrate if skills acquired through simulation training are retained over time 3.50 1 0.00 3.50 1 0.00 5 Evaluation of simulation needs to be done contextually and over time thro ugh long range studies 3.44 1 0.00 3.38 1 0.00 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual clinical performance 3.19 .25 0.75 3.44 1 0.00 7 Need valid research co ncerning the impact of simulation on student learning outcomes for allied healthcare skills 3.50 1 0.00 3.50 1 0.00 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes 3.13 0 1.00 3.25 1 0.00 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments 3.19 .25 0.75 3.19 .25 0.75

PAGE 146

146 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques 3.27 0 0 .25 2.73 .5 0.50 New Conduct researc h to find best andragogical strategies concerning when to use simulation with large class sizes versus condensed but personalized simulation scenarios (when to use team based, individual scenario, student observation, etc.) 3.30 1 0.00 3.17 0.75 0.25 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available for session, and experience level of the parti cipants 3.65 1 1.00 3.63 1 0.00 12 Need to develop guidelines concerning the best mix of current clinical and simulation based training for optimal learning outcomes 3.39 1 0.00 3.53 1 0.00 13 Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies 4.00 .5 1.50 3.88 1 1.00 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., basic anatomy and physiology courses requir ed by all disciplines) 3.31 1 0.00 2.67 1 0.00

PAGE 147

147 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respiratory therapists) 2.88 .25 1.75 2.59 1 0. 00 16 Need validated method of measurement to verify transferability of simulation performance to the clinical setting 3.25 .25 0.75 3.25 .25 0.75 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too com plex 3.31 1 0.00 3.63 1 0.00 18 Need to develop guidelines which ensure faculty are not overwhelmed with scenarios that are too complex 3.25 1 0.00 3.56 1 0.00 19 Clear learning outcomes must be developed in a way that is easily communicated to and under stood by students using simulation 4.25 1 0.00 4.19 .25 0. 7 5 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare 3.43 1 0.00 3.34 1 0.00 21 Need for faculty agreement on purpose and meth odology for debriefing sessions 3.79 1 0.00 3.49 1 0.00 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient care) 3.60 1 0. 00 3.41 1 0.00 23 Need for all stakeholders to create measurable (objective) benchmarks for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation 2.93 0 0.25 2.94 0 0.00

PAGE 148

148 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs 2.35 .5 0.50 2.29 0 1.00 25 Need to verify if commercially available scenarios addres s tasks and objectives at the proper level for allied healthcare education 3.44 1 0.00 3.20 1 0.00 26 Need to verify if pre written scenarios are appropriate for the average educational level of the allied healthcare student 3.50 1 0.00 3.37 .75 0.25 27 Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and a ttention during feedback/debriefing sessions 3.07 0 1.00 3.07 0 0.25 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education 3.67 1 0.00 3.31 1 0.00 29 Need to develop validated methods to ass ess if positive outcomes were a result of an effective simulation or effective teacher 2.63 1 0.00 2.73 1 0.75 30 Need to address the issues that accreditation, licensing, and certification do not always allow for required clinical experience to be substi tuted with simulated experience when it has been proven to be a sufficient substitute 2.86 .75 0.25

PAGE 149

149 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum 3.14 0 1.00 3.06 0 0.00 32 Need to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system 3.53 1 0.00 3.27 .06 0.19 33 Need to find ways to address institutional or accreditation pressures which can hamper the implementat ion of flexible methods of training 3.07 0 0.75 2.61 1 0.13 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could benefit the greatest number of students 3.56 1 0.00 3.31 1 1.00 35 Need to identify if a nd how simulation can be used to teach students to deal with moral and ethical dilemmas 3.50 1 0.00 3.20 1 0.00 36 Need to establish a validated matrix concerning when to employ traditional teaching methods versus simulation 3.27 .06 0.94 3.06 0 0.00 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) 3.07 .50 0.75 3.07 .50 0.50 38 Institutions need to give faculty t he ability to network through observation of clinical simulation use in other programs/institutions 3.50 1 0.00 3.47 1 0.00

PAGE 150

150 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 39 Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and support simulation based education 3.13 0 1.25 3.00 0 1.25 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations providing de facto regulation 2.43 1 0.00 2.27 0 0.50 41 Institutions should work to address the lack of a systematic or coordinated means to identify issues in simulation based allied healthcare education 2.47 1 0.00 2.33 1 0.00 42 In stitutions should network with other institution to optimize simulation research 3.31 1 0.00 3.20 0 1.00 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios 3.56 1 0.00 3.40 .5 0.50 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources 3.35 1 0.00 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximi ty to universities and medical centers when using the consortium model 2.46 1 0.00 2.57 1 0.00 46 Need to find ways to ensure educational issues are considered during the early stages of simulator design (a framework) 3.36 1 0.00 3.19 0 0.00

PAGE 151

151 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 47 Need to f ind ways to ensure simulators are designed with feedback features that are pedagogically sound 3.34 .63 0.38 3.36 .70 0.30 48 Develop ways to mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions f rom attempting to implement new technology 2.55 1 0.00 2.59 1 0.00 49 Need to find a way to ensure that the design, development, integration, and use of simulator technology becomes an integrated enterprise with developers, clinicians, and educators worki ng together towards the same goal 2.66 1 0.50 2.70 .75 0.63 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex 3.58 1 0.00 3.58 1 0.00 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex 3.09 .63 0.38 3.34 .63 0.38 52 Need to address the fact that there is a lack of simulation technology designed specifically for allied healthcare curriculums 2.73 .50 0.50 3.00 0 0 .00 53 Need to develo p clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives 3.44 1 0.00 3.30 .85 0.15 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., vis ual, haptic, olfactory) to use based on learning levels and objectives 3.05 .38 1.63 3.27 .50 0.50

PAGE 152

152 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs 2.73 1 0.75 3.00 0 0.50 56 Need to find ways to ensure that educators, simulation technicians, and clinical faculty work together, they can have a tendency of not communicating and integrating ideas, needs, methods, and resources 3.37 1 0.00 3.44 1 0.00 57 Need to address the fact that simulation is often vie wed as an added burden by faculty members and is therefore not fully supported 3.31 1 0.00 3.38 1 0.00 58 Need to address the fact that simulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in educat ion 3.39 .88 0.13 3.46 1 0.00 59 Need to address the fact that faculty does not have the time to prepare complex simulation scenarios 3.38 .25 0.75 3.53 1 0.00 60 Need to address the fact that faculty development needs to include the proper use of simu lator technology 3.55 1 0.00 3.31 1 0.00 61 Need to address the fact that faculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology 3.67 1 0.00 3.50 1 0.00 62 Need to address the fact that faculty development should include the use of the debriefing sessions; why, when, and how to conduct them 3.69 1 0.00 3.81 .25 0.25

PAGE 153

153 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 63 Need to address the fact that faculty development should include the integration of simulation in to the curriculum which includes an understanding of the different modalities and technologies available 3.69 1 0.00 3.72 .63 0.38 64 Need to address the fact that often too much time is needed to develop simulation programs; faculty and staff do not hav e the time to properly develop 3.36 1 0.25 3.38 1 0.00 65 Ne e d to address how best to provide simulation specialists continuing education opportunities to keep abreast of changes in the field 3.43 1 0.00 3.28 1 0.00 66 Need to create standardized certif ication opportunities for those teaching with simulation 3.44 1 0.00 3.50 1 0.00 67 Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students 2.73 .50 .50 2.69 1 1.00 68 Need an acceptable conti nuing education curriculum for faculty and staff teaching with simulators/simulation 3.61 1 0.00 3.33 1 0.00 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios 3.37 .75 0.13 3.34 1 0.00 70 Need to addr ess the fact that an aging workforce will lead to critical faculty shortages in community and technical colleges 2.77 1 1.00 2.73 1 0.00 71 Need to address the fact that low salaries make retention of a qualified workforce difficult 2.60 1 0.00 2.50 1 0.0 0

PAGE 154

154 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 72 Need to address the fact that a lack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) 2.80 1 0.75 2.93 0 0.00 73 edit Need to address the fact that there can be a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation 3.25 .25 0 .25 3.27 .50 0.25 74 Need to address the fact that individual faculty often have to take the initiative and be motivated to le arn how to use the simulators on their own 3.15 0 0.06 3.06 0 0.00 75 Need to address the lack of time and resources to train incumbent faculty, meaning simulators that are purchased go unused or underused 3.13 0 0.00 2.84 .13 0.00 76 Need to develop gu idelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to achieve a specific teaching method, and if the session should be a team or individual interaction 3.50 1 0.00 3.63 1 0.00 7 7 Need to show return on investment to decrease pressure to reduce high cost programs 2.81 .25 0.25 3.06 0 0 .00 78 Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive 3.00 0 0.00 2.60 1 0. 00 79 Need to address how to best equip and support facilities to properly run the appropriate scenarios 3.06 0 0.00 2.73 .50 0.25

PAGE 155

155 Table 4 5. Continued Item Technical Feasibility Mean IQR IQR Change Instructional Feasibility Mean IQR IQR Change 80 Need to understand the appropriate level of support equipment such as cameras, crash carts, catheters supplied by the institution 3.05 0 0 .00 2.73 1 0 .00 81 Need to address the fact administrators do not fully fund ongoing maintenance and training in their annual budgets 2.81 1 0.00 2.70 1 0.00 82 Need to address the fact that it is not known what the best mix of clinic al and simulation is in order to meet cost effectiveness 3.13 0 0.00 2.93 0 0.00 83 Need to find solutions to the fact that individual institutions are hampered in their expansion of simulation based educational opportunities by a lack of resources includ ing funding, available faculty, and space 2.57 1 0.0 2.69 1 0.25 84 Need to find ways to educate stakeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student lea rning, funding cutbacks, increased student load 3.16 .63 0.38 3.07 0 0.25 Consensus items had no change (0.00) Only Need to address the issues that accreditation, licensing, and certification do not al ways allow for required clinical experience to be substituted with simulated experience when it has been proven to be a sufficient substitute 4 IF Need for institutions using or thinking of using simulation to develop a shared database or portal for resources between round 2 and round 3 ( 0.50 and 0.38 respectively). Broad issues such as

PAGE 156

156 accreditation were problematic for several panel members as it was questionable if the return would be wor th not just the time and money, but the frustration of trying to form consensus across disciplines and institutions. Sharing resources is a recommendation in some of the literature concerning simulation implementation (Adamson, 2010; Jansen, Johnson, Lars on, Berry, & Brenner, 2009 ), but was seen by many of the panelists as too challenging. Item 39TF and IF Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and support simulatio n based education moved from no consensus (IQR=1.25) to absolute consensus (IQR=0). This item touches on political concerns, but the consortium model is becoming more accepted as a means to develop successful simulation centers ( Jeffries & Battin, 2012 ; Mase & Wattenbarger, 1980 ; Sportsman et al., 2009 ) The mean scores of 39 TF=3.13 and 39 concerns about difficulties in finding commonalities between educational institutions and other health science institutions while acknowle dging the practical need to collaborate to share expenses and resources. Some Delphi studies remove items that have reached consensus and ask the panel to only rank the non consensus items in later rounds I t was decided that for this study all items woul d be present in all three rounds to give voice to divergent scores as well as to allow panelists to change their mind due to previous comments or further reflection. The following 14 items displayed a higher IQR in round three but remained within the acc eptable range of consensus: Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques

PAGE 157

157 Need to find ways to address institutional or accreditation pressures which can hamper the implementation of flexible methods of training 49TF (2.66 .47) and Need to find a way to ensure that the design, development, integration, and use of simulator technology becomes an integ rated enterprise with developers, clinicians, and educators working together towards the same goal Need to address the fact that there is a lack of simulation technology designed specifically for allied healthcare curriculums 62IF (3.8 Need to address the fact that faculty development should include the use of the debriefing sessions; why, when, and how to conduct them Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios Need to address the fact that an aging workforce will lead to critic al faculty shortages in community and technical colleges Need to address the fact that a lack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) 77TF Need to show return on investment to decrease pressure to reduce high cost programs Need to address how to best equip and support facilities to properly run the appropriate scenarios Need to find solu tions to the fact that individual institutions are hampered in their expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space These items covered the entire range of issues and the chan ge in their scoring demonstrates the convoluted nature of the problems and possible solution s as well as the ability of the Delphi to engage panel members in reflective thinking The number of NA responses went down in round three from 105 to 91. Twenty t wo items (10, 27, 29, 30, 33. 35, 37, 39, 40, 41, 44, 50, 52, 54, 55, 58, 68, 70, 71, 72,

PAGE 158

158 78, and 83) had at least one NA response for both TF and IF. There were 14 items that expansion of simulation based educational opportunities by a lack of reso urces including funding, available faculty, and space These were seen as issue that did not impact simulation, but were more cultural or political in nature. Item 83 had two NA responses on TF and three on IF demonstrating that instructional and technical feasibility are not always interconnected. The same participants from round two with NA responses gave at least one NA response for round three: 1387, 1419, 1428, 1438, 1453, and 1471. Participant 1399, who had several NA resp onses in round two, did not return a questionnaire for round three. It must be noted that participant 1399 had 10 NA responses in round two, so it is probable that at least some of the difference between rounds comes from the lack of response from this pa rticipant. The average standard deviation was 0.83, meaning that the data points tended to be closer to the mean than on previous rounds. Standard deviation ranged from a low of 0. 25 on 31 IF (3.06.25) Need to develop a clear understanding of when to int roduce simulation supported IPSE into the allied healthcare curriculum 40 on 58 TF (3.391.40) Need to address the fact that simulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in education. It is worth noting that both of these items are concerned with the lack of a clear research based simulation methodology. IPSE was seen by many panelists as an important model for

PAGE 159

159 the future and this is supported by the literature ( Gough et al., 2012 ; He ndee, 1971; Sportsman et al 2009 ; Zhang et al., 2011 ). The finding s of round three show that t he majority of items (n=121) had a rounded mean score of 3, placing them in the feasible range There were 4 2 items that scored in the very feasible range mea ning that these items had manageable obstacles Surprisingly, 10 of these items were concerned with research issues, which then leads to the question, why is there so little research concerning simulation in allied healthcare education? Seven items fell in the probably un feasible range: 24TF, 24IF, 40TF, 40IF, 41TF, 41IF, 45TF. Consistently, items with the lowest mean score or with multiple NA responses tended to deal with collaboration issues or areas where consensus between different institutions, disc iplines, or accrediting agencies would be needed. There were no items with an averaged mean score on either end of the spectrum (1 = not at all feasible or 5 = extremely feasible). While no item was rated as being not at all feasible, emonstrate that all items have at least some barriers to their implementation. Future research may seek to use a panel of experts to rank these items in order o f priority as time and resources are limited. Discovering if an item is technically and instru ctionally feasible is only the first step; the item should also be needed before resources are spent pursuing it. There were fewer comments in round three (n=107) than in round two (n=187), and they were more evenly distributed among the eight participants : P1419=32, P1373=21, P1428=17, P1471=13, P1387=9, P1388=7, P1414=5, P1367=3. Participant 1453, who had 58 % of the comments in round two, did not leave any comments for round three. Results from round two showed that the small sample size makes

PAGE 160

160 compariso ns across participants difficult and ineffectual when trying to tie input to experience or field of practice. There were 12 instances in round three where a participant noted agreement with a comment from round two. This helped validate the Delphi method in that participants took note of the input of others and on occasion found it useful or persuasive. Not all comments are listed in Table 4 6, only those coded more than once as being a concern, or coded as a justification for a ranking outside the mean. The same item is sometimes listed twice due to the fact that more than one panelist made a comment on an item that was coded with the same theme. The same themes were used from round two when appropriate. Some comments have been edited to include only the relevant text, but they were not edited for grammar or spelling

PAGE 161

161 Table 4 6. Round three, e merging areas of concern by topic area Theme Count Item # Example Associated Statement(s) Student specific 3 8TF, 8TF, 8IF preferred learning styles of different students should be taken into account (P1387 8TF ) Faculty limitations 3 2IF, 11TF, 59IF but each faculty member has a different level of teaching perception which influe nces the validated matrix's use (P1414 11TF ) More development/ R&D 4 10TF, 17TF, 18TF, 37IF research and development needed (P 1419 37IF ) Context specific/ contextual 4 5TF, 5IF, 12TF, 15TF from state to state thus making a standardized consensus pr oblematic (more research or development on that 15TF) Difficult to accomplish/ study 5 17TF, 18TF, 37TF, 37IF, 75TF chicken and egg situation that needs organisational good will to address. (P1373 75TF ) Justification 7 1TF, 11I F, 13TF, 23TF, 35TF, 64TF, 64IF There is no lack of technology that is preventing this from being completed. (P1428 35TF ) This is based on our experience [Score 5] (P1419 64TF & IF ) Cost 11 2IF, 6IF, 14TF, 15TF, 33TF, 45TF, 45TF, 70IF, 78IF, 84TF, 84IF Not sure organizations are willing to finance this level of assessment. (P1414 2IF ) cost is the challenge for many rural schools (P1471 45TF ) Not relevant/ needed 12 4TF, 15IF, 24TF, 24IF, 33TF, 33IF, 52TF, 52IF, 55TF, 55IF, 68TF, 68IF, Does it really matter as we can't/don't validate that traditional non simulation t rg is retained over time either (P1414 4TF ) Not sure this is a "need". Might be nice to have but not needed. (P1367 15IF ) Politics 14 23TF, 37IF, 37TF, 37IF, 39TF, 39TF, 39IF, 40IF, 41IF 44IF, 44IF, 46TF, 56TF, 56IF politically challenging, feasible IF stakeholders value creation of consistent benchmarks (P1471 23TF ) This is a political question, not a technological one (P1428 39TF ) Lack of political c onsensus will influence choices (P 1388 41IF )

PAGE 162

1 62 experience and sometimes included a statement that their score may be valid only in that limited context. Panelists noted that some of these issues were not technical or instruc tional issues, but political or financial in nature. The ability of simulation based education to support different learning styles was introduced in round three and has been noted in some of the literature (Fountain & Alfred, 2009; Jansen et al., 2009). Due to the fact that there was no survey instrument to display comments from round three, all round three comments can be found at the end of Appendix I. Summary of Findings Chapter 4 reported the findings of this three round modified Delphi study as it p ertained to the questions: 1. In the opinion of a panel of experts, what technical issues will impact the future use of simulators/ simulation in allied healthcare education? 2. In the opinion of a panel of experts, what instructional issues will impact the futur e use of simulators/ simulation in allied healthcare education? 3. What is the technical feasibility of each issue impacting the future use of simulators/ simulation in allied healthcare education? 4. What is the instructional feasibility of each issue impacting t he future use of simulators/ simulation in allied healthcare education? This study began with 168 items to rate (84 issues scored on both technical and instructional feasibility) and the panel added only one item throughout the study, making a total of 170 items rated The panel reached consensus on all but two issues, scoring all items between probably unfeasible and very feasible. No item was rejected as being not at all feasible. The 91 NA response s while allowing for minority opinions. These minority opinions often noted that other issues, such as costs or politics, had greater impact on the item being feasible or

PAGE 163

163 unfeasible than either technology or instructional issues. No item was removed due to a co nsensus opinion of it not being needed/applicable. There was no item that ranked extremely feasible, meaning that out of 170 items, none is currently rated as being completely ready to be implemented. Healthcare s imulation is not a new concept, but it sti ll lacks a solid educational simulation centers (Gaba, 2004b). The re are still many questions concerning its effectiveness and long term impact in skills training, but eve n with all the unknowns institutions, including military institutions continue to explore and expand their use of simulation ( McGaghie et al., 2010 ; Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, 2013 ; Rosen, 2008 ). The results of this study support the idea that simulation will continue to be integrated into allied healthcare curriculum at ever increasing rates but to be fully utilized the implementation process should be thoughtful contextual, and grounded in rese arch based instructional practices. This is not always feasible when there is little rigorous research to rely on for decision making. Chapter 5 uses the results of this study to make some recommendations for strategic decision makers concerning the impl ementation and use of simulation in allied healthcare education and training.

PAGE 164

164 CHAPTER 5 DISCUSSION, RECOMMENDATIONS, AND FUTURE WORK This Delphi study explored and identified issues imp acting the future of simulation in allied healthcare education and tr aining through the consolidation and assess ment of expert opinions about the technical and instructional feasibility of successfully addressing the critical issues facing the future use of simulator technology and simulation methodology Chapter 5 discuss es the major findings broken down by issue type (research, curriculum, collaboration, tools and simulator technology, faculty and staff, and resources) then looks at recommendations for decision makers as well as lessons learned from this Delphi study. Fi nally, this chapter discusses the implication s for future research. The discussion is concerned with only the results from the third and final Delphi round. Healthcare educators have an important responsibility educating and training students so they have the knowledge and skills necessary to successfully and both civilian and military. There is a growing amount of simulation literature which supports the idea that simulated practice may improve the quality o f delivered care (Schmidt et al., 2013; Shear et al., 2013), but many questions still exist as to how, when, and where to best utilize simulation in allied healthcare education. This study provide s some insight and recommendation s which can be used to bas e future decisions and research. Discussion This Delphi study used an extensive literature review to develop 84 items to be rated by a panel of experts for their technical and instruction feasibility The a rticles chosen for the literature review focused on several aspects of simulation in education

PAGE 165

165 and training; however, l ess than a dozen dealt solely or in part with the use of simulation in allied healthcare education. This gap makes strategic decision making difficult as and stake holders must see evidence that the use of Ringsted, stergaard, & Dieckmann, 2011, p. 155). There is little research concerning the future of simulation in allie d healthcare education or on the current use of simulation in allied healthcare education; therefore, it was necessary to research articles which discussed simulation issues, theories, and practices among all levels of healthcare related education and extr apolate. This meant that some of the items were viewed as not needed or appropriate by some of the panelists. Other items were seen as not appropriate to this study as they were not issues of technical or instructional feasibility. The iterative nature o f the Delphi made it possible for pane l members to anonymously review, comment on items, and modify or maintain their initial ratings. This allowed panel members to take a broad view of the items as each member brought a different background and experienc e s to the process. The results of this study demonstrate that it is possible to find consensus among a wide range of allied healthcare educators and practitioners, but that there remain differences and concerns pertaining to the methodology, theory, and p ractice of simulation based education The major finding s of this study are discussed below. Research Issues There were ten original items listed under the theme of research issues. The andragogical strategies concerning when to use simulation with large class sizes versus condensed but personalized simulation scenarios

PAGE 166

166 fy compare simulation debriefing sessions to other educational methods and techniques While 10TF reached absolute consensus it also had one of the largest standard deviations, meaning there were minority scores ranging from two to five for this item. Item 10 also demonstrated that the relationship between technical and instructional feasib ility was not always strong as 10IF received the lowest mean score There is a need to research the impact of simulated learning as it impacts real world skills application received a score in the very feasible range (4.13 0.72). Research in this area has been noted as a gap in the literature ( Holzinger, Kickmeier Rust, Wassertheuer, & Hessinger, 2009; Kardong Edgren Adamson & Fitzgerald, 2010; McGaghie, Issenberg, Petrusa, & Scalese, 2010; Oku da et al., 2009) but as some panelists noted, conducting it wou ld be time consuming and costly It is recommended that the time and money be invested as understanding simulation impact on real world skills is important to recognizing how and when it sh ould be implemented in education and training environments Items 10TF and 10IF were also the only research items which received a score of NA. Comments on item 10IF included that politics would be the greatest hurdle and is not really an instructional o r technical feasibility issue. The comments seen most often under research issues were related to the high level of difficulty due to the amount of data needed or number of variables present. When dealing with healthcare education it is difficult to pin point a single variable or track

PAGE 167

167 a large number of students over time (Bradley, 2006 ; Bradley & Postlethwaite 2003b ; Garden 2008) literature are due to the fact that rigorous research req uires extensive resources, time, money, and the ability to account for outside factors which may impact results. In allied healthcare education it can be difficult to remove those outside factors without negatively affecting student lea rning and/or patien t outcomes While research may be difficult, it is still needed to validate the practice of simulation based education Practitioners, educators, and researchers should collaborate to develop robust research designs which can be used to validate current practice and positively influence future implementation Curriculum Issues Curriculum issues accounted for items 11 36 of the questionnaire. All curriculum items reached consensus, but only nine reached absolute consensus: 23TF (2.93 0.86) and IF (2.94 Need for all stakeholders to create measurable (objective) benchmarks for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions 31TF (3.14 1.13) Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum

PAGE 168

168 Need to find ways to address institutional or accreditation pressures which can hamper the implementation of flexible methods of training Need to establish a validated matrix concerning when to employ traditional teaching methods versus simulation These items received mean scores that ranged from probably unfeasible to jus t above feasible but had standard deviations that ranged from 0.44 to 1.30. Curriculum items had some of the widest range of responses. Those panelists giving the highest mean scores 1387 (average score 3.65) and1428 (average score 4.25) had slightly different backgrounds: a managing director for an organization who trains with simulation and researches simulation issues and a simulation coordinator, respectively. Both had more than 11 years of experience. Those giving the lowest average mean scores 1380 (2.92) and 1414 (2.85) were a simulation related business owner/simulation coordinator and director of a surgical simulation center. Both responded as having six to ten years of experience. From the demographic data it does not appear that either y ears of experience or current job explain the wide variance in responses. There were 12 items (20TF, 22TF, 23TF, 24IF, 25IF, 26IF, 27IF, 28TF, 29IF, 30IF, and 35TF and IF) that had one NA response. Seven items (24TF, 27TF, 29TF, 30TF, 31TF, 33TF and IF) had two NA responses. Three curriculum items had some of the highest feasibility scores of the entire study: Item Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies 25 0.58), Clear learning outcomes must be developed in a way that is easily communicated to and understood by students using simulation guidelines and objectives usin g current technology and teaching methodologies. Some simulation companies and groups such as the National League of Nursing have lesson

PAGE 169

169 plans and strategies ( Jeffries 2007; Kumar & Sherwood, 2007) which can be used as a starting point for developing gui delines and objectives for allied healthcare education. Alternatively, there were 11 curriculum items which had some of the lowest scores of the study: Need to find consensus concerning curriculum content for simulation across allied h ealth disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) Need to find consensus concerning curriculum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respiratory therapists) Need for all stakeholders to create measurable (objective) benchmarks for assessing a good clinician so it can be possible to assess st udent clinical success through the use of simulation Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postse condary education programs Need to develop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher Need to address the issues that accredi tation, licensing, and certification do not always allow for required clinical experience to be substituted with simulated experience when it has been proven to be a sufficient substitute Need to find ways to address institutional or a ccreditation pressures which can hamper the implementation of flexible methods of training Item 30IF (3.07 1.09) was one of two items in the entire study that did not reach consensus (IQR=1.50). This seems to mirror the literature which is split on whe ther simulation is an adequate replacement for clinical experience ( Bradley & Postlethwaite, 2003b; Cantrell, & Deloney, 2007; Chiniara et al., 2012; Colliver, 2002; Dawson, 2002; Gaba, 2004 b ; Gibbs, Durning, & Van Der Vleuten 2011 ; Haluck et al., 2007; O kuda et al., 2009) This is a very important issue which will impact the future use of simulation

PAGE 170

170 in allied healthcare education and training so further research in this area is vital. It is also important to understand when and if simulation can be used to replace clinical experience from a patient safety perspective ( Nishisaki et al. 2007) I t should not be used as a replacement for clinical time if the outcomes include greater risk to patients. Comments concerning the curriculum items noted that many of the issues listed were not simulation issues so much as educational issues. The differences in allied healthcare disciplines and even state regulations concerning accreditation and licensing were also noted as barriers to consensus building. Developi ng benchmarks, core curriculum, and objectives were seen as worthwhile pursuits, but difficult to achieve due to practical differences and political issues between individual instructors, administrators, and accrediting bodies. Curriculum issues are often contextual and developing solutions to these problems will probably be done within disciplines even though IPSE is becoming more valued ( Gough et al 2012). Finding individualized solutions may also be more cost effect ive and politically palatable. The se targeted solutions may lead to more far reaching solutions over time if these solutions are shared with the wider community An approach to using simulation in allied healthcare education which takes into account the entirety of the learning process has a better chance of influencing positive student outcomes than one which looks at simulation as a discrete variable. Just adding simulation to the curriculum does not improve learning; it is one tool that can be used in combination with other teaching str ategies when appropriate ( Chiniara et al., 2012; Gaba 2004 ; Jeffries, 2006) Simulation should be assessed and implemented as

PAGE 171

171 part of the instructional design process, integrating it into the curriculum if and when appropriate. Collaboration Issues Close ly related to curriculum issues, collaboration issues consisted of items 37 45. Collaboration a ppeared to be a concept that is seen as valuable, but difficult to attain. All but four collaboration issues (38IF, 42TF, 43TF, 45TF) had one NA response. Mos t NA responses came from two panelists, 1453 and 1387, a simulation coordinator and a managing director for an organization who trains with simulation and researches simulation issues. Both have over 11 years of simulation experience. Neither participant left comments supporting their reason for the NA responses. Participant 1453 did not leave any comments on their round three questionnaire; however, they had the same responses in round two, and left numerous comments stating that politics would be the b iggest obstacle to many collaboration issues. In round three several other participants agreed that politics, including an institutional aversion to shar ing resources, would play a big role in the feasibility of collaboration issues. Four collaboration it ems reached absolute consensus: Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and support simulation based education 40IF (2.27 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations providing de facto regulation 42IF (3.20 1 Institutions should network with other institution s to optimize simulation research Item 40IF had the lowest mean score of any item in this Delphi study as this would require a wide assortment of stakeholders to come to a consensus concerning

PAGE 172

172 accred itation and oversight. The average mean score in this group was 2.97, with the Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios IPSE was one area that panelists noted was already being explored and implemented and this trend is also noted in the literature (Gough et al 2012; Hendee, 1971; Sportsman et al Need for institution s using or thinking of using simulation to develop a shared database or portal for resources with little return on investment. The Federal Medical Simulation and Training Consortium ( FMSTC) Educational Framework Initiative (EFI) (Educational Framework Initiative, Office of Naval Research, 2014) is currently working on a simulation database for the DoD and other federal agencies, but it is not currently known what impact this will have on the field or if it will be shared outside the DoD community. Most panelists agreed that pursuing collaboration issues could be beneficial for the field, but worried about the practicality of institutions with varying budgets, programs, and regulations working together. It is important that collaboration between institutions, programs, educational and clinical sites, as well as between simulation developers and educators is pursued if simulation is to positively impact student and patient outcomes ( Ada mson, 2010; Jansen et al., 2009) This may require a change in the culture of healthcare education but economic factors may help drive solutions that were once politically unattainable. Tools and S imulator T echnology I ssues Items 46 55 were concerned wit h the tools and simulator technology available for use in allied healthcare education. The average mean for these items was 3.09,

PAGE 173

173 Need to develop clear guidelines concerning the nee ded level of fidelity of a simulation for teaching specific skills or learning objectives had the highest mean score. Nine items had one NA response: 47IF Need to find ways to ensure simulators are designed with feedback features that ar e pedagogically sound Develop ways to mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technology Need to find a way to ensure that the design, development, integration, and use of simulator technology becomes an integrated enterprise with developers, clinicians, and educators working together towards the same goal Proper care nee ds to be taken to ensure students are not overwhelmed with technologies that are too complex Need to address the fact that there is a lack of simulation technology designed specifically for allied healthcare curriculums 54IF (3.05 1 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives Need to develop guidelines concerning t he level of realism needed for human tissue and organs Two items, 52IF (3.00 1.31) and 55IF (3.00 1.26), had two NA responses. Two panelists, 1438, a program director with over three years of experience, and 1453, a simulation coordinator with over 11 years of experience, accounted for all NA responses in this topic area. Comments on these items showed that some panelists felt that realism, or a high level of fidelity, is unnecessary for simulation activities to be successful. It was also noted that it is not necessary to have simulation technology specifically for allied

PAGE 174

174 healthcare education, only that it is necessary to have simulation that is used in such a way that it meets the needs of allied healthcare education (Bruhn & Philips, 1985). Overal l there were few comments in this section with most of the concerns being on the proper use of the technology as opposed to the technology itself. Several noted that more development is needed to fully address feedback and cost issues, but remarked that t he relationship between users and developers is complex and not always driven by the needs of discreet fields. A needs analysis should be done by any program or institution looking at implementing simulation to help assess if the appropriate technology ex ists to address its specific instructional needs. Unfortunately, w ithout more research concerning issues of fidelity, skills assessment, feedback, and student outcomes it may prove difficult to accurately gauge the appropriateness of the technology for sp ecific tasks and learning outcomes. Faculty and Staff Issues Items 56 76 were concerned with faculty and staff issues. There were nine items having a mean score of < 3.00, but the average mean for faculty and staff items was 3.28 placing many of these it ems in the feasible range. The lowest scoring were often related to budgetary concerns: Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students 70TF (2.7 Need to address the fact that an aging workforce will lead to critical faculty shortages in community and technical colleges Need to address the fact that low salaries make retention of a qualified workforce difficult Need to address the fact that a lack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss)

PAGE 175

175 75I Need to address the lack of time and resources to train incumbent faculty, meaning simulators that are purchased go unused or underused All but 75IF had at least one NA and 70TF received three NAs as this was not seen as a priority. Ther e were 13 items that had one NA response (56TF, 61TF, 64TF, 67TF, 68IF, 69TF, 70IF, 71TF and IF, 72TF and IF, 73IF, and 74TF). Items 58TF and Need to address the fact that simulation is not supported because there is a lack of strong theoretical and p hilosophical basis for its use in education Need an acceptable continuing education curriculum for faculty and staff teaching with simulators/simulation NA responses as budgetary con cerns were viewed as a bigger problem than an aging workforce. There were six panelists who responded NA at least once in this section, giving it the largest percentage of NAs of any topic area. Comments for faculty and staff issues addressed concerns suc h as the need for more research and development, better education for faculty and administration concerning simulation capabilities, political issues and budgetary concerns Some comments noted that more collaboration in sharing best practices and resour ces could help alleviate some of the time and money constraints associated with developing simulation programs, but as rankings and comments in previous topic area s demonstrated, collaboration is not always feasible. Several panelists noted that their ins titution had successfully dealt with these challenges but recognized they could still be This is not an issue in my department but may be an issue for others Sharing these experiences and lessons le arned can help make the future implementation of simulation easier and more effective.

PAGE 176

176 Technology innovations will probably continue to outpace advances in methodology (Greenberg, 2004), but as some panelists noted, integrating simulation into the curricul um helps alleviate some instructional issues as well as addresses some concerns about faculty and staff training and support ( Adamson 2010; Jansen et al., 2009) Successfully addressing faculty and staff issues can be politically challenging and resource dependent, but educating and training faculty and staff in simulator technology and simulation methodology is recommended as an essential piece for any successful simulation implementation Resource Issues The last and smallest topic group was resource i ssues, items 77 84. This section had some of the lowest scores, having a mean average of 2.88 meaning that most scored in the range between probably unfeasible and feasible. Only seven items had a and none rounded up to the very feasible lev el : Need to show return on investment to decrease pressure to reduce high cost programs Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive 79TF (3 Need to address how to best equip and support facilities to properly run the appropriate scenarios Need to understand the appropriate level of support equipment such as cameras, crash carts, catheters supplied by the insti tution Need to address the fact that it is not known what the best mix of clinical and simulation is in order to meet cost effectiveness Need to find ways to educate stakeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load

PAGE 177

177 The most common comments were concerned about political obstacles, cost, and t he need for more development Some issues such as budgetary constraints were viewed as difficult to find solutions for and not directly tied to the use or abandonment of simulation Some panelists noted that solutions will p robably be local and contextua l ( e.g., grants or donations ) Several panelists commented that 84IF and TF could be very useful to research but caution ed that the different stakeholders should be viewed as independent variables (e.g., administrators, governance boards, and the public). Three panelists ( 1387, 1438, and 1453 ) all had NA responses on at least two Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive Need to find solutions to the fact that individual institutions are hampered in their expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space respo nses. These were viewed as budgetary and political concerns as opposed to technical or instructional issues. Limited resources are not a simulation specific issue, but the success of a simulation program is often dependent on having found solutions to t he issues noted in (Jeffries & Battin, 2012 p. 5). The more information an inst itution has concerning what works and what does not work the better they can manage their costs and resources. Ignoring the complexity and inter related nature of simulation technology may cause some institutions to inadequately prepare for acquiring and maintaining support

PAGE 178

178 equipment and personnel. Th is lack of appropriate resources can lead to improper or discontinued use of expensive simulation equipment. It is recommended that institutions looking to implement simulation take a broad view and factor i n all elements, including the time needed to develop run, and debrief scenarios, into their decision making process. Recommendations This Delphi study along with the support ing literature provides insight into the future of simulation in allied healthc are education and training. This study shows that there are weaknesses in all the areas reviewed (research, curriculum, collaboration, tools and technology, faculty and staff, and resources ) due to the complex ity of a successful learning environment. The results from this study, including the literature the following recommendations The se recommendations are broken into two parts: considerations for specific stakeholders and general recommendations for institution s Stakeholder Considerations Administrators. Administrators need to understand the scope of what the institution is trying to accomplish and plan small steps to reach this goal. There are many aspects that need to be considered before any technology is purchased. The resources, faculty staff, and facilities issues need to be addressed and will vary based on simulation type and method. For instance, integrating a few case study simulations into a course may not require dedicated staff, new technolog y, or additional space, but a simulation lab to for IPSE may require all three. Faculty and staff need to be involved in the decision making process in order to make them vested partners. Identify individual faculty and staff members who are interested o r have previously used simulation and

PAGE 179

179 recruit them as simulation champions. The faculty and staff need to understand how it will affect them what the expectations are. There should be a single point of contact throughout the institution for questions, con cerns, and implementation issues. Administrators should research the possibility of a consortium model and other collaboration options including renting out simulation labs to the wider communit y. To garner community support plan community outreach inclu ding tours and press releases Researchers. Researchers need to explore the possibilities of collaboration with other researchers at different institutions. This can help with larger sample sizes and improve the ability to look at outcomes across group s. Research needs to be valid and have results that can impact the practice. Before starting any new research, conduct an extensive literature review to understand weaknesses in the existing literature and understand gaps in the research. Talk with educ ators and simulation developers to understand what their needs are and what issues are most important to address. Faculty and Staff. Simulation should be part of the faculty development experience. There are national organizations (e.g., Society for Simu lation in Healthcare) and users groups (e.g., SimGHOSTS) that can help support training. Research courses offered for simulation professionals and review scenarios available through organizations such as the National League for Nursing. It is important t o understand how and where simulation will be integrated into the curriculum, but not everything needs to be done at once. If simulation is new take integration in measured steps, look at the gaps and areas where students are struggling and choose one or two specific objectives to address with simulation. Low fidelity task trainers or case studies may be sufficient as an initial step. Plan how simulation will be used, e.g., as part of

PAGE 180

180 classroom instruction, skills lab, formative assessment, or in support of clinical time. The use of simulation should be in support of student learning and align with the learning objectives. Students. Potential students reviewing programs should review the use of simulation. Some questions to ask include: how is it used ( e.g., skills training or assessment), is there a set amount of simulation time required, what is the ratio of simulation to clinical time ? Each student will need to decide what type of simulation use they are looking for, but it should be part of the proc ess when deciding which institution and program is the best fit G eneral R ecommendations General recommendations reiterate some of the suggestions noted above, but look at the process of integrat ing and using simulation from a broader institutional standpo int. It is important for institutions to take a holistic approach as many of these recommendations are interconnected, such as collaborating to gain insights on requirements. Modular approach. The future implementation of simulation in allied healthca re education and training should be a modular approach allowing for flexibili ty and contextual customization. Allied healthcare is diverse and evolving; a single solution will not work in all situations and may be quickly outdated It is recommended that institutions do not get tied to a specific technology that cannot be adapted due to changing needs or new methodologies ( Bradley & Postlethwaite, 2003a ; Girzadas et al. 2009) When researching appropriate simulator technology it is recom mended that inst itutions take a n approach which includes consideration of cross utilization for a wide variety of programs within the institution and/or the wider community. Shared resources

PAGE 181

181 across programs can help alleviate many of the implementation barriers and incre ase educator awareness including how to use simulation in courses and programs not traditionally associated with simulation based learning (Jansen et al. 2009) A modular approach requires an honest assessment of the requirements. For example, if six pro grams need to learn BLS, is there a way to set up a single lab and schedule classes to rotate through? What is the lowest fidelity/lowest cost simulator that will work for the task? Or is there a slightly more expensive simulator that will work for BLS an d airway management allowing it to be used for multiple objectives? Starting with easy to use technology aimed at supporting specific tasks allows faculty and staff to get used to the technology and increase their confidence with its integration P lanning It is recommended that institutions understand and plan for student to simulation ratios. Students not actively engaged in the simulation will still need to actively participate in learning, e.g., watching previously recorded content, autonomously stud ying and critiquing the actions of the participants, or independently using another technology. Additionally, different types of simulation or different teaching modalities can be used in combination when the learning objective combines different types of skills (Bradley, 2006; Lane et al., 2001 ), thereby giving the students opportunities for self directed or project based learning. Computer simulations can be self directed and used as a teaching or study method Whatever methodology or technology is use d, the simulation curriculum should include valid, reproducible, and standardized learning experiences so all students receive the same experience Students can be active observers during a simulation and record actions for written or verbal evaluation. I f rotating students through a simulation lab, an instructor

PAGE 182

182 or other proctor will need to be available to work with st udents outside the lab This time can be utilize d as an independent study, an instructor can run a small group session, or students could view previously recorded simulation sessions. The number of students per simulator may vary some based on the task and technology, but for most allied healthcare education it is recommended to have 2 4 students per simulator. However, in some instances such as for team communication, it is more important to mirror real world experience and have the same number and types of roles as will be in a clinical setting. The number of students can be increased for basic courses through the use of audio/visual feeds for large groups Students can view the instructor actions through technology to learn basic ideas; however they cannot practice skills in large group settings. Requirements gathering. Objective strategic planning concerning the future use of sim ulation in allied healthcare education and training should be made by individuals with broad knowledge of the factors that influence its success (e.g., research, politics, cost, manpower) ( Wicklein, 1993 ) It is recommended that institutions collect data from simulation companies, educators, administrators, support staff, and practitioners concerning the current state of simulation technology and then validate requirements ( Haluck et al. 2007 ; McLaughlin et al. 2010 ; Ro sen, 2004 ; Shaffer et al., 2001 ) Research and development is currently being conducted, and the technology is continually evolving while the literature does not always keep pace ; therefore it is necessary to look beyond the literature and communicate wit h current practitioners and developers to make informed decisions

PAGE 183

183 If an institution is new to simulation they should consider hiring someone with experience to assist in requirements gathering. This individual should speak with all stakeholders and plan short and long term strategies and goals. They may need to begin the process with educating the stakeholders on simulation methodology and simulator technology, openly discussing the opportunities and concerns. It is difficult to gather requirements whe n individuals do not understand the possibilities. Classroom s paces may need to be reconfigured to accommodate simulation Some task trainers and lower fidelity simulators can be stored and moved in and out of classrooms as needed, but they will still ne ed storage space maintenance, and knowledgeable people to run them Collaboration. The optimal use of simulator types and simulation modalities is not currently known ( Hammoud et al., 2008 ; Jeffries, 2006) Designing and conducting research which incl udes learner characteristics, skills retention, and scenario complexity is vital to understanding how to appropriately design and utilize simulation programs ( McLaughlin et al. 2010 ) It is recommended that institutions and clinical sites collaborate on research to ensure studies have a large and robust sample size as well as the ability to track long range outcomes. message boards to engage with other researchers concerning opportunities. It is also recommended th at institutions share resources including curricula and scenarios in order to improve consistency, decrease development time, build shared knowledge, and increase ongoing communication concerning what works and what has been problematic. A strong foundati on for simulation based education can be built on shared knowledge (Adamson, 2010 ; Curtin & Dupuis, 2008 ) Simulation takes time and

PAGE 184

184 money to learn, implement, and support. Sharing experiences, successes, and challenges may improve future efforts to inte grate simulation into the curricula. In situations where funds, facilities, and staff are limited, institutions should look at the feasibility of collaborating on a mobile simulation lab. The costs are then shared, new facilities are not needed, and spec ialized staff is available for multiple locations. Human factors. The human role is an often overlooked aspect of simulation, but understanding the educational theories and technological capabilities supporting simulation and its practice is fundamental t o building a successful learning environment. It is recommended that institutions take into account and prioritize the investment in initial and ongoing education of all stakeholders in order to help the technology meet its full potential ( Bonnel and Smit h 2010 ). Knowing how to run the simulator is not the same as understanding scenario development, debriefing techniques, learning theories which support simulation, and evaluating student outcomes. It is recommended that institutions consider innovative solutions including sharing specialized faculty, involving faculty or students from other departments for actor patients, hiring graduate students as support staff, and opening simulation labs to outside users for a fee (Jansen et al. 2009 ; Ladyshwesky et al. 200 0 ) to lessen the impact of new simulation. It is necessary to ensure long term use and integration of simulation through the investment of training and education of the faculty and staff. All aspects need to be addressed, from curriculum integra tion to the debrief. A working group can help facilitate the exchange of ideas and help ensure a shared workload. Conferences and training courses are available through processional organizations such as the Society for Simulation in Healthcare, user gro ups for specific simulator s such as Laerdal and

PAGE 185

185 courses through institutions already utilizing simulation including the University of California Irvine and the University of Washington. Webinars are also available through organizations such as the Instit ute for Medical Simulation these can help train multiple staff and faculty without having to pay travel costs Curricula integration. It is recommended that programs integrating new simulation rework their curricul a and not just add in simulation to exis ting lesson plans Issues such as feedback, type of tasks to be taught, desired learning outcomes, proper placement of I SPE, appropriate time on task, and faculty preparation need to be addressed before the technology is even purchased ( Morgan et al. 200 6 ; Rosen, 2004 ) Time and effort spent on integrating simulation into the curriculum before implementation can ease faculty concerns, ensure proper technology is adopted, and positively influence outcomes Starting with one or two tasks that students are struggling with may help decrease the burden of curriculum integration. Analyze the gaps through observation of student performance, instructor feedback, clinical site input, and assessment outcomes. Is there a skill clinical sites say is missing or is there a task students do not feel confident performing? Begin assessing the use of simulation to address these needs, then work to integrate its use in a way that can be measured. Once there is enough data to understand if and how simulation is impactin g outcomes it can be integrated into the curriculum in more areas. Institutional structure. It is recommended that an institution has a single point of contact for simulation in order to help ensure communication with all stakeholders and promote the inte gration of ideas, needs, methods, and resources (J effries & Battin,

PAGE 186

186 2012 ; Mase & Wattenbarger, 1980 ; Sportsman et al., 2009 ) This individual should be involved in simulation acquisition scheduling, curriculum design, and budget planning. A Director of S imulation Operations or a Simulation Coordinator with a healthcare and education background can help provide oversight and execution of the simulation program or center. A single point of contact can help facilitate the development of curricula and traini ng materials across programs. They work to coordinate with administrators, deans, department chairs, faculty, staff, and the community helping to ensure proper planning This individual will assist with requirement validation and procurement to facilitat e cross utilization and proper use of funds. This individual is not responsible for the day to day set up and application of simulation, but oversees and helps to ensure simulation technicians are properly trained and supported. They work to s trategical ly plan and facilitate the growth of the simulation program and help ensure the effectiveness of the program to limit redundancies and assist with garnering facu lty support and adoption. Summary Many solutions will be contextual as each situation is differ type of simulation will need to be adapted to the educational needs of the learner and does not mean that results from research i n other fields is not us eful only that its applicability may be limited I t is simply not practical for each allied healthcare field to conduct its own research, and for the majority of initial skills training, team building and communication training it is not necessary ; ther efore, it is important that institutions utilizing and implementing simulation have open communication with each other

PAGE 187

187 Mistakes and successes should be shared so the knowledge base can be improved and expanded. There is consensus that simulation is an appropriate tool for allied healthcare education and training, but that it has not reached its full potential. Those seeking to implement simulation should take an unbiased look at the healthcare education system in its entirety before making decisions co ncerning the appropriateness of simulation for their specific needs. Objective guidelines concerning valid and appropriate simulation use can be developed if it is looked at as part of the system and not an independent variable. Future Work The Delphi pro cess allow s for areas of disagreement and therefore highlight s opportunities for further research. There is a need for more research concerning the use and effectiveness of simulation in healthcare education in general, and the use and effectiveness of si mulation in allied healthcare education in particular. N early every aspect of the practice and methodology requires more research; however, there are some areas where research may be the most beneficial. The future of simulation in allied healthcare educ ation and training may partially rest in the ability of the simulation community to perform rigorous research which a nswers how and when simulation a s a teaching method is as good as or better than other models, including clinical rotations. The aviation industry has shown that simulation training can positively impact team building, error handling, and performance, but these lessons have not seamlessly transferred to the healthcare field (Sexton, Thomas, & Helmreich, 2000). Taking another look at the suc cess of simulation based education in aviation over the past

PAGE 188

188 decade may help to pinpoint some of the areas that can be most improved upon, such as patient safety. The practice of simulation in allied healthcare education is complex, and there are many int er connected issues that still need to be addressed concerning the technical and instructional feasibility of simulator technology and simulation methodology in allied healthcare education Those items with the highest mean There i s a need to research the impact of simulated learning as it impacts real world skills application Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies Clear learning outcome s must be developed in a way that is easily communicated to and understood by students using simulation this does not mean that they are the most important issues needing to be addressed ; as several panel ist s noted, there is a difference between need and feasibility. Future research should seek to use a panel of experts to rank these items in order of priority as resources to pursue and implement solutions are limited. In several places panelists noted that simulatio n issues were learner dependent, thereby compelling the question what impact do learner characteristics such as previous knowledge, learner motivation, and social skills have on skills acquisition (Issenberg et al., 2011) ? It is not known at this time if these characteristics have any impact on student success or skills retention. Instructional and technical feasibility are not the only feasibility issues that need to be explored. Political and financial pressures are just two of the noted areas of conce rn which greatly impact the future of simulation in a llied healthcare education and

PAGE 189

189 training. One of the best way s to address political concerns and help alleviate financial ones is to recommend evidence based information and solutions With the level o f research available today institutions cannot make well informed decisions identifying critical elements for healthcare education (Issenberg et al., 2011). It is vital to conduct research which helps educators design programs which effectively meet insti tutional needs and demonstrate objective outcomes. Currently even recognizing and agreeing to outcomes measurement is a source of contention. Research needs to be conducted in allied healthcare education which can demonstrate that simulation use results in highly effective clinical practice. In order to accomplish many of the research needs, research collaborations should cross not only institutional boundaries, but include input from clinicians, simulation developers, researchers, and educators. Small sample sizes have limited the validity of current research, by addressing issues of collaboration larger student populations and long term outcomes could be assessed (Adamson, 2010). This study attempted to find differences in panelist responses due to b ackground or years of experience, but the sample size was too small to form any conclusions. To see if there are differences between allied healthcare fields, research would need to be conducted using separate groups of experts from different allied healt hcare fields. This may answer some questions about the validity of the responses and their applicability to specific fields or institutions. Lessons Learned The Delphi is a flexible approach, that is used commonly within the health and social sciences, y et little guidance exists to help researchers undertake this method of The fact that need, importance, and applicability of the items was not researched is a weakness of this study, but including

PAGE 190

190 these in a single study would have been burdensome for the panelists and may have resulted in lower response rates. The results of this Delphi study lead to more questions that still need to be explored. comments exposed under lying assumptions or experiences leading to differing judgments (Hasson et al., 2000). Further communication with panel members may have resulted in richer data concerning these areas of concern and difference. A more traditional Delphi study may have del ivered very different results as the panel members would have developed the items based on their own knowledge and experience. I want ed to g round the project in the literature and give the panel the opportunity to focus on existing items in the first roun d, therefore this traditional first step was skipped The panel was encouraged to add new items but the fact that there were already 84 items to score for technical and instructional feasibility possibly discourage people from providing suggestions for ne w items. This approach may have biased the responses or limited the available options (Hasson et al., 2000). When the literature is poor and assumptions have to be made concerning relevancy, having the panel develop the items would probably yield differe nt and fewer items more closely tied to their individual practice. The ability to then utilize the results for individual practice may have been clearer. A smaller more manageable number of items may have positively influenced the amount of time panelist s spent considering each item and increased retention In this case follow on work using a Delphi study to rank or score for need is suggested as those results may better align with strategic decision making When deciding on the type of Delphi and the m ethods of analysis the researcher is left to delve through existing studies, trying to align their research with an existing

PAGE 191

191 study design. The flexibility of the Delphi is both its strength and weakness as it allow s for a research design that is contextua l, but then leaves questions concerning the us efulness of the data gathered. Conclusion As this study shows, there is still research that needs to be done to address the issues pertaining to the use of simulation in allied healthcare education and trainin g. This work was very successful in finding consensus among a panel of experts on the 170 technical and instructional items in the study. Even though there was consensus on all but two items, there were also strong minority opinions on many items. This divergence of opinion indicates a strong need for further research, including gauging the need to pursue each item. It is important to not only identify and define technical and instructional issues which may impact the future of simulations in allied hea lthcare education, but to develop improved understanding of conceptual issues and evidence of their effectiveness will ultimately guide the development, use, and funding of s 2011, p. 163). This study makes a contribution to the field of allied healthcare education and training in that it sheds light on current areas of strength and weakness in order to better address these issues for future pract ice This study showed that a panel of experts can help inform decision makers in their strategic planning initiatives by analyzing the technical and instructional feasibility of issues impacting the field The Delphi method was used in this study as it allowed for an iterative process to collect and distill anonymous judgments of experts in order to improve understanding of the problem and feasible so lutions (Skulmoski et al. 2007). This project utilized the

PAGE 192

192 modified Delphi method proposed by Custer, Scarcella, and Steward (1999) in that it began with a set of carefully selected items drawn from an extensive literature review concerning issues affecting the future of simulation in allied healthcare education and training The items were then subjected to a three round Delphi to ascertain expert agreement concerning the items and the technical and instructional feasibility of addressing these issues. The Delphi method was applied in order to facilitate an anonymous exchange of ideas and opinions among a panel of experts concerning the items derived from the literature (Fleuren, Wiefferink, & Paulssen, 2004). By utilizing the Delphi method experts from a variety of geographic locations with varied backgrounds and professions were able to participate Th ere is a distinct research need in allied healthcare and training. This work was able to shed some light on the strengths and weakness in the current use of simulation in allied healthcare education and training, and the results can help to create a roadm ap concerning steps that can be taken to strengthen the field. Simulation implementation and utilization is complicated and the solutions may be as complex as the educational system in which it functions (Issenberg et al., 2011). Each individual or insti tution should view the results of this study and contextually decide which areas are of the most important to their future use of simulation. The need for highly skilled allied healthcare professionals will not be going away any time soon, so there is a n eed for the field to find research based methods for adapting and adopting simulation. Improved understanding can lead to improved implementation, and that can lead to improved results for practitioners and patients

PAGE 193

193 APPENDIX A INVITATION TO PARTICIPATE Dear I am writing today to invite you to participate in an exciting endeavor that I believe will have a positive impact in the field of allied healthcare education I am asking you to become a member of an expert panel of approximately 25 to 30 persons with varied backgrounds in the field of simulation in allied healthcare education For this study t he term simulation includes all modalities including mannequin simulators, task trainers, actor patients, or screen based simulations T he term expert is defined as someone who has designed, developed or supported a simulation lab for allied health care program ( s ) or has a minimum of three experience in any of the following roles within allied healthcare programs : operating simulations ; assessing sim ulation based competencies ; using simulation as a teaching or assessment tool ; developing new curriculum for the integration of simulation ; or having p ublished peer reviewed research concerning simulations at the allied healt hcare level This work is bein g undertaken as a part of my doctoral dissertation at the University of Florida. This study will use the modified Delphi method to explore the feasibility of successfully addressing the critical issues facing the future use of simulator technology and simu lation methodology in allied healthcare education Over the course of three rounds, i t seeks to consolidate and assess expert opinions in order to help develop guidelines and recommendations for strategic decision making. The first round will consist of r ating the technical and instructional feasibility of items developed from an intensive literature review During this round you will also have to opportunity to add new items or comment on existing items During rounds two

PAGE 194

194 and three you will be able to v As a member of the panel you will remain anonymous to all other panel members All survey interactions are conducted t hrough email and each panel member will be identified by a random number This number will be used on all correspondence including the questionnaires The value of this study will be determined by the quality of its participants Without the input of t he most qualified individuals in the field of simulation use in allied healthcare education the results of this endeavor will not be optimal Your willingness to commit to this endeavor is critical to its success Each member of the panel who complete th e three rounds will receive a copy of the findings I believe that the results of this study will be of use to you and others in the field of allied healthcare education The approximate timetable for this study, subject to some adjustment to accommodat Table A 1. Study timeline Task Date Demographic questionnaire completed June 23 rd 2013 Round One Questionnaire distributed to panel June 24 th 2013 Round One Questionnaire returned to the researcher July 7 th 2013 Rou nd Two Questionnaire distributed to panel July 2 1 st 2013 Round Two Questionnaire returned to the researcher August 4 th 2013 Round Three Questionnaire distributed to panel August 1 7 th 2013 Round Three Questionnaire returned to the researcher August 2 5 t h 2013

PAGE 195

195 I would be grateful if you would consent to becoming a member of the expert panel by completing the demographic survey Please go to http://www.surveymonkey.com ... to access the demographic survey You will notice the informed consent information at the begging of the survey, by completing the demographic survey you will have given your consent to participate in this research If you would prefer to receive hardcopies of all questionnaires please e mail me with your complete mailing address and I will send you hardcopies and a self addressed stamped envelope Following the completion of the demographic survey you will be sent the first round questionnaire or you will be notified that you did not mee t the qualification requirements. Thank you for giving this project your consideration I hope that you will find this study of interest to your professional and/or academic work and will consent to participate I would be happy to hear from you at jokke nney@ufl.edu or at 210 347 6200 if you have any questions concerning this project or if you wish to nominate someone else for this work Please forward this invitation to others who you feel would be interested and qualified for this study. Sincerely, Jo hanna Kenney

PAGE 196

196 APPENDIX B REJECTION LETTER Dear I am writing to thank you for taking an interest this study concerning the future of simulation in allied healthcare education and for taking the time to fill out the demographics survey While I was impr essed with your background and experience, d ue to the very specific needs of this Delphi study it was decided that other individuals were better fits for the panel Thank you again for your time, Sincerely, Johanna Kenney

PAGE 197

197 APPENDIX C INFORMED CONSENT Inf ormed Consent Protocol Title: The Future of Simulations in Allied Healthcare Education and Training: A Modified Delphi Study Identifying Their Instructional and Technical Feasibility Please read this consent document carefully before you decide to partici pate in this study and complete the demographic survey. Purpose of the research study: The purpose of this study is to consolidate and assess expert opinions about the feasibility of successfully addressing the critical issues facing the future use of sim ulator technology and simulation methodology in Allied Healthcare Education in order to help develop guidelines and recommendations for strategic decision making What you will be asked to do in the study: Members of the expert panel will be asked to pa rticipate in a three round Delphi study This process includes filling out an initial demographic questionnaire and then rate items and, if necessary add new items and provide qualitative feedback Time required: This will vary some based on the amount of input and feedback given The demographic survey should take more approximately 10 15 minutes The initial round may take 2 to 4 hours to complete The second and third rounds should take approximately 1 2 hours to complete All together the panel participants will probably spend 4 to 7 hours completing the surveys over the course of 3 months. Risks and Benefits: We do not anticipate any direct risk or benefit from participating in this project Compensation: There is no compensation for partici pating in this research Confidentiality: Your identity will be kept confidential to the extent provided by law Your information will be assigned a code number me office This code will be used to communicate to all panel members during the Delphi study When the study is completed and the data have been analyzed, the list will be destroyed Your name will not be used in any report Voluntary participation: Your participation in this study is completely voluntary There is no penalty for not participating

PAGE 198

198 Right to withdraw from the study: You have the right to withdraw from the study at any time without consequence Whom to contact if you have questio ns about the study: Johanna Kenney, Graduate Student, School of Teaching & Learning, phone: 210 347 6200, e mail: jokkenney@ufl.edu Dr Albert Ritzhaupt, Assistant Professor, Educator Technology, phone: 352 273 4180, e mail: artizhaupt@coe.ufl.edu Whom t o contact about your rights as a research participant in the study: IRB02 Office, Box 112250, University of Florida, Gainesville, FL 32611 2250; phone 392 0433 Agreement: By beginning the survey, you acknowledge that you have read this consent form an d agree to participate in this research

PAGE 199

199 APPENDIX D DEMOGRAPHICS SURVEY PERSONAL CHARACTERISTICS Name (For tracking purposes only, you will receive a participant number for use during the Delphi study) ______________________________ Preferred contact me thod ______________________________ Age group _ 20 30 __ 31 40 __ 41 50 __ 51 60 __ 61 years or older Highest Level of Education __ High School graduate __ Some College __ Associates Degree __ College Degree __ Graduate Degree Please list any relevant certifications _________________________________________________ _____________________ Years of experience supporting, using, researching, and/or evaluating simulator technologies and or simulation modalities in allied healthcare education. __ 0 2 __ 3 5 _ 6 10 __ 11 or greater Current position Title ___________ ______________________________ Years in current position ____________ Have you previously held a simulation related position (s) at a community or technical college? __ Yes __ No If yes Title__ _________ ______________________________ Years in position ____________ Title___________ ______________________________

PAGE 200

200 Years in position ____________ How many hours per week spent using or working with simulators/simulation? __ 1 10 __ 11 15 __ 16 20 __ 2 1 or more Type of simulator(s)/simulation used ______________________________________________________________________ ______________________________________________________________________ ____________________________________________________________________ __ ______________________________________________________________________ ______________________________________________________________________ Please list any simulation related publications _______________________________________________________________ _______ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ Other relevant information _________ _____________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________ ________________ Reason for interest in the study ______________________________________________________________________ ______________________________________________________________________ _________________________________________________________________ _____ ______________________________________________________________________ INSTITUTIONAL CHARACTERISTICS __ I am not currently associated with a technical or community college Institution Name ______________________ Student Enrollment Total Student Enrollment _____________ Total Degree Seeking Enrollment _____________ Total Non Degree Seeking/Other Enrollment _____________

PAGE 201

201 Allied Health Programs Offered ______________________________________________________________________ ________________________ ______________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ _____________________________________________________________________ Carnegie C lassification If you are unsure of your classification please go to http://classif ications.carnegiefoundation.org/lookup_listings/institution.php and look up your institution __ Public Rural serving Small __ Public Rural serving Medium __ Public Rural serving Large __ Public Suburban serving Single Campus __ Public Suburban serving Mu lticampus __ Public Urban serving Single Campus __ Public Urban serving Multicampus __ Public Special Use __ Private Not for profit __ Private For profit __ Public 2 year colleges under 4 year universities __ Public 4 year Primarily Associate's __ Private Not for profit 4 year Primarily Associate's __ Private For profit 4 year Primarily Associate's Programs Currently Utilizing Human Patent Simulations ______________________________________________________________________ ____________________________________ __________________________________ ______________________________________________________________________ Programs Currently Utilizing Patient Actor Simulations ______________________________________________________________________ _________________________ _____________________________________________ ______________________________________________________________________ Programs Currently Utilizing Screen Based Simulations ______________________________________________________________________ _______________ _______________________________________________________ ______________________________________________________________________ Thank you for taking the time to complete this survey I will contact you by May 24 th with information concerning the first roun d of the Delphi survey.

PAGE 202

202 APPENDIX E DEMOGRAPHICS SURVEY RESULTS Results for most of the demographic survey questions are shown in this section Any question which included personally identifiable information (PII) including publications and place of work has been removed No edits were done for grammar spelling or consistency (e.g., PhD versus Ph.D.) Table E 1 What is your preferred contact method ? Answer Options Response Percent Response Count email 96.0 24 hard copies 4.0 1 Please specify email or physical address 17 Answered question 25 Skipped question 0 Table E 2 Age group Answer Options Response Percent Response Count 20 29 0.0 0 30 39 16.0 4 40 49 44.0 11 50 59 16.0 4 60 or older 24.0 6 Answered question 25 S kipped question 0 Table E 3 Gender Answer Options Response Percent Response Count Male 48.0 12 Female 52.0 13 Answered question 25 Skipped question 0

PAGE 203

203 Table E 4 What is the highest level of education you have completed? Answer Options R esponse Percent Response Count High school 0.0 0 Some college 4.0 1 Associates degree 12.0 3 Baccalaureate degree 20.0 5 Some graduate school 0.0 0 Graduate degree 64.0 16 Other (please specify) 4 Answered question 25 Skipped question 0 PhD in process ; Commercial pilot training ; doctoral studies in progress ; Chiropractic degree also Table E 5 Degree(s) earned Response Text Response Count A nswered question 25 S kipped question 0 Integrated Marketing and Communications BSN, MSN Master of Health Science BS Biochemistry AA EMS Management Bachelor of Applied Science (Physiotherapy) Master of Physiotherapy Doctor of Philosophy MSN MS in Family Nursing, Nurse Education BS BHSc, MS PhD in Education Training and Performa nce Improvement BSN, MA BS in Physical Therapy MHS DHS Ph.D D.C. Bachalor of Science in Liberal Studies BS, MS Aeronautics and Astronautics B.S., M.Ed MSN BSN Diploma School of Nursing PhD MEd BA DASE Cert Ed Business, Accounting Associate i n Applied Science Allied Health Sciences Bachelor of Science in Health Sciences B.A. AS Science AAS, BA, MAT, PhD Master Degree in Aeronautical Science (MAS)

PAGE 204

204 Table E 6 Please list any relevant certifications Response Text Response Count A nswe red question 17 S kipped question 8 Certified TeamSTEPPS Master Trainer Certified Laerdal Educator Certified Anesthesia Technologist ANCC Certification Nursing Professional Development ACLS/PALS/PEPP/BLS/AMLS/EMPACT/PHTLS/TC3 Instructor; NAEMT Leve l II Instructor IBM Certified Developer in six areas of expertise Registered Vascular Technologist (RVT) Airline Pilot Licence Fellow of the Royal Aeronautical Society Crew Resource Management Instructor Airline Type Rating Examiner Certified Healt hcare Simulation Educator Certified Perioperative Registered Nurse Certified Nurse Educator Licensed Practitioner with HeartMath Certified health education specialist (CHES) National Registry Paramedic Oregon Paramedic RPTA, NREMT P CNE CCRN Alumn i FAcadMEd Certificate of Achievement in Clinical Simulation EMT P Paramedic Paramedic license, ACLS instructor Private Pilot, Instrument rating and Floatplane rating Table E 7. Years of experience supporting, using, researching, and/or evaluating simulator technologies and or simulation modalities in allied healthcare education Answer Options Response Percent Response Count 0 2 8.0 2 3 5 16.0 4 6 10 48.0 12 11 or greater 28.0 7 Answered question 25 Skipped question 0

PAGE 205

205 Ta ble E 8. Have you had any simulation specific training? Answer Options Response Percent Response Count Yes 84.0 21 No 16.0 4 If yes, please specify 21 Answered question 25 Skipped question 0 I was a trained simulation specialis t at WISER for 5 years, and I am now a simulation educator. Currently enrolled in Postgraduate Certificate Program Healthcare Simulation Have completed 3 of 4 courses. On line modules http://www.nhet sim.edu.au/ manufacturer courses on equipment use programming, repair, etc. 40 hour course through the U.S Army. Through Laerdal and Gaumard plus some courses taken on my own Conferences, reading of journals, peer education, education by product makers. IML basic user trainig for the HeratWorks 3D e chocardiography simulator. Approved Synthetic Flight Instructor Course Comprehensive Workshop in Debriefing at CMS Cambridge LAERDAL SUN User Conferences, Drexel University Simulation Certificate Course 8/2010 Laeredol Simulation mannequins Instructor status with the American Heart Association, American Red Cross, American Safety and Health Institute and the Emergency Care and Safety Institute I have also initiated immediate feedback systems with our CPR manikens and begun a unit using Second Life fo r age related health changes for students this semester. Several trainings through Laerdal Several trainings through the Oregon Simulation Alliance While I have never had any simulation specific "training", I have been a leader in developing simulation methods for over 40 years in multiple industries With respect to healthcare: founder and director of a very large multidiscipline simulation center in a hospital, residency, community environment. conferences and on site trainings 3 hour onsite trainin g session provided by METI Hands on in the United States Air Force Attended the CAE Healthcare Human Patient Simulator Conference (HPSN 2011, 2012). Completed all factory training with CAE (METI) and am fluent with physiological modeling user interface. Training on Laerdal and CAE simulators Various local courses from other medical disciplines (nursing and medical school faculty) and simulation equipment manufactures Also training through grad school on simulations use in the aviation field.

PAGE 206

206 Table E 9. What is the job title for your current position? Response Text Response Count A nswered question 25 S kipped question 0 Simulation Educator Chief Nurse Lecturer and Stream coordinator Radiation therapy Simulation Coordinator Train ing Specialist U.S Army Senior Lecturer Simulation Coordinator Innovative Practice Center Coordinator Program Director Managing Director of (redacted) Director Currently not employed due to disability Assistant Professor Assistant Professor in H ealth Education Program Chair, Instructor Director Director Professor freelance consultant in continuing medical education Owner (redacted) Simulation Coordinator Patient Simulation Laboratory Coordinator Simulation Technician Simulation Coordina tor Dean of Health Occupations/Director of Simulation Program Director and Department Chair Table E 10. About how long have yo u been in your current position ? Answer Options Response Average Response Count Years 6.32 Months 5.45 answer ed question 25 skipped question 0

PAGE 207

207 Table E 11. Have you previously held a simulation related position(s) at a community or technical college? Answer Options Response Percent Response Count Yes 24.0 6 No 76.0 19 Answered question 25 S kipped question 0 Table E 12. What is the job title for your current position? Response Text Response Count A nswered question 6 S kipped question 19 Associate Dean, Academic Support Adjunct Faculty San Antonio College Adjunct Faculty E MT Simulation Technician Adjunct Faculty EMS Instructor and Assistant Professor Table E 13. Time in this position Answer Options Response Average Response Count Years 3.50 Months 3.50 Answered question 6 Skipped question 19

PAGE 208

208 Table E 14. Other relevant previous experience Response Text Response Count A nswered question 17 S kipped question 8 2 years Chief Nurse -education and training fell under chief nurse; low fidelity simulation used animal emergency medicine technician (ICU care, assist in surgeries and procedures, phlebotomy, run lab tests, prep and administer meds, check in out and triage, etc) I have worked in staff development since 1998 Also have some academic teaching experience. FAculty in nursing edu cation, facilitated simulations with students, wrote simulations Developed and implemented computer based simulations for educational purposes over a 15 year period while employed in IT/consulting roles Simulator trainer (redacted) Airways 1998 2005 Past Certified EMT (redacted) Firefighter ADLS, BDLS Instructor TNCC 20 yrs Army Nurse Corp Chiropractic training included the use of cadavers (3 year program) Instructor for 20+ years ACLS, PALS, PHTLS, PEPP Paramedic instructor last 14+ years Professor and P rogram Director at another four year college Sr Clinical Education Specialist with Patient Simulator Manufacturer Adjunct faculty at (redacted) Community College 2004 2006 (Time of initial simulation development 15 years university teaching and research; lead educator for (redacted) (UK) 1996 to present; ATLS educator Simulation Consultant, (redacted) Author Was a Nationallly Certified EMT, license is expired currently. Certified AHA CPR BLS Healthcare & Heartsaver Instructor, actively teaching yearly. A/ V technician Simulation and WMD tech for (redacted) Health Systems emergency Department Also work ~36 hours per month as a paramedic to try and stay current in the field...

PAGE 209

209 Table E 15. On average, how many hours per week do you spend using or working with simulators/simulation? Answer Options Response Average Response Count 1 10 45.8 11 11 15 16.7 4 16 20 8.3 2 21 or more 29.2 7 Other (please specify)* 45.8 3 Answered question 6 Skipped question 19 I was a trained simulation specialist at WISER for 5 years, and I am now a simulation educator. currently not working directly with simulation It varies, some weeks lots, others none. 40 hours per week in simulation training ( 7 years )

PAGE 210

210 Table E 16. Type(s) of simula tor(s)/simulation used Response Text Response Count A nswered question 24 S kipped question 1 Laerdal Sim Essential Meti Man Noelle Sim Mom Sim Junior Laerdal Sim Man 3G Laerdal Sim Man Laerdal ALS Man Laerdal Sim Baby Laerdal Sim NewBBLS Torso Other Partial Task Trainers hard task trainer, low fidelity; high fidelity (3G SimMan) virtual reality software, authentic clinical environments, manequins various adult peds infant manikins from METI, Laerdal and Gaumard, CAE VR endoscopy, simple and co mplex task trainers (including Harvey and LSAT), and more Laerdal Sim Man/Sim Baby I have had interests and participation in projects which use online simulation and patient examples. Sim Man and megacode Kelly more than others Haven't started actively u sing our other models yet Also some medium fidelity and low fidelity as well as SPs and task trainers. High fidelity, medium fidelity, low fidelity task trainers Laerdol, Guamard productts IML HeartWorks 3D echocardiography simulator Clinical human m annequin SimMan 3G Noelle SimBaby Nursing Kid Neonate Chest tube insertation LP Rectal exam, IV starts, Sim IV, Harvey, Haptic Lap Mentor, Haptic Endoscopy, Difficult Airway Torso, Arterial Sticks, Suturing and Knot Tying, Vascular Suturing, Blood dr aws Patient actor simulations Mannequin si mulators with computer software Manikins, second life Live, task, low fidelity, high fidelity All Types All types of medical simulators: Hi fidelity full body, task trainers, VR, Standardized patients, hybrids ( we were the pioneers of surgical hybrids around 2007 ) METI Standard Man METE pediatric HPS A lower fidelity Laerdal simulator" Laerdal Laerdal, METI, Guamard We have 10 high fidelity simulators Eight are CAE Healthcare manikins and Four are Gaumard ma nikins. High fidelity: Laerdal SimMan, Sim Junior, Sim NewB, Sim Baby PTT: IV arms, intubating heads, LP backs, foley, TEE/TTE We do lab based and in situ simulations Task trainers or static simulators Computer based software simulation Dynamic full bo dy simulators CAE Istan, MetiMan; Laredal SimMan, SimMan 3G, SimBaby, Gaumard Noelle High fidelity field based simulation for both paramedic students and currently active field providers Have used or currently using a wide range of equipment (mostly laer dal), paid actors and volunteers.

PAGE 211

211 Table E 17. Please list any simulation related publications Response Text Response Count A nswered question 12 S kipped question 13 Note: publications not listed due to PII Table E 18. Other re levant information Response Text Response Count A nswered question 7 S kipped question 18 Developed a "Basic Use of Sim Man 3G" course at the (redacted) Graduate capstone project was in simulation happy to share if it would be helpful Member of the Hosp ital Emergency Response Team Society for Simulation in Secretary 2009 Nominations Committee 2006 Star Wars Finals Judge (2010) (redacted). (redacted) Presented at the Society for Simulation in Healthcare Conference, Orlando, FL (redacted) (redacted). Pres ented at the Society for Simulation in Healthcare Conference, Orlando, FL (redacted) Panel presentation at Society for Simulation in Healthcare Conference, Orlando, FL (redacted). Presented at the first Bio skills conference, Orlando, FL (redacted ) (redacted). Presented at the Congressional Caucus National Training and Simulation Association, Virginia Beach, VA (redacted). (redacted). Presented at the Society for Simulation in Healthcare Conference, Orlando, FL (redacted). (redacted) Presente d at the Capital Medical Center, Olympia, WA (redacted). (redacted) Presented at the Capital Medical Center, Olympia, WA (redacted) (2004) Masters in Nursing Commencement Speech Presented at (redacted). (2005 2008) Various Tour Presentations to DOD VIPs Presented at (redacted) (1985 2005) Various briefings to Soldiers and Higher Command Presented during 20 years as U.S Army Nurse officer. Using bio communication technology from the Institute of HeartMath and the EVOX from the Zyto corporatio n for the past 6 years Helped set up simulation centers at other locations Designed simulation centers for groups in foreign countries Acted as a resource to train people setting up centers in other locations, both national and international Have par ticipated in many national surveys of this type. Owner of (redacted) Simulation Specialist Dacum panel member N/A

PAGE 212

212 Table E 19. Reason for interest in the study Response Text Response Count A nswered question 21 S kipped question 4 More info rmation to share with the simulation community. Simulation is used throughout the (redacted) enlisted medical training Simulation is often taught OJT with no formal training program for military instructors This is minimal published research in the eff ectiveness/best use of simulation in allied health care. Passionate about simulation to advance the field To gather and provide information which would benefit sim man use in the helath care educational setting My interest in research is growing and I woul d like to take part in more studies I'm all about the simulation To provide assitance in the project, and curious as to the findings as well Table E 20. I am currently associated with a technical or community college Answer Options Response Per cent Response Count Yes 37.5 9 No 62.5 1 5 Answered question 24 Skipped question 1 Table E 21. Institution Name Response Text Response Count A nswered question 9 S kipped question 16 Note: Institution name redacted due to PII

PAGE 213

213 T able E 22. Allied health programs offered (please include name and type, e.g., associate degree, continuing education, certificate) Response Text Response Count A nswered question 9 S kipped question 16 Continuing Education Physiotherapy Occup ational Therapy Human Movement Health Science Podiatry Medical Raditions Occupational Therapy Pre physical therapy, BS Pre occupational therapy, BS Community Health, BS Biotechnology minor as part of a BS degree Paramedic Associate of Applied Science also Fire Science and Nursing AS Programs Nuclear medicine radiography sonography respiratory care dental hygiene dental assistant interdisciplinary studies medical assisting clinical lab science physical therapy occupation therapy Nursing Surgical Technology C ertificate programs CNA PCT Massage Therapy EMT Paramedic Phlebotomy Sterile Processing Community Nursing AD Respiratory Therapy AD Physical Therapy Assistant AD Occupational Therapy Assistant AD Laboratory Tech AD Medical OffIce Assistant AD Surgical Tech nology AD Health Information Technology AD ADN Associate Degree Nursing PN Practical Nursing Surgical Technician Certificate Medical Assistant Certificate Massage Therapy Certificate Medical Record Coding Certificate Pharmacy Technician Certi ficate We offer two Nursing degrees and 17 other Allied Health degrees, diploma and certificates Within my department we offer: EMT certificaiton AAS in Paramedicine BS in EMS Management (starting in Fall 2013) Other departments within my university off er degrees in Respiratory therapy (BS), Nursing (RN and BSN through transfer), Clinical Lab Science (BS), Medical Imagining (BS).

PAGE 214

214 Table E 23. Carnegie Classification If you are unsure of your classification please go to http://classifications. carnegiefoundation.org/lookup_listings/institution.php and look up your institution Answer Options Response Average Response Count Public Rural serving Small 0.0 0 Public Rural serving Medium 22.2 2 Public Rural serving Large 11.1 1 Public Suburban serving Single Campus 0.0 0 Public Suburban serving Multicampus 22.2 2 Public Urban serving Single Campus 11.1 1 Public Urban serving Multicampus 22.2 2 Public Special Use 0.0 0 Private Not for profit 0.0 0 Private For profit 0.0 0 Public 2 year colleges under 4 year universities 11.1 1 Public 4 year Primarily Associate's 0.0 0 Private Not for profit 4 year Primarily Associate's 0.0 0 Private For profit 4 year Primarily Associate's 0.0 0 Answered question 6 Skipped question 19 Table E 24. Programs currently utilizing human patient simulators Response Text Response Count A nswered question 9 S kipped question 16 EMT Basic and Paramedic programs I know of one program Physiotherapy Nursing Exercise Physiology Corpora te Fitness Athletic Training Biology Physical Education Psychology Sociology Health Studies Pre med Paramedic, Fire, Nursing All of the above listed Nursing Respiratory Therapy Medical Assistant Associate Degree Nursing and Practical Nursing Departments Nu rsing, Paramedic, EMT, Respiratory Therapy, Pharmacy Technician, OTA, PTA EMT, Paramedic, Nursing (presumably) Not sure about who else uses them as I'm not involved in their departments

PAGE 215

215 Table E 25. Programs currently utilizing patient actor simulation Response Text Response Count A nswered question 9 S kipped question 16 EMT Basic and Paramedic programs I know of one program Physiotherapy Unknown none Nursing PCT Respiratory Care Unknown Associate Degree Nursing Nursing, Paramedic, EMT, Respiratory Therapy, Pharmacy Technician, OTA, PTA EMT, Paramedic, Nursing (presumably) Not sure about who else uses them as I'm not involved in their departments. Table E 26. Programs currently utilizing screen based simulation Response Tex t Response Count A nswered question 9 S kipped question 16 EMT Basic and Paramedic programs I know of one program Medical Raditions Same as 23 [Pre physical therapy, BS Pre occupational therapy, BS Community Health, BS Biotechnology minor as part of a BS degree] none Only our one day medical encounters, currently building this area Unknown N/A Phlebotomy not in our department and unknown use in the university.

PAGE 216

216 APPENDIX F ROUND ONE QUESTIONNAIRE Dear Panelist, Thank you for agreeing to participate i n this modified Delphi study exploring the instructional and technical feasibility of successfully addressing the issues facing the future use of simulator technology and simulation methodology in allied healthcare education and training Attached to this email you will find the first round Delphi questionnaire with your unique participant ID This questionnaire was built after an extensive literature review, but you may add new items or give feedback on the existing items during this round Your input w ill be of great value in creating guidelines and recommendations based on the feasibility of successfully addressing the critical issues facing the use of simulator technology and simulation methodology within allied health care education Also attached is the contact information I currently have for you If you would like to add any information or correct this information please do so and return with the completed questionnaire by July 7th Technical issues are concerned not only with the simulation techn ology itself, but can include issues with peripherals such network support, simulator maintenance, technical difficulties in running the simulators, and the design and development of new technologies Instructional issues may include pedagogical issues su ch as the choosing of the most effective instructional method, policy issues such as accreditation requirements, curriculum issues such as assessment validation, and faculty issues such as training The purpose of this research is to develop items based n ot only the strengths and weakness of the technology, but also on the strength and weaknesses of the methodology It is important to not only identify and define technical and

PAGE 217

217 instructional issues which may impact the future of simulations in allied healt hcare education, but to develop a consensus concerning if these issues can be dealt with successfully The Delphi method is used to work towards group consensus concerning the list of items on the questionnaire Development of the items began with a lit erature review concerning the current use of healthcare simulations in education, specific examples of the use of simulation in allied healthcare, and the possible future of healthcare simulations in healthcare education Each item is to be rated on its t echnical and instructional feasibility as both are necessary if the item is to be strategically pursued You are requested to do the following: 1. Review all the issues listed in the questionnaire 2. Make any comments or suggestions, give feedback, or request clarification on any item in the spaces provided If commenting on a current item please note the item number before the comment. 3. Add new items as you deem appropriate 4. Rate t he technical (TF) and instructional feasibility (IF) of each item based on the included scale (more details in PDF) 5. Return your response by July 7th 2013. Thank you for your prompt attention Should you have any questions or concerns please feel free t o contact me by email at jokkenney@ufl.edu or at 210 347 6200.

PAGE 218

218 Rating Scale for Technical Feasibility 1 1= Not at all feasible, 2= Probably unfeasible 3= Feasible, 4= Very Feasible, 5= Extremely Feasible This is not intended to be an all encompassing list but an example of some issues at each level. 1. Not at all feasible Cannot be implemented Basic research and development of technology or methodology is needed Current technologies do not exist Cost of technology is excessive 2. Probably unfeasible Implementat ion not possible under current conditions Major research and development are needed before implementation Cost of technology and other resources would require additional funding Complexity of technology makes implementation difficult 3. Feasible Possible impl ementation Some research and development is still needed Cost of technology and other resources is manageable Complexity of technology makes implementation challenging 4. Very feasible Some indication that implementation is possible Small amount of research a nd development still needed (research and/or development is in process) Cost of technology and other resources can be covered Complexity of technology is manageable with training 5. Extremely feasible Can be implemented Current research and development is ade quate Necessary resources are available Training is available and adequate 1 Adapted from Ziglio, E (1996) The Delphi method and its contr ibution to decision making. In M Adler and E Ziglio (Eds. ), Gazing into the oracle: The Delphi method and its application to social policy and public health (3 33) London, Englan d: Jessica Kingley Publishers.

PAGE 219

219 Rating Scale for Instructional Feasibility 2 1= Not at all feasible, 2= Probably unfeasible 3= Feasible, 4= Very Feasible, 5= Extremely Feasible This is not intended to be an all e ncompassing list but an example of some issues at each level. 1. Not at all feasible Cannot be implemented Faculty unwilling to support changes No possibility of being acceptable for certification of skills Curriculum would need a major re write to support s imulation Politically unacceptable 2. Probably unfeasible Implementation not possible under current conditions Majority of faculty opposes changes Low possibility of being acceptable for certification of skills Extensive modifications are needed to curriculum Political obstacles 3. Feasible Possible implementation Some research and development are still needed Some possibility of being acceptable for certification of skills Moderate modifications are needed to curriculum Some political obstacles 4. Very feasible Som e instructional obstacles Faculty concerns can be addressed Possibility of being acceptable for certification of skills Some modification is needed to curriculum Minor political obstacles 5. Extremely feasible No major instructional obstacles Will be acceptab le to faculty Will be approved by accrediting/certification boards for certification of skills Can be easily integrated into existing curriculum No significant political obstacles 2 Adapted from Ziglio, E (1996) The Delphi method and its contr ibution to decision making. In M Adler and E Ziglio (Eds. ), Gazing into the oracle: The Delphi method and its application to social policy and publi c health (3 33) London, England: Jessica Kingley Publishers.

PAGE 220

220 Round 1 Questionnaire Rate and technical feasibility ( TF ) and instructio nal feasibility ( IF ) on a scale of 1 to 5: 1= Not at all feasible, 2= Probably unfeasible 3= Feasible 4= Very feasible, 5= Extremely f easible Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA Research issues 1 Need to set up a clear research agenda for simulations in allied healthcare education TF IF 2 There is a need to research the impact of simulated learning as it impacts real wor ld skills application TF IF 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning specific procedural skills TF IF 4 Research is needed to demonstrate if skills acquired through sim ulation training are retained over time TF IF 5 Evaluation of simulation needs to be done contextually and over time through long range studies TF IF 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual clinical performance TF IF 7 Need valid research concerning the impact of simulation on student learning outcomes for allied healthcare skills TF IF 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes TF IF

PAGE 221

221 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments TF IF 10 Res earch needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques TF IF Additional items or comments on existing research items (For comments on existing items, please state the item number) Curriculum issues 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available for session, and experience level of the participants TF IF 12 N e ed to develop guidelines concerning the best mix of clinical and simulation based training for optimal learning outcomes TF IF 13 Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies TF IF 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) TF IF 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respiratory therapists) TF IF

PAGE 222

222 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 16 Need validated method of meas urement to verify transferability of simulation performance to the clinical setting TF IF 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too complex TF IF 18 N eed to develop guidelines which ensu re faculty are not overwhelmed with scenarios that are too complex TF IF 19 Clear learning outcomes must be developed in a way that is easily communicated to and understood by students using simulation TF IF 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare TF IF 21 Need for faculty agreement on purpose and methodology for debriefing sessions TF IF 22 Need for all stakeholders to have an understanding of the limitati ons of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient care) TF IF 23 Need for all stakeholders to create measurable (objective) benchmark s for assessing a good clinician so it can be possib le to assess student clinical success through the use of simulation TF IF 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary educ ation programs TF IF

PAGE 223

223 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare education TF IF 26 Need to verify if pre written scenarios are appropriate for the average educati onal level of the allied healthcare student TF IF 27 Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions TF IF 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education TF IF 29 Need to de velop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher TF IF 30 Need to address the issues that accreditation, licensing, and certification do not always allow for required clinical experi ence to be substituted with simulated experience TF IF 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum TF IF 32 Need to address the issue that simulators are often s een as stand alone objects, not as a part of an integrated system TF IF 33 Need to find ways to address institutional or accreditation pressures which can hamper the implement ation of flexible methods of training TF IF

PAGE 224

224 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could benefit the greatest number of students TF IF 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas TF IF 36 Need to establish a validated matrix concerning when to employ traditional teaching methods versus simulation TF IF Additional items or comments on existing curriculum items (For comments on existing items, please state the item number) Col laboration issues 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) TF IF 38 Institutions need to give facul ty the ability to network through observation of clinical simulation use in other programs/institutions TF IF 39 Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and sup port simulation based education TF IF 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations provi ding de facto regulation TF IF

PAGE 225

225 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 41 Institution s should work to address the lack of a systematic or coordinated means to identify issues in simulation based allied healthcare education TF IF 42 Institutions should network with other institution to optimize simulation research TF IF 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios TF IF 44 Need for institutions using or thinking of using simulation to devel op a shared database or portal for resources TF IF 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medical centers w hen using the consortium model TF IF Additional items or comments on existing collaboration items (For comments on existing items, please state the item number) Tools and simulator technology issues 46 Need to find ways to ensure e ducational issues are considered during the early stages of simulator design (a framework) TF IF 47 Need to find ways to ensure s imulators are designed with feedback features that are pedagogically sound TF IF

PAGE 226

226 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 48 Develop ways to mitigate the cost of keeping up with the changing pace of the tec hnology which may dissuade some institutions from attempting to implement new technology TF IF 49 Need to find a way to ensure that the design, development, integration, and use of simulator technology become s an integrated enterprise with developers clinicians, and educators working together towards the same goal TF IF 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex TF IF 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex TF IF 52 Need to address the fact that there is a l ack of simulation technology designed specifically for allied health care curriculum s TF IF 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives TF IF 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learn ing levels and objectives TF IF 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs TF IF

PAGE 227

227 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA Additional items or comments on existing technology items (For comments on existing items, please state the item number) Faculty and staff issues 56 Need to find ways to ensure that e ducators, simulation technicians, and clinical faculty work together they can have a tendency of not communicating and integrating ideas, needs, methods, and resources TF IF 57 Need to address the fact that s imulation is often viewed as an added burden b y faculty members and is therefore not fully supported TF IF 58 Need to address the fact that s imulation is not supported because there is a lack of strong theor etical and philosophical basis for its use in education TF IF 59 Need to address the fact that f aculty does not have the time to prepare complex simulation scenarios TF IF 60 Need to address the fact that f aculty development needs to include th e proper use of simulator technology TF IF 61 Need to address the fact that f aculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology TF IF 62 Need to addre ss the fact that f aculty development should include the use of the debriefing sessions; why, when, and how to conduct them TF IF

PAGE 228

228 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 63 Need to address the fact that f aculty development should include the integration of simulation into the curriculum whi ch includes an understanding of the different modalities and technologies available TF IF 64 Need to address the fact that often t oo much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop TF IF 65 N e ed to address how best to provide simulation specialist s continuing education opportunities to keep abreast of changes in the field TF IF 66 Need to create standardized certification opportunities for those teaching with simulation TF IF 67 Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students TF IF 68 Need an acceptable continuing education curriculum for faculty and staff teaching with simulators/simulation TF IF 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios TF IF 70 Need to address the fact that an a ging workforce will lead to critical faculty shortages in community and technical colleges TF IF 71 Need to address the fact that l ow salaries make retention of a qualified workforce difficult TF IF

PAGE 229

229 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 72 Need to address the fact that a l ack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerni ng the technology (knowledge loss) TF IF 73 Need to address the fact that t here is a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation TF IF 74 Need to address the fact that indiv idual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own TF IF 75 Need to address the l ack of time and resources to train incumbent faculty mean ing simulators that are purchased go unused or un derused TF IF 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to achieve a specific teaching method, and if the session should be a team or individua l interaction TF IF Additional items or comments on existing faculty and staff items (For comments on existing items, please state the item number) Resources 77 Need to show return on investment for high cost simulation programs TF IF 78 Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive TF IF

PAGE 230

230 Item No. Issue Statement (in no particular order) 1. Not at all Feasible, 2. Probably unfeasible 3. Feasibl e 4. Very feasible 5. Extremely feasible NA 79 Need to address the fact that f acilities are inadequate to properly run the scenarios TF IF 80 Need to address the fact that there is a l ack of appropriate support equipment such as cameras, crash carts, catheters supplied by the institution TF IF 81 Need to address the fact a dministrators do not fully fund ongoing maintenance and training in their annual bud gets TF IF 82 Need to address the fact that i t is not known what the best mix of clinical and simulation is in order to meet cost effectiveness TF IF 83 Need to find solutions to the fact that i ndividual institutions are hampered in their expan sion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space TF IF 84 N eed to educate s takeholders better so they have an understanding of when simulation may be an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load TF IF Additional items or comments on existing resource items (For comments on existing items, please state the item number)

PAGE 231

231 APPENDIX G ROU ND TWO QUESTIONNAIRE Dear Panelist #, Thank you for your participation and input on Round One of this modified Delphi study exploring the instructional and technical feasibility of successfully addressing the issues facing the future use of simulator techn ology and simulation methodology in allied healthcare education and training As some of you noted, this research is only looking at one small part of a complex issue as feasibility and need is not always the same thing and this study will not be able to address every issue Your participation is greatly appreciated and I anticipate even more interesting insights in this round Attached to this email you will find the second round Delphi questionnaire You can use Adobe Acrobat or Adobe Reader to fill i n this form If you are using a MAC you may experience problems if you use MAC Preview. Attached is the questionnaire for Round Two of the study Your comments and feedback as well as one new item have been compiled and added to the questionnaire Some of the comments have been edited for clarity and consistency A few items were edited based on panel feedback for clarity; these are noted as edited in the item number and consist of strikeouts for deletion of words and bold for additional words Items l isted with an interquartile range (IQR) of less than 1.2 are noted with an asterisk (*) and have statistically reached consensus; however, you still have an opportunity to modify your rating There were a number of items where some respondents answered NA Reasoning for those responses are provided when given If an item reaches consensus on an NA response (70 % ) the item will be removed for Round Three.

PAGE 232

232 For this round you are requested to do the following: 1. Review all the issues listed in the questionnair e. 2. Review the group responses to Round One items The mean, standard deviation, and IQR are given along with the number of NA responses. 3. Review your Round One response against the group mean Any comments given during Round One are included Decide if y ou wish to re rank the item or leave your Round One ranking Check the box under No change to response if you wish to keep your rating from Round One If you wish to change, type in a number 1 5 or NA in the Rank column Note : Those listed with an aster isk next to the IQR number have reached consensus, but you may still change your rating. 4. Rate the technical (TF) and instructional (IF) feasibility of the new item based on the included scale by typing the number 1 5 or NA in the Rank column. 5. If your ranki ng is outside of group mean, please explain your reasoning in the last column You may also add any additional feedback you may think helpful to your fellow panelists Your input from this round could impact the way other panelists respond, so please be clear and concise. 6. Please return your response by August 4th Clicking on the Submit button on the first page will send it directly back to me. Thank you for your prompt attention Should you have any questions or concerns please feel free to contact me by email at jokkenney@ufl.edu or call me at 210 347 6200. Johanna Kenney

PAGE 233

233 Round 2 Questionnaire Rate technical feasibility (T F ) and instructional feasibility (I F ) on a scale of 1 to 5: 1= Not at all feasible, 2= Probably un feasible, 3= Feasible 4= Ver y feasible, 5= Extremely f easible Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. Research issues 1 Need to set up a clear research agenda for simulations in allied healthcare education Mean: SD: IQR: %NA 3.72 1.26 1.75 5.26% 5 TF Mean: SD: IQR: %NA 3.56 1.26 1* 5.26% 5 IF 2 There is a need to research the impact of simulated learning as it impacts real world skills application Mean: SD: IQR: %NA 3.74 0.87 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.94 1.28 2 5.26% 5 IF 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning specific procedural skills Mean: SD: IQR: %NA 3.63 1.16 1.5 0.00% 5 TF Mean: SD: IQR: %NA 3.67 1.39 1.75 5.26% 5 IF

PAGE 234

234 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 4 Research is needed to demonstrate if skills acquired through simulation training are retained over time Mean: SD: IQR: %NA 3.84 0.90 1.5 0.00% 5 TF Mean: SD: IQR: %NA 3.83 1.34 2 5.26% 5 IF 5 Evaluation of simulation needs to be do ne contextually and over time through long range studies Mean: SD: IQR: %NA 3.53 1.02 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.42 0.84 1* 0.00% 4 IF 6 Need for verification or refutation of relationship between increased student confidence as a re sult of using simulation and actual clinical performance Mean: SD: IQR: %NA 3.22 1.03 1* 5.26% 3 TF Mean: SD: IQR: %NA 3.50 1.00 1* 5.26% 3 IF 7 Need valid research concerning the impact of simulation on student learning outcomes for allied h ealthcare skills Mean: SD: IQR: %NA 3.56 1.12 1* 5.26% 5 TF Mean: SD: IQR: %NA 3.72 1.12 1* 5.26% 5 IF

PAGE 235

235 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes Mean: SD: IQR: %NA 3.33 1.07 0.75* 5.26% 5 TF Mean: SD: IQR: %NA 3.44 1.24 1* 5.26% 5 IF 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments Mean: SD: I QR: %NA 3.37 1.12 1.5 0.00% 3 TF Mean: SD: IQR: %NA 3.28 1.24 1.75 5.26% 3 IF 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques Mean: SD: IQR: %NA 3.11 1.31 1.5 5.26% 5 TF Mean: SD: IQR: %NA 2.76 1.26 1* 10.53% 3 IF Comments on Item 10: Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational met hods and not needed The same metrics should be used to evaluate educational outcomes for any method or technique, with many examples are well est ablished in the literature New Conduct research to find best andragogical strategies concerning when to use simulati on with large class sizes ver sus condensed but personalized simulation scenarios (when to use team based, individual scenario, student observation, etc.) Added New TF Added New IF

PAGE 236

236 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. Curriculum issues 11 Institutions need to develop a validated m atrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available for session, and experience level of the participants Mean: SD: IQR: %NA 3.72 1.58 2.75 5.26% 5 TF Mean: SD: IQR: %NA 3.63 1.07 1* 0.00% 3 IF 12 edited N e ed to develop guidelines concerning the best mix of current clinical and simulation based training for optimal learning outcomes Mean: SD: IQR: %NA 3.67 1.39 2 5.26% 3 TF Mean: SD: IQR: %NA 3.74 0.93 1.5 0.00% 3 IF Comments on Item 1 2 : assuming we are optimizing the mix using current technology 13 Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies Mean: SD: IQR: %NA 4.00 1.32 2 5.26% 3 TF Mean: SD: IQR: %NA 3.84 0.96 2 0.00% 3 IF 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., basic anatomy and physiology co urses required by all disciplines) Mean: SD: IQR: %NA 3.39 1.23 1* 5.26% 4 TF Mean: SD: IQR: %NA 2.89 1.05 2 0.00% 2 IF

PAGE 237

237 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across institution s (e.g., standard curriculum for all respiratory therapists) Mean: SD: IQR: %NA 3.17 1.45 2 5.26% 5 TF Mean: SD: IQR: %NA 2.58 1.22 1* 0.00% 1 IF 16 Need validated method of measurement to verify transferability of simulation performance to t he clinical setting Mean: SD: IQR: %NA 3.33 1.38 1* 5.26% 3 TF Mean: SD: IQR: %NA 3.28 1.24 1* 5.26% 3 IF 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too complex Mean: SD: IQR: %NA 3.50 1. 34 1* 5.26% 2 TF Mean: SD: IQR: %NA 3.83 1.21 1* 5.26% 2 IF 18 N eed to develop guidelines which ensure faculty are not overwhelmed with scenarios that are too complex Mean: SD: IQR: %NA 3.47 0.96 1* 0.00% 2 TF Mean: SD: IQR: %NA 3.78 1.22 1* 5.26% 2 IF

PAGE 238

238 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 19 Clear learning outcomes must be developed in a way that is easily communicated to and understood by students using simulation Mean: SD: IQR: %NA 4.11 1.33 1.75 5.26% 5 TF Mean: SD: IQR: %NA 4.11 0.94 1.5 0.00% 5 IF 2 0 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare Mean: SD: IQR: %NA 3.50 1.34 1.75 5.26% 4 TF Mean: SD: IQR: %NA 3.53 0.96 1* 0.00% 2 IF 21 Need for faculty agreement on purp ose and methodology for debriefing sessions Mean: SD: IQR: %NA 3.83 1.30 1.75 5.26% 5 TF Mean: SD: IQR: %NA 3.67 1.22 1* 5.26% 3 IF 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulat ion, (e.g., simulated patients cannot teach the responsibility of patient care) Mean: SD: IQR: %NA 3.67 1.35 1* 5.26% 3 TF Mean: SD: IQR: %NA 3.68 1.06 1.5 0.00% 3 IF

PAGE 239

239 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 23 Need for all stakeholders to create measurable (objective) benchmark s fo r assessing a good clinician so it can be possible to assess student clinical success through the use of simulation Mean: SD: IQR: %NA 3.12 1.51 2 10.53% 2 TF Mean: SD: IQR: %NA 3.16 0.96 1.5 0.00% 2 IF 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs Mean: SD: IQR: %NA 2.60 1.51 1* 21.05% NA TF Mean: SD: IQR: %NA 2.44 1.43 1* 15.79% NA IF 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare education Mean: SD: IQR: %NA 3.56 1.26 1* 5.26% 3 TF Mean: SD: IQR: %NA 3.29 1.43 2 10.53% 3 IF 26 Need to verify if pre written scenarios are appropriate for the average educational level of the allied healthcare student Mean: SD: IQR: %NA 3.61 1.22 1* 5.26% 3 TF Mean: SD: IQR: %NA 3.61 1.22 1* 5.26% 3 IF

PAGE 240

240 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 27 Need to address the constructivist aspects of si mulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions Mean : SD: IQR: %NA 3.13 1.61 2 15.79% 3 TF Mean: SD: IQR: %NA 3.17 1.33 1.75 5.26% 3 IF Comments on Item 27 : not needed, redundant, or not specifically applicable or attributable to the use of simulations in healthcare education speci fically, bu t exist as educational issues in general 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education Mean: SD: IQR: %NA 3.59 1.47 1* 10.53% 4 TF Mean: SD: IQR: %NA 3.42 0.84 1* 0.00% 2 IF 29 Need to develop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher Mean: SD: IQR: %NA 2.81 1.42 1.25 15.79% 2 TF Mean: SD: IQR: %NA 3.11 1.31 1.5 5.26% 2 IF Comments on Item 29 : not needed, redundant, or not specifically applicable or attributable to the use of simulations in healthcare education speci fically, but exist as educational issues in general

PAGE 241

241 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 30 edited Need to address the issues that accreditation, licensing, and cer tification do not always allow for required clinical experience to be substituted with simulated experience when it has been proven to be a sufficient substitute Mean: SD: IQR: %NA 3.06 1.54 2 15.79% 3 TF Mean: SD: IQR: %NA 3.12 1.44 2 10.53% 4 I F Comments on Item 30 : technology is not always adequate for replacing clinical experience ; not needed, redundant, or not specifically applicable or attributable to the use of simulations in healthcare education speci fically, but exist as educationa l issues in general 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum Mean: SD: IQR: %NA 3.25 1.48 1* 10.53% 3 TF Mean: SD: IQR: %NA 3.32 0.95 1* 0.00% 3 IF 32 Need to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system Mean: SD: IQR: %NA 3.72 1.39 1.75 5.26% 5 TF Mean: SD: IQR: %NA 3.44 1.48 2 5.26% 5 IF 33 Need to find ways to address institut ional or accreditation pressures which can hamper the implement ation of flexible methods of training Mean: SD: IQR: %NA 3.07 1.57 0.5* 21.05% 3 TF Mean: SD: IQR: %NA 2.88 1.57 1* 15.79% 2 IF Comments on Item 33 : not needed, redundant, or no t specifically applicable or attributable to the use of simulations in healthcare education specifically, but exist as educational issues in general

PAGE 242

242 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could be nefit the greatest number of students Mean: SD: IQR: %NA 3.42 1.07 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.37 1.26 2 0.00% 3 IF 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas Mean: S D: IQR: %NA 3.24 1.56 2 10.53% 4 TF Mean: SD: IQR: %NA 3.28 1.29 1* 5.26% 3 IF Comments on Item 29 : not needed, redundant, or not specifically applicable or attributable to the use of simulations in healthcare education speci fically, but exi st as educational issues in general 36 Need to establish a validated matrix concerning when to employ traditional teaching methods versus simulation Mean: SD: IQR: %NA 3.32 1.06 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.11 0.94 1* 0.00% 3 IF Collab oration issues 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) Mean: SD: IQR: %NA 3.17 1.41 1.75 5.26% 4 TF Mean: SD: IQR: %NA 3.28 1.52 1* 5.26% 4 IF

PAGE 243

243 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 38 Institutions need to give faculty the ability to network through observation of clinical simulation use in other programs/institutions Mean: SD: IQR: %NA 3.53 1.57 1* 10.53% 5 TF Mean: SD: IQR : %NA 3.74 1.15 2 0.00% 5 IF 39 Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and support simulation based education Mean: SD: IQR: %NA 3.00 1.34 2 5.26% 2 TF M ean: SD: IQR: %NA 3.06 1.41 2 5.26% 2 IF 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations pr oviding de facto regulation Mean: SD: IQR: %NA 2.71 1.43 1* 10.53% 2 TF Mean: SD: IQR: %NA 2.41 1.21 1* 10.53% 2 IF 41 Institution s should work to address the lack of a systematic or coordinated means to identify issues in simulation based a llied healthcare education Mean: SD: IQR: %NA 2.82 1.43 1* 10.53% 3 TF Mean: SD: IQR: %NA 2.50 1.12 1* 5.26% 3 IF

PAGE 244

244 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 42 Institutions should network with other institution to optimize simulation research Mean: SD: IQR: %NA 3.42 1.02 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.35 1.49 1* 10.53% 5 IF 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios Mean: SD: IQR: %NA 3.42 1.17 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.39 1.36 1* 5.26% 5 IF 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources Mean: SD: IQR: %NA 3.56 1.46 1.75 5.26% 4 TF Mean: SD: IQR: %NA 3.28 1.45 2 5.26% 4 IF 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medical centers w hen using the consortium model Mean: SD: IQR: %NA 2.74 1.15 1* 0.00% 3 TF Mean: SD: IQR: %NA 2. 89 1.28 1.75 5.26% 3 IF

PAGE 245

245 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. Tools and simulator technology issues 46 Need to find ways to ensure e ducational issues are considered during the early stages of simulator design (a framework) Mean: SD: IQR: %NA 3.74 0.93 1.5 0.00% 4 TF Mean: SD: IQR : %NA 3.37 0.90 1* 0.00% 4 IF 47 Need to find ways to ensure s imulators are designed with feedback features that are pedagogically sound Mean: SD: IQR: %NA 3.42 0.84 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.39 1.18 1* 5.26% 5 IF 48 Develop way s to mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technology Mean: SD: IQR: %NA 2.68 1.06 1* 0.00% 5 TF Mean: SD: IQR: %NA 2.83 1.25 1* 5.26% 5 IF 49 Need to find a way to ensure that the design, development, integration, and use of simulator technology become s an integrated enterprise with developers, clinicians, and educators working together towards the same goal Mean: SD: IQR: %NA 2.84 1.12 1 0.00% 3 TF Mean: SD: IQR: %NA 2.78 1.21 1* 5.26% 3 IF

PAGE 246

246 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex Mean: SD: IQR: %NA 3.56 1.38 1* 5.26% 5 TF Mean: SD: IQR: %NA 3.72 1.26 1* 5.26% 5 IF Comments Item 50 : likely not needed or unnecessary, or the question is based on a false perception of need, e.g simulations specifically for allied health 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex Mean: SD: IQR: %NA 3.26 1.05 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.47 0.90 1* 0.00% 5 IF 52 Need to address the fact that there is a l ack of simulation technology designed specifically for allied health care cur riculum s Mean: SD: IQR: %NA 3.06 1.29 0.75* 5.26% 3 TF Mean: SD: IQR: %NA 3.06 1.45 1* 10.53% 3 IF Comments Item 52 : likely not needed or unnecessary, or the question is based on a false perception of need, e.g simulations specifically for allied health 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives Mean: SD: IQR: %NA 3.53 0.96 1* 0.00% 2 TF Mean: SD: IQR: %NA 3.37 1.01 1* 0.00% 2 IF

PAGE 247

24 7 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives Mean: SD: IQR: %NA 3.17 1.25 1.75 5.26% 3 TF Mean: SD: IQR: %NA 3.33 1. 17 1* 5.26% 3 IF Comments Item 54 : likely not needed or unnecessary, or the question is based on a false perception of need, e.g simulations specifically for allied health 55 Need to develop guidelines concerning the level of realism needed for huma n tissue and organs Mean: SD: IQR: %NA 3.00 1.34 2 5.26% 3 TF Mean: SD: IQR: %NA 3.24 1.49 1* 10.53% 3 IF Comments Item 54 : L likely not needed or unnecessary, or the question is based on a false perception of need, e.g simulations specifica lly for allied health Faculty and staff issues 56 Need to find ways to ensure that e ducators, simulation technicians, and clinical faculty work together they can have a tendency of not communicating and integrating ideas, needs, methods, and resources M ean: SD: IQR: %NA 3.47 1.56 1* 10.53% 4 TF Mean: SD: IQR: %NA 3.58 1.02 1* 0.00% 4 IF 57 Need to address the fact that s imulation is often viewed as an added burden b y faculty members and is therefore not fully supported Mean: SD: IQR: %NA 3.33 1.21 1* 5.26% 4 TF Mean: SD: IQR: %NA 3.32 0.89 1* 0.00% 4 IF

PAGE 248

248 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 58 Need to address the fact that s imulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in education Mean: SD: IQR: %NA 3.50 1.68 1.25 15.79% 5 TF Mean: SD: IQR: %NA 3.53 1.46 1* 10.53% 5 IF 59 Need to address the fact that f aculty does not have the time to prepare complex simulation scenarios Mean: SD: IQR: %NA 3.31 1.62 1* 10.53% 5 TF Mean: SD: IQR: %NA 3.61 1.39 1* 0.00% 5 IF 60 Need to address the fact that f aculty development needs to include the proper use of simulator technology Mean: SD: IQR: %NA 3.74 0.87 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.47 0.84 1* 0.00% 2 IF 61 Need to addr ess the fact that f aculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology Mean: SD: IQR: %NA 3.72 1.22 1* 5.26% 5 TF Mean: SD: IQR: %NA 3.68 0.95 1* 0.00% 3 IF

PAGE 249

249 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 62 Need to address the fact that f aculty development should include the use of the debriefing sessions; why, when, and how to conduct them Mean: SD: IQR: %NA 3.74 0.87 1* 0.00% 5 TF Mean: SD: IQR: %NA 3.84 0.83 0* 0.00% 5 IF 63 Need t o address the fact that f aculty development should include the integration of simulation into the curriculum which includes an understanding of the different modalities and technologies available Mean: SD: IQR: %NA 3.68 0.89 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.68 0.95 1* 0.00% 4 IF 64 Need to address the fact that often t oo much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop Mean: SD: IQR: %NA 3.24 1.49 2 10.53% 3 TF Mean: SD: IQR: %NA 3.16 1.07 1.5 0.00% 3 IF 65 N e ed to address how best to provide simulation specialist s continuing education opportunities to keep abreast of changes in the field Mean: SD: IQR: %NA 3.53 0.90 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.42 0.9 0 1* 0.00% 3 IF

PAGE 250

250 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 66 Need to create standardized certification opportunities for those teaching with simulation Mean: SD: IQR: %NA 3.58 0.90 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.58 0.96 1* 0.00% 4 IF 67 Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students Mean: SD: IQR: %NA 2.89 1.10 0.75* 5.26% 3 TF Mean: SD: IQR: %NA 2.89 0.94 1* 0.00% 3 IF 68 Need an acceptable continuing education curriculum for faculty and staf f teaching with simulators/simulation Mean: SD: IQR: %NA 3.56 1.26 1* 5.26% 4 TF Mean: SD: IQR: %NA 3.32 1.00 1* 0.00% 4 IF 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios Mean: SD: IQR : %NA 3.24 1.41 1* 10.53% 5 TF Mean: SD: IQR: %NA 3.21 1.03 1* 0.00% 5 IF

PAGE 251

251 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 70 Need to address the fact that an a ging workforce will lead to critical faculty shortages in community and technical colleges Mean: SD: IQR: %NA 3.06 1.54 2 15.79% 4 TF Mean: SD: IQR: %NA 3.11 1.35 2 5.26% 4 IF 71 Need to address the fact that l ow salaries make retention of a qualified workforce difficult Mean: SD: IQR: %NA 2.75 1.60 2.25 15.79% 4 TF Mean: SD: IQR: %NA 2.41 1.38 3 10.53% 4 IF 72 Need to address the fact that a l ack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) Mean: SD: IQR: %NA 3.06 1.15 2 5.26% 4 TF Mean: SD: IQR: %NA 3.06 0. 99 0.75* 5.26% 4 IF 73 Need to address the fact that t here is a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation Mean: SD: IQR: %NA 3.06 1.41 1* 10.53% 5 TF Mean: SD: IQR: %NA 3.13 1.54 1* 15.79% 5 IF

PAGE 252

252 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 74 Need to address the fact that individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own Mean: SD: IQR: %NA 3.17 1.20 1* 5.26% 4 TF Mean: SD: IQR: %NA 3.16 0.69 1* 0.00% 4 IF 75 Need to address the l ack of time and resources to train incumbent faculty mean ing simulators that are purchased go unused or underused Mean: SD: IQR: %NA 3.06 1.15 0.75* 5.26% 3 TF Mean: SD: IQR: %NA 2.94 1.03 1.75 5.2 6% 3 IF 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to achieve a specific teaching method, and if the session should be a team or individual intera ction Mean: SD: IQR: %NA 3.47 0.84 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.53 0.77 1* 0.00% 4 IF Resources 77 Need to show return on investment to decrease pressure to reduce high cost programs Mean: SD: IQR: %NA 3.11 0.99 2 0.00% 2 TF M ean: SD: IQR: %NA 3.11 0.99 2 0.00% 2 IF

PAGE 253

253 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 78 Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive Mean: SD: IQR: %NA 3.24 1.37 0* 5.26% 3 TF Mean: SD: IQR: %NA 2.82 1.35 1* 10 .53% 3 IF 79 edited Need to address the fact that facilities are inadequate to properly run the scenarios Need to address how to best equip and support facilities to properly run the appropriate scenarios Mean: SD: IQR: %NA 3.06 1.29 2 5.26% 5 TF Mean: SD: IQR: %NA 2.94 1.47 2 15.79% 3 IF 80 Need to address the fact that there is a lack of appropriate support equipment such as cameras, crash carts, catheters supplied by the institution Need to understand the appropriate level of support e quipment such as cameras, crash carts, catheters supplied by the institution Mean: SD: IQR: %NA 3.18 1.34 1* 10.53% 4 TF Mean: SD: IQR: %NA 2.81 1.38 2 15.79% 4 IF 81 Need to address the fact a dministrators do not fully fund ongoing maintena nce and training in their annual budgets Mean: SD: IQR: %NA 2.82 1.17 1* 5.26% 3 TF Mean: SD: IQR: %NA 2.71 1.12 1* 10.53% 3 IF

PAGE 254

254 Issue No. Issue Statement (in no particular order) Round 1 Group Response Your Round 1 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasib le or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 82 Need to address the fact that i t is not known what the best mix of clinical and simulation is in order to meet cost effectiveness Mean: SD: IQR: %NA 3.05 0.78 0* 0.00% 4 TF Mean: SD: IQR: %NA 3.06 1.19 0* 5.26% 2 IF 83 Need to find solutions to the fact that i ndividual institutions are hampered in their expansion of simulation based educational opport unities by a lack of resources including funding, available faculty, and space Mean: SD: IQR: %NA 2.94 1.26 1* 5.26% 3 TF Mean: SD: IQR: %NA 3.00 1.39 1.25 15.79% 3 IF 84 edited N eed to find ways to educate s takeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load Mean: SD: IQR: %NA 3.25 1.59 2 5.26% 3 TF Mean: SD: IQR: %NA 3.12 1.32 0* 5. 26% 3 IF Comments on resource items: T here should be a distinction made between NEED and FEASIBILITY Many things are feasible to study, but are not needed, with c omparatively few the opposite. M any of the items above are also very specific to site/i nstitution, or so wide reaching that one wonders who exactly would be doing the research; a consortium of users? Manufacturers of simulations? Both?

PAGE 255

255 APPENDIX H ROUND THREE QUESTIONNAIRE Dear Panelist #, Welcome to the third and final round of this modi fied Delphi study exploring the instructional and technical feasibility of successfully addressing the issues facing the future use of simulator technology and simulation methodology in allied healthcare education and training Thank you for your particip ation and input on the first two rounds All but a dozen items have reached consensus, however, all but five items include comments from Round Two Based on some input I considered adding a ranking concerning whether or not the implementation of each ite m was feasible After much debate I decided that I could not do justice to that issue within the scope of this research; that needs to be researched on its own as that issue, as well as need and political feasibility, are extremely important to discuss an d deserve more time and effort than can be given for this particular study. Attached to this email you will find the third round Delphi questionnaire You can use Adobe Acrobat or Adobe Reader to fill in this form If you are using a MAC you may experien ce problems if you use MAC Preview Your comments and ratings have been compiled and added to the questionnaire Some of the comments have been edited for clarity and consistency One item was edited based on feedback for clarity; this is noted as edite d in the item number and consists of bold text for additional words Items listed with an interquartile range (IQR) of less than 1.2 are noted with an asterisk (*) and have statistically reached consensus; however, you may still modify your rating There were a number of items where some panelists answered NA Reasoning for those responses

PAGE 256

256 are provided when given If an item reaches consensus on an NA response (70%) the item will be removed for the final analysis. For this round you are requested to do the following: 1. Review all the issues listed in the questionnaire. 2. Review the group responses to Round Two items The mean, standard deviation, and IQR are given along with the number of NA responses. 3. Review your Round Two response against the group mean Comments given during Round Two are included Decide if you wish to re rank the item or leave your Round Two ranking Check the box under No change to response if you wish to keep your rating from Round Two If you wish to change, type in a number 1 5 or NA in the Rank column Note : Those listed with an asterisk next to the IQR number have reached consensus, but you may still change your rating. 4. If your ranking is outside of group mean, please explain your reasoning in the last column You may still a dd additional feedback; this will be included in the final analysis. 5. Please return your response by August 31st Clicking on the Submit button on the first page will send it directly back to me or you may just send it as an email attachment. Thank you for your prompt attention I will return a copy of the final analysis to you as soon as it is available Should you have any questions or concerns please feel free to contact me by email at jokkenney@ufl.edu or call me at 210 347 6200. Johanna Kenney

PAGE 257

257 Round 3 Questionnaire I nstructional feasibility (I F ) and technical feasibility (T F ) rated on a scale of 1 to 5: 1= Not at all feasible, 2= Probably unfeasible 3 = Feasible, 4= Very feasible, 5= Extremely f easible Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. Research issues 1 Need to set up a clear research agenda for simulations in allied healthcare education Mean: SD: IQR: %NA 3.65 0.79 1 0.00% 5 TF Mean: SD: IQR: %NA 3.59 0.80 1* 0.00% 4 IF 1TF: Changed from a 5 to a 4 to account for the lack of resources currently available to support quantitative research in this area. Many of these items are feasible If you could ever get consensus of a group of educator to agree within a few years My experience has been that you get more than two to three people together to work on a project, the project can be difficult to complete in a timely manner. "Simulation should not be an ""add on"" nor a side effect of any program Most graduate and some under grad progra ms have established research methods To include simulation to the research agenda would require a folk to actually sit and talk about it however, simulation is still somewha t new and not many understand effective implementation of the many platforms of s imulation to research Implementation has already occurred in some research programs, development might need tweaking to suit the type of research Cost as less prohibitive than in previous years Simulation technology is very manageable with minimal tr aining. 1IF: Instruction obstacles would result from lack of simulation technologies which would probably cover the major faculty concern other than cost Many credentialing bodies have accepted simulation as capable of providing a high level of fidelity in certification of skills Curriculum modification would be to incorporate the simulation Political obstacles can easily be overcome. 2 There is a need to research the impact of simulated learning as it impacts real world skills application Mean: SD: IQR: %NA 3.88 0.70 1* 0.00% 4 TF Mean: SD: IQR: %NA 4.03 0.84 2 0.00% 5 IF 2TF: This may be time consuming but to determine if simulation has impacted the learner could simply r equire reassessment of skills 2IF: From an instructional standpo int there should not be much difference then the technical feasibility.

PAGE 258

258 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning specific procedural skills Mean: SD: IQR: %NA 3.71 0.85 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.79 0.81 1* 5.56% 5 IF 3TF: This would be very much dependent on the specific procedure Traditional methods in the past have used live models (starting IV on each other vs a simulator) Simulation technology is cont inuing to evolve requiring a little R & D 3IF: Many skills assessments have been certified through simulation thus making the instructional feasibility higher. 4 Research is needed to demonstrate if skills acquired through simulation training are retained over time Mean: SD: IQR: %NA 3.79 0.88 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.74 0.83 1* 5.56% 4 IF 4TF: It seems it would be difficult to do a controlled study proving Retaining skills learned and tested on a simulator should not be confused w ith the validity of the skill. 4IF & TF: This is also time consuming but has been implemented in some programs Some R & D could be needed depending on the skill being assessed. 5 Evaluation of simulation needs to be done contextually and over time throu gh long range studies Mean: SD: IQR: %NA 3.59 1.00 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.41 0.87 1* 0.00% 4 IF 5TF : I have undertaken a longitudinal study and apart from some impatience and the risk of anticipating outcomes, there are real b ene fits in gaining perception over time 5TF & IF : This variable is mostly dependent on time and context as well as technological hurdles (how can you assess a procedure if the test subject moves or completes the program) Not sure that it is useful for long ra nge studies as practice and conceivably simulation practices may changeevolve over time more useful to get current data

PAGE 259

259 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual clinical performance Mean: SD: IQR: %NA 3.29 0.77 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.53 0.72 1* 0.00% 4 IF 6TF : I think it is difficult to isolate the impact of discrete variables 6TF & IF : A simple metric or rubric would be needed during fo llow up to assess confidence. 7 Need valid research concerning the impact of simulation on student learning outcomes for allied healthcare skills Mean: SD: IQR: %NA 3.59 0.71 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.65 0.70 1* 0.00% 4 IF 7TF & IF: Technologically how would one asses impact from simulation? Complexity of technology could make this challenging 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes Mean: SD: IQR: %NA 3.29 0 .69 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.29 0.92 1* 0.00% 4 IF 8 TF & IF : This would be very student specific

PAGE 260

260 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as acc urate as hands on assessments Mean: SD: IQR: %NA 3.35 0.86 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.18 0.88 1* 0.00% 4 IF 9TF : I think it is difficult to isolate the impact of discrete variables 9IF : I think this would be beneficial data to gain Mo re simulation technology research could be required 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques Mean: SD: IQR: %NA 3.31 1.11 .25* 5.56% 3 TF Me an: SD: IQR: %NA 2.88 0.85 0* 5.56% 3 IF 10TF : I don't know how much technology would play a part in comparing debriefing techniques It would be difficult to create Comment related to the Item 10 comment from round Item 10 above is not needed T he same metrics should be used to evaluate educational outcomes for any method or technique, with many examples are well established in the literature. My understanding of this survey effort is for the panel to rate the implementation feasibility (T or I) of the items you provide on the survey, not to provide our opinions on if the survey items are needed or would be valuable in the industry. 10IF : I foresee politics as the greatest hurdle here. N1 (Added from Round 1) Conduct research to find best and ragogical strategies concerning when to use simulation with large class sizes ver sus condensed but personalized simulation scenarios (when to use team based, individual scenario, student observation, etc.) Mean: SD: IQR: %NA 3.41 0.95 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.28 0.77 1* 0.00% 3 IF NEWTF : There are a number of variables introduced here

PAGE 261

261 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. Curriculum issues 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process b ased on issues including complexity of the scenarios, objectives, time available for session, and experience level of the participants Mean: SD: IQR: %NA 3.73 1.15 2 0.00% 4 TF Mean: SD: IQR: %NA 3.71 0.92 1* 0.00% 4 IF 11TF : Development of a matrix for validation could be implemented currently in some programs. 11IF : Institution often designates one facilitator for session, based on availability and not skill level experience. Instructional feasibility would be very instructor dependent but is seemingly not difficult to implement. 12 N e ed to develop guidelines concerning the best mix of current clinical and simulation based training for optimal learning outcomes Mean: SD: IQR: %NA 3.55 1.27 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.68 0. 85 1* 0.00% 4 IF 12TF : Guideline development would not require much in terms of technology it would mostly require faculty discussion I don't know how this would be possible with so many different disciplines and contexts 12IF : To study outcome resul ts of clinical vs simulation training requires follow up with clinical participants (usually something IRB approved) thus should not incur majo r political obstacles. 13 Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies Mean: SD: IQR: %NA 4.06 0.90 2 0.00% 5 TF Mean: SD: IQR: %NA 3.94 0.90 2 0.00% 4 IF 13TF & IF : Guidelines w ould be based on current simulation technology (you wouldn't prepare guidelines based on technology yet to be available)

PAGE 262

262 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., b asic anatomy and physiology courses required by all disciplines) Mean: SD: IQR: %NA 3.29 0.77 1* 0.00% 4 TF Mean: SD: IQR: %NA 2.81 0.81 1* 0.00% 3 IF 14TF & IF : This is very institution resource dependent Some programs have more resources to commit to simulation curriculum content then others Some courses may not require simulation as the platform to include in the curriculum. 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across i nstitutions (e.g., standard curriculum for all respiratory therapists) Mean: SD: IQR: %NA 3.12 1.17 2 0.00% 4 TF Mean: SD: IQR: %NA 2.68 0.92 1* 0.00% 3 IF 15TF & IF : This varies from state to state For example, Respiratory Therapist traini ng, scope of practice, etc .. varies from state to state thus making a standardized consensus problematic (more research or development on that would be needed) 16 Need validated method of measurement to verify transferability of simulation performanc e to the clinical setting Mean: SD: IQR: %NA 3.24 0.90 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.29 0.77 1* 0.00% 4 IF 16TF : Studies in this have been conducted with more being done, however this can be started at the individual institution level Some simulation methods have already been validated to serve in place of some clinical experiences (if students come up a little short on procedures, accrediting bodies with allow simulated proced ures to substitute, i e ., a student needs 3 central lines to complete the requirement and have already completed their last clinical commitment could use simulated central line training to be checked off) 16IF : Some political obstacles could be incurred depending on the discipline More development may be needed

PAGE 263

263 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too complex Mean: SD: IQR: %NA 3.47 1.01 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.76 0.83 1* 0.00% 4 IF 17TF : Again not sure this is technically feasi ble huge amount of data required 17IF : There may be two stages to this process: during design phase (where are the potential challenges knowledge, expertise; and during practice is a scenario difficult to bring to a successful closure 17TF & IF : Deve lopment of this variable is in progress and dependent on each individual program We begin our students gently with basics (task trainers) gradually increasing to complex simulated scenarios just prior to graduation. 18 N eed to develop guidelines which e nsure faculty are not overwhelmed with scenarios that are too complex Mean: SD: IQR: %NA 3.41 1.00 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.71 0.85 1* 0.00% 4 IF 18TF : Again not sure this is technically feasible huge amount of data required 18I F : There may be two stages to this process: during design phase (where are the potential challenges knowledge, expertise; and during practice is a scenario difficult to bring to a successful closure 18TF & IF : Scenario development is technology driven 19 Clear learning outcomes must be developed in a way that is easily communicated to and understood by students using simulation Mean: SD: IQR: %NA 4.24 0.75 1* 0.00% 5 TF Mean: SD: IQR: %NA 4.18 0.73 1* 0.00% 4 IF 19TF & IF : As part of any lab a clearly defined outcome should be established prior to use of simulation "At the end of this lab the learner should......"

PAGE 264

264 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare Mean: SD: IQR: %NA 3.53 1.21 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.44 0.93 1* 0.00% 4 IF 20TF : Framework development would not have any tech nological issue that I can think of 20IF : Such a development would could have faculty issue in determining which would be the best framework, it would be curriculum sp ecific requiring potential modification. 21 Need for faculty agreement on purpose and me thodology for debriefing sessions Mean: SD: IQR: %NA 3.86 0.86 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.59 0.80 1* 0.00% 3 IF 21TF : Debriefing methodology at its core requires an audio and video recording that can be played back Such technology e xists in multiple forms and on multiple platforms 21IF : This could possibly be implemented depending on faculty agreement. 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient care) Mean: SD: IQR: %NA 3.75 1.23 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.62 0.93 1* 0.00% 3 IF 22IF : Most people do not know what simulators are much less understand their limitations

PAGE 265

265 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 23 Need for all stakeholders to create measurable (objective) benchmark s for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation Mean: SD: IQR: %NA 3.19 1.17 0.25* 5.56% 4 TF Mean: SD: IQR: %N A 3.12 0.93 0* 0.00% 3 IF 23IF : There may be some instructional obstacles from an instructional standpoint resulting from politics 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs Mean: SD: IQR: %NA 2.46 1.28 1* 11.11% NA TF Mean: SD: IQR: %NA 2.27 1.22 1* 5.56% NA IF 24TF & IF : Not sure this is relevant 24IF : This is highly dependent on school budg ets that have (as of late) been cut so deep simulation equipment and proper instructional with said simulators is prohibitive 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare educ ation Mean: SD: IQR: %NA 3.65 0.79 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.38 1.13 1* 5.56% 3 IF 25TF : Each simulator manufacturer has a bank of purchasable pre programmed scenarios available Current technology would determine is objectives are met or able to be met. I would be little inclined towards off the shelf purchases of scenarios Bespoke ones should meet objectives as part of design process 25IF : This is more of a technology driven question

PAGE 266

266 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 26 Need to verify if pre written scenarios ar e appropriate for the average educational level of the allied healthcare student Mean: SD: IQR: %NA 3.71 0.77 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.50 1.16 1* 5.56% 4 IF 26TF : I would be little inclined towards off the shelf purchases of scenari os Bespoke ones should meet objectives as part of design process 26IF : I cannot see faculty arriving at an agreement on this matter. 26TF & IF : Each simulator manufacturer has a bank of purchasable pre programmed scenarios available Current technology would determine is objectives are met or able to be met. 27 Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued b y the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions Mean: SD: IQR: %NA 3.13 1.39 1* 11.11% 4 TF Mean: SD: IQR: %NA 3.19 1.06 0.25* 5.56% 3 IF 27TF : Feedback has a lot of challenges and not knowing what you don't know offers a particular challenge to effective feedback 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education Mean: SD: IQR: %NA 3.81 1.18 1* 5.56% 4 TF Mean: SD : IQR: %NA 3.41 0.80 1* 0.00% 4 IF 28IF : Some modifications to curriculums may be required to determine if objectives can be met with simulation

PAGE 267

267 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 29 Need to develop validated methods to assess if positive outcomes were a result of an effective simula tion or effective teacher Mean: SD: IQR: %NA 2.85 1.28 1* 11.11% 4 TF Mean: SD: IQR: %NA 2.94 1.03 0.25* 5.56% 3 IF 29TF : I think this is about teacher assessment vital. 29IF : Important as not all teachers can get the best out of simulation 30 Need to address the issues that accreditation, licensing, and certification do not always allow for required clinical experience to be substituted with simulated experience when it has been proven to be a sufficient substitute Mean: SD: IQR: %NA 3. 07 1.31 1* 11.11% 2 TF Mean: SD: IQR: %NA 3.13 1.20 2 5.56% 2 IF 30IF : As time and simulation technology improve, more accrediting bodies will accept simulated experiences for certifications and l icensing Some currently do. 31 Need to develo p a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum Mean: SD: IQR: %NA 3.38 1.13 1* 5.56% 3 TF Mean: SD: IQR: %NA 3.24 0.75 0* 0.00% 3 IF 31IF : May encounter some political obstacles.

PAGE 268

268 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 32 Need to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system Mean: SD: IQR: %NA 3.66 1.12 1* 5.56% 3 TF Mean: SD: IQR: %NA 3.38 1.19 0.25* 5.56% 3 IF 32TF : Current technology exist s and if delivered in a high fidelity context can be easily incorporated into a program curriculum. 32IF : Current limitations are faculty agreement and involvement Some programs do not have the faculty resources to adequately incorporate simulation and i t ends up being a stand alone thing. 33 Need to find ways to address institutional or accreditation pressures which can hamper the implement ation of flexible methods of training Mean: SD: IQR: %NA 3.21 1.50 0.75* 16.67% 4 TF Mean: SD: IQR: %NA 2.75 1.35 0.86* 16.67% NA IF 33TF & IF : Not sure this is relevant 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could benefit the greatest number of students Mean: SD: IQR: %NA 3.65 0.86 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.29 1.05 2 0.00% 4 IF 34TF & IF : This seems like a great outcome

PAGE 269

269 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas Mean: SD: IQR: %NA 3.56 1.22 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.31 1.11 1* 5.56% 3 IF 35TF : We currently incorporate this in our crisis management training. 35IF : Depending on the school and program, could encounter some political difficulties. 36 Need to establish a validated matri x concerning when to employ traditional teaching methods versus simulation Mean: SD: IQR: %NA 3.37 0.78 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.18 0.73 0* 0.00% 3 IF 36TF & IF : This is program specific and may require more development 36IF : Not su re what this might look like: there are some learning outcomes that an experienced educator can look at and think this modali ty will work better than that one It is based on intuition and experience and cannot be reduced to a checklist. Collaboration is sues 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) Mean: SD: IQR: %NA 3.13 1.30 1.25 5.56% 4 TF Mean: SD : IQR: %NA 3.13 1.20 1* 5.56% NA IF 37TF &IF : I feel this is too big and could be to the detriment discreet disciplines 37IF : More research and development needed Some political obstacles.

PAGE 270

270 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 38 Institutions need to give faculty the ability to networ k through observation of clinical simulation use in other programs/institutions Mean: SD: IQR: %NA 3.56 1.27 1* 5.56% 3 TF Mean: SD: IQR: %NA 3.62 0.99 1* 0.00% 3 IF 38IF : I foresee big political obstacles 39 Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and support simulation based education Mean: SD: IQR: %NA 3.19 1.22 1.25 5.56% 3 TF Mean: SD: IQR: %NA 3.25 1.30 1.25 5.56% NA IF 39TF : Some R & D still needed Over time simulation costs will decline as technology becomes commonplace 39IF : Some political obstacles 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and c ollaborations make the adoption of simulation a patchwork of organizations providing de facto regulation Mean: SD: IQR: %NA 2.63 1.21 1* 11.11% 3 TF Mean: SD: IQR: %NA 2.33 0.97 0.5* 11.11% NA IF 40IF : Big political obstacles 41 Institution s should work to address the lack of a systematic or coordinated means to identify issues in simulation based allied healthcare education Mean: SD: IQR: %NA 2.75 1.00 1* 5.56% 3 TF Mean: SD: IQR: %NA 2.56 0.94 1* 5.56% NA IF 41IF : Lots of pol itical issues

PAGE 271

271 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 42 Institutions should network with other institution to optimize simulation research Mean: SD: IQR: %NA 3.41 0.87 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.31 1.17 1* 5.56% NA IF 42TF : Some political issues 42IF : Technology can be sh ared via distance in some cases 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios Mean: SD: IQR: %NA 3.59 0.87 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.31 1.17 1* 5.56% NA IF 43TF : W e have been working in some collaborative areas with other disciplines within the institution. 43IF : A s always, politics 43TF & IF : IP education is the way of the future so simulation fits nicely 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources Mean: SD: IQR: %NA 3.58 1.27 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.20 1.28 2 5.56% NA IF 44IF : This can be a big task I am more enthusiastic about portable models of good p ractice, rather than portable examples Politics many institutions would not be as open to sharing as others

PAGE 272

272 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medic al centers w hen using the consortium model Mean: SD: IQR: %NA 2.71 0.69 1* 0.00% 3 TF Mean: SD: IQR: %NA 2.72 0.93 1* 5.56% NA IF 45TF : Technology could be a small issue in terms of cost and complexity 45IF : Some curriculum modification may be needed Tools and simulator technology issues 46 Need to find ways to ensure e ducational issues are considered during the early stages of simulator design (a framework) Mean: SD: IQR: %NA 3.46 0.71 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.29 0.77 0 0.00% 3 IF 46TF : Issues considered would be influenced by available technology 46IF : Politics 47 Need to find ways to ensure s imulators are designed with feedback features that are pedagogically sound Mean: SD: IQR: %NA 3.32 0.68 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.34 1.06 1* 5.56% 3 IF 47TF : Some R & D may be required

PAGE 273

273 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 48 Develop ways to mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technolo gy Mean: SD: IQR: %NA 2.59 0.87 1* 0.00% 3 TF Mean: SD: IQR: %NA 2.75 1.28 1* 5.56% 3 IF 48TF : More R & D needed 49 Need to find a way to ensure that the design, development, integration, and use of simulator technology become s an integrated enterprise with developers, clinicians, and educators working together towards the same goal Mean: SD: IQR: %NA 2.79 0.81 0.5* 0.00% 3 TF Mean: SD: IQR: %NA 2.84 1.04 0.13* 5.56% 3 IF 50 Proper care needs to be taken to ensure students are n ot overwhelmed with technologies that are too complex Mean: SD: IQR: %NA 3.61 1.37 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.73 1.23 1* 5.56% 4 IF 50TF : ere to replicate patient response Accordingly, complexity arises from Not sure how this could be assessed

PAGE 274

274 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologie s that are too complex Mean: SD: IQR: %NA 3.21 1.02 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.44 0.86 1* 0.00% 3 IF 51TF : on se Accordingly, complexity arises from Not sure how this could be assessed 52 Need to address the fact that there is a l ack of simulation technology designed specifically for allied health care curriculum s Mean: SD: IQR: %NA 2.88 1.16 0* 5.56% 4 TF Mean: SD: IQR: %NA 3.00 1.46 0* 11.11% 4 IF 52TF & IF : simulations spec 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives Mean: SD: IQR: %NA 3.41 0.94 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.29 0.94 1 0.00% 3 IF 53TF & IF : Each simulation session should always be at the highest level of fidelity If you are using a IV task trainer, the learner should be encouraged to go through every step (PPE, Aseptic technique etc...) they would normally go thr ough If they are learning epidural placement, the learner should be in sterile PPE as if they are in the live clinical environment.

PAGE 275

275 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfacto ry) to use based on learning levels and objectives Mean: SD: IQR: %NA 3.05 1.10 2 5.56% 4 TF Mean: SD: IQR: %NA 3.31 0.99 1* 5.56% 4 IF 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs Mean: SD: IQR: %NA 2.88 1.05 0.25* 5.56% 4 TF Mean: SD: IQR: %NA 3.00 1.27 0.5* 11.11% 4 IF 55TF & IF : Unnecessary Faculty and staff issues 56 Need to find ways to ensure that e ducators, simulation technicians, and clinical faculty work together th ey can have a tendency of not communicating and integrating ideas, needs, methods, and resources Mean: SD: IQR: %NA 3.53 1.26 1* 5.56% 4 TF Mean: SD: IQR: %NA 3.53 0.87 1* 0.00% 3 IF 56TF : This can be a challenge and needs to address underlyi ng structural features of inter professional behavior. 56IF : This is not an issue in my department but may be an issue for others As the sole coordinator all simulation activities funnel through me for all campuses (our main one and 3 distance sites that require per the COA an commensurate experience as the main campus students receive)

PAGE 276

276 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 57 Need to address the fact that s imulation is often viewed as an added burden b y faculty members and is therefore not fully supported Mean: SD: IQR: %NA 3.47 0.80 1* 5 .56% 3 TF Mean: SD: IQR: %NA 3.41 0.71 1* 0.00% 3 IF 57TF & IF : Technology advances have eased faculty concerns and as ours is integrated into the curriculum, faculty participation is not l acking at all 58 Need to address the fact that s imulat ion is not supported because there is a lack of strong theoretical and philosophical basis for its use in education Mean: SD: IQR: %NA 3.50 1.48 1* 11.11% 3 TF Mean: SD: IQR: %NA 3.57 1.41 1* 11.11% NA IF 58TF: Most unsupported simulation is budget driven or stems from lack of simulator familiarity with technology. 58IF : Politics 59 Need to address the fact that f aculty does not have the time to prepare complex simulation scenarios Mean: SD: IQR: %NA 3.47 0.87 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.62 0.93 1* 0.00% 3 IF 59TF : Limitation here would be on the level of technology and such activities can be delegated to the tech coordinator 59IF : Primary here is the faculty communication of their objective and the tech to prepare the ses sion to the faculty needs Once complex scenarios are written, they can be saved. 59TF & IF : Resources require time to develop

PAGE 277

277 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 60 Need to address the fact that f aculty development needs to include the proper use of simulator technology Mean: SD: IQR: % NA 3.75 0.75 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.53 0.80 1* 0.00% 3 IF 60IF : Many of the older faculty members still use the see one, do one, teach one model 61 Need to address the fact that f aculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology Mean: SD: IQR: %NA 3.81 1.18 1* 5.56% 3 TF Mean: SD: IQR: %NA 3.65 0.70 1* 0.00% 3 IF 61IF : Politics 62 Need to address the fact that f aculty develo pment should include the use of the debriefing sessions; why, when, and how to conduct them Mean: SD: IQR: %NA 3.71 0.77 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.82 0.73 0* 0.00% 4 IF 62IF : Simulation debriefing is considered terrifying by most par ticipants The faculty would need some minimal training on softer debriefing methods. 63 Need to address the fact that f aculty development should include the integration of simulation into the curriculum which includes an understanding of the different m odalities and technologies available Mean: SD: IQR: %NA 3.71 0.69 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.74 0.56 1* 0.00% 4 IF

PAGE 278

278 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 64 Need to address the fact that often t oo much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop Mean: SD: IQR: %NA 3.21 1.23 1.25 5.56% 3 TF Mean: SD: IQR: %NA 3.24 0.90 1* 0.00% 3 IF 64TF & IF : Strongly agree need assistance 64IF : Shared educational resources (curricula, scenarios) would cut down on development time. Politics 65 Ne e d to address how best to provide simulation specialist s continuing education opportunities to keep abreast of changes in the field Mean: SD: IQR: %NA 3.52 0.79 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.38 0.70 1* 0.00% 3 IF 65TF : In UK this is a resource issue availability of courses, who pays etc. 65IF : More development is required 66 Need to create standardized certification opportunities for those teaching with simulation Mean: SD: IQR: %NA 3.53 0.72 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.59 0.71 1* 0.00% 3 IF 66TF : Institutional autonomy can be a challenge here perhaps regional or national organisations could step in e.g ASPiH in UK 66TF &IF : Politics

PAGE 279

279 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 67 Need to address the fact that there is o ften not enough staff to run the simulation controls and oversee the students Mean: SD: IQR: %NA 2.94 0.97 0* 5.56% 3 TF Mean: SD: IQR: %NA 2.88 0.70 0* 0.00% 3 IF 67IF : Politics and budgets can be the main obstacles 68 Need an acceptable co ntinuing education curriculum for faculty and staff teaching with simulators/simulation Mean: SD: IQR: %NA 3.70 1.39 1* 11.11% 4 TF Mean: SD: IQR: %NA 3.44 1.03 1* 5.56% 3 IF 68TF & IF : Unnecessary over kill 68IF : Politics 69 Need to addre ss the fact that faculty is not properly trained to write pedagogically sound scenarios Mean: SD: IQR: %NA 3.34 1.00 0.63* 5.56% 3 TF Mean: SD: IQR: %NA 3.32 0.68 1* 0.00% 3 IF 69IF : This is program specific and faculty may not understand how to write scenarios.

PAGE 280

280 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 70 Need to address the fact that an a ging workforce will lead to critical faculty shortages in community and technical colleges Mean: SD: IQR: %NA 2.85 1.42 0* 16.67% 3 TF Mean: SD: IQR: %NA 2.93 1.28 1* 5.56% 3 IF 70TF & IF : 70TF : Decrease in college student pool also noted; less population in this age group upcoming in the next few years 70IF : This is a budgetary concern and obstacle 71 Need to address the fact that l ow salaries make retent ion of a qualified workforce difficult Mean: SD: IQR: %NA 2.71 1.30 1* 11.11% 3 TF Mean: SD: IQR: %NA 2.63 1.16 1* 5.56% NA IF 71TF : With a less qualified technical staff, the technology will need to be much more intuitive, reliable and versa tile. 71IF : Politics....ohh yea Lots of politics in this one =) 72 Need to address the fact that a l ack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) Mean: SD: I QR: %NA 2.94 0.97 0.25* 5.56% 3 TF Mean: SD: IQR: %NA 3.00 0.95 0* 5.56% NA IF 72TF : With a less qualified technical staff, the technology will need to be much more intuitive, reliable and versatile. 72IF : more research is needed as well as so me development

PAGE 281

281 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 73 edited Need to address the fact that there can be a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation Mean: SD: IQR: %NA 3.07 1.36 0.50* 11.11% 3 TF Mean: SD: IQR: %NA 3.07 1.50 0.75* 16.67% NA IF 73TF : Lack of knowledge from simulation technologies requires administration and faculty become more aware of simulation capabiliti es. 73 IF : See 73 TF We know at face value, simulation works however more research ma y be needed to validate 74 Need to address the fact that individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own Mean: SD: IQR: %NA 3.14 0.90 0.06* 5.56% 3 TF Mean: SD: IQR: %NA 3.18 0.53 0* 0.00% 3 IF 74IF : Political obstacles 75 Need to address the l ack of time and resources to train incumbent faculty mean ing simulators that are purchased go unused or underused Mean: SD: IQR: %NA 3.06 0.93 0* 5.56% 3 TF Mean: SD: IQR: %NA 2.91 0.90 0.13* 5.56% 3 IF 75 IF : Political obstacles 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to achieve a specific teaching method, and i f the session should be a team or individual interaction Mean: SD: IQR: %NA 3.59 0.62 1* 0.00% 4 TF Mean: SD: IQR: %NA 3.65 0.61 1* 0.00% 4 IF

PAGE 282

282 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. Resources 77 Need to show return on investment to decrease pressure to reduce high cost programs Mean: SD: IQR: %NA 2.96 0.76 0* 0.00% 3 TF Mean: SD: IQR: %NA 3.12 0.78 0* 0.00% 3 IF 77TF : Could require more technological development My response (2) may be based on ignorance of possible value for money assessments 77TF &IF : Proving that simulation improves clinical performance has been next to impossible, beyond reducing central line infections with standardiz ed training 77IF : Political obstacle 78 Need to address the fact that the cost associated with hiring dedicated experts for simula tion programs is prohibitive Mean: SD: IQR: %NA 3.20 1.29 0* 5.56% 3 TF Mean: SD: IQR: %NA 2.76 1.17 1* 11.11% NA IF 78IF : Political obstacles, budget concerns 79 Need to address how to best equip and support facilities to properly run the appropriate scenarios Mean: SD: IQR: %NA 3.18 0.81 0* 0.00% 3 TF Mean: SD: IQR: %NA 2.94 1.03 0.25* 5.56% NA IF 79TF : Cost is manageable however alternative methods can be utilized...i.e., donations Models can vary to meet local resources, ra ther than impose external equipment levels

PAGE 283

283 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 80 Need to understand the appropriate level of support equipment such as cameras, crash carts, catheters supplied by the institution Mean: SD: IQR: %NA 3.12 0.60 0* 0.00% 3 TF Mean: SD: IQR: %NA 2.81 0.93 1* 5.56% NA IF 80TF : Depending on the technological means, complexity can be the challenge 80IF : Political obstacles arise from determining what is actually needed vs wanted 81 Need to address the fact a dministrators do not fully fund ongoing m aintenance and training in their annual budgets Mean: SD: IQR: %NA 2.81 0.93 1* 5.56% 3 TF Mean: SD: IQR: %NA 2.70 1.11 1* 11.11% NA IF 81TF & IF : Simulation technicians should have, as part of their knowledge base, the basics on simulation m aintenance and repair More development is needed 82 Need to address the fact that i t is not known what the best mix of clinical and simulation is in order to meet cost effectiveness Mean: SD: IQR: %NA 3.24 0.66 0* 0.00% 3 TF Mean: SD: IQR: %NA 3.06 0.99 0* 5.56% NA IF 82TF : Not sure that there is a fixed best mix: this depends of availability of resources, staffing levels etc. More development needed 82IF : Some political obstacles

PAGE 284

284 Issue No. Issue Statement (in no particula r order) Round 2 Group Response Your Round 2 Response No change to response Rank 1= Not at all feasible, 2= Somewhat feasible, 3= Feasible, 4= Very feasible, 5= Extremely feasible or NA If no change and outside of group mean, please explain You may also add any feedback to new items here. 83 Need to find solutions to the fact that i ndividual insti tutions are hampered in their expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space Mean: SD: IQR: %NA 2.73 1.23 1* 11.11% 3 TF Mean: SD: IQR: %NA 2.93 1.37 0.75* 16.67% NA IF 84 N eed to find ways to educate stakeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load Mean: SD: IQR: %NA 3.26 1.03 1* 0.00% 3 TF Mean: SD: IQR: %NA 3.19 1.17 0.25* 5.56% NA IF 84TF : I have a problem with the notion of all stakeholders having the same rights and responsibilities and the same claims to havin g these met. This is something u seful to research 84IF : Could make a significant impact

PAGE 285

APPEN DIX I SURVEY RESULTS

PAGE 286

286 Round 1 Results Rat ing technical feasibility (T F ) and instructional feasibility (I F ) on a scale of 1 to 5: 1= Not at all feasible, 2= Probably un feasible, 3= Feasible 4= Very feasible, 5= Extremely f easible Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response Research issues 1 Need to set up a clear research agenda for simulations in allied healthcare education TF Mean: SD: IQR: %NA 3.72 1.26 1.75 5.26% IF Mean: SD: IQR: %NA 3.56 1.26 1* 5.26% 2 There is a need to research the impact of simulated learning as it impacts real world skills application TF Mean: SD: IQR: %NA 3.74 0.87 1* 0.00% IF Mean: SD: IQR: %NA 3.94 1.28 2 5.26% 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning specific procedural skills TF Mean: SD: IQR: %NA 3.63 1.16 1.5 0.00% IF Mean: SD: IQR: %NA 3.67 1.39 1.75 5.26% 4 Researc h is needed to demonstrate if skills acquired through simulation training are retained over time TF Mean: SD: IQR: %NA 3.84 0.90 1.5 0.00% IF Mean: SD: IQR: %NA 3.83 1.34 2 5.26%

PAGE 287

287 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 5 Evaluation of simulation needs to be done contextually and over tim e through long range studies TF Mean: SD: IQR: %NA 3.53 1.02 1* 0.00% IF Mean: SD: IQR: %NA 3.42 0.84 1* 0.00% 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual cl inical performance TF Mean: SD: IQR: %NA 3.22 1.03 1* 5.26% IF Mean: SD: IQR: %NA 3.50 1.00 1* 5.26% 7 Need valid research concerning the impact of simulation on student learning outcomes for allied healthcare skills TF Mean: SD: IQR: %NA 3.56 1. 12 1* 5.26% IF Mean: SD: IQR: %NA 3.72 1.12 1* 5.26% 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes TF Mean: SD: IQR: %NA 3.33 1.07 0.75* 5.26% IF Mean: SD: IQR: %NA 3.44 1.24 1* 5.26% 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments TF Mean: SD: IQR: %NA 3.37 1.12 1.5 0.00% IF Mean: SD: IQR: %NA 3.28 1.24 1.75 5.26%

PAGE 288

288 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques TF Mean: SD: IQR: %NA 3.11 1.31 1.5 5.26% IF Mean: SD: IQR: %NA 2.76 1.26 1* 10.53% Cur riculum issues 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available for session, and experience level of the participants TF Mean: SD: IQR: %NA 3.72 1.58 2.75 5.26% IF Mean: SD: IQR: %NA 3.63 1.07 1* 0.00% 12 N e ed to develop guidelines concerning the best mix of clinical and simulation based training for optimal learning outcomes TF Mean: SD: IQR: %NA 3.67 1.39 2 5.26% IF Mean: SD: IQR: %NA 3.74 0.93 1.5 0.00% 13 Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies TF Mean: SD: IQR: %NA 4.00 1.32 2 5.26% IF Mean: SD: IQR: %NA 3.84 0.96 2 0.00% 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) TF Mean: SD: IQR: %NA 3.39 1.23 1* 5.26% IF Mean: SD: IQR: %NA 2.89 1.05 2 0.00%

PAGE 289

289 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respiratory therapists) TF Mean: SD: IQR: %NA 3.17 1.45 2 5.26% IF Mean: SD: IQR: %NA 2.58 1.22 1* 0.00% 16 Need validated method of measurement to verify transferability of simulation performance to the clinical setting TF Mean: SD: IQR: %NA 3.33 1.38 1* 5.26% IF Mean: SD: IQR: %NA 3.28 1.24 1* 5.26% 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too complex TF Mean: SD: IQR: %NA 3.50 1.34 1* 5.26% IF Mean: SD: IQR: %NA 3.83 1.21 1* 5.26% 18 N eed to develop guidelines which e nsure faculty are not overwhelmed with scenarios that are too complex TF Mean: SD: IQR: %NA 3.47 0.96 1* 0.00% IF Mean: SD: IQR: %NA 3.78 1.22 1* 5.26% 19 Clear learning outcomes must be developed in a way that is easily communicated to and underst ood by students using simulation TF Mean: SD: IQR: %NA 4.11 1.33 1.75 5.26% IF Mean: SD: IQR: %NA 4.11 0.94 1.5 0.00%

PAGE 290

290 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare TF Mean: SD: IQR: %NA 3.50 1.34 1.75 5.26% IF Mean: SD: IQR: %NA 3.53 0.96 1* 0.00% 21 Need for faculty agreement on purpose and methodology for debriefing sessions TF Mean: SD: IQR: %NA 3.83 1.30 1.75 5.26% IF Mean: SD: IQR: %NA 3.67 1.22 1* 5.26% 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient care) TF Mean: SD: IQR: %NA 3.67 1.35 1* 5.26% IF Mean: SD: IQR: %NA 3.68 1.06 1.5 0.00% 23 Need for all stakeholders to create measurable (objective) benchmark s for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation TF Mean: SD: IQR: %NA 3.12 1.51 2 10.53% IF Mean: SD: IQR: %NA 3.16 0.96 1.5 0.00% 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs TF Mean: SD: IQR : %NA 2.60 1.51 1* 21.05% IF Mean: SD: IQR: %NA 2.44 1.43 1* 15.79%

PAGE 291

291 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare education TF Mean: SD: IQR: %NA 3.56 1.26 1* 5.26% IF Mean: SD: IQR: %NA 3.29 1.43 2 10.53% 26 Need to verify if pre written scenarios are appropriate for the average educational level of the allied healthcare student TF Mean: SD: IQR: %NA 3.61 1.22 1* 5.26% IF Mean: SD: IQR: %NA 3.61 1.22 1* 5.2 6% 27 Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions TF Mean: SD: IQR: %NA 3.13 1.61 2 15.79% IF Mean: SD: IQR: %NA 3.17 1.33 1.75 5.26% 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced educati on TF Mean: SD: IQR: %NA 3.59 1.47 1* 10.53% IF Mean: SD: IQR: %NA 3.42 0.84 1* 0.00% 29 Need to develop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher TF Mean: SD: IQR: %NA 2.81 1.4 2 1.25 15.79% IF Mean: SD: IQR: %NA 3.11 1.31 1.5 5.26%

PAGE 292

292 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 30 Need to address the issues that accreditation, licensing, and certification do not always allow for required clinical experience to be substituted with simulated experience TF Mean: SD: IQR: %NA 3.06 1.54 2 15.79% IF Mean: SD: IQR: %NA 3.12 1.44 2 10.53% 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum TF Mean: SD: IQR: %NA 3.25 1.48 1* 10.53% IF Mean: S D: IQR: %NA 3.32 0.95 1* 0.00% 32 Need to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system TF Mean: SD: IQR: %NA 3.72 1.39 1.75 5.26% IF Mean: SD: IQR: %NA 3.44 1.48 2 5.26% 33 Need t o find ways to address institutional or accreditation pressures which can hamper the implement ation of flexible methods of training TF Mean: SD: IQR: %NA 3.07 1.57 0.5* 21.05% IF Mean: SD: IQR: %NA 2.88 1.57 1* 15.79% 34 Need to identify core curr iculum across allied healthcare programs where the use of simulation could benefit the greatest number of students TF Mean: SD: IQR: %NA 3.42 1.07 1* 0.00% IF Mean: SD: IQR: %NA 3.37 1.26 2 0.00%

PAGE 293

293 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 35 Need to identify if and how simulation can be use d to teach students to deal with moral and ethical dilemmas TF Mean: SD: IQR: %NA 3.24 1.56 2 10.53% IF Mean: SD: IQR: %NA 3.28 1.29 1* 5.26% 36 Need to establish a validated matrix concerning when to employ traditional teaching methods versus simu lation TF Mean: SD: IQR: %NA 3.32 1.06 1* 0.00% IF Mean: SD: IQR: %NA 3.11 0.94 1* 0.00% Collaboration issues 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulatio n in allied healthcare education (consortium model) TF Mean: SD: IQR: %NA 3.17 1.41 1.75 5.26% IF Mean: SD: IQR: %NA 3.28 1.52 1* 5.26% 38 Institutions need to give faculty the ability to network through observation of clinical simulation use in ot her programs/institutions TF Mean: SD: IQR: %NA 3.53 1.57 1* 10.53% IF Mean: SD: IQR: %NA 3.74 1.15 2 0.00% 39 Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and sup port simulation based education TF Mean: SD: IQR: %NA 3.00 1.34 2 5.26% IF Mean: SD: IQR: %NA 3.06 1.41 2 5.26%

PAGE 294

294 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and colla borations make the adoption of simulation a patchwork of organizations providing de facto regulation TF Mean: SD: IQR: %NA 2.71 1.43 1* 10.53% IF Mean: SD: IQR: %NA 2.41 1.21 1* 10.53% 41 Institution s should work to address the lack of a systemati c or coordinated means to identify issues in simulation based allied healthcare education TF Mean: SD: IQR: %NA 2.82 1.43 1* 10.53% IF Mean: SD: IQR: %NA 2.50 1.12 1* 5.26% 42 Institutions should network with other institution to optimize simulatio n research TF Mean: SD: IQR: %NA 3.42 1.02 1* 0.00% IF Mean: SD: IQR: %NA 3.35 1.49 1* 10.53% 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios TF Mean: SD: IQR: %NA 3 .42 1.17 1* 0.00% IF Mean: SD: IQR: %NA 3.39 1.36 1* 5.26% 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources TF Mean: SD: IQR: %NA 3.56 1.46 1.75 5.26% IF Mean: SD: IQR: %NA 3.2 8 1.45 2 5.26%

PAGE 295

295 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medical centers w hen using the consortium model TF Mean: SD: IQR: %NA 2.74 1.15 1* 0.00% IF Me an: SD: IQR: %NA 2.89 1.28 1.75 5.26% Tools and simulator technology issues 46 Need to find ways to ensure e ducational issues are considered during the early stages of simulator design (a framework) TF Mean: SD: IQR: %NA 3.74 0.93 1.5 0.00% IF Mea n: SD: IQR: %NA 3.37 0.90 1* 0.00% 47 Need to find ways to ensure s imulators are designed with feedback features that are pedagogically sound TF Mean: SD: IQR: %NA 3.42 0.84 1* 0.00% IF Mean: SD: IQR: %NA 3.39 1.18 1* 5.26% 48 Develop ways to mi tigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technology TF Mean: SD: IQR: %NA 2.68 1.06 1* 0.00% IF Mean: SD: IQR: %NA 2.83 1.25 1* 5.26% 49 Need to fin d a way to ensure that the design, development, integration, and use of simulator technology become s an integrated enterprise with developers, clinicians, and educators working together towards the same goal TF Mean: SD: IQR: %NA 2.84 1.12 1 0.00% IF Mean: SD: IQR: %NA 2.78 1.21 1* 5.26%

PAGE 296

296 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex TF Mean: SD: IQR: %NA 3.56 1.38 1* 5.26% IF Mean: SD: IQR: %NA 3.72 1.26 1* 5.26% 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex TF Mean: SD: IQR: %NA 3.26 1.05 1* 0.00% IF Mean: SD: IQR: %NA 3.47 0.90 1* 0.00% 52 Need to address the fact that there is a l ack of simulation technol ogy designed specifically for allied health care curriculum s TF Mean: SD: IQR: %NA 3.06 1.29 0.75* 5.26% IF Mean: SD: IQR: %NA 3.06 1.45 1* 10.53% 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teachi ng specific skills or learning objectives TF Mean: SD: IQR: %NA 3.53 0.96 1* 0.00% IF Mean: SD: IQR: %NA 3.37 1.01 1* 0.00% 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives TF Mean: SD: IQR: %NA 3.17 1.25 1.75 5.26% IF Mean: SD: IQR: %NA 3.33 1.17 1* 5.26%

PAGE 297

297 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs TF Mean: SD: IQR: %NA 3.00 1.34 2 5.26% IF Mean: SD: IQR: %NA 3.24 1.49 1* 10.53% Faculty and staff issues 56 Need to find ways to ensure that e ducators, simulation technicians, and clinical faculty work together they can have a tendency of not communicating and in tegrating ideas, needs, methods, and resources TF Mean: SD: IQR: %NA 3.47 1.56 1* 10.53% IF Mean: SD: IQR: %NA 3.58 1.02 1* 0.00% 57 Need to address the fact that s imulation is often viewed as an added burden b y faculty members and is therefore not fully supported TF Mean: SD: IQR: %NA 3.33 1.21 1* 5.26% IF Mean: SD: IQR: %NA 3.32 0.89 1* 0.00% 58 Need to address the fact that s imulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in edu cation TF Mean: SD: IQR: %NA 3.50 1.68 1.25 15.79% IF Mean: SD: IQR: %NA 3.53 1.46 1* 10.53% 59 Need to address the fact that f aculty does not have the time to prepare complex simulation scenarios TF Mean: SD: IQR: %NA 3.31 1.62 1* 10.53% IF M ean: SD: IQR: %NA 3.61 1.39 1* 0.00%

PAGE 298

298 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 60 Need to address the fact that f aculty development needs to include the proper use of simulator technology TF Mean: SD: IQR: %NA 3.74 0.87 1* 0.00% IF Mean: SD: IQR: %NA 3.47 0.84 1* 0.00% 61 Need to addres s the fact that f aculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology TF Mean: SD: IQR: %NA 3.72 1.22 1* 5.26% IF Mean: SD: IQR: %NA 3.68 0.95 1* 0.00% 62 N eed to address the fact that f aculty development should include the use of the debriefing sessions; why, when, and how to conduct them TF Mean: SD: IQR: %NA 3.74 0.87 1* 0.00% IF Mean: SD: IQR: %NA 3.84 0.83 0* 0.00% 63 Need to address the fact tha t f aculty development should include the integration of simulation into the curriculum which includes an understanding of the different modalities and technologies available TF Mean: SD: IQR: %NA 3.68 0.89 1* 0.00% IF Mean: SD: IQR: %NA 3.68 0.95 1* 0.00% 64 Need to address the fact that often t oo much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop TF Mean: SD: IQR: %NA 3.24 1.49 2 10.53% IF Mean: SD: IQR: %NA 3.16 1.07 1.5 0.00%

PAGE 299

299 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 65 Ne e d to address how best to provide simulation specialist s continuing education opportunities to keep abreast of changes in the field TF Mean: SD: IQR: %NA 3.53 0.90 1* 0.00% IF Mean: SD: IQR: %NA 3.42 0.90 1* 0.00% 66 Need to create standardized c ertification opportunities for those teaching with simulation TF Mean: SD: IQR: %NA 3.58 0.90 1* 0.00% IF Mean: SD: IQR: %NA 3.58 0.96 1* 0.00% 67 Need to address the fact that there is often not enough staff to run the simulation controls and over see the students TF Mean: SD: IQR: %NA 2.89 1.10 0.75* 5.26% IF Mean: SD: IQR: %NA 2.89 0.94 1* 0.00% 68 Need an acceptable continuing education curriculum for faculty and staff teaching with simulators/simulation TF Mean: SD: IQR: %NA 3.56 1.26 1* 5.26% IF Mean: SD: IQR: %NA 3.32 1.00 1* 0.00% 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios TF Mean: SD: IQR: %NA 3.24 1.41 1* 10.53% IF Mean: SD: IQR: %NA 3.21 1.03 1* 0.00%

PAGE 300

300 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 70 Ne ed to address the fact that an a ging workforce will lead to critical faculty shortages in community and technical colleges TF Mean: SD: IQR: %NA 3.06 1.54 2 15.79% IF Mean: SD: IQR: %NA 3.11 1.35 2 5.26% 71 Need to address the fact that l ow salarie s make retention of a qualified workforce difficult TF Mean: SD: IQR: %NA 2.75 1.60 2.25 15.79% IF Mean: SD: IQR: %NA 2.41 1.38 3 10.53% 72 Need to address the fact that a l ack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) TF Mean: SD: IQR: %NA 3.06 1.15 2 5.26% IF Mean: SD: IQR: %NA 3.06 0.99 0.75* 5.26% 73 Need to address the fact that t here is a lack of support from administration, faculty, and tech nical staff due to concerns about the validity of simulation TF Mean: SD: IQR: %NA 3.06 1.41 1* 10.53% IF Mean: SD: IQR: %NA 3.13 1.54 1* 15.79% 74 Need to address the fact that individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own TF Mean: SD: IQR: %NA 3.17 1.20 1* 5.26% IF Mean: SD: IQR: %NA 3.16 0.69 1* 0.00%

PAGE 301

301 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 75 Need to address the l ack of time and resources to train incumbent faculty mean ing simulators that are purchased go unused or underused TF Mean: SD: IQR: %NA 3.06 1.15 0.75* 5.26% IF Mean: SD: IQR: %NA 2.94 1.03 1.75 5.26% 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective tech nique is to achieve a specific teaching method, and if the session should be a team or individual interaction TF Mean: SD: IQR: %NA 3.47 0.84 1* 0.00% IF Mean: SD: IQR: %NA 3.53 0.77 1* 0.00% Resources 77 Need to show return on investment to decr ease pressure to reduce high cost programs TF Mean: SD: IQR: %NA 3.11 0.99 2 0.00% IF Mean: SD: IQR: %NA 3.11 0.99 2 0.00% 78 Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive TF Mean: SD: IQR: %NA 3.24 1.37 0* 5.26% IF Mean: SD: IQR: %NA 2.82 1.35 1* 10.53% 79 Need to address the fact that f acilities are inadequate to properly run the scenarios TF Mean: SD: IQR: %NA 3.06 1.29 2 5.26% IF Mean: SD: IQR: %NA 2.94 1.47 2 15.79%

PAGE 302

302 Issue No. Issue Statement (in no particular order) Feasibility Round 1 Group Response 80 Need to address the fact that there is a l ack of appropriate support equipment such as cameras, crash carts, catheters supplied by the institution TF Mean: SD: IQR: %NA 3.18 1.34 1* 10.53% IF Mean: SD: IQR: %NA 2.81 1.38 2 15.79% 81 N eed to address the fact a dministrators do not fully fund ongoing maintenance and training in their annual budgets TF Mean: SD: IQR: %NA 2.82 1.17 1* 5.26% IF Mean: SD: IQR: %NA 2.71 1.12 1* 10.53% 82 Need to address the fact that i t is not known wh at the best mix of clinical and simulation is in order to meet cost effectiveness TF Mean: SD: IQR: %NA 3.05 0.78 0* 0.00% IF Mean: SD: IQR: %NA 3.06 1.19 0* 5.26% 83 Need to find solutions to the fact that i ndividual institutions are hampered in t heir expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space TF Mean: SD: IQR: %NA 2.94 1.26 1* 5.26% IF Mean: SD: IQR: %NA 3.00 1.39 1.25 15.79% 84 N eed to educate s takeholders better so they have an understanding of when simulation may be an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load TF Mean: SD: IQR: %NA 3.25 1.59 2 5.26% IF Mean: SD: IQR: %NA 3.12 1.32 0* 5.26%

PAGE 303

303 Round 2 Results Rat ing technical feasibility (T F ) and instructional feasibility (I F ) on a scale of 1 to 5: 1= Not at all feasible, 2= Probably un feasible, 3= Feasible 4= Very feasible, 5= Extremely f easible Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response Research issues 1 Need to set up a clear research agenda for simulations in allied healthcare education TF Mean: SD: IQR: %NA 3.65 0.79 1 0.00% IF Mean: SD: IQR: %NA 3.59 0.80 1* 0.00% 2 There is a need to research the impact of simulated learning as it impacts real world skills application TF Mean: SD: IQR: %NA 3.88 0.70 1* 0.00% IF Mean: SD: IQR: %NA 4.03 0.84 2 0.00% 3 Research is needed which demonstrates that simulat ions are more effective than other teaching methods for learning specific procedural skills TF Mean: SD: IQR: %NA 3.71 0.85 1* 0.00% IF Mean: SD: IQR: %NA 3.79 0.81 1* 5.56% 4 Research is needed to demonstrate if skills acquired through simulation training are retained over time TF Mean: SD: IQR: %NA 3.79 0.88 1* 0.00% IF Mean: SD: IQR: %NA 3.74 0.83 1* 5.56%

PAGE 304

304 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 5 Evaluation of simulation needs to be done contextually and over time through long range studies TF Mean: SD: IQR: %NA 3.59 1.00 1 0.00% IF Mean: SD: IQR: %NA 3.41 0.87 1* 0.00% 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual clinical performance TF Mean: SD: IQR: %NA 3.29 0.77 1* 0.00% IF Mean: SD: IQR: %NA 3.53 0.72 1* 0.00% 7 Need valid research concerning the impact of simulation on student learning outcomes for allied healthcare skills TF Mean: SD: IQR: %NA 3.59 0.71 1* 0.00% IF Mean: SD: IQR: %NA 3.65 0.70 1* 0.00% 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes TF Mean: SD: IQR: %NA 3.29 0.69 1* 0.00% IF Mean: SD: IQR: %NA 3.29 0.92 1* 0.00%

PAGE 305

305 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments TF Mean: SD: IQR: %NA 3.35 0.86 1* 0.00% IF Mean: SD: IQR: %NA 3.18 0.88 1* 0.00% 10 Research needs to be completed to determine common metr ics to compare simulation debriefing sessions to other educational methods and techniques TF Mean: SD: IQR: %NA 3.31 1.11 .25* 5.56% IF Mean: SD: IQR: %NA 2.88 0.85 0* 5.56% N1 (Added from Round 1) Conduct research to find best andragogical strateg ies concerning when to use simulation with large class sizes ver sus condensed but personalized simulation scenarios (when to use team based, individual scenario, student observation, etc.) TF Mean: SD: IQR: %NA 3.41 0.95 1* 0.00% IF Mean: SD: IQR: % NA 3.28 0.77 1* 0.00% Curriculum issues 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available fo r session, and experience level of the participants TF Mean: SD: IQR: %NA 3.73 1.15 2 0.00% IF Mean: SD: IQR: %NA 3.71 0.92 1* 0.00% 12 N e ed to develop guidelines concerning the best mix of current clinical and simulation based training for optima l learning outcomes TF Mean: SD: IQR: %NA 3.55 1.27 1* 5.56% IF Mean: SD: IQR: %NA 3.68 0.85 1* 0.00%

PAGE 306

306 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 13 Faculty and staff need guidelines concerning how simulation should be combined with other teaching strategies TF Mean: SD: IQR: %NA 4.06 0.90 2 0.00% IF Mean: SD: IQR: %NA 3.94 0.90 2 0.00% 14 Need to find consensus concerning curriculum content for simulation across allied health disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) TF Mean: SD: IQR: %NA 3.29 0.77 1* 0.00% IF Mean: SD: IQR: %NA 2.81 0.81 1* 0.00% 15 Need to find consensus concerning curriculum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respirato ry therapists) TF Mean: SD: IQR: %NA 3.12 1.17 2 0.00% IF Mean: SD: IQR: %NA 2.68 0.92 1* 0.00% 16 Need validated method of measurement to verify transferability of simulation performance to the clinical setting TF Mean: SD: IQR: %NA 3.24 0.90 1* 0.00% IF Mean: SD: IQR: %NA 3.29 0.77 1* 0.00% 17 Need to develop guidelines which ensure students are not overwhelmed with scenarios that are too complex TF Mean: SD: IQR: %NA 3.47 1.01 1* 0.00% IF Mean: SD: IQR: %NA 3.76 0.83 1* 0.00%

PAGE 307

307 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 18 N eed to develop guidelines which ensure faculty are not overwhelmed with scenarios that are too complex TF Mean: SD: IQR: %NA 3.41 1.00 1* 0.00% IF Mean: SD: IQR: %NA 3.71 0.85 1* 0.00% 19 Clear learning outcomes must be developed in a way that is e asily communicated to and understood by students using simulation TF Mean: SD: IQR: %NA 4.24 0.75 1* 0.00% IF Mean: SD: IQR: %NA 4.18 0.73 1* 0.00% 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulat ion in healthcare TF Mean: SD: IQR: %NA 3.53 1.21 1* 5.56% IF Mean: SD: IQR: %NA 3.44 0.93 1* 0.00% 21 Need for faculty agreement on purpose and methodology for debriefing sessions TF Mean: SD: IQR: %NA 3.86 0.86 1* 0.00% IF Mean: SD: IQR: % NA 3.59 0.80 1* 0.00% 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient care) TF Mean: SD: IQR: %NA 3.75 1.23 1* 5.56% IF Mean: SD: IQR: %NA 3.62 0.93 1* 0.00%

PAGE 308

308 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 23 Need for all stakeholders to create measurable (objective) benchmark s for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation TF Mean: SD: IQR: %NA 3.19 1.17 0.25* 5.56% IF Mean: SD: IQR: %NA 3.12 0.93 0* 0.00% 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs TF Mean: SD: IQR: %NA 2.46 1.28 1* 11.11% IF Mean: SD: IQR: %NA 2.27 1.22 1* 5.56% 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare education TF Mean: SD: IQR: %NA 3.65 0.79 1* 0.00% IF Mean: SD: IQR: %NA 3.38 1.13 1* 5.56% 26 Need to verify if pre written scenarios are appropriate for the average educational level of the allied healthcare student TF Mean: SD: IQR: %NA 3.71 0.77 1* 0.00% IF Mean: SD: IQR: %NA 3.50 1.16 1* 5.56% 27 Need to address the constructivist aspects of simulated learning where some actions taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions TF Mean: SD: IQR: %NA 3.13 1.39 1* 11.11% IF Mean: SD: IQR: %NA 3.19 1.06 0.25* 5.56%

PAGE 309

309 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education TF Mean: SD: IQR: %NA 3.81 1.18 1* 5.56% IF Mean: SD: IQR: %NA 3.41 0.80 1* 0.00% 29 Need to develop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher TF Me an: SD: IQR: %NA 2.85 1.28 1* 11.11% IF Mean: SD: IQR: %NA 2.94 1.03 0.25* 5.56% 30 Need to address the issues that accreditation, licensing, and certification do not always allow for required clinical experience to be substituted with simulated e xperience when it has been proven to be a sufficient substitute TF Mean: SD: IQR: %NA 3.07 1.31 1* 11.11% IF Mean: SD: IQR: %NA 3.13 1.20 2 5.56% 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the alli ed healthcare curriculum TF Mean: SD: IQR: %NA 3.38 1.13 1* 5.56% IF Mean: SD: IQR: %NA 3.24 0.75 0* 0.00%

PAGE 310

310 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 32 Need to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system TF Mean: SD: IQR: %NA 3.66 1.12 1* 5.56% IF Mean: SD: IQR: %NA 3.38 1.19 0.25* 5.56% 33 Need to find ways to address institutional or accreditation pressures which can hamper the implement ation of flexible methods of training TF Mean: SD: IQR: %NA 3.21 1.50 0.75* 16.67% IF Mean: SD: IQR: %NA 2.75 1.35 0.86* 16.67% 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could benefit the greatest number of students TF Mean: SD: IQR: %NA 3.65 0.86 1* 0.00% IF Mean: SD: IQR: %NA 3.29 1.05 2 0.00% 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas TF Mean: SD: IQR: %NA 3.56 1.22 1* 5.56% IF Mean: SD: IQR: %NA 3.31 1.11 1* 5.56% 36 Need to establis h a validated matrix concerning when to employ traditional teaching methods versus simulation TF Mean: SD: IQR: %NA 3.37 0.78 1* 0.00% IF Mean: SD: IQR: %NA 3.18 0.73 0* 0.00%

PAGE 311

311 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 37 Institutions need to create a network of community based, multi secto r, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) TF Mean: SD: IQR: %NA 3.13 1.30 1.25 5.56% IF Mean: SD: IQR: %NA 3.13 1.20 1* 5.56% 38 Institutions need to give faculty the abilit y to network through observation of clinical simulation use in other programs/institutions TF Mean: SD: IQR: %NA 3.56 1.27 1* 5.56% IF Mean: SD: IQR: %NA 3.62 0.99 1* 0.00% 39 Educational institutions should find ways to collaborate with other hea lth science institutions in the community to help pay for and support simulation based education TF Mean: SD: IQR: %NA 3.19 1.22 1.25 5.56% IF Mean: SD: IQR: %NA 3.25 1.30 1.25 5.56% 40 Need to address the disparate voluntary oversight of education al institutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork of organizations providing de facto regulation TF Mean: SD: IQR: %NA 2.63 1.21 1* 11.11% IF Mean: SD: IQR: %NA 2.33 0.97 0. 5* 11.11% 41 Institution s should work to address the lack of a systematic or coordinated means to identify issues in simulation based allied healthcare education TF Mean: SD: IQR: %NA 2.75 1.00 1* 5.56% IF Mean: SD: IQR: %NA 2.56 0.94 1* 5.56%

PAGE 312

312 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 42 Institutions should network with other institution to optimize simulation research TF Mean: SD: IQR: %NA 3.41 0.87 1* 0.00% IF Mean: SD: IQR: %NA 3.31 1.17 1* 5.56% 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios TF Mean: SD: IQR: %NA 3.59 0.87 1* 0.00% IF Mean: SD: IQR: %NA 3.31 1.17 1* 5.56% 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources TF M ean: SD: IQR: %NA 3.58 1.27 1* 5.56% IF Mean: SD: IQR: %NA 3.20 1.28 2 5.56% 45 Need to find a way to address the inequality which may exists between rural and urban schools due to their proximity to universities and medical centers w hen using the consortium model TF Mean: SD: IQR: %NA 2.71 0.69 1* 0.00% IF Mean: SD: IQR: %NA 2.72 0.93 1* 5.56% Tools and simulator technology issues 46 Need to find ways to ensure e ducational issues are considered during the early stages of simulator design (a framework) TF Mean: SD: IQR: %NA 3.46 0.71 1* 0.00% IF Mean: SD: IQR: %NA 3.29 0.77 0* 0.00%

PAGE 313

313 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 47 Need to find ways to ensure s imulators are designed with feedback features that are pedagogically sound TF Mean: SD: IQR: %NA 3.32 0.68 1* 0.00% IF Mean: SD: IQR: %NA 3.34 1.06 1* 5.56% 48 Develop ways to mitigate the cost of keeping up with the changing pace of the technology which may dissuade some institutions from attempting to implement new technology TF Mean: SD: IQR: %NA 2.59 0.87 1* 0 .00% IF Mean: SD: IQR: %NA 2.75 1.28 1* 5.56% 49 Need to find a way to ensure that the design, development, integration, and use of simulator technology become s an integrated enterprise with developers, clinicians, and educators working together towa rds the same goal TF Mean: SD: IQR: %NA 2.79 0.81 0.5* 0.00% IF Mean: SD: IQR: %NA 2.84 1.04 0.13* 5.56% 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex TF Mean: SD: IQR: %NA 3.61 1.3 7 1* 5.56% IF Mean: SD: IQR: %NA 3.73 1.23 1* 5.56%

PAGE 314

314 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 51 Proper care needs to be taken to ensure faculty are not overwhelmed with technologies that are too complex TF Mean: SD: IQR: %NA 3.21 1.02 1* 0.00% IF Mean: SD: IQR: %NA 3.44 0.86 1* 0.00% 52 Need to address the fact that there is a l ack of simulation technology designed specifically for allied health care curriculum s TF Mean: SD: IQR: %NA 2.88 1.16 0* 5.56% IF Mean: SD: IQR: %NA 3.00 1.46 0* 11.11% 53 Need to develop clear guidelin es concerning the needed level of fidelity of a simulation for teaching specific skills or learning objectives TF Mean: SD: IQR: %NA 3.41 0.94 1* 0.00% IF Mean: SD: IQR: %NA 3.29 0.94 1* 0.00% 54 Need to develop guidelines concerning the appropriat e type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives TF Mean: SD: IQR: %NA 3.05 1.10 2 5.56% IF Mean: SD: IQR: %NA 3.31 0.99 1* 5.56% 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs TF Mean: SD: IQR: %NA 2.88 1.05 0.25* 5.56% IF Mean: SD: IQR: %NA 3.00 1.27 0.5* 11.11%

PAGE 315

315 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response Faculty and staff issues 56 Need to find ways to ensure that e ducators, simulation technicians, and clinical fac ulty work together they can have a tendency of not communicating and integrating ideas, needs, methods, and resources TF Mean: SD: IQR: %NA 3.53 1.26 1* 5.56% IF Mean: SD: IQR: %NA 3.53 0.87 1* 0.00% 57 Need to address the fact that s imulation is often viewed as an added burden b y faculty members and is therefore not fully supported TF Mean: SD: IQR: %NA 3.47 0.80 1* 5.56% IF Mean: SD: IQR: %NA 3.41 0.71 1* 0.00% 58 Need to address the fact that s imulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in education TF Mean: SD: IQR: %NA 3.50 1.48 1* 11.11% IF Mean: SD: IQR: %NA 3.57 1.41 1* 11.11% 59 Need to address the fact that f aculty does not have the time to prepare complex sim ulation scenarios TF Mean: SD: IQR: %NA 3.47 0.87 1* 0.00% IF Mean: SD: IQR: %NA 3.62 0.93 1* 0.00% 60 Need to address the fact that f aculty development needs to include the proper use of simulator technology TF Mean: SD: IQR: %NA 3.75 0.75 1* 0. 00% IF Mean: SD: IQR: %NA 3.53 0.80 1* 0.00%

PAGE 316

316 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 61 Need to address the fact that f aculty development needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology TF Mean: SD: IQR: %NA 3. 81 1.18 1* 5.56% IF Mean: SD: IQR: %NA 3.65 0.70 1* 0.00% 62 Need to address the fact that f aculty development should include the use of the debriefing sessions; why, when, and how to conduct them TF Mean: SD: IQR: %NA 3.71 0.77 1* 0.00% IF Mean : SD: IQR: %NA 3.82 0.73 0* 0.00% 63 Need to address the fact that f aculty development should include the integration of simulation into the curriculum which includes an understanding of the different modalities and technologies available TF Mean: SD: IQR: %NA 3.71 0.69 1* 0.00% IF Mean: SD: IQR: %NA 3.74 0.56 1* 0.00% 64 Need to address the fact that often t oo much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop TF Mean: SD: IQR: %NA 3. 21 1.23 1.25 5.56% IF Mean: SD: IQR: %NA 3.24 0.90 1* 0.00% 65 Ne e d to address how best to provide simulation specialist s continuing education opportunities to keep abreast of changes in the field TF Mean: SD: IQR: %NA 3.52 0.79 1* 0.00% IF Mean : SD: IQR: %NA 3.38 0.70 1* 0.00%

PAGE 317

317 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 66 Need to create standardized certification opportunities for those teaching with simulation TF Mean: SD: IQR: %NA 3.53 0.72 1* 0.00% IF Mean: SD: IQR: %NA 3.59 0.71 1* 0.00% 67 Need to address the fact that th ere is often not enough staff to run the simulation controls and oversee the students TF Mean: SD: IQR: %NA 2.94 0.97 0* 5.56% IF Mean: SD: IQR: %NA 2.88 0.70 0* 0.00% 68 Need an acceptable continuing education curriculum for faculty and staff teac hing with simulators/simulation TF Mean: SD: IQR: %NA 3.70 1.39 1* 11.11% IF Mean: SD: IQR: %NA 3.44 1.03 1* 5.56% 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios TF Mean: SD: IQR: %NA 3.34 1.00 0.63* 5.56% IF Mean: SD: IQR: %NA 3.32 0.68 1* 0.00%

PAGE 318

318 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 70 Need to address the fact that an a ging workforce will lead to critical faculty shortages in community and technical colleges TF Mean: SD: IQR: %NA 2.85 1.42 0* 16.67% IF Mean: SD: IQR : %NA 2.93 1.28 1* 5.56% 71 Need to address the fact that l ow salaries make retention of a qualified workforce difficult TF Mean: SD: IQR: %NA 2.71 1.30 1* 11.11% IF Mean: SD: IQR: %NA 2.63 1.16 1* 5.56% 72 Need to address the fact that a l ack of time for orientation and mentoring of new workers mean new employees are not knowledgeable concerning the technology (knowledge loss) TF Mean: SD: IQR: %NA 2.94 0.97 0.25* 5.56% IF Mean: SD: IQR: %NA 3.00 0.95 0* 5.56% 73 edited Need to address th e fact that there can be a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation TF Mean: SD: IQR: %NA 3.07 1.36 0.50* 11.11% IF Mean: SD: IQR: %NA 3.07 1.50 0.75* 16.67% 74 Need to addre ss the fact that individual faculty often have to take the initiative and be motivated to learn how to use the simulators on their own TF Mean: SD: IQR: %NA 3.14 0.90 0.06* 5.56% IF Mean: SD: IQR: %NA 3.18 0.53 0* 0.00%

PAGE 319

319 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 75 Need to address the l ack of time and resources to train incumbent faculty mean ing simulators that are purchased go unused or underused TF Mean: SD: IQR: %NA 3.06 0.93 0* 5.56% IF Mean: SD: IQR: %NA 2.91 0.90 0.13* 5.56% 76 Need to develop guidelines for debriefing session s which include when the debriefing session is required and what the most effective technique is to achieve a specific teaching method, and if the session should be a team or individual interaction TF Mean: SD: IQR: %NA 3.59 0.62 1* 0.00% IF Mean: SD : IQR: %NA 3.65 0.61 1* 0.00% Resources 77 Need to show return on investment to decrease pressure to reduce high cost programs TF Mean: SD: IQR: %NA 2.96 0.76 0* 0.00% IF Mean: SD: IQR: %NA 3.12 0.78 0* 0.00% 78 Need to address the fact that th e cost associated with hiring dedicated experts for simulation programs is prohibitive TF Mean: SD: IQR: %NA 3.20 1.29 0* 5.56% IF Mean: SD: IQR: %NA 2.76 1.17 1* 11.11% 79 Need to address how to best equip and support facilities to properly run t he appropriate scenarios TF Mean: SD: IQR: %NA 3.18 0.81 0* 0.00% IF Mean: SD: IQR: %NA 2.94 1.03 0.25* 5.56%

PAGE 320

320 Issue No. Issue Stat ement (in no particular order) Feasibility Round 2 Group Response 80 Need to understand the appropriate level of support equipment such as cameras, crash carts, catheters supplied by the institution TF Mean: SD: IQR: %NA 3.12 0.60 0* 0.00% IF Mean: SD: IQR: %NA 2.81 0.93 1* 5.56% 81 Need to address the fact a dministrators do not fully fund ongoing maintenance and training in their annual budgets TF Mean: SD: IQR: %NA 2.81 0.93 1* 5.56% IF Me an: SD: IQR: %NA 2.70 1.11 1* 11.11% 82 Need to address the fact that i t is not known what the best mix of clinical and simulation is in order to meet cost effectiveness TF Mean: SD: IQR: %NA 3.24 0.66 0* 0.00% IF Mean: SD: IQR: %NA 3.06 0.99 0* 5.56% 83 Need to find solutions to the fact that i ndividual institutions are hampered in their expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space TF Mean: SD: IQR: %NA 2.73 1.23 1* 11.11% IF Mean: SD: IQR: %NA 2.93 1.37 0.75* 16.67% 84 N eed to find ways to educate stakeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student learnin g, funding cutbacks, increased student load TF Mean: SD: IQR: %NA 3.26 1.03 1* 0.00% IF Mean: SD: IQR: %NA 3.19 1.17 0.25* 5.56%

PAGE 321

321 Round 3 Results Rat ing technical feasibility (T F ) and instructional feasibility (I F ) on a scale of 1 to 5: 1= Not at a ll feasible, 2= Probably un feasible 3= Feasible 4= Very feasible, 5= Extremely f easible Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response Research issues 1 Need to set up a clear research agenda for simulations in a llied healthcare education TF Mean: SD: IQR: %NA 3.53 0.72 1 0.00% IF Mean: SD: IQR: %NA 3.53 0.81 1* 0.00% 2 There is a need to research the impact of simulated learning as it impacts real world skills application TF Mean: SD: IQR: %NA 3.75 0. 77 1* 0.00% IF Mean: SD: IQR: %NA 4.13 0.72 1 0.00% 3 Research is needed which demonstrates that simulations are more effective than other teaching methods for learning specific procedural skills TF Mean: SD: IQR: %NA 3.58 0.83 1* 0.00% IF Mean : SD: IQR: %NA 3.63 0.72 1* 0.00% 4 Research is needed to demonstrate if skills acquired through simulation training are retained over time TF Mean: SD: IQR: %NA 3.50 0.82 1* 0.00% IF Mean: SD: IQR: %NA 3.50 0.73 1* 0.00%

PAGE 322

322 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 5 Evaluation of simulat ion needs to be done contextually and over time through long range studies TF Mean: SD: IQR: %NA 3.44 0.96 1* 0.00% IF Mean: SD: IQR: %NA 3.38 0.89 1* 0.00% 6 Need for verification or refutation of relationship between increased student confidence as a result of using simulation and actual clinical performance TF Mean: SD: IQR: %NA 3.19 0.75 .25* 0.00% IF Mean: SD: IQR: %NA 3.44 0.73 1* 0.00% 7 Need valid research concerning the impact of simulation on student learning outcomes for allied h ealthcare skills TF Mean: SD: IQR: %NA 3.50 0.63 1* 0.00% IF Mean: SD: IQR: %NA 3.50 0.63 1* 0.00% 8 Need for defensible research to identify methods and technologies that work best for specific learning outcomes TF Mean: SD: IQR: %NA 3.13 0.72 0 0.00% IF Mean: SD: IQR: %NA 3.25 0.93 1* 0.00%

PAGE 323

323 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 9 Research needs to be completed to determine if computer based simulations for skills assessment for certification procedures is as accurate as hands on assessments TF Mean: SD: IQR: %NA 3.19 0.75 0 .25* 0.00% IF Mean: SD: IQR: %NA 3.06 0.68 0.25* 0.00% 10 Research needs to be completed to determine common metrics to compare simulation debriefing sessions to other educational methods and techniques TF Mean: SD: IQR: %NA 3.27 1.12 0* 5. 88 % I F Mean: SD: IQR: %NA 2.73 0.81 0.5* 5.88% N1 (Added from Round 1) Conduct research to find best andragogical strategies concerning when to use simulation with large class sizes ver sus condensed but personalized simulation scenarios (when to use team bas ed, individual scenario, student observation, etc.) TF Mean: SD: IQR: %NA 3.30 0.88 1* 0.00% IF Mean: SD: IQR: %NA 3.17 0.65 0.75 0.00% Curriculum issues 11 Institutions need to develop a validated matrix which clearly defines the role(s) of the facilitator in the debriefing process based on issues including complexity of the scenarios, objectives, time available for session, and experience level of the participants TF Mean: SD: IQR: %NA 3.65 0.70 1* 0.00% IF Mean: SD: IQR: %NA 3.63 0.62 1 0.00% 12 N e ed to develop guidelines concerning the best mix of current clinical and simulation based training for optimal learning outcomes TF Mean: SD: IQR: %NA 3.39 1.10 1* 5.88 % IF Mean: SD: IQR: %NA 3.53 0.62 1 0.00%

PAGE 324

324 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 13 Faculty and staff ne ed guidelines concerning how simulation should be combined with other teaching strategies TF Mean: SD: IQR: %NA 4.00 0.73 0.5* 0.00% IF Mean: SD: IQR: %NA 3.88 0.72 1* 0.00% 14 Need to find consensus concerning curriculum content for simulation acr oss allied health disciplines within an institution (e.g., basic anatomy and physiology courses required by all disciplines) TF Mean: SD: IQR: %NA 3.31 0.48 1 0.00% IF Mean: SD: IQR: %NA 2.67 0.60 1 0.00% 15 Need to find consensus concerning curri culum content for simulation within allied health disciplines across institutions (e.g., standard curriculum for all respiratory therapists) TF Mean: SD: IQR: %NA 2.88 0.81 0.25 0.00% IF Mean: SD: IQR: %NA 2.59 0.61 1 0.00% 16 Need validated meth od of measurement to verify transferability of simulation performance to the clinical setting TF Mean: SD: IQR: %NA 3.25 0.45 0.25 0.00% IF Mean: SD: IQR: %NA 3.25 0.45 0.25 0.00% 17 Need to develop guidelines which ensure students are not overwh elmed with scenarios that are too complex TF Mean: SD: IQR: %NA 3.31 0.95 1* 0.00% IF Mean: SD: IQR: %NA 3.76 0.81 1* 0.00%

PAGE 325

325 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 18 N eed to develop guidelines which ensure faculty are not overwhelmed with scenarios that are too complex TF Mean: SD: IQR : %NA 3.25 0.93 1* 0.00% IF Mean: SD: IQR: %NA 3.56 0.81 1* 0.00% 19 Clear learning outcomes must be developed in a way that is easily communicated to and understood by students using simulation TF Mean: SD: IQR: %NA 4.25 0.58 1* 0.00% IF Mean: SD: IQR: %NA 4.19 0.54 0.25 0.00% 20 Develop a dedicated framework and supporting taxonomy for instructional design concerning simulation in healthcare TF Mean: SD: IQR: %NA 3.4 3 1.05 1* 5.88 % IF Mean: SD: IQR: %NA 3.3 4 0.70 1* 0.00% 21 Need f or faculty agreement on purpose and methodology for debriefing sessions TF Mean: SD: IQR: %NA 3.79 0.75 1* 0.00% IF Mean: SD: IQR: %NA 3.49 0.51 1* 0.00% 22 Need for all stakeholders to have an understanding of the limitations of what can be taught through simulation, (e.g., simulated patients cannot teach the responsibility of patient care) TF Mean: SD: IQR: %NA 3.60 1.09 1* 5.88% IF Mean: SD: IQR: %NA 3.41 0.61 1* 0.00%

PAGE 326

326 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 23 Need for all stakeholders to create measurable (objective) benchmar k s for assessing a good clinician so it can be possible to assess student clinical success through the use of simulation TF Mean: SD: IQR: %NA 2 .9 3 0.86 0 5. 88 % IF Mean: SD: IQR: %NA 2.94 0.57 0* 0.00% 24 Need to address the lack of a shared K 12 framework for healthcare curriculum which ensures students are prepared for and can easily enter state and local postsecondary education programs TF Mean: SD: IQR: %NA 2.35 1.27 0.5* 11.76 % IF Mean: SD: IQR: %NA 2.29 1.15 0* 5.88% 25 Need to verify if commercially available scenarios address tasks and objectives at the proper level for allied healthcare education TF Mean: SD: IQR: %NA 3.44 0.63 1* 0.00% IF Mean: SD: IQR: %NA 3.20 1.03 1* 5.88% 26 Need to verify if pre written scenarios are a ppropriate for the average educational level of the allied healthcare student TF Mean: SD: IQR: %NA 3.50 0.63 1* 0.00% IF Mean: SD: IQR: %NA 3.37 1.03 0.75* 5.88% 27 Need to address the constructivist aspects of simulated learning where some action s taken within the simulation may seem logical to the learner based on their personal experience but not valued by the teacher and therefore not given appropriate time and attention during feedback/debriefing sessions TF Mean: SD: IQR: %NA 3.07 1.30 0* 1 1.76% IF Mean: SD: IQR: %NA 3.07 0.89 0* 5.88%

PAGE 327

327 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 28 Need to develop consensus concerning which learning objectives are appropriate for simulation enhanced education TF Mean: SD: IQR: %NA 3.67 1.15 1* 5.88% IF Mean: SD: IQR: %NA 3.31 0.70 1 0.00 % 29 Need to develop validated methods to assess if positive outcomes were a result of an effective simulation or effective teacher TF Mean: SD: IQR: %NA 2.63 1.13 1* 11.76% IF Mean: SD: IQR: %NA 2.73 0.89 1* 5.88% 30 Need to address the issues t hat accreditation, licensing, and certification do not always allow for required clinical experience to be substituted with simulated experience when it has been proven to be a sufficient substitute TF Mean: SD: IQR: %NA 3.07 1.09 1.5 5.88% IF Mean: S D: IQR: %NA 3.13 1.20 2 5.56% 31 Need to develop a clear understanding of when to introduce simulation supported IPSE into the allied healthcare curriculum TF Mean: SD: IQR: %NA 3.14 1.13 0* 11.76% IF Mean: SD: IQR: %NA 3.06 0.25 0 0.00%

PAGE 328

328 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 32 Ne ed to address the issue that simulators are often seen as stand alone objects, not as a part of an integrated system TF Mean: SD: IQR: %NA 3.53 0.62 1* 0.00% IF Mean: SD: IQR: %NA 3.27 0.77 0.06 0.00% 33 Need to find ways to address institutional o r accreditation pressures which can hamper the implement ation of flexible methods of training TF Mean: SD: IQR: %NA 3.07 1.25 0* 11.76% IF Mean: SD: IQR: %NA 2.61 1.13 1* 11.76% 34 Need to identify core curriculum across allied healthcare programs where the use of simulation could benefit the greatest number of students TF Mean: SD: IQR: %NA 3.56 0. 73 1* 0.00% IF Mean: SD: IQR: %NA 3.31 0.79 1* 0.00% 35 Need to identify if and how simulation can be used to teach students to deal with moral and ethical dilemmas TF Mean: SD: IQR: %NA 3.50 1.18 1* 5.88% IF Mean: SD: IQR: %NA 3.20 1.03 1 5.88% 36 Need to establish a validated matrix concerning when to employ traditional teaching methods versus simulation TF Mean: SD: IQR: %NA 3.3 2 0.57 0.06 0.00% IF Mean: SD: IQR: %NA 3.06 0.44 0* 0.00%

PAGE 329

329 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 37 Institutions need to create a network of community based, multi sector, multi disciplinary collaborations for the support of simulation in allied healthcare education (consortium model) TF Mean : SD: IQR: %NA 3.07 1.09 0.5* 5.88% IF Mean: SD: IQR: %NA 3.07 1.09 0.5* 5.88% 38 Institutions need to give faculty the ability to network through observation of clinical simulation use in other programs/institutions TF Mean: SD: IQR: %NA 3.50 1 .13 1* 5.88% IF Mean: SD: IQR: %NA 3.47 0.81 1* 0.00% 39 Educational institutions should find ways to collaborate with other health science institutions in the community to help pay for and support simulation based education TF Mean: SD: IQR: %NA 3 .13 1.06 0* 5.88% IF Mean: SD: IQR: %NA 3.00 0.98 0* 5.88% 40 Need to address the disparate voluntary oversight of educational institutions in the form of accrediting bodies, societies, and collaborations make the adoption of simulation a patchwork o f organizations providing de facto regulation TF Mean: SD: IQR: %NA 2.43 0.86 1* 5.88% IF Mean: SD: IQR: %NA 2.27 0.81 0* 5.88% 41 Institution s should work to address the lack of a systematic or coordinated means to identify issues in simulation b ased allied healthcare education TF Mean: SD: IQR: %NA 2.47 0.79 1* 5.88% IF Mean: SD: IQR: %NA 2.33 0.75 1* 5.88%

PAGE 330

330 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 42 Institutions should network with other institution to optimize simulation research TF Mean: SD: IQR: %NA 3.31 0.70 1* 0.00% I F Mean: SD: IQR: %NA 3.20 1.03 0 5.88 % 43 Educational institutions should work to find ways to schedule inter professional simulation based education (IPSE ) scenarios TF Mean: SD: IQR: %NA 3.56 0.81 1* 0.00% IF Mean: SD: IQR: %NA 3.40 1.11 0.5 5. 88 % 44 Need for institutions using or thinking of using simulation to develop a shared database or portal for resources TF Mean: SD: IQR: %NA 3.35 1.02 1* 5.88 % IF Mean: SD: IQR: %NA 2 .95 1.07 1.63 5.88% 45 Need to find a way to address the ine quality which may exists between rural and urban schools due to their proximity to universities and medical centers w hen using the consortium model TF Mean: SD: IQR: %NA 2.71 0.69 1* 0.00% IF Mean: SD: IQR: %NA 2.57 0.88 1* 5.88% Tools and simulat or technology issues 46 Need to find ways to ensure e ducational issues are considered during the early stages of simulator design (a framework) TF Mean: SD: IQR: %NA 3.36 0.48 1* 0.00% IF Mean: SD: IQR: %NA 3.19 0.40 0* 0.00%

PAGE 331

331 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 47 Need to find ways to ensure s imulators are designed with feedback features that are pedagogically sound TF Mean: SD: IQR: %NA 3.34 0.60 0.63 0.00% IF Mean: SD: IQR: %NA 3.36 1.03 0.70 5.88% 48 Develop ways to mitigate the cost of keeping up with the changing pace o f the technology which may dissuade some institutions from attempting to implement new technology TF Mean: SD: IQR: %NA 2.55 0.81 1* 0.00% IF Mean: SD: IQR: %NA 2.59 1.09 1* 5.56% 49 Need to find a way to ensure that the design, development, integr ation, and use of simulator technology become s an integrated enterprise with developers, clinicians, and educators working together towards the same goal TF Mean: SD: IQR: %NA 2.66 0.47 1 0.00% IF Mean: SD: IQR: %NA 2.70 0.81 0.75* 5.88% 50 Proper care needs to be taken to ensure students are not overwhelmed with technologies that are too complex TF Mean: SD: IQR: %NA 3.58 1.20 1.00 5.88% IF Mean: SD: IQR: %NA 3.58 1.20 1.00 5.88%

PAGE 332

332 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 51 Proper care needs to be taken to ensure faculty are not o verwhelmed with technologies that are too complex TF Mean: SD: IQR: %NA 3.09 0.93 0.63 0.00% IF Mean: SD: IQR: %NA 3.34 0.79 0.63 0.00% 52 Need to address the fact that there is a l ack of simulation technology designed specifically for allied hea lth care curriculum s TF Mean: SD: IQR: %NA 2.73 0.96 0.50* 5.88% IF Mean: SD: IQR: %NA 3.00 1.31 0* 11.76% 53 Need to develop clear guidelines concerning the needed level of fidelity of a simulation for teaching specific skills or learning objective s TF Mean: SD: IQR: %NA 3.44 0.73 1* 0.00% IF Mean: SD: IQR: %NA 3.30 0.69 0.85 0.00% 54 Need to develop guidelines concerning the appropriate type of human simulator interface (e.g., visual, haptic, olfactory) to use based on learning levels and objectives TF Mean: SD: IQR: %NA 3.05 1.01 0.38* 5.88% IF Mean: SD: IQR: %NA 3.27 0.93 0.50* 5.88% 55 Need to develop guidelines concerning the level of realism needed for human tissue and organs TF Mean: SD: IQR: %NA 2.73 1.03 1* 5.88% IF Mea n: SD: IQR: %NA 3.00 1.26 0* 11.76%

PAGE 333

333 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response Faculty and staff issues 56 Need to find ways to ensure that e ducators, simulation technicians, and clinical faculty work together they can have a tendency of not communicating and integrating ideas, needs, methods, and resources TF Mean: SD: IQR: %NA 3.37 1.09 1* 5.88% IF Mean: SD: IQR: %NA 3.44 0.51 1* 0.00% 57 Need to address the fact that s imulation is often viewed as an added burden b y faculty members and is therefore not fully supported TF Mean: SD: IQ R: %NA 3.31 0.48 1* 0.00% IF Mean: SD: IQR: %NA 3.38 0.50 1* 0.00% 58 Need to address the fact that s imulation is not supported because there is a lack of strong theoretical and philosophical basis for its use in education TF Mean: SD: IQR: %NA 3. 39 1.40 0.88* 11.76% IF Mean: SD: IQR: %NA 3.46 1.32 1* 11.76% 59 Need to address the fact that f aculty does not have the time to prepare complex simulation scenarios TF Mean: SD: IQR: %NA 3.38 0.72 0.25 0.00% IF Mean: SD: IQR: %NA 3.53 0.81 1* 0.00% 60 Need to address the fact that f aculty development needs to include the proper use of simulator technology TF Mean: SD: IQR: %NA 3.55 0.62 1* 0.00% IF Mean: SD: IQR: %NA 3.31 0.60 1* 0.00%

PAGE 334

334 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 61 Need to address the fact that f aculty develo pment needs to include an understanding of simulation methodology and the underlying learning theories that support the methodology TF Mean: SD: IQR: %NA 3.67 1.15 1* 5.88% IF Mean: SD: IQR: %NA 3.50 0.52 1* 0.00% 62 Need to address the fact that f aculty development should include the use of the debriefing sessions; why, when, and how to conduct them TF Mean: SD: IQR: %NA 3.69 0.70 1* 0.00% IF Mean: SD: IQR: %NA 3.81 0.54 0 .25 0.00% 63 Need to address the fact that f aculty development shoul d include the integration of simulation into the curriculum which includes an understanding of the different modalities and technologies available TF Mean: SD: IQR: %NA 3.69 0.48 1* 0.00% IF Mean: SD: IQR: %NA 3.72 0.45 0.63 0.00% 64 Need to addre ss the fact that often t oo much time is needed to develop simulation programs; faculty and staff do not have the time to properly develop TF Mean: SD: IQR: %NA 3.36 1.21 1* 5.88% IF Mean: SD: IQR: %NA 3.38 0.81 1* 0.00% 65 Ne e d to address how best to provide simulation specialist s continuing education opportunities to keep abreast of changes in the field TF Mean: SD: IQR: %NA 3.43 0.72 1* 0.00% IF Mean: SD: IQR: %NA 3.28 0.58 1* 0.00%

PAGE 335

335 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 66 Need to create standardized certification opportunities for those teaching with simulation TF Mean: SD: IQR: %NA 3.44 0.63 1* 0.00% IF Mean: SD: IQR: %NA 3.50 0.63 1* 0.00% 67 Need to address the fact that there is often not enough staff to run the simulation controls and oversee the students TF Mean: SD: IQR: %NA 2.73 0.81 0.50* 5.88% IF Mean: SD: IQR: %NA 2.69 0.48 1 0.00% 68 Need an acceptable continuing education curriculum for faculty and staff teaching with simulators/simulation TF Mean: SD: IQR: %NA 3.61 1.36 1* 11.76% IF Mean: SD: IQR: %NA 3.33 0.96 1* 5.88% 69 Need to address the fact that faculty is not properly trained to write pedagogically sound scenarios TF Mean: SD: IQR: %NA 3.37 1.03 0.75* 5.88% IF Mean: SD: IQR: %NA 3.34 0.70 1* 0.00%

PAGE 336

336 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 70 Need to address the fact that an a ging workforce will lead to critical faculty shortages in community and technical colleges TF Mean: SD: IQR: %NA 2.77 1.34 1* 17.65% IF Mean: SD: IQR: %NA 2.73 1.03 1* 5.88% 71 Need to address the fact that l ow salaries make retention of a qualified workforce difficult TF Mean: SD: IQR: %NA 2.60 0.96 1* 5.88% IF Mean: SD: IQR: %NA 2.50 0.94 1* 5.88% 72 Need to address the fact that a l ack of time for orientation and mentoring of new workers mean new employees are not knowledgeable c oncerning the technology (knowledge loss) TF Mean: SD: IQR: %NA 2.80 0.96 1* 5.88% IF Mean: SD: IQR: %NA 2.93 0.93 0* 5.88% 73 edited Need to address the fact that there can be a lack of support from administration, faculty, and technical staff due to concerns about the validity of simulation TF Mean: SD: IQR: %NA 3.25 0.68 0.25* 0.00% IF Mean: SD: IQR: %NA 3.27 1.06 0.50* 5.88% 74 Need to address the fact that individual faculty often have to take the initiative and be motivated to learn ho w to use the simulators on their own TF Mean: SD: IQR: %NA 3.15 0.86 0* 5.88% IF Mean: SD: IQR: %NA 3.06 0.44 0* 0.00%

PAGE 337

337 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 75 Need to address the l ack of time and resources to train incumbent faculty mean ing simulators that are purchased go unused or underused TF Mean: SD: IQR: %NA 3.13 0.50 0* 5.56% IF Mean: SD: IQR: %NA 2.84 0.51 0.13* 0.00% 76 Need to develop guidelines for debriefing sessions which include when the debriefing session is required and what the most effective technique is to a chieve a specific teaching method, and if the session should be a team or individual interaction TF Mean: SD: IQR: %NA 3.50 0.52 1* 0.00% IF Mean: SD: IQR: %NA 3.63 0.50 1* 0.00% Resources 77 Need to show return on investment to decrease pressure to reduce high cost programs TF Mean: SD: IQR: %NA 2.81 0.54 0.25 0.00% IF Mean: SD: IQR: %NA 3.06 0.57 0* 0.00% 78 Need to address the fact that the cost associated with hiring dedicated experts for simulation programs is prohibitive TF Mean: SD : IQR: %NA 3.00 1.09 0.00 5.88% IF Mean: SD: IQR: %NA 2.60 1.00 1.00 11.76% 79 Need to address how to best equip and support facilities to properly run the appropriate scenarios TF Mean: SD: IQR: %NA 3.06 0.68 0* 0.00% IF Mean: SD: IQR: %NA 2.73 0.81 0.50 5.88%

PAGE 338

338 Issue No. Issue Statement (in no particular order) Feasibility Round 3 Group Response 80 Need to understand the appropriate level of support equipment such as cameras, crash carts, catheters supplied by the institution TF Mean: SD: IQR: %NA 3.05 0.55 0* 0.00% IF Mean: SD: IQR: %NA 2.73 0.89 1* 5.88 % 81 Need to address the fact a dministrators do not fully fund ongoing maintenance and training in their annual budgets TF Mean: SD: IQR: %NA 2.81 0.66 1.00 0.00% IF Mean: SD: IQR: %NA 2.70 0.96 1* 5.88 % 82 Need to address the fact that i t is not known what the best mix of clinical and simulation is in order to meet cost effectiveness TF Mean: SD: IQR: %NA 3.13 0.50 0* 0.00% IF Mean: SD: IQR: %NA 2.93 0.86 0* 5.88% 83 Need to find solutions to the fact that i ndividual institutions are hampered in thei r expansion of simulation based educational opportunities by a lack of resources including funding, available faculty, and space TF Mean: SD: IQR: %NA 2.57 1.06 1* 11.76% IF Mean: SD: IQR: %NA 2.69 1.22 1* 17.65% 84 N eed to find ways to educate st akeholders better so they have an understanding if and when simulation is an appropriate solution for dealing with institutional issues such as student learning, funding cutbacks, increased student load TF Mean: SD: IQR: %NA 3.16 0.81 0.63 0.00% IF M ean: SD: IQR: %NA 3.07 1.09 0* 5.88%

PAGE 339

339 Table I 1. Comments f rom r ound 3 Item Comments 1TF & IF Regardless of challenges (which I recognize) this still is a priority. In UK, there are a number of bodies that are encouraging a degree of collaboration, if only in identifying a research agenda. (P 1373) 1 TF I don't see a real problem in the UK. I think it is very feasible here. (P 1387) 2 IF I think so long as the instructors properly understand the link between technical and non technical skills it should be entirely practicable (P 1387) 4 TF surely the sustainability will erode over time whether using sim or not depending on opportunity to maintain recency in the skill? (P 1387) Does it really matter as we can't/don't validate that traditional non simulati on trg is retained over time either. (P 1414) The question deals only with research on skill retention, not with the validity of the skill or its transference to real patients, Retention has been studied thoroughly in other areas and should transfer to sim ulation fairly easily. (P 1428) 5 TF This is an essential ingredient and one which will be responsive to developments that bring theoretical and practical aspects of the curriculum closer together. (P 1373) 5 TF & IF taken a longitudinal study and apart from some impatience and the risk of anticipating outcomes, 6 TF time and context as we ll as technological hurdles (how can you assess a procedure IF conceivably simulation practices may change/ evolve over time more use I continue to have reservations about the design that would permit this but would like to see proposals. (P 1373) confidence is a function of the debrief skills of the instructor as well as success or otherwise. So long as the experience has been realistic it should be achievable (P 1387) 6 IF Not sure organizations are willing to finance this level of assessment. (P 1414) 8 TF preferred learning styles of different students should be taken into account (P 1387) 8 TF & IF I believe this is too student dependent to be easily accomplished. (P 1428) 9 TF I continue to have reservations about the design that would permit this but would like to see proposals. Also not convinced that computer based simulations will have the same f idelity. (P 1373) 9 IF Learning value on the computer is worth looking into. (P 1414)

PAGE 340

340 Table I 1 Continued Item Comments 10 TF video debriefing would be highly beneficial, not always available in other situations (P 1387) The technology needed to compar e debriefing sessions to other educational methods is fairly well developed, unlike the actual debriefing content. (P 1428) N1TF The number of variables and time required makes this difficult (P 1471) 11TF Is doable but each faculty member has a differen t level of teaching perception which influences the validated matrix's use. (P 1414) Developing a validated instructor role matrix should be done with currently available technology, not by relying on possible future developments. (P 1428) 11 IF There is a n increasing body of literature (and I think, the will) to develop a more robust debrief procedure and maybe my 5 is an aspiration rather than a reality. (P 1373) 12 TF require much in terms of te chnology it would mostly require faculty 13 TF I have been a long term simulation trainer and see no problem here. I have run multiple train the trainer courses (P 1387) 13 IF Subject to reservations expressed below. (P 1373) [previous comment: Guidelines would be based on current simulation technology (you wouldn't prepare guidelines based on technology yet to be available)] 14 TF Perhaps dev eloping less expensive, therefore more easily adopted, technologies would aid in reaching a consensus. (P 1428) 15 TF example, Respiratory Therapist training, scope of practice, etc. ... vari es from state to state thus making a standardized consensus problematic (more research or development on that would be Perhaps developing less expensive, therefore more easily adopted, technologies would aid in reaching a consensus. (P 14 28) 15 IF Not sure this is a "need". Might be nice to have but not needed. (P 1367) 17 TF not sure this is technically feasible Having more choice among technologies available for a scenario would make it easier to fine tune the complexity. (P 1428) 17 IF I think this is essential and should result from careful mapping of curriculum elements (theory and practice) (P 1373) 18 TF Having more choice among technologies available for a scenario would make it easier to fine tune the complexity. (P 1428) 18 IF I think this is essential and should result from careful mapping of curriculum elements (theory and practice) (P 1373)

PAGE 341

341 Table I 1 Continued Item Comments 19 TF can not see a problem (P 1387) 21 IF there is substantial evidence for how to perform debriefing successfully from other industries like aviation. However in medical matters there can be a reluctance to adopt successful practice from elsewhere. (P 1387) 23 TF politically challenging, feasible IF stakeholders val ue creation of consistent benchmarks (P 1471) I think this is a challenge and I will look forward to seeing what this would look like (P. 1373) Actually thought this might be easier than it probably would be. (P 1388) 24TF & IF Relevancy? (P 1419) 25TF N ot convinced that commercial concerns are the best drivers (P. 1373) 33TF If less expensive, more versatile technology was developed, implementation would be easier. (P 1428) 33TF & IF Relevancy? (P 1419) 34TF & IF Our faculty has a multitude of Allied health professions so I see this as really useful (P 1419) 35TF There is no lack of technology that is preventing this from being completed. (P 1428) 36TF Question deals with establishing a matrix for current technology, not a matrix for potential new de velopments. (P 1428) 37 IF 37 TF & IF political barriers sharing resources is threatening to many faculty (P 1471) is too big and could be to the 38 TF Watching current technology in action does not require the development of new technology. (P 1428) 39 TF This is a political question, not a technological one. (P 1428) 39 TF & IF political obstacles (P 1471) 40 IF Politics will be big. (P 1388) 41 IF Lack of political consensus will influence choices. (P 1388) 43 TF & IF 44IF i ndividuals at the same institution are not always willing to share making institution sharing difficult (P 1471) agree with both comments below (P 1373) [ Previous Comments: This can be a big task. I am more enthusiastic about portable models of good pract ice, rather than portable examples; Politics many institutions would not be as open to sharing as others] 45 TF cost is the challenge for many rural schools (P 1471) Costs will influence balance. (P 1388) 46TF There could be political issues with a coll aboration between users and

PAGE 342

342 Table I 1 Continued Item Comments developers. (P 1428) 50TF not sure which technologies are being referred to here. Technological savvy seems not to be a problem for many students (P 1373) 51TF sim leaders are often technology leaders also (P 1373) not sure ho w this would be assessed (P 1419) 52 TF & IF question is based on a false perception of need, e.g. simulations 55 TF & IF 5 6 TF An issue that needs to be openly addressed. (P 1373) 56 TF & IF Political and other hurdles. (P 1428) 59 TF complexity of technology requires time to understand (P 1471) 59 IF time is a barrier (P 1471) Lack of staffing ($) has a big impact. (P 1388) 6 3 IF Possible (P 1388) 64 IF Efforts need to be made to ensure administrators understand the complexities of curriculum design and evaluation (P 1373) 64TF & IF This is based on our experience [Score 5] (P 1419) Efforts need to be made to ensure administra tors understand the complexities of curriculum design and evaluation (P 1373) 68 TF & IF Overkill (P 1419) 69 TF & IF Disagree that this is a fact (P 1419) 70 IF Age is not the barrier, budget is limiting full time employment options (P 1471) 70 TF & IF Not a priority (P 1419) 71 IF lack of benefits (especially healthcare) is the most common reason for attrition at my institution (P 1471) 75 TF chicken and egg situation that needs organisational good will to address. (P 1373) 78 IF Big budget issues. (P 1388) 79 TF limited resources can be a challenge. (P 1373) 80 IF need versus want within a limited budget is a challenge (P 1471) 84 TF & IF based on our experience this would be helpful (P 1419)

PAGE 343

343 LIST OF REFERENCES Adamson, K. (2010). Integrating human patient simulation into associate degree nursing curricula: Faculty experiences, barriers, and facilitators. Clinical Simulation in Nursing 6(3), e75 e81. Alexander, D. R. (2008). A modified Delphi study of future crises and required leader competencies (Doctor al dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 304299225). Alinier, G. (2007). A typology of educationally focused medical simulation tools. Medical Teacher, 29 (8), 243 250. doi: 10.1080/01421590701551185 Allied Hea lth Center of Excellence. (2012). Allied health core curriculum project Retrieved from http://www2.yvcc.edu/coe/core.html American Dental Education Association (ADEA). (2012). Explore healthcareers.org: Allied health professions overview Retrieved from http://explorehealthcareers.org/en/Field/1/Allied_Health_Profession Anastakis, D. J., Regehr, G., Reznick, R. K., Cusima no, M., Murnaghan, J., Brown, M., & Hutchison, C. (1999). Assessment of technical skills transfer from the bench training model to the human model. The American Journal of Surgery, 177 (2), 167 170. Annells, M., DeRoche, M., Koch, T., Lewin, G., & Lucke, J. (2005). A Delphi study of district nursing research priorities in Australia Applied Nursing Research, 18 (1), 36 43. Arena, R. A., Goldberg, L. R., Ingersoll, C. D., Larsen, D. S., & Shelledy, D. (2011). Research in the allied health professions: Why fund it? A Report of the ASAHP Research Committee. Journal of Allied Health, 40 (3), 161 166. Association of Schools of Allied Health Professions (ASAHP). (2012). Allied health professionals. Retrieved from htt p://www.asahp.org/definition.htm Bck Pettersson, S., Hermansson, E., Sernert, N., & Bjrkelund, C. (2008). Research priorities in nursing a Delphi study among Swedish nurses. Journal of Clinical Nursing, 17 (16), 2221 2231. Bailie, J. L. (2011). Effective online instructional competencies as perceived by online university faculty and students: A sequel study. Journal of Online Learning and Teaching, 7 (1), 82 89. Retrieved from http://jolt.merlot .org/vol7no1/bailie_0311.pdf

PAGE 344

344 Barab, S. A., MaKinster, J G. & Scheckler, R. (2004). Designing system dualities: Characterizing an online professional development community. In S. A. Barab, R. Kling & James H. Gray (Eds.), Designing for virtual communities in the service of learning (pp. 53 90). Cambridge, UK: Cambridge University Press. Baxter, P., & Norman, G. (2011). Self assessment or self deception ? A lack of assessment and performance. Journal of Advanced Nurs ing, 67 (11), 2406 2413. Beatty, P. C. & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71 (2), 287 311. Bell, P. F., Daly, J., & Chang, E. M. (2008). A study of the educational and research priori ties of registered nurses in rural Australia. Journal of Advanced Nursing, 25 (4), 794 800. Health Affairs, 21 (3), 80 90. doi: 10.1377/hlthaff.21.3.80 Blackstock, F. C., & Jull, G. A. (2007). High fidelity patient simulation in physiotherapy education. Australian Journal of Physiotherapy, 53 (1), 3. Blair, S. & Uhl, N.P. (1993). Using the Delphi method to improve the curriculum. Canadian Journal of Higher Education, 23 (3), 107 128. B londy, L. C. (2007). Evaluation and application of andragogical assumptions to the adult online learning environment. Journal of Interactive Online Learning, 6 (2), 116 130. Bond, S., & Bond, J. (1982). A Delphi survey of clinical nursing research prioritie s. Journal of Advanced Nursing, 7 (6), 565 575. Bonnel, W. E. & Smith, K. V. (2010). Teaching technologies in nursing and the health professions: Beyond simulation and online learning New York, NY: Springer Publishing Company, LLC Boyce, W., Gowland, C., R ussell, D., & Goldsmith, C. (1993). Consensus methodology in the development and content validation of a gross motor performance measure. Physiotherapy Canada, 45 94 94. Bradley, P. (2006). The history of simulation in medical education and possible futur e directions. Medical Education, 40 (3), 254 262. Bradley, P., & Postlethwaite, K. (2003a). Setting up a clinical skills learning facility. Medical Education, 37 (s1), 6 13. Bradley, P., & Postlethwaite, K. (2003b). Simulation in clinical learning. Medical E ducation, 37 (s1), 1 5. doi: 10.1046/j.1365 2923.37.s1.1.x

PAGE 345

345 Brancheau, J. C., & Wetherbe, J. C. (1987). Key issues in information systems management. MIS Quarterly, 11 (1), 23 45. Brill, J. M., Bishop, M. J., & Walker, A. E. (2006). The competencies and chara cteristics required of an effective project manager: A web based Delphi study Educational Technology Research and Development, 54 (2), 115 140. Brookfield, S. (1984). The contribution of Eduard Lindeman to the development of theory and philosophy in adult education. Adult Education Quarterly, 34 (4), 185 196. Brookfield, S. D. (1992). Developing criteria for formal theory building in adult education. Adult Education Quarterly, 42 (2), 79 93. Broomfield, D., & Humphris, G. M. (2001). Using the Delphi technique to identify the cancer education requirements of general practitioners. Medical Education, 35 (10), 928 937. Bruhn, J. G., & Philips, B. U. (1985). The influence of technology on the future of allied health professionals. Journal of Allied Health, 14 (3), 2 89 295. Buelow, J. R., Rathsack, C., Downs, D., Jorgensen, K., Karges, J. R., & Nelson, D. (2008). Building interdisciplinary teamwork among allied health students through live clinical case simulations. Journal of Allied Health, 37 (2), e109 e123. Burgess, C. C. (2007). Developing a collaborative regional nursing simulation hospital. Teaching and Learning in Nursing, 2 (2), 53 57. Burnett, J., Kumar, S., Grimmer, K. (2005). Development of a generic critical appraisal tool by consensus: Presentation of first round Delphi survey results. The Internet Journal of Allied Health Sciences and Practice, 3 (1), 1 18. Retrieved from http://ijahsp.nova.edu/articles/vol3num1/burnett.pdf Buxbaum, P. (201 2). Medical simulation training. Military Medical & Veterans Affairs Forum, 16 (8): 21 23. Cabaniss, K. (2001). Counseling and computer technology in the new millennium: An Internet Delphi study (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No.304728569). Cantrell, M. J., & Deloney, L. A. (2007). Integration of standardized patients into simulation. Anesthesiology Clinics, 25 (2), 377 383. Cavanagh, S. J. (1990). Educational aspects of cardiopulmonary resuscitation ( CPR) training. Intensive Care Nurse, 6 (1), 38 44.

PAGE 346

346 Childress, R. M., Jeffries, P. R., & Dixon, C. F. (2007). Using collaboration to enhance the effectiveness of simulated learning in nursing education. In P. R. Jefferies (Ed.), Simulation in nursing educati on: From conceptualization to evaluation (pp. 123 135). New York, NY: National League of Nursing. Chiniara, G., Cole, G., Brisbin, K., Huffman, D., Cragg, B., Lamacchia, M., & Norman, D. (2012). Simulation in healthcare: A taxonomy and a conceptual framewo rk for instructional design and media selection. Medical Teacher, (0), e1 e16. doi:10.3109/0142159X.2012.733451 Clayton, M. J. (1997). Delphi: A technique to harness expert opinion for critical decision making tasks in education. Educational Psychology: An International Journal of Experimental Educational Psychology, 17 (4), 373 386. doi : 10.1080/0144341970170401 Clevin, L. & Grantcharov, T. P. (2008). Does box model training improve surgical dexterity and economy of movement during virtual reality laparosco py? A randomized trial. Acta obstetricia et gynecologica Scandinavica 87 (1), 99 103 Cohen, M. Z., Harle, M., Woll, A. M., Despa, S., & Munsell, M. F. (2004). Delphi study of nursing research priorities. Oncology Nursing Forum, 31 (5), 1011 1017. Colliver, J. A. (2002). Educational theory and medical education practice: A cautionary note for medical school faculty. Academic Medicine, 77 (12), 1217 1220. Commission on Accreditation of Allied Health Education Programs Profession (CAAHEP). (2009). Profession de scription & certification information Retrieved from http://www.caahep.org/Content.aspx?ID=19 Cooper, J. B. & Taqueti, V. R. (2008). A brief history of the development of mannequin simulators for cl inical education and training. Postgraduate Medical Journal, 84 (997), 563 570. doi:10.1136/qshc.2004.009886 Cooper, R. A. (2001). Healthcare workforce for the twenty first century: The impact of nonphysician clinicians. Annual Review of Medicine, 52 51 61 Cougar, J. D. (1988). Key human resources issues in IS in the 1990s. Views of IS executives versus human resource executives. Information and Management, 14 (4), 161 174. Crisp, J., Pelletier, D., Duffield, C., Adams, A., & Nagy, S. (1997). The Delphi met hod? Nursing Research, 46 (2), 116 118. Curtin, M. M., & Dupuis, M. D. (2008). Development of human patient simulation programs: Achieving big results with a small budget. Journal of Nursing Education, 47 (11), 522 523.

PAGE 347

347 Cusick, A., McIntosh, D., & Santiago, L. (2004). New graduate therapists in acute care hospitals: Priorities, problems and strategies for departmental action. Australian Occupational Therapy Journal, 51 (4), 174 184. Custer, R. L., Scarcella, J. A., Stewart, B. R. (1999). The modified Delphi te chnique A rotational modification. Journal of Vocational and Technical Education, 15 (2), 50 58. Dajani, J. S., Sincoff, M. Z., & Talley, W. K. (1979). Stability and agreement criteria for the termination of Delphi studies. Technological Forecasting and S ocial Change, 13 (1), 83 90. Dalkey, N. C. (1969b). The Delphi method: An experimental study of group opinion Santa Monica, CA: The RAND Corporation (RM 5888 PR). Dalkey, N.C. (1969a). An experimental study of group opinion: The Delphi method. Futures, 1 (5 ), pp. 408 426. Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9 (3), 458 467. Daniels, L., & Ascougi, A. (1999). Developing a strategy for cancer nursing research: identifying pr iorities. European Journal of Oncology Nursing, 3 (3), 161 169. Davenport, J. & Davenport, J. A. (1985). A chronology and analysis of the andragogy debate. Adult Education Quarterly, 35 (3), 152 159. Dawson, S. L. (2002). A critical approach to medical simul ation Bulletin of the American College of Surgeons, 87 (11), 12 18. Delahaye, B. L., Limerick, D. C., & Hearn, G. N. (1994). The relationship between andragogical and pedagogical orientation and the implications for adult learning. Adult Education Quarterl y, 44 (4), 187 200. Dietrich, H. & Ehrlenspiel, F. (2010). Cognitive interviewing: A qualitative tool for improving questionnaires in sport science. Measurement in Physical Education and Exercise Science, 14 (1), 51 60. Delbecq, A. L., Van de Ven, A. H., & G ustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes Glenview, IL: Scott, Foresman. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education New York, NY: The Free P ress.

PAGE 348

348 Duffy and D. H. Jonassen (Eds.), Constructivism and the technology of instruction: a conversation (pp. 91 98). Hillsdale, NJ: Lawrence Erlbaum Associates. Donohoe, H., Stellefson, M., & Tennant, B. (2012). Advantages and limitations of the e Delphi technique: Implications for health education researchers. American Journal of Health Education, 43 (1), 38 46. Drennan, J. (2003). Cognitive interviewing: Verbal data in th e design and pretesting of questionnaires. Journal of Advanced Nursing, 42 (1), 57 63. Duffield, C. (1993). The Delphi technique: A comparison of results obtained using two expert panels. International Journal of Nursing Studies, 30 (3), 227 237. doi.org/10 .1016/0020 7489(93)90033 Q. Educational Framework Initiative, Office of Naval Research. (2014). In Federal Medical Simulation and Training Consortium (FMSTC) Educational Framework Initiative (EFI) Retrieved from http://www.cse.ucla.edu/focusareas/military/medical.php Elder, O. C., Jr., & Nick, T. G. (1997). Moving toward a core curriculum in schools of allied health professions: Knowledge and skills considered important by department chairs in four disciplines. Journal of Allied Health, 26 (2), 51 56. Fanning, R. M., & Gaba, D. M. (2007). The role of debriefing in simulation based learning. Simulation in Healthcare, 2 (2), 115 125. Feinstein, A. H., & Cannon, H. M. (2002). Constructs of simulation evaluation. Simulation & Gaming, 33 (4), 425 440. Ferdowsian, H. (2011). Human and animal research guidelines: Aligning ethical constructs with new scientific developments. Bioethics, 25 (8), 472 478. Fitzgerald, S. H. (November 08, 2006). Allied health shortage a recruiting challenge, St Louis Medical News Retrieved from http://saintlouismedicalnews.com/content/allied health shortage recruiting cha llenge Fleuren, M. Wiefferink, K. & Paulssen, T. (2004). Determinants of innovation within healthcare organizations. International Journal of Quality in Healthcare, 16 (2): 107 123. doi: 10.1093/intqhc/mzh030 Flexner, A. (1910). Medical education in the Un ited States and Canada: A report to the Carnegie Foundation for the Advancement of Teaching (No. 4). Carnegie Foundation for the Advancement of Teaching.

PAGE 349

349 Fountain, R. A. & Alfred, D. ( 2009 ). Student satisfaction with high fidelity simulation: Does it corre late with learning styles ? Nursing Education Perspectives 30 (2), 96 98 Gaba, D. M. (2004a). A brief history of mannequin based simulation & application. In W. F. Dunn (Ed.), Simulators in critical care and beyond (pp. 7 14). Des Plaines, IL: Society of Critical Care Medicine. Gaba, D. M. (2004b). The future vision of simulation in healthcare. Quality and Safety in Healthcare 13 i2 i10. doi: 10.1136/qshc.2004.009878 Gaba, D. M. (2006). What's in a name? A mannequin by any other name would work as well. S imulation in Healthcare, 1 (2), 64 65. Gaba, D. M., Howard, S. K., Fish, K. J., Smith, B. E., & Sowb, Y.A. (2001). Simulation based training in anaesthesia crisis resource management (ACRM): A decade of experience. Simulation Gaming, 32 (2), 175 193. Gaba, D M., & Raemer, D. (2007). The tide is turning: organizational structures to embed simulation in the fabric of healthcare. Simulation in Healthcare, 2 (1), 1 3. Gabard, D. L., Lowe, D. L., & Chang, J. W. (2012). Current and Future Instructional Methods and Influencing Factors in Anatomy Instruction in Physical Therapy and Medical Schools in the U.S. Journal of Allied Health, 41 (2), 53 62. Garden, A. (2008). Research in simulation. In R. H. Riley (Ed.), Manual of simulation in healthcare (pp. 227 238). New Yo rk, NY: Oxford University Press. Gibbs, T., Durning, S., & Van Der Vleuten, C. (2011). Theories in medical education: Towards creating a union between educational practice and research traditions. Medical Teacher, 33 (3), 183 187. Gillett, B., Peckler, B., Sinert, R., Onkst, C., Nabors, S., Issley, S., ... & Arquilla, B. (2008). Simulation in a disaster drill: Comparison of high fidelity simulators versus trained actors. Academic Emergency Medicine, 15 (11), 1144 1151. Girzadas Jr, D. V., Antonis, M. S., Zert h, H., Lambert, M., Clay, L., Bose, S., & Harwood, R. (2009). Hybrid simulation combining a high fidelity scenario with a pelvic ultrasound task trainer enhances the training and evaluation of endovaginal ultrasound skills. Academic Emergency Medicine, 16 ( 5), 429 435. Glass, R. I. (1997). Telling good numbers from bad ones. IEEE Software, 14 (4), 15 19. Gordon, S., Mendenhall, P., & O'Connor, B. B. (2012 ). Beyond the checklist: What else health care can learn from aviation teamwork and safety Cornell Univ ersity Press.

PAGE 350

350 Gough, S., Hellaby, M., Jones, N., & MacKinnon, R. (2012). A review of undergraduate interprofessional simulation based education (IPSE). Collegian: Journal of the Royal College of Nursing Australia, 19 (3), 153 170. doi.org.lp.hscl.ufl.edu/10 .1016/j.colegn.2012.04.004 Gracht, H. von der (2012). Consensus measurement in Delphi studies: Review and implications for future quality assurance. Technological Forecasting & Social Change, 79 1525 1536. Greatorex, J., & Dexter, T. (2000). An accessibl e analytical approach for investigating what happens between the rounds of a Delphi study. Journal of Advanced Nursing, 32 (4), 1016 1024. Greenberg, R. (2004). Technology enhance simulation: Looking ahead to 2020. In G. K Loyd, C. L. Lake, & Greenberg, R. (Eds.), Practical healthcare simulations (pp. 575 580). Philadelphia, PA: Elsevier Mosby. Grober, E. D., Hamstra, S. J., Wanzel, K. R., Reznick, R. K., Matsumoto, E. D., Sidhu, R. S., & Jarvi, K. A. (2004). The educational impact of bench model fidelity o n the acquisition of technical skill: the use of clinically relevant outcome measures. Annals of Surgery, 240 (2), 374 381. Grol, R., & Grimshaw, J. (2003). From best evidence to best practice: effective implementation of change in patients' care. Lancet, 3 62 (9391), 1225 1230. Grundy, M., & Ghazi, F. (2009). Research priorities in haemato oncology nursing: Results of a literature review and a Delphi study. European Journal of Oncology Nursing, 13 (4), 235. Guillaume, A. (2007). A typology of educationally foc used medical simulation tools. Medical Teacher, 29 (8), e243 e250. Retrieved from http://informahealthcare.com/doi/full/10.1080/01421590701551185 Hall, E. (2009). The Delphi primer: Doing real world or academic research using a mixed method approach. [Kindle Edition] In C. Lentz (Ed.), The refractive thinker: An anthology of learning volume II: Research methodology Retrieved from Amazon.com Hall, R. E., Plant, J. R., Bands, C J., Wall, A. R., Kang, J., & Hall, C. A. (2005). Human patient simulation is effective for teaching paramedic students endotracheal intubation. Academic Emergency Medicine, 12 (9), 850 855. Haluck, R. S., Satava, R. M., Fried, G., Lake, C., Ritter, E. M., Sachdeva, A. K., ... Wilks, D. (2007). Establishing a simulation center for surgical skills: What to do and how to do it. Surgical Endoscopy, 21 (7), 1223 1232. doi: 10.1007/s00464 006 9150 3

PAGE 351

351 Hamilton, R. (2005). Medical education for the future. Journal of the Royal Society of Medicine, 98 (5), 208 209 Hammoud, M. M., Nuthalapaty, F. S., Goepfert, A. R., Casey, P. M, Emmons, S, Espey, of simulators in surgical training. Amer ican Journal of Obstetrics & Gynecology, 199 (4), 338 343. Hanafin, S. (2004). Review of literature on the Delphi Technique Retrieved from http:/ /www.childrensdatabase.ie/documents/publications/Delphi_Technique_A_Li terature_Review.pdf Hasson, F., & Keeney, S. (2011). Enhancing rigour in the Delphi technique research. Technological Forecasting and Social Change, 78 (9), 1695 1704. Hasson, F., Keene y, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32 (4), 1008 1015. Hayden, J. (2010). Use of simulation in nursing education: National survey results. Journal of Nursing Regulation, 1 (3), 52 57. Health Professions Network. (2010). For students Retrieved from http://www.healthpronet.org/students/index.html Healthcare Association of New York State. (2012). A changing, growing healthc are sector: Demand continues for more nurses and allied healthcare professionals: 2012 nursing and allied healthcare professionals workforce survey report Rensselaer, NY. Retrieved from http://www.hanys.org/workforce/docs/2012 05 10_workforce_allied_2012_survey_report.pdf Hendee, W. R. (1971). A collaborative program in allied health training. Journal of Medical Education, 46 (8), 658 665. Henrichs, B., R ule, A., Grady, M., & Ellis, W. (2002). Nurse anesthesia students' perceptions of the anesthesia patient simulator: A qualitative study. AANA Journal, 70 (3), 219 225. Holden, M. C. & Wedman, J. F. (1993). Future issues of computer mediated communication: T he results of a Delphi study. Educational Technology Research and Development, 41 (4), 5 24. doi: 10.1007/BF02297509 Holey, E., Feeley, J., Dixon, J., & Whittaker, V. (2007). An exploration of the use of simple statistics to measure consensus and stability in Delphi studies. BMC Medical Research Methodology, 7 (1), 52.

PAGE 352

352 Holzinger, A., Kickmeier Rust, M. D., Wassertheuer, S., & Hessinger, M. (2009). Learning performance with interactive simulations in medical education: Lessons learned from results of learning complex physiological models with the HAEMOdynamics SIMulator. Computers & Education, 52 292 301. doi:10.1016/j.compedu.2008.08.008 Hovancsek, M. T. (2007). Using simulation in nursing education. In P. R. Jefferies (Ed.), Simulation in nursing education: From conceptualization to evaluation (pp. 1 20). New York, NY: National League of Nursing. Hsu, C. & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12 (10), 1 8. Huang, G. C., Sacks, H., Passiment, M. (2012). Characteristics of simulation activities at North American medical schools and teaching hospitals: An AAMC SSH ASPE AACN collaboration. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 7 (6), 329 333. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century Washington, DC: Committee on Quality of Health Care in America. Issenberg, S.B., McGaghie, W.C., Hart, I. R., Mayer, J. W., Felner, J. M., Petrusa, E.R. training and assessment. JAMA: The Journal of the American Medical Association, 282 (9), 861 866. Issenberg, S. B., Ringsted, C., stergaard, D., & Dieckmann, P. (2011). Setting a research agenda for simulation based healthcare education: A synthesis of the outcome from an Utstein style meeting. Simulation in Healthcare, 6 (3), 155 167. Jansen, D. A., Johnson, N., Larson, G., Berry, C ., & Brenner, G. H. (2009). Nursing faculty perceptions of obstacles to utilizing manikin based simulations and proposed solutions. Clinical Simulation in Nursing, 5 (1), e9 e16. Retrieved from http://www.sciencedirect.com/science/article/pii/S1876139908000455 Jeffries, P. R. (2005). A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nursing Education Perspectives, 26 (2 ), 96 103. Jeffries, P. R. (2006). Designing simulations for nursing education. Annual Review of Nursing Education, 4 161 177. Jeffries, P. R. (2007). Preface. In P. R. Jefferies (Ed.), Simulation in nursing education: From conceptualization to evaluation (pp. xi xii). New York, NY: National League of Nursing.

PAGE 353

353 Jeffries, P. R., & Battin, J. (2012). Developing successful health care education simulation centers New York, NY: Springer Publishing. Jenkins, D. A., & Smith, T. E. (1994). Applying Delphi methodo logy in family therapy research. Contemporary Family Therapy, 16 (5), 411 430. Jonassen, D., Davidson, M., Collins, M., Campbell, J. & Haag, B. B. (1995). Constructivism and computer mediated communication in distance education. American Journal of Distance Education, 9 (2), 7 26. Jones, A., & Sheppard, L. (2007). Can human patient simulators be used in physiotherapy education? Internet Journal of Allied Health Sciences and Practice, 5 1 5. Jones, J., & Hunter, D. (1995). Consensus methods for medical and he alth services research. British Medical Journal, 311 (7001), 376 380. Journal of the American Medical Association 290 (6), 830. (Reprinted from Journal of the American Medical Associa tion 41 1903, 429). Kalkman, C. J. ( 2012 ). Serious play in the virtual world: Can we use games to train young doctors? Journal of Graduate Medical Education, 4 (1), 11 13. Kardong Edgren, S., Adamson, K. A., & Fitzgerald, C. (2010). A review of currently published evaluation instruments for human patient simulation. Clinical Simulation in Nursing 6(1), e25 e35. doi:10.1016/j.ecns.2009.08.004 Katz, R. (2006). Methods of training using pelvic trainers. Current Urology Reports, 7 (2), 100 106. Keeney, S., Has son, F., & McKenna, H. (2011). The Delphi technique in nursing and health research Oxford, UK: Wiley Blackwell. Keeney, S., Hasson, F., & McKenna, H. (2011). The Delphi technique in nursing and health research Oxford, UK: Wiley Blackwell. Kerdoka, A. E, Cotinb, S. M., Ottensmeyerb, M. P., Galeaa, A.M., Howe, R.D., & Dawson, S.L. (2003). Truth cube: Establishing physical standards for soft tissue simulation. Medical Image Analysis, 7 283 291. Keyes, J. A., Wilson, M. P., & Becker, J. (1975). The future of medical education: Forecast of the Council of Deans. Journal of Medical Education, 50 (4), 319 327. Kneebone, R. (2003). Simulation in surgical training: educational issues and practical implications. Medical Education, 37 (3), 267 277.

PAGE 354

354 Kneebone, R. (2005). Evaluating clinical simulations for learning procedural skills: A theory based approach. Academic Medicine, 80 (6), 549 553. Knowles, M. S. (1970). The modern practice of adult education: Andragogy versus pedagogy New York, NY: Association Press. Knowles, M. S. (1984). Andragogy in action San Francisco, CA: Jossey Bass. Knowles, M. S. (1988). The modern practice of adult education : From pedagogy to andragogy Englewood Cliffs: Prentice Hall/Cambridge. Koch, T. (2006). Establishing rigour in qualitative re search: the decision trail. Journal of advanced nursing 53 (1), 91 100 Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (Eds.). (2000). To err is human: Building a safer health system (Vol. 627). Washington, DC: National Academies Press. Kolb, D. A. (1984 ). Experiential learning Englewood Cliffs, NJ: Prentice Hall. Kreber, C. (2002). Controversy and consensus on the scholarship of teaching. Studies in Higher Education, 27 (2), 151 167. doi: http:// dx.doi.org/10.1080/03075070220119995 Kumar, D. D., & Sherwood, R. D. (2007). Effect of a problem based simulation on the conceptual understanding of undergraduate science education students. Journal of Science Education and T echnology, 16 (3), 239 246. Kun kler, K. (2006). The role of medical simulation: an overview. The International Journal of Medical Robotics and Computer Assisted Surgery, 2 (3), 203 210. high fidelity hu man patient simulator. Teaching and Learning in Nursing, 2 (2), 46 52. Kuznar, K. A. (2009). Effects of high fidelity human patient simulation experience on self efficacy, motivation and learning of first semester associate degree nursing students (Doctora l dissertation). Available from ProQuest Dissertations and Theses database (UMI No. 304932094). Retrieved from http://search.proquest.com/docview/304932094?accountid=10920 Ladysh ewsky, R., Baker, R., Jones, M., & Nelson, L. (2000). Reliability and validity of an extended simulated patient case: A tool for evaluation and research in physiotherapy. Physiotherapy Theory and Practice, 16 (1), 15 25. Lampotang, S. (2008). Medium and hig h integration mannequin patient simulators. InR. H. Riley (Ed.), Manual of simulation in healthcare (pp. 51 64). New York, NY: Oxford University Press.

PAGE 355

355 Landeta, J. (2005). Current validity of the Delphi method in social sciences. Technological Forecasting & Social Change, 73 467 482. Lane, J. L., Slavin, S., & Ziv, A. (2001). Simulation in medical education: A review. Simulation & Gaming, 32 (3), 297 314. Lee, L. Y. K., Lee, J. K. L., Wong, K. F., Tsang, A. Y. K., & Li, M. K. (2010). The establishment of an integrated skills training centre for undergraduate nursing education. International Nursing Review, 57 359 364. Leigh, G., & Hurst, H. (2008). We have a high fidelity simulator, now what? Making the most of simulators. International Journal of Nursing E ducation Scholarship, 5 (1), article 33. Retrieved from http://www.bepress.com/ijnes/vol5/iss1/art33 Lindeman, C. A. (1975). Delphi survey of priorities in clinical nursing research. Nursing Rese arch, 24 (6), 434 441 Lindeman, E. C. (1926). The meaning of adult education New York, NY: New Republic. Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3 12). R eading, MA: Addison Wesley. Lipton, H. L. (2009). Home is where the health is: Advancing team based care in chronic disease management. Archives of Internal Medicine, 169 (21), 945 1948. doi:10.1001/archinternmed.2009.428. Liu, A., Tendick, F., Cleary, K., & Kaufmann, C. (2003). A survey of surgical simulation: applications, technology, and education. Presence: Teleoperators & Virtual Environments, 12 (6), 599 614. criticisms exa mined. International Education Journal, 6 (3), 386 399. Lizarondo, L., Kumar, S., Hyde, L., & Skidmore, D. (2010). Allied health assistants and what they do: A systematic review of the literature Journal of Multidisciplinary Healthcare, 3 (1): 143 153. Lyn agh, M., Burton, R., Sanson Fisher, R. (2007). A systematic review of medical skills laboratory training: Where to from here? Medical Education, 41 (9): 879 887. Maas, N. A., & Flood, L. S. (2011). Implementing high fidelity simulation in practical nursing education. Clinical Simulation in Nursing, 7 (6), e229 e235. doi: 10.1016/j.ecns.2010.04.001 MacDonald, J., Williams, R. G., & Rogers, D. A. (2003). Self assessment in simulation based surgical skills training. The American Journal of Surgery, 185 (4), 319 322.

PAGE 356

356 Magee, J. H. (2003): Validation of medical modeling & simulation training devices and systems. Studies in Health Technology and Informatics. 94 196 198. Malekzadeh, S., Pfisterer, M. J., Wilson, B., Na, H., & Steehler, M. K. (2011). A novel low cost sinus surgery task trainer. Otolaryngology -Head and Neck Surgery, 145 (4), 530 533. Martino, J.P. (1972). An introduction to technological forecasting New York, NY: Gordon and Breach Science Publishers. Mase, D. J. & Wattenbarger, J. L. (1980). An overvi ew of allied health in the community colleges. Community College Review, 8 (1), 45 51. doi: 10.1177/009155218000800109 Masys, D. R. (2002). Effects of current and future information technologies on the healthcare workforce. Health Affairs, 21 (5), 33 41. doi : 10.1377/hlthaff.21.5.33 Mayer, R. E. (2001). Multimedia learning Cambridge (UK): Cambridge University Press. Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning Cambridge (MA): Cambridge University Press. McGaghie, W. C., Issenber g, S.B., Petrusa, E.R., & Scalese, R.J. (2010). A critical review of simulation based medical education research: 2003 2009. Medical Education, 44 (1), 50 63. McKenna, H., Hasson, F., & Smith, M. (2002). A Delphi survey of midwives and midwifery students to identify non midwifery duties. Midwifery, 18 (4), 314 322. McLachlan, J. C., Bligh, J., Bradley, P., & Searle, J. (2004). Teaching anatomy without cadavers. Medical Education, 38 (4), 418 424. McLaughlin, Starobin, & Laana, (2010). Medical simulation in the community college health science curriculum: A matrix for future implementation. Community College of Research and Practice, 34 (6), 462 476. Meijer, R., Ihnenfeldt, D., Vermeulen, M., De Haan, R., & Van Limbeek, J. (2003). The use of a modified Delphi pro cedure for the determination of 26 prognostic factors in the sub acute stage of stroke. International Journal of Rehabilitation Research, 26( 4), 265 270. Melnick, E. R., Nielson, J. A., Finnell, J. T., Bullard, M. J., Cantrill, S. V., Cochrane, D. G., ... & Richardson, L. D. (2010). Delphi consensus on the feasibility of translating the ACEP clinical policies into computerized clinical decision support. Annals of Emergency Medicine, 56 (4), 317 320.

PAGE 357

357 Miles Tapping, C., Dyck, A., Brunham, S., Simpson, E., & Ba rber, L. (1990). Canadian therapists' priorities for clinical research: A Delphi study. Physical Therapy, 70 (7), 448 454. Misener, T. R., Watkins, J. G., & Ossege, J. (1994). Public health nursing research priorities: a collaborative Delphi study. Public H ealth Nursing, 11 (2), 66 74. Moercke, A. M., & Eika, B. (2002). What are the clinical skills levels of newly graduated physicians? Self assessment study of an intended curriculum identified by a Delphi process. Medical Education, 36 (5), 472 478. Moore, M. & Kearsley, G. (2005). Distance education: A systems view (2nd ed.). Belmont, CA: Thomson Wadsworth. Moreno Ger, P., Torrente, J., Bustamante, J., Fernndez Galaz, C., Fernndez Manjna, B., & Comas Rengifo, M. D. (2010). Application of a low cost web base International Journal of Medical Informatics, 79 (6), 459 467. Morgan, P. J., Cleave Hogg, D., Desousa S., & Lam Mcculloch, J. (2006). Applying theory to practice in undergraduate educ ation using high fidelity simulation. Medical Teacher, 28 (1), e10 e15. doi:10.1080/01421590600568488 Morgan, P. J., Cleave Hogg, D., Desousa S., & Lam Mcculloch, J. (2006). Applying theory to practice in undergraduate education using high fidelity simulati on. Medical Teacher, 28 (1), e10 e15. doi:10.1080/01421590600568488 Mort, T. C. & Donahue, S. P. (2004). Debriefing: The basics. In W. F. Dunn (Ed.). Simulators in critical care and beyond (pp. 76 83). Des Plaines, IL: Society of Critical Care Medicine. Mun z, Y., Kumar, B. D., Moorthy, K., Bann, S., & Darzi, A. (2004). Laparoscopic virtual reality and box trainers: is one superior to the other? Surgical Endoscopy, 18 (3), 485 494. Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., Sanderson, C. F., As kham, J., & Marteau, T. (1998). Consensus development methods, and their use in clinical guideline development. Health Technology Assessment, 2 (3), 1 88. Murry, J. W. & Hammons, J. O. (1995). Delphi: A versatile methodology for conducting qualitative resea rch. The Review of Higher Education, 18 (4), 423 436. Nehring, W. M. & Lashley, F. R. (2004). Current use and opinions regarding human patient simulators in nursing education: An international survey Nursing Education Perspectives, 25 (5), 244 248.

PAGE 358

358 Nishisak i, A., Keren, R., & Nadkarni, V. (2007). Does simulation improve patient safety?: Self efficacy, competence, operational performance, and patient safety. Anesthesiology Clinics, 25 (2), 225 236. Nworie, J. (2011). Using the Delphi technique in educational t echnology research. TechTrends: For Leaders in Education & Training, 55 (5), 24 30. doi: 10.1007/s11528 011 0524 6 Nyssen, A. S., Larbuisson, R., Janssens, M., Pendeville, P., & Mayn, A. (2002). A comparison of the training value of two types of anesthesia simulators: computer screen based and mannequin based simulators. Anesthesia & Analgesia, 94 (6), 1560 1565. R. H. Riley (Ed.), Manual of simulation in healthcare (pp. 227 238). New York, NY: Oxford University Press. Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (2013). Report to Congress on the strategy to transition to use of human based methods for certain medical training ( H.Doc Report 112 705 ). Washington: Government Printing Office. Retrieved from http://www.nabr.org/uploadedFiles/nabrorg/Content/Hosted_Files/Congress%20 Report.p df Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: An example, design considerations and applications. Information & Management, 42 (1), 15 29. Okuda, Y., Bryson, E. O., DeMaria, S., Jacobson, L., Quinones, J., Shen, B. & Levin e, A. I. (2009). The utility of simulation in medical education: What is the evidence? Mount Sinai Journal of Medicine, 76 (4) 330 343. doi: 10.1002/msj.20127 O'Neill, S., Scott, M. &, Conboy, K. (2011). A Delphi study on collaborative learning in distance education: The faculty perspective. British Journal of Educational Technology, 42 (6), 939 949. doi: 10.1111/j.1467 8535.2010.01132.x Parker, J. T., & Taylor, P. G. (1980). The Delphi survey: CBAE through the eyes of leading educators Belmont, CA: Fearon Pitman Publishers. Passiment, M., Sacks, H., & Huang, G. (2011). Medical simulation in medical education: Results of an AAMC survey Washington, D.C.: Association of American Medical Colleges. Pollard, C. & Pollard, R. (2004). Research priorities in educat ional technology: A Delphi study. Journal of Research on Technology in Education, 37 (2), 145 160. Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41 (4), 376 382. doi: 10.1046/j.1365 2648.2003.02537.x

PAGE 359

359 Priede, C. & Farrall, S. (2011). Comparing results from different styles of cognitive International Journal of Social Research Methodology, 14 (4), 271 287. Raine, S. (2006). Defining the Bobath concept using the Delphi technique. Physiotherapy Research International, 11 (1), 4 13. Rauch, W. (1979). The decision Delphi. Technological Forecasting and Social Change, 15 (3), 159 169. Reid, N. (1988). The Delphi technique: Its contribution to the evaluation of professional pra ctice. In R. Ellis (Ed.) Professional competence and quality assurance in the caring professions (pp. 230 262). London: Croom Helm. Reznek, M. A. (2004). Current status of simulation in education and research. In G. K Loyd, C. L. Lake, & Greenberg, R. (Eds .), Practical health care simulations (pp. 27 47). Philadelphia, PA: Elsevier Mosby. Rieger, W. G. (1986). Directions in Delphi developments: Dissertations and their quality. Technological Forecasting and Social Change, 29 (2), 195 204. Robinson, T. C. (19 93). Overview of allied health issues in contemporary healthcare. Journal of Dental Education, 57 (8), 616 618. Rodgers, D. L. (2007). High fidelity patient simulation: A descriptive white paper report. Healthcare Simulation Strategies, 10 (4), 68 77. Retrie ved from http://sim strategies.com/downloads/Simulation%20White%20Paper2.pdf Rogers, M. R. & Lopez, E. C. (2002). Identifying critical cross cultural school psychology com petencies. Journal of School Psychology, 40 (2), 115 141. Rollin, B. E. (2006). The regulation of animal research and the emergence of animal ethics: a conceptual history. Theoretical Medicine and Bioethics, 27 (4), 285 304. Rosen, K. R. (2004). The history of medical simulation. In G. K Loyd, C. L. Lake, & Greenberg, R. (Eds.), Practical health care simulations (pp. 3 26). Philadelphia, PA: Elsevier Mosby. Rosen, K. R. (2008). The history of medical simulation. Journal of Critical Care, 23 (2), 157 166. Row e, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15 (4), 353 375. Sackman, H. (1974). Delphi critique; expert opinion, forecasting, and group process Lexington, MA.: Lexingto n Books.

PAGE 360

360 Sadosty, A. T., Bellolio, M. F., Laack, T. A., Luke, A., Weaver, A., & Goyal, D. G. (2011). Simulation based emergency medicine resident self assessment. The Journal of Emergency Medicine, 41 (6), 679 685. Scalese, R. J. & Issenberg, S. B. (2005). The effective use of simulations for the teaching and acquisition of veterinary professional and clinical skills. Journal of Veterinary Medical Education, 32 (3), 461 467. Schibler, K. M. (2008). A Delphi study of skills that will help first year elementary principals in California address critical issues they will face by 2012 (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 304376372) Schmidt, E., Goldhaber Fiebert, S. N., Ho, L. A., & McDonald, K. M. (2013). Si mulation exercises as a patient safety strategy: A systematic review. Annals of Internal Medicine, 158 (5 Part 2), 426 432. Schmidt, R. C. (1997). Managing Delphi surveys using nonparametric statistical techniques. Decision Sciences, 28 (3), 763 774. Schmidt V. V. (1995). Awakening intuition: A Delphi study (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 304231930). Schwid, H. A. (2004). Screen based simulation for critical care medicine. In W. F. Dunn (Ed.), Sim ulators in critical care and beyond (pp. 61 66). Des Plaines, IL: Society of Critical Care Medicine. Schwid, H. A., Rooke, G. A., Ross, B. K., & Sivarajan, M. (1999). Use of a computerized advanced cardiac life support simulator improves retention of advan ced cardiac life support guidelines better than a textbook review. Critical Care Medicine, 27 (4), 821 824. Scott, D. J., Bergen, P. C., Rege, R. V., Laycock, R., Tesfay, S. T., Valentine, R. J., ... & Jones, D. B. (2000). Laparoscopic training on bench mod els: better and more cost effective than operating room experience? Journal of the American College of Surgeons, 191 (3), 272. Seropian, M. A. (2003). General concepts in full scale simulation: Getting started. Anesthesia & Analgesia, 97 (6), 1695 1705. Sero pian, M. A., Brown, K., Gavilanes, J. S., & Driggers, B. (2004). Simulation: Not just a manikin. The Journal of Nursing Education, 43 (4), 164 169. Andersen, D. K., & Satava, R. M. (2002). Virtual reality training improves operating room performance: results of a randomized, double blinded study. Annals of Surgery, 236 (4), 458 464.

PAGE 361

361 Sexton, J. B., Thomas, E. J., & Helmreich, R. L. (2000). Error, stress, and teamwork in medicine and aviation: Cross sectional surveys. BMJ: British Medical Journal, 320 (7237), 745 749. Shaffer, W., Dawson, S. L., Meglan, D., Cotin, S., Ferrell, M., Norbash, A., Muller, J. (2001). Design principles for the use of simulation as an aid in interventional ca rdiology training. Minimally Invasive Therapy & Allied Technologies, 10 (2), 75 82. Sharkey, S. B., & Sharples, A. Y. (2001). An approach to consensus building using the Delphi technique: developing a learning resource in mental health. Nurse Education Toda y, 21 (5), 398 408. Shear, T. D., Greenberg, S. B., & Tokarczyk, A. (2013). Does training with human patient simulation translate to improved patient safety and outcome?. Current Opinion in Anesthesiology, 26 (2), 159 163. Siddall, V. J. (2004). Simulation i n respiratory therapy. In G. K Loyd, C. L. Lake, & Greenberg, R. (Eds.), Practical health care simulations (pp. 543 547). Philadelphia, PA: Elsevier Mosby. Sinz, E. H. (2004). Partial task trainers & simulations in critical care medicine. In W. F. Dunn (E d.) Simulators in critical care and beyond (pp. 33 41). Des Plaines, IL: Society of Critical Care Medicine. Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6 1 21. Smith, M. K. (2002). Malcolm Knowles, informal adult education, self direction and andragogy, The Encyclopedia of Informal Education Retrieved from www.infed.org/thinkers/et knowl.htm Snyder Halp ern, R., Thompson, C. B., & Schaffer, J. (2000). Comparison of mailed vs. Internet applications of the Delphi technique in clinical informatics research In Proceedings of the AMIA Symposium, pp. 809 813. American Medical Informatics Association. Solomon, G. & Schrum, L. (2007). Web 2.0: New tools, new schools Eugene, OR: International Society for Technology in Education. Sportsman, S., Bolton, C., Bradshaw, P., Close, D., Lee, M., Townley, N., & Watson, M. N. (2009). A regional simulation center partnersh ip: Collaboration to improve staff and student competency. The Journal of Continuing Education in Nursing, 40 (2), 67 73. Steurer, J. (2011). The Delphi method: An efficient procedure to generate knowledge. Skeletal Radiology, 40 (8), 959 961.

PAGE 362

362 Stelfox, H. T. Quality and Safety in Health Care, 15 (3), 174 178 Stewart, J., O'Halloran, C., Harrigan, P., Spencer, J. A., Barton, J. R., & Singleton, S. J. (1999). Identifying appropriate tasks for the preregistration year: Modified Delphi technique. British Medical Journal, 319 (7204), 224 229. Stillwell, C. A. (1999). A Delphi study on critical issues facing higher education for the first decade of the 21st century based on the perspectives of the executive leaders of the Exe cutive Leaders of the International Association of University Presidents (IAUP) (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 304520299) Sumsion, T. (1998). The Delphi technique: an adaptive research tool. The British Journal of Occupational Therapy, 61 (4), 153 156. Swanson, D. B., & Stillman, P. L. (1990). Use of standardized patients for teaching and assessing clinical ski lls. Evaluation & the Health Professions, 13 (1), 79 103. Tam, M. (2000). Constructivism, instructional systems design, and technology: Implications for transforming distance learning. Educational Technology & Society, 3 (2), 50 60. Tanaka, H., & Kobayashi, E. (2006). Education and research using experimental pigs in a medical school. Journal of Artificial Organs, 9 (3), 136 143. Thompson, C. R., Repko, K., & Staggers, N. (2003). A Delphi study to validate competencies required of Air Force medical surgical (4 6N3) nurses in mobilized environments. Military Medicine, 168 (8), 618 625. Thorne, S. (2000). Data analysis in qualitative research. Evidence Based Nursing, 3 (3), 68 70. Tough, D. T. (2009). Developing a consensus driven, core competency model to shape fut ure audio engineering technology curriculum: A web based modified Delphi study (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 305128843). Tuoriniemi, P., & Schott Baer, D. (2008). Implementing a high fidelity s imulation program in a community college setting. Nursing Education Perspectives, 29 (2), 105 109. Turoff, M. (1970). The design of a policy Delphi. Technological Forecasting and Social Change, 2 (2), 149 171.

PAGE 363

363 Turoff, M. (1975). The policy Delphi. In H. A. L instone & M. Turoff. (Eds.). The Delphi method: Techniques and applications (pp. 84 101). Reading, MA: Addison Wesley. Tysinger, J. W., Klonis, L. K., Sadler, J. Z., & Wagner, J. M. (1997). Teaching ethics using small group, problem based learning. Journal of Medical Ethics, 23 (5), 315 318. U.S. Department of Health and Human Services. (1999). Building the future of allied health: Report of the implementation task force of the national commission on allied health Rockville, Maryland: U.S. Department of Hea lth and Human Services, Public Health Service, Health Resources and Services Administration, Bureau of Health Professions, Division of Associated, Dental, and Public Health Professions. Uhl, N. P. (1983). Using the Delphi technique in institutional plannin g. New Directions for Institutional Research, 10 (1), 81 94. Van Zolingen, S. J., & Klaassen, C. A. (2003). Selection processes in a Delphi study about key qualifications in Senior Secondary Vocational Education. Technological Forecasting and Social Change, 70 (4), 317 340. Vleuten, C. P. & Swanson, D. B. (1990). Assessment of clinical skills with standardized patients: state of the art. Teaching and Learning in Medicine: An International Journal, 2 (2), 58 76. Vygotsky, L. S. (1978). Mind in society: The deve lopment of higher psychological processes (M. Cole, V. John Steiner, S. Scribner & E. Souberman, Eds. and Trans.). Cambridge, MA: Harvard University Press. Walker, A. M. (1994). A Delphi study of research priorities in the clinical practice of physiothera py. Physiotherapy, 80 (4), 205 207. Walker, A. M., & Selfe, J. (1996). The Delphi method: a useful tool for the allied health researcher. International Journal of Therapy and Rehabilitation, 3 (12), 677 681. Weaver, W. T. (1971). The Delphi forecasting metho d. Phi Delta Kappan, 52 (5), 267 271. Wenk, M., Waurick, R., Schotes, D., Wenk, M., Gerdes, C., Van Aken, H. K., & Ppping, D. M. (2009). Simulation based medical education is no better than problem based discussions and induces misjudgment in self assessme nt. Advances in Health Sciences Education, 14 (2), 159 171. West, J. F. & Cannon, G. S. (1988). Essential collaborative consultation competencies for regular and special educators. Journal of Learning Disabilities, 21 (1), 56 63.

PAGE 364

364 Whittemore, R., Chase, S. K. & Mandle, C. L. (2001). Validity in qualitative research. Qualitative Health Research, 11 (4), 522 537. doi: 10.1177/104973201129119299 Wicklein, R. C. (1993). Identifying critical issues and problems in technology educations using a modified Delphi techn ique. Journal of Technology Education, 5 (1). Retrieved from http://scholar.lib.vt.edu/ejournals/JTE/v5n1/bigwick3.html Williams, P. L. & Webb, C. (1994). The Delphi technique: A metho dological discussion. Journal of Advanced Nursing, 19 (1), 180 186. Willis, G. (2005). Cognitive interviewing: A tool for improving questionnaire design Thousand Oaks, CA: Sage. Willis, G. B., Royston, P., & Bercini, D. (1991). The use of verbal report met hods in the development and testing of survey questionnaires. Applied Cognitive Psychology 5 (3), 251 267. Wilson, M., Shepherd, I., Kelly, C., & Pitzner, J. (2005). Assessment of a low fidelity human patient simulator for the acquisition of nursing skills Nurse Education Today, 25 (1), 56 67. Wolf, Z. R. (2003). Exploring the audit trail for qualitative investigations. Nurse Educator, 28 (4), 175 178. Woods, R. H. & Baker, J. D. (2004). Interaction and immediacy in online learning. The International Revie w of Research in Open and Distance Learning, 5 (2 ) Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticle/186/268 Yaeger, K. A ., Halamek, L. P ., Coyle, M ., Murphy, A ., Anderson, J ., Boyle, K ., . & Smith, B. (2004). High fidelity simulation based training in neonatal nursing. Advances in Neonatal Care, 4 (6), 326 331. Zeedick, D. M. (2010). The modified Delphi method to analyze the application of instru ctional design theory to online graduate education (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3397676). Zhang, C ., Thompson, S ., & Miller, C. (2011). A review of simulation based interprofessional educatio n. Clinical Simulation in Nursing, 7 (4), e117 e126. Ziglio, E. (1996). The Delphi method and its contribution to decision making. In M. Adler and E. Ziglio (Eds. ), Gazing into the oracle: The Delphi method and its application to social policy and public health (3 33). London, England: Jessica Kingley Publishers. Ziv, A ., Erez, D ., Munz, Y ., Vardi, A ., Barsuk, D ., Levine, I ., The Israel Center for Medical Simulation: A paradigm for cultural change in medical education. Academic M edicine, 81 (12), 1091 1097.

PAGE 365

365 Ziv, A., Wolpe, P. R ., Small, S. D ., & Glick, S. (2003). Simulation based medical education: an ethical imperative. Academic Medicine, 78 (8), 783 788. Zunker, N. D. (2009). A modified Delphi study to identify the significant w orks pertaining to the understanding of reading comprehension and content analysis of the identified works (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 89202198).

PAGE 366

366 BIOGRAPHICAL SKETCH Johanna Kenney received her Doctor of Education in curriculum and instruction from the University of Florida in the spring of 2014. She received her BA in political science and w g ender studies from Eckerd College and her MA in social sciences f rom the University of Colorado at Denver include modeling and simulations in healthcare education social media and Web 2 0, online communities of practice (CoPs), and serious games for education She currently works as a Le arning Systems Developer for the United States Air Force Johanna uses instructional design methods along with adult learning theories and validated research to find innovative approaches to integrate technology into military medical education and trainin g