<%BANNER%>

Mixed Reality Interpersonal Simulation Affords Cognitive, Psychomotor, and Affective Learning

Permanent Link: http://ufdc.ufl.edu/UFE0041156/00001

Material Information

Title: Mixed Reality Interpersonal Simulation Affords Cognitive, Psychomotor, and Affective Learning
Physical Description: 1 online resource (363 p.)
Language: english
Creator: Kotranza, Aaron
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: education, haptics, human, medical, simulation, virtual
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: In fields such as medicine, military, and law enforcement, success in interpersonal scenarios requires mastering a complex group of cognitive, psychomotor, and affective skills. These interpersonal scenarios involve tasks which call upon multiple skill sets concurrently. For example, in a clinical breast exam, the doctor must concurrently recall a pattern of palpation (cognitive) while palpating with correct pressure (psychomotor) and also keeping a patient comforted (affective). The concurrent use of these skill sets mandates that learners practice concurrent actuation of these skills; each skill set should not be learned in isolation. However, traditional education approaches provide a level of practice that is inadequate for many novices to achieve competency in the three skill sets and to learn how to concurrently call on multiple skill sets to succeed in the interpersonal scenario. This is due to reasons such as lack of standardization and inability to recreate these interpersonal scenarios with peers or human actors. Interpersonal simulation with virtual humans promises on-demand, highly immersive learning experiences which could augment these curricula to provide learners the additional practice required to achieve competency. For this reason, interpersonal simulation is receiving increased attention from early adopters in medicine, military, and law enforcement fields. However, interpersonal simulation has yet to be deployed as part of curricula in these fields. We have identified two shortcomings of current approaches to interpersonal simulation: the absence of touch ? which contributes to training of all three skill sets, and a lack of evidence that learning and skills transfer takes place in users of the simulations. To address the shortcomings of current approaches, we introduce a new approach to interpersonal simulation ? mixed reality interpersonal simulation (MRIPS). MRIPS incorporates passive and active haptic interfaces instrumented with sensors to afford touch between human and virtual human as well as the manipulation of hand-held tools. Thus, in addition to bidirectional verbal and gestural capabilities of traditional interpersonal simulation, MRIPS provides touch from human to virtual human, touch from virtual human to human, and touch from hand-held tools to virtual human. These touch inputs and outputs are used to affect the psychomotor, cognitive, and affective components of the simulated interpersonal scenario. The incorporation of these haptic interaction capabilities in MRIPS addresses problems with prior approaches to interpersonal simulation, expanding the applicability of interpersonal simulation to training cognitive, psychomotor, and affective skills and their concurrent use: 1. Problem: Psychomotor task components involving touch could not be simulated. a. Approach: MRIPS incorporates touch from human to virtual human and from hand-held tool to virtual human to afford the simulation of psychomotor task components involving touch. 2. Problem: Prior interpersonal simulation approaches do not afford touch for communication between the human and virtual human. Touch is an essential component of communication between two humans and the lack of touch results in incomplete communication in these interpersonal simulators. a. Approach: By incorporating touch from human to virtual human and from virtual human to human, MRIPS affords interpersonal communication to accomplish both cognitive task components, e.g. achieving compliance in getting a patient to assume a specific pose for a physical exam, and affective components, e.g. a comforting touch. 3. Problem: Prior approaches to interpersonal simulation afforded only simple vision-based gesture interfaces, e.g. pointing and iconic gestures, or encumbering gesture interfaces, e.g. body suits. a. Approach: MRIPS incorporates instrumentation, e.g. six degree-of-freedom tracking, of the haptic interfaces. This affords simulation of hand-held tools for performing complex psychomotor task components not involving touch. These tools also serve as grounding objects to enhance communication between the human and virtual human. 4. Problem: Feedback and reflection motivated by feedback is necessary for learning. Prior interpersonal simulations could not provide feedback of psychomotor performance without encumbering interfaces, e.g. gloves, body suits or expert observers. a. Approach: MRIPS incorporates instrumentation of the non-encumbering haptic interfaces for touching the virtual human and for tool manipulation. This affords quantitative measuring of the learner?s psychomotor, cognitive, and affective skills performance as well as real-time feedback to guide and elicit reflection on the learner?s performance. By providing real-time feedback of learner performance, MRIPS increases the potential for learning in the simulated interpersonal scenario. We applied MRIPS to simulate two interpersonal scenarios in medicine ? the clinical breast exam and the neurological exam ? which could not be simulated through prior interpersonal simulation approaches. User studies established the validity of MRIPS for practicing and evaluating learners? psychomotor, cognitive, and affective skills. We then incorporated visual feedback of user performance in these three skill sets, to enhance the potential for learning. Additional user studies were then conducted to determine what learning occurs in users of MRIPS and whether skills learned in MRIPS transfer to the real-world scenarios being simulated.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Aaron Kotranza.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Lok, Benjamin C.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2011-12-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0041156:00001

Permanent Link: http://ufdc.ufl.edu/UFE0041156/00001

Material Information

Title: Mixed Reality Interpersonal Simulation Affords Cognitive, Psychomotor, and Affective Learning
Physical Description: 1 online resource (363 p.)
Language: english
Creator: Kotranza, Aaron
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: education, haptics, human, medical, simulation, virtual
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre: Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: In fields such as medicine, military, and law enforcement, success in interpersonal scenarios requires mastering a complex group of cognitive, psychomotor, and affective skills. These interpersonal scenarios involve tasks which call upon multiple skill sets concurrently. For example, in a clinical breast exam, the doctor must concurrently recall a pattern of palpation (cognitive) while palpating with correct pressure (psychomotor) and also keeping a patient comforted (affective). The concurrent use of these skill sets mandates that learners practice concurrent actuation of these skills; each skill set should not be learned in isolation. However, traditional education approaches provide a level of practice that is inadequate for many novices to achieve competency in the three skill sets and to learn how to concurrently call on multiple skill sets to succeed in the interpersonal scenario. This is due to reasons such as lack of standardization and inability to recreate these interpersonal scenarios with peers or human actors. Interpersonal simulation with virtual humans promises on-demand, highly immersive learning experiences which could augment these curricula to provide learners the additional practice required to achieve competency. For this reason, interpersonal simulation is receiving increased attention from early adopters in medicine, military, and law enforcement fields. However, interpersonal simulation has yet to be deployed as part of curricula in these fields. We have identified two shortcomings of current approaches to interpersonal simulation: the absence of touch ? which contributes to training of all three skill sets, and a lack of evidence that learning and skills transfer takes place in users of the simulations. To address the shortcomings of current approaches, we introduce a new approach to interpersonal simulation ? mixed reality interpersonal simulation (MRIPS). MRIPS incorporates passive and active haptic interfaces instrumented with sensors to afford touch between human and virtual human as well as the manipulation of hand-held tools. Thus, in addition to bidirectional verbal and gestural capabilities of traditional interpersonal simulation, MRIPS provides touch from human to virtual human, touch from virtual human to human, and touch from hand-held tools to virtual human. These touch inputs and outputs are used to affect the psychomotor, cognitive, and affective components of the simulated interpersonal scenario. The incorporation of these haptic interaction capabilities in MRIPS addresses problems with prior approaches to interpersonal simulation, expanding the applicability of interpersonal simulation to training cognitive, psychomotor, and affective skills and their concurrent use: 1. Problem: Psychomotor task components involving touch could not be simulated. a. Approach: MRIPS incorporates touch from human to virtual human and from hand-held tool to virtual human to afford the simulation of psychomotor task components involving touch. 2. Problem: Prior interpersonal simulation approaches do not afford touch for communication between the human and virtual human. Touch is an essential component of communication between two humans and the lack of touch results in incomplete communication in these interpersonal simulators. a. Approach: By incorporating touch from human to virtual human and from virtual human to human, MRIPS affords interpersonal communication to accomplish both cognitive task components, e.g. achieving compliance in getting a patient to assume a specific pose for a physical exam, and affective components, e.g. a comforting touch. 3. Problem: Prior approaches to interpersonal simulation afforded only simple vision-based gesture interfaces, e.g. pointing and iconic gestures, or encumbering gesture interfaces, e.g. body suits. a. Approach: MRIPS incorporates instrumentation, e.g. six degree-of-freedom tracking, of the haptic interfaces. This affords simulation of hand-held tools for performing complex psychomotor task components not involving touch. These tools also serve as grounding objects to enhance communication between the human and virtual human. 4. Problem: Feedback and reflection motivated by feedback is necessary for learning. Prior interpersonal simulations could not provide feedback of psychomotor performance without encumbering interfaces, e.g. gloves, body suits or expert observers. a. Approach: MRIPS incorporates instrumentation of the non-encumbering haptic interfaces for touching the virtual human and for tool manipulation. This affords quantitative measuring of the learner?s psychomotor, cognitive, and affective skills performance as well as real-time feedback to guide and elicit reflection on the learner?s performance. By providing real-time feedback of learner performance, MRIPS increases the potential for learning in the simulated interpersonal scenario. We applied MRIPS to simulate two interpersonal scenarios in medicine ? the clinical breast exam and the neurological exam ? which could not be simulated through prior interpersonal simulation approaches. User studies established the validity of MRIPS for practicing and evaluating learners? psychomotor, cognitive, and affective skills. We then incorporated visual feedback of user performance in these three skill sets, to enhance the potential for learning. Additional user studies were then conducted to determine what learning occurs in users of MRIPS and whether skills learned in MRIPS transfer to the real-world scenarios being simulated.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Aaron Kotranza.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Lok, Benjamin C.
Electronic Access: RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2011-12-31

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0041156:00001


This item has the following downloads:


Full Text

PAGE 1

1 MIXED REALITY INTERPERSONAL SIMULATION AFFORDS COGNITIVE, PSYCHOMOTOR, AND AFFECTIVE LEARNING By AARON ANDREW KOTRANZA A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2009

PAGE 2

2 2009 Aaron Andrew Kotranza

PAGE 3

3 To my wife Sarah for her love and support

PAGE 4

4 ACKNOWLEDGMENTS I thank my research advisor and supervisory committee chair, Dr. Benjamin Lok, for his advice, direction and support. I also thank my collaborators in medi c al education which made this work possible and provided valuable insight into real -world applications of this work: Dr. D. Scott Lind, Dr. Carla Pugh, Dr. Juan Cend an, Dr. Adeline Deladisma, and Andy Laserna. Thank you to Dr. Samsun Lampotang, Dr. Jorg Peters, Dr. Paul Fishwick, and Dr. Alireza Entezari for being my supervisory committee members, and for their ideas and support in my research. Kyle Johnsen, Andrew Raij, Brent Rossen, John Quarles, Xiyong Wang, Joon Chuah, and the rest of the past and present members of the Virtual Experiences Research Group provided invaluable assistance in conducting this research and were always there to discuss new ideas, no matt er how off -the wall. I thank the University of Florida alumni who provided financial support for my work through a University of Florida Graduate Alumni Fellowship. I thank my family for their love and support: my parents Steve and Rosemary, my brother E van, my sister Alissa, and especially my ever encouraging wife Sarah.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ...................................................................................................... 4 LIST OF TABLES .............................................................................................................. 12 LIST OF FIGURES ............................................................................................................ 14 LIST OF ABBREVIATIONS .............................................................................................. 17 ABSTRACT ........................................................................................................................ 18 CHAPTER 1 INTRODUCTION ........................................................................................................ 22 1.1 Motivation: Enhancing Educational Methods for Teaching Interpersonal Scenarios ................................................................................................................. 25 1.1.1 Peer Simulation ........................................................................................... 26 1.1.2 Human Actor Simulation ............................................................................. 28 1.1.3 Interpersonal Simulation with Virtual Humans ........................................... 30 1.1.3.1 Current approaches to interpersonal simulation lack touch ............ 31 1.1.3.2 Feedback of learner performance is limited ..................................... 33 1.1.3.3 Learning and training transfer have not been demonstrated in current approaches to interpersonal simulation. ....................................... 34 1.2 Motivation: Augmenting Education of Medical Interpersonal Scenarios Underserved by Current Educational Approaches ................................................. 35 1.2.1 Enhancing Intimate Exam Education ......................................................... 35 1.2.1.1 Intimate exams require touch for cognitive, psychomotor, and affective task components and require concurrent use of these three skill sets ...................................................................................................... 36 1.2.1.2 Intimate exam training is underserved by existing educational approaches ................................................................................................. 37 1.2.1.3 Improving intimate exam training has potential for broad social benefit ......................................................................................................... 41 1.2.2 Increasing Exposure to Abnormal Physical Findings ................................ 42 1.2.2.1 Training neurological examination with abnormal findings is underserved by existing educational approaches ..................................... 42 1.2.2.2 Neurological examination requires touch and concurrent use of cognitive, psychomotor, and affective skill sets ........................................ 44 1. 2.2.3 Increasing exposure to abnormal findings in neurological exams has potential for broader social benefit ..................................................... 45 1.3 Thesis .................................................................................................................... 46 1 .4 Overview of Approach .......................................................................................... 46 1.4.1 Technological Innovation of MRIPS and Application to Medical Interpersonal Scenarios .................................................................................... 47

PAGE 6

6 1.4.2 E valuation of the Validity of MRIPS for Practicing and Evaluating Cognitive, Psychomotor, and Affective Task Components ............................. 48 1.4.3 Incorporation or Real -time Feedback of Cognitive, Psychomotor, and Affective Task Performance ............................................................................. 48 1.4.4 Evaluation of Learning and Training Transfer ........................................... 50 1.5 Innovations ............................................................................................................ 53 2 REVIEW OF LITERATURE ........................................................................................ 58 2.1 Foundations for Interpersonal Simulation with Virtual Humans .......................... 58 2.1.1 Social Responses to Virtual Humans ......................................................... 59 2.1.2 Toward Changing Human Behavior ........................................................... 60 2.2 Interpersonal Simulation with Virtual Humans ..................................................... 62 2.2.1 Current Approaches to Interpersonal Simulation ....................................... 62 2.2.2 Existing Interpersonal Simulators ............................................................... 66 2.2.3 Incorporation of Feedback in Interpersonal Simulation ............................. 68 2.3 Motivation for Touch in Interpersonal Simulation ................................................ 69 2.3.1 The Role of Touch in Communication ........................................................ 70 2.3.2 Touch in Virtual Environments ................................................................... 71 2.4 Other Approaches to Medical Interpersonal Simulation...................................... 73 3 MRIPS DESIGN PRINCIPLES AND DEVELOPMENT OF MRIPS-CBE ................. 75 3.1 MRIPS Des ign Principles ..................................................................................... 76 3.2 Clinical Breast Examination ................................................................................. 79 3.2.1 CBE Procedure ........................................................................................... 79 3.2.2 Cognitive, Psychomotor, and Affective Components ................................ 82 3.2.3 Current Approaches to Teaching Clinical Breast Examination ................. 83 3.3 MRIPS -CBE .......................................................................................................... 85 3.3.1 Motivations and Goals ................................................................................ 85 3.3.2 Merging and Augmenting in MRIPS -CBE .................................................. 86 3.3.3 Cognitive, Psychomotor, and Affective Affordances of MRIPS -CBE ........ 87 3.4 Visual Interface ..................................................................................................... 88 3.4.1 Life -sized Virtual Human Characters ......................................................... 89 3.4.2 Augmenting the Virtual World with Real Objects ....................................... 89 3.4.3 Display of the Visual Interface .................................................................... 92 3.4.3.1 Head Mounted Display (HMD) ......................................................... 93 3.4.3.2 Projection display .............................................................................. 93 3.4.4 Perspective Correct Viewing of the Visual Interface ................................. 94 3.4.5 Registering Visual and Physical Interfaces ................................................ 94 3.5 Physical Interface ................................................................................................. 95 3.5.1 Active Sensing of User Touch .................................................................... 95 3.5.2 Passive Detection of User Touch and Manipulation of Tools and Props ................................................................................................................. 97 3.5.3 Bidirectional Touch: Enabling the Virtual Human to Touch the User ........ 99 3.5.3.1 Purely virtual touch ........................................................................... 99 3.5.3.2 Physical touch ................................................................................. 100

PAGE 7

7 3.6 Mixed Reality Human Simulation ....................................................................... 103 3.6.1 Maintaining and Applying Simulation State ............................................. 104 3.6.2 Incorporating User Touch of the Physical Interface into the Simulation 105 3.6.3 Touch-Driven Communication .................................................................. 106 3.7 Example MRIPS -CBE Interaction ...................................................................... 108 3.8 Pilot Study ........................................................................................................... 110 3.8.1 Population and Procedure ........................................................................ 110 3.8.2 Observations ............................................................................................. 111 3.8.3 Discussions ............................................................................................... 112 3.8.4 Conclusion and Further Evaluation .......................................................... 113 4 VALIDITY OF MRIPS -CBE FOR PRACTICE AND EVALUATION OF COGNITIVE, PSYCHOMOTOR, AND AFFECTIVE SKILLS .................................. 123 4.1 Introduction ......................................................................................................... 124 4.2 Study MRIPS -SP: Comparing MRIPS -CBE to CBE of an SP .......................... 126 4.2.1 Study Design and Procedure ................................................................... 127 4.2.2 Measures .................................................................................................. 129 4.2.3 Statistical Analysis .................................................................................... 130 4.2.4 Results and Discussion ............................................................................ 131 4.2.4.1 Order effects .................................................................................... 131 4.2.4.2 Cognitive performance .................................................................... 132 4.2.4.3 Psychomotor ................................................................................... 134 4.2.4.4 Affective ........................................................................................... 135 4 .2.5 Limitations of the Study ............................................................................ 137 4.2.6 Conclusions ............................................................................................... 138 4.3 MRIPSx2 ............................................................................................................. 139 4.3.1 Study Design and Procedure ................................................................... 139 4.3.2 Measures .................................................................................................. 140 4.3.3 Analyzing the Impact of Experience on Performance ............................. 141 4.3.4 Results ...................................................................................................... 14 4 4.3.4.1 Cognitive .......................................................................................... 144 4.3.4.2 Psychomotor ................................................................................... 144 4.3.4.3 Affective ........................................................................................... 145 4.3.5 Discussion ................................................................................................. 146 4.4 Conclusion .......................................................................................................... 146 5 MRIPS -NEU R O ........................................................................................................ 152 5.1 Introduction ......................................................................................................... 153 5.1.1 The Neurological Exam Req uires Cognitive, Psychomotor, and Affective Skills ................................................................................................. 154 5.1.2 Evaluating MRIPS -NEURO ...................................................................... 156 5.2 The Neurological Exam ...................................................................................... 157 5.3 A Virtual Human Agent to Simulate Cranial Nerve Disorders ........................... 158 5.3.1 Eye Movement Model ............................................................................... 158 5.3.2 Virtual Human Abilities to Support Neurological Tests ............................ 161

PAGE 8

8 5.4 The Haptic Interface ........................................................................................... 162 5.4.1 Prior Approaches ...................................................................................... 162 5.4.2 Haptic Interface: Wii -Remote and External Sensing ............................... 163 5.4.3 Virtual Hand-Held Tools and Hand Gestures .......................................... 165 5.4.3.1 Ophthalmoscope ............................................................................. 165 5.4.3.2 Eye chart ......................................................................................... 166 5.4.3.3 Hand gesture tool ............................................................................ 167 5.4.3 The Haptic Interface Enhances Communication in Interpersonal Simulation ....................................................................................................... 170 5.5 Usability and Content Validity of MRIPS -NEURO for Practicing Diagnoses of Abnormal Findings ............................................................................................ 171 5.5.1 Study Design and Procedure ................................................................... 171 5.5.2 Results ...................................................................................................... 172 5.5.3 Observations ............................................................................................. 174 5.5.4 Conclusions and Continued Evaluation ................................................... 175 6 REAL -TIME EVALUATION AND FEEDBACK OF PERFORMANCE ..................... 184 6.1 Motivation for Feedback ..................................................................................... 185 6.2 Unique Capabilities of MRIPS to Evalua te Performance and Provide Feedback ............................................................................................................... 186 6.3 Choice of the Visual Channel to Provide Feedback .......................................... 189 7 FEEDBACK IN MRIPS-CBE .................................................................................... 191 7.1 Introduction ......................................................................................................... 191 7.1.1 Cognitive Components ............................................................................. 191 7.1.2 Affective Components ............................................................................... 192 7.1.3 Psychomotor Components ....................................................................... 193 7.2 Procedural Checklist ........................................................................................... 194 7.3 Thought Bubbles ................................................................................................ 195 7.3.1 Automated Evaluation of Affective Performance ..................................... 197 7.3.2 Feedback to Reinforce and Correct Affective Performance .................... 199 7.4 Touch Map .......................................................................................................... 200 7.4.1 Feedback Goals ........................................................................................ 200 7.4.2 Capturing Palpation Pressure and Pressure in an Experts CBE ........... 201 7.4.3 Guiding and Evaluating Complete Coverage .......................................... 203 7.4.4 Calculating the Palpation Pressure Levels .............................................. 204 7.4.5 Design of Feedback Elements to Guide, Reinforce, and Correct ........... 208 7.4.6 Presenting Feedback In-situ with the Virtual Human and Physical Breast .............................................................................................................. 210 7.4.7 Design Choices ......................................................................................... 210 7.4.7.1 How many exper ts are needed to model psychomotor performance? ........................................................................................... 210 7.4.7.2 Visual feedback elements occlude the learners hands ................ 211 7.4.7.3 Drawbacks of an image -based approach ...................................... 211 7.5 Patternof -Search Map ....................................................................................... 212

PAGE 9

9 7.5.1 Feedback Goals ........................................................................................ 212 7.5.2 Modeling Correct Patternof -Search ........................................................ 212 7.5.3 Guiding and Evaluating Learner Patternof -Search ................................ 213 7.5.4 Design of the Feedback Elements for Guiding, Reinforcement, and Correction ........................................................................................................ 215 7.6 Post Experiential Feedback ............................................................................... 216 7.7 Face Validity of Touch Map and Patternof -Search Map Feedback ................. 217 8 FEEDBACK IN MRIPS-NEURO ............................................................................... 230 8.1 Introduction ......................................................................................................... 230 8.2 H Map ................................................................................................................. 230 8.3 Patient Vision ...................................................................................................... 232 8.3.1 Feedback Goals ........................................................................................ 232 8.3.2 Prior Work in Motivating Perspective Taking ........................................... 233 8.3.3 Patient Vision Feedback ........................................................................... 234 8.4 Evaluating the Impact of Feedback on Cognitive, Psychomotor, and Affective Performance ........................................................................................... 235 8.4.1 Study Design and Procedure ................................................................... 235 8.4.2 Population ................................................................................................. 238 8.4.3 Metrics ....................................................................................................... 238 8.4.3.1 Evaluating cognitive and affective performance ............................ 238 8.4.3.2 Evaluating psychomotor performance ............................................ 239 8.4.3 Hypotheses ............................................................................................... 241 8.4.4 Resu lts and Discussion ............................................................................ 242 8.4.4.1 Hypothesis Patient -Vision improves affective. Experiencing patient vision increases concern for patient safety: accepted ................ 242 8.4.4.2 Hypothesis Patient -Vision improves cognitive. Experiencing patient vision aids diagnosis of CN disorder: rejected but with a positive result ........................................................................................... 244 8.4.4.3 Hy pothesis H Map improves psychomotor completeness. H Map visualization results in a more complete eye movements test: rejected ..................................................................................................... 245 8.4.4.4 Hypothesis H Map improves psychomotor efficiency. H -Ma p visualization results in a more efficient eye movements test: accepted 249 8.4.5 Conclusions ............................................................................................... 250 9 LEARNING, TRAINING TRANSFER, AND IMPACT OF REAL -TIME FEEDBACK IN MRIPS-CBE .................................................................................... 258 9.1 Introduction ......................................................................................................... 259 9.2 Study Design ...................................................................................................... 260 9.2.1 Evaluating Learning and Training Transfer .............................................. 260 9.2.2 Evaluating the Impact of Real -Time Feedback on Performance ............ 262 9.2.3 Control Groups for Investigating the Validity of Study Results ............... 263 9.3 Population ........................................................................................................... 264 9.4 Statistic al Analysis .............................................................................................. 265

PAGE 10

10 9.5 Cognitive Performance ....................................................................................... 266 9.5.1 Measures .................................................................................................. 267 9.5.1.1 Breast history completeness .......................................................... 267 9.5.1.2 Visual inspection completeness ..................................................... 269 9.5.2 Hypotheses ............................................................................................... 269 9.5.3 Results: Breast History Learning and Training Transfer ......................... 270 9.5.4 Results: Impact of Feedback on Breast History Completeness .............. 274 9.5.5 Visual Inspection Learning and Training Transfer ................................... 276 9.5.6 Discussion ................................................................................................. 278 9 .6 Psychomotor and Cognitive-Psychomotor Performance .................................. 278 9.6.1 Measures .................................................................................................. 279 9.6.1.1 Coverage and correct pressure ...................................................... 279 9.6.1.2 Correct pattern of search ................................................................ 282 9.6.1.3 Finding masses ............................................................................... 283 9.6.2 Hyp otheses ............................................................................................... 284 9.6.3 Results: Coverage and Pressure Learning and Transfer ........................ 285 9.6.3.1 Coverage learning ........................................................................... 285 9.6.3.2 Coverage transfer ........................................................................... 287 9.6.3.3 Pressure learning ............................................................................ 288 9.6.3.4 Pressure transfer ............................................................................. 289 9.6.4 Results: Impact of Real -Time Feedback on Coverage and Pressure .... 290 9.6.5 Results: Pattern of -Search Learning and Transf er .................................. 293 9.6.6 Results: Impact of Real -Time Feedback on Pattern of -Search .............. 297 9.6.7 Results: Finding Masses and False Positives Le arning and Transfer .... 298 9.6.8 Discussion ................................................................................................. 301 9.7 Affective Performance ........................................................................................ 302 9.7.1 Measures .................................................................................................. 302 9.7.2 Hypotheses ............................................................................................... 306 9.7.3 Results: Empathy Learning ...................................................................... 306 9.7.4 Results: Impact of Feedback .................................................................... 307 9.7.5 Results: Empathy Transfer ....................................................................... 309 9.7.6 Discussion ................................................................................................. 310 9.8 Validity of Results ............................................................................................... 311 9.8.1 Impact of Multiple MRIPS Practice Opportunities ................................... 311 9.8.2 Impact of an SP Pre-test Interaction on Subsequent SP Performance .. 312 9.9 Study Limitations ................................................................................................ 313 9.10 Revisitin g Meta -Hypotheses ............................................................................ 314 10 SUMMARY AND FUTURE DIRECTIONS ............................................................... 330 10.1 Review of Results ............................................................................................. 330 10.2 Future Directions .............................................................................................. 331 APPENDIX A STUDY MRIPS -SP DATA ........................................................................................ 334

PAGE 11

11 A.1 Breast History Checklist Data ............................................................................ 334 A.2 Empathy Video Review Instrument ................................................................... 334 A.3 Empathy Video Review Data ............................................................................. 335 B STUDY MRIPS x2 DATA ......................................................................................... 337 B.1 Study MRIPSx2 Video Review Instrument ........................................................ 337 B.2 Study MRIPSx2 Video Review Data ................................................................. 337 B.3 Study MRIPSx2 Breast History Checklist Data ................................................. 338 B.4 Study MRIPSx2 Palpation Completeness Data ................................................ 338 C STUDY MRIPS -NEURO QUESTIONNAIRES ......................................................... 339 C.1 Study MRIPS -NEURO Post -Patient Vision Survey .......................................... 339 C.2 Study MRIPS -NEURO Post -Exam Survey ....................................................... 339 D STUDY MRIPS -LEARNING INSTRUMENTS AND DATA ...................................... 341 D.1 Study MRIPS Learning Breast History Checkl ist Data ..................................... 341 D.2 Study MRIPS Learning Coverage and Pressure Data ..................................... 342 D.3 Study MRIPS Learning Patternof -Search Data ............................................... 343 D.4 Study MRIPS Learning Empathy Video Review Instrument ............................ 343 D.5 Study MRIPS Learning Empathy Video Review Data ...................................... 344 D.6 Affective Ratings of Participants in MRIPS -CBE .............................................. 346 D.7 Pilot Study Video Rating Instrument and Data ................................................. 346 LIST OF REFERENCES ................................................................................................. 349 BIOGRAPHICAL SKETCH .............................................................................................. 363

PAGE 12

12 LIST OF TABLES Table page 1 -1 Affordances and drawbacks of prior educational approaches and MRIPS for teaching cognitive, psychomotor, and affective aspects of high-stakes interpersonal scenarios .......................................................................................... 55 1 -2 T raditional and simulation approaches to teaching intimate exams ..................... 56 1 -3 Feedback in MRIPS ............................................................................................... 56 3 -1 List of simulation inputs and outputs. .................................................................. 114 3 -2 U sing system state to direct the conversation between user and MRH ............. 114 4 -1 Items in the medical history completen ess checklist .......................................... 148 4 -2 Results of video review of critical moments ........................................................ 148 4 -3 Population breakdown for Study MRIPSx2. ........................................................ 149 4 -4 Cognitive performance results for Study MRIPSx2. ........................................... 149 4 -5 Psychomotor performance results for Study MRIPSx2. ..................................... 149 5 -1 List of cranial nerves which can be examined using MRIPS -NEURO. .............. 176 5 -2 Focused neurological exam tasks ....................................................................... 176 5 -3 Usability ratings of MRIPS -NEURO. .................................................................... 176 9 -1 Instrument used to evaluate the completeness of breast history taking in MRIPS and SP interactions ................................................................................. 318 9 -3 Performance in breast history taking in Study MRIPS Learning. ....................... 319 9 -4 Changes in the number of participants asking about specific risk factor s ......... 319 9 -5 Number of participants performing any visual inspection and complete visual inspections in each interaction ............................................................................. 319 9 -6 In strument used to evaluate coverage and use of correct pressure in the SP interactions ........................................................................................................... 320 9 -7 Summary of acceptance and rejection of hypotheses of psychomotor and cognitive-psychomotor task pe rformance............................................................ 320

PAGE 13

13 9 -8 Coverage and use of deep pressure of the Study MRIPS -Learning participants ........................................................................................................... 321 9 -9 Total deviation from expert patternof -search in the three MRIPS interactions of Study MRIPS Learning. ................................................................................... 321 9 -10 Number of participants finding real masses and false positive masses in MRIPS. ................................................................................................................. 321 9 -11 Participants finding masses palpated a larger percentage of the MRIPS breast with deep pressure .................................................................................... 322 9 -12 Affective performance in MRIPS -CBE interac tions. ............................................ 322 9 -13 Expert ratings of participants affective performance in the SP interactions ...... 3 22 9 -14 Performance in pilot study used to assess impact of a single MRIPS CBE interaction on a subsequent CBE of an SP ......................................................... 323 9 -15 Performance in a CBE of an SP after three MRIPS -CBE practice opportunities and after one MRIPS -CB E practice opportunity ........................... 323 9 -16 Concurrent improvement in the three skill sets ................................................... 324

PAGE 14

14 LIST OF FIGURES Figure page 1 -1 The simulator of Pugh et al. [38] ............................................................................ 57 3 -1 A learner performs a CBE in MRIPS -CBE .......................................................... 115 3 -2 System de sign of MRIP S -CBE ............................................................................ 116 3 -3 MRIPS -CBE visual interface presented on a projection screen. ........................ 117 3 -4 The physical interface of MRIPS -C BE ................................................................ 117 3 -5 The first approach taken to tracking the physical gown and providing a corresponding virtual gown used a background subtraction approach .............. 118 3 -6 Affording bidirectional touch for communication by allowing the MRH to touch the user ................................................................................................................. 119 3 -7 The physical interface incorporates a mechanical right arm, allowing a ctive haptic touch from virtual human to human user. ................................................. 120 3 -8 The layers of the framework for abstracting a virtual environments control of physical actuators. ................................................................................................ 120 3 -9 Progression of one time step of the simulation module ...................................... 121 3 -10 The medical interview portion of the interaction with the MRH breast exam patient ................................................................................................................... 121 3 -11 Two of the poses required for visual inspection .................................................. 122 3 -12 The xml script that defines relationships between servos, constraints and animation of servos to allow the MRH to touch the user on the hand. ............ 122 4 -1 Appearance of the MRIPS -CBE patient for Study MRIPS -SP ........................... 150 4 -2 The appearance of the MRIPS -CBE mixed reality human in Study MRIPSx2 .. 151 4 -3 Visualization of a participants CBE completeness ............................................. 151 5 -1 An expert performs a neurological exam of Vic, a virtual human patient with double vision due to CN6 palsy. .......................................................................... 177 5 -2 The cardinal eye movements of a normal, unaffected eye ................................. 177 5 -3 Cardinal movements with the left eye affected by CN3 palsy ............................ 178

PAGE 15

15 5 -4 Cardinal movements with the left eye affected by CN6 palsy. ........................... 179 5 -5 The cardinal vectors for normal, CN3, and CN6 eye movements are graphed as (yaw, pitch) pair associated with each axis .................................................... 180 5 -6 A) Testing the pupillary reflex with the ophthalmoscope. B) Performing the fundoscopic test with the ophthalmoscope. ........................................................ 181 5 -7 Visual acuity test with the virtual eye chart. ........................................................ 181 5 -8 The finger counting test ........................................................................................ 182 5 -9 Checking the eye movement of a virtual human patient with a left eye affected by CN6. ................................................................................................... 182 5 -10 Testing facial sensitivity by touching the virtual humans face. .......................... 183 7 -1 The breast history portion of the procedural ch ecklist is displayed above the virtual humans head ............................................................................................ 220 7 -2 The visual inspection portion of the procedural checklist expands to show the three poses required for visual inspection ........................................................... 221 7 -3 The procedural checklist also incorporates feedback to aid in the cognitive task of recalling which peripheral areas of l ymph nodes should be examined .. 221 7 -4 Thought bubble feedback when the learner responds with empathy ................. 222 7 -5 Thought bubble feedback when the learner responds inappropriately .............. 223 7 -6 The touch map provides feedback of coverage and use of correct palpation pressure ................................................................................................................ 224 7 -7 The pairing of the color and infrared seeing cameras and the haptic interface to the virtual human. ............................................................................................. 224 7 -8 A) The boundary of the area required for complete coverage of the breast cone. B) Complete coverage is indicated when this area is filled. .................... 225 7 -9 Informal correctness of the model is demonstrated by showing that the output of the model fits the expected progression of pressure levels ........................... 225 7 -10 The color of the feedback provides guidance, reinforcement, and correction of the learners palpation pressure thr ough real -time changes in color ............. 226 7 -11 A learner follows an experts vertical strip patternof -search. ............................. 226 7 -12 Modeling patternof -search .................................................................................. 227

PAGE 16

16 7 -13 The touch map and patternof -search map for the same exam ......................... 227 7 -15 Feedback is provided concerning cognitive elements such as the procedure of visual inspection and cognitive psychomotor elements such as completeness of p alpation ................................................................................... 228 7 -16 Summary feedback of affective performance...................................................... 229 8 -1 Progression of the H Map visualization as the learner perfor ms the eye movement test. ..................................................................................................... 252 8 -2 T he VSP feedback experience of Raij et al ......................................................... 253 8 -4 The initial implementation of patient vision alpha blended each eyes image to present double vision on a non-stereoscopic display ..................................... 254 8 -5 Study procedure ................................................................................................... 255 8 -6 Physical setup of the study .................................................................................. 256 8 -7 Views during the exam ......................................................................................... 257 9 -1 Procedure for Study MRIPS Learning. ................................................................ 325 9 -2 Participants performance in breast history completeness in the two SP and three MRIPS interactions. .................................................................................... 326 9 -3 Real -time feedback appears to be more effectiv e than the post experiential feedback ............................................................................................................... 327 9 -4 A) Participant drawing of correct locations of the two masses in the MRIPS breast. B) Participant drawing of correct location of the mass in the S P breast. ................................................................................................................... 328 9 -5 Participants use of deep pressure in MRIPS. ..................................................... 328 9 -6 Patterns of -search closely following and significantly de viating from an expert ................................................................................................................... 329

PAGE 17

17 LIST OF ABBREVIATION S CBE Clinical Breast Examination. Refers both to the exam itself and the act of performing the exam. A Clinical breast examination involves conversing with the patient to take a breast history, visually inspecting the patients breasts, and palpating the patients breasts to search for abnormalities. MRIPS Mixed Reality Interpersonal Simulation. Our expansion of interpersonal simulation to incorporate touching of the virt ual human, manipulation of hand-held tools, and real -time feedback of performance. CN Cranial nerve. Refers to one of the twelve cranial nerves.

PAGE 18

18 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy MIXED REALITY INTERPERSONAL SIMULATION AFFORDS COGNITIVE, PSYCHOMOTOR, AND AFFECTIVE LEARNING By Aaron Andrew Kotranza December 2009 Chair: Benjamin Lok Major: Computer Engi neering In fields such as medicine, military, and law enforcement, success in interpersonal scenarios requires mastering a complex group of cognitive, psychomotor, and affective skills. These interpersonal scenarios involve tasks which call upon multiple skill sets concurrently. For example, in a clinical breast exam, the doctor must concurrently recall a pattern of palpation (cognitive) while palpating with correct pressure (psychomotor) and also keeping a patient comforted (affective). The concurrent use of these skill sets mandates that learners practice concurrent actuation of these skills; each skill set should not be learned in isolation. However, traditional education approaches provide a level of practice that is inadequate for many novices to ac hieve competency in the three skill sets and to learn how to concurrently call on multiple skill sets to succeed in the interpersonal scenario. This is due to reasons such as lack of standardization and inability to recreate these interpersonal scenarios with peers or human actors. Interpersonal simulation with virtual humans promises ondemand, highly immersive learning experiences which could augment these curricula to provide learners the additional practice required to achieve competency. For this r eason,

PAGE 19

19 interpersonal simulation is receiving increased attention from early adopters in medicine, military, and law enforcement fields. However, interpersonal simulation has yet to be deployed as part of curricula in these fields. We have identified two shortcomings of current approaches to interpersonal simulation: the absence of touch which contributes to training of all three skill sets, and a lack of evidence that learning and skills transfer takes place in users of the simulations. To address the shortcomings of current approaches, we introduce a new approach to interpersonal simulation mixed reality interpersonal simulation (MRIPS). MRIPS incorporates passive and active haptic interfaces instrumented with sensors to afford touch between human and virtual human as well as the manipulation of handheld tools. Thus, in addition to bidirectional verbal and gestural capabilities of traditional interpersonal simulation, MRIPS provides touch from human to virtual human, touch from virtual human to hu man, and touch from hand -held tools to virtual human. These touch inputs and outputs are used to affect the psychomotor, cognitive, and affective components of the simulated interpersonal scenario. The incorporation of these haptic interaction capabilities in MRIPS addresses problems with prior approaches to interpersonal simulation, expanding the applicability of interpersonal simulation to training cognitive, psychomotor, and affective skills and their concurrent use: 1 Problem: Psychomotor task component s involving touch could not be simulated. a Approach: MRIPS incorporates touch from human to virtual human and from handheld tool to virtual human to afford the simulation of psychomotor task components involving touch.

PAGE 20

20 2 Problem: Prior interpersonal simu lation approaches do not afford touch for communication between the human and virtual human. Touch is an essential component of communication between two humans and the lack of touch results in incomplete communication in these interpersonal simulators. a Approach: By incorporating touch from human to virtual human and from virtual human to human, MRIPS affords interpersonal communication to accomplish both cognitive task components, e.g. achieving compliance in getting a patient to assume a specific pose for a physical exam, and affective components, e.g. a comforting touch. 3 Problem: Prior approaches to interpersonal simulation afforded only simple vision based gesture interfaces, e.g. pointing and iconic gestures or encumbering gesture interfaces, e.g. bo dy suits. a Approach: MRIPS incorporates instrumentation, e.g. six degreeof freedom tracking, of the haptic interfaces. This affords simulation of handheld tools for performing complex psychomotor task components not involving touch. These tools also serve as grounding objects to enhance communication between the human and virtual human 4 Problem: Feedback and reflection motivated by feed back is necessary for learning Prior interpersonal simulations could not provide feedback of psychomotor performance without encumbering interfaces, e.g. gloves, body su its or expert observers a Approach: MRIPS incorporates instrumentation of the nonencumbering haptic interfaces for touching the virtual human and for tool manipulation. This affords quantitative m easuring of the learners psychomotor, cognitive, and affective skills performance as well as real time feedback to guide and elicit reflection on the learners performance. By providing real time feedback of learner performance, MRIPS increases the potential for learning in the simulated interpersonal scenario. We applied MRIPS to simulate two interpersonal scenarios in medicine the clinical breast exam and the neurological exam which could not be simulated through prior interpersonal simulation approaches. User studies established the validity of MRIPS for practicing and evaluating learners psychomotor, cognitive, and affective skills. We then incorporated visual feedback of user performance in these three skill sets, to enhance the potential for l earning. Additional user studies were then conducted to determine what learning occurs in users of MRIPS and whether skills learned in MRIPS transfer to the real world scenarios being simulated.

PAGE 21

21 Results show that MRIPS affords psychomotor, cognitive, and affective learning required for success in real world interpersonal scenarios and that skills learned in MRIPS translate to the real world scenarios. This work demonstrates the validity of using MRIPS to train real world interpersonal scenarios and moti vates further incorporation of MRIPS into interpersonal skills curricula

PAGE 22

22 CHAPTER 1 INTRODUCTION Success in interpersonal scenarios depends on one s ability to draw from a complex mix of cognitive, psychomotor, and affective skills. By acquiring these skill sets and mastering the concurrent actuation of these skills, learners attain competency in these scenarios. However, there is increasing evidence that the traditional curricula tasked with teaching these interpersonal scenarios fail, for a signific ant1 percentage of students, in bringing students to competency [ 1 ][ 2 ][ 3 ] Coupled with inadequate opportunities for evaluating students, this results in graduates of these curricula lacking skills necessary to succeed in the interpersonal scenarios being taught [ 4 ][ 5 ]. Instead, competency in these scenarios is achieved through trial -by fire in later apprenticeship stages of education, e.g. the residency following medical school. It is the goal of educators to bring learners to competency before the learners are placed in these real world interpersonal scenarios in which failure carries with it high risks, e.g. of harming a patient malpractice lawsuits The shortcomings of traditional curricula in achieving this goal are due to the limited set of training tools available to educators. Lectures, practice with peers and actors, and purely physical simulation (e.g. anatomical models) all have qualities that limit their availability and effectiveness at training concurrent use of cognitive, psychomotor, a nd affective skills [4] [ 6 ][ 7 ][ 8 ] [ 9 ] 1 For the example scenario of the clinical breast exam, the percentage of students graduating without achieving competency v aries by experiment and scenario component: 83% of 4thyear students reported needing additional training in CBE [2]; 65% of 1styear residents failed a standardized patient assessment of CBE performance [3]; graduating students failed to use a correct pat tern of search 55% of the time, failed to perform visual inspection 25% of the time, and found only 40% of masses present [1].

PAGE 23

23 My work seeks to provide educators with an additional tool, one that is autonomous, always available, and able to concurrently train all three skill sets. This tool is mixed reality interpersonal simulation (MRIPS), a novel approach to simulation of interpersonal scenarios that merges haptic interfaces, physical sensing, and virtual humans, and is effective in training the cognitive, psychomotor, and affective components of interpersonal scenarios. MRIPS provides pass ive and active haptic interfaces to a life -sized virtual human agent. The haptic interfaces are instrumented with sensors which provide quantitative measures of manipulation of the haptic interfaces. This allows the virtual human to respond to the manipu lating of these haptic interfaces and affords automated evaluation of learners cognitive, psychomotor, and affective performance. This is a fundamentally new approach to interpersonal simulation with virtual humans the simulation of an interpersonal s cenario by replacing one s interaction partner with a virtual human. Prior approaches to interpersonal simulation have allowed users to communicate with the virtual human using speech (bidirectional verbal communication) and limited gestures (e.g. poi n tin g) and for the virtual human to communicate using facial expressions and gestures [ 10 ][ 11 ]. MRIPS provides a richer, more complete (as compared to human-human interaction) set of interaction capabilities. In addition to bidirectional verbal communication and unidirectional gestural communication from the virtual human, MRIPS provides: Bidirectional touch from human to virtual human and from virtual human to human. Touch is enabled for cognitive and affective aspects of communication as well as for psycho motor task performance. Hand -held tool manipulation including touching of the virtual human using the handheld tools.

PAGE 24

24 Robustly recognized user hand -gestures to communicate with the virtual human and serve as conversational grounding elements. Real -time feedback of the learners performance in cognitive, psychomotor, and affective components of the scenario, including feedback to guide affective aspects of communication, such as expressions of empathy. These affordances of MRIPS uniquely enable concurrent l earning of the cognitive, psychomotor, and affective components of an interpersonal scenario and skills learned in MRIPS transfer to the real world interpersonal scenario. This dissertation presents the development and evaluation of MRIPS in the followi ng progression: 1 Design of haptic interfaces to enable touch and handheld tool use in the interpersonal simulation, as a means of performing the cognitive, psychomotor, and affective components of interpersonal scenarios. Application of these interfaces t o two interpersonal scenarios in the medical domain: a clinical breast exam and a neurological exam with abnormal findings. 2 Evaluation of MRIPS to establish the validity of MRIPS for practicing the cognitive, psychomotor, and affective components of these medical examination scenarios. This evaluation focuses on demonstrating that learners treat the examination of a virtual human using MRIPS similarly to how they treat the examination of a human patient. Learners performances in MRIPS are shown to be sim ilar in quality to their exams of human patients. Additionally, learners of different levels of experience with the real world interpersonal scenario are able to perform in MRIPS in a manner consistent with their experience level. These evaluations establish the validity of MRIPS as an additional practice opportunity to augment underserved aspects of traditional curricula. Establishing this validity sets the groundwork for evaluating learning within MRIPS and training transfer of skills learned in MRIPS to real world interpersonal scenarios 3 Enhancement of the educational capabilities of MRIPS through the addition of novel real -time and post experiential feedback of quantitatively measured learner performance. Feedback has been shown to be necessary for l earning in the scenarios to which MRIPS is targeted [ 12 ]. The novel feedback capabilities of MRIPS are designed to elicit reflection and result in improvement of cognitive, psychomotor, and affective task components. We present results of user studies wh ich demonstrate that this feedback improves learners performance in these three skill sets. 4 Evaluation of the efficacy of MRIPS for cognitive, psychomotor, and affective learning, and the training transfer of learned skills to the real world interpersonal scenario.

PAGE 25

25 1.1 Motivation: Enhancing Educational Methods for Teaching Interpersonal Scenarios Interpersonal simulation is the recreation of a real world interpersonal scenario by substituting a virtual human for one s interaction partner. Interpersonal si mulation has been proposed to train high-stakes interpersonal scenarios with the goal of bringing novices to a level of competency that will allow them to succeed in the real world scenario. Examples of high -stakes scenarios are medical examination and mi litary negotiation. These scenarios are characterized by the quality that failure is both unacceptable and highly likely without having previously achieved competency [ 10][ 13 ]. Interpersonal simulation is well suited for practicing these high-stakes scen arios because it presents a learning environment in which it is acceptable to fail [ 14 ]. This quality allows learners to potentially undergo the progression of failure, feedback, reflection, and improvement which is critical to attaining competency [ 12], before being exposed to the real world scenario. As a result, practicing with interpersonal simulation is expected to provide learners with an increased chance of success in the high-stakes real world scenario [ 13][ 15 ]. The driving force behind using interpersonal simulation to practice these scenarios is the finding that learning these high -stakes scenarios is underserved by traditional educational methods [ 4 ][ 12]. Traditional approaches to interpersonal scenario education involve lecture or book based l earning, followed by a learning by doing ex perience. This experience is a re -creation of the interpersonal scenario with a peer or human actor [ 16 ]. Interpersonal simulation seeks to recreate these experiences with a virtual human in place of the peer or human actor. The affordances and drawbacks of each approach are listed in Table 1-1.

PAGE 26

26 1.1.1 Peer S imulation Simulating an interpersonal scenario using a peer as the interaction partner affords practice of communication, including the cognitive component s of progressing through a set of questions to ask or topics to converse about and responding correctly to the peer, psychomotor components such as touch for communication and physical examination, and affective components such as perspective taking and ex pressing empathy. However, using peers to practice psychomotor components involving touch is limited to a set of scenarios for which touch of another is ethical and does not make either participant uncomfortable. While psychomotor tasks such as suturing a simulated wound on a suturing pad worn by the peer (as used in [ 7 ]) can be practiced, touching of the peer is not acceptable in intimate examinations, especially in mixed gender peer pairs. Surveys of medical students have found that a majority of stud ents of both genders are uncomfortable with having intimate exams performed on them by a peer [ 17 ][18 ]. For these high-stakes interpersonal scenarios, practice with peers has largely been replaced by practice with human actors [ 6 ]. Even for non -intimate exams, practice with peers was rated as the fourth most effective approach to learning physical examination in a survey of 83 American medical schools. Peer learning was surpassed by exams of standardized human patients (hum an actors) and real patients a s well as observing expert exams of human actors [ 6 ]. Another drawback of practicing with peers is that the experience is often (~33% of the time)2 not treated seriously, especially in embarrassing and awkward medical 2 There is no widely reported measure of how seriously role playing with peers is taken. From a limited sample of published stud ies, roleplaying with peers is a negative experience for roughly one third of

PAGE 27

27 physical exam scenarios. This causes the affective components of the scenario to not be effectively practiced [ 8 ]. Other shortcomings of this approach are that abnormal findings and cultural differences can not be simulated. This limits the set of scenarios to which the peer approach can be applied [ 19 ]. Abnormal physical findings, specific to the medical domain, are physical abnormalities or pathologies present in the interaction partner, e.g. a lazy eye. Practicing interpersonal scenarios with a partner of a different culture is a goal o f cultural competency training. Examples include addressing cultural differences between European -American and African-American patients [ 20 ] and dealing with non native English speakers [ 21 ] or criminal suspects with a mental handicap [ 22 ]. Simulation o f abnormal findings and cultural differences with peers is difficult simply because peers with the desired abnormalities and peers from diverse cultural backgrounds are rarely available. With respect to cultural differences, this is partially due to the d emographics of medical students. From 2006 to 2008, more than 64% of matriculants of United States medical schools were Caucasian, with Asians (both East and South Asian) the next largest group at just under 20% [ 23 ]. When culturally diverse peers are av ailable, the use of these peers in physical examination scenarios is further limited due to cultural and religious beliefs [ 19 ]. The drawback of peer simulation for teaching cultural competencies is readily apparent in situations in which the medical stud ent (peer) population has a different cultural makeup than the surrounding patient population. For example, at our collaborating institution, the Medical College of students. Commonly reported reasons for the negative experience are a lack of realism and inability or unwillingness to become engaged (i.e. take the experience seriously). One study reported that roleplay with peers is the teaching method least preferred by 32% of novice medical students [24]. Another study in support of role play reported that 22% of participants had prior negative roleplay experiences; however, this number actually represented 33% of participants who had prior role play experiences [8].

PAGE 28

28 Georgia, the majority of the student population is Caucasian, while the majority of the po pulation of the surrounding city and county is African-American [ 25 ]. The military domain is one in which peers are actively recruited for cultural competency training [ 26 ], e.g. the recruitment of native Arabic speakers as peers for training scenarios including negotiating with civilians. Seeking out, compensating, and training an individual to serve as an interaction partner describes the human actor approach. 1.1.2 Human Actor S imulation In the human actor approach, an interaction partner is sought w ho possesses a specific set of physical traits. This actor is then trained to portray an individual of a specific group. For example, medical educators might recruit a post menopausal female to portray a patient with multiple risk factors for breast canc er, e.g. family history, smoking, hormone replacement use. In medical education, human actors are known as standardized human patients (SPs). SPs are the gold -standard for providing medical students with practice in medical interview and examination [ 16 ] This approach has many of the same affordances and drawbacks as the peer approach, but provides a more standardized approach. The actor is more standardized than the peer, as the same actor may be used for an entire group of learners, while the peer group typically divides a group pair wise. The actor is also trained from a script developed by educators. The recruiting of actors with specific backgrounds or traits allows cultural differences to be simulated. The experience is taken seriously by lear ners and has been validated in many domains as an equivalent substitution for the real world scenario [ 16]. The actor is compensated monetarily for a specific scenario, so issues related to touching and psychomotor tasks are eliminated, e.g. a female SP m ay have a breast exam conducted by medical students regardless of student gender.

PAGE 29

29 Drawbacks of the human actor approach are largely logistical: finding an actor with specific characteristics, maintaining availability of the actor for an entire group of le arners, and compensating the actor [ 6 ]. Because of the issue of availability, the inability to simulate abnormal findings persists with the human actor approach. It is rare to find a person with the desired abnormality, such as a breast mass, who is willi ng to be subjected to examination by a class of medical students. Some abnormalities can not be simulated with actors because the abnormalities are life threatening or require immediate treatment. An example of this category of abnormal finding is double vision due to a cranial nerve abnormality. Additionally, it is logistically difficult to find actors for certain groups such as the elderly or minorities. It may be unethical to recruit other specific patient types such as children or people with psychological disorders. Some institutions have policies preventing the use of actors for intimate exam scenarios because of privacy concerns [ 6 ]. These logistic issues also extend to issues of standardization. Actors are costly and each actor may be available for a limited set of hours, resulting in a limited number of practice opportunities for learners. The same actor is unlikely to be available for an entire group of learners (each U.S. medical school had an average of ~135 1styear students enroll in 2008 [ 23 ]) An actors performance may vary depending on his or her mood, and some actors will follow the script more closely and act more convincingly than other actors. For these reasons, a group of learners may receive highly varying experiences, making evaluation of the learners performances difficult (e.g. in a medical interview scenario, medical educators would have to normalize learner scores based on

PAGE 30

30 how easily an SP divulged her medical history). Howley provides a review of these issues and prior evaluations of the efficacy of SPs. Even with these drawbacks, human actors are widely used in training medical interpersonal scenarios. Seventy -five percent of medical schools use SPs to teach or evaluate examination skills [ 27 ]. 1.1.3 Interpersonal S im ulation with Virtual H umans Interpersonal simulation with virtual humans was developed to address the drawbacks of the human actor approach. As the virtual human interaction partner is an autonomous agent, the simulation is always available. The virtual human can be programmed to follow a script more closely than a human actor, providing a greater degree of standardization and less variance in the learners experiences [ 14 ]. Virtual humans can also be developed to present abnormal findings such as double vision, a breast mass, or a facial burn, and can be given specific cultural traits for cultural competency training [ 20 ][ 28 ]. For these reasons, interpersonal simulation has received attention from early adopters in fields training high stakes interpers onal scenarios, including medical interview, informed consent, and examination [ 15][ 29 ][ 30 ], mental health [ 31][32] military leadership and negotiation [ 13][ 21 ][ 33 ], and law enforcement training [ 22 ]. However, interpersonal simulation has yet to be incor porated into curricula to train real end-users in these fields. We have identified three main shortcomings of current approaches to interpersonal simulation which may contribute to the lack of curriculum integration: 1) current approaches to interpersona l simulation universally lack touch between human and virtual human, 2) feedback of learner performance is not widely used and is primarily limited to post experiential review (after action review), and 3)

PAGE 31

31 there is a lack of evidence that learning and skil ls transfer takes place in users of existing interpersonal simulations. 1.1.3.1 Current approaches to interpersonal simulation lack tou ch Touch is a crucial part of the cognitive, psychomotor, and affective components of interpersonal scenarios. Touch contributes to both communication and the performance of psychomotor tasks. Within communication, touch is used in both affective and cognitive components. For example, touch can be used to comfort or express an emotional connection (affective) or to inst ruct the interaction partners movement or to achieve her compliance with one s instructions (cognitive) [ 34 ][ 35][ 36 ]. Without providing interfaces for touch between human and virtual human, current interpersonal simulations can not fully simulate communi cation between two humans. MRIPS provides bidirectional touch touch from human to virtual human and from virtual human to human. The addition of touching affords a set of communication modalities more similar to real world interpersonal scenarios. In addition to communicating by touching of one s communication partner, touch contributes to communication by enabling manipulation of other objects in the environment. Manipulated objects serve as a common ground for communication between interaction partners. A common ground is a pool of mutually agreed upon information which serves as a way to ensure that a verbal or nonverbal message intended to be communicated is received by the other communication partner [ 37 ]. For example, in a neurological examinati on scenario, an ophthalmoscope manipulated by the doctor serves as a grounding object. The doctor asks the patient to follow the light with your eyes. Because the patient sees that the doctor is holding an

PAGE 32

32 ophthalmoscope, the patient knows that the light refers to the light of the ophthalmoscope, and the patient follows the ophthalmoscope as it is moved by the doctor. Grounding is especially important in communication between human and virtual human because of the error present in verbal interfaces s uch as speech recognition. Even with hours of speech recognition training, recognition of a users speech is imperfect. In the same neurological exam scenario, if the doctor asks follow the light with your eyes the speech recognition may produce text s uch as follow the lead with your tires (actual output from Microsoft speech recognition SDK 5.1). Without the presence of grounding objects, there is not enough information present in this nonsensical utterance to allow the virtual human to respond appr opriately. However, if the virtual human knows that the doctor is holding an ophthalmoscope with the light turned on, the keyword follow provides enough information for the virtual human to know to follow the position of the ophthalmoscope with his eyes In addition to contributing to communication, touch is used to perform tasks with psychomotor components. These tasks may be purely psychomotor, for example, palpating (touching) a clinical breast exam patients breast with specific movements of the han d. Other tasks in which touch is used may require concurrent cognitive and psychomotor skills. Examples of this class of task include recalling (cognitive) a cultural greeting wi th elements of touch and speech and enacting (psychomotor) this greeting whe n meeting a foreign business client, or palpating (psychomotor) a breast with a hidden breast mass while recognizing (cognitive) whether the tissue feels like fatty tissue, fibrous tissue, or an abnormality. We refer to these as compound tasks. These com pound tasks require the use of touch to perform a psychomotor task concurrently

PAGE 33

33 with a cognitive and/or affective task. An example of a compound psychomotor affective task is comforting a nervous patient during a physical examination. Compound tasks may draw on all three skills sets, for example suturing an injured soldier while keeping him calm while instructing the soldier in how to apply pressure to the wound. Novices typically perform poorly at compound tasks because they have not attained competency in all three skill sets and have not had adequate practice actuating all three skill sets concurrently [ 38]. Current approaches to interpersonal simulation can not provide the necessary practice to achieve competency in these joint tasks, as they lack in terfaces that afford touch between human and virtual human. Thus the set of real world interpersonal scenarios to which current interpersonal simulations can be applied is limited by the lack of touch. A main innovation of this dissertation is the design of haptic interfaces to life -sized virtual humans which enable touching of the virtual human. These interfaces expand on the capabilities of prior approaches to interpersonal simulation, affording touch for enhancing communication and performing psychomot or and compound tasks. The design of these haptic interfaces is explored in the implementation of the breast exam and neurological exam simulations in Chapters 3 and 5. 1.1.3.2 Feedback of learner performance is limited There are few instances of feedback of learner performance in existing interpersonal simulations. Existing approaches at providing feedback are typically limited to post experiential feedback, e.g. the after action review system of Raij et al. [84]. The existing approaches at providing feedback of performance in interpersonal simulation are discussed in Section 2.2.3. Prior work investigating real world interpersonal scenarios such as medical physical examination has found that learners

PAGE 34

34 prefer real -time feedback to post experiential feedb ack. Additionally, feedback should be coupled to learner actions, i.e. learner actions should elicit changes in the feedback in real -time [12] MRIPS expands on existing approaches to providing feedback of learner performance in interpersonal simulation by: 1) providing both real -time and post experiential feedback; 2) providing feedback that guides, reinforces, and corrects performance in cognitive, psychomotor, and affective performance; and 3) coupling real time feedback to learner actions, e.g. in the clinical breast exam, palpating the patients breast results in visual changes in the area of the breast palpated, to indicate correctness of the pressure used in palpation. 1.1.3.3 Learning and training transfer have not been demonstrated in current appr oaches to interpersonal simulation. In addition to lacking touch, the efficacy of current approaches to interpersonal simulation to train the cognitive, psychomotor, and affective components of real world interpersonal scenarios has not been evaluated. T he goal of interpersonal simulation is the transfer of learned skills to real world interpersonal scenarios yet this has not been demonstrated. In the literature, there are instances in which interpersonal simulation has elicited short -term changes in user behavior (discussed in Section 2.1.2). However, I was not able to find any published instances of learning the skills required for a real world interpersonal scenario using interpersonal simulation, or improvement in a real world interpersonal scenar io due to practice with interpersonal simulation. A main innovation of this dissertation is to determine whether learning occurs as a result of practice with mixed reality interpersonal simulation. We specifically evaluate whether a novice learners cogni tive, psychomotor, and affective skills improve with

PAGE 35

35 repeated use of MRIPS, and evaluate whether skills learned in MRIPS transfer to the real world interpersonal scenario. This is explored in Chapter 9. 1.2 Motivation: Augmenting Education of Medical Inte rpersonal Scenarios Underserved by Current Educational Approaches We are motivated to choose applications for MRIPS which: 1) require touch for cognitive, psychomotor, affective, and compound task components; 2) require concurrent use of skills from these three skill sets; 3) are underserved by existing educational approaches; and 4) may provide broader societal benefit as a result of enhancing educational approaches. Along with collaborators at the Medical College of Georgia and University of Florida Colle ge of Medicine, we have evaluated two areas of medical education that can benefit from MRIPS: intimate physical examination and physical examination with abnormal findings. Although both applications are medical physical exams, each has distinct sets of cognitive, psychomotor, and affective components and distinct uses of touch. 1.2.1 Enhancing Intimate Exam E ducation Intimate exams are physical exams of intimate areas of the patient, e.g. female breast exams, male and female pelvic exams, and digital re ctal exams. Due to their intimate nature these exams are anxiety provoking for both the patient and healthcare provider [ 38 ]. Because of this, affective components such as understanding the patients emotional state and empathizing with the patient are especially important. The physical examination itself is also especially difficult to learn and perform in the context of the stressful, anxiety -pro voking patient encounter. The combination of an anxious novice learner with the difficult exam (e.g. 76% of experienced clinicians feel they need

PAGE 36

36 to improve in breast examination [ 4 ]) causes novices to be unable to communicate effectively while simultaneously performing incomplete and incorrect exams [ 38 ]. We have focused specifically on clinical breast exam ina tion (CBE), as practice and evaluation of CBE skills can benefit from the unique combination of haptics and sensing provided by MRIPS. A critical component of CBE is palpating the breast at three distinct levels of pressure. The correctness of this palpation pressure can only be determined with sensing [ 39 ]. In our application of MRIPS to simulating CBE, MRIPS CBE, MRIPS is able to provide realistic feeling simulated breast tissue and sensing of correct examination skills including correct palpation pres sure. A unique aspect of MRIPS is to provide these affordances within the broader context of the affective and cognitive tasks of communicating with a patient. 1.2.1.1 Intimate exams require touch for cognitive, psychomotor, and affective task components and require concurrent use of these three skill sets All intimate exams rely on touch extensively for communication and psychomotor task performance. As intimate exams are anxiety provoking for the patient (and the practitioner), touch is commonly used t o comfort the patient [ 35 ]. Touch is also relied upon for attaining patient compliance and for instructing the patient, e.g. in clinical breast examination, to assume specific positions for visual inspection and palpation [ 34 ][36 ]. Specific to the clinic al breast exam, scenario components can be classified as cognitive, psychomotor, affective, and compound cognitivepsychomotor. These components typically take place concurrently. Cognitive: 1) recalling of a series of questions to assess the patients ri sk of breast cancer and 2) thinking of appropriate verbal and nonverbal responses to patient statements and questions.

PAGE 37

37 Psychomotor: palpating the patients breast in circular motions at three levels of pressure, superficial (low), medium, and deep (high), without palpating at a toohard level of pressure that may cause the patient pain. Affective: 1) keeping track of the patients verbal and nonverbal cues as to her emotional state, and 2) in response to patient emotions, alleviating the patients anxiety addressing her concerns, and expressing empathy when appropriate. In order to foresee when empathy is appropriate, learners must gain some understanding of the patients perspective by engaging in perspective taking [ 40 ][41 ]. Cognitive -psychomotor: Recalling of a procedural pattern, the patternof -search, in which the breast should be palpated and maintaining this pattern of palpations. While progressing through a series of palpations, recognizing which areas of the breast remain to be palpated to ensur e palpation of the entire breast. At each palpation, interpreting whether the breast tissue feels like normal tissue or an abnormality. Other compound tasks: These include affectivepsychomotor and cognitivepsychomotor tasks such as keeping the patient c omforted during anxiety -provoking moments such as visual inspection of the patients breasts, opening the patients gown, and palpating the breast. 1.2.1.2 Intimate exam training is underserved by existing educational approaches Educators note that clinic al breast examination is difficult to learn and teach due to novice learners anxieties, the complex set of task components, and the difficulty in providing feedback of learners performance [ 9 ][38 ]. Typical approaches to practicing other physical exams, such as practice with human actors (standardized human patients), can not be provided frequently enough3 and do not provide the precise feedback (e.g. of palpation pressure, pattern of search) needed to achieve competency. SPs provide high -level objective ratings of cognitive and psychomotor performance (e.g. did the student palpate the cone of the breast, yes or no, did the student perform a complete review of systems, yes or no) and subjective rating of affective performance 3 At many institutions, including those we have partnered with to evaluate MRIPS, standardized patients are used primarily to evaluate learners, typically once at the end of the first year and then again before graduation.

PAGE 38

38 (rating the student encouraged the patient to express emotions using a Likert scale). Some SPs are trained to provide more precise feedback, such as a qualitative, subjective determination of whether correct deep pressure was used in palpation. Regardless of the precision, the re is no widely used standardized feedback [ 9 ]. The approach most widely accepted as effective for teaching CBE is to have an expert to observe the novice perform the exam on a human patient and for the expert to provide feedback as to the quality of the novices exam. However, this expert observation can not be provided frequently due to the high demand on the experts time, and some students graduate without being evaluated in performing a CBE. The current approaches to teaching CBE and intimate exam t raining in general allow medical students to graduate without a rigorous assessment of their competency in intimate exams. Without having received adequate practice opportunities and feedback, these graduating medical students express low confidence in th eir intimate exam skills [ 4 ][ 9 ]. Simulation approaches have addressed the shortcomings of traditional educational methods for intimate examination, but have their own drawbacks, which are listed alongside traditional approaches in Table 1-2. Purely physica l simulation, i.e. silicone anatomical models, provides learners with ondemand practice of physical exam skills and exposure to abnormal findings (e.g. simulated breast masses incorporated into the model). However, the ondemand nature of this approach d oes not extend to providing feedback on the quality of learner performance. Feedback is typically provided only when used in peer to -peer teaching sessions incorporated into course curricula. The feedback received from a peer may not be the same quality available from an expert. Notably, purely physical simulation

PAGE 39

39 does not provide the patient context (conversing with a patient) required for practicing cognitive and affective interpersonal skills. Standardized human patients (SPs) provide high-level fee dback (e.g. for CBE, the SP may notify the learner that he has missed the lymph nodes near the breast). Other than real patients, SPs provide the highest fidelity simulation of interpersonal skills. SPs have been validated as a replacement for real patient interactions in this regard [ 16 ]. However, SPs are rarely able to present abnormal findi ngs, e.g. a breast mass or a lazy eye. To provide exposure to abnormal findings, hybrid approaches have SPs wear the silicone anatomical models, such that the mod els appear to be part of the SPs body. This provides exposure to abnormal findings within the patient context needed for interpersonal skills practice [ 7 ]. However, this approach retains the drawbacks of SPs related to availability (Table 1 -1). Pugh et al. have augmented silicone anatomical models with physical sensors to create simulators for female pelvic, rectal, and breast exams. This simulation approach provides students with immediate, objective feedback of their physical exam performance. Howe ver, this approach lacks the patient context needed for interpersonal skills practice. As with all approaches which do not afford practice of the cognitive and affective components involved in these interpersonal skills, this approach does not provide an opportunity to practice the concurrent actuation of cognitive, psychomotor, affective skill sets. Feedback in this physical sim + sensing approach takes the form of a series of meters and charts indicating correctness and completeness of exam technique (Figure 1 1). This presentation was designed to provide experts with

PAGE 40

40 a more detailed (i.e. quantitative) method of evaluating learner performance. However, this feedback is not presented in a form meaningful to novice learner s.[5 ][ 38 ][42 ]. Approaches w hich do not simulate the interpersonal aspects of intimate exams are often used in conjunction with SP interactions. Typically the physical simulator is used to first train the cognitive and psychomotor components of physical examination. This is followe d by SP interactions to train the other cognitive, psychomotor, and affective components of the interpersonal scenario. This separation hampers the learners ability to draw from all three skill sets concurrently, which is required for success in the real world scenario [ 7 ]. MRIPS addresses the drawbacks of each approach. Similar to hybrid simulation, MRIPS provides opportunities to practice concurrent use of cognitive, psychomotor, and affective skill sets, but expands on this approach with the addition of on demand availability and objective, quantitative, more precise feedback of student performance. Expanding on Pughs approach of incorporating sensing with physical simulation [ 5][ 38 ], MRIPS also simulates interpersonal components of the scenario, and provides more detailed feedback in a form expected to be more meaningful to novice learners (e.g. providing color -coded feedback at palpation at the three levels of pressure, vs. providing a meter from 0 volts to 5 volts). A main innovation of this disser tation is the creation of automated, real -time feedback and evaluation of a learners cognitive, psychomotor, and affective performance. Surveys of novice medical students have shown that 70% of students learn best from experiences that combine visual and kinesthetic information [ 43 ]. Expert observation favors auditory learners (a small minority of the population) as these

PAGE 41

41 students can be talked through the exam [ 42 ]. MRIPS provides visual feedback in combination with kinesthetic information from the h aptic interfaces, potentially providing a tool that can educate a large segment of medical (and perhaps other) students underserved by existing traditional and simulation approaches. Of the approaches detailed in Table 12, MRIPS is the only approach whic h simulates both the interpersonal and physical exam components of the intimate exam while also targeting visual kinesthetic learners. 1.2.1.3 Improving intimate exam training has potential for broad social benef it Clinical breast exam and other intimate exams such as the prostate exam are essential components of screening for early detection of several cancers and competence of intimate exams is a critical skill for all healthcare professionals [ 4 ][ 9 ]. CBE may find up to 10% of cancers that are not detec table with imaging techniques such as mammography [ 44 ]. Breast cancer is the most common form of cancer in women and is the second most common cause of cancer death [ 45 ]. Without CBE, approximately 10,000 breast cancers might otherwise go undetected each year [ 4 ]. If MRIPS -CBE is shown to be successful in training CBE and is incorporated into medical curricula and remediation programs, MRIPS -CBE has the potential to enhance early detection of cancers. This can potentially improve the quality of life for thousands of patients. Improving cognitive and affective skills through additional practice with MRIPS has the potential to improve patient outcomes through making better testing and treatment decisions and forging closer (i.e. friendlier, less businessl ike) doctor -patient relationships.

PAGE 42

42 1.2.2 Increasing Exposure to Abnormal Physical Findings The second application area chosen for MRIPS is simulation of abnormal physical findings that can not be simulated using traditional or existing simulation approach es. One such scenario is a neurological exam of a patient with cranial nerve palsy. Patients with cranial nerve palsies present with abnormal physical findings such as: a pupil that does not contract, an eye that does not move through a full range of motion, or double vision [ 46 ]. 1.2.2.1 Training neurological examination with abnormal findings is underserved by existing educational approaches The diagnosis of the cranial nerve palsy is based primarily on interpreting these abnormal findings in the cont ext of the patients medical history [ 47 ]. Currently, medical students learn diagnosis through lecture, textbook, video-based instruction, and supervised patient encounters [ 48 ]. Exposure to abnormal findings in human patients is not standardized and occ urs only if a student happens to be in the neurological clinic at the time that a patient arrives with abnormal findings, i.e. exposure is catch as catch can. Thus medical students may graduate without experiencing abnormal findings in a neurological ex am. Students diagnostic skills are typically tested by an expert observer at most once in a neurology clerkship, and some students go untested. When these skills are tested, students arrive at (what an expert would consider) a correct diagnosis 50% of t he time [ 49]. The lack of exposure to abnormal findings may also hamper affective skills with patients presenting with these abnormal conditions. Patients are often fearful because of social or cosmetic problems (e.g. lazy eye) and because they worry th at they will lose sight completely [ 50 ] and their symptoms may indicate a serious condition such as

PAGE 43

43 aneurys m or brain tumor [ 51 ]. Without the experience of talking to patients with these fears and taking these patients perspective, novice learners may no t address the patients concerns or consider the patients safety (e.g. did a patient with severe double vision drive to the clinic?). It is imperative for the physician to address issues of patient safety [ 52 ]. As with intimate exams, the limited opport unities for practice and evaluation causes students to report a low level of knowledge of the neurological exam and low confidence in their abilities [ 53 ][ 54]. Simulation approaches to providing additional practice of the neurological examination include p urely physical and purely virtual simulation. Purely physical simulation has yet to completely recreate abnormal findings in a neurological examination. Recreation of abnormal findings such as restricted eye movements and partial loss of sensation in th e face would require robotics beyond that used in sophisticated physical simulators such as the Human Patient Simulator [ 55 ] Instead of attempting to simulate a full exam, physical simulation has focused on training narrower components such as diagnosing abnormal findings in the fundoscopic exam (looking at the patients retina through the ophthalmoscope). Slides of photos of abnormal retinas are inserted in the eyes of a mannequin head [ 56 ]. Purely virtual simulation approaches have more completely si mulated the neurological exam and abnormal findings. A purely virtual webbased approach (2D graphics) has been developed by the University of California Davis [ 57 ] to train diagnosis of cranial nerve palsies. A pair of disembodied eyes follows the curso r, providing the user with information to make a diagnosis based on eye movements.

PAGE 44

44 There are currently no simulation approaches which address a complete set of abnormal findings (i.e. eye internal and external appearance, movement, and reaction to light; drooping eyelid, sensation or lack of sensation in the face and eyes; expression of seeing double based on where the eyes are looking) or which simulates the interpersonal aspects of the scenario. Our application of MRIPS to simulating a neurological exam with abnormal findings, MRIPS -NEURO, represents the first attempt to recreate this interpersonal scenario in a manner that affords practice of the cognitive, psychomotor, and affective skills required by the scenario. Users of MRIPS -NEURO are currently able to perform tests required to diagnose disorders of seven cranial nerves : 2, 3, 4, 5, 6, 7, and 12 (there are a total of 12 cranial nerves). 1.2.2.2 Neurological examination requires touch and concurrent use of cognitive, psycho motor, and affective ski ll sets A neurological exam with abnormal findings requires touch (including hand -held tool use) for cognitive, psychomotor, and affective components. These components occur concurrently, requiring concurrent actuation of the three skill sets. The exam components of a neurological exam are many [ 54 ][ 58 ]. Those which are required for diagnosing cranial nerve palsies are: Cognitive: Conducting a medical history. Recalling various tests and interpreting the results of the tests: testing pupillary reflex by shining the light of the ophthalmoscope into the patients eyes, examining the fundus (retina) using the ophthalmoscope, checking for double vision by holding fingers up and asking the patient how many fingers he sees, have the patient read from an eye chart to test visual acuity, shake a finger in the patients peripheral vision to test for peripheral vision disorders, ask the patient to blink, wink, stick out his tongue, turn his head from side to side, and touch his chin to his chest. Psychomotor: Te sting the patients eye movements by moving a finger or light in the shape of an uppercase H with a size and shape that tests the extremes of the patients eye movements: far left, far right, far upper left, lower -left, upper -right, and lower -right. Touching the patients face with a finger to test for loss of sensitivity in the face.

PAGE 45

45 Affective: Alleviating the patients anxiety and addressing the patients concerns, through engaging in taking the patients perspective and expressing empathy when appropr iate. Additionally the user must consider issues related to patient safety [ 52 ]. Although all of the tests performed in the neurological exam have psychomotor components, from discussion with medical experts, two tasks are considered to be more difficult. These are the fundoscopic exam, which requires skilled use of the ophthalmoscope, and the eye movements test. Notably, MRIPS -NEURO does not have the goal of training ophthalmoscope use practice with a real ophthalmoscope and peer is a higher fidelity and more commonly available method for learning how to use the ophthalmoscope. MRIPS -NEURO focuses less on training psychomotor components than MRIPS -CBE, instead focusing primarily on the integration of information gained from the many simple tests and m edical history to arrive at a correct diagnosis. The one psychomotor task which MRIPS -NEURO seeks to train is learning the H pattern to test eye movements. The size and shape of the H vary with the depth of the users finger to the patients eyes with experience, clinicians learn to perform this exam up close and have memorized the movements that result in the patients eyes moving to the extremes (the six endpoints of the H). MRIPS -NEURO may be able to assist novice learners in memorizing these movements in addition to providing novices with increased exposure to patients with abnormal findings. 1.2.2.3 Increasing exposure to abnormal findings in neurological exams has potential for broader social benefit The majority of patients presenting wit h neurological disorders are not first seen by a neurologist, but instead by a general practitioner or emergency room clinician [ 59]. Up to twenty percent of acute medical admissions are neurological disorders these

PAGE 46

46 patients require immediate treatment. Thus all healthcare practitioners need to be able to recognize the symptoms of a variety of cranial nerve disorders [ 60 ]. By increasing medical students exposure to and practice diagnosing abnormal findings in a neurological exam, MRIPS -NEURO may impr ove patient outcomes in a scenario that is underserved by current educational approaches. An innovation of this dissertation is the application of MRIPS to simulating clinical breast examination and neurological examination with abnormal findings, providin g an additional tool in the educators arsenal for educating future generations of medical students and maintaining the skills of current residents and clinicians. 1.3 Thesis The thesis deals with the novel integration of haptic interfaces and physical sensing with virtual human simulation and real -time feedback to, as a whole, initiate learning of the cognitive, psychomotor, and affective skill sets of an interpersonal scenario and demonstrate improvement in the real world interpersonal scenario being simu lated. Thesis statement: Interpersonal simulation incorporating instrumented haptic interfaces and providing real -time evaluation and feedback of performance improves users scenario-specific psychomotor, cognitive, and affective skills. Skills improvemen t transfers to the real world interpersonal scenarios being simulated, demonstrated as improved performance in the real world interpersonal scenario. 1.4 Overview of Approach To investigate the truth of this thesis statement we focused on innovating techn ology and evaluating its impact on learners, and finally evaluating learning and training transfer: 1 Technological innovation: Two interpersonal simulators incorporating instrumented haptic interfaces were designed: MRIPS -CBE and MRIPS -NEURO.

PAGE 47

47 Evaluation: Tw o user studies (total of 52 participants) were conducted to demonstrate that learners cognitive, psychomotor, and affective performances in MRIPS are indicative of learners performance in the real world scenario. This content validation of MRIPS laid th e groundwork for studying learning in MRIPS. As skill in MRIPS reflects real world skill, we should expect skills learned in MRIPS to transfer to the real world. 2 Technological innovation: Novel real time and post experiential feedback of learners cogniti ve, psychomotor, and affective performance was integrated into MRIPS. This type of feedback is known to be required for learning in interpersonal scenarios [ 12]. Evaluation: User studies were conducted to demonstrate that this feedback positively impacts learners cognitive, psychomotor, and affective performance in MRIPS. Because performance in MRIPS has been shown to be indicative of performance in the real world, skills improvement due to feedback should also result in improvement in the real world sce nario. 3 Evaluation of learning: Learning in interpersonal scenarios takes place as a result of a repeated cycle of practice and feedback [ 12 ]. To evaluate whether learning takes place in MRIPS, we conducted a user study in which learners underwent a treat ment of three MRIPS experiences each including real -time and post experiential feedback. Before (pre-test) and after (post -test) this treatment, the learners were evaluated in the real world scenario being simulated. Improvement from pre-test to post -tes t is taken as indication of learning and training transfer. 1.4.1 Technological Innovation of MRIPS and Application to Medical Interpersonal Scenarios MRIPS -CBE, a simulation of the clinical breast examination was developed. A haptic interface instrumented with force sensors and cameras captures the users touching of the virtual human and other physical objects. This touch is incorporated into a virtual human simulation allowing for performance of the cognitive, psychomotor, and affective components of c linical breast examination on the virtual human. MRIPS -NEURO, a simulation of a neurological examination with abnormal physical findings was then developed. A Nintendo Wii -Remote augmented with external sensing in the form of six -degreeof -freedom pose tr acking provided a haptic interface for touch, handgestures, and hand-held tool use. This interface provided the shape,

PAGE 48

48 feel, and correct kinesthetic information of real handheld tools used in the neurological exam, as well as vibratory force feedback to indicate touching of the virtual humans face. The augmented Wii -Remote also provides a substitute for the users hand, affording a robust (high update rate, low error) gesture interface. 1.4.2 Evaluation of the V alidity of MRIPS for P racticing and Evalu ating C ognitive, Psychomotor, and A ffective T ask C omponents We conducted two observational studies to determine whether MRIPS -CBE elicited real world cognitive, psychomotor, and affective skill performance. The first study demonstrated equivalent performance with MRIPS -CBE and an SP. The second study demonstrated that the learners prior experience with CBE of human patients impacts performance in MRIPS. These studies established the validity of MRIPS -CBE for practicing and evaluating the three skill set s of CBE. 1.4.3 Incorporation or R eal -time F eedback of C ognitive, Psychomotor, and A ffective T ask Performance We were motivated to incorporate real -time feedback into MRIPS by the knowledge that the learning process is driven by feedback of performance [ 12 ] and the finding that immediate, specific, non-judgmental feedback is the most important motivator for sustained learning in medical education and CBE [ 61 ][62 ]. We expect that incorporation of real -time, objective feedback from quantitative measures of t ask performance will thus improve the learning potential of MRIPS. Feedback was created to specifically target the cognitive, psychomotor, and affective task components of the clinical breast and cranial nerve exams (summarized in Table 1 3). For MRIPS-CBE feedback consists of:

PAGE 49

49 Feedback of the completeness and correctness of the exam are given in the form of two visualizations which address cognitive and psychomotor components of the exam. The touch map visualizes the amount of tissue palpated (and amount of tissue remaining to palpated) as well as the correctness of palpation pressure. The pattern of -search map aids in recall of the correct patternof -search and visualizes how well the user is following this pattern (Figure 4-2). A procedural checklist lists the important topics to ask about in the medical history and displays icons representing the positions the patient should assume during visual inspection of the breasts and the areas of the breast that should be palpated during the physical exam (Fi gure 43). This feedback is targeted to help novice learners perform more complete medical histories and exams. Thought bubbles appearing next to the patients head provide feedback of how the users verbal and nonverbal behaviors are affecting the pati ents emotions (comfort, fear, and attitude towards the user). This feedback is targeted to guide the user to better recognize how his actions affect a patients feelings and when and how to better express empathy. MRIPS -NEURO incorporated the thought bub ble feedback as well as two scenario specific visualizations : The H map visualizes the H pattern used to assess whether a patient has a full range of eye movement. In this assessment, the doctor sweeps his finger or a light in the pattern of an upperc ase H. The H map visualizes this pattern, the shape of which is dependent on the depth of the doctors finger from the patients head. This visualization is targeted to aid novice learners in assessing the extremes of the patients vision. If an inc orrect pattern is used, the extremes of the patients vision are not adequately assessed, which may lead to incorrect diagnosis. The patient vision feedback is a simulation of what a person with the cranial nerve disorder sees. By wearing an HMD, the user is able to see the virtual world through the patients eyes and experience the double vision and incomplete range of eye movement experienced by the patient. This feedback is targeted to aid the cognitive task of diagnosis and the affective task of persp ective taking. Evaluation of the feedbacks impact on learner behavior is provided by the evaluation of learning and training transfer (for MRIPS -CBE) and a separate user study that directly evaluated the impact of each feedback component of MRIPS -NEURO Results of this study indicated that the patient vision feedback improved cognitive and

PAGE 50

50 affective performance, and that the H Map feedback improved the efficiency of learners eye movement tests. 1.4.4 Evaluation of L earning and T raining T ransfer We desig ned a user study Study MRIPS Learning to evaluate learning in users of MRIPS, and transfer of skills to a real world interpersonal scenario. MRIPS -CBE was chosen for this study, as historical data from students learning with traditional approaches was av ailable for comparison. Beginning 3rdyear medical students with no prior hands on experience with CBE were chosen as the study participants, as these participants were about to start a womens health clerkship involving performing CBEs of real patients. MRIPS -CBE is targeted for inclusion in the curriculum before this clerkship. The study procedure was as follows: A baseline of participant performance in cognitive, psychomotor, and affective tasks was obtained by having participants perform a CBE of a s tandardized human patient (SP). Participants then performed three CBEs in MRIPS -CBE in which they received real time feedback of cognitive, psychomotor, and affective performance. Post experiential feedback of performance in these three skill sets was al so provided after each MRIPS -CBE interaction. Improvement was evaluated with another CBE of an SP. We chose multiple MRIPS interactions because learning requires repeated experience, feedback and reflection [ 12 ]. Three interactions were chosen because this produced a MRIPS curriculum lasting three weeks, which is of reasonable length to incorporate into a medical school curriculum (in which rotations or clerkships typically last 2 -3 weeks). The MRIPS interactions were spaced one week apart as this is standard in studies of learning from repeated treatments [ 63 ] and if integrated into a

PAGE 51

51 curriculum would allow educators or curriculum overseers adequate time to review a students performance and intervene (e.g. for remediation) if necessary before the next interaction. MRIPS provided automated evaluation of performance through data collected by the haptic and speech interfaces. This was augmented by video review by the experimenter. Expert clinicians and medical educators reviewed video of the SP intera ctions to evaluate participant performance with the SPs. Learning and training transfer was evaluated by analyzing changes in performance from the baseline SP to the evaluation SP and throughout the three MRIPS interactions. To evaluate the impact of rea l -time feedback on performance, participants performance in the first MRIPS -CBE interaction was compared to historical control groups of both novices and experts performing CBEs in MRIPS without real-time feedback. Historical control groups of novice med ical students CBEs of SPs allowed us to investigate the impact of repetitive MRIPS practice and eliminate the baseline SP interaction as a source of improved performance. For many cognitive, psychomotor, and affective tasks, participants improved signific antly throughout the three MRIPS interactions. For other tasks, incremental, but not significant, improvement was demonstrated. However, participants encountered a ceiling effect on three cognitive and psychomotor tasks, likely due to the real -time feedb ack. Generally, participants improved their performance concurrently in all three skill sets during the three practice opportunities afforded by MRIPS -CBE. Training transfer in the form of improvement from the SP baseline to the SP evaluation CBE was dem onstrated for tasks in all three skill sets.

PAGE 52

52 Results of this study also demonstrated significant positive impact of real -time feedback on learner performance. In cognitive and psychomotor tasks, participants in Study MRIPS Learning significantly outperf ormed those in the historical control group not receiving real -time feedback. In particular, Study MRIPS -Learning participants receiving real time feedback of the completeness and correctness of their palpation performed CBEs with expert -level or greater completeness and use of correct palpation pressure. Learning from repetitive use of MRIPS was less clear. Finally, we investigate the impact of having multiple MRIPS -CBE practice opportunities (vs. only one) and the impact of the baseline SP interaction on skills improvement. Compared to one MRIPS -CBE practice opportunity, participants receiving three MRIPS -CBE practice opportunities performed significantly better in cognitive and affective components of a CBE of an SP and non-significantly better in ps ychomotor components. After four weeks of no CBE learning opportunities (approximately the length of Study MRIPS -Learning) any improvements in performance from an SP interaction were no longer retained, demonstrating that assessing participants baseline skills with a CBE of an SP does not contribute to improved performance in the post -treatment evaluation with an SP Instead, all improvements in performance from baseline to evaluation SP interactions were due to learning during MRIPS -CBE interactions and retention of the skills learned in MRIPS -CBE. This study demonstrated that the incorporation of haptic interfaces and sensors to enable simulation of cognitive, psychomotor, and affective components of an interpersonal scenario along with real -time feed back of performance in these components results in concurrent learning of all three skill sets in users of the

PAGE 53

53 interpersonal simulation. Repeated use of the MRIPS elicits skills improvement that transfers to the real world interpersonal scenario. This demonstrates that an interpersonal simulation can make the leap from laboratory test bed to a deployed curriculum component to aid in training the next generation of doctors, soldiers, and other professionals who must perform in high-stakes interpersonal s cenarios. 1.5 Innovations The innovations of this work were in designing and evaluating a novel approach to interpersonal simulation for interpersonal skills training. We designed haptic interfaces instrumented with physical sensing, e.g. force sensors and optical tracking, to afford touch between a human and a life -sized virtual human agent. These interfaces are the first haptic interfaces to a life -sized virtual human. Bidirectional (human to virtual human and virtual human to human) touch is incorpor ated into the virtual humans simulation as a means of performing the cognitive, psychomotor, affective, and compound cognitivepsychomotor tasks of the interpersonal scenario. In its cognitive and affective role, touch provides a means for communicating with the virtual human and for the virtual human to communicate with the user. This is the first incorporation of touch as a means for communication with a virtual human. The haptic interfaces afford psychomotor and cognitivepsychomotor task performance that involves touching of the virtual human and manipulation of handheld tools. The interfaces are designed in a manner that provides kinesthetic feedback consistent with the of the real world scenario being simulated i.e. using these haptic interface s, one can learn the muscle movements to perform the psychomotor task in the real world scenario.

PAGE 54

54 We also designed novel real -time, objective, dynamically generated, in -situ presented feedback of quantitatively measured learner performance. This perform ance feedback was designed to train psychomotor, cognitive, affective, and cognitive psychomotor task components. In user studies of MRIPS CBE and MRIPS -NEURO, cognitive, psychomotor, and affective feedback improved user performance in those three skill s ets. These feedback methods also provide educators with quantitative measurements of user performance in psychomotor, affective, and joint cognitivepsychomotor task components, uniquely allowing educators to automatically, and more precisely evaluate learner performance. Applied to simulating CBE and neurological examination, MRIPS is the first simulation of medical physical examination which targets the visual -kinesthetic learning style favored by a majority of medical students, while also simulating bot h interpersonal and physical exam components of the exam scenarios. Finally, we provide evidence that learning occurs in users of interpersonal simulation and that learned skills transfer to the real world scenario being simulated. Although our study of l earning was confined to the CBE scenario and should be considered only a first pass at evaluating learning and training transfer, it represents a comprehensive attempt to determine whether interpersonal simulation is ready for deployment into real world interpersonal curricula.

PAGE 55

55 Table 1 1. Affordances and drawbacks of prior educational approaches and MRIPS for teaching cognitive, psychomotor, and affective aspects of high-stakes interpersonal scenar ios Approach Cognitive Psychomotor Affective Drawbacks P eer Yes Yes, for a limited set of scenarios: there is no place for peer genital, rectal, or female breast exams in the curriculum [ 17 ] No, not taken seriously by ~33% of students [ 8 ] Can not be used in high-stakes intimate exams, limiting the set of appl icable scenarios [ 6 ]. No abnormal findings and limited cultural differences [ 19 ]. Human actor [ 16 ] Yes Yes Yes Limited availability and set of scenarios [ 6 ]. No abnormal findings [ 7 ]. IPS [ 10 ] Yes Yes, limited to pointing and iconic gestures or encumbered by gloves or body suits [ 10 ][ 11 ][ 64 ] Yes, if taken seriously Psychomotor tasks limited to pointing and iconic gestures No touch [ 10 ][ 11 ]. MRIPS Yes (evaluated in Chapter 5) Yes (evaluated in Chapter 5) Yes, if taken seriously (evaluated in Chapter 5) ** ~10% do not approach seriously. ** Similar drawbacks as IPS with regard to robustness of verbal interaction (Section 2.2.1). (*) In Section 3.4 we present the results of a pilot study which demonstrated that learners affective behaviors in MRIPS wer e more similar to a human patient interaction than were learners affe ctive behaviors in IPS [82 ]. (**) Section 4.2 presents evidence that if treated seriously, learners affective performance in MRIPs is equivalent to that with human actors. However thr oughout the evaluations of MRIPS, we have found that a small percentage (<10%) do not approach MRIPS seriously.

PAGE 56

56 Table 1 2. Traditional and simulation approaches to teaching intimate exams Approach Interpersonal aspects simulated? Exposure to abnormal find ings? Feedback? Learning requires real time and post experiential [ 12 ] Learning style emphasized: SPs [ 16 ] Yes No Real -time: affective, performance (social, auditory) Post experience: completeness of interview and exam (auditory) Kinesthetic Expert obser ver [ 12 ] Yes By chance (catch as catch can) Real time: correctness of exam technique (auditory) Kinesthetic, auditory [ 42] Physical sim [ 65 ][66 ] No Yes Only if used in peer group learning. Real -time: correctness of exam technique (auditory) Kinesthetic Hybrid (SP + physical sim) [ 7 ] Yes Yes Real -time: affective performance (social) Post experience: completeness of interview and exam (auditory) Kinesthetic Physical sim + sensing [ 39 ] No Yes Real time: charts and diagrams (visual) Kinesthetic, visual MR IPS Yes Yes Real time and post experience: affective performance, completeness and correctness of exam and interview (social, auditory, visual) Kinesthetic, visual Table 1 3. Feedback in MRIPS Simulation Cognitive Psychomotor Affective MRIPS CBE Touch m ap (coverage) Pattern of -search map Procedural checklist (breast history, visual inspection) Touch map (pressure) Thought bubbles MRIPS NEURO Procedural checklist H -map Patient vision H map Thought bubbles Patient vision

PAGE 57

57 Figure 11. The simulator of P ugh et al. [ 38 ] provides feedback of pressure as a series of meters representing the value at each of the 11 sensors, and shows coverage as lit/unlit dots representing whether each sensor has been touched.

PAGE 58

58 CHAPTER 2 REVIEW OF LITERATURE The goals of this chapter are to impart an understanding of prior approaches to interpersonal simulation and to motivate MRIPS incorporation of touch to enhance the capabilities of interpersonal simulation to train cognitive, psychomotor, and affective tasks. Pri or work relative to our goal of educating interpersonal scenarios through the use of simulation is reviewed. The review of prior work related to the motivation and implementation of specific components of this dissertation is left to these components res pective chapters. In this chapter, Section 2.1 investigates a theoretical foundation for the use of virtual humans for interpersonal simulation. This prior work demonstrated social responses to virtual humans and investigated behavioral changes resultin g from social interaction with virtual humans. Section 2.2 details current approaches to interpersonal simulation with virtual humans, connecting these previous approaches to the novel approach of MRIPS. Section 2.3 motivates the incorporation of touch i nto interpersonal simulation by detailing the many uses of touch in real world interpersonal scenarios. As we have applied MRIPS to medical physical examination interpersonal scenarios, it is useful to be aware of related approaches other than virtual humans. Section 2.4 provides a brief overview of previous and contemporary approaches to simulating physical examination. 2.1 Foundations for Interpersonal Simulation with Virtual Humans This section investigates a theoretical foundation for the use of virtual humans for interpersonal simulation. This prior work demonstrated social responses to virtual

PAGE 59

59 humans and investigated behavioral changes resulting from social interaction with virtual humans. 2.1.1 Social R esponses to Virtual H umans The goal of interper sonal simulation is for a virtual human to effectively simulate a human interaction partner. This goal can not be achieved without first demonstrating that virtual humans are treated as social entities. The prior work detailed in this section has accompl ished this, demonstrating that virtual humans elicit social responses that are consistent with humanhuman interaction in a variety of real world scenarios. A field which has extensively used virtual humans to provoke social responses is the field of virt ual reality exposure therapy (VRET). VRET provides exposure therapy (used to conquer phobias, post -traumatic stress, and other psychological disorders) in a virtual world, allowing the clinician to standardize and control stimuli to which the patient is e xposed. In VRET, real world social fears have been elicited by virtual worlds inhabited by virtual humans, e.g. a virtual audience elicits anxiety in users with a fear of public speaking [ 67 ]. The presence of virtual humans also provided the social stimu li to increase cravings for smoking among users of a virtual environment to treat nicotine addiction [ 68 ]. A bar virtual environment inhabited by virtual humans who directed their attention towards the user elicited anxiety in socially phobic users [ 69 ] Other real world social responses to VHs include social inhibition when being observed by a virtual human during a complex task [ 70 ] and affording a virtual human a similar amount of personal space as afforded to a human [ 71 ]. Social interactions with v irtual humans also elicit behavioral expression of users implicit biases, such as skin tone bias [ 20 ].

PAGE 60

60 It is important to note that these social responses were elicited without extensive communication between human and virtual human. In much of the pri or research, the users actions were not able to impact the virtual humans behaviors (i.e. the virtual human simulation accepted no input from the user). Instead it was the virtual humans appearance or unidirectional verbal and nonverbal communication t hat impacted the users attitudes and actions. This differs from interpersonal simulation, in which verbal and nonverbal interaction between human and virtual human is the focal point. 2.1.2 Toward Changing Human B ehavior The goal of MRIPS is to improve a learners cognitive, psychomotor, and affective behavior in an interpersonal scenario. The prior work covered in this section has shown that human behavior can be altered by interacting with a virtual human. However, this prior work has stopped short of demonstrating learning of real world social scenarios as a result of using interpersonal simulation. Human opinions are known to be altered by the expression of the opinions of their social interaction partners. This phenomenon was demonstrated for virtu al human social interaction partners as well. Zanbaka et al. found that virtual humans presenting an unfavorable argument, e.g. arguing for tuition increases, were able to persuade their human interaction partners to change their attitudes towards the topic [ 72 ]. Further work has shown that a persons attitudes towards a subject can be changed by an interaction with a virtual human. In an effort to pique middle school girls interest in engineering, an interaction with a female virtual human engineer chan ged students attitudes towards engineering. Conversing with the virtual human increased students interest in engineering and performance in math. This was attributed to improving students belief in their abilities [ 73 ].

PAGE 61

61 Beyond the altering of expresse d attitudes, it has been shown that in some scenarios, social interaction with virtual humans can improve human social behaviors. VRET incorporating virtual humans as observers has improved patients management of their fears in the real world scenario being simulated. Repeated delivery of a speech to a virtual human audience, along with anxiety management treatment from a human expert, resulted in decreased anxiety in later speeches to human audiences This work demonstrates that these virtual experienc es can alter behavior. However it did not directly demonstrate training of a skill, as an increase in speech quality was not measured [ 74 ]. The decrease in anxiety demonstrate s a positive impact of a virtual experience on user behavior but was partially due to conditioning and use of breathing exercises. Improvement in the real world scenario was not demonstrated to be a result of interpersonal skills learned in the simulation. Interaction with virtual humans has also been targeted towards teaching psyc homotor skills. Users unfamiliar with South Indian greeting protocols performed greetings more correctly after watching a virtual human perform the greeting than after reading a text based instructional booklet [ 64 ]. This work stopped short of demonstrat ing short -term learning by not measuring an improvement in participants greetings from before to after viewing the virtual human greeting. A similar approach was taken to improving childrens behavior in the cognitive task of determining a safe gap in tra ffic for crossing an intersection on a bicycle. Childrens bicycle riding behavior was influenced by a virtual peer. When riding with a virtual peer who made unsafe crossings through traffic, the child mimicked these unsafe behaviors [ 75 ]. Although the authors assumed this would persist as a change in behavior in real -

PAGE 62

62 world bicycling, the study conducted was not designed to measure learning or behavioral changes. This previous work has shown that social interactions with virtual humans have the potentia l to alter human behavior, at least in the short -term and in the presence of the virtual human. Our work goes beyond this to investigate improvement in specific real world interpersonal scenarios as a result of practicing these interpersonal scenarios wit h a virtual human. 2.2 Interpersonal Simulation with Virtual Humans MRIPS expands on prior approaches to interpersonal simulation by incorporating touch as a means to perform cognitive, psychomotor, and affective components of interpersonal scenarios. Thi s section describes the prior approaches to interpersonal simulation and aspects of these approaches that MRIPS expands upon. 2.2.1 Current Approaches to Interpersonal Simulation We have defined interpersonal simulation as the recreation of a real world in terpersonal scenario which replaces a human with a virtual human confederate. The primary goal of interpersonal simulation has been to train users for corresponding real world interpersonal scenarios. For this reason, many current approaches to interper sonal simulation have strived to provide natural interfaces to interact with the virtual human. Typically the virtual human is presented at life -size in a head -mounted display (HMD) or on a large screen or projected display. Users are able to communicate with the virtual human through natural speech, which is processed by speech recognition software and the resulting text matched to a database of phrases understood by the virtual human. When the user

PAGE 63

63 utters a phrase in this database, the virtual human replies with speech, either prerecorded or text -to -speech, and gestures and facial expressions [ 10 ] [13] [ 15 ] [21] [ 31 ][ 33 ]. The primary communication modality in these simulations is speech. Speech recognition rates vary from user to user (e.g. due to accents and speaking style long utterances and mumbling are two qualities which result in poor speech recognition). Additionally, speech matching rates vary based on the matching approach, e.g. 60-70% [ 15 ], 53 62% [ 76 ], 75% [77 ]. These figures may include a significant number of false positives (incorrect matches which trigger virtual human responses which appear to the user to be on-topic but may not convey the same information as the correct response). Artstein et al. report 33% false positives using th e matching approach of Leuski et al. [ 76 ][78 ]. These interpersonal simulations are meant to be autonomous, and their imperfect speech interfaces negatively impact the usability of these systems in an autonomous fashion (i.e. the user might get stuck because the virtual human can not understand his question). For these reasons interpersonal simulations which rely primarily or solely on speech as the means for communication with the virtual human are often augmented by a human controller to evaluate the pot ential of the system to simulate a specific scenario. This behind -the -scenes wizard of oz, is able to manually trigger virtual human responses if speech recognition or matching fails, e.g. [ 14 ]. In this work, we used a wizardof oz to augment speech re cognition when investigating whether MRIPS has the potential to elicit real world psychomotor, cognitive, and affective behaviors (Chapter 5). In these user studies, the wizard of -oz triggered the virtual humans response if speech recognition failed for a users question. However, when evaluating MRIPS for learning and training transfer, verbal communication relied

PAGE 64

64 solely on the automated speech matching approach of Johnsen et al. [ 15 ]. With respect to the efficacy of speech interfaces, we have also obs erved that the incorporation of touch and tool use as additional interaction modalities appears to mitigate user frustration in the presence of poorly performing speech recognition and matching. A small number of interpersonal simulation approaches allow t he user to communicate nonverbally with the virtual human using a small set of simple gestures. Users of a medical interview simulation wore a glove augmented with optically tracked fiducials on their dominant hand. This glove allowed the user to point t o a spot on the virtual humans body and ask does it hurt here? [ 10]. Cassell et al. used position data of the users hands and head, captured using unencumbering vision based tracking, as inputs to Rea, a virtual human real estate agent. Gesture input was limited to detecting when the user was moving his or her hands, indicating that the user wanted to speak [ 11 ]. In an evolution of this system, Gandalf, a cartoonish virtual character who served to guide users through a virtual solar system, was able to recognize iconic gestures, e.g. holding a hand up to signal stop, in addition to pointing, by having users wear a suit incorporating orientation and position sensors on the hands and chest [ 79 ]. Although not used as inputs to the simulation, users wh o practiced cultural greetings with a virtual human wore position orientation sensors on the hands, waist, and head to record their body movements during the greeting [ 64 ]. These prior approaches to incorporating nonverbal communication into interpersonal simulations have primarily relied on encumbering tracking devices (e.g. gloves, wires, headbands, body suits) and have enabled only simple iconic gestures to be input to the virtual human. My work incorporates unencumbered, robustly -tracked

PAGE 65

65 haptic inter faces for hand held tool manipulation for handgestures and accomplishing psychomotor tasks not involving touch. Universally, prior interpersonal simulators lack a natural means of touching for hands on psychomotor interaction. For this reason, interpers onal simulation applied to scenarios requiring psychomotor tasks (with or without touching) has relied on unnatural keyboardandmouse interfaces. To apply interpersonal simulation to scenarios requiring tool use, touch, or complex gestures, nonnatural mouse-keyboard interfaces were used in addition to the natural speech interface. This approach has been applied to scenarios such as clinical examination and battlefield operations [ 30 ]. The portion of the scenario requiring touching or other psychomotor tasks is conducted separately from the portion of the scenario involving only verbal interaction. This separation of components by interaction modality prevents touching from being a mechanism for nonverbal communication with the virtual human and makes the simulation dissimilar from the real world scenario. Additionally, use of the unnatural mouse keyboard interface for touch negates the possibility of training psychomotor components of these interpersonal scenarios, as different muscle movements are us ed with the mouse and keyboard than are used in the real world scenario. Expanding on the affordances of these prior approaches to interpersonal simulation, MRIPS incorporates natural interfaces for verbal, gestural, and touch interaction with a life -sized virtual human. The incorporation of haptic interfaces and sensing of user manipulation of these interfaces affords concurrent use of speech,

PAGE 66

66 gestures, and touch as means for communication (cognitive and affective) and performance of psychomotor tasks required of the interpersonal scenario. 2.2.2 Existing Interpersonal Simulators This section describes the application of prior interpersonal simulation approaches to the fields of military, law enforcement, and medicine. Evaluation of these simulators has t ypically been restricted to face validity or usability, though the medical history simulator of Johnsen et al. has been validated for evaluating medical students cognitive history taking skills [ 81 ]. Within the military domain, i nterpersonal simulators ha ve been employed to train end-users communication skills in military crisis management [ 13 ] and negotiation [ 33 ] scenarios. The simulation which addressed these scenarios is the Mission Rehearsal Exercise system (MRE) developed by the University of South ern California Institute for Creative Technologies (ICT) The MRE allows a user to communicate verbally using natural speech, with virtual humans playing the roles of allied s oldiers and foreign civilians. Bridging military and medical domains, USC ICT has also developed male and female virtual humans, Justin and Justina, who simulate patients with conduct disorder (Justin) and post traumatic stress disorder (Justina). Preliminary testing with mental health clinicians have established face validity of this interpersonal simulation to provide novice clinicians exposure to these mental health conditions [ 31 ] [ 32 ] Within the medical domain, Johnsen et al. created IPS (The Interpersonal Simulator). The goal of IPS is to train health profession students communication skills in medical interviews [ 10] In IPS, the user converse s with a life -sized virtual human patient who responds to user speech with pre-recorded human speech, gestures, and

PAGE 67

67 facial expressions Additional nonverbal input was provided through a simple gesture interface limited to recognizing pointing [ 10 ]. IPS does not provide any mechanism for touching the virtual human. Users of this simulation treated the virtual human patient similarly to a standardized human patient, but were not sin cere in their use of empathy and had di fficulty achieving rapport [ 80 ]. However, this interpersonal simulation was validated for evaluating novice medical students interview skills, as students performance with the virtual human predicted their performance with a standardized human patient [ 81 ]. The speech recognition and matching capabilities and virtual human appearance of MRIPS are directly based on the simulation of Johnsen et al. [81] With the incorporation of touch as a means for communication, we have observed improvement in the frequency of users empathy towards the virtual human [ 82 ]. Also in the medical domain, smaller than life -sized virtual humans were incorporated in simulations for practicing clinical examination skills [ 30 ] and informed consent interviews [ 29 ]. These systems used a natural speech interface in addition to a mouse andkeyboard interface for performing nonverbal components of the scenarios. The same approach was taken to simulating police interviews of mentally ill suspects [ 22 ]. While these systems have received various degrees of evaluation, none have been evaluated for learning or improvement in the real world scenario being simulated. In this dissertation, we take this next step, evaluating what (if any) learning occurs in MRIPS and whether skills learned in MRIPS transfer to the real world.

PAGE 68

68 2.2.3 Incorporation of Feedback in Interpersonal Simulation In real world interpersonal scenarios, receiving feedback during and after one s performance is necessary for learnin g [ 12][61][62 ]. Interpersonal simulations are thus beginning to be augmented with real time and post experiential (i.e. After Action Review) feedback. MRIPS also incorporates real -time and post experiential feedback of a learners performance. The feedb ack provided in MRIPS draws on the work described in this section to innovate new approaches to providing feedback of cognitive, psychomotor, and affective components of learner performance. In the Virtual Environment Cultural Training for Operational Read iness (VECTOR) simulation by CHI Systems Inc, soldiers negotiating with virtual Iraqi civilians receive feedback of the virtual humans emotional state in real -time, displayed textually, e.g. Neutral or Anxious appears over the characters head and changes depending on the users actions [ 21 ]. A similar approach was taken with USC ICTs ELECT BiLAT, also a cultural competency training system. The virtual humans level of trust of the user is indicated by a onedimensional meter, similar to a completion or loading meter [ 83 ]. No evaluation of this feedback was reported. This feedback guided the users affective and cognitive performance, and is the only instance of real -time feedback of affective performance we have come across in interpersonal simulation. MRIPS takes a different approach to affective guidance. Instead of explicitly stating how well the learner is performing (e.g. 50% trust), the virtual human emits thought bubbles indicating how the quality of the users affective performance might affect a real patients emotions and attitudes. To provide post experiential feedback of cognitive performance, Raij et al. incorporated an After Action Review system into the interpersonal simulation of

PAGE 69

69 Johnsen et al. for practicing medical history taking [ 10 ]. This system allowed users to receive feedback in a self -guided manner, navigating a traditional monitor and-keyboard interface which provided video of the users performance, expert performance, graphs of how users progressed through topics i n the medical history, and visualization of the users head gaze at the virtual human patient. The system aided users in reflecting on their performance, exhibited by changes in self -ratings of performance, and may have motivated users to improve their behavior in future interviews, as users reported they would change, but this was not explored wi th further user studies [ 84 ]. Expanding this approach to affective feedback of performance, Raij and myself built a post experiential feedback system which allowed medical students to relive their breast exam of a virtual human through the eyes and body of the virtual human [ 85 ]. This experience had the goal of improving medical students empathic behavior and perspective taking (the affective components of th e medical interview). The system created by Raij et al. serves as inspiration for a novel feedback experience designed to enhance both cognitive and affective performance in MRIPS (Chapter 8). 2.3 Motivation for Touch in Interpersonal Simulation One of th e primary innovations of MRIPS is to incorporate touch as a means of communication between human and virtual human. We are motivated to incorporate touch into interpersonal simulation due to the important and widespread roles touch plays in communication between two humans. Interpersonal touch is a critical component of communication in real world interpersonal scenarios; thus, without affording touch, current interpersonal simulation provides an incomplete simulation of a real world interpersonal scenario. Additionally, the incorporation of touch into both noninhabited (no virtual humans) and collaborative (multiple non-colocated human users

PAGE 70

70 with avatars) virtual worlds has been shown to enhance these virtual worlds believability and usability [ 86 ]. 2. 3.1 The Role of Touch in Communication In clinical, professional, and casual social situations, touch between two humans is used to communicate disagreement, agreement, appreciation, interest, intent, understanding, affection, caring, support and comforting [ 35][ 87 ]. Bidirectional touch between interaction partners elicits more positive attitudes towards the interaction partners than communication through speech and gestures alone [ 88 ]. Even an unexpected touch from a stranger can elicit positive affect t owards the stranger [ 34 ]. Touch is essential in many social interactions, such as those between medical doctors and patients. Caregivers touch patients to communicate reassurance and empathy, convey the idea that the caregiver is helping the patient, and achieve patient compliance, e.g. following of treatment plans [ 36 ][89 ]. Touching the patient improves doctor -patient communication by increasing patient verbalization, self -disclosure, and rapport, and producing positive attitudes towards the caregiver [ 9 0 ][91 ]. Touch from patient to caregiver is used to communicate friendliness, gratefulness, and establish a positive patient doctor relationship [ 92 ]. By incorporating touch into interpersonal simulation, not only is performance of psychomotor task components enabled, but communication between the human and virtual human is made more like communication between two humans. In evaluations of MRIPS, we have observed users touching the virtual human to communicate many of these constructs, e.g. empathy, concer n and comforting; instruction and compliance; and social norms such as touch in greeting [ 93 ].

PAGE 71

71 2.3.2 Touch in Virtual Environments Prior work has demonstrated that adding touch to uninhabited virtual environments and collaborative (inhabited by other humans avatars) virtual environments increases the realism and usability of these virtual environments. This provides a theoretical background against which we have made observations that, with the addition of touch, MRIPS users treat the MRIPS virtual human more similarly to a real human than has been observed in prior interpersonal simulations lacking touch [ 82 ]. Adding haptics to virtual environments has been shown to increase the believability of the experience. The addition of passive haptics to the UNC Pit environment, a stress invoking walk over a virtual chasm, increased users sense of presence, i.e. being there [ 144]. Haptics also increases the sense of co presence, i.e. being with another, in collaborative virtual environments. A passive haptic l azy susan increased remote users sense of being co located around a virtual table [ 94]. Remote users collaborated in a shared Unified Modeling Language editor more effectively when using activehaptic cursors provided by a PHANTOM Omni [ 95 ]. Remote use rs collaborating to move a ring along a curved wire in a virtual world reported a higher sense of togetherness when activehaptic feedback was given than when only visual feedback was given [ 86 ]. Haptic interfaces have also been used to interact with a vir tual human in game like scenarios. An active -haptic interface allowed a user to play catch with a virtual human [ 145]. The passive -haptic interface of a real checkers set allowed a human to play a game of checkers with a virtual human [ 96 ]. Prior to MRIPS, the work that had come the closest to enabling touch as a means for communication with a virtual human is that of virtual interpersonal touch. Bailenson

PAGE 72

72 and Yee proposed virtual interpersonal touch as the touching of a smaller than life -size virtual hu man using an active-haptic interface. An active -haptic interface is one that mechanically applies force to the user, as the user manipulates the interface elements. A study found that participants who used a Phantom Omni force -feedback device to clean vi rtual dirt off of a virtual humans body touched female virtual humans with less force than male virtual humans. This result fit with known results from psychology literature concerning gender effects on interpersonal touch between two humans. However, th e virtual human did not communicate with the user or react to the touch. It is important to note that this was not considered to be an interpersonal touch between a human and virtual human, as the cleaning was not a type of social touch and there was no c ommunication between the human and virtual human [ 97 ] Bailenson and Yees approach of affording touch using currently available activehaptic devices to provide a realistic feel of touch would be logistically and mechanically difficult to extend to touch ing of a life -sized virtual human. For this reason, MRIPS takes the approach of using passive haptics when a large area of the life -sized virtual humans body is to be touched by the user. A passive haptic interface does not mechanically apply force to t he user, but provides the user feedback through the interface elements shape, weight, and texture. Being touched has yet to be explored in interpersonal simulation or social virtual worlds. However, it has been shown that purely virtual stimuli can elici t feelings of being physically touched. These visual touches or pseudo -haptics were explored by Biocca et al. Participants who manipulated a virtual spring, the visual analog of a physical force, reported sensing a haptic resistance [ 98 ]. Pusch et al. f ound that an

PAGE 73

73 illusory haptic sensation was achieved by visually displacing the users hand when placed into a virtual force-field [ 99 ]. My work expands on prior research into the impact of touch by using passive and active haptic interfaces to afford bi -directional interpersonal touch between human and virtual human, providing touch as an additional modality for performing the cognitive, psychomotor, and affective components of interpersonal scenarios. 2.4 Other Approaches to Medical Interpersonal Simulat ion We have targeted MRIPS to simulate medical physical examination scenarios which are underserved in terms of practice opportunities, standardization, and feedback [ 9 ]. Approaches other than virtual humans have been previously used to provide additional practice in medical procedures. These approaches have incorporated computer based simulation but are predominantly physical (i.e. mechanical) approaches. One of the most mature and sophisticated simulation approaches for learning medical procedures is t he human patient simulator (HPS). The HPS is a mannequin simulator that provides a realistic, full -sized human shaped haptic interface to an underlying agent controlling the patients physiological state. The mannequin can breathe, blink, and can be intubated, anesthetized, injected with medication, and can have his blood pressure and heart rate measured. The healthcare provider can communicate verbally with the HPS H owever, communication is not agent controlled. Instead a wizardof oz human controller listens and speaks for the HPS patient. For this reason, the HPS is primarily used to train scenarios in which interpersonal skills are not emphasized, e.g. procedures in which the patient has been anesthetized. Additionally, though realistic, f eedback consists only of feedback that is received during

PAGE 74

74 real procedures, e.g. monitoring of vital signs [55] the HPS does not provide feedback that is unavailable with a human patient, e.g. viewing inside the patient to see how anesthetic flows through the patients system Though the haptic in terface of MRIPS is not as sophisticated, MRIPS enables ondemand (completely automated) practice of interpersonal and physical exam skills, and provides feedback beyond that available in purely physical environm ents, by co -locating visual feedback of performance with the anatomy being examined. Fledgling approaches to medical simulation using robotics are in development. Robotics techniques have recently been applied to create human-form robot patients capable o f verbal communication [ 100 ] and realistic movement. Nonverbal communication of this approach appears limited. The current version of the robot patients nonverbal gestures consist only of exhibiting symptoms T ouch input to the simulation is limited to triggering 1 2 sensors placed at the appendix. Using touch for communication with this approach is certainly possible but has not yet been demonstrated. These predominantly physical approach es encounter difficulties in presenting feedback, and simulati ng abnormal findings and cultural differences. By taking a mixed reality approach that merges the flexibility of virtual human agents for simulating a wide variety of presentations and communication with the psychomotor capabilities of physical simulation, MRIPS captures the best of both worlds.

PAGE 75

75 CHAPTER 3 MRIPS DESIGN PRINCIP LES AND DEVELOPMENT OF MRIPS -CBE This chapter describes the design principles behind a mixed reality interpersonal simulation (MRIPS) and the development of MRIPS -CBE, an interpersonal simulation of a clinical breast examination. This simulation along with pilot study results was published in the proceedings of the IEEE Virtual Reality 2008 conference (best paper) [ 82 ]. An expanded version of this paper incorporating results of a s tudy comparing usability and acceptability of MRIPS -CBE to standardized human patients was published in the IEEE Transactions on Visualization and Computer Graphics journal, May/June 2009 issue [ 101]. Implementation of bidirectional touch and related user study results were published in the proceedings of the IEEE Virtual Reality 2009 conference [ 93 ]. The treatment given in this chapter adds significant detail concerning the cognitive, psychomotor, and affective elements of MRIPS -CBE, the haptic interface implementation, and the virtual human simulation implementation. Collaborators : The speech interface was designed by Kyle Johnsen. The virtual human simulation module of MRIPS -CBE is an expanded version of the virtual human simulation module created by Kyle Johnsen and Andrew Raij [ 10 ]. The appearance of the virtual human was created by Brent Rossen, Corey Forbus, and myself. Medical collaborators Scott Lind and Adeline Deladisma provided medical information pertaining to the clinical breast exam. The haptic interface of MRIPS -CBE is a significantly expanded version of a breast palpation trainer developed by Carla Pugh at Northwestern University. Personal contributions : I developed the current version of the haptic interface, significantly expanded an existing virtual human simulation module, incorporating touch

PAGE 76

76 inputs and outputs into the simulation, developed a framework and hardware for touch outputs (bidirectional touch), incorporated video of haptic interface manipulation into an existing rendering module, and incorporated manipulation of physical objects into the simulation. Relevance to thesis: The thesis states that interpersonal simulation incorporating instrumented haptic interfaces and providing real -time evaluation and feedback of performan ce improves users psychomotor, cognitive, and affective skills in an interpersonal scenario. This chapter describes the design of a mixed reality interpersonal simulation incorporating haptic interfaces and providing real -time feedback (Chapter 6 -7). Th e simulation described in this chapter, MRIPS -CBE, is used to evaluate learning and training transfer in MRIPS (Chapter 9). 3.1 MRIPS Design Principles Mixed interpersonal simulation (MRIPS) incorporates haptic interfaces augmented with real -time sensing i nto an interpersonal simulation augmenting bidirectional verbal communication with bidirectional touch between human and virtual human to enhance communication and train psychomotor tasks. The two main components of MRIPS are a virtual human simulation and a haptic interface. However, MRIPS does more than just merge the communicationoriented virtual human simulation and the psychomotor oriented physical simulation (of the haptic interface). In addition to merging virtual humans and physical simulation, MRIPS also augments both components: the communication affordances of the virtual human simulation are augmented with touch-driven communication (e.g. touch to communicate instruction); the affective affordances of the virtual human simulation are augment ed with the affective use of touch (e.g. a comforting touch); and the psychomotor

PAGE 77

77 affordances of physical simulation are augmented with sensing (e.g. force sensors, cameras) to track manipulation of the haptic interface and extract data of learner performa nce. The result is a simulation of the cognitive, psychomotor, and affective components of an interpersonal scenario. The merging and augmenting of affordances is paralleled by the merging and augmenting of the physical and virtual components of MRIPS. The haptic interface and virtual human are merged through spatial registration, meaning they occupy the same volume in space. Registration conceptually merges the virtual human and haptic interface into a single entity, a mixed reality human, that exists in physical (tangible) and virtual (intangible) forms. The concept of the mixed reality human is extended to the augmentation of physical and virtual components. The visual appearance of the mixed reality human consists of a virtual human augmented by r eal -time video streams of the haptic interface and the users manipulation of the haptic interface. The simulation controlling the mixed reality human provides verbal inputs and outputs and gestural outputs of a virtual human simulation, augmented with touch inputs and outputs. These touch inputs are fed with data read from sensors augmenting the haptic interface; touch outputs drive active haptic components of the haptic interface. These virtual and physical components of the MRIPS design are presented as two interfaces and a simulation which takes inputs and produces outputs at each interface: the visual interface, the physical interface, and the mixed reality human simulation. Taken as a whole, these three pieces of this design simulate the cognitiv e, psychomotor, affective, and joint cognitive-psychomotor components of an interpersonal

PAGE 78

78 simulation. Although the affordances of the components are adapted to the specific interpersonal scenario simulated, a general set of affordances is given here: The physical interface represents the physical embodiment of the mixed reality human. The physical interface incorporates sensing of the users actions and manipulations of passivehaptic and activehaptic interface elements, producing information describing the users touching of the mixed reality human, props, and tools. The affordances of the physical interface are touching of the mixed reality human, props, and tools for cognitive tasks (e.g. touches communicating instruction), affective tasks (e.g. comf orting touches), and psychomotor tasks (e.g. physical examination). The physical interface also affords the mixed reality humans touching of the user, through activehaptic interface elements, for purposes of communication. These affordances can be summ arized as bidirectional nonverbal touch communication, human-to virtual human nonverbal gestural communication, and psychomotor and cognitive-psychomotor task performance. The visual interface displays the virtual human, augmentation with physical interfac e elements and a real -time video avatar of the user. The affordances of the visual interface are bidirectional verbal communication, and virtual human-to -human nonverbal communication (gestures and facial expressions). The mixed reality human simulation t akes inputs of touch, speech, and handheld tool use, and produces as outputs virtual human speech, gestures, facial expressions, and touch. The remainder of this chapter describes the application of the MRIPS design principles to the simulation of the int erpersonal scenario of clinical breast exam. The

PAGE 79

79 implementation of this simulator, MRIPS -CBE, begins in Section 3.3; Section 3.2 first provides background on the clinical breast exam interpersonal scenario. 3.2 Clinical Breast Examination 3.2.1 CBE Proced ure A clinical breast exam has two components which can happen concurrently: the medical interview and the physical exam The medical interview is a 10minute conversation in which the healthcare provider and patient exchange information. Each of the co mmunication partners possesses unique goals for this interaction. The goals of the healthcare provider are to gather key facts of the patients condition (e.g. principal complaint: the patient has found a hard mass in her left breast; family history: her s ister had breast cancer) and to put the patient at ease and develop rapport with the patient. These goals are achieved through asking questions of, and expressing appropriate empathy to, the patient. When interacting with a human patient, interpersonal t ouch is critical for allowing the healthcare provider to develop rapport and express empathy [34] [ 36 ][89] Previous user studies of the interpersonal simulator of Johnsen et al. revealed that the healthcare provider had diffi culty building rapport with a virtual human patient with a breast cancer fear due to the lack of interpersonal touch [ 10 ][80 ]. MRIPS -CBE allows the healthcare provider to communicate both verbally and through touch, and the patient to synthesize verbal and touch inputs, in order to better accomplish these goals. The patient has two main goals: to receive information about her condition, and to be comforted. The patient accomplishes these goals by asking questions of the healthcare provider however, the patients willingness to revea l information and ask

PAGE 80

80 questions is dependent on the rapport achieved with the healthcare provider. Appropriate use of interpersonal touch by the provider can help in this regard [36] [ 89 ][ 90 ]. MRIPS simulates the patients anxieties and desire to find out what may be wrong with her, by verbally challenging the healthcare provider with questions such as do you think I could have cancer? The physical exam consists of a visual inspection of the patients breasts followed by palpation (touching). Interpersonal skills continue to be required during the exam, as the interview may continue during the physical exam and the healthcare provider should talk the patient through the procedures as they are about to be performed. Visual inspection begins in a sitting position, as the healthcare provider asks the patient to remove her gown and assume three poses: arms relaxed, arms raised above the head, and hands pressed on hips with chest flexed. The patient is then asked to close her gown and lie down, and the heal thcare provider again visually inspects the breasts in the prone position. In this inspection, the healthcare provider looks for visible abnormalities such as asymmetry, puckering, and redness. The provider then palpates one breast at a time, with only on e breast exposed at a time to minimize patient discomfort and embarrassment. MRIPS -CBE allows the learner to manipulate a physical gown in order to achieve this correct draping of the patient during palpation. Though there are multiple approaches to pal pating the patients breasts, the currently recommended approach is the Mammacare method which has been found to maximize sensitivity of the palpation to finding breast masses [66] [ 102]. The patient lies on her back with the arm corresponding to the breas t being palpated placed behind her

PAGE 81

81 head. The breast is palpated in a vertical -strip patternof -search, i.e. a lawnmower pattern. Each palpation consists of circular motions at low pressure (feeling the superficial layer), medium pressure, and high pres sure (pressing to the chest wall), with each palpation lasting approximately one second. Visual and verbal communication is also important during palpation, as the patient may experience pain which could be expressed verbally but may be more subtly expres sed in facial expressions. The healthcare provider should recognize these expressions, and might also preface the exam by asking let me know if you experience any tenderness. Verbal communication from the provider should also be used to put the patient at ease and to communicate what the palpation will feel like for the patient. Additionally, if e.g. the patient has previously found a mass, the provider may ask questions such as is this the mass that you found? when the mass is palpated. After palpation is completed, the provider should close the patients gown and have the patient sit up on the exam bed. The provider should share any findings, e.g. masses found, with the patient while taking care not to unnecessarily upset the patient, and to addr ess patient concerns. The provider and patient will also discuss the next steps in treatment or diagnosis, e.g. mammograms or biopsies. Both mammograms and biopsies are viewed as fearful experiences by significant portions of the population, with reasons of fear of finding cancer, pain, and radiation being significant barriers to women getting regular screening mammograms. This is especially prevalent in African American and Hispanic populations. Similarly, patients fear that if the mass is cancerous, t he biopsy will spread cancer throughout the breast [ 103 ]. The provider must be prepared to address these fears when discussing these diagnostic tests. In our

PAGE 82

82 evaluations o f MRIPS -CBE, we have created scenarios in which the patient is fearful of mammogram s, because they may be painful and because the patient associates a mammogram with causing her mothers breast cancer, and scenarios in which the patient is fearful of a biopsy because it may spread cancer throughout the breast. 3.2.2 Cognitive, Psychomotor, and Affective Components With respect to the skills that must be trained during the CBE, the CBE is a compound affective cognitive and psychomotor task these three skill sets must be trained in a way that the healthcare provider is able to actuate all three concurrently The affective components of CBE are taking the patients perspective (perspective taking [ 40 ]) and displaying appropriate empathic behavior. These two elements allow the healthcare professional to comfort the patient, obtain an op timal amount of information disclosure and following of the healthcare professionals treatment instructions, achieve rapport, and form a close doctor -patient relationship. Touching of the patient is often used to convey that the healthcare professional u nderstands the patients perspective, as a component of empathic behavior, and as a means of comforting the patient [34][35] [ 36 ][104 ]. The cognitive components of the CBE are split between the medical history and the physical exam. In the medical history, the cognitive components are following proper procedure asking the correct questions, integrating the information divulged by the patient to arrive at a correct differential diagnosis and diagnostic workup (further diagnostic steps, e.g. a mammogram, an d treatment plan). In the physical exam, the cognitive and psychomotor components can be elucidated by labeling each component with a keyword from Bloom and Simpsons taxonomies of cognitive and psychomotor tasks [ 105 ][106 ]. The cognitive tasks are to

PAGE 83

83 rec all a procedural pattern, the patternof -search, in which the breast should be palpated, and the search task for unseen breast masses recognizing which areas of the breast remain to be palpated in order to ensure complete coverage of the breast, and interpreting whether breast tissue feels like normal tissue or abnormality. The psychomotor task is palpation of the breast in circular motions with a specific motion and palpation pressure (at the low, medium, and high levels of pressure). This psychomotor task starts out as a guided response (trial and error) and must be practiced until it becomes an overt response (accurate and skillfully performed). 3.2.3 Current Approaches to Teaching Clinical Breast Examination Accepted approaches to teaching and learni ng CBE follow a progression of: lectures covering technique, practice with anatomical models, e.g. a silicone breast, and practice on standardized human patients (SPs), actors trained to portray a patient with a breast complaint [9] [ 107]. This progression takes place in the 2nd or 1st and 2nd years of medical school, after which, in the beginning of the 3rd year, students begin seeing patients in clinic. In the 3rd year, students education in CBE stems entirely from experiences in clinic. Though the i ndividual approaches in this progression have been validated as providing an improvement in learning over lectures alone, e.g. [62] [ 65][107][ 108 ], individual approaches have significant drawbacks. While practice with anatomical models allows learners to experience abnormal findings in an exam, e.g. a breast mass, this approach does not provide the context of a patient interaction that is critical to succeeding in examinations of human patients [ 7 ]. Students are unable to practice interpersonal skills or t he merging of interpersonal and physical exam skills in this approach. Additionally, feedback on performance is rare, and typically consists of

PAGE 84

84 verbal feedback provided by peers [ 62]. Practice with SPs allows students to focus on both interpersonal and p hysical exam skills, however, availability of SPs for intimate exams is scarce and SPs with abnormal findings, e.g. a breast mass, are unavailable. Opportunities for practice on SPs are further limited because of high monetary cost. Because students get few opportunities to practice with SPs, the merging of the interpersonal and cognitive and psychomotor physical exam skill sets may not occur through practice with SPs indeed, it is common that students performing intimate exams on patients in clinic hav e low confidence in their physical exam skills and anxiety pertaining to both the intimate, interpersonal nature of the exam and the performance of the physical exam itself [4] [ 38]. Additionally, novice students report fear of hurting the patient and fear of missing a lesion as causing significant anxiety during these exams, enough to prevent them from performing an effective exam and communicating with the patient. The fear of missing a lesion persists even in graduating medical students [ 38 ]. In the 3rd year of medical school, as students treat patients in clinic, their education in intimate exams comes primarily or completely from their experiences in clinic. Evaluation of students intimate exam skills in the 3rd year is not standardized, but a common and accepted approach is to receive feedback from an expert observer of the students exam [ 9 ]. This feedback has been shown to improve students physical exam skills in future intimate exams [ 61 ][ 62 ]. However, the requirement of an expert clinician or educator to provide feedback reduces the frequency of these evaluations. Medical educators have identified a need for increased number of practice opportunities and the incorporation of standardized, immediate, objective, and detailed feedback [ 9 ]. MRIPS CBE is targeted to train medical students prior to entering clinic in the 3rd year, to

PAGE 85

85 provide novice students with additional practice opportunities and feedback of their quantitatively measured performance, with the goal of improving cognitive, psychomotor, and affective components of their CBE performance. 3.3 MRIPS -CBE MRIPS -CBE applies the MRIPS design principles to simulation of the interpersonal scenario of clinical breast examination. As an intimate exam, CBE is a high-stakes interpersonal scenari o in which it is not acceptable to fail. Using MRIPS -CBE, novice learners of CBE can practice cognitive, psychomotor, and affective CBE tasks in a learning environment in which it is safe to fail. 3.3.1 Motivations and Goals Together with mammography and breast self examination, the clinical breast exam (CBE) is an essential part of screening for the early detection of breast cancer [9][ 44 ]. Each year in the United States, 200,000 women are diagnosed with breast cancer and 40,000 die of the disease. The CBE is critical in early detection of the disease as CBE is effective at younger ages than yearly mammography is recommended. Additionally, CBE can find masses not found by mammography [ 44 ] each year, up to 10,000 otherwise undetected cancers can be det ected through CBE and it is not cost effective to make mammography the first step in screening [ 4 ]. Because of these factors, competence in CBE is required of all health care professionals both in the physical exam and in communication skills to asses s patient risk, reduce patient anxieties, and address findings of the exam. Clinical breast exam is thus a high-stakes interpersonal scenario. The prevalent means of learning CBE is live-fire practice on real patients in clinic this is not a learnin g environment in which it is safe to fail. This high-stakes aspect along with the

PAGE 86

86 intimate nature of the scenario and complex set of cognitive, psychomotor, and affective skills required for success results in medical students and even practicing clinicians express ing low confidence in their CBE technique and fear that they will miss a breast abnormality [ 9 ][38 ]. Teaching and learning of CBE is hampered by a high degree of learner anxiety [ 38], a lack of standardized, objective, precise feedback, and limi ted opportunities for practice before the live -fire real patient interactions [ 9 ]. The goal of MRIPS -CBE is to provide novice learners of CBE with additional opportunities to practice the cognitive, psychomotor, and affective components of CBE and to rec eive objective, precise feedback concerning these three components (see Chapters 6 and 8 for a description of this feedback). The coupling of additional practice with feedback and reflection is targeted to improve learner performance in CBE of human patients (evaluated in Chapter 9), potentially resulting in more effective early screening for breast disease. We next describe the design of MRIPS -CBE using the merging and augmenting paradigm presented in section 3.1, describe the affordances of MRIPS -CBE wit h respect to the cognitive, psychomotor, and affective components of CBE, and then present the implementation of the MRIPS -CBE visual interface, haptic interface, and mixed reality human simulation. 3.3.2 Merging and Augmenting in MRIPS -CBE The physical in terface of MRIPS -CBE takes the form of a mannequin with a silicone breast in the place of the left breast, and wearing a hospital gown. The mannequins right arm incorporates servo motors allowing it to move across the torso and touch the user. The physi cal interface is registered to the pose of the life -sized virtual human lying on her back on a virtual hospital bed. This affords touching of the

PAGE 87

87 virtual human when she is lying on the bed the pose in which the exam takes place and in which communicativ e touches are most likely (e.g. to instruct to put her arm behind her head to make it easier to identify breast tissue and palpate the armpit; to comfort when the patient expresses fear over the exam or anxiety about having her breast touched). The appearance of the virtual human is augmented by a video stream that shows the users hands, the silicone breast of the physical interface, and the hospital gown; the rest of the physical interface is segmented out of this video stream. The physical interface is augmented with 64 force sensors which report touching of the virtual human. Manipulation of the hospital gown is tracked using infrared optical tracking, reporting when the gown is opened or closed on the left side (i.e. whether the left breast is exposed). The mixed reality human simulation captures the users speech, head pose, force sensor data, and gown manipulation data. This simulation drives outputs: virtual human speech, facial expressions, and gestures, and movement of the mannequins right arm. 3.3.3 Cognitive, Psychomotor, and Affective Affordances of MRIPS -CBE MRIPS -CBE attempts to afford performance of all of the cognitive, psychomotor, and affective components of CBE. MRIPS -CBE affords the cognitive task of conducting a medical interview (as sessing the virtual human patients medical history and risk factors for breast disease) through user speech and virtual human speech and nonverbal responses. The psychomotor task of palpating the breast in three levels of pressure is afforded by the inclu sion of a silicone breast into the physical interface of MRIPS CBE. This breast mimics the feel and soft tissue deformation of a human breast (though it

PAGE 88

88 does not mimic thermal qualities), allowing the user to perform the psychomotor palpation task as he w ould on a human patient. Feedback of the quality of this psychomotor task is also presented using the visual interface (feedback described in Chapters 67). The joint cognitive-psychomotor task of recognizing when the entire breast is palpated and maintai ning a specific pattern of search can be performed by feeling the breast in the physical interface and viewing the breast in the visual interface. Feedback is also integrated into the visual interface to aid these tasks (see Chapters 67). The affective t ask of comforting the patient can be engaged in using MRIPS -CBE. The mixed reality human simulation incorporates specific conversational topics and touch -triggered scenarios (e.g. opening the patients gown to begin palpation) which cause the virtual human to prompt (verbally and nonverbally; but never explicitly) for comfort. To comfort the patient, the user can verbally express understanding of her situation or touch the shoulder or upper arm of the physical interface (comforting touch). In response to the users attempt at comforting the virtual human, the virtual human expresses that she is (or is not) comforted with speech or through visually expressed thoughts (thought bubble feedback discussed in Chapter 7). 3.4 Visual Interface The visual interf ace consists of a life-sized virtual human character as well as elements of the real world that (a) augment this VH character (e.g. physical simulators, physical clothes and other objects) and (b) provide the user with high -fidelity self avatar in the virt ual world.

PAGE 89

89 3.4.1 Life -sized Virtual Human Characters The life -sized virtual human character has f a cial animation and gestural capabilities including lip -synched speech, eye blinking, breathing, pointing, idle behaviors (e.g. swinging legs, looking around), the ability to maintain eye contact with the user, as well as scenario specific keyframebased an imations. The virtual character is able to gesture through offline -created keyframebased animations. The virtual characters facial expressions consist of real -time morphing between offline -created meshes. The virtual characters appearance and animation capabilities were designed using Autodesk Maya and Di -O Matic Facial Studio by other researchers at UF CISE. Specific to MRIPS -CBE, the virtual character h as keyframe -based animations specific to the scenario of a breast exam. She is able to point to locations in her left breast where she has found a mass and where she is ex periencing pain. She also has animations to transition between four positions used in a breast exam: si t ting with arms relaxed, sitting with hands pressed on hips and chest flexed sitting with arms raised above head, and lying down with arms raised above her head. Her facial expressions include neutral, happy (smiling), sad (frowning), confused (this is occasionally used when user speech is not recognized), inpain (grimacing), and fearful (a combination of sad and in-pain). 3.4.2 Augmenting the Virtual World with Real Objects Through additional sensors (webcams) and computer vision tec hniques, elements of the real world are made to augment the virtual human, and the user is given a visually faithful self -avatar. Two webcams, one seeing in color, and one seeing in infrared, are mounted above, looking down upon, the physical interface. For a general application of MRIPS, the color webcam (640x480 resolution, 30Hz) is used to

PAGE 90

90 incorporate a visually faithful self avatar of the user consisting of real time video of the users hands and forearms. Other augmentations of the virtual world and the virtual human are application specific. In MRIPS -CBE, the same color -seeing webcam and computer vision techniques are used to augment the virtual human with a physical breast and a physical hospital gown. Independent of application, we are motivated to allow the user to see his hands when touching the physical interface, as this feature was demanded by pil ot study participants [ 82 ], and it has been shown that seeing ones hands in a virtual world improves the believability of the experience [ 109 ]. Additionally, it has been found that when conducting hands on tasks in a virtual environment, users prefer a visually faithful avatar to a generic avatar i.e. a user wants to see his hand, not a generic hand [ 110 ]. However, providing the user with a vi sually faithful avatar in a virtual world is difficult to do in an unconstrained scenario the users hands must be segmented out from the real world and incorporated into the virtual world in the position and orientation of the users hands the resulti ng avatar is often noisy, containing real world pixels which do not belong to the users body [ 110]. This is not acceptable in many scenarios we anticipate applying MRIPS e.g. in the scenario of a medical physical exam, the user must see how his hands t ouching the patient deforms the patients tissue; this deformation can not be observed if real world background pixels occlude the virtual patients tissue. Fortunately, in our scenario of performing a clinical exam, the user only needs to see his hands when touching the VH. This constrains the problem to providing the user with a visually faithful avatar when his hands are within a small volume surrounding the

PAGE 91

91 MRIPS physical interface. We take advantage of this constraint to simplify the segmentation t ask. All objects in the area of the physical interface, and all parts of the physical interface, which are visible in the webcam images used for augmentation and which should not be visible in the virtual world, are colored black. This allows these objec ts to be segmented from the background with little noise. The segmentation proceeds as: 1 Perform a Gaussian blur (3x3) on the incoming camera image to remove noise within areas that should be marked as foreground. 2 Perform color segmentation, where background pixels are defined as those having luminance less than some predefined threshold (fixed at the time the system is setup, depending on real world lighting conditions). For each background pixel, set its alpha channel value to zero. Alpha blending will be used to incorporate the final image into the virtual world areas of the image with an alpha value of zero will not be visible. 3 Use a 3x3 mean filter to blur the alpha channel, reducing noise at foreground edges. One benefit of this approach allows desirable real world shadows (providing important depth cues) to be included in the virtual world and undesirable real world shadows (over the black -colored areas) to be discarded. In my experience, this would be difficult using color -based or background -subtraction-based image segmentation in an unprepared environment. Furthermore, because the background color is known a priori, the range of colors which the algorithm relegates to the background is very small and is known to be disjoint from the range of human skin colors. This allows the algorithm to succeed for any user, no matter how dark skinned. Once the image of the desired augmentation is extracted from the current video stream image, the augmentations must be incorporated into the virtual world. Because the pose of the camera which perform s this augmentation is fixed (and is not the same pose as the users head pose), we take a simple approach to displaying real

PAGE 92

92 augmentations of the virtual human character in the virtual world at a believable approximation of the users perspective. This approach is to project the processed video stream, which provides these augmentations, onto a polygonal mesh of the physical interface. This mesh is obtained through the 3D -reconstruction technique of laser scan ning the physical interface elements (breast, mannequin). The laser -scanned mesh is registered in the virtual world with the virtual character, and the video stream is a projected texture, projected from a virtual camera having a similar pose as the physi cal webcams, onto this mesh. This method provides depth and occlusion cues in the virtual world, without requiring video to be taken from the users viewpoint. Specific to MRIPS -CBE, augmentation of the virtual world other than the users self -avatar cons ists of augmenting the virtual human character with a physical breast model and a physical hospital gown. The advantage of augmenting the virtual human character with a physical breast model is that users can be provided with realistic appearance and behavior (realistic soft -tissue deformation) of the virtual breast. In a user study we conducted to evaluate the importance of the visual fidelity of the virtual human [ 101], it was found that providing a realistic virtual breast resulted in increased believa bility, realism, and perceived educational benefit of the simulated interaction. 3.4.3 Display of the Visual Interface The visual interface is displayed using one of two display devices: a headmounted display or a projection screen (Figures 3-1 and 33). The goal of each of these displays is to allow the user and the visual interface to occupy the same physical space, allowing the user to see his touching of the virtual human in -situ with the physical silicone breast being touched. The headmounted displ ay (HMD) embeds the user

PAGE 93

93 within the virtual world, while the projection display provides a window into the virtual world, embedding the virtual human within the real world [ 14 ]. 3.4.3.1 Head -Mounted Display (HMD) The HMD used is an eMagin z800 with 800x6 00 resolution and 40-degree diagonal field of view. This HMD is light weight, <0.5 lbs, and has an unobtrusive form factor, making it less encumbering to wear than higher resolution and field of view (and weight and size) HMDs. When viewing the MRIPS vis ual interface using an HMD, haptics and visuals are co-located. The MRIPS visual interface is colocated with the MRIPS physical interface the visual and physical interfaces occupy the same volume in 3D -space. Thus the HMD display device provides an ex perience most similar to the real world, at the expense of encumbering the user (covering the users face, wires that attach the HMD to a PC restricting the users movement ). 3.4.3.2 Projection display The MRH is projected on a planar surface causing the MRIPS visual component to appear non-co -located with its physical component, but instead in a configuration similar to that of Figure 3 -3 Using the projection screen display, users touch the virtual human in one volume and see the virtual human in a dis joint volume haptics and visuals are not co located in the real world, but the virtual human and haptic interface are still conceptually co-located (to touch the virtual human, one touches the haptic interface). The projection screen display provides an experience dissimilar from the real world, but with the advantage of unencumbered movement of the user. We have conducted a user study which compares the HMD and projection displays on the dimensions of usability, acceptability, and user behavior; no sig nificant differences were

PAGE 94

94 found, demonstrating that despite their differences, the displays are equivalently appropriate for displaying the MRIPS -CBE visual interface. 3.4.4 Perspective Correct Viewing of the Visual Interface Independent of which display t ype is used, the users head pose (6 degrees of freedom: 3 degrees of -freedom position, 3 degrees of -freedom orientation) must be tracked to provide the user with a perspective-correct view of the virtual world, and for the users head pose to be an input into the simulation (e.g. allowing the virtual human to make eye contact with the user). When using the HMD, one infrared-reflective tracking marker is affixed to the HMD. Tracking of this marker by an infrared tracking system [ 14] composed of two infrar ed-seeing NaturalPoint Optitrack cameras provides the users head position. The users head orientation is provided by an inertial sensor, an Intersense InertiaCube 2, mounted on the HMD. When using the projection screen display, the user wears a hat, to which three infrared -reflective tracking markers are attached. These markers are tracked to calculate the users head position and orientation, which allows the virtual human to be rendered from the users perspective. 3.4.5 Registering Visual and Physic al Interfaces The physical interface is registered to the visual interface in a one-time calibration step. It is this calibration that allows the user to see the virtual human character in-situ with its physical representation. At setup time, an infrared marker is placed on the webcam overlooking the physical interface, and an additional infrared marker is placed on the physical interface at a location that corresponds to a known 3D -coordinate in the virtual world (e.g. for the breast exam MRH, this marker is placed on the nipple, the position of which is known in the virtual world). By tracking these two infrared markers using the same infrared tracking system used to track the users head pose, the position

PAGE 95

95 of the physical interface and the webcam in th e virtual world are calculated. The error of the registration of the visual interface to the physical interface is estimated at ~1cm at the nipple. The error is higher for other parts of the physical interface, as the shape of the physical interface and virtual human do not match in all aspects. 3.5 Physical Interface The physical interface represents a portion, or all, of the virtual humans body. The physical interface provides a focal point for touch of the virtual human for communication and psychomotor task performance. In MRIPS -CBE, this physical interface takes the form of a semi articulated (neck, arm, and elbow joints) torso arms andhead mannequin having a human form. The mannequins left breast is a silicone breast with the feel of human bre ast tissue. The right arm of the mannequin is replaced with a mechanical arm capable of simple movements to touch the user. The mannequin is covered with a skin that detects the users touch. 3.5.1 Active Sensing of User Touch The skin of the physical interface consists of two thin layers (~3mm thickness) of high density foam. Between the two foam layers is a layer of force sensing resistors. The initial prototype contained only 10 force sensors and provided limited sensing area and resolution. The l atest version of the physical interface of MRIPS CBE contains 64 force sensing resistors (Figure 3 4), affording dense sensing in the full area required for a clinical breast exam and less dense sensing in areas that may be touched for communication, such as the upper arm and shoulder. The resolution at which touch is sensed in the breast is approximately 2cm, though an interpolation technique was

PAGE 96

96 developed which provides continuous sensing of touch throughout the breast (Chapter 7). The force sensing resi stors are placed in circuit s which cause each force sensor to output a voltage (05 volts) that varies (approximately) linearly with the force exerted on the sensor. The output s of these circui ts are sampled by 8bit analog -to digital converters at 60 Hz. This process is controlled by a Motorola 68HC12 microcontroller, which transfers the digital output of the 64 force sensors to the simulation mod ule over a serial link. Through this process, the users touch of the physical interface becomes an input to the mixed reality human simulation. Before the sensor values are received by the mixed reality human simulation module, a baseline for each sensor is found as the average of its values over a period of 30 seconds (in which the physical interface is not touched). This baseline may fluctuate during an exam due to noise or a sensor becoming stuck (after being compressed in by a palpation, thicker areas of the silicone breast may take a few seconds of not being touched in order to become uncompressed). St uck sensors can negatively influence detection of user touch and measurement of user psychomotor task performance. To address this, the value of each sensor is input into a noise gate, which produces an output value of zero if the sensor is stuck above its baseline for a period of time, but allows the sensor to contribute its values once it becomes unstuck, indicated by the value dropping below the baseline for a similar period of time. User touch of the physical input is also detected using computer vision techniques. The infrared-seeing webcam mounted above the physical interface tracks infrared markers which are affixed to physical objects which augment the physical interface (e.g.

PAGE 97

97 the physical hospital gown worn by the breast exam patient). By tr acking the position and other aspects of these physical objects, the manipulation of these physical objects become inputs to the simulation (e.g. opening and closing of the physical hospital gown worn by the breast exam patient). 3.5.2 Passive Detection of User Touch and Manipulation of Tools and Props User touch of the physical interface, which is not a direct touch of the virtual humans body (e.g. touching the virtual humans clothes, manipulating other real objects such as a stethoscope) is tracked usin g the infrared -seeing webcam mounted above the physical interface. Components of the physical interface, such as the virtual humans clothes and tools used to interact with the virtual human, are affixed with infrared reflective tracking markers. In MRIP S-CBE, this approach is used to track a hospital gown worn by the virtual human. The virtual human breast exam patient wears a hospital gown which opens in the front. This is accomplished by having a physical hospital gown be worn by the physical interfac es mannequin, and a virtual correlate (textured mesh) to this physical gown be worn by the virtual character. The gown is an integral part of the breast exam: both breasts must be visually and physically examined, but to maintain patient comfort, only on e breast should be exposed at a time. The gown provides the user with haptic feedback, and the opening and closing of the gown are momentous events in the interpersonal scenario of the clinical breast exam (e.g. a patient fearful of having the exam perfor med may exclaim Wait, Im scared! when the user is detected opening the gown). We explored two approaches to tracking the physical hospital gown in order to render a visually faithful virtual counterpart. The first approach was to use background

PAGE 98

98 subtrac tion to create an image which indicated the area of the webcam image o ccupied by the gown (Figure 35 A ). The gown is segmented from the webcam image using a Gaussianmodel background subtraction approach (see [ 111] for a review of background subtraction techniques). This produces a binary foreground image. To reduce noise in the segmentation caused by shadows cast by the user, the binary foreground image is passed through a variation of a smoothing filter. This filter classifies an image region as for eground if the region contains a number of foreground pixels greater than a predefined threshold, and classifies the region as background otherwise (Figure 35 B). The resulting binary foreground image is ANDed with the visually faithful gown texture. T his texture is applied at 30Hz (maximum webcam frame rate) on top of the virtual characters mesh using multi -texturing. This allows current configuration of the physical gown to be displayed as a virtual gown covering the patient (Figure 35 C ). Using this method, it could be determined whether the gown was open or closed based on the number of background pixels in the binary image. Although this method provided both a virtual correlate to the physical gown, and also detected gown opening and closing using a single color webcam, the resulting visuals are less realistic and convincing than taking the approach used to produce the visuals in Figures 31 and 3-3. These visuals incorporate the second approach to tracking the gown the color webcam uses color segmentation to make the physical gown augment the virtual world (similarly to providing the user with a self -avatar), and an infrared webcam, with a similar viewpoint as the color webcam, tracks a strip of infrared-reflective tape placed on the physic al gown to determine if the gown is open or closed.

PAGE 99

99 The edges of the gown (the infrared-reflective tape) are segmented from the image by color segmentation (the IR reflective edges show up as white in the image, while the background is grey/black). Before the user begins the examination of the patient, a line is fit to the foreground pixels, providing a baseline of where the edges of the gown are located when the gown is closed. For subsequent frames, as the user manipulates (opens, closes) the gown, a li ne is fit to the foreground pixels using least squares. The line of the current frame and the baseline frame are clipped to the edges of the image. This results in four points which form a quadrilateral which is an estimation of the open area of the gown The area of this rectangle is calculated and compared to a pre-set value in order to determine if the gown is open or closed. The advantage of this method is that it produces a higher fidelity visual representation of the virtual gown while detecting o pening and closing with less noise than the first method described. 3.5.3 Bidirectional Touch: Enabling the Virtual Human to Touch the User We have taken two approaches to allow the VH to make an interpersonal touch of the user for the purpose of communicat ing: purely virtual touch and physical (activehaptic) touch. 3.5.3.1 Purely virtual touch The first approach is to incorporate the virtual humans touching of the user into the visual interface: the virtual human character is given an animation in which the virtual humans hand appears to touch the users self avatar. This approach is known as visual touch or pseudohaptics [ 98][ 99 ]. In this approach, the perception of a haptic stimulus from the purely visual stimuli is an effect of synesthesia, a phys iological

PAGE 100

100 phenomenon in which data from one sense (e.g. vision) fills in missing data of other senses (e.g. touch). 3.5.3.2 Physical touch The second approach to allow the virtual human to make a communicative touch of the user is to provide the virtual human a means of physically touching the user. This active haptic touch is provided by a simple mechanical arm which augments the physical interface. This arm was designed for the specific p urpose of the MRIPS CBE patient touching the doctor on the hand du ring the clinical breast exam (Figure 36 ). The arm consists of upper arm and forearm hand sections connected by a servo motor. Another servo motor attaches the shoulder of the arm to the instrumented mannequin of the tangible interface. The two servo motors provide 180 degrees of flexion/extension motion at the shoulder and elbow joints. A passive joint at the wrist and padding of the hand reduce the force of the moving arm to that of a social touch (Figure 37 ). The mechanical arm uses two Hitec HS -564 5 MG digital servos which are capable of 168 oz.* in. of torque and 60o of rotation in 0.18 seconds. Desired rotations of the digital servos are achieved by having a hardware servo controller send a pulse width modulated signal to the servo [ 112 ]. The du ty cycle of this signal specifies the absolute angle the servo should be rotated to. The controller and servos are abstracted by a highlevel framework which allows movement of the virtual humans mechanical arm to appear to the controlling application to be a keyframe-based animation, similar to those used to animate the virtual human. These mechanical animations are defined off -line as a series of sequential or concurrent rotations of servos. At runtime, the controlling

PAGE 101

101 application calls the desired animation, resulting in the predefined movement of the arm to physically touch the user. To enable the movement of the mechanical arm to appear to the application as a simple animation call, an extensible framework was designed. This framework is a highlevel API allowing a virtual reality application to control locally or remotely located groups of actuators, including servos, as a means for incorporating mechanical devices into virtual experiences. The framework is visualized in Figure 38 At the appli cation layer, the movement of the mechanical devices is abstracted as an animation, allowing control of mechanical devices in virtual reality applications to be conceptualized in the same manner as traditional animations in computer graphics (e.g. runAnimation(point to user)). The actuator abstraction and actuator network abstraction layers abstract a single actuator and a network of actuators, respectively. An actuator is an electro-mechanical device (such as a servo motor) which takes as input an ele ctrical signal and produces a mechanical output (physical motion). An actuator network consists of one or more locally or remotely located actuators. Communication between remote systems is enabled through the Virtual Reality Peripheral Network library V RPN [ 113]. At this level, the animation is decomposed into individual commands given to actuators described by a template of (e.g. rotate servo_elbow 90o). This level also enforces constraints of the networks of actuators. Constraints allow for simulation of skeletons of actuators. A skeleton defines relationships between actuators in a network and the conditions these relationships impose. For example, the mechanical arm of the MRH breast exam patient has a skeleton consist ing of two servos:

PAGE 102

102 servo_shoulder_extend and servo_elbow_extend The relationship between these two actuators is that servo_elbow_extend should only operate if servo_shoulder_extend has an angle of greater than 114o (highlighted in yellow in Figure 3-1 2 ). The hardware controller abstraction is a software interface written for a generic hardware controller. It is at this level that the abstraction of having an actuator perform some action is translated into, e.g., having a servo perform a rotation. The h ardware controller interface is a software interface written for a specific hardware controller or set of controllers. This layer incorporates any APIs of commercial controllers used. For example, this layer translates rotateServo (String servoname, float degree) into a hardware specific API call phidgets_api_move_servo(int controllerId, int servoId, float degree), where phidgets is a set of commercial hardware controllers. These layers fit within a mechanical device controller application, which automates initialization of all hardware controllers used in the active animation, and runs all commands of the active animation, specifically handling thread management allowing for concurrent control of multiple actuators. As part of the evaluation of the MRIP S-CBE and its usability for training communication (cognitive and affective) aspects of CBE, we conducted a user study to determine if pseudo-haptic (purely virtual) or activehaptic touch (physical touch) were preferred for allowing the virtual human to i nitiate interpersonal touch and communicate with the user. Participants indicated that both approaches were successful at allowing the virtual human to communicate, but that activehaptic touch may be more effective for communication and was rated as more realistic [ 93 ].

PAGE 103

103 3.6 Mixed Reality Human Simulation The mixed reality human simulation module is an extended version of the simulation module used in the interpersonal simulator of Johnsen et al. [ 10 ]. By taking in the inputs and producing the outputs list ed in Table 3 -1, the simulation module affords bidirectional verbal, gestural, and haptic communication between virtual human and human. The design of the simulation module is shown in Figure 3-9 For each scenario, an xml database of triggers and responses is created by a scenario designer with input from domain experts. The function of the simulation is to map simulation inputs to a trigger, which, as defined in the xml database, is in turn mapped to a set of responses. These responses are executed by the rendering module, resulting in the simulation output the speech and nonverbal behavior of the virtual human character of the visual interface. To map the simulation inputs to triggers in the xml database, the current set of inputs and the current si mulation state are examined. Each trigger can be thought of as a vector of trigger variables T and a vector of pre-condition expressions P (e.g. user_said_hello != 0). For each trigger, if T is found to be a subset of the simulation inputs I and P is found to be a subset of the current simulation state S, the trigger is activated and the corresponding responses responses (T ) are executed. At each timestep in the simulation, the funct ion of the simulation module is that of the following pseudocode meth od match (Triggers). Responses R NULL For each pair ( T P ) in Triggers, For each pre-condition p in P If p S, exit Else, continue For each trigger variable t in T If matchSubset ( t I ), continue

PAGE 104

104 Else, exit R responses (T ) execute(R ) Each type of input listed has a different matching algorithm ( matchSubset ) associated with it. Matching of speech and user tracking data has previously been described by Johnsen [ 14 ]. The speech matching algorithm matches text of the users r ecognized speech to keywords defined in the trigger variable vector T My contributions to the simulation module have been to add a flexible system of maintaining state, and to incorporate haptic interaction inputs (user touch of the physical interface and tracking of the manipulation of physical interface items) into the simulation 3.6.1 Maintaining and Applying Simulation State Simulation state is maintained in a simple fashion each set of executed responses R contains a vector SR containing dyads . After executing the response, the new value of each state variable s in the system state vector S is updated as, s f (v ). By allowing each trigger to require that some subset of state variables have specified values (i.e. have pre-con ditions), the interaction between the human and VH can be directed down certain paths, or be allowed to proceed in a free-form manner. Providing the simulation with the ability to direct the human virtual human communication is necessary when the space of inputs is expanded from only speech input to combinations of speech input and haptic input (e.g. touching the virtual human while asking does this hurt?). In the example shown in Table 3-2, each of the users statements is extremely similar so similar that the keyword matching system will likely not be able to tell them

PAGE 105

105 apart and these statements should elicit different responses from the virtual human. The virtual human should not provide the 2nd response if she has not already provided the 1st resp onse. The current state of the simulation after the first virtual human response is taken into account when the 2nd user speech is processed, and results in the appropriate 2nd virtual human speech. This example illustrates the ability of maintaining sim ulation state to provide the virtual human with the appearance of understanding the context of the users speech. The simulation state is designed to be flexible in its use, e.g. it can also be used to keep the virtual human from performing a certain acti o n until the user has performed prerequisites for this action. We use this function of the simulation state to guide the user and virtual human through a three part exchange when the user recommends the virtual human receive a mammogram. The virtual human first expresses fear that the mammogram will hurt, then expresses fear that it will find cancer, and, with further prompting from the user, agrees to have a mammogram performed. Keeping track of simulation state can also allow for asynchronous arrival o f speech and touch inputs which are part of the same query to trigger the correct response (e.g. touch followed 2 seconds later by does this hurt?; does this hurt? followed 3 seconds later by touch; these can both trigger the correct response: its a little tender there, even though the speech never arrives at the exact time as the painful area is being touched. 3.6.2 Incorporating User Touch of the Physical Interface into the Simulation Haptic inputs to the simulation are the users touch of the phys ical interface, and the users manipulation of other physical objects. User manipulation of physical objects is handled by the simulation just as any other position orientation tracking data input to the simulation. User touch of the physical interface i s handled through simple touch

PAGE 106

106 templates which define the mapping of low level sensor inputs to high-level touch gestures. E.g., the low -level sensor inputs sensor4=2.0v ; sensor5=2.5v are mapped to the gesture user_touching_mrh_patients_left_breast by this template: = link=and/> = link=or/> Using a simple hierarchical approach, the simulation module matches the current set of sensor values to a touch-template. From a software engineering standpoint, it is valuable to take a hierarchical, multi -level approach, in order to provide extensibility of the implementation. Howev er in practice, we have found that a single level (flat hierarchy) is sufficient for most touch gestures. Only a handful of higher level gestures benefit from the multi level approach (certainly these can also be described in a single level, but creation of the xml script becomes unwieldy), e.g.: Above, touch_breast i s the template described in Listing 3 3, and touch_neck is another template not shown that involves many sensor inputs. The touchgestures which are the output of this matching step are trigger variables which are matched to triggers using the matching al gorithm for which pseudocode is provided in match (Triggers). 3.6.3 Touch Driven Communication The design of MRIPS affords bidirectional verbal and nonverbal communication between human and virtual human. By providing this verbal and nonverbal communication including communication through touch, MRIPS provides a solution to

PAGE 107

107 the difficult human -computer interaction problem of simulating a social interaction between two humans. Communication through touch is afforded by a feedback loop. This feedback loop pr oceeds as: The user touches the VH. The resulting values of the touch sensors are examined to recognize the corresponding touchgesture. The corresponding touch-gesture is matched to a VH response. The VH communicates with the user as a result of the use r touching the VH (the VH may also visually or physically touch the user in response). The VHs verbal, gestural, or haptic response leads the user to his next verbal or haptic communication. The following example of touch -driven communication was incorpor ated into one of the user studies described in Chapter 4. When the user begins to palpate the patients breast, the users touch is detected by the force sensing resistors (for illustrative purposes, the touch is detected at sensors 4 and 5); the sensor v alues at sensors 4 and 5 are matched to the touchgesture user_touching_mrh_patients_left_breast ; this touch-gesture is mapped to a trigger which has as its responses the VH assuming a fearful facial expression and exclaiming Wait! Im scared that you might find something wrong with me will it be ok? The virtual humans verbal and gestural responses should lead the user to express empathy towards the virtual human with a comforting verbal response and possibly a comforting interpersonal touch of the virtual human. Additional touchdriven communication events have been built into MRIPS -CBE. During the exam, the virtual human patient is able to tell the user when a painful area of her breast is palpated ( Yeah, that hurts a little accompanied by a grim ace) and describe relative levels of discomfort between two areas of her breast being palpated (e.g. participant: Does it hurt more or less when I press over here? could elicit virtual human responses such as About the same or It hurts the most below the nipple ). After palpation is completed, the virtual human asks the user does everything feel ok? challenging the user to gently broach the subject of any masses which had been found.

PAGE 108

108 During palpation, the virtual human tended to stare at the ceilin g of the exam room; however, by touching the physical interface on the upper arm or shoulder, the participant could get the virtual humans attention in the form of eye contact from the virtual human. 3.7 Example MRIPS -CBE Interaction For two of the user s tudies conducted with MRIPS -CBE, the MRH patient was a 34year old female who has been experiencing a persistent pain in her lower left breast for the past three weeks. She has lost her mother to breast cancer within the past two years, and is fearful that the exam will find an indication that she too has cancer. The user spends the first five minutes conversing with the VH patient to take a breast history (Figure 3-1 0 ). The goals of this conversation (a cognitive task) are to elicit: the patients current complaint, past medical history and social history, as well as her family medical history. This involves the user first introducing himself, and asking the patients name and age and eliciting information concerning the patients complaint: User: Hi I m Matthew a medical student, how are you doing today? VH: Im ok, Im a little nervous though. User: Ok, may I ask your name VH: My name is Amanda Jones User: How old are you Miss Jones? VH: Im 34 User: What brings you in today? VH: Ive had pain in my left breast for a while. User: How long is a while? VH: I guess Ive had pain for about a month. Ive just been scared to come in. User: Can you describe the pain? VH: Its kind of a dull pain. Its really sensitive to the touch though. The user then investigates the patients past medical history (e.g. Have you ever had breast pain before?), the patients social history (e.g. Are you sexually active? ), and past family history:

PAGE 109

109 User: Has anyone in your family ever had breast canc er? VH: I lost my mother to breast cancer two years ago. User: Im sorry to hear that. It must make this very hard for you. The user then conducts a visual inspection of the patients breasts, asking her to first remove her gown, and then to pose i n two positions: with arms raised over her head (Figure 3-11 ) and with hands pressed to her hips and chest flexed. After the visual inspection, the patient is asked to lie down and put her left arm behind her head (as in Figures 3-1, 3 -3). The user then proceeds with palpation of the patients left breast, after first opening the patients gown to expose the left breast (psychomotor and cognitive-psychomotor tasks are performed during palpation). When the user is first detected palpating the patients br east, the patient expresses fear that the examination might find something bad (cancer): wait! Im kind of scared about this. Is it going to be ok? This expression of fear and prompting for reassurance should be responded to empathetically by the part icipant (an affective task). When finished with palpation, the user discusses his findings with the patient, stating that he found what may be a breast mass, and recommending the patient receive a mammogram for further diagnosis. The patient expresses a fear of mammograms: arent mammograms painful? and do I really have to get a mammogram? I mean, my mom was fine, then she had a mammogram, then all of the sudden she was really sick. This is an opportunity for the user to reassure and comfort the pat ient (affective tasks), while achieving compliance with the recommended diagnostic procedure of receiving a mammogram (cognitive task). After expressing that a mammogram is not painful and is important to determine what may be wrong with the patient, the patient expresses that

PAGE 110

110 she will consent to a mammogram: if thats what you think is best The user then concludes the encounter. 3.8 Pilot Study A pilot study was conducted to determine whether MRIPS -CBE is usable to perform the cognitive, psychomotor, and affective components of the CBE. The question this study posed was: do learners apply the skills they have previously learned in real world interpersonal scenarios to the simulated interpersonal scenario of MRIPS -CBE? Observed participant behavior revealed that learners of MRIPS -CBE applied the cognitive, psychomotor, and affective skills they had previously learned through traditional educational methods. Of note, participants frequently touched the virtual human to comfort her during expressions of fear and pain. 3.8.1 Population and Procedure Eight 2ndyear physician assistant students at the Medical College of Georgia conducted a CBE using MRIPS -CBE. These students were inexperienced in CBE. Although they had all received lecture-based teachi ng of CBE procedure, only one of the students had previous experience performing CBE on a human patient. All participants had conducted medical interviews of human patients (ranging from 1 to 6 interviews with an average of 2.4). Using MRIPS -CBE, particip ants began with a medical history of the patient la sting approximately 10 minutes, and then performed a physical exam lasting from 5-10 minutes. During palpation, the virtual human expressed pain when participants touched a pre defined (but unknown to part icipants) section of her breast. The patient exclaimed

PAGE 111

111 ouch! or that hurts! Participants responses to this moment and use of comforting touches were observed to evaluate whether participants were able to apply their real world learned affective skil ls to the simulated interpersonal scenario MRIPS -CBE. Participants were not given any instruction regarding touching of the virtual human in MRIPS -CBE. 3.8.2 Observations All participants were able to successfully perform the cognitive task of taking a medical history of the patient using MRIPS -CBE. Participants discovered important pieces of information such as: the patient had found a walnut sized mass in her left breast, the patient had a family history of breast cancer (sister), the patient was postmen opausal, and the patient was taking hormone replacement. All participants were also able to perform the psychomotor and cognitivepsychomotor tasks involved in palpation. In this pilot study we did not evaluate the quality of this task performance; partic ipants were inexperienced and not expected to perform these well, but all participants attempted these tasks to the best of their ability. A majority of participants performed successfully at the affective task of comforting the patient. Seven of the part icipants elicited the pain response from the virtual human, indicating a need for comforting through the expression of empathy. Five participants responded with empathic statements, e.g. I know its tender there. Ill try to be more gentle. Non empath ic responses from the other two participants indicated that they at least understood the patient was in pain, e.g. Oh, it hurts right there? Compared to a previous study of 27 2ndyear medical students interviewing a virtual human with a breast mass usi ng the IPS system of Johnsen et al. [ 15 ] (in which touch was not afforded) more participants exhibited empathy and appropriate responses to a prompt

PAGE 112

112 for comforting using MRIPS -CBE. In the prior study with IPS, 70% of participants did not respond appropri ately to a virtual humans expression of fear; only 10% responded empathetically and 20% completely ignored the prompt for comforting. The participants using MRIPS responded to the virtual humans need for comfort in the manner they were taught and are ex pected to respond to a human patient; the participants using IPS responded dissimilarly from the manner they are taught to treat human patients. Participants also used touch for cognitive and affective tasks such as instructing and comforting the patient. Seven of the eight participants touched the virtual human for communicative purposes. Each participant used an average of 1.4 (std. dev. of 0.9) of these touches. We sought to determine if this was a similar amount of touch as used in the standardized p atient interactions in which these students had learned the skill of interpersonal touch. We were able to obtain data from a user study in which 76 students examined an SP with abdominal pain (data for an intimate exam scenario was unavailable). In the S P scenario, participants used an average of 1.8 (std. dev = 1.8) touches. A statistical test of equivalence [ 114 ] indicated that it was likely that we would find an equivalent amount of touch in MRIPS and SP scenarios if a larger population was obtained for the MRIPS scenario. 3.8.3 Discussions Participants performed all three task sets of CBE: cognitive, psychomotor, and affective. The observations made in this pilot study presented positive preliminary evidence that MRIPS presents advantages over previ ous interpersonal simulations, as the haptic interface of MRIPS provided touch which was used by participants to perform cognitive and affective tasks in addition to the expected psychomotor task performance. Additionally, the improvement in affective per formance over the prior IPS scenario could

PAGE 113

113 indicate that the additional interaction modality of touch makes users treat the MRIPS scenario more like a real world interpersonal scenario. 3.8.4 Conclusion and Further Evaluation The pilot study established th at users of MRIPS -CBE were able to perform the cognitive, psychomotor, and affective components of the CBE as they had been taught. Chapter 4 presents two additional user studies which establish that MRIPS -CBE elicits performance indicative of the learner s performance with a standardized human patient and that performance in MRIPS -CBE differentiates between users of different skill and experience levels. These studies establish the validity of MRIPS -CBE for practicing and evaluating a learners cognitive, psychomotor, and affective performance in CBE. After establishing that MRIPS -CBE can be used for evaluating a learners CBE performance, we can get to the heart of the thesis: evaluating whether a learners CBE performance can be improved by use of MRIPS -CBE (Chapter 9).

PAGE 114

114 Table 3 1. List of simulation inputs and outputs. Inputs from physical sensors Outputs of the virtual human simulation User speech VH speech (pre recorded human speech) User touch of physical interface VH gestures (keyframe animation) Tracking of physical interface VH facial expressions (mesh morphing) User tracking data (e.g. head pose) VH vis ual touch of the user (keyframe animation) VH physical touch of the user (mechanical actuation) Table 3 2. An example of using system state to direct the conversation between user and MRH to take a specific path. State (transitions are marked): value pre speech value post speech ) Input (user) / Output (virtual human) Told_need_ mammogram Afraid_of_ mammogram Accepted_ mammogram 0 0 0 User : Based on your family history, you should have a mammogram. 0 1 0 0 VH: Ive never had a mammogram. Ive always been afraid that a mammogram would hurt. 1 0 1 0 User (is trained to continue in this vein to arrive at VH compliance): Well, there is s ome pain associated with a mammogram, but right now it is the best option for you 1 1 0 1 VH: Well, ok, if you think a mammogram is best

PAGE 115

115 Figure 31. A learner performs a CBE in MRIPS -CBE. He touches the physical interface and s ees his touch reflected in the visual interface.

PAGE 116

116 Figure 32. System design of MRIP S-CBE. The learners touch of the physical interface, speech, tracked head pose, and manipulation of the hospital gown are inputs into the simulation. The outputs of the simulation are the virtual humans speech, facial expressions, and gestures.

PAGE 117

117 Figure 33. MRIPS -CBE visual interface presented on a projection screen. Figure 34. The physical interface of MRIPS -CBE. Beneath the modular foam skin and silicone bre ast are 64 force sensors.

PAGE 118

118 A B C Figure 35 The first approach taken to tracking the physical gown and providing a corresponding virtual gown used a background subtraction approach. This approach was noisy and less realistic than the video augmentation method seen in Figures 3 -1 and 33.

PAGE 119

119 A B Figure 36 Affording bidirectional touch for communication by allowing the MRH to touch the user. A) a purely virtual pseudo-haptic touch and, B) a physical active haptic touch from the robotic right arm of the physical interface.

PAGE 120

120 A B Figure 37 The physical interface incorporates a mechanical right arm, allowing activehaptic touch from virtual human to human user. Figure 38 The layers of the framework for abstracting a virtual environments control of physical actuators.

PAGE 121

121 Figure 39 Progression of one time step of the simulation module At time t the simulation module takes an input vector I (t ), the set of inputs detected by physical sensors placed on and around the user and simulation state vector S(t ), and produces an output vector O (t + dt ), the set of VH responses and a modified simulation state vector S( t + dt ). Figure 31 0 The medical interview portion of the interaction with the MRH breast exam patient. The studen t converses with the virtual human to gain information concerning the virtual humans medical history and current condition.

PAGE 122

122 Figure 31 1 Two of the poses required for visual inspection. The virtual human raises her arms above her head and presses on her hips to flex her chest to allow the healthcare provider to inspect her breasts for abnormal appearance (e.g. asymmetry or redness). Figure 31 2 The xml script that defines relationships between servos, constraints, and animation of servos to all ow the MRH to touch the user on the hand.

PAGE 123

123 CHAPTER 4 VALIDITY OF MRIPS -CBE FOR PRACTICE AND EVALUATION OF COGNIT IVE, PSYCHOMOTOR, AND AFF ECTIVE SKILLS This chapter describes two user studies which evaluated the potential for MRIPS CBE to educate the cognitive, psychomotor, and affective skill sets of its users and for educators to use MRIPS -CBE to evaluate learners performance. Study MRIPS -SP compared novice learners CBE performance with MRIPS -CBE and with a standardized human patient (SP) using a within-subjects design. Results of this study demonstrated content validity of MRIPS -CBE for practicing CBE. The validity of MRIPS -CBE for evaluating learners CBE skills was additionally demonstrated for cognitive and psychomotor skill sets, but not fo r affective skills. The lack of evidence of validity for evaluating affective skills was due to high variability in expert ratings of participants affective performance. The second study, Study MRIPSx2 further evaluated the validity for evaluating learners CBE skills by investigating the ability of MRIPS -CBE to distinguish between learners of different skill sets on dimensions of cognitive, psychomotor, and affective performance. Expert ratings of affective performance were again used, but a simpler ratings instrument was used. Results of this study established validity of MRIPS -CBE for evaluating learners cognitive, psychomotor, and affective performance. These studies serve to demonstrate that MRIPS -CBE can be substituted in place of an SP for pra ctice and evaluation of CBEs, and motivate further evaluation to determine what learning takes place in users of MRIPS CBE. Collaborators: Medical collaborators Scott Lind, Adeline Deladisma, Andy Laserno, and Angela Gucwa recruited study participants and assisted in running the studies described in this chapter. Thanks to Scott Lind, Angela Gucwa, Teresa Lord,

PAGE 124

124 Hevil Shah, and other clinicians and medical educators who participated in rating videos of participant interactions. Thanks goes to Adeline Dela disma for assisting in rating participant notes for correctness of diagnosis and diagnostic workup. Personal contributions : I designed all studies and performed all analysis described in this chapter. Relevance to thesis: The thesis focuses on demonstrat ing learning and training transfer with MRIPS. But, why should we expect users of MRIPS -CBE to learn (improve in) CBE or for this learning to transfer to CBE of human patients? The studies described in this chapter lay the foundation for evaluating learn ing and training transfer, by demonstrating that users of MRIPS -CBE were able to apply their cognitive, psychomotor, and affective skill sets in a fashion similar to CBE of a human patient. These studies demonstrated the validity of MRIPS -CBE for practici ng CBE and evaluating components of learners CBEs. 4.1 Introduction This chapter describes two user studies that were conducted to evaluate the validity of MRIPS -CBE for practicing and evaluating the cognitive, psychomotor, and affective components of l earners CBEs. Content validity of MRIPS -CBE for practicing CBE is established if learners are able to perform similarly using MRIPS -CBE and a previously validated means for practicing CBE. Study MRIPS SP sought to establish this validity. Novice learn ers performed a CBE using MRIPS -CBE and a CBE of a standardized human patient (SP). Cognitive, psychomotor, and affective performance in MRIPS -CBE was found to be statistically equivalent or noninferior to performa nce with the SP. Learners were able

PAGE 125

125 to use the same skills with MRIPS -CBE and the SP, establishing validity of MRIPS -CBE as an additional practice opportunity for CBE. Additionally, learners cognitive and psychomotor performances with MRIPS -CBE and the SP were significantly correlated, estab lishing the validity of MRIPS -CBE to evaluate learners on cognitive and psychomotor components of the CBE. Affective performance in MRIPS -CBE and with the SP was not correlated, due to high variability in expert ratings of learners affective performances Because of this we further investigated the validity of MRIPS -CBE for evaluating CBE skills, using another approach. Another approach to establish a simulations validity in evaluating real world skills is to show that the simulation distinguishes between users of different skill levels [ 42 ]. We took this approach in Study MRIPSx2, in which 2nd, 3rd, and 4th year medical students as well as interns, residents, and clinicians had their cognitive, psychomotor, and affective performance evaluated in a CBE using MRIPS -CBE. Results established the validity of MRIPS -CBE for evaluating learners cognitive, psychomotor, and affective performance in CBE. These studies demonstrate that users of mixed interpersonal simulation treat their mixed reality human inter action partners similarly to how they treat human interaction partners, and that an interpersonal simulator incorporating haptic interfaces elicits and can be used to evaluate users real world cognitive, psychomotor, and affective skills in an interperson al scenario. MRIPS -CBE presents a novel opportunity for learning CBE which would be a valid addition to a CBE curriculum.

PAGE 126

126 4.2 Study MRIPS -SP: Comparing MRIPS -CBE to CBE of an SP This study set out to demonstrate that MRIPS -CBE can elicit cognitive, psycho motor, and affective performance similar to the standardized human patient (SP). We chose to compare MRIPS -CBE to the SP because the SP is the gold-standard for learning and evaluating CBE performance [ 16 ] To evaluate whether learners performed similarly with MRIPS -CBE and the SP, we conducted a user study in which novice medical students performed CBEs on each of the MRIPS -CBE mixed reality human patient and an SP. We do not present hypotheses for this study as none were written a priori As our goal is for MRIPS -CBE to become a tool for practicing and evaluating learners CBE, the analysis focuses on determining whether MRIPS -CBE is valid for practice and evaluation of CBE performance. We designed the study and conducted analysis to query the: 1 Valid ity of MRIPS -CBE as a practice tool for CBE. If participants cognitive, psychomotor, and affective performances with MRIPS -CBE are statistically equivalent or non-inferior to their performances with the SP, we establish the content validity of MRIPS -CBE for practicing CBE. Content validity for practicing CBE indicates that MRIPS -CBE can be used in place of an SP for practicing CBE, as it is known that learners perform equivalently in both MRIPS CBE and SP. Note that we only address practice of CBE; eval uation of learning is left to Chapter 9. 2 Validity of MRIPS -CBE for evaluating medical students CBEs along cognitive, psychomotor, and affective dimensions. Validity of an interpersonal simulation for evaluating performance is demonstrated by showing significant correlations between learner performance in the interpersonal simulation and the gold-standard for evaluation [ 81 ]. For CBE, this gold -standard is the SP. Thus if significant correlations are found between learners cognitive, psychomotor, and af fective performances with MRIPS -CBE and with the SP, we will accept that MRIPS -CBE is valid for evaluating learners CBE skills. Establishing equivalence or non-inferiority of MRIPS -CBE for practice of CBE indicates that MRIPS -CBE can be substituted for an SP interaction. As there are

PAGE 127

127 logistical problems limiting the use of SPs the ability to validly substitute a MRIPS -CBE interaction for an SP interaction can potentially increase the quantity and frequency of learners practice opportunities in a curricu lum for learning CBE, without a reduction in the quality of the practice experience. 4.2.1 Study Design and Procedure Ten (n = 10) medical students in their 2nd semester of medical school conducted two CBEs, one using MRIPS -CBE and the other of an SP. A c ounterbalanced design was used with five participants examining first the SP and then MRIPS -CBE (Group SPMRIPS ), and five participants examining first MRIPS -CBE and then the SP (Group MRIPS -SP). None of the participants had previous experience performing CBE on a human patient or SP, although all had conducted medical interviews of SPs (an average of 2.0 SP interviews). For this scenario, the mixed reality human patient of MRIPS -CBE was a 34year old female who has been experiencing a persistent pain in h er lower left breast for the past three weeks. She has lost her mother to breast cancer within the past two years, and is fearful that her breast pain is due to cancer. Two simulated masses were placed in the MRIPS breast model, each a hard mass with approximately 0.5 cm radius. The SP was a middleaged female who has found a breast mass that comes and goes for the last six months. Recently the mass has increased in size. Due to SP availability, three actresses played the SP. To account for this var iability, the varying of the actress was balanced between Groups SP -MRIPS and MRIPS -SP. In order to evaluate participants ability to find masses in MRH and SP breasts, an SP with a breast abnormality was required. As SPs with breast masses are not avail able, participants did not perform palpation on the SPs breasts but instead palpated a silicone breast

PAGE 128

128 model placed on the hospital bed next to the SP. This breast model was the same used in MRIPS -CBE. The SP observed the palpation and was trained to re spond to the participants palpation of the silicone breast, expressing pain if a tender area was palpated and acknowledging the location of the mass if asked by the participant (e.g. is this the mass you found?). It has previously been shown that simil ar integration of simulators into an SP encounter provides a similar educational experience to performing the exam directly on the SP [ 115]. This method has also been validated for practice of physical exams and procedures which can not be performed direc tly on SPs [ 7 ]. A single 0.5 cm radius hard mass was placed in the SP silicone breast. Although the number of masses in the MRIPS -CBE and SP breasts differed, finding of masses was determined to be acceptable in judging completeness of participants exam s. The masses in the MRIPS -CBE and SP breasts were judged by collaborating medical educators as equally difficult to palpate. Since the masses are of equal difficulty to find, an exam is complete if it finds all masses regardless of the number of masses present [4] [ 66 ]. Three critical moments were integrated into the MRIPS -CBE and SP interactions. A critical moment is an instance in the scenario which prompts the learner to utilize his affective skill set, to take the patients perspective and express em pathy or concern. While explaining her symptoms, the patient expressed her fear that she could have cancer by fearfully asking could it be cancer? Later in the exam, when the participant began to palpate, the patient expressed fear that the examination might have a negative outcome (i.e. cancer): wait! Im kind of scared about this. Is it going to be ok? After

PAGE 129

129 palpation was complete, the patient inquired about what would happen to her, asking can you tell me what happens next? 4.2.2 Measures Evalu ating cognitive performance consisted of evaluating the completeness of the medical history and correctness of the differential diagnosis and diagnostic workup (what tests or procedures to perform after the exam). To evaluate the completeness of the medi cal history, an objective, quantitative score of completeness was obtained by reviewing logs and videos of the MRIPS -CBE and SP interactions. The cognitive performance of each interaction was scored as the number of items of Table 4 -1 that the participant queried. The items in this medical history completeness checklist are taken from instruments used in curricula at the University of Florida and the Medical College of Georgia, and consist of crucial topics which should be addressed to evaluate a patient s risk factors for breast cancer. Data for this checklist is presented in Appendix A. The correctness of the differential diagnosis and diagnostic workup were evaluated by medical educators at the Medical College of Georgia. A correct differential diagno sis for the MRIPSCBE and SP was a malignant or benign tumor or cyst and a correct diagnostic workup must have included a mammogram. Psychomotor performance was evaluated as the completeness and effectiveness of the physical exam. Completeness was measured as the time spent palpating, and effectiveness was measured as the number of breast masses found. The amount of time spent palpating indicates the effort put into the exam and is positively correlated with the number of masses found [4] [ 65 ]. Time spent palpating has been questioned as a measure of CBE performance due to differences in efficiency among practitioners of

PAGE 130

130 different experience levels. However, it is valid to use this metric in this study because all participants in this study had the same experience level in CBE [ 4 ]. Affective performance was evaluated by having clinicians and medical education experts rate each of the three critical moments using a 5-item questionnaire. The questionnaire and data are presented in Appendices B and C This questionnaire consisted of one item assessing the appropriateness of the learners response to the critical moment (would the response be appropriate with a real patient) and four items assessing the empathic quality of the learners response (does it encourage the patient to express emotion, validate the patients feelings, explore the patients feelings, and include appropriate nonverbal behavior). The four items dealing with empathy are taken from the empathy subscale of a validated instrument for evalu ating medical interview skills [ 41 ]. 4.2.3 Statistical Analysis In presenting the results of this study, we refer to equivalence and non-inferiority. Equivalence indicates that the results of two treatments, while not identical, are so close that the trea tments are equally preferable. However, equivalence is too restrictive for comparing MRIPS -CBE and SP as it does not allow MRIPS -CBE to outperform SP. For this reason, we additionally test for the non-inferiority of MRIPS-CBE. Non-inferiority of a novel treatment (MRIPS -CBE) indicates that the novel treatment is equally or more preferable than the existing treatment it is compared to (SP). Non-inferiority can be thought of as a lesser test of equality than equivalence. If two treatments are equivalent, each treatment is non-inferior compared to the other treatment. However, the reverse is not true, as non-inferiority does not imply equivalence.

PAGE 131

1 31 Statistical equivalence is demonstrated by calculating a 95% confidence interval and comparing it to a clinic ally chosen zone of indifference. If the confidence interval lies completely inside the zone of indifference, the two treatments are equivalent. The zone of indifference is subjectively chosen based on knowledge of the scenario and measures used. In thi s experiment, many of the measures have a granularity of 1 unit (e.g. one point on the 1-5 scales used in video coding; one item on the cognitive checklist). Thus we pick a zone of indifference to be a closed interval of length one: (-.5, .5). Noninferiority is evaluated in a similar fashion but only the lower bound of the confidence interval must lie within the zone of indifference. The lower bound of this one -sided 95% confidence interval is equal to the lower bound of a standard 90% confidenc e interval, so in practice the 90% confidence interval (CI) is calculated [ 114][ 116]. To determine the validity of using MRIPS -CBE to evaluate learner performance, we look for a positive linear relationship between a participants MRIPS -CBE performance and his SP performance. This is evaluated by calculating Pearsons correlation coefficient. All analysis was performed using the SPSS15 software package. In addition to evaluating equivalence and non-inferiority, we looked for effects of the order of treatm ents by performing a Students t -test between Group SP MRH and Group MRH SP. 4.2.4 Results and Discussion 4.2.4.1 Order effects No effect of order was found on any measures. One might expect participants to exhibit improved performance in their second int eraction. However, we did not expect

PAGE 132

132 this to be the case. The two interactions were performed within an hour of each other and participants received no feedback as to the quality of their first performance. Thus we did not expect learning and a corresp onding improvement in the second interaction to occur. 4.2.4.2 Cognitive performance Cognitive performance in MRIPS -CBE was significantly correlated with cognitive performance with the SP. This establishes the validity of MRIPS -CBE for evaluating learners cognitive performance in CBE. One participant was removed from cognitive and affective measures analysis due to clearly demonstrating a disinterest in approaching the MRIPS -CBE interaction seriously. This participant first performed the SP interaction i n which he queried 12 items on the cognitive checklist. However, in his MRIPS -CBE interaction he queried only 5 items on the cognitive checklist, only asking questions directly related to the manual exam in what appeared to be an attempt to complete the exam in a minimal amount of time. This was also the only participant to receive a rating of 1 exceptionally inappropriate from the medical expert video reviewers for his response to one of the critical moments. We chose to remove this participant from further analysis because we do not expect that this negative attitude towards MRIPS CBE would be exhibited if MRIPS -CBE was integrated into a curriculum (i.e. if the participant was graded on his performance with MRIPS -CBE). Accordingly, we attach to the results the caveat that learners must approach the MRIPS interaction seriously in order to benefit. It is our opinion that this caveat extends to all educational experiences In the medical history checklist measure, the remaining nine participants querie d an average of 10.3 1.8 items from the MRIPS -CBE virtual human patient and queried

PAGE 133

133 an average of 9.7 2.2 items from the SP. A significant correlation between the number of items queried in MRIPS CBE and SP was found, with r(7) = 0.86 ( r2=0.74), p = 0 medical history of MRIPS -CBE is predictive of performance in a medical history of an SP. Johnsen et al. previously showed that medical student performance in a history taking interpersonal simulation predicted performance in taking a medical history of an SP [ 81]. We expand on this approach further by evaluating non -inferiority of MRIPS CBE. Participants conducted more complete medical histories of the virtual human pat ient in MRIPS -CBE than of the SP, but not significantly so (by paired t -test: t = 1.8, n s. ). The 90% CI is [ 0.19, 1.53], demonstrating that MRIPS -CBE is non -inferior to an SP for practicing the cognitive task of medical history taking. This finding goes beyond the work of Johnsen et al., to demonstrate that for practicing taking a medical history a MRIPS -CBE interaction is an equally preferable substitute for an SP. All participants arrived at the same differential diagnoses and diagnostic workups with M RIPS -CBE and the SP. In other words, if the participant arrived at the correct diagnosis for the SP, he also arrived at the correct diagnosis with MRIPS -CBE. Thus for diagnosis and diagnostic workup, participants performances in MRIPS -CBE and SP were eq uivalen t and also perfectly correlated (i.e. r2 = 1.0). Participants performed well in these two tasks, as 9 of 10 participants arrived at correct differential diagnoses and diagnostic workups.

PAGE 134

134 These results demonstrate the validity of MRIPS -CBE for pract ice and evaluation of learners skills in the cognitive tasks of taking a medical history and integrating findings into a correct diagnosis. 4.2.4.3 Psychomotor There were no sensors present in the SP breast, so to compare performance with MRIPS and the SP we employed measures previously used in evaluating performance with SPs and Mammacare silicone breast models. Psychomotor performance was measured as the completeness of palpation: masses found and time spent palpating. These measures have previously b een used to measure completeness of palpation, and Hall et al. indicated a positive linear relationship between these measures and completeness [ 65][ 66 ]. The time spent palpating the patients breasts was significantly correlated ( r2(8) = 0.64, p < 0.01) The participants ability to find masses in the SPs breast was also significantly correlated with the participants ability to find the masses in the MRIPS -CBE breast ( r2(8) = 0.63, p < 0.05). Five participants found all masses in both the MRIPS -CBE a nd SP breasts; three participants did not find the masses in either breast; and only one participant found the mass in the SP breast but failed to find the masses in the MRIPS -CBE breast. These correlations validate MRIPS -CBE for evaluating learners skil l in the psychomotor task of palpation. On average, participants spent more time palpating the MRIPS -CBE breast (83.3 35.4 sec onds ) than they did palpating the SP breast (65.9 25.7 seconds ). This was true for 9 or the 10 participants, even those who p erformed the SP exam before the MRIPS -CBE exam. The differenc e in average palpation duration was significant by a paired t test: t(9) = 2.6, p < 0.05. The MRIPS -CBE mannequin included an armpit which could also be palpated as part of the exam, while the SP breast did not. However, time

PAGE 135

135 spent palpating the armpit was not included in this comparison. The longer palpation duration for the MRIPS -CBE breast may be explained by the virtual human occasionally looking at the participant during palpation. The v irtual humans face was in the participants visual field of view during palpation, and alternated looking at the back wall of the room (gaze aversion) and at the participants head position. In contrast, the SPs face was not in participants field of view during palpation, and the SP tended to watch the palpation of the silicone breast the entire time. This allowed the SP to have context to answer participant questions such as is this the mass you found? Whether lengthened by meeting eyes with the vi rtual human or not, it is clear that participants put at least as much effort into palpating the MRIPS -CBE breast as the SP breast. For this reason, we accept MRIPS -CBE as valid as a substitute for the SP for practice of the CBE psychomotor task of palpat ion. Because the breasts palpated in the SP and MRIPS -CBE exams were the same simulated (silicone) breast, the SP and MRIPS CBE exams were more similar than they would have been if the SP exam was performed on a human breast. However, silicone breasts hav e been validated for evaluating and learning self breast exams and CBEs [4] [ 65 ][66 ]. This prior finding allows us to conclude that MRIPS -CBE is valid for evaluating learners psychomotor skills in CBE. 4.2.4.4 Affective Results of the video review of the three critical moments are presented in Table 42. Video review evaluated the appropriateness and empathic content of participants responses to the critical moments in MRIPS -CBE and SP. Ratings demonstrated that participants performances in MRIPS -CBE w ere either non -inferior or equivalent to their performances with the SP. Participants exhibited similar affective behaviors with the

PAGE 136

136 virtual human as with the SP. This establishes the validity of MRIPS CBE for practicing affective components of clinical breast examination. However, no correlation was found between affective performance in MRIPS -CBE and with the SP. Although MRIPS -CBE and SP are equally preferable for practicing affective components of CBE, MRIPS -CBE is not validated for evaluating affect ive performance in CBE. We believe the lack of correlation between affective performances in MRIPS -CBE and SP is due to the subjective nature of evaluating empathy and appropriateness of the participants responses to the critical moments. Reviewer diffi culties in rating affective performance are demonstrated by the lack of inter -rater reliability. Inter -rater reliability was not established, as calculated inter -rater reliability coefficients (see Appendix C ) were less than 0.70 for every item in the vid eo review survey [ 117]. This emphasizes a need for objective measures of affective performance, and motivates our later approach of detecting key -phrases associated with empathy within user responses to critical moments (described in Section 7.3.1 and use d in the user study of Chapter 9). The medical educators and experts who performed the video review noted that the overall use of empathy was low. Indeed, only 5 of 10 participants were rated positively on expressing empathy in response to the virtual hum ans could this be cancer? critical moment, and no participants were rated positively on expressing empathy in response to the virtual humans wait! Im scared! critical moment. Performance was actually worse for the SP critical moments (2/10 and 0/10 respectively), though this may be due to video reviewers increased expectations of performance with the SP. This low level of empathy likely contributes to the lack of correlation between MRIPS -CBE and

PAGE 137

137 SP affective performance. However, this low level of expressed empathy is expected behavior of novice students. According to Pugh et al., novice learners of CBE perform poorly on affective and cognitive components until they master the psychomotor and cognitive-psychomotor components (e.g. palpation) of the exam [ 38 ]. This observation motivates the incorporation of real -time visual feedback for guidance of psychomotor, cognitive, and affective performance (Chapters 6-8) and subsequent evaluation of learning of all three skill sets in MRIPS -CBE (Chapter 9 ). 4.2.5 Limitations of the Study A small population was obtained for this study, allowing us to find only large-size effects. Similarly to statistical tests of difference, results of statistical equivalence and non-inferiority tests will hold with larger population sizes. However, it is likely that a medical school considering curriculum integration of a simulation such as MRIPS -CBE would desire a similar study be run with a larger population. The results of this study are only applicable to populations of novice learners of CBE, as all participants were nearing the end of their first year. This 1st-2nd year medical student group is the target enduser group for MRIPS -CBE. However, these novice learners performed poorly on cognitive and affective aspect s of the exam with both MRIPS -CBE and the SP: on average asking only 10 of the 20 medical history questions and generally receiving negative (< 3.0) ratings on use of empathy in the critical moments. This poor performance is expected of novice learners. However, more experienced learners may not approach MRIPS -CBE with the similar positive attitude with which these novices approached both MRIPS -CBE and the SP. For this reason, equivalence and noninferiority should be retested with more experienced learn ers if MRIPS-CBE were to be incorporated into, e.g. 3rdyear curriculum.

PAGE 138

138 In this study we wished to evaluate learners interactions with the virtual human and their ability to perform a CBE using the haptic interface of MRIPS -CBE. For these reasons, if speech recognition failed to produce reasonable text input from the participants utterance, the experimenter triggered the virtual human to reply appropriately to the participant. The result is a MRIPS interaction with idealized speech interface performa nce. It is possible that this artificially improved learners performance in the cognitive task of history taking. It is unlikely that it significantly impacted affective performance, as the critical moments, other than the moment concerning the patient s mother, were triggered from manipulation of the haptic interface. The correlation between MRIPS and SP performance would not be impacted by a reduction in speech interface performance, provided all participants experienced a similar reduction in speech interface performance in MRIPS However, this approach potentially impacted the result of equivalence or noninferiority. We do not view this as impacting the conclusions of this study. The focus of this dissertation is not on speech interfaces, and we expect speech recognition and understanding to continue to improve, shrinking the gap between typical and ideal speech interface performance. Even using the current speech interface, performance can be improved through a training session with speech recog nition. This training session was omitted from this study because of time constraints, but could be incorporated if MRIPS was used in a curriculum. 4.2.6 Conclusions This study provided statistical evidence that for practicing the cognitive, psychomotor, and affective components of CBE, MRIPS -CBE is equally preferable as an SP. This establishes the use of MRIPS -CBE as an additional or even alternative tool for medical educators to give students more practice opportunities in CBE. However,

PAGE 139

139 before MRIPS -CB E can be integrated into a curriculum in which students are graded, we must establish validity of MRIPS -CBE for evaluating a learners performance. Study results validated MRIPS -CBE for evaluating cognitive and psychomotor performance in CBE, but did not e stablish validity for evaluating affective performance. The next study, Study MRIPSx2, employs another accepted technique [ 42] for establishing validity of a simulation for evaluating real world skill. Study MRIPSx2 seeks to determine whether MRIPS -CBE c an distinguish between participants of varied (known) skill and experience levels on dimensions of cognitive, psychomotor, and affective performance in CBE. 4.3 MRIPSx2 In Study MRIPSx2, participants of varied experience levels with CB examined a mixed rea lity human patient using MRIPS -CBE. If performance with MRIPS -CBE reflects users experience level with CBE, MRIPS -CBE will be validated for evaluating learners CBE skill. This approach has previously been used by Balkissoon et al. to establish the vali dity of a rectal exam simulator for evaluating learners rectal examination skills [ 42 ]. Although we have already established validity for evaluating cognitive and psychomotor skills, we will again evaluate users in all three skill sets: cognitive, psychomotor, and affective. 4.3.1 Study Design and Procedure The study was conducted in two stages, in late May and early July 2008, in order to capture 2nd, 3rd, and 4thyear medical students as well as interns, residents, and clinicians. Forty -two medical students, residents, and clinicians at the Medical College of Georgia enrolled in the study. The population breakdown is presented in Table 4 -3.

PAGE 140

140 In the May 2008 study, participants completed a background survey to assess experience in CBE and then performed a CBE using MRIPS -CBE. The July 2008 study participants performed CBEs of two mixed reality human patients using MRIPS -CBE. In between the two MRIPS -CBE experiences, participants completed a review session in which they reviewed their CBE from the perspe ctive of the patient. The analysis presented here focuses only on the first MRIPS CBE performed by participants. 4.3.2 Measures Measures once again focus on cognitive, psychomotor, and affective components of the CBE. The measure of cognitive performance is the same medical history checklist used in Study MRIPS -SP, shown in Table 4 -1. Data collected with this checklist is shown in Appendix F. The psychomotor component evaluated was the completeness of palpation of the breast. Study MRIPSx2 marked the introduction of an improved version of the MRIPS CBE mannequin, containing 64 force sensors (42 in the breast). This improvement (over the 12 force sensors in the breast used in Study MRIPS -SP) allowed us to use a more sensitive, precise measure of palpation completeness: the percent of breast tissue palpated. The location of each sensor was labeled by hand in an image of the mannequin. The area of tissue in which each sensor could detect force was estimated manually by observing sensor values while apply ing force to the breast in the pattern of a grid of approximately 1 x 1 squares. The area in which each sensor detected force was modeled as a circle. During the exam, if a sensor reported a value indicating palpation (light pressure or greater), the ci rcle corresponding to that sensor was included in the area measured. The output generated by a participants exam is shown in Figure 4 -3. The percent area palpated by each participant is given in Appendix G.

PAGE 141

141 We have since developed an automated, more pre cise method for determining the area of tissue palpated as well as the force palpated with; this method is presented in Section 7.4 To measure affective performance, we once again had medical experts review video of critical moments in participants exams. Three critical moments were included in the scenario used in this study. The first critical moment occurred when the participant asked about a family history of breast disease or cancer. The patient indicated that her mother had pas sed away recently fr om cancer: I lost my mom to cancer two years ago. The second critical moment was triggered by the participants action of opening the physical hospital gown of the MRIPS -CBE haptic interface. This occurred as the participant was about to begin palpating the patients breast. The patient exclaimed Wait! Im scared. What if you find cancer? The third critical moment occurred after palpation when participants instructed the patient that she needed a mammogram. The patient asked Do I really have to g et a mammogram? I mean, my mom was fine, then she had a mammogram and all of the sudden she was really sick? The instrument used by the video reviewers was simplified from Study MRIPS -SP, in an effort to reduce the inter -rater variability observed in Stu dy MRIPS -SP. The new instrument contained the same item querying appropriateness but reduced the four empathy items to a single item. The instrument is presented in Appendix A.2 and the ratings of participants in Appendix A.3 Video review was conducted only for the 19 participants who completed the study in July 2008. 4.3.3 Analyzing the Impact of Experience on Performance Similar prior work in medical simulation has taken the approach of including participants of a wide variety of experience levels and performing post -hoc analysis to

PAGE 142

142 find a grouping of experience levels which were discriminated between by the simulator (i.e. the analysis looked for two or more groups with significantly different performance using the simulator). For example, in evaluat ing validity of a rectal exam simulator, Balkissoon et al. started with participants in three groups: medical students, residents, and clinicians. However, the final analysis included only two groups based on the number of rectal exams previously performed: low experience (< 5) and high experience (>= 5 exams) [ 42 ]. Because we analyze three distinct aspects of performance, we considered different groupings of experienced and inexperienced participants for each of the cognitive, psychomotor, and affective p erformance. Affective performance consisted of comforting a fearful patient and expressing empathy concerning the patients loss of her mother to breast cancer. Expressing empathy and comforting a patient are skills which are learned in a variety of scena rios. Thus experience with CBE is not necessarily a good predictor of affective performance. Instead, overall experience with human patients is expected to be a better criterion for classifying affective performance. Consulting with medical educators a t the Medical College of Georgia, we grouped participants into two groups to compare affective performance. The affective experienced group contained residents and clinicians and the affective -inexperienced group contained medical students and interns. Ps ychomotor performance in CBE can only be improved through performing CBEs [ 4 ][ 65 ]. In evaluating psychomotor performance, we grouped participants based on their experience in CBE of human patients. This yielded two groups. The psychomotor experienced gr oup contained participants who had completed a womens health

PAGE 143

143 clerkship including having their CBE evaluated by an expert in clinic. The psychomotor inexperienced group had not yet begun this clerkship. This classification corresponded with the <5 exams vs. >= 5 exams classification used by Balkissoon [ 42]. In this analysis, we included only medical students and interns, as these participants had received training in CBE at the Medical College of Georgia. This institution teaches the Mammacare method of examination (circular palpations, vertical strip pattern, three levels of pressure) [ 65 ]. The Mammacare method is known to be the most effective method for finding masses, but many other methods exist [ 4 ]. By restricting analysis to participants who hav e been taught the Mammacare method we avoided incurring additional variance due to the method of examination used. Cognitive performance was measured using the medical history checklist of Table 4 -1. This checklist focuses on medical history items related to assessing breast health history and breast cancer risk factors. For this reason, we expect experience in CBE to be a predictor of performance in this cognitive task. Thus the cognitiveexperienced and cognitiveinexperienced groups used the same clas sification criteria as the psychomotor groups. As this measure assesses completeness of the history as the number of items addressed in the medical history, we omitted clinicians and residents from this analysis. It is known that tactics in taking a medi cal history change as advanced levels of experience are reached [ 12]. The more experienced residents and clinicians are likely to use a smaller more focused set of questions and infer information from their past experiences or omit a question if the same information can be obtained in the physical exam (e.g. omitting asking about nipple discharge because nipple discharge can be tested in the exam). Indeed, use of checklists in SP encounters has

PAGE 144

144 been found to be biased against these more efficient clinici ans or experts [ 118]. Expert behavior is in contrast to behavior we have observed with medical students, who tend to stick to a script of questions they have learned in lecture. Thus the number of relevant questions asked appears to be a good indicator of student skill and performance, but not a good indicator of highly experienced residents and clinicians. 4.3.4 Results Results are summarized in Tables 4-4 through 46. Performance in MRIPS -CBE distinguished between experienced and inexperienced students in cognitive, psychomotor, and affective tasks. This further establishes the validity of MRIPS -CBE for evaluating learners in these three skill sets. 4.3.4.1 Cognitive The experienced medical students and interns who had receiv ed expert tutelage in the women s health clerkship asked an average of three more questions (from the medical history completeness instrument, Table 4-1) than the inexperienced medical students (Table 4-4). Experienced students asked 12.3 2.3 critical questions compared to 9.3 2.9 for the inexperienced students. This difference was significant by an independent samples t -test: t(26) = 2.9, p < 0.01. This result provides evidence in addition to that of Study MRIPS -SP to validate MRIPS -CBE for evaluating the cognitive componen ts of learners CBEs. 4.3.4.2 Psychomotor Experienced medical students and interns with five or more prior CBEs performed significantly more complete examinations of the breast than did inexperienced medical students with less than five prior CBEs. Experi enced participants palpated an average of 84.2 % 7.6% of the breast tissue with light or higher pressure, while inexperienced

PAGE 145

145 participants palpated 74.8 % 13.4 % of the breast tissue. The difference was -te st: t(30) = 2.4, p = 0.03. This result provides evidence in addition to that of Study MRIPS -SP to establish the validity of MRIPS -CBE for evaluating the psychomotor component of learners CBEs. 4.3.4.3 Affective Six medical educators and clinicians evaluated participants affective performances in three critical moments (see Section 4.3.2) using the instrument of Appendix A.2 As in Study MRIPS -SP, interrater reliability was not above adequate agreement indicated by a coefficient value of 0.70 [ 117]. Considering that adequate inter -rater reliability was not achieved using the previously validated instrument used in Study MRIPS -SP nor with the simplified instrument used in this study, it appears as though rating of empathy is too subjective to achieve high inter -rater reliability. The six expert reviewers scores were averaged to produce a score of acceptability and empathy for each participants response to each of the three critical moments (Table 4-6). The experienced group outperformed the inexper ienced group in both acceptability of response and empathic content of response for all three critical moments. However, the difference in appropriateness was significant for two out of three moments and the difference in empathy was significant for only the critical moment concerning the patients fear of having a mammogram. Given better inter -rater agreement, we expect that the amount by which the experienced participants outperformed the inexperienced participants will become significant. Without inter-rater agreement we still accept that MRIPS -CBE is able to distinguish between experienced and inexperienced affective performance, as the mammogram critical moment produced

PAGE 146

146 significantly different performance based on experience. This result establishes the validity of MRIPS -CBE for evaluating the affective component of learners CBEs. 4.3.5 Discussion The results of this study establish the validity of MRIPS -CBE for evaluating learners cognitive, psychomotor, and affective performance. As in Study MRI PS-SP, the experimenter triggered virtual human responses when speech recognition failed. This was performed uniformly across participants, so it does not affect the statistical results of the study. 4.4 Conclusion Study MRIPS -SP established that users of MRIPS -CBE use their real -world skills to perform CBEs that are similar to CBEs of SPs along cognitive, psychomotor, and affective dimensions. Study MRIPSx2 established that experienced participants are able to perform more complete and correct CBEs than inexperienced participants, along cognitive, psychomotor, and affective dimensions. These two studies established the validity of MRIPS -CBE as a tool for practicing CBEs and as an instrument for evaluating learners cognitive, psychomotor, and affective s kills in CBE. From a computer science or human-computer interaction perspective, the main result of these studies is to provide evidence that users of mixed interpersonal simulation treat their mixed reality human interaction partners similarly to how they treat human interaction partners. From a simulation perspective, the main result of these studies is to show that an interpersonal simulator incorporating haptic interfaces elicits and can be used to evaluate users real world cognitive, psychomotor, and affective skills in an interpersonal scenario.

PAGE 147

147 From a medical education perspective, the main result of these studies is to establish MRIPS CBE as an additional practice opportunity for learners of CBE. Since current medical curricula do not provide enoug h practice opportunities for many learners to reach competence in CBE [ 9 ], MRIPS -CBE is a viable addition to a medical curriculum for teaching CBE. MRIPS -CBE can be used to augment an existing SP curriculum and has been shown to be a valid substitute for an SP interaction. Prior medical simulators have been incorporated into medical curricula as a result of establishing validity for evaluating learner skill (e.g. Pughs breast palpation, pelvic, and rectal exam trainers into Northwestern Universitys curri culum [ 5 ][ 38 ][42 ]). However, prior interpersonal simulations, for which content validity has been established, have not been accepted into interpersonal skills curricula (e.g. the medical history simulator of Johnsen et al [ 10 ] is not currently used in th e curricula of either medical school which assisted in its development). One of the driving goals of this work is to establish that mixed interpersonal simulation is an invaluable addition to a curriculum in which the simulated interpersonal scenario is c urrently underserved (e.g. CBE education in current medical curricula [ 9 ]). For this reason, we continue beyond establishing the validity of MRIPS -CBE, to investigate learning and trainin g transfer in MRIPS -CBE (Chapter 9). Before evaluating learning and training transfer, we incorporate real time and post experiential feedback of learner performance (Chapters 6-8). Such feedback is mandated for learning in interpersonal scenarios [ 12].

PAGE 148

148 Table 4 1. Items in the medical history completeness checklist used to evaluate participants cognitive performance. History of Present Illness Description of current complaint (for MRIPS, pain; for SP, breast mass) Location of current complaint (for MRIPS, pain; for SP, breast mass) Presence of discharge Other breast changes (e.g. redness, skin puckering) Medical H istory Menarche: age of onset Menarche: currently occurring or post menopausal Use of hormones for birth control or hormone replacement Past pregnancies Past breast problems Screening: yearly clinical breast exams by a doctor Screening: monthly self breast exams Screening: yearly mammograms Past hospitalizations Past surgeries Current medications Family History Family history of cancer Other family history of medical problems Social History Sm oking / use of tobacco Drinking / use of alcohol Health risks involved in employment Table 4 2. Results of video review of critical moments Study MRIPS SP (scores are averages of the multiple experts ratings) MRIPS SP CI Result Wait! Im scared. What if you find something bad? Appropriate 3.92 0.39 3.48 0.30 [ 0.10, 0.78] Non inferior Empathic 1.30 0.17 1.28 0.007 [ 0.10, 0.15] Equivalent Can you tell me what happens next? Appropriate 3.89 0.49 3.24 0.85 [ 0.28, 1.02] Non infer ior Do you think it could be cancer? Appropriate 3.64 0.68 3.67 0.54 [ 0.44, 0.38] Equivalent Empathic 3.10 0.93 2.95 0.81 [ 0.40, 0.71] Non inferior

PAGE 149

149 Table 4 3. Population breakdown for Study MRIPSx2. Classification Population Study stage Educational experience in CBE Medical student Yr. 2 5 May Lecture Medical student Yr. 3 12 May, July Womens health clerkship with real patients Medical student Yr. 4 12 May, July Clinic (real patients) Intern year 3 July Clinic (real patients) Resident 5 July Clinic (real patients) Clinician 5 July Clinic (real patients) Table 4 4. Cognitive performance results for Study MRIPSx2. Group Pop. Size Mean stdev 95% CI Test Sig. Experienced* 12 12.3 2.3 items [10.8, 13.7] t test, t(26) = 2. 9 p < 0.01 Inexperienced** 16 9.3 2.9 items [ 7.8, 10.9] (*) Post womens health clerkship; (**) Pre womens health clerkship. Table 4 5. Psychomotor performance results for Study MRIPSx2. Group Pop. Size Mean stdev 95% CI Test Sig. Experience d* 14 84.2% 7.5 % [79.9, 88.6] t test, t(30) = 2.4 p < 0.05 Inexperienced** 18 74.8% 13.4 % [68.1, 81.4] (*) Post womens health clerkship; (**) Pre womens health clerkship.

PAGE 150

150 Table 4 6. Affective perfor mance results for Study MRIPSx2 (scores are averages of multiple experts ratings). Group Pop. Size Mean stdev. Test statistic Sig. 1. Wait! Im scared. What if you find cancer? a. Appropriate? 3.9 0.7 t(25) = 0.5 ns Experienced* 8 3.8 0.4 Inexperienced** 19 b. Empathetic? Ex perienced* 8 2.5 1.1 t(25) = 0.8 ns Inexperienced* 19 2.4 0.9 2. I lost my mother to breast cancer two years ago a. Appropriate? Experienced* 9 3.9 0.5 t(23) = 2.2 p < 0.05 Inexperienced** 16 3.4 0.6 b. Empathetic? Experience d* 9 2.3 0.9 t(23) = 0.9 ns Inexperienced* 16 2.0 0.9 3. Do I really have to get a mammogram a. Appropriate? Experienced* 8 4.2 0.3 t(21) = 2.4 p < 0.05 Inexperienced** 15 3.7 0.5 b. Empathetic? Experienced* 8 3.3 0.9 t(21) = 2.6 p < 0.05 Inexperienced* 15 2.2 0.9 (*) Residents and clinicians; (**) 2nd4th year students and interns. Figure 41. Appearance of the MRIPS -CBE patient for Study MRIPSSP. Participants wore a head -mounted display to view the virtual human.

PAGE 151

151 Figure 42. The appearance of the MRIPS -CBE mixed reality human in Study MRIPSx2. The users view in the HMD is shown as the outlined inset. Figure 43. Visualization of a participants CBE completeness. Green indicates tissue palpated and red indicates tissue missed in the exam.

PAGE 152

152 CHAPTER 5 MRIPS -NE U R O This chapter describes the development of MRIPS -NEURO, an interpersonal simulation of a neurological examination with abnormal findings of cranial nerve palsy. An early version of the system was published in the proceedings of the IEEE Symposium on 3D User Interfaces 2009 [ 28 ]. The design of the system and ability to simulate the neurological exam has since been significantly enhanced. Collaborators : Kyle Johnsen developed the original eye movement model, virtual human abilities such as counting the users fingers and reading from an eye chart, and a portion of the tool interaction. The appearance of the virtual human and tools used in the exam was developed by Brent Rossen, Kyle Johnsen, and me. Medical collaborators Juan Cendan, Bayard Miller, Lou Ritz, and Thea Nalls provided information on neurological exam content and feedback concerning the correctness of the abnormal eye movements. Personal contributions : I developed a new eye movement model, integrated touch into the interaction, developed virtual human nonverbal actions to support additional neurological exam tests, and developed tool use such as the fundoscopic exam. Relevance to thesis: The thesis states that interpersonal simulation incorporating instrumented haptic interfaces and providing real -time evaluation and feedback of performance improves users cognitive, psychomotor, and affective performance in an interpersonal scenario. To prove this statement, we must develop such an interpersonal simulation and evaluate the impact of feedback on learner performance within this interpersonal simulation. This chapter describes the design of a mixed reality

PAGE 153

153 interpersonal simulation incorporating haptic interfaces and providing real -time feedback (Chapter 8). The simulation described in this chapter, MRIPS -NEURO, is used to evaluate the impact of this real -time feedback on learners cognitive, psychomotor, and affective performance (Chapter 8). 5.1 Introduction A mixed reality interpersonal simulation, MRIPS -NEURO, was developed to simulate a neurological exam with abnormal physical findings, e.g. abnormal eye movements. The motivation for developing this simulation was to provide a learning opportunity not afforded by current medical educat ion and simulation approaches. MRIPS -NEURO affords communication and physical examination of a life -sized virtual human agent through speech and touch. MRIPS -NEURO also introduces a new affordance to MRIPS: handheld tool use and gestures through the manipulation of the haptic interface. We have developed ophthalmoscope, eye chart, and gesture (appearing as the users right hand) tools for communication and exam performance. Using MRIPS -NEURO, Figure 5.1, novice learners of neurological exams can practic e history taking (verbally evaluating nonvisible symptoms such as headache), physical exam tasks (eliciting abnormal physical findings), synthesizing the information collected in history taking and physical exam into a differential diagnosis of the patien ts abnormality, and practice affective exam components such as addressing the patients concerns and issues of patient safety. In addition to the science oriented advances of incorporating hand-held tool use and hand gestures into the interaction with the virtual human, MRIPS -NEURO incorporates an engineeringoriented advance: creation of a virtual human that is able to simulate cranial nerve abnormalities.

PAGE 154

154 As with MRIPS -CBE, the design of MRIPS -NEURO is divided into three parts: the appearance of the vir tual human and virtual world, the underlying virtual human simulation, and the haptic interface. The virtual human simulation of MRIPS -NEURO operates similarly to the simulation module of MRIPS -CBE described in Chapter 3, so we omit a full treatment from this chapter. This chapter provides background information concerning the neurological exam, then focuses on the development of a virtual human capable of presenting the physical findings necessary for the exam, and on the incorporation of a haptic interf ace for performing the physical exam. 5.1.1 The Neurological Exam Requires Cognitive, Psychomotor, and Affective Skills As with the CBE, the neurological exam is composed of cognitive, psychomotor, and affective components. The cognitive parts of the crani al nerve exam are: Recall of important questions to ask in order to compile a medical history and assess symptoms that are not physically expressed, e.g. onset of vision problems, headache [ 47 ]. Recall of physical exam tasks that should be used to evaluat e disorders of the patients 12 cranial nerves. Synthesis of information gathered from the medical history and the cranial nerve tests into a differential diagnosis determining what nerve is affected. The synthesis of information is known to be the most difficult component of the exam, as there are many tests and a dozen cranial nerves, and learners do not have experience conducting a physical exam with abnormal findings alongside taking a medical history of a patient (because learners do not have opportu nities to examine peers, SPs, or real patients with abnormal findings).

PAGE 155

155 Many of the tests used in the physical exam involve manipulation of hand-held tools or gestures, and could be thought of as psychomotor tasks. However, educators do not identify the m ajority of these tasks as providing difficulties for learners. Thus only two tasks will be considered to be psychomotor tasks in this work. One of the two psychomotor tasks is using the ophthalmoscope to conduct a fundoscopic exam. The ophthalmoscope i s a complex instrument which requires significant practice to master. MRIPS -CBE does not aim to teach mastery of the ophthalmoscope, as this is better served by existing purely physical approaches such as practice with a peer or mannequin [ 56]. Instead M RIPS -CBE incorporates a simplified fundoscopic exam which focuses on the cognitive task of interpreting the appearance of the patients retina. The second psychomotor task is testing the patients eye movements by sweeping ones finger (or light) in the shape of an uppercase H while the patient follows the finger with his eyes. This task requires that the shape of the H and depth of the finger from the patients head be such that the extremes of the patients vision are examined (e.g. the patient mus t move his eyes as far left as possible). If the novice learner does not use an H that elicits the extremes of eye movement, the learner misses information that may indicate an abnormality with one of the cranial nerves, e.g. the patients inability to move an eye to one of the extremes. In addition to these cognitive and psychomotor tasks, the learner can also practice affective components of the exam. As with the CBE, the patient is often anxious and it is the learners job to keep the patient comfort ed through appropriate expressions of empathy. Unique to the neurological exam are concerns of the patients immediate

PAGE 156

156 safety. For example, did a patient with severe double vision drive to the clinic? Is it safe for him to drive home? A learner who is more concerned with a correct diagnosis than understanding what the patient is going through may disregard issues of safety. The patients safety is additionally the doctors responsibility and even liability [ 119], so it bodes well for a learners future to b e exposed to issues of patient safety as a novice. 5.1.2 Evaluating MRIPS NEURO The evaluation portion of my dissertation focuses on two components: 1) evaluating the impact of real -time feedback on learner performance and 2) evaluating whether learni ng occurs in MRIPS and transfers to the real world interpersonal scenario. The first component is addressed in MRIPS -NEURO: the impact of real -time feedback of learners cognitive, psychomotor, and affective skills is evaluated in a formal study described in Chapter 8. However, the second component can not be fully evaluated in MRIPS -NEURO because a standardized real world scenario is unavailable for evaluation of learning and training transfer. Neither SPs nor real patients with abnormal findings are available to evaluate learners skills in a real world neurological exam. In fact, MRIPS -NEURO represents the first step towards a standardized platform for evaluating learners skills in the interpersonal scenario of a neurological exam with abnormal findings. Without the capability to directly evaluate learning in MRIPS -NEURO, we focus on a necessary step towards learning: content validity. Establishing content validity answers the question: Does MRIPS -NEURO simulate abnormal physical findings and the per formance of neurological exam tasks to a highenough fidelity that novice

PAGE 157

157 learners are able to practice diagnosing the cranial nerve disorder and arrive at a correct diagnosis? Such a practice opportunity is not provided by current medical education and simulation approaches due to the inability to simulate abnormal findings. Thus demonstrating the content validity of MRIPS NEURO would be a significant step towards providing medical educators and students with a platform affording practice and evaluation of the neurological exam scenario. To establish the content validity of MRIPS -NEURO, we conducted a user study which focused on whether medical students (2nd and 3rd year) were able to arrive at a correct differential diagnosis through performing an exam ination in MRIPS -NEURO. Novice learners have previously learned the mapping of symptoms and abnormal physical appearances to disorders of specific cranial nerves (from lectures and books). To establish content validity, we needed to show that these novic e learners could use MRIPS -NEURO to perform all of the tests necessary for the learner to collect enough information to arrive at a correct diagnosis. These tests included manipulating handheld tools, gestures, and communicating with the virtual human. Twelve of fourteen participants were able to arrive at a correct diagnosis; this was a proportion significant beyond chance, by a oneway Chi -square test ( X2 = 5.8, p = 0.02), establishing the content validity of MRIPS -NEURO. This study is described furth er in Section 5.5. 5.2 The Neurological Exam In this work we use the term neurological exam to refer to a focused exam of the cranial nerves. There are twelve cranial nerves; MRIPS -NEURO focuses on simulating tests used to examine cranial nerves 2, 3, 4, 5, 6, 7, and 12. The functions of these nerves and symptoms resulting from disorders of these nerves are listed in Table 51.

PAGE 158

158 Examining this subset of nerves requires manipulation of an ophthalmoscope with a light, an eye chart, and verbal and nonverbal c ommunication involving hand gestures and references to tools. The virtual human developed for MRIPS -CBE complains of experiencing double vision. The primary tasks of an exam to evaluate double vision are listed in Table 5 -2. The virtual humans abilitie s to support these tasks and other components of a neurological exam are discussed in Section 5.3.2. 5 .3 A Virtual Human Agent to Simulate Cranial Nerve Disorders We developed a virtual human agent capable of displaying the physical symptoms of cranial ner ve 3 and cranial nerve 6 palsies and capable of performing many of the tasks asked of the patient in a neurological exam of cranial nerves 2, 3, 4, 5, 6, 7, and 12. The design of this virtual human is broken into two sections: developing a model of eye movements that presents physiologically accurate abnormalities and designing the virtual human to perform tasks used to diagnose cranial nerve disorders. 5 .3.1 Eye Movement Model The most important aspect of simulating a cranial nerve disorder is displaying correct abnormal eye movements. These eye movements are the primary basis for diagnosis, in addition to other aspects of the patients appearance such as tilting of the head indicating CN4 palsy and secondary aspects such as headache [ 120] or trauma to th e head. In MRIPS -CBE the virtual humans left eye is the eye affected by the CN3 or CN6 disorder; the right eye retains normal movement. Both normal and abnormal movements are controlled by the same model. Moving the human eye to look at an object or perf orm a task such as following the doctors fingers requires a comp lex interaction of six muscles The original eye

PAGE 159

159 movement model, developed by Kyle Johnsen, sought to replicate the effects of each muscle on the yaw and pitch of the eye [ 28 ]. However, thi s approach only modeled the primary function of each muscle, making it unable to simulate physiology such as greater angle of pitch when looking directly up than when looking up and to the right. Additionally, this approach could not simulate abnormalitie s related to cranial nerve palsies such as having the affected eye look down and out when the virtual human is attempting to look straight ahead. For these reasons, a new eye movement model was developed that is not physically based, but does produce outpu t consistent with real physiology. This model uses linear interpolation of the eight cardinal eye positions to restrict eye movement in a way that appears physiologically correct for the cranial nerve disorder. The eight cardinal eye positions are displayed in Figures 52 through 54 for no CN disorder, CN3 palsy, and CN6 palsy respectively. Typically up and down are not considered cardinal positions, but they are included here. Through review of case data and textbook diagrams [ 60 ], the UC Davis ey e simulator [ 57], and discussion with medical collaborators, we estimated the yaw and pitch of each of the cardinal poses (and the default, looking straight pose) for normal eye movement, CN3, CN4, and CN6 these are the three cranial nerve palsies which result in abnormal eye movement. For each eye, the movement model defines a set of eight vectors corresponding to the cardinal movements, each 45 degrees apart. Each vector is associated with a (yaw, pitch) pair representing the maximum yaw and pitch of t he eye for that cardinal position. If we think of the Cartesian plane as having dimensions of yaw and pitch, these vectors divide the plane into eight sectors. This is illustrated in Figure 55.

PAGE 160

160 Given a desired gaze position (e.g. look to the left, follow my finger, or gazing at the users head position), this model outputs a new gaze position altered by the constraints imposed on eye movement by the cranial nerve disorder. For an affected eye, the process proceeds as (illustrated in Figure 55): 1 Giv en the desired gaze position, calculate the (yaw, pitch) required to rotate the eye from its default (straight ahead) position. 2 This (yaw, pitch) pair defines a vector d the desired gaze vector. 3 Assuming d originates from the origin of the normal eye model, determine which two normal eye movement vectors v1 v2 define the sector in which d lies. 4 Calculate the angles between d and v1 v2 as a1 and a2 5 Normalize and invert a1 and a2: a a1 = a1 / ( a1 + a2 ); a2 = a2 / ( a1 + a2) b a1 = 1.0 a1 ; a2 = 1.0 a2 6 a1 and a2 are now weights for linear interpolation. The smaller the angle between d and the neighboring vector ( v1 or v2 ), the larger the weight. 7 Using the abnormal vectors v1 and v2 corresponding to the v1 and v2 find the vector d defining the ma ximum allowed (yaw, pitch) along the desired gaze vector d : d = ( v1 a1 + v2 a2 ) / ( a1 + a2 ) 8 Desired gaze length ld = || d ||; Maximum gaze length lm = || d || 9 If ld > lm, set ld = lm. 10. Let s indicate the (yaw, pitch) vector corresponding to the abnormal ey e looking straight ahead. 11. The final gaze vector g is then calculated as: g = s + d*( ld / lm). Example vectors d and g are illustrated for the case of CN6 in Figure 55. The same process is performed for the normal eye, with the substitution of normal eye vectors for the abnormal eye vectors used in Step 7 (i.e. v1 = v1; v2 = v2 ). Once the final gaze yaw and pitch angles are calculated, the eye does not rotate to these angles instantly, but moves over time. To accomplish this we define the maximal an gle the eye can rotate per second and linearly interpolate between the current eye yaw and pitch angles and the desired gaze yaw and pitch angles based on the maximal angle per second and the duration of the current frame.

PAGE 161

161 We consulted with neurologists and eye movement experts at the University of Florida s College of Medicine to establish the correctness of the eye movements produced by this model. 5.3.2 Virtual Human Abilities to Support Neurological Tests In addition to displaying physiologically accura te eye movements, the virtual human has verbal and nonverbal behaviors to support additional neurological tests. The virtual human supports additional tests of cranial nerves 2, 3, 4, 5, 6, 7, and 12, with the following abilities: Pupillary response: The virtual humans pupils constrict when the ophthalmoscope light is shined into an eye. In the CN3 affected eye, the pupil remains dilated even in the presence of light. This test requires manipulation of the ophthalmoscope tool using the haptic interface. (Tests CN 2, 3). Eye movement: The patient can hold his head still and follow the ophthalmoscope light or the doctors finger with his eyes. This test requires user speech and manipulation of the gesture or ophthalmoscope tool using the haptic interfac e. (Tests CN3, 4, 6). For CN3, the virtual human can be asked to hold his drooping left eyelid up to enhance the doctors view of the eye movement. The virtual human can verbally express whether he sees double depending on where his eyes are looking. For example, with CN6 the patients double vision goes away if he looks to the right. Visual acuity: The virtual human can read from an eye chart, either reading the smallest line he can make out, or reading a line that the doctor points to using the haptic i nterface. This test requires user speech and manipulation of the eye chart tool using the haptic interface (Tests CN2, 3, 4, 6). With both eyes open, the virtual human will not be able to read any lines with CN3, 4, or 6. The virtual human is able to cov er either eye with his hand, allowing him to read the eye chart with 20/20 visual acuity. Facial sensation: The virtual humans facial sensation can be tested by touching the virtual humans face using the haptic interfaces gesture tool. If the eye is a ccidentally touched, e.g. by the ophthalmoscope during the fundoscopic exam, the virtual human blinks and jerks his head away as a reflex. (Tests CN 5).

PAGE 162

162 Facial maneuvers: The virtual human can smile, frown, raise his eyebrows, puff out his cheeks, blink, and wink. The patient can also stick out his tongue. These are all elicited through verbal instructions from the user. (Tests CN 7 and 12). 5.4 The Haptic Interface The haptic interface of MRIPS -NEURO takes a different approach than the haptic interface of MRIPS -CBE. Unlike MRIPS -CBE, the MRIPS-NEURO haptic interface does not provide a physical representation of the patient. Although touching the patients face is part of the neurological exam, the exam predominantly consists of tool manipulation, verb al references to manipulated tools, and hand gestures. Thus the haptic interface for MRIPS -NEURO should focus on providing natural manipulation of handheld tools and robust detection of hand gestures. 5.4.1 Prior Approaches Prior attempts to incorporate natural manipulation of handheld tools and hand gestures into mixed and virtual environments have taken two approaches: encumbering the user with gloves and wires or vision-based gesture recognition. Gloves containing bend and pinch sensors can provide l ow -noise recognition of gestures [ 121] suitable for this application. However gloves alone do not provide the feel and weight of hand-held tools. Most importantly, in our experience with over 500 endusers in the medical profession (at the University of Florida and Medical College of Georgia) requiring the user to wear encumberances such as gloves significantly decreases acceptability of the simulation and would be a significant hurdle to the incorporation of the simulation into a medical curriculum. Une ncumbered alternatives to gloves have focused on vision -based tracking of the users hands and has afforded simple hand and gesture based interaction in VEs [ 122][ 123]. However the amount and complexity of recognizable gestures is suitable

PAGE 163

163 only for simple interfaces such as those used for virtual environment navigation and selection (e.g. point, pinch), and the accuracy and update-rate of such systems would also reduce acceptability of the simulation. Former collaborator Xiyong Wang developed a minimally e ncumbering (no wires or gloves) interface based on optical tracking of user worn infrared fiducials. However the optical tracking proved to be too noisy for the system to be usable to conduct the neurological exam. Problems that are trivial for humans, s uch as distinguishing one finger from two fingers prove to be extremely difficult to accomplish using vision and optical techniques. Because of the drawbacks of prior approaches, we chose to base our haptic interface around a handheld tracked device, sa crificing the naturalness of gestures for noise -free gesture recognition, and preserving natural manipulation of hand-held tools. 5.4.2 Haptic Interface: Wii-Remote and External Sensing The haptic interface for MRIPS -NEURO is a hand -held tracked input devi ce with the weight and shape of many hand held tools, such as the ophthalmoscope used in the neurological exam, and button inputs to simulate affordances of real tools. We chose a single hand-held device instead of having one device for each tool. This is motivated by the ability to overload the single handheld device to simulate multiple tools and perform multiple tasks. In contrast, the approach of having many tool -specific devices requires switching between devices any time a new tool is desired, and requires significant development if a new tool is required [ 124]. The single handheld device takes a Swiss army knife approach, affording the simulation of multiple handheld tools using a single interface which is representative of a wide class of handheld tools. Thus the hand -held tools in MRIPS -CBE are virtual and manipulated by a

PAGE 164

164 handheld device that serves as the physical haptic interface (providing both passive haptic feedback through shape and weight, as well as activehaptic vibratory force feedback) The virtual tools of MRIPS -CBE are an ophthalmoscope, an eye chart, and the gesture tool which appears as the users right hand and fingers. The tools and their use in the exam are described in Section 5.4.3. The hand-held device used in the haptic interface was chosen to be the Nintendo Wii-Remote (wiimote) which we augmented with external six degree of -freedom tracking using a four camera Naturalpoint OptiTrack infrared tracking system. The wiimote was chosen because it is shaped as a hand-held tool and has high degree of freedom control. The wiimote features an array of integrated sensors that are reported at 100Hz update rate over a Bluetooth connection: 11 buttons, 3 orthogonal accelerometers (+/ 3g), and a 45-degree field of view infr ared camera (128x96) that tracks up to 4 points at 1/8 to 1 pixel resolution depending on the size of the infrared point. In addition, the wiimote can display information through integrated LEDs, speaker, and vibration motor. Open source software was use d to acquire data from the wiimote across the Bluetooth interface [ 125 ]. A summary of the wiimotes potential as an interaction device and descriptions of many applications can be found in the work of Lee [ 126]. External infrared optical tracking of the wiimote provides the position and orientation of the wiimote at 100Hz update rate and sub-centimeter accuracy. The features of this approach are: Robust tool use: The six degrees of -freedom pose (position and orientation) of the wiimote are measured at a high update rate (100Hz) with low noise (sub-centimeter). The virtual tools take on the six -degree of freedom pose of the wiimote.

PAGE 165

165 Tool control and gestures: Beyond six degreeof -freedom rigid pose tracking, the wiimote controls other aspects of the tool s through button presses, e.g. turning the light of the ophthalmoscope on and off, and the virtual hand can form a variety of poses (e.g. grip, point, lie flat). Button presses are sampled at 100Hz, providing robust control over the gestures. Noise-free g esture recognition: With the gesture tool representing the users hands, gestures are chosen at the press of a button. If the user intends to make a gesture of two fingers held up, he presses the finger -up button twice. This instructs the simulation t o display the virtual hand with two fingers held up. The user sees his virtual hand holding up two fingers and knows that the virtual human also recognizes that he is holding up two fingers. There is no opportunity for error in the gesture recognition, i n contrast with the other gesture interface approaches described in Section 5.4.1. Correct kinesthetic information: The wiimote is manipulated using similar muscle movements that are used with the real physical tools being simulated. Passive -haptic feedback: The wiimote provides the weight and shape similar to many hand held tools, providing passive-haptic feedback similar to that of the tools being simulated. Active -haptic feedback: providing vibratory force -feedback when the virtual tools collide with the virtual world. 5.4.3 Virtual Hand Held Tools and Hand Gestures Three virtual tools were created for MRIPS -CBE: an ophthalmoscope, an eye chart, and the hand and fingers gesture tool. The user switches between tools by pressing a button on the wiimote. 5.4.3.1 Ophthalmoscope An ophthalmoscope is a handheld tool equipped with a lens for viewing the back of the inside of the patients eye (the fundus) to determine the health of the patients retina and look for symptoms of cranial nerve disorders such as retinal hemorrhages. The ophthalmoscope moves with six degrees of -freedom and its position is mapped one-to one to the position of the wiimote.

PAGE 166

166 The ophthalmoscope size, shape, and weight are closely approximated by the wiimote, providing passive haptic f eedback. As the ophthalmoscope is often held close to the patients head and sensitive eyes, it is useful to inform the user if they contact the patient with the (virtually) metal and non-sterile ophthalmoscope. Active-haptic feedback is provided in the form of force feedback when the virtual ophthalmoscope makes contact with the virtual humans head. If the ophthalmoscope makes contact with the virtual humans eye, vibratory force feedback is received and the virtual human blinks and jerks his head back The ophthalmoscope is typically used to perform two tests: the pupillary reflex test and fundoscopic test. The pupillary reflex test is performed by turning on the light of the virtual ophthalmoscope by pressing the trigger button on the rear of the wiimote, and manipulating the wiimote to aim this light into each of the virtual humans eyes (Figure 5 -6 A). The fundoscopic test is performed by turning on the light and moving the ophthalmoscope close to the virtual humans eye (<8 cm). When this is detected, an image of the fundus of that eye is displayed above the virtual humans head (Figure 5-6 B). This provides a simplistic simulation of the fundoscopic exam, as use of the ophthalmoscope to view the fundus is simplified, e.g., we use a static im age of the fundus instead of an ophthalmoscopeorientation-dependent image. However, our goal is not to train the fundoscopic exam, but to allow a learner to obtain the information provided by a fundoscopic exam for use in diagnosing the cranial nerve dis order. 5.4.3.2 Eye chart The virtual eye chart is used to test the patients visual acuity. The eye chart is fixed in 3D space, much as a physical eye chart is affixed to a wall. Rather than the eye chart position and orientation being controlled by the wiimote, a virtual finger pointing to

PAGE 167

167 a line on the eye chart is manipulated. The eye chart contains eight lines. To point to a line, the user translates the wiimote vertically. The vertical (Y axis) position of the wiimote is binned into eight interval s corresponding to the eight lines on the eye chart. This illustrates the adaptation of user input to specific tool characteristics. The eye chart is used in the visual acuity test While the virtual finger is pointing to a line on the eye chart, the user is able to ask the patient to read the pointed-to line, e.g. can you read this line? The virtual human reads the currently pointed to line if he is able to depending on the CN affected (Figure 5 -7 left). With CN3, 4, and 6, the patient can not read any of the lines with both eyes open. The user is able to ask the patient to cover one eye (Figure 57 right) and try again with one eye closed, the virtual human has 20/20 visual acuity. Alternatively the user can ask the virtual human to report what is the lowest line you can read? 5.4.3.3 Hand gesture tool To provide gesture inputs used in neurological exam tests, a virtual hand and fingers tool is provided. The hand moves with six degrees of -freedom and its position is mapped one to one to the position of the wiimote. Gestures that can be performed using this tool include making a fist, holding between 1 and 5 fingers up, pointing with one finger, and shaking a finger. The hand tool is used in many of the neurological exam tests. Because of the noise -free gesture recognition, the state of the virtual hand represents both the state of the users hand and the hand that the virtual human sees and responds to there is no ambiguity resulting from gesture recognition error. Finger counting test: The hand can transform from an open hand to a clenched fist. The number of fingers the doctor is holding up can range from 0 5. The up and

PAGE 168

168 down arrows on the directional pad of the wiimote are used to raise and lower fingers. To test the patients ability to maintain binocular vision in all fields of gaze, the user will hold one or more fingers up and ask the patient to look straight ahead and report how many fingers do you see? (Figure 5-8). If the fingers are held outside of the virtual humans field o f view, he will answer I cant see your hand. For the double vision disorders (CN3, 4, and 6) if the fingers are in view of only one eye the virtual human will report the number of fingers held up; if the fingers are in view of both eyes, the virtual hu man will report twice the number of fingers held up. If the fingers are in view of only one eye, or one eye is covered, the virtual human will report the correct number. Because of the noise -free gesture recognition, if the virtual human reports twice the number of fingers held up by the virtual hand, the user can be certain that the virtual human is experiencing double vision; there is no ambiguity that the gesture recognition may be malfunctioning. Oculomotor (eye movement) test: By asking the patient to follow my finger (alternatively the ophthalmoscope can be used for this test follow the light), the user can test the functionality of the patients oculomotor muscles and, correspondingly, cranial nerves that innervate these muscles (Figure 5-9). The virtual human holds his head still facing forwards, and attempts to follow the position of the finger with both of his eyes. Because the wiimote and finger move in a one-to one correspondence, the user receives the same kinesthetic feedback as he woul d in the real world exam. This correct kinesthetic information is necessary for learning the psychomotor task of moving the finger in the shape of an uppercase H to test the extremes of the patients vision.

PAGE 169

169 Peripheral vision test: To test the patient s peripheral vision, the user holds the virtual hand outside of the virtual humans peripheral vision, instructs the patient tell me when you see my hand and then proceeds to move the hand into the patients peripheral vision. The virtual human answers I can see it now when the finger enters the field of view of either eye. Alternatively the user can raise one finger on the hand, hold the hand in the patients peripheral vision, shake the wiimote, and ask the patient to tell me when you see my finge r shake. Shaking is detected as changing values in the wiimotes internal accelerometers. Facial sensitivity test: To test if the patient has feeling in the face, the user can poke the virtual humans face with one or more fingers and ask can you feel this (Figure 510). The user knows when he is making contact with the virtual humans face because vibratory force -feedback is provided by the wiimote when the fingers or hand collide with the face. Collision detection is performed using the meshes of t he virtual humans head and the hand and finger tool, using the OPCODE Optimized Collision Detection library incorporated in the Ogre 3D rendering engine. It is important to note that if a test requires both user speech and tool manipulation, as is the cas e for all the tests using the hand gesture tool, the simulation module is designed to perform the test asynchronously or synchronously, whichever is appropriate for the test. For example, how many fingers do you see? wants synchronous information, so th e virtual human responds (instantly) based on the number of fingers held up on the virtual hand at the time when the simulation module receives the user speech input. However, tell me when you see my finger shake and can you feel this ambiguously refe r to events that could be happening at that point in

PAGE 170

170 time or in the near future, and are handled as asynchronous commands. When one of these utterances is received by the simulation module, it puts the simulation into a state in which it is actively looki ng for a finger shake or a collision between hand and head. If the finger is shaking, was recently shaking ( e.g. within 4 seconds into the past), or begins to shake within the next 10 seconds (and before another command is given by the user or the hand tool is deselected), the virtual human will report that he sees the finger shaking. Allowing for asynchronous events provides more robust communication e.g. the user does not have to continuously shake the wiimote while asking the question multiple times until the two actions coincide. 5.4.3 The Haptic Interface Enhances Communication in Interpersonal Simulation The tool, gesture, and speech interaction afforded by the haptic interface and speech interface enhance the communication of the interpersonal sim ulation. Touch of the virtual human can be used for communication, similarly to MRIPS -CBE. Touch of the hand -held tools also enhances communication by providing conversational grounding. A common ground is a pool of mutually agreed upon information and s erves as a way to ensure that the message intended to be communicated is received intact by ones communication partner [ 37 ]. Grounding has been provided in a limited form in previous interpersonal simulations. For example in Gandalf, a solar system educ ation application, the user could direct her head gaze to point to a planet and tell Gandalf lets go there [ 79 ]. Grounding is enhanced in MRIPS -NEURO, due to the noise -free gesture recognition. Because the state of the hand held tools is never ambiguous to either the human user or the virtual human, the tools serve as grounding objects in many facets of the neurological exam scenario. Grounding examples (with the

PAGE 171

171 information provided by the tool in parentheses) include: can you read this line (which line?), how many fingers do you see (held up on the users hand), follow the light (the light on the ophthalmoscope), and whats the lowest one you can read (lowest line on the eye chart), can you feel this (the users finger). Just as interpersonal touch enhances communication in MRIPS -CBE, touch for manipulation of hand-held tools enhances communication in MRIPS -NEURO. 5.5 Usability and Content Validity of MRIPS NEURO for Practicing Diagnoses of Abnormal Findings The main goal of MRIPS -NEURO is to provide novice learners with increased exposure to abnormal findings in the clinical context of a patient interaction. Learners are able to practice synthesizing physical findings with information gleaned through conversation with the patient. To dete rmine if this goal is met by the design of MRIPS CBE, we evaluated the usability of MRIPS -NEURO for practicing diagnosing abnormal findings in a focused (on cranial nerves) neurological exam. If a significant proportion of participants are able to correct ly diagnose the virtual human using the tests afforded by MRIPS -NEURO, this will also establish the content validity of MRIPS NEURO. 5 .5.1 Study Design and Procedure An observat ional study was conducted with nine 2nd year medical students at the University of Florida. Participants filled out a background survey concerning their experience in neurological examination. All students were considered novices in the neurological exam. However, they had different experience levels with neurological examination: All had coursework and some relevant web -based simulator use. However, four of the participants had just completed a neurology clerkship (had experience performing neurological exams on human patients) while five of the

PAGE 172

172 participants were just beginning a neurology clerkship and had no experience performing a neurological exam of a human patient. The distinct difference in experience level is beneficial to the study: more experienced students should be able to reach a correct differential diagnosis (the v irtual human had CN3 palsy), while less experienced students may not be able to reach correct diagnosis. If the experienced students can diagnose CN3 disorder in the virtual human, then MRIPS -NEURO is usable for collecting information through speech, gest ures, and tool use, and synthesizing this information into a diagnosis in the abnormal neurological exam. An additional five 2nd and 3rdyear medical students from the Medical College of Georgia were also recruited. All students had completed a neurology clerkship and are considered to be part of the experienced student group. These students completed the same procedure, but a different post experience survey was used. They are included here only to evaluate whether learners are able to reach a correct diagnosis using the symptoms presented by the virtual human and the tests afforded by MRIPS -NEURO. Participants completed brief speech recognition volume and quality tests (using Dragon Naturally Speaking 9.5). The participants then began the exam. During the exam, participants could press the home button on the wiimote to bring up a tutorial screen for the selected tool (Figure 5-1 1 ). After the exam, participants completed a brief survey to assess usability of the interface to complete the exam and rep ort their findings. 5.5.2 Results All 9 of the more experienced students (4 at UF and 5 at MCG), in addition to 3 of the 5 less experienced students, arrived at the correct diagnosis of CN3 palsy. This

PAGE 173

173 demonstrates the usability of MRIPS -CBE to collect th e data needed to arrive at a correct diagnosis in a neurological exam with abnormal physical findings. This also establishes the content validity of MRIPS -NEURO: MRIPS -NEURO simulates the neurological exam scenario to a highenough degree of fidelity that learners are able to apply their knowledge to arrive at a correct differential diagnosis. Twelve of 14 participants, 85%, a significant majority by a one way Chi -square test ( X2(1) = 5.8, p = 0.02) arrived at a correct diagnosis of a CN3 palsy. Because this is not a large population, the demonstration of content validity should be viewed as a preliminary result. In Section 8.4, a second user study is described, in which 17 of 18 participants arrived at a correct diagnosis of two virtual human patients w ith a cranial nerve disorder. Over these two studies, 29 of 32 participants were able to use MRIPS -NEURO to gather information regarding symptoms and synthesize this information into a correct diagnosis. This provides strong support of the content validi ty of MRIPS -NEURO, as 29 of 32 participants is a significant proportion by oneway Chi -square ( X2(1) = 19.5, p < 0.0001). In self -report data of the usability of MRIPS, p articipants rated the interface and MRIPS -NEURO as usable for performing the neurologi cal exam. In the usability survey, participants rated (1) the usability (dimensions of effectiveness efficiency and satisfaction ) of the simulator for the physical examination portion of the encounter, ( 2 ) the usability of the interface (controlling thr ee tools with one hand -held device), and ( 3 ) the usability of each of the tools The dimensions of usability were rated on the scale: 2(strongly disagree), 1(disagree), 1(agree), 2(strongly agree). Each item was phrased as clearly as possible to avoid confusion. For example, I was SATISFIED with the

PAGE 174

174 technology during the PHYSICAL EXAM of the patient. I felt comfortable using the technology and did not get frustrated." Given the small sample, the data for each item was collapsed into a higher level view of usability as "Yes" or "No." This was calculated by summing across each dimension and then using a threshold of 0 (neutral) to determine "Yes" or "No". Results are displayed in Table 53 5.5.3 Observations Overall, participants were fairly positive c oncerning the usability of MRIPS -NEURO for performing the neurological exam. Participants were strongly positive concerning the haptic interface. Eight of the nine participants rated the interface positively on all three dimensions of usability. The ophthalmoscope was rated less positively than the other two tools; we expect this is related to issues with tracking, as the physical study configuration did not allow tracking of the wiimote close to the LCD screen on which the virtual human was displayed. Performing the fundoscopic exam tool with the wiimote near the edge of the tracked area, some students tried to move it closer where it could not be tracked. More experienced participants performed more tests. The less experienced students performed the pupillary reflex test, fundoscopic test, finger counting, eye movement test, and visual acuity test. The more experienced students also conducted the peripheral vision test and other neurological tests unrelated to the eyes (smile, frown, stick out the ton gue). Touch and avoidance of touch was observed. Five of the participants collided the ophthalmoscope with one of the patients eyes during the fundoscopic test, and three later avoided another collision when performing the fundoscopic test on the other eye.

PAGE 175

175 One participant was observed colliding with the left eye, then pulling the wiimote back so he could move the ophthalmoscope around the nose, avoiding further contact with the face. During this study, the virtual human did not have the ability to flin ch if the eye was contacted by the ophthalmoscope, so all avoidances of touch are due to receiving the force feedback of the wiimote, which appears to be adequate to convey touching. 5 .5.4 Conclusions and Continued Evaluation This observational study provi ded evidence of the usability of MRIPS -NEURO for its stated goal: provide additional exposure for novice learners in diagnosing abnormal findings in a neurological exam. Additionally, the haptic interface of MRIPS -CBE was rated as usable for performing th e neurological exam and appears to be usable for conveying touch. Most importantly, a significant proportion of learners were able to use the hand -held tool use, gestures, and communication affordances of MRIPS -NEURO to correctly diagnose the virtual huma ns cranial nerve disorder. This establishes the content validity of MRIPS -NEURO. The second component of evaluating MRIPS NEURO is to evaluate the impact of feedback designed to improve learners cognitive, psychomotor, and affective performance. This e valuation is presented in Chapter 8. We first provide an overview and motivation of feedback designed for MRIPS -CBE and MRIPS -NEURO.

PAGE 176

176 Table 5 1. List of cranial nerves which can be examined using MRIPS -NEURO. Nerve(s) Function Abnormal symptoms CN 2 Visu al acuity, visual fields Poor acuity, peripheral vision loss, abnormal appearance of fundus (retina) CN 2, 3 Pupillary response Pupil does not respond to light CN 3, 4, 6 Movement of the eyes, raising of eyelids Limited movement, ptosis (drooping eyelid) CN 5 Facial sensation Loss of sensation in face CN 7 Facial movements Asymmetry in smile, frown, eyebrow raise CN 12 Movement and protrusion of the tongue Tongue crooked when protruded Table 5 2. Focused neurological exam tasks and information gained from each to aid in diagnosis of the cranial nerve disorder. Examination task Information gained Conduct medical history interview Determines what present illnesses, medication, social, family, and sexual history may be involved in the current neurologic al problem Test pupillary reflex Checks for pupil abnormalities Visually examine fundus (rear of inside of eye, e.g. retina) Checks for intracranial pressure Hold fingers up and ask how many fingers the patient sees Tests patient's binocular vision or double vision Move index finger in the shape of an H in front of patients eyes Tests for limitation of movement of one or both of the patients eyes Have the patient read from an eye chart Tests the patient's visual acuity Move or shake finger in pe ripheral vision while patient looks straight forward Tests for peripheral vision disorders Ask the patient to blink or wink his eyes Tests for ptosis, drooping of an eyelid. Table 5 3. Usability ratings of MRIPS -NEURO ( n = 9). Usable? Participants ratin g Yes Participants rating No Physical Examination 5 4 Interface 8 1 Ophthalmoscope 5 4 Hand 6 3 Eye Chart 8 1

PAGE 177

177 Figure 51. An expert performs a neurological exam of Vic, a virtual human patient with double vision due to CN6 palsy. Figure 5 2. The cardinal eye movements of a normal, unaffected eye. The (yaw, pitch) in degrees of the left eye is displayed next to each depiction.

PAGE 178

178 Figure 53. Cardinal movements with the left eye affected by CN3 palsy. In addition to the left eye pointing down and out when the virtual human looks straight, CN3 is notable for ptosis, the drooping of the eyelid of the affected eye.

PAGE 179

179 Figure 54. Cardinal movements with the left eye affected by CN6 palsy.

PAGE 180

180 Figure 55. The cardinal vectors for normal, CN 3, and CN6 eye movements are graphed as (yaw, pitch) pair associated with each axis. The green dot indicates the (yaw, pitch) of the eye when attempting to look straight ahead. The mechanism of the model is visualized for CN6: the vectors d in the normal and CN6 graph indicate the (yaw, pitch) needed to rotate to the desired gaze position. To find the maximum (yaw, pitch) capable by the CN6 affected eye along vector d cardinal vectors 1 and 2 are interpolated to calculate the vector g The magnitude of d is greater than that of the g thus the (yaw, pitch) represented by g is chosen as the final gaze yaw and pitch.

PAGE 181

181 A B Figure 56. A) Testing the pupillary reflex with the ophthalmoscope. B) Performing the fundoscopic test with the ophthalm oscope. Figure 57. Visual acuity test with the virtual eye chart.

PAGE 182

182 A B Figure 58. The finger counting test. A) With both eyes open, the virtual human with a CN3 affected left eye sees double. B) By closing an eye, he sees the correct number of fingers. Figure 59. Checking the eye movement of a virtual human patient with a left eye affected by CN6.

PAGE 183

183 Figure 510. Testing facial sensitivity by touching the virtual humans face. A B Figure 511. On-screen tutorials for A) the ophthalmoscope and B) gesture tools.

PAGE 184

184 CHAPTER 6 REAL -TIME EVALUATION AND FEEDBACK OF PERFORMANCE Chapters 68 describe the incorporation of real time evaluation and feedback of user performance into MRIPS -CBE and MRIPS -NEURO. This chapter briefly describes the motivation for feedback; Chapter 7 describes the implementation of the feedback in MRIPS -CBE and Chapter 8 describes the implementation of the feedback in MRIPS NEURO. A portion of Chapter 7 was published in the proceedings of the IEEE and ACM Symposium on Mixed and Augmented Reality 2009 [ 127]. Collaborators : Suggestions by medical collaborators D. Scott Lind and Adeline Deladisma motivated creation of the touch map feedback in MRIPS CBE. One feedback mechanism for affective performance ( thought bubbles) arose from group discussions with Andrew Raij, Brent Rossen, and Joon Hoa Chuah. In Chapter 8 we briefly describe a previous feedback system for enhancing perspective taking in MRIPS : Virtual Social Perspective -taking ( VSP) which was pri marily designed by Andrew Raij with contributions from me. Other than the VSP feedback, I designed and implemented all feedback mechanisms. Relevance to thesis: The thesis states that an interpersonal simulation incorporating instrumented haptic interfac es and providing real -time evaluation and feedback of performance improves users psychomotor, cognitive, and affective skills in an interpersonal scenario. To prove this statement, we must first develop mechanisms for providing this real time evaluation and feedback of performance; these are described in Chapter 7 (MRIPS -CBE) and 8 (MRIPS-NEURO). Next, we must demonstrate that the real time feedback plays a role in improving psychomotor, cognitive, and affective skill sets. Chapter 8 describes a formal study which evaluated the impact of feedback

PAGE 185

185 on these skill sets in MRIPS -NEURO. Evaluation of the impact of feedback in MRIPS CBE is left to Chapter 9, in which learning and training transfer is also evaluated. 6.1 Motivation for Feedback Learning takes place as a result of performing a task and reflecting, during and after the task, on how future performance can be improved [ 12 ]. However, novice learners are not competent to assess their own performance and thus require feedback from an external sourc e, e.g. expert observer feedback or automated feedback, to initiate reflection [ 12 ]. In medical education, it has been shown that immediate, specific, non-judgmental feedback is the single greatest motivator for learning [ 61][ 62 ]. In fact, feedback is nec essary from the very beginning of a medical education: Students treatment of affective aspects of patient encounters (dealing with learner and patient fears and discomfort) become more difficult to modify as learners gain experience. Without feedback, l earners may develop incorrect skills if these incorrect skills happen to achieve positive outcomes. For example, finding a mass during a clinical breast exam reinforces the technique used by the student even if it is not the correct technique likely to fi nd masses in future exams. Additionally, the absence of feedback causes learners to lose desirable behaviors, for example novice medical students instinctively use openended questions in medical histories because their knowledge level is low. Without f eedback that this process is desirable, students begin to use more close ended questions (moving to a rigid decision tree) as their knowledge level increases. Closeended questioning may cause the student to miss a crucial piece of information and reduces patient information disclosure, e.g. about a secondary complaint that may be more urgent than the given reason for coming to see the doctor [ 12 ]. Thus feedback is not only beneficial to learning, but the absence of feedback degrades learning and contributes to negative training transfer. Prior research into the impact of feedback in medical education specifically investigated the feedback given by an expert observer during a physical exam. While expert feedback has the presumed advantage of being of high quality (because it

PAGE 186

186 comes from an expert), this method of feedback has several disadvantages: it is inherently subjective, often presented in a judgmental manner, and provides a high -level qualitative evaluation (that looks right or palpate until you f eel the chest wall vs. more specific and quantitative feedback such as you have 26% of the breast tissue left to palpate) [ 12]. Additionally, this feedback may favor auditory learners who can be talked through the exam [ 42 ]. These learners make up only 19% of novice medical students [ 43 ], leaving the majority of medical students underserved by this approach. Perhaps the biggest drawback of this approach is the limited opportunities to be observed by an expert especially in exams in which practice i s already limited such as intimate exams and exams with abnormal findings which leads to many medical students graduating without receiving any feedback concerning their exam performance [ 4 ]. When feedback is given, it often comes too late, e.g. at the end of a clerkship or rotation. At this point there are no more opportunities to practice applying what is learned from this feedback. This conundrum breeds residents and practicing clinicians with questionable competence, low confidence, and high anxiet y [4] [ 9 ]. For these reasons, recommendations have been made to provide more detailed (precise), quantitative, objective, and frequent feedback of exam performance [4][9] Based on these recommendations and evidence of the efficacy of feedback for learning in traditional curricula, we are motivated to incorporate into MRIPS real time feedback to guide cognitive, psychomotor, and affective skills from real -time quantitative, objective evaluation of performance. 6.2 Unique Capabilities of MRIPS to Evaluate Pe rformance and Provide Feedback The haptic interface and sensing of its manipulation, in combination with verbal input, provides MRIPS with the novel abilities to quantitatively and autonomously

PAGE 187

187 evaluate and provide feedback concerning learners cognitive, psychomotor, and affective skills performance. As MRIPS is the only interpersonal simulation (as of this writing, we know of no other, published or unpublished) to incorporate verbal and touch inputs, MRIPS is uniquely able to evaluate user performance wit h data from both verbal and touch channels as well as data from tracking e.g. user head pose and hand position. Previous interpersonal simulations captured only the verbal channel in addition to tracking of user head pose and hand position. These simulat ions thus had the potential to provide feedback on cognitive tasks involving speech and psychomotor tasks not involving touch. With the addition of the touch input channel, MRIPS is able to evaluate and provide feedback on performance in cognitive, psychomotor, compound e.g. cognitivepsychomotor tasks and affective tasks For example, in the CBE scenario MRIPS is able to provide feedback in the cognitive tasks of taking a breast history and visually inspecting the patients breasts; the psychomotor tas k of palpating with correct pressure; and the compound cognitivepsychomotor task of using a correct patternof -search. These psychomotor and cognitive-psychomotor tasks could not be evaluated using prior approaches at interpersonal simulation. Compound cognitive-psychomotor tasks, such as recalling a correct pattern of -search and recognizing which areas of the breast remain to be palpated can be aided by feedback which incorporates data from the haptic interfaces sensors in addition to tracking data providing the position of the users hand. Psychomotor components such as palpating with correct pressure clearly require the haptic interface and its sensing of the force applied to the interface by the user.

PAGE 188

188 Automated e valuation of affective tasks is not a s cut and-dry, as evaluation of affective performance, e.g. determining when proper empathy is used, is largely subjective. However, as touch is a common means of comforting and expressing empathy (both affective tasks) [ 35], the combination of speech and touch inputs provides MRIPS with more information (than prior speechonly interpersonal simulations) with which to evaluate affective performance. The automated evaluation and feedback of affective performance in MRIPS should be considered a first attempt; the inno vation lies in obtaining enough information to make such an attempt. An example occurs in the clinical breast exam. When the virtual human expresses fear of having a breast exam, it provides an opportunity for the learner to comfort the patient Touching the patient on the shoulder is a comforting response [ 35]; MRIPS is able to sense this touch. If the learner makes such a touch in response to the virtual human expressing fear, this touch can be made to elicit a virtual human response express ing her gratitude towards the learners comforting gesture. In addition, speech input can also be processed to look for phrases which express understanding (indicating empathy). If the user touches the virtual humans shoulder while saying I understand how scary this must be for you with your mother dying of breast cancer MRIPS has more confidence that this input should elicit the virtual human to express comfort. By incorporating a haptic interface augmented with sensing of manipulation of that interf ace, as well as traditional speech interface and tracking of head pose and hand position MRIPS is uniquely able to evaluate performance and provide real time feedback to guide learner performance in the three skill sets.

PAGE 189

189 6.3 Choice of the Visual Channel t o Provide Feedback Feedback has been designed for MRIPS -CBE and MRIPS -NEURO to guide user performance in scenario specific cognitive, psychomotor, and affective skills. All of these feedback elements are presented visually. There are two reasons for choosing to present feedback visually. We believe that the strength of MRIPS lies in the mixed reality approach the combination of haptics providing realistic kinesthetic information with the virtual world providing a rich set of visual information visual information that both recreates the real world and goes beyond what the real world can offer. The real world can not provide visual feedback of performance in-situ with physical objects being manipulated, but by registering the real and virtual components of the mixed world, this is easily accomplished. By presenting visual feedback in-situ with the virtual human and her registered (conceptually, but not necessarily physically, co -located) haptic interface, MRIPS avoids additional cognitive load imposed f rom looking back and forth from a visual feedback presentation and objects being manipulated (as in the approach of Pugh et al [ 5 ]; see Figure 11). Additionally, the visual channel is hypothesized to have the highest bandwidth, explaining the visual do minance (over haptic and auditory senses) [ 128]. Thus the amount of information that can be presented through feedback, without overwhelming the task at hand, may be maximized by presenting the feedback visually. In Section 6.1 we described one drawback o f expert observational feedback to be its predominantly auditory presentation. While only 19% of novice medical students favor a predominantly auditory learning style, 70% of students favor a learning style combining visual and kinesthetic information pre sentation [ 43 ]. MRIPS already provides

PAGE 190

190 an experience in which visual and kinesthetic information dominate (the exception being the verbal responses of the virtual human), so choosing a visual presentation for feedback serves to further tailor MRIPS to fav or the 70% majority of medical students favoring a visual and kinesthetic learning style. By providing feedback in the visual channel, presented in-situ with the haptics being manipulated and the virtual human being examined, MRIPS can maximize the amount of information provided through feedback and maximize the amount of novice learners who are likely to benefit from this feedback.

PAGE 191

191 CHAPTER 7 FEEDBACK IN MRIPS-CBE 7.1 Introduction This chapter describes methods of evaluating and providing feedback of us er performance in order to guide and motivate improvement in cognitive, psychomotor, and affective components of the clinical breast exam. Each type of feedback is provided in two forms, a real time visualization presented in the mixed environment and a p ost experiential summary of performance presented in a traditional desktop environment. 7.1.1 Cognitive Components There are two cognitive components of CBE for which automated evaluation and guiding feedback are provided. The first task is to recall the series of questions to ask the patient to take a breast history. The breast history queries the patients current complaint (e.g. breast pain) and assesses the patients risk of breast cancer. The second task is to recall the poses the patient must assum e for the learner to perform a visual inspection of the patients breast. These tasks are typically evaluated by expert review of video. In this review, the expert uses a checklist, a series of dichotomous (yes/no) items, to rate the learners performance. Examples of items on the checklist used to evaluate students in clinic at the Medical College of Georgia are: Exam: inspected both breasts arms relaxed and [asked about] Risk Factor/Symptom: Breast self exam. In MRIPS, feedback is provided to guide learners in performing complete breast histories and visual inspections. The feedback is provided in the form of a list of questions to ask (text) and visual inspection poses (icons). This procedural checklist (Section 7.2) changes appearance in real -time as the learner progresses through the breast history and visual inspection. An item is automatically checkedoff

PAGE 192

192 the list when the learner asks a question related to the item. This relies solely on the speech interface which, while present in prior interpersonal simulations, has not been used to provide real -time feedb ack for similar cognitive tasks. This type of feedback could be provided with SPs, though it would require additional effort by the SP to provide this feedback during the interview and exam, and thus is not typically provided in real -time [4][16]. 7.1.2 Affective Components Affective components of CBE are to alleviate the patients anxiety and address the patients concerns effectively maintaining the patients comfort level by engaging in perspective taking and expressing empathy (understanding of the patients concerns) when appropriate. Other than patient speech, the patients facial expressions and tone of voice are the only cues to guide learners in these tasks. Because n ovice learners are under a high cognitive load during a CBE, they perform poorly in recognizing these cues and do not engage in affective elements of the exam [ 39]. MRIPS -CBE provides learners with an additional cue to guide learners to consider the patients concerns and emotional state. These cues take the form of thought bubble feedback (Section 7.3), in which the virtual humans internal state (emotions, concerns, feelings towards the learner) are presented in the visual form of a cartoon thought bu bble. Thought bubbles are used to prompt learners when empathy or comforting is appropriate (cueing learners empathic responses) and to show learners how their speech and actions can negatively or positively change the virtual humans affective state (pr oviding feedback on appropriateness of learners empathic or nonempathic responses).

PAGE 193

193 7.1.3 Psychomotor Components The psychomotor components of CBE are palpation of the breast using the middle three fingers. Palpation should be performed using the middle three fingers (finger pads) pressing in overlapping circular motions at three successive levels of pressure. Each palpation covers these three levels in sequence: light (subcutaneous), medium (midlevel), and deep (to the chest wall) [ 9 ]. This is the rec ommended procedure known as the Mammacare method. We chose to evaluate users of MRIPS CBE according to this method because it is known to outperform other methods at finding breast masses [ 66 ], and it is the method taught at both the Medical College of Georgia and the University of Florida College of Medicine, with whom we collaborate. The sensing approach used in MRIPS CBE is not able to accurately distinguish palpation with three fingers from palpation with e.g. two fingers, so evaluation and feedback is not provided for this aspect of palpation. Collaborating medical experts did not consider this a concern because remembering to use three fingers is considered minor compared to palpating with correct pressure. Feedback of the use of pressure is provi ded in the form of the touch map visualization, a series of color coded visual elements overlaid on the virtual humans breast (Section 7.4). Feedback is also provided for two compound cognitivepsychomotor tasks. The first is recalling and following the correct pattern of -search in which to palpate. Learners patterns of -search are guided by the pattern of -search map visualization, which displays an experts pattern and the learners deviation from the expert pattern in real -time (Section 7.5). The second compound cognitivepsychomotor task is recognizing which areas of the breast remain to be palpated (coverage of the breast). This task is guided as part of the touch map visualization described in Section 7.4.

PAGE 194

194 7.2 Procedural Checklist To guide learners through the cognitive tasks of 1) recalling what questions to ask in order to evaluate the virtual humans breast cancer risk and 2) recalling the poses for visual inspection, we provide a graphical checklist of text and icons. An item on this checklist is automatically checked off or highlighted when a user utterance is matched to a virtual human response corresponding to the item. The checklist items are divided into three parts: breast history (Figure 7-1), visual inspection (Figure 7-2), and palpat ion (Figure 73). To illustrate how this visualization changes to guide participants through the cognitive tasks, in Figure 71 part B, the learner has asked the virtual human about the location of the pain in her breast and whether she has experienced ni pple discharge or any other changes in her breasts. Novice medical students are under a high cognitive load during the CBE. This high cognitive load causes novice learners to miss elements of cognitive tasks as these require verbal communication; learners are too focused on performing a correct manual exam (palpation) to communicate effectively [ 38 ]. It is the goal of the breast history procedural checklist feedback to help to reduce this load by guiding students through the task of assessing the patient s cancer risk. The checklist includes important questions to ask divided by topic (history of present illness, medical history, family history, social history). This feedback may also help to keep students on topic, as Raij et al. previously found that novice medical students have a difficult time following a logical sequence of topics but instead jump from topic to topic. This behavior causes them to miss important questions which may lead to failure in a CBE of a human patient [ 84 ]. The breast histor y p rocedural checklist items are taken

PAGE 195

195 from validated instruments used to evaluate medical students at the University of Floridas College of Medicine and at the Medical College of Georgia. The palpation checklist in Figure 7-3 is designed to primarily aid the cognitive task of recalling the areas of lymph nodes peripheral to the cone of the breast which also require palpation. The palpation procedural checklist changes appearance in response to the learners palpation of these areas on the MRIPS physi cal interface. The completeness of a learners CBE with respect to breast history, visual inspection, and palpation of peripheral breast tissues is also provided after the MRIPS CBE interaction. This feedback is provided as part of a desktop based post ex periential feedback application (Section 7.6). The overall goal of the real -time and post experiential procedural checklist feedback is to guide novice learners through performing a more complete CBE. 7.3 Thought Bubbles The thought bubbles feedback augments the communication between virtual human and human to include visual representations of the virtual humans thoughts. This feedback has the goal of informing the user how well he is handling the affective components of the CBE. Through thought bubbles the virtual human indicates when she requires comforting (is scared, anxious, sad). The users attempts to comfort the patient through expressions of empathy and comforting touches are evaluated by capturing the users speech and touching of the haptic interface. In response to a comforting attempt by the user, the virtual human indicates the effect that the users actions have on her emotional state through thought bubbles and speech. Expressing the virtual humans emotional state through thoughts demonstrates to the learner that his patient may be emotionally affected by the learners actions and

PAGE 196

196 words even if the patient does not verbalize these emotions. Prior work in medical education has shown that novice medical students are largely unaware of the patients emotions [ 129 ]. Showing learners how their actions change the patients emotional state may help prime learners to be more aware of their patients emotions. Previous work in visualizing internal state of virtual humans in interpersonal sim ulations has been limited to the ELECT BiLAT system [ 83 ], a negotiation and cultural competency trainer. In ELECT BiLAT, the emotional state is depicted as text hovering above the virtual humans head. A later version included other internal state inform ation visualized in the style of progress bars, e.g. a green line progressing between 0% trust and 100% trust of the user. We believe that our visualization of emotional state as thoughts allows for more realistic and detailed emotional expressions than th ese prior attempts at visualizing virtual human emotions. As an example, the virtual human can think the doctor seems more concerned with the exam than with my feelings instead of text indicating that she is annoyed or a graph indicating 10% sadness. T hought bubble feedback is used to indicate when the virtual human is in need of comforting, when she has been comforted, and when the user has failed to comfort her. Comforting of the virtual human through speech and touch can only be triggered when the virtual human has first prompted for comfort, putting the simulation in a state in which it recognizes speech and touch inputs as comforting or non-comforting. Thus, we are able to quantitatively evaluate affective performance as the percent of opportunit ies for comforting in which the learner successfully comforts the patient. The goal of this feedback is to aid novice learners in becoming cognizant of emotional situations within

PAGE 197

197 interpersonal communication and addressing these situations with appropriat e expressions of empathy. 7.3.1 Automated Evaluation of Affective Performance Determining the amount of empathy expressed by a users response is highly subjective, however in our experience with nearly 100 prior users of MRIPS -CBE, there is a clear distin ction between responses that empathize, sympathize, patronize, or ignore emotional content (e.g. by moving on to a new topic). Empathic responses are distinct in their expression of understanding, while sympathetic responses express being sorry, and pat ronizing responses take the form of an instruction, e.g. dont be scared or theres nothing to be scared about. Responses may also ignore the emotional content of the patients speech. For example, when the virtual human states I lost my mom to brea st cancer two years ago; I miss her everyday, examples of empathic, sympathetic, patronizing, and ignoring responses that have been elicited in user studies of MRIPS -CBE are: Empathic: That must make this [exam] very difficult for you. I understand how difficult it can be to lose a loved one. Are you handling it ok? Have you been able to talk to anyone about your feelings? Sympathetic: Im sorry about your mother. Im sorry to hear that. Patronizing: Dont be sad. Itll be ok. Ignoring emoti on: How old was your mother when she died? To determine if a user utterance was empathic, sympathetic, or patronizing, we grouped user responses given in prior evaluations of MRIPS -CBE (Chapter 4) which were rated by experts for their empathic content on a scale of 1-5, where ratings of 3 5 indicated empathy, and for their appropriateness (also from 15, 1 being extremely inappropriate, 3 being neutral, and 5 being an exemplary response). Experts also rated responses as empathic, sympathetic, or both. Reducing the appropriateness rating to a

PAGE 198

198 dichotomous (inappropriate / appropriate) variable tended to group empathic and sympathetic responses together as appropriate and to group patronizing and ignoring responses together as inappropriate. We then categorized other non -rated responses by their similarity, albeit subjective similarity, to the expert -rated responses and extracted key phrases for each of the groups. However, instead of trying to differentiate between these four categories as the expert ratings did not specifically do this we chose to group utterances based on the expected effect on the patients comfort level and affect (positive or negative) towards her doctor. Thus user utterances could trigger a virtual human expression of being c omforted, an expression of not being comforted but recognizing the users attempt to comfort her, and an expression of negative affect towards the user. User utterances which contained empathic key phrases triggered the virtual human to indicate that she was comforted by the users speech. Additionally, we allowed touching of the shoulder or arm of the haptic interface to comfort the virtual human, as we observed touching of the virtual humans upper arm to comfort the patient in prior observational studies [ 82 ][ 93 ] and touch is the most commonly used method of comforting [ 35]. This response is aimed to give the user reinforcement of appropriate handling of the emotional situation. Utterances which contain key phrases indicating a sympathetic or patroniz ing response triggered the virtual human to indicate that she was not comforted by the users speech but understood that the user was trying to comfort her. This response is aimed to indicate to the user that he needs to handle the emotional situation wit h a more empathic approach.

PAGE 199

199 If the user responds to the virtual humans prompt for comforting by ignoring the emotional content, e.g. moving on to another topic which may or may not be related to the prompt for comforting, the virtual humans response indi cates negative affect towards the user, because the user is not paying attention to her feelings. This is certainly a limited approach to evaluating empathy, and should be considered only a first attempt which produces only an approximate rating of empathi c performance. In addition to touch, this attempt considers only the words spoken by the user, not the prosody (inflection, rhythm, and stress) which is an important conveyer of emotional content [ 130]. Additionally, the reliance on speech recognition to provide these words will cause some attempts at empathy to go unrecognized, as speech recognition performance is less than perfect. In the future we hope to augment this detection of empathy with a battery of further sensing, e.g. user posture and prosod y. 7.3.2 Feedback to Reinforce and Correct Affective Performance The feedback elements take the appearance of cartoon thought bubbles, as this is a common (in American culture) method of expressing when a person is thinking. The thought bubble and its two trailing bubbles are textured quadrilaterals that always face the camera (i.e. billboards). A simple heuristic is used to ensure the visibility of the bubble. The bubble has an affinity for the right side of the patients head (from the users point of view), as the left side is occupied by the procedural checklist. However if there is more screen space above or to the left of the virtual humans head, the bubbles will appear in that location instead. When the patient is lying down for palpation, the b ubbles appear above her head.

PAGE 200

200 Two different situations requiring comforting (empathic responses) and the virtual human responses to correct and incorrect handling of these situations are depicted in Figures 7 4 and 75. 7.4 Touch Map The touch map evaluate s the use of correct palpation pressure at light, medium, and deep levels and presents feedback to reinforce correct use of pressure and indicate the need for correction of incorrect (too hard) pressure. This is accomplished by comparing the learners use of pressure at each palpation to the pressure used by an expert in a pre -recorded CBE. Thus, the touch map is able to evaluate the novice learners use of correct pressure in relation to an experts use of correct pressure. The touch map also guides and evaluates palpation of the entire breast. Completeness is also evaluated in relation to an experts CBE. The process of providing feedback of palpation pressure and palpation coverage completeness is to first capture the palpation pressures and positions of an experts CBE (Section 7.4.2), processing this data to define light, medium, and deep pressure levels (Section 7.4.3), and then, during the learners exam, determining in what pressure level the learners palpations belong (Section 7.4.4). 7.4.1 Fee dback Goals The goal of the touch map feedback is to provide a precise and quantitative evaluation of palpation pressure to equip learners with a skill that will lead to more complete CBEs and more effective detection of breast masses. Feedback of palpati on pressure with the precision of touch maps has not previously been available. The correctness of a learners palpation pressure can not be evaluated without the sensing approach taken by MRIPS and the simulator of Pugh et al. (Figure 11) [ 39]. Touch

PAGE 201

201 ma ps expand on the feedback provided by Pugh et al. MRIPS uses more sensors in a more dense configuration than Pugh et al., evaluates all three levels of pressure instead of two (hard, too -hard), and provides continuous feedback of pressure over the entire breast instead of at discrete areas on the breast. 7.4.2 Capturing Palpation Pressure and Pressure in an Experts CBE In order to determine what constitutes correct light, medium, and deep pressure and complete coverage, an expert performs an exam using MR IPS -CBE. Palpation position and pressure data captured during this exam is later processed to model the three pressure levels (and a too-hard pressure level), correct patternof -search (Section 7.5), and complete coverage. An expert exam is performed onc e per setup of the MRIPS system as during setup, the image-space of the augmentation and infraredtracking cameras is registered with the physical interface. A ny number of learner exams can be perfor med after this calibration step; in our installation at the Medical College of Georgia, over 50 exams have been performed by students without need for recalibration. The experts palpation pressure is defined as the set of 64 floating point values reported by the set of 64 force sensors embedded in the haptic interface of the virtual human (detailed in Chapter 3). The force sensors are sampled at 35 Hz with a maximum delay of ~90 ms between applying a force to the haptic interface and receiving the set of 64 sensor values. The values received by the MRIPS sim ulation module are relative to automatically made baseline measurements. The rest of the process of displaying feedback takes ~30 ms, resulting in a total of slightly more than one tenth of a second of delay between palpating and receiving feedback. This upper bound on delay was measured by forcing the application to block (wait) on a new

PAGE 202

202 camera frame and set of sensor data, and measuring the time from receiving both to displaying an updated frame. This delay is acceptable as the MammaCare method of CBE calls for each palpation motion to have a duration of 12 seconds. Palpation position is calculated in the image space of the color camera that provides the MRIPS video augmentation. The touch map feedback is later rendered into this video stream. This v ideo stream is in turn projected onto the mesh of the virtual human, using a projected texture. This process displays the touch map in-situ with the physical breast being palpated. The expert wears a piece of infrared reflective tape on the fingernails o f her middle three fingers. The positions of all pixels belonging to this piece of tape are captured by the infrared camera paired with the color augmentation camera (the pairing is again shown in Figure 7-7). The transform from a pixel in the infrared c amera to a pixel in the color camera has previously been calculated. This transform is simplified to three degrees of translation and one degree of rotation, and is calculated by waving an infrared marker around in view of both cameras. The marker positi on is found in each cameras image space, and the transform is calculated by applying Horns algorithm [ 131] to the resulting point clouds. After transformation into the color cameras image space, all pixels belonging to the infrared marker are recorded, along with their centroid and a logical timestamp (an integer number used to order the sets of pixels temporally). Throughout the expert exam, the area of the infrared marker is estimated. The mean estimate is taken to be the area of a circle defining th e size of a palpation motion. The radius of this circle is stored and later used to draw the feedback elements during the learners exam.

PAGE 203

203 7.4.3 Guiding and Evaluating Complete Coverage The union of all infrared marker pixels from each camera frame of the experts exam represents the area that must be palpated for a complete exam (complete coverage). The boundary of this area is presented to the learner as part of the touch map feedback to indicate the area needed for a complete exam (Figure 78). To find this boundary, simple edge detection is first performed. Minor smoothing of this boundary is needed because of noise in the detection of the infrared pixels. This noise can produce cracks, thin areas in which the expert palpated but infrared pixels wer e not detected. These are undesirable while the expert area need not be convex, the shape should be fairly simple and there should not be gaps in the area within the cone of the breast. To achieve these desired qualities, the boundary is smoothed. Firs t, boundary pixels are ordered by walking the boundary clockwise and discarding pixels that are farther than five pixels from the current ordered boundary (the distance of 5 pixels is governed by the earlier edge detection method). This removes sharp changes of direction of the boundary line. The ordered boundary pixels are then filtered, to smooth the appearance of the boundary line, by convolving with the 1D filter { }. To calculate the area bounded by this smoothed boundary line, the frame is flood filled starting from a pixel known to be outside the boundary. The inverse of the filled area now represents the area required for a complete exam. The number of pixels in this area will be used to calculate the percentage of breast tissue the learner has palpated. The frame created from this process (pixels with alpha = 1.0 represent the boundary; 0.5 inside the boundary; and 0.0 outside the boundary) is retained to use as a mask for later calculations of the area the learner has palpated.

PAGE 204

204 During the learners exam, each palpation creates a circular element whose color (and alpha value) represent the pressure level. These elements are accumulated in a texture functioning as an accumulation buffer. Parts of elements that extend beyond the boundary ar e discarded using the boundary mark. Thus at any time during the learners exam, the total area that has been palpated by the learner is simply the number of pixels with alpha value greater than zero in the accumulation buffer. The buffers pixels are al so processed to count the number of pixels belonging to each of the light, medium, deep, and toohard pressure levels. This produces a measure of the area of the breast palpated by the learner, in pixels. The percentage of breast tissue that the learner has palpated at each pressure level (and total percentage) can then be calculated by dividing by the previously calculated area (in pixels) required for complete coverage. These percentages are provided to the learner in the post experiential feedback (Se ction 7.6; Figure 715). As the learner palpates, he sees the area of the breast within the boundary becoming covered with touch map elements, guiding the cognitive task of palpating all breast tissue. 7.4.4 Calculating the Palpation Pressure Levels After capturing the palpation pressure and position for the experts complete exam, this data is processed to create a model of correct pressure at the three required pressure levels (light, medium, and deep) and an inappropriate toohard level of pressure. Mo deling correct pressure consists of determining the range of sensor values corresponding to each pressure level at each possible palpation position. Because of the high dimensionality of the sensor data (64 real values, one from each sensor) and

PAGE 205

205 the large size of the space of possible palpation positions (order of 105 in the 640x480 image), we instead model the pressure ranges at each sensor. During the learners exam, the pressure level is calculated at each sensor, and then the pressure level at the pal pation position is calculated as a weighted average of the pressure levels at the sensors. Modeling pressure ranges at each sensor avoids the computational expense of working with high dimensional sensor data. At each sensor, a pressure level can be model ed in one dimension if pressure levels were modeled at each palpation position, one dimension per sensor would be required. This approach also provides the option of using the sensor values to estimate palpation position. However, in evaluations we hav e calculated learner palpation position by tracking a 0.25 cm radius infrared marker on the learners middle fingernail. Modeling pressure levels at each sensor: The low, medium, and high pressure ranges are naturally present in the sensor data of the exp erts exam. Calculating these ranges is an unsupervised learning problem which can be solved using clustering. A Gaussian mixture model (GMM) with three onedimensional components (corresponding to light, medium, and deep pressure levels) is fit to the s et of non zero values reported by each sensor during the experts calibration exam. Each component of the GMM takes the form of Equation 11 ) | (2 k k s k GMMv N (1 -1) Initial values for the GMMs expectationmaximization algorithm are provi ded by first applying k means ( k = 3) to the data. The toohigh pressure level is modeled as an

PAGE 206

206 additional one-dimensional Gaussian component, which is a shifted (an experimentally determined shift of +2.5 std. deviations) duplicate of the high pressure c omponent. Modeling the influence of each sensor: The relationship between the values reported by a sensor s and the possible palpation positions are modeled as a 2D Gaussian centered at the position of sensor s in image-space (the mean of the 2D Gaussian) The position of sensor s is estimated as the weighted mean of expert palpation positions corresponding to non-zero values of sensor s using the values of sensor s as the weights. To reduce the impact of noise in the sensor data, this calculation inclu des only those palpation positions corresponding to values of sensor s that are one std. deviation greater than the mean value reported by sensor s during the expert exam. This adaptive thresholding heuristic calculates the sensors position in image-spac e to within the radius of the sensing element, resulting in ~0.5cm (or ~5 pixels) of error. The covariance of the 2D Gaussian is calculated as the weighted covariance, again with the sensors values as weights, but without thresholding. After using the experts exam data to calculate the 2D Gaussian of Equation 12 for each sensor, the learners palpation position can be estimated by Equation 1 -3, where vs is the value of sensor s ) | (, 2 2 2 s s img sx N (1 -2) s s s s s imgv v x, 2 (1 -3 ) k k k s k GMM k k k s k GMM sv N v N k l ) | ( ) | (2 2 (1 -4)

PAGE 207

207 s s s img s s s s img s s xx N x N l l ) | ( ) | (, 2 2 2 2 2 2 (1 -5) Calculating level of pressure at the learners palpation position: The model of correct pressure can be described completely by the four -component 1D GMM and 2D Gaussian at each se nsor. During the learners exam, the model is evaluated at the set of sensor values V, and the model returns a continuous value in the range [1 = light, 4 = too high] representing the learners palpation level. Given the set of sensor values V and the le arners palpation position ximg reported by 2D infrared tracking or calculated using Equation 13 the learners palpation pre ssure level is calculated using Equation 1 -4 and Equation 1-5. For each sensor s with non zero value vs, calculate the pressure le vel ls at sensor s using Equation 14. The pressure level lx at ximg is then calculated by Equation 1-5 by interpolating between pressure levels at each sensor. The correctness of this model is evaluated informally by demonstrating that it produces, for t he range of all sensor values, output consistent with 4 distinct levels of pressure. It is expected that as the user reaches either of the light, medium, and deep levels of pressure, there is a range of sensor values for which the pressure remains in the same level. The values of these ranges vary with the thickne ss of the breast tissue and cannot be known a priori these ranges are discovered by fitting of the GMM to the expert calibration data. Also, as the user transitions between levels, the output of the model should be approximately linear, as the value returned by a force sensor scales linearly with the force applied. The model reproduces this expected behavior, as shown in Figure 7 -9.

PAGE 208

208 We have explored other methods for modeling correct palpation pressure, including the nave approach of finding a nearest neighbor in the experts sensor data to the learners sensor data, but the high-dimensionality of the sensor data makes these approaches too computationally expensive to evaluate in real time. I n contrast, the presented model is computationally inexpensive (evaluation of 5 Gaussian distributions at each active sensor with typically no more than 5 sensors active at once), allowing the learners use of correct pressure to be evaluated in real -time, to guide, reinforce, and correct the learners palpation pressure. 7.4.5 Design of Feedback Elements to Guide, Reinforce, and Correct The touch map provides visual feedback of the learners use of correct pressure and coverage of the breast. The touch map applies two rules to present this information visually: the coverage is encoded as the feedback elements shape, a circle, and the pressure is encoded as the color of the feedback element a multicolored scale with distinct colors at each of the four pr essure levels. Because each palpation consists of applying pressure using a circular motion, the shape of each visual element of the touch map is a circle. The radius of the circle is calculated during the expert calibration exam to provide a circle of approximately the area that the experts middle finger covers during the palpation motion. Each palpation of the breast results in one of these circular elements. The union of these circles represents the area of the breast tissue palpated, guiding the l earner in palpating the entire breast. The level of pressure (light, medium, deep, toohigh) the learner uses is represented by the color of this circle. A multicolored scale with a distinct color at each pressure level was chosen, as multicolored scales are preferred for identification tasks

PAGE 209

209 (i.e. identifying areas of the breast which have not been palpated with light, medium and deep pressure) [ 132 ]. The colors chosen for each pressure level are influenced by prior information visualization literature and discussion with medical educators. The ability of the color scale to convey use of correct pressure was informally evaluated by feedback from medical students (Section 7.7). As a blue green yellow scale best encodes continuous data [ 133 ], these color s are chosen for the low, medium, and high pressure levels (low = blue, medium = yellow, high = green). The order of green and yellow were swapped so that greens connotation with good would match good to reaching the high pressure level. Red was cho sen for the toohigh pressure level, as red connotes stop Given the continuous pressure level value lx outputted by the model of correct pressure, the color of the visual element is calculated by linearly interpolating between the colors at the neighbor ing pressure levels floor( lx) and floor( lx)+1. The color of the element guides use of correct palpation pressure and indicates the need for correction of incorrect palpation pressure (Figure 7-10). The learner is guided to increase pressure until the el ement becomes colored green. Reinforcement of correct pressure occurs by seeing the element colored green, associating the learners muscle movements with the knowledge that correct pressure was applied. As the pressure applied begins to exceed appropriate deep pressure, the element begins to turn red, indicating to the learner to stop increasing pressure. Correction is provided from seeing the element colored red, indicating that inappropriately high pressure was used.

PAGE 210

210 7.4.6 Presenting Feedback In -situ with the Virtual Human and Physical Breast An improvement of MRIPS over previous approaches to providing feedback of palpation in intimate exams (e.g. Figure 11) is to display the feedback elements in-situ with the anatomy being palpated. The learner needs only to look in one place, which will reduce the increase in cognitive load imposed by the feedback. To achieve in-situ display of the feedback elements, we render the feedback elements into a real -time video stream of the physical breast. An image-bas ed approach is taken to locating the visual feedback insitu with the virtual human and the physical breast of the haptic interface. The visual elements of the touch map and the patternof -search map are rendered into the real -time video stream of the learners hands and the physical breast, which augments the virtual human. Fragment shader programs and render to texture are used to simulate an accumulation buffer for each of the touch map and patternof -search map. As the touch map accumulates visual el ements, existing color is overwritten only by color representing higher pressure. The final color of each touch map element thus represents the highest pressure used at that position. For the patternof -search map, more recent elements occlude older elem ents. For each frame of video, the latest touch map and pattern of search map, in that order, are alpha blended with the video frame. The video frame is then projected onto the mesh of the virtual human using a projected texture. The result is that the feedback appears located on the virtual humans breast. 7.4.7 Design Choices 7.4.7.1 How many experts are needed to model psychomotor performance? We chose to base the model of expert palpation pressure on a single experts CBE. Using a single expert is s tandard in mixed environments which seek to teach

PAGE 211

211 psychomotor skills through mimicking a pre -recorded expert performance; e.g. learning tai -chi [ 134], tennis [ 135], rehabilitation [ 136], and laparoscopy [ 137]. However, the validity of the model of expert performance may be increased by incorporating data from multiple experts. The model of palpation pressure is trivially extended to multiple experts; multiple expert calibration exams are performed and the aggregate data processed. However, every expert w ill use slightly different amounts of pressure, so with multiple sets of expert data we expect that it may require many experts to find distinct clusters representing the three pressure levels. In this respect, the approach of using a single expert may pr ove to be more practical, but with the risk of being less representative of a large population of breast examination experts. 7.4.7.2 Visual feedback elements occlude the learners hands The touch map and patternof -search map are rendered on-top of the video stream of the learners hands and the physical breast, with a learner adjustable partial transparency. This maximizes the visibility of the visual feedback. However this may make it difficult for learners to locate their fingertips in the virtual w orld. We have experimented with adaptive-k Gaussian mixture models [ 138] for segmenting the learners hands from the video stream, in order to render the feedback under the hands, but this has not been incorporated in MRIPS -CBE because of the high computational complexity. 7.4.7.3 Drawbacks of an image-based approach Because we take an image based approach to locating the visual feedback in-situ with the virtual human and physical breast, a new expert calibration exam is required if the cameras are moved w ith respect to the physical breast model. However, a one time

PAGE 212

212 installation of MRIPS CBE at the Medical College of Georgia has been used in more than 50 interactions without the need to recalibrate. 7.5 Pattern -of -Search Map The pattern of -search map evalu ates and provides feedback of the learners following of an experts correct patternof -search, guiding, reinforcing, and correcting the learner in the cognitive task of recalling and following a correct patternof -search. 7.5.1 Feedback Goals Following a systematic pattern of search, such as the vertical strip pattern, is known to result in CBEs which are more successful in finding breast abnormalities [ 4 ][ 9 ]. The goals of the pattern of search feedback are to allow a medical educator to define a pattern of search for learners to follow; and to guide learners to follow this pattern to develop more successful CBE technique. 7.5.2 Modeling Correct Pattern-of -Search A model of correct pattern of -search takes as input a recent subset of the learners palpation positions, and outputs a quantitative measure of the learners deviation from expert patternof -search. Modeling correct patternof -search consists of recovering the experts pattern from the palpation position data of the expert calibration exam, and cr eating a real time evaluable model to calculate the learners deviation from this expert pattern. } ,..., {1 np p P (1 -6) Recovering the experts patternof -search: The set of palpation positions captured in the expert calibration exam is g iven by Equation 1 -6. This set contains clusters corresponding to each distinct palpation. This is shown for a vertical strip pattern in

PAGE 213

213 Figure 712A The centroids of these clusters are calculated by processing P in temporal order and creating a new cl uster when the distance between the current clusters centroid and the next position pi is greater than the radius r of a circle representing the area the experts fingertips cover in each palpation. Resulting centroids are shown in Figure 712B Because the noise present in the IR tracking of the experts palpation positions influences the centroid calculation, the centroids are then filtered in temporal order by convolving with the neighborhood (, ). The final expert path is created by constructin g a Catmull -Rom spline with the filtered centroids as control points (Figure 712C ). The Catmull -Rom spline was chosen because it passes through all control points. Direction indicators are added when rendering the expert path (Figure 712E ). The spline reconstruction of the expert pattern is stored as a sequence of line segments, S, which will be used to evaluate the learners pattern also represented a sequence of line segments L between the learners successive distinct palpation positions. 7.5.3 Gu iding and Evaluating Learner Patternof -Search In CBE, the learners goal is to use the same patternof -search as an expert, but it is not necessary to follow the experts trajectory exactly for example, the learner should not be penalized for following the pattern in reverse temporal order or for a small translation between learner and expert patterns. Thus, Euclidean distance is a poor metric of learner deviation from the expert pattern. Deviation from the expert pattern is instead taken to be the ang le between matched segments of learner and expert patterns. We experimented with a nave approach to matching learner and expert patterns: the two nearest (in Euclidean distance) expert segments to the current learner segment

PAGE 214

214 were found, and the deviation calculated as the average of the angles between the learner segment and the two expert segments. However, this approach exhibited poor performance in portions of the pattern with high curvature and penalizes small translational offsets between learner and expert. An approach which avoids these problems is to not explicitly define a matching between learner and expert segments, but instead to create from the expert pattern a vector field which serves as a look -up -table. Our approach is to place radial ba sis functions, of the form of Equation 17, at the midpoints of the line segments of the experts pattern, where mi is the midpoint of segment si and r is the radius of the circle representing the area of each palpation. Each radial basis is associated wi th the normalized vector i of the line segment at which it is placed. The vector field value at ximg is calculated by Equation 1-8. Instead of storing the vector field, a scalar field is stored to simplify computation during the learners exam. The scalar field contains the absolute value of the dot product of v (ximg) with the reference vector (0,1). The absolute value causes forward and reverse traversal of the pattern to be equivalent. This scalar field is visualized in Figure 712D. ) exp( ) (2 2 i img img im x r x f (1 -7) i img i i i img i imgx f s x f x v ) ( ) ( ) ( (1 -8) To calculate the deviation of the current line segment of the learners pattern, the scalar field values s(x1), s(x2) at the endpoints of the segment are retrieved and the dot product d between the learners current line segment li ne and the reference vector calculated. The learners deviation from the expert pattern is then calculated as the average of the differences |d -s(x1)| and |d-s(x2)|.

PAGE 215

215 This method affords quantitative feedback of the learners performance relative to the expert: the number of learner palpations vs. the number of expert palpations; the percentage of learner palpations which correctly followed the expert pattern (deviation of < 15 degrees); the percentage of learner palpations which fell into low dev iation (< 15 degrees), medium deviation (15 -30 degrees), and high deviation (>30 degrees); and the total amount of deviation of the learners pattern from the experts pattern. The number of learner and expert palpations and the percentage of learner palp ations correctly following the pattern were chosen by our medical collaborators as the most meaningful to learners; this numerical feedback is incorporated into the post experiential feedback described in Section 7.6 and depicted in Figure 7-15. 7.5.4 Desi gn of the Feedback Elements for Guiding, Reinforcement, and Correction The pattern of -search map encodes the progression of the search pattern as a series of arrows, and encodes the deviation of the students pattern from the experts pattern as a multicol ored scale. Patterns of search are typically presented in medical texts as a series of arrows (e.g. [ 9 ]). Thus, the series of line segments which reconstruct the learners pattern are visualized as a series of arrows which point towards increasing time. The appearance of an arrow is constructed in real -time as a series of polygons, rather than offline created sprites. The rendering of each arrow is parameterized with parameters of arrow tail and head widths, arrow length, and arrow color. The color of each arrow represents its deviation from the experts pattern of search. We chose a three-colored scale of green, yellow, and red, as with traffic lights (go caution stop). Green encodes that the student deviates by less than 15 degrees

PAGE 216

21 6 (low deviation range); yellow that the student deviates between 15 and 30 degrees (medium deviation range); and red encodes deviation of greater than 30 degrees (high deviation range). As with the color of the touch map elements, the color of an arrow is calculated by l inearly interpolating between the two neighboring ranges of the learners deviation value. Tracking the learners palpation position by an infrared marker on the middle finger allows the patternof -search map to indicate the deviation of a move before the learner actually palpates. This extra arrow is visually distinct as it is not outlined (see Figure 711). The learner is guided through correct pattern of -search by the expert pattern. The color of the current arrow between the last palpation position and the current hand position also provides guidance by indicating whether palpating at the current hand position would follow or deviate from the experts pattern. Reinforcement and correction are provided by arrows of prior palpations being colored green or red. The touch map and patternof -search map are presented in combination (Figure 714) to provide the learner with feedback of cognitive and psychomotor components of CBE: palpation completeness, palpation pressure, and patternof -search. 7.6 Post -Exp eriential Feedback Learning is maximized when feedback and reflection occurs during and after an experience [ 12 ]. After a MRIPS -CBE interaction, learners receive feedback which provides a summary of their performance on cognitive, psychomotor, and affecti ve aspects of the CBE. Feedback is divided into performance in the medical history (cognitive) and affective portions of the exam (Figure 715) and performance in the visual inspection and palpation portions of the exam (cognitive and psychomotor; Figure 716).

PAGE 217

217 Feedback is quantitative, to motivate learners to improve their scores by improving their exam performance. These quantitative ratings of performance can also be used by educators to more precisely grade portions of the CBE which can not be graded q uantitatively without MRIPS: correct pressure, coverage, and following of the patternof -search. 7.7 Face Validity of Touch Map and Pattern-of Search Map Feedback The face validity of the touch map and patternof -search map was established through feedback from novices and experts in an informal evaluation. Face validity indicates that the touch map and patternof -search map appears to assist the learner in performing a more complete and correct exam, with respect to palpation coverage, palpation pressure, and patternof -search. Whether this feedback does improve learners exam skills will be evaluated in Chapter 9. To establish face validity, expert clinicians and 2ndyear medical students provided informal feedback concerning the touch map and patterno f -search map feedback. After receiving lecturebased CBE teaching, six 2ndyear medical students performed their first CBE using a version of MRIPS -CBE integrating the touch map visualization (but not the pattern of -search map ). All students reported that the touch map assisted them in palpating the entire breast and in using correct pressure, and felt that receiving this feedback was valuable in the learning process. A portion of students responses are shown below, with italicized phrases indicating how each quote relates to coverage or palpation pressure e.g.: Being able to see which areas I had covered, helped me to examine the entire breast. The map helped me realize how deep I was pressing, and how large an area I covered.

PAGE 218

218 It was good being able to watch and see when I hadn't gone far enough or when my next row was too far over. [this student is referring to the feedback allowing her to see when she has left too much space between rows of her vertical strip patternof search] I thought the touc h map was an excellent way to learn how to do a breast exam. It was nice to feel how deep you are supposed to palpate. I thought the most important feature was the pressure sensing capability. [Before using MRIPS -CBE] we don't really know how much force to apply and I think this would be very useful if implemented into our curriculum prior to our 3rd yr rotations [in which students are graded on CBE]. Students comments indicate that their psychomotor skill of palpating with correct pressure was undevel oped before using MRIPS -CBE, and that the touch map feedback assisted in developing this psychomotor skill by allowing them to link the amount of force applied with the level of pressure visualized. Students also indicated that the touch map assisted them in the cognitive task of palpating the entire breast, helping them keep track of the area they had palpated and making sure adjacent palpations were adequately close together. All students also reported that the multicolor scale of the touch map was easil y interpreted, though one student suggested alternately using the progression of colors of the visible light spectrum. Additionally, four expert clinicians who have each conducted thousands of CBEs informally evaluated the touch map and pattern of -search m ap feedback, providing feedback concerning how the real time visual feedback could benefit students learning the CBE. Portions of the experts views are reproduced here, with italics indicating the connection between the expert quote and the coverage, pre ssure, and patternof -search feedback: Expert 1: One of the biggest complaints students have when they finish medical school is that they dont have faculty or qualified people observe them conducting a

PAGE 219

219 breast exam to the extent that they would like. T his is information that they wouldnt get otherwise. This technology allows [students] to get real -time feedback on the quality of their exam and the ability to incorporate that feedback into their learning and improve their examination skills, so I think it could have a significant impact on their ability to perform breast exams in the future. Expert 2: This seems good for making sure students examine all portions of the breast. I have not seen much focus on pressure before. I am sure it will help st udents with their own comfort level in the CBE which can be a hurdle for some students. Expert 3: Feedback is probably much better than what students normally receive. Particularly with the pressure feedback. Expert 4: Too often the breasts are prov ided only a cursory exam. The idea of placement, appropriate pressure and pattern of examination is quite useful as an educational tool. [The expert expects that] having had experience on the mannequin will give them more confidence and understanding and will enable them to complete an exam on a living patient more efficiently. The expert clinicians identified that the touch map and pattern of -search map visual feedback provided students with feedback on palpation pressure, patternof search, and coverage of the breast which students are not provided in purely physical learning environments. The experts also remarked on the potential of the real -time visual feedback to overcome barriers to learning CBEs, such as student comfort (anxiety) and confidence.

PAGE 220

220 A B C Figure 71. A) The breast history portion of the procedural checklist is displayed above the virtual humans head. In B) and C), items and topics are highlighted as the user asks questions corresponding to the items on the list.

PAGE 221

221 Figure 72. The visual inspection portion of the procedural checklist expands to show the three poses required for visual inspection: relaxed with arms at sides, chest flexed with hands on hips, and arms held above head. A B C Figure 73. The procedural chec klist also incorporates feedback to aid in the cognitive task of recalling which peripheral areas of lymph nodes should be examined. This also provides high-level feedback as to the completeness of palpation, as areas are highlighted as they are palpated by the user. This highlighting is illustrated in the progression from A) to B) to C). These areas are axillary, infraclavicular, and supraclavicular. The cone of the breast is also highlighted as it is palpated, though more detailed feedback of completeness of palpation of the cone is given by the touch map feedback of Section 7.4.

PAGE 222

222 A B C D Figure 74. A) The virtual human prompts for empathy B) If the learner then expresses empathy, e.g. I understand how hard this must be for you, the virtu al humans thoughts indicate positive affect towards the learner. C) If the learner ignores the prompt the virtual human responds to indicate the learner needs to pay more attention to emotional content of the interaction. D) If the learner responds inappropriately, the virtual humans thoughts express negative feelings towards the learner.

PAGE 223

223 A B C Figure 75. When the physical exam begins, A) the virtual human expresses fear. B) If the learner ignores the opportunity to comfort the patient, her re sponse indicates negative affect towards the learner. C) If the learner responds with a patronizing response, e.g. there is nothing to be scared about, the virtual human responds with similar negative affect towards the learner.

PAGE 224

224 Figure 76. The touc h map provides feedback of coverage and use of correct palpation pressure through color -coded visual elements presented in-situ with the virtual human and the haptic interface. Figure 77. The pairing of the color and infrared seeing cameras (hanging ab ove the mannequin) and the haptic interface to the virtual human.

PAGE 225

225 A B Figure 78. A) The boundary of the area required for complete coverage of the breast cone. B) Complete coverage is indicated when this area is filled. Figure 79. Informal cor rectness of the model is demonstrated by showing that the output of the model fits the expected progression of pressure levels. Shown here is data for a single sensor. This behavior is consistent across all sensors and repeatable across multiple calibrat ion data sets.

PAGE 226

226 A B C D Figure 710. The color of the feedback provides guidance, reinforcement, and correction of the learners palpation pressure through real -time changes in color. A) Light pressure is indicated by a blue color; B) Medium pressure by yellow; C) Deep pressure by green; and D) Too high pressure by red. A B C Figure 711. A, B) A learner follows an experts vertical strip patternof -search. C) The pattern of -search map indicates the learners failure to follow a systematic pattern.

PAGE 227

227 A B C D E F Figure 712. Modeling patternof -search: A) Expert palpation position data contains clusters. B) The centroids of these clusters. C) The resulting expert path after filtering and spline interpolation. E) The vertical strip pattern is then rendered with direction arrows added. F) The same process applied to a spiral expert pattern of -search. D) The scalar field used to calculate learner deviation from the pattern of ( e ) is shown. A B C D E Figur e 713. A) The touch map and B) patternof -search map and C) the combination of the two for the same exam. D, E) The progression of the combined visualizations.

PAGE 228

228 Figure 715. Feedback is provided concerning cognitive elements such as the procedure of visual inspection and cognitive-psychomotor elements such as completeness of palpation, palpation with correct pressure, and following of the vertical strip pattern of search.

PAGE 229

229 Figure 716. Summary feedback of affective (Your use of emp athy) and cognitive (Your information gathering) performance. If the learner did not ask about a topic, it appears as red in this list. This list includes items in addition to those in the procedural checklist real time feedback (Section 7.2). Items on this list, other than the patient concerns, are taken from validated checklists use to grade medical students at the University of Floridas College of Medicine and at the Medical College of Georgia.

PAGE 230

230 CHAPTER 8 FEEDBACK IN MRIPS-NEURO 8.1 Introduction T his chapter describes the development and evaluation of two visualizations providing feedback to guide cognitive, psychomotor, and affective performance in MRIPS -NEURO. The thesis states that an interpersonal simulation incorporating instrumented haptic interfaces and providing real -time evaluation and feedback of performance will improve users psychomotor, cognitive, and affective skills. To prove this statement, we must demonstrate that real -time feedback improves performance in these skill sets. This chapter describes a formal study designed to evaluate whether providing H -Map and Patient Vision feedback improves learners performance in these three skill sets. We have developed two visualizations to provide guiding and motivating feedback in cognitive, psychomotor, and affective components of the cranial nerve exam. The H Map displays a path for the learner to follow during the eye movement test, guiding the psychomotor task of moving the finger or ophthalmoscope in an H pattern to test the extrem es of the patients vision. The patient vision visualization allows the learner to view the virtual world through the eyes of the patient. We expect that experiencing the patients double vision firsthand will assist learners in the cognitive task of d iagnosing the affected cranial nerve (based on eye movements) and improve learners perspective taking and concern for the patient, improving their affective performance in the exam. 8.2 H -Map The H Map (Figure 81) visualizes the uppercase H pattern use d to assess the patients range of eye movements. In this assessment, the learner sweeps his finger or

PAGE 231

231 the ophthalmoscope light in the H shape, testing the lateral extremes of the patients vision and at each lateral extreme (left and right) testing the vertical extremes of the patients vision. If the patient can not move his eyes (or one of his eyes) through the entire H pattern, this indicates a cranial nerve 3, 4, or 6 disorder. For example, if the patient has a CN3 affected left eye, he can not adduct the left eye towards his nose. The eye movements are the most important test for diagnosing cranial nerve palsies that restrict eye movements, making this test important for learners to perform correctly. The H Map guides learners to ensure they are testing the extremes of the vision. This H -Map is implemented as colored targets (colored quadrilateral primitives) connected by two vertical and one horizontal bar (also colored quads). The targets represent the positions the ophthalmoscope or hand sh ould reach in order to elicit the six cardinal extremes of the patients eye movements. The position of the targets varies with the depth of the ophthalmoscope or hand tool from the virtual humans eyes. Targets are placed at the intersection of rays cas t from the unaffected right eye of the virtual human with the XY plane placed at the depth of the tool. Each of the targets corresponds to casting of a ray oriented in one of the six extreme rotations of the eye. Intersecting the targets with the gesture (hand) tool or ophthalmoscope tool changes the color to green to indicate the eye movement extreme corresponding to that target has been tested. Feedback on this design was received through informal testing with a neurologist and four neurology residents These experienced performers of the neurological exam indicated that they performed the eye movements test close in depth to the patients head, approximately 1 ft. or less. This allowed them economy of movement, and they

PAGE 232

232 could focus on the patients eyes while keeping their finger/ophthalmoscope in their primary vision. Based on this feedback, we altered the H Map to indicate to the learner a range of depth that would provide this economy of movement. The horizontal and vertical bars of the H -Map cha nge to a green color when the ophthalmoscope or hand is approximately 512 inches in depth from the patients eyes (Figure 87). 8.3 Patient Vision 8.3.1 Feedback Goals The patient vision feedback allows learners to view the virtual world through the eyes of the patient, providing learners with firsthand experience of life with double vision The goal of this experience is to motivate learners to engage in social perspective taking, imagining what it is like to be the patient. When a person engages in per spective taking, he first considers what he knows of the other persons knowledge, senses, and experiences. This leads him to an affective understanding of the other understanding the others emotions and stateof mind [ 139 ]. The outcome of this proces s is expression of this affective understanding, through empathy and concern for the other. Thus the goal of the patient vision feedback is to aid learners in understanding how the patients double vision affects the patients life and emotional state. We expect the outcome of this understanding to be increased consideration of the patients safety, e.g. is it safe for the patient to drive home from the clinic, and expression of empathy improving the learners affective performance in MRIPS -NEURO. We are motivated to incorporate the patient vision feedback by prior work which allowed a learner to relive a clinical breast exam from within the body of the patient, and

PAGE 233

233 demonstrated impact on learners self awareness of his affective performance in the CBE. 8 .3.2 Prior Work in Motivating Perspective Taking Raij developed an after action -review experience, virtual social perspective taking (VSP), which sought to improve medical students use of perspective taking in the clinical breast exam scenario [ 85 ]. Lear ners performed CBE of a virtual human patient using MRIPS -CBE, and then relived their interview and exam from within the body of the virtual human. The learner looked through the virtual humans eyes, seeing what the virtual human saw during the exam th e virtual world and the learner. As the virtual human, the learner heard what the learner had said during the exam and was asked to speak what the virtual human spoke during the exam. The learner embodied the avatar of the virtual human and controlled th e pose of the avatars head. To emphasize that the learner was reliving the experience in the body of the virtual human, the learner could see the movements of his avatar in a virtual mirror (Figure 8-2). Raij and I conducted a study of 16 medical student s, residents, and clinicians at the Medical College of Georgia to evaluate the impact of VSP on perspective taking and empathy. Participants rated their affective performance along the dimensions of perspective taking and empathy before and after the VSP. Ratings decreased after VSP, demonstrating that reliving the experience from the patients point of view helped learners become more aware of their affective performance. The VSP feedback motivated reflection, which leads to change. Participants indicated that they would change how they approached perspective taking and empathy in future patient interactions, but change was not specifically measured in this study.

PAGE 234

234 8.3.3 Patient Vision Feedback Patient vision is a simulation of what the virtual human wit h cranial nerve disorder sees. By wearing an HMD, the user is able to see through the patients eyes and experience the double vision and incomplete range of eye movement experienced by the patient. This visualization is targeted to improve the learners cognitive performance in diagnosing the cranial nerve disorder and to improve the learners affective performance by motivating perspective taking which is expected to increase empathy and concern for the patients safety. The patient vision feedback is a novel simulation of the experience of double vision and a cranial nerve disorder. Prior work has developed visual simulation of other vision disorders: myopia and effects of laser surgery on myopic vision [ 140], recreating an individuals vision by scann ing the retina [ 141 ], and multitexturing to simulate glaucoma and diabetic retinopathy [ 142]. The learner literally sees through the eyes of the patient, as the virtual world is rendered from two cameras having the position and orientation of the virtual humans eyes (Figure 8 3). The virtual humans left eye image is presented to the users left eye and the virtual humans right eye image is presented to the users right eye. Just as in a patient with double vision, the job of fusing the two images is le ft to the users brain. If the virtual human is seeing double, then the user will be unable to fuse the two images and as a result will also see double. During the neurological exam, the virtual humans behaviors allow the user to experience what a patient with double vision sees during a neurological exam. When the patient is asked to follow the doctors finger with his eyes, the user is able to determine in what fields of vision (e.g. left, right, far, close) the patient sees double. The patient vision also reflects other changes in the patients vision during

PAGE 235

235 the exam. For example, when the patient is asked to close or cover one eye (e.g. in the visual acuity and finger counting tests), the covered eyes image is not rendered, providing the user with vision in only one eye. Initially we had experimented with combining the left and right eye images into a composited image, by alpha blending the left eye on top of the right eye with alpha = 0.5 (Figure 84). This approach provided a method for experiencing the patients double vision displayed directly on the non-stereoscopic large screen display. During the exam, the virtual human would say let me show you what I see and the learners view of the virtual human would be replaced by the virtual human s view of the virtual world. While this approach could benefit the affective component of perspective taking, it is unable to assist the cognitive task of diagnosing which cranial nerve is affected, because the left and right eyes can not be distinguished. To afford diagnosis during patient vision, a stereo display, such as an HMD, is required. For this reason, and to facilitate the study design of Section 8.4, the implementation was altered to render each virtual human eye view to its respective HMD scre en (left, right). 8.4 Evaluating the Impact of Feedback on Cognitive, Psychomotor, and Affective Performance We conducted a user study to evaluate the impact of the H map and patient vision feedback on learners cognitive, psychomotor, and affective perfor mance. 8.4.1 Study Design and Procedure To directly evaluate the impact of the patient vision feedback on participant performance in the cognitive task of diagnosing the CN disorder and in the affective task of perspective taking, participants were divided into two groups. Group A experienced

PAGE 236

236 the patient vision feedback before examining the patient and Group B did not experience patient vision before examining the patient. The procedure is shown in Figure 8 -5. Two participants arrived at a time. Each par ticipant completed a background survey assessing their experience with the neurological exam. The experimenter then explained to both participants how to talk to the virtual human and how to use the Wiimote to manipulate the virtual ophthalmoscope, hand, and eye chart tools. The experimenter pointed out the series of icons on the left hand side of the screen which illustrated the tests that could be performed on the virtual human (Figure 8-7). Participants were told the patient vision feedback would let them experience the patients double vision during the exam and that they should try to figure out which eye and cranial nerve is abnormal during the patient vision. Participants were told that the H -Map feedback would appear during the eye movements tes t. They were told that the size of the H would decrease as they moved the hand/ophthalmoscope closer to the patient, that the H would change color when the hand/ophthalmoscope was an optimal distance from the patient, and that the targets represented th e six cardinal extremes of the patients eye movements. Participants were not told that they we re required to follow the H Map, though all participants did follow the H Map. After receiving instruction, Participant A donned a stereoscopic HMD displaying t he virtual exam room and Participant B wore a hat augmented with infrared markers used to track her head pose. Participant B viewed the virtual human and exam room rendered on the non-stereoscopic large screen display. By tracking her head pose, the

PAGE 237

237 virt ual scene is rendered from her perspective, as if the display was a window into the virtual world (Figure 8 -6). Participant B performed a medical history and exam of a virtual human with a CN6 affected left eye. In this exam Participant B received the H map feedback during the eye movement test. During this exam, participant A viewed the virtual world, including the tools manipulated by Participant B, through the eyes of the virtual human. This allowed Participant A to experience the patients double v ision during an exam, allowing Participant A to evaluate the severity of the patients double vision and the patients eye movements. Participant A was a passive observer; the movement of virtual tools was controlled by Participant B and the speech, actions, and eye movements that Participant A experienced were controlled by the virtual human simulation. After Participant B completed the history and exam of the virtual human patient with CN6 disorder, Participant A removed the HMD and completed the post -pa tient vision survey (Appendix H ). Participant B then performed an exam (without taking a medical history) of a virtual human with a CN3 affected left eye. In this exam, the H Map feedback was not provided. Participant B then completed a post exam survey (Appendix I ). The participants then switched roles and repeated the procedure, with Participant B experiencing the patients vision and Participant A performing a history and exam of the CN6 virtual human patient followed by an exam of the CN3 virtual huma n patient. The counterbalanced design of Group A and Group Bs procedures allows us to evaluate the impact, on cognitive and affective performance, of experiencing a patients abnormal vision before examining the same patient. By having each participant

PAGE 238

238 e xamine both a CN6 and a CN3 patient, this design additionally allows us to make within -subjects comparisons of the completeness and efficiency of the eye movement tests when H Map feedback is provided (CN6 patient) and when H Map feedback is not provided ( CN3 patient). The eye movement test required for the CN6 and CN3 patients is equivalent. 8.4.2 Population Eighteen 2ndyear medical students at the University of Florida s College of Medicine participated. All participants had experience in neurological exams of standardized or real patients, with four participants having 15 experiences, eight having 6 -10 experiences, and six having more than 10 experiences. It is not known whether participants had previous exposure to CN3 or CN6 in human patients. Nin e participants were enrolled in each of Group A (patient vision before exam) and Group B (patient vision after exam). 8.4.3 Metrics 8.4.3.1 Evaluating cognitive and affective performance After completing the patient vision experience, participants were ask ed to diagnose the cranial nerve and eye affected based on what they saw through the patients eyes. Participants are also asked to describe how the participant felt the double vision affected the patients everyday life. This survey assesses the impact of patient vision on performance in the cognitive task of diagnosing the cranial nerve disorder and the affective task of perspective taking. After examining the CN3 and CN6 patients, participants were asked to provide a diagnosis of the cranial nerve and eye affected in each of two patients. This was used to evaluate cognitive performance. Participants were also asked to list any concerns

PAGE 239

239 they would like to relay to the patient or the patients family. This item was also used to evaluate perspective tak ing. In evaluating the two free response items in the post -patient vision and post exam surveys, we looked for expression of participant concern for patient safety. These consisted of expressions that the patient should not engage in specific tasks that have become dangerous to perform due to the patients double vision. Specifically, we expected to find instructions to the patient that he should not drive a vehicle. The patient stated early in the exam that the double vision had started when he was dr iving home from work. If asked, the patient also states during the exam that he drove to the doctors office and would be driving home. We specifically focused on driving as an aspect of patient safety because driving is an everyday task that all participants knew the patient performed. Based on the patients double vision, the participant should do her best to dissuade the patient from driving due to the danger to the patient and others [ 52 ]. Expression for patient concern was also evaluated by observi ng video of the participants history taking and exam, i.e. to determine if participants verbally instructed the patient that he should not drive or expressed concern about how the patient would get home from the doctors office. 8.4.3.2 Evaluating psychom otor performance The psychomotor component of the exam is the eye movement test. The impact of the H Map feedback on the completeness and efficiency of the eye movement test was measured by recording the position and orientation of the virtual tools at a minimum of 30 Hz. Completeness of the eye movement test was evaluated as the difference of 1) the angles of the six cardinal eye movements elicited by the participant and 2) the

PAGE 240

240 maximum angles of t he six cardinal eye movements. The yaw and pitch elicited by the participant was treated as a two-dimensional vector. The maximum yaw and pitch was also treated as a two-dimensional vector. The difference between the participant elicited yaw and pitch and the maximum yaw and pitch was calculated as the Euclid ean distance between these two vectors. For example, in CN3 the right down extreme is (yaw = 10, pitch = 10). If the participant elicited (yaw = 10, pitch = 7), the distance for this extreme is calculated to be 3 degrees. The differences at each of the six extremes were summed for each participant. A sum of zero indicated the most complete eye movement test possible. As described in Section 8.2, medical professionals experienced at neurological examination perform the eye movement test with the finger or ophthalmoscope held one foot (or less) in depth from the patients eyes. This affords an economy of motion of the practitioners hand/arm and allows the practitioner to view the patients eyes up close while keeping the practitioners finger (or the ophthalmoscope) within the practitioners primary vision. The horizontal and vertical bars of the H Map visualization became colored green when the virtual hand or ophthalmoscope was held at a level of depth of ~5 to ~11. These depths are indicated to be approximate because the implementation calculated depth in centimeters. The depths in centimeters were indicated by an expert neurologist testing MRIPS -NEURO. An efficient eye movement test would test all six extremes at such a depth. To evaluate effic iency of each participants eye movement test, we counted the number of extremes that were tested within the efficient depth range. We also calculated the average depth at which the six extremes were tested and the standard deviation of the depths at whic h the six extremes were tested. This

PAGE 241

241 was performed for each of the CN6 exam in which the H Map was provided and for the CN3 exam in which the H Map was not provided. 8.4.3 Hypotheses Hypotheses concern the impact of the Patient -Vision feedback on improving the affective task of perspective taking and the cognitive task of diagnosis, as well as the impact of the H Map feedback on the completeness and efficiency of the psychomotor task of testing patient eye movements. Hypothesis Patient -Vision Improves Affe ctive: Participants experiencing the patient vision feedback (CN6) before performing an exam of the patient (CN6) will exhibit increased concern for patient safety, expressed verbally to the patient or written in the post -patient vision or post exam survey s. o Null hypothesis: Participants in Group A and Group B will exhibit no difference in verbal or written concern for patient safety. Hypothesis Patient -Vision Improves Cognitive: Participants experiencing the patient vision feedback (CN6) before performing an exam of the CN6 patient will more often correctly diagnose the cranial nerve disorder of the CN6 patient. o Null hypothesis: There will be no difference in the number of participants in Group A and Group B who diagnose the CN6 patient correctly. Hypothes is H Map Improves Psychomotor Completeness: Participants will perform more complete eye movement tests when the H Map is provided than when the H Map is not provided. o Null hypothesis: Participants eye movement tests of the CN6 and CN3 patients will be equivalently complete. Hypothesis H Map Improves Psychomotor Efficiency: Participants will perform more efficient eye movement tests when the H Map is provided than when the H Map is not provided. o Null hypothesis: Participants eye movement tests of the CN6 and CN3 patients will be equivalently efficient. We do not expect to improve participants psychomotor skills from a single interaction, thus we evaluate the H Map on its ability to elicit more complete and

PAGE 242

242 efficient performance when the H Map is present Prior work has demonstrated that similar psychomotor tasks can be learned through similar mixed reality visualizations of tool or hand trajectories [ 143 ]. If learners demonstrate more accurate (complete) and efficient eye movement tests when the H Ma p feedback is provided than when the H Map feedback is not provided, we can infer that repeated practice of the eye movement test with the H Map feedback will lead to more accurate and efficient eye movement tests. 8.4.4 Results and Discussion Two particip ants, one in each group, expressed that they did not experience double vision at any time during the patient vision feedback. We were unable to determine whether this was due to malfunctioning of the HMD or a peculiarity of the participants vision. It i s possible for the HMD to automatically toggle between nonstereoscopic (left eye duplicated for both left and right eyes) and stereoscopic display; however, the experimenter tested the HMD before each participant and did not note any problems in this rega rd. However, because these participants did not actually experience the virtual patients double vision, and were evenly split between the two groups, they were removed from analysis for Hypothesis Patient -Vision Improves Affective and Hypothesis Patient -Vision Improves Cognitive. 8.4.4.1 Hypothesis Patient Vision improves affective. Experiencing patient vision increases concern for patient safety: accepted Affective performance was evaluated as expressed (verbal or written) concern for patient safety, i. e. informing the patient that he should not drive a vehicle. Significantly more participants who experienced patient vision before examining the virtual human patient (Group A) expressed concern that the patient should not drive than did

PAGE 243

243 participants who did not experience patient vision before the exam (Group B). Seven of eight participants in Group A expressed this concern vs. zero of eight participants in Group B. This is significant at p < 0.005 by Fishers exact probability test. Almost all particip ants who experienced patient vision before the exam expressed concern that the patient should not drive, with five participants expressing this on the post patient vision survey, one participant (of the five) directly telling the patient during the exam, and three participants expressing this on the post exam survey (one participant in this group had previously indicated this in the post -patient vision survey). None of the participants in Group B, who performed the exam before experiencing patient vision, expressed concern that the patient should not be driving. All patients were equall y primed to think about driving, as the patient first began to experience double vision while he was driving home from work : I was driving home from work and all of the sudden the lines on the road started to cross However, only those participants who had seen through the eyes of the patient before assuming the role of the doctor expressed concern for the patients safety. This indicates that literally providing particip ants with the patients visual perspective caused the participants to engage in perspective taking. The participants experiencing Patient Vision considered the patients feelings and the impact of the double vision on the patients life and developed a c oncern for the patients safety that later (as the doctor) allowed them to identify driving as a danger to the patient Thus participants affective performance was improved by providing them with the patient vision feedback prior to their examination of the virtual human patient.

PAGE 244

244 8.4.4.2 Hypothesis Patient Vision improves cognitive. Experiencing patient vision aids diagnosis of CN disorder: rejected but with a positive result Nearly all participants in both groups were able to correctly diagnose both the CN3 and CN6 virtual patients through physical examination (9 of 9 in Group A and 8 of 9 in Group B). This leads us to reject Hypothesis Patient Vision Improves Cognitive, as performance in the cognitive task of diagnosis was equivalent for those participa nts who received patient vision prior to the exam and those participants who did not receive patient vision prior to the exam. As participants in both groups performed well in this task, we can not conclude whether the patient vision improved, or did not improve, performance. However, there is evidence that patient vision alone is adequate for diagno sing the cranial nerve disorder. Eighty one percent of participants correctly diagnosed the cranial nerve disorder while viewing Patient -Vision. Seven of the eight participants in Group A were able to correctly diagnose the virtual patient with CN6 palsy from experiencing patient vision alone (i.e. before conducting an exam of the patient). Additionally, 6 of 8 participants in Group B were able to diagnose CN 6 from the patient vision feedback. Although Group B had previously examined a virtual human patient with CN6, they had most recently examined a virtual human with CN3 and were not told what disorder they would experience with patient vision (it could hav e been CN3, CN6, or other). Overall, 13/16 participants were able to correctly diagnose CN6 from experiencing the patients vision. This is a significant percentage of the participants (one way chi -square test: X2 = 5.1, p < 0.05). This result shows tha t viewing a (virtual) neurological exam through the eyes of a virtual human patient with cranial nerve palsy

PAGE 245

245 provides the information needed to diagnose the cranial nerve disorder affecting the virtual humans eye movements. This is a positive result for t he impact of Patient Vision on cognitive performance: experiencing the patients double vision aided in the diagnosis of the cranial nerve disorder. The rejecting of Hypothesis Patient -Vision Improves Cognitive is caused by a too restrictive wording of th e hypothesis and a ceiling effect in participant s diagnostic performance. W hile Patient Vision is sufficient for diagnosing the cranial nerve disorder, we were unable to measure the impact of Patient -Vision on the correctness of diagnosis reported after an exam of the patient. To determine if patient vision is a significant factor in arriving at a correct diagnosis after additionally performing an exam of the patient, we should revisit this experiment with a more difficult -to -diagnose cranial nerve disor der. In addition to evaluating the cognitive impact of Patient -Vision, the finding that 17 of 18 participants arrived at correct diagnoses provides additional evidence of the content validity discussed in Section 5.5. In the prior study of Section 5.5, 12 of 14 participants were able to use MRIPS -NEURO to diagnose one cranial nerve disorder. In this study, 17 of 18 participants were able to use MRIPS -NEURO to correctly diagn ose two cran ial nerve disorders. This result strengthens our claim of the content validity of MRIPS -NEURO: MRIPS -NEURO simulates a neurological exam with abnormal findings to fidelity sufficient to allow learners to arrive at a correct diagnosis. 8.4.4.3 Hypothesis H -Map improves psychomotor completeness. H -Map visualization results in a more complete eye movements test: rejected We expected participants to perform a more complete eye movements test, eliciting all extremes of the patients eye movement, when the H Map was provided.

PAGE 246

246 This expectation was not supported by the data. Fiv e participants performed a more complete eye movements test with the H Map present (avg. of 4.8 2.8 degrees more complete with H Map). Four participants had equivalent tests with and without the H Map present (total difference < 1.0 degree). The remaining nine participants performed better without the H Map present (avg. of 4.3 1.3 degrees less complete with H Map). A t -test comparing the magnitude of the difference for the 5 who performed better with H -Map and the 9 who performed worse with H Map revealed no significant difference: t(12) = .61, ns A within -subjects paired samples t test revealed no significant improvement from viewing the H Map: t(17) = 0.86, ns We conclude that the H Map did not assist participants in eliciting the extreme eye m ovements of the patient. In investigating why the H Map did not improve completeness of participants exams, we looked for specific eye movement extremes that caused difficulty for participants and reviewed participant comments. The average differences li sted in the previous paragraph seem small considering they are sums over the six extreme eye rotations. However, more in-depth analysis reveals that these differences are primarily due to missing two of the six extremes: the right up extreme and the right -down extreme. With the H -Map, examining the CN6 patient, 12 participants missed the right up extreme by an average of 2.9 degrees and 12 participants missed the right -down extreme by an average of 3.0 degrees. Without the H Map, examining the CN3 patient, 12 participants missed the right up extreme by an average of 4.0 degrees. It was not the same 12 participants missing all three extremes, though there were some participants who missed two.

PAGE 247

247 Participants difficulty eliciting the right down and right u p extremes does not appear to be linked to the cranial nerve disorders examined. A CN6 affected eye has normal movement in the right -down and right up extremes, while a CN3 affected eye has abnormal movement in these extremes. Notably, all participants held the Wiimote in their right hands. As the right -up and right -down extremes required moving the Wiimote to the left of the participants body, the handedness of the participant may have played a role. By examining participant feedback and behavior, we have identified two other potential factors linked to participant skill and system design. During the exam, participants reported difficulty in judging the depth of the ophthalmoscope or hand from the virtual humans head. This was likely due to the virt ual world being displayed on a non-stereoscopic display. The lack of stereo depth cue appeared to cause participants difficulty in following the H -Map. While following the H pattern, participants would tilt the top of the Wiimote towards the screen, caus ing the depth of the virtual hand/ophthalmoscope to become closer to the patient. This in turn caused the H Map to become smaller. Participants appeared to perceive the decrease in size of the H -Map to indicate the H -Map had moved farther away from them, and in turn continued moving the hand/ophthalmoscope closer to the patient, chasing the H with the virtual hand/ophthalmoscope. This occurred because of the lack of stereo depth cues and two other factors: (1) Participants seemingly forgot prior inst ruction that the H appeared at the same depth as the virtual hand/ophthalmoscope. (2) Participants did not use appropriate movements of the Wiimote e.g. when having a human patient follow the light on the ophthalmoscope, tilting the ophthalmoscopes ti p

PAGE 248

248 towards the patient would cause the light to aim at the floor and the patient would no longer see the light they were instructed to follow. We believe that providing a stereoscopic display would eliminate factor (1). Eliminating factor (2) appears to be more complex. The reason for these inappropriate movements of the Wiimote appears to be that the participant tended to move the Wiimote in orbit about his or her own body instead of more appropriately moving the Wiimote as though it was in orbit arou nd the virtual human patients head. We believe this is partially due to participant inexperience i.e. lack of skill in this psychomotor task i.e., novices might make these same egocentric movements with a real ophthalmoscope. However, participants beh aviors raises questions for future work: do participants have difficulty in mapping the movement of the Wiimote to the movement of the virtual tool perceived to be beyond the display surface? Perhaps the approach of adding a physical mannequin head would decrease the cognitive load imposed by this mapping. While the H Map in its current form does not appear to guide learners to perform more complete eye movement tests, further evaluation reveals that practice with the H Map has the potential to improve the efficiency of participants exams. Potential solutions must mitigate the high variance in the tracked depth position, reducing the impact of user errors such as pointing the wiimote at the screen. One such solution is to track the position of the wiimot e as the centroid of the wrist. In the study, the position was tracked as the tip of the wiimote because this corresponded closely with the tip of the virtual hands finger which the patient was to follow with his eyes. However, the offset between the tip of the wiimote and the users wrist can be estimated with

PAGE 249

249 reasonable confidence, making it possible to use the tip of the wiimote as the target for the patients eyes to follow and the position of the wrist as the depth of the H Map. 8.4.4.4 Hypothesis H -Map improves psychomotor efficiency. H Map visualization results in a more efficient eye movements test: accepted Following the H Map allowed participants to perform more efficient eye movement tests than when the H Map feedback was not provided. An ef ficient eye movements test would test all six extremes within the optimal depth range described in 8.4.3.2, and would test all six extremes at the same depth, i.e. with minimal deviation in depth. Sixteen of the 18 participants examined more of the six extremes within the optimal depth range when the H Map feedback was provided than when the H -Map was not provided. This is a significant majority by oneway chi -square ( X2 = 9.4, p < 0.005). On average, participants tested significantly more extremes wi thin the optimal depth range when viewing the H Map: 4.7 1.3 extremes vs. 1.7 1.8 extremes without the H Map. This difference is significant by paired samples t -test: t(17) = 6.3, p < 0.005. Participants also examined the six extremes of the patients vision with significantly less variance in depth when they were able to follow the H -Map than when the H Map was not displayed. The standard deviation of the depths at which the six extremes were tested: with H Map: 2.4 1.7 in. and without H Map: 3.6 1.8 in. The H Map allowed these novice learners of the neurological exam to perform an eye movements test of similar efficiency as an expert. Without the H Map present, participants performed this psychomotor task using a previously learned, less efficient method. To illustrate this, without the H Map some participants performed the eye movements test at a depth that required them to hold the Wiimote out at arms length, with the virtual hand/ophthalmoscope no longer appearing on the screen. If novice

PAGE 250

250 use rs of MRIPSNEURO were to practice the eye movements test repeatedly with the H -Map feedback, we would expect these novices to improve in efficiency of this psychomotor task. 8.4.5 Conclusions In this study we evaluated the impact of the patient vision an d H -Map feedback on participants performance in the affective, cognitive, and psychomotor components of the neurological exam. The patient vision feedback was targeted to the affective task of taking the patients perspective and a resulting expression of concern for the patients safety. Participants who experienced the patients vision before examining the patient expressed concern that it would be dangerous for the patient to drive. These participants demonstrated increased affective performance in perspective taking over the participants who did not experience the patient vision before examining the patient. Experiencing the patients vision before examining the patient was also expected to aid in the cognitive task of diagnosing the patients crani al nerve disorder. We expected participants who experienced patient vision to arrive at a correct diagnosis from the exam more often than those participants who did not experience the patients vision prior to the exam. This was not shown to be true as n early all participants in both groups arrived at a correct diagnosis. However, a significant majority of the participants were able to diagnose the disorder solely from experiencing the patient vision, demonstrating that the patient vision feedback does aid in the cognitive task of diagno sing the cranial nerve disorder. The patient vision feedback was successful in its goals of improving affective and cognitive performance. In the right scenario, this feedback may be a powerful method

PAGE 251

251 of emphasizing both affective and cognitive aspects of the neurological exam to novice learners. For example, in the scenario of an elderly patient with dementia who is unable to adequately express his vision problem, patient vision feedback could be used to emphasize the im portance of certain tests, e.g. fundoscopic examination, as well as provide the learner with an emotional grounding that would aid in making an affective connection with the patient, i.e. achieving rapport with a patient who has difficulty communicating. The H Map feedback was expected to improve the completeness and efficiency of the psychomotor portion of the neurological exam: the eye movements test. The presence of the H Map improved the efficiency of participants eye movements tests, but did not im pact the completeness of these tests. Possible causes highlighted potential system design improvements such as providing a stereoscopic display and a less cognitively demanding merging of the virtual and physical spaces. Overall, the introduction of the t wo feedback elements into MRIPS -NEURO had a positive impact on participants affective, cognitive, and psychomotor performance. After demonstrating that real -time feedback elements can positively impact performance in these skill sets, we turn our focus t o learning. We next examine whether repeated practice with MRIPS -CBE while receiving real time and post experiential feedback leads to affective, cognitive, and psychomotor learning in the clinical breast exam.

PAGE 252

252 A B C Figure 81. Progression of the H Map visualization as the learner performs the eye movement test.

PAGE 253

253 A B Figure 82. In the VSP feedback experience of Raij et al, the learner: A) performs CBE of a virtual human using MRIPS -CBE, and then B) relives the experience in the avatar of the virtual human. A B C D E F Figure 83. Patient Vision with a left eye affected by CN3 (A, C, E) and CN6 (B D, F). Looking: A,B) straight ahead; C,D) down -left; and E,F) to the right.

PAGE 254

254 Figure 84. The initial implementation of patient vision alpha blended each eyes image to present double vision on a non-stereoscopic display. However, this approach makes it difficult to distinguish which image corresponds to the left and right eyes.

PAGE 255

255 Figure 85. Study procedure. Group A received patient vision feedback before examining the virtual human patients. Group B did not receive this feedback before the exams. Group B received patient feedback after the exams in order to provide both groups with an equivalent set of experienc es.

PAGE 256

256 Figure 86. Physical setup of study. The standing participant is performing an exam of the virtual human while the sitting participant wears an HMD to view the exam through the virtual humans eyes.

PAGE 257

257 A B Figure 87. Views during the exam: A ) view of the participant performing the exam; B) view of the participant experiencing patient vision. Image B is sized to enable the reader to experience the double vision. If the reader begins to cross his eyes while focusing on the finger, he should see double two fingers.

PAGE 258

258 CHAPTER 9 LEARNING, TRAINING T RANSFER, AND IMPACT OF REAL -TIME FEEDBACK IN MRIPS -CBE This chapter describes a user study, Study MRIPS -Learning, which evaluated learning of cognitive, psychomotor, and affective skills within MRIPS CBE and transfer of learned skills to the real world interpersonal scenario of performing a CBE of a standardized human patient (SP). Participants completed a baseline evaluation consisting of a CBE of an SP. Participants then completed a series of three MRIPS CBE interactions including real -time and post experiential feedback. Improvement from the repetitive practice with MRIPS -CBE was measured by a second, subsequent CBE of an SP. Learning was evaluated as improvement throughout the three MRIPS -CBE i nteractions. Training transfer was measured as improvement from the baseline (pretest) SP interaction to the subsequent post -test SP interaction. Learning and training transfer were demonstrated for cognitive, psychomotor, and affective components of th e CBE. Study MRIPSLearning also evaluated the impact of real time feedback on performance, through comparison to historical control groups. Results of these comparisons demonstrated significant improvement in cognitive, psychomotor, and affective task p erformance as a result of receiving real time feedback. Collaborators: Educational technology professor Rick Ferdig was consulted in designing the study. Medical collaborators Scott Lind and Brenda Rossen recruited participants for the study. Scott Lind, Brenda Rossen, Andy Laserno, James McLoughlin, Jamison Weir, Steven Blackwood, and Amrew El Alamad assisted with evaluating participants SP psychomotor performance through video review. Those video raters and Joanna Lind, Carson Kisner, and Jennifer Car rick assisted with video review to evaluate affective performance in SP interactions.

PAGE 259

259 Personal contributions: I was the primary designer of the study and performed all analysis other than the video review of SP interactions. Relevance to thesis: The thesis states: Interpersonal simulation incorporating instrumented haptic interfaces and providing real -time evaluation and feedback of performance (MRIPS) improves users scenario-specific psychomotor, cognitive, and affective skills. Skills improvement trans fers to the real world interpersonal scenario being simulated, demonstrated as improved performance in the real world interpersonal scenario. Study MRIPS Learning directly evaluates the thesis statement for the CBE scenario by evaluating learning, traini ng transfer, and the impact of feedback on performance. 9.1 Introduction This chapter describes a user study, Study MRIPS -Learning, that investigated learning (retained skills improvement) in MRIPS -CBE and transfer of learned skills to the real world inter personal scenario of CBE of a human patient. Study MRIPS Learning sought to accomplish four tasks, each with a corresponding meta hypothesis (formal hypotheses concerning individual measures of performance are given in Sections 9.5 through 9.7). Determine what learning occurs in users of MRIPS -CBE. o Meta hypothesis: Participants will improve in cognitive, psychomotor, and affective performance throughout repetitive practice with MRIPS -CBE. Performance in these tasks will significantly improve from the firs t MRIPS interaction to the third MRIPS interaction. Determine whether improvement in skills within MRIPS CBE transfers to the real world, in the form of improvement in performance in CBE of human patients. o Meta hypothesis: After practice with MRIPS -CBE, pa rticipants cognitive, psychomotor, and affective performance in CBE of an SP will have

PAGE 260

260 significantly improved in relation to baseline levels taken before practice with MRIPS -CBE. Determine whether the presence of real -time feedback causes learners to sign ificantly outperform past users of MRIPS -CBE (without real -time feedback) in cognitive, psychomotor, and affective tasks. o Meta hypothesis: Participants in Study MRIPS -Learning will perform significantly better in cognitive, psychomotor, and affective tasks (for which real -time feedback was provided) than prior users of MRIPS -CBE (participants in Study MRIPSx2, Section 4.3) who did not receive real -time feedback. If learning occurs, determine whether skill sets are learned concurrently; or, whether one skill set must be maximized before performance in other skill sets can improve. o Meta hypothesis: We will not observe the following: performance in a single skill set requires maximization before the other skill sets can improve. To accomplish these tasks, we conducted a study with novice medical students at the Medical College of Georgia. Participants were evaluated in a CBE of a standardized human patient (SP), then received a simulationbased curriculum consisting of three MRIPS -CBE interactions, and were f inally re evaluated in a CBE of an SP. 9.2 Study Design 9.2.1 Evaluating Learning and Training Transfer The MRIPS Learning study procedure is visualized in Figure 91. The novice medical students recruited for Study MRIPS -Learning had received lecture bas ed learning (textbooks, power point, and expert demonstration with silicone model) of CBE, and had no experience with CBE of standardized patients (SPs) or real patients. The study procedure was as follows: Participants were consented and completed a back ground survey assessing their experience with CBE.

PAGE 261

261 Immediately following the consent and survey, participants performed a CBE of an SP, the SP-Pretest. The SP wore a silicone breast vest, incorporating a simulated mass, to evaluate participants on their ability to find breast masses. Video of this CBE was later reviewed by medical educators and myself to establish a baseline for the students performance in the cognitive, psychomotor, and affective components of CBE. CBE of an SP was chosen as the real world interpersonal scenario in which to evaluate training transfer, because: o SPs have previously been validated for evaluating clinical skills, providing an experience equivalent to a real patient [16]. o SPs provide each participant with an equivalent exp erience. The SPs were trained to answer participant questions with responses taken from a script (the database of responses used by MRIPS -CBE). While a real patient might describe an ailment slightly differently to each participant, the SP provided the s ame response to all participants. Providing participants with nearly identical experiences in the SP Pre -test and SP Post -test helps to maintain internal validity of the study e.g. a participant does not have a harder SP in the SP Pre test than in the SP Post -test (or vice versa), which would skew the delta measured between baseline and final evaluations. o Real patients with a breast mass were not available for the study. Evaluating participants ability to detect breast masses would have required the real patient to wear a breast vest, as the SP did; this would reduce the main benefit associated with the real patient, namely the palpating of the patients breasts. o Real patient interactions could not be videotaped (for later review by experts and the experimenters) because of privacy concerns related to the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Privacy Rule. Video recording participants CBEs allowed multiple experts to rate the participants performances without the logi stical restriction of having multiple experts present at the time of the CBE. Approximately one week after the SP Pre-test, participants completed the first of three MRIPS -CBE interactions. Two MRIPS -CBE interactions followed with approximately one week b etween interactions. The schedule of the MRIPS interactions had to be approximate in order to accommodate medical student schedules. One week between interactions is a standard time used in repetitive learning studies [63]. Three practice opportunities each spaced approximately one week apart were chosen because this created a MRIPS curriculum of similar length to existing clerkships (23 weeks in length) used to teach intimate exams in medical curricula. The content of the three MRIPS -CBE interactions was identical, i.e. the appearance, symptoms, and concerns of the virtual human patient were the same. Each MRIPS -CBE interaction incorporated the real time and post experiential

PAGE 262

262 feedback of cognitive, psychomotor, and affective performance described in Chapter 7. Approximately one week after the third MRIPS -CBE interaction, participants completed a CBE of an SP, the SP Post -test. The content of this SP interaction was the same as the content of the SP Pre test interaction. This enabled us to evaluate improvement due to repetitive practice with MRIPS CBE as the delta in performance between SP Pre-test and SP Post test. Participants received no education in CBE between SP Pre-test and SP Post test other than the MRIPS -CBE interactions. The time of one week between the end of treatment (the third MRIPS CBE interaction) and post -test is standard in evaluating whether treatment to post -test improvement is due to skills improvement and retention (i.e. learning) as opposed to short -term memorization [63]. 9. 2.2 Evaluating the Impact of Real-Time Feedback on Performance In addition to evaluating learning and training transfer with the above procedure, Study MRIPS Learning also evaluated the impact of real -time feedback on learner performance. To evaluate the impact of real -time feedback on performance, we compared cognitive, psychomotor, and affective performance of Study MRIPS -Learning participants to that of participants in Study MRIPSx2 (Section 4.3). The content of the MRIPS interactions in Study MRIPSx2 and Study MRIPS Learning is identical (i.e. same virtual human appearance, behaviors, responses, and critical moments). Other than real -time feedback, the difference between the two studies was the use of the Wizard of -Oz in Study MRIPSx2. In Study MRIPS x2, if a participant repeatedly asked a question (corresponding to a response in the virtual human response database) but did not get a response due to speech recognition or speech understanding failure, the experimenter triggered the virtual human to prov ide the appropriate response. This mechanism was not used in Study MRIPS -Learning. For MRIPS -CBE to function in its intended role as an on-demand learning opportunity, responses to participant speech depended wholly on speech recognition and understandin g modules. This difference has the potential to put Study MRIPS -Learning participants at a comparative

PAGE 263

263 disadvantage in tasks involving speech (e.g. taking a complete medical history). In the sections detailing this analysis, we discuss the performance of the speech interface and how measures were designed to mitigate the effect of this difference. 9.2.3 Control Groups for Investigating the Validity of Study Results Prior to conducting Study MRIPS Learning, we conducted a pilot study which had two goals: d etermining the impact of a single MRIPS interaction on performance in CBE of an SP, and determining the impact of a single SP interaction on performance in subsequent SP interactions. This pilot study provided baseline data which allowed us to evaluate th e validity of the Study MRIPS -Learning results. The pilot study consisted of two phases. In the first phase, Group MRIPS -SP, eleven medical students with the same experience level as Study MRIPS -Learning participants, performed an MRIPS interaction. On e week later, this was followed by an SP interaction. The MRIPS interaction provided real -time feedback of coverage and correct palpation pressure, and provided post experiential feedback of coverage, pressure, visual inspection, and breast history completeness in a web -based interface. This webbased interface also provided self -driven feedback of affective performance in the form of reviewable videos of the participants and an experts interactions. Also in phase one, Group SP, an additional eight medical students, completed only the SP interaction. Medical educators, clinicians, and residents rated participant performance in the SP interactions of both groups by reviewing video of the interactions. Comparing the two groups performance in the SP int eraction allows us to evaluate how much of an impact one MRIPS interaction has on performance in a CBE of an SP. The second phase involved five participants from Group MRIPS SP performing a CBE of an SP in the womens health clinic. These participants wer e evaluated in -

PAGE 264

264 person by a single expert. In clinic, exams can not be videotaped for review by multiple experts. This SP interaction came approximately one month after Group MRIPS -SPs first SP interaction. These two SP interactions mimic the SP Pre -tes t and SP Post -test interactions of Study MRIPS -Learning. Evaluating improvement in performance between these two SP interactions allows us to determine whether the SP Pre test of Study MRIPS could cause observed improvement from SP Pre test to SP Post tes t. If, in this pilot study, there is no improvement (or there is a decrease in performance) between the two SP interactions, than any improvement from SP Pre-test to SP Post test must be due to the three MRIPS -CBE interactions. Analysis of performance in this pilot study and its impact on the validity of the Study MRIPS Learning results is presented in Section 9.8. 9.3 Population Participants were recruited from 3rdyear medical students beginning a womens health clerkship at the Medical College of Georgia. The total size of this population was 30 students. Prior to conducting the study, we conducted a power and sample size analysis using G*Power3 [ 146]. With 5 repetitions and an alpha of 0.05, we could expect to find large effects (Cohens f2 >= 0.4) with a sample size of 17 participants. Our collaborators at the Medical College of Georgia were able to recruit 16 of the 30 medical students for the study. Participants were unpaid volunteers. With a study taking place over the course of one month, we expected to lose a few participants. All 16 participants completed the background portion of the study in which they received a lecture in which an expert clinician demonstrated a CBE. These participants also completed the background survey. Of these par ticipants, only 12 participants completed the SP Pre test and first MRIPS -CBE interaction, and only 8 participants

PAGE 265

265 completed the entire study. With 8 participants, we can still find large effects ( f2 = 0.5). Statistically, these effects must be larger th an those at f2 = 0.4 (i.e the test statistic, e.g. t or F, must be larger to indicate significance). Because the population was small, we view Study MRIPS Learning as only a first step at investigating learning in MRIPS. For evaluation of learning we anal yze the data of the 8 participants who completed the entire treatment. In evaluating the impact of feedback, we will also incorporate the data of the 12 participants who completed the SP Pre -test and first MRIPS interaction. Larger historical control groups are also used in comparison to evaluate the impact of feedback. All participants had the same background in clinical breast examination. None of the participants had performed a CBE of an SP or a real patient. Participants experience in CBE was limi ted to receiving a lecture and demonstration from an expert clinician. During the course of the study, no participants received education or practice in clinical breast examination from any sources other than the studys treatments. 9.4 Statistical Analys is Because a small population was used, we can not assume a normal distribution of participant performance. For this reason we primarily used non-parametric tests in our analysis. When comparing changes in performance from SP Pre-test to SP Post -test, w e used the Wilcoxon signed-rank test [ 147]. The Wilcoxon test is a non -parametric version of the paired samples t -test, used to evaluate within-subjects change in a 2 level repeated measure (i.e. before and after treatment).

PAGE 266

266 To compare changes in perfor mance throughout the three MRIPS treatments, the Friedman test was used. The Friedman test is a nonparametric version of a one way analysis of variance with a k -level repeated measure (k > 2) [ 148]. For dichotomous (binary outcome) categorical variables, McNemars test is used to assess significant change in a 2-level repeated measure (i.e. before and after treatment) [ 149]. When comparing to historical controls, if the control group is significantly large enough to assume a normal distribution, we used 2 way analysis of variance (ANOVA) because of the relative common usage and familiarity to readers. For all tests, the 95% in rejecting a null hypothesis. Since comparison to historical controls is a between subjects comparison, we also use Fishers exact test when comparing dichotomous variables between -subjects. Fishers exact test is a non parametric substitute for the Chi -square test which is used with small sa mples sizes (i.e. when the expected value for any cell in the 2x2 contingency table is less than 10) [ 150]. When describing changes in performance, we will occasionally refer to the median change. In the small population of Study MRIPS Learning, a single outlier could significantly impact the mean, but the median is less sensitive to these outliers. 9. 5 Cognitive Performance The cognitive tasks on which participants were evaluated are: Breast history completeness Evaluation of the patients medical histor y and risk factors for breast cancer. The participant must recall the important questions to ask in order to evaluate the patients history of present illness (breast pain), risk factors, and relevant medical history.

PAGE 267

267 Visual inspection completeness. Performing a complete visual inspection. The participant must recall and have the patient assume the three poses required for a complete visual inspection: arms relaxed at sides, hands on hips with chest flexed, and arms raised above head. 9. 5 .1 Measures 9.5.1 .1 Breast history completeness The completeness of the breast history is measured as the number of questions asked from a 27 item list (Table 9-1). These items are taken from validated instruments used to evaluate students in breast history taking of SPs and real patients at the University of Floridas College of Medicine and the Medical College of Georgia. Twenty one of these items are present in the procedural checklist real -time feedback described in Chapter 7. During the MRIPS -CBE interactions, these i tems are displayed on the screen above the virtual human, ordered by topic (history of present illness, medical history, family history, social history). This feedback is expected to improve performance in the cognitive task of recalling the critical ques tions to ask and aid in keeping track of the items previously queried. All twenty -seven items are reviewed in the post experiential feedback viewed by participants after each MRIPS -CBE interaction. By leaving six items off of the real -time feedback and displaying them only in post experiential feedback, we are able to observe whether real time feedback has benefits beyond those of post experiential feedback for the breast history taking task. The MRIPS -CBE speech interface automatically records participants asking of items on this list. For the SP interactions, the completeness was also assessed by reviewing video.

PAGE 268

268 The experimenter also reviewed video of participant performances to confirm or correct the automated rating. A discussion of the automated rating is provided in Section 9.5.5. The manual review was performed to remove two types of errors: false positives and false negatives. False positives occurred when the participant asked a question and received an unrelated response which provided info rmation related to an item in the list. Eight false positives occurred across 28 MRIPS interactions. It is possible that receiving this unrelated information would cause the participant to not query this item further. However, in practice, we observed t hat participants typically pursued their original line of questioning until they received a related answer, and then asked for confirmation of the earlier unrelated piece of information. Even with this behavior, because we did not include false positives in participants MRIPS scores, these scores may be viewed as a lower bound on actual performance. In contrast, false positives were included in SP scores. With the SP, false positives occurred when SPs volunteered more information than the script called for in response to a participant question. There were two instances (out of 20 SP interactions) in which a false positive was not counted. In both instances, the SP volunteered information that her mother had breast cancer in a critical moment speech at the end of an interaction (after the exam). The participants had not asked about family history and were closing the patient -doctor interview, when the SP used the speech (similar to) Im scared it could be cancer because my mom died of breast cancer The choices made in how to handle scoring false positives reflect the qualities of the virtual human and real human interactions: the virtual human interaction follows a more rigid questionanswer form than the more openended conversation with the SP.

PAGE 269

269 F alse negatives occurred when participants repeated a question multiple times but could not get a related response. These occurred due to consistent speech recognition and understanding errors specific to that question (due to southern accent, poor enunciation, or odd phrasing which could not be matched to the correct response). There were 43 false negatives across the 28 MRIPS interactions, an average of ~1.5 per interaction and a mode of 2 per interaction. SP interactions did not have false negatives. 9 .5.1.2 Visual inspection completeness In the MRIPS interactions, the completeness of the visual inspection was evaluated automatically. For the SP interactions, the completeness of the visual inspection was evaluated by review of video. For a complete vi sual inspection, the patient should be examined with arms relaxed by her sides, hands pressed on hips, and arms raised above head. Participants were judged on whether they performed any visual inspection (one or more poses) and whether they performed a complete visual inspection (all three poses). Participants visual inspections are guided in MRIPS, as the procedural checklist real -time feedback in MRIPS displays icons depicting the poses used in visual inspection (Figure 7 2). 9. 5 .2 Hypotheses Hypothesi s Breast History Completeness Learning and Transfer : The completeness of participants history taking and breast cancer risk assessment will increase from SP Pre-test through the three MRIPS interactions. The completeness of participants history taking and breast cancer risk assessment will increase significantly from SP Pre-test to SP Post -test. o Null hypothesis : Participants scores on the breast history completeness instrument (Table 9 1) will not significantly improve throughout the two SP and three M RIPS interactions.

PAGE 270

270 Hypothesis Breast History Completeness Feedback Impact : For the breast history items which are included in the real -time feedback, participants completeness in the first M R IPS interaction will be significantly greater than in historica l data in which real -time feedback was not provided. o Null hypothesis : Completeness of participants breast history taking will not be significantly different from historical MRIPS data. Hypothesis Visual Inspection Learning and Transfer : The number of par ticipants who perform a complete visual inspection will increase from SP Pre-test through the three MRIPS interactions. The number of participants performing a complete visual inspection will increase significantly from SP Pre -test to SP Post -test. o Null hypothesis : The number of participants performing a complete visual inspection will not significantly improve throughout the two SP and three MRIPS interactions. A summary of results of acceptance and rejection of these hypotheses is shown in Table 9 2. 9. 5 3 Results: Breast History Learning and Training Transfer Means and standard deviations as well as ranges are shown in Table 9-3. Scores for the eight participants are visualized in Figure 9 -2. Participants significantly improved the completeness of their breast histories from SP Pre -test to SP Post -test, by Wilcoxon test: Z = 2.4, p = 0.02. Seven of eight participants increased the completeness of their breast histories and one participant queried the same number of items (though not the same items) in pre -test and post -test. The median change was an increase by 5.5 items (20% of the 27 items). The largest component of the improvement was from SP Pre-test to MRIPS #1, as all participants improved their completeness by an average of 8.3 items. Completeness in MRIPS #1 was significantly greater than in the SP Pre-test, by a Wilcoxon test: Z = 2.5, p < 0.01.

PAGE 271

271 Throughout the three MRIPS interactions, participants did not significantly increase the number of breast history items asked on average. From MRIPS #1 to MRIPS #2, three participants improved, two scored the same, and three had their scores decrease. From MRIPS #2 to MRIPS #3, five increased their scores, and three had their scores decrease. Overall, from MRIPS #1 to MRIPS #3, the average score incr eased by one question asked. For six of the participants, the real -time and post experiential feedback regarding the completeness of their breast history appears to have caused a ceiling effect in the first or second MRIPS interaction. The other two part icipants ( participants 20 and 22 in Figure 9 -2 ) improved in each repeated MRIPS interaction. These two participants represented the lower bound on performance in each of MRIPS #1 and MRIPS #2 interactions. The effect of the repetition was to bring these participants up to the level of performance achieved by the other participants. With respect to the ceiling effect experienced by the majority of participants, the ceiling was set at or close to 21, the number of items appearing in the real time feedback. For measuring the ceiling value, we included false positives, as false positives caused the items to appear checked off in the feedback. As such, participants could not use the feedback to keep track of whether they had actually asked about the false positive item. In MRIPS #1 and MRIPS #2, six of 8 participants asked 19 or more of the 21 items appearing in the real time feedback. In MRIPS #3, all participants asked 20 or 21 items. Reasons for not asking all 21 items may include not wanting to break up the topic flow (i.e. if an item related to medical history is skipped and the participant moves on to family history, the participant may not feel that it is necessary to return to medical history later), frustration with speech recognition (which

PAGE 272

272 was apparent only in interaction by one participant), or not wanting to rely completely on the real time feedback. Some participants clearly did rely on the real -time feedback, as the real -time feedback guided their line of questioning. Typically, medical stu dents have a difficult time asking all questions within a specific topic (e.g. family history) before moving along to the next topic [84]. The real time feedback appeared to help guide the progression of questions and topics, as 5 of 8 participants follow ed the progression displayed in the real -time feedback exactly for at least one MRIPS interaction. One participant followed the progression exactly for all MRIPS interactions, with appropriate introductions to each topic. This participant followed the feedback exactly but was not progressing mindlessly through the items displayed by the feedback, i.e. trying to game the system by checking off all items. If a participant asked a nonsensical question for the purpose of checking off an item without dis playing knowledge of the medical information the item refers to, we did not count that item in the participants total. We observed only two instances of this: in MRIPS #1 and MRIPS #2 interactions, the same participant asked are you having any problems with your hormones in reference to the hormones item. The hormones item refers to the risk factor of taking replacement hormones (if the patient is post menopausal) or taking birth control. By the third MRIPS interaction, this participant realized wha t hormones referred to and asked an appropriate question about hormone use.

PAGE 273

273 Aside from the five participants following the progression exactly, two additional participants followed the progression of topics, but asked questions within each topic in an or der different from the progression shown in the feedback. It appears clear that the feedback and repetition helped participants learn which important questions to ask the patient. From SP Pre test to MRIPS #3, the median improvement was 9.5 items or 35% o f the list. Most of this improvement was retained with the removal of the feedback in the SP Post test. From MRIPS #3 to SP Post test, the median retention rate was 84% (calculated by dividing SP Post -test performance by MRIPS #3 performance, and taking the median result among participants). The improvement in number of items asked does not appear to be a random effect of guessing at possibly important questions, as there were specific items which a significantly larger percentage of the population querie d in SP Post -test than in SP Pre test. In the SP Pre -test, no participants asked about the age at which the patient began menarche. Early onset of menarche is a well -confirmed correlate of increased risk for breast cancer [ 151]. All participants asked about onset of menarche in MRIPS #3, and 5 of 8 remembered to ask in the SP Post test. Other such items are summarized in Table 9 4. Of the items that received large (+3 or more) increases, the patients age is the only item not included in the real -time feedback. The patient age item is part of the post experiential feedback. A discussion of the comparative impact of the real -time and post experiential feedback is given in Section 9.5.4. As the result of this analysis, we accept Hypothesis Breast Histor y Completeness Learning and Transfer Participants significantly increased the number of breast history items asked from before to after the MRIPS treatment. The improvement was not solely

PAGE 274

274 due to repetition, but due to repetition with feedback, as a ceil ing effect was observed starting in the first MRIPS interaction. The improvement was not from random guesswork, as specific critical items saw significant improvement. MRIPS combination of repetition while receiving real -time feedback led to significant improvement in the completeness of learners breast histories in the real world scenario of a CBE of an SP. 9. 5 4 Results: Impact of Feedback on Breast History Completeness To evaluate whether the procedural checklist real time feedback improved breast hi story completeness in MRIPS, we compared participants in Study MRIPS -Learning to a historical control group. The historical control group consisted of experienced (having performed >= 5 CBEs) and inexperienced (04 CBEs) students from Study MRIPSx2 (Secti on 4.3). This group contained 29 participants. The residents and clinicians in Study MRIPSx2 were not included because, as discussed in Section 4.3, the number of questions asked is not a valid means of querying expert performance. To gain a slight incr ease in power for this comparison, we included all 12 participants in Study MRIPS Learning who completed the first MRIPS interaction. The addition of the four participants that only completed one MRIPS interaction actually served to decrease the mean scor e for participants in Study MRIPS -Learning. The instrument used to evaluate breast history completeness in the historical control contained 20 items. These items were a subset of the 21 items in the real -time feedback of Study MRIPS -Learning (nonhighligh ted items in Table 9 -1). All participants in Study MRIPS Learning queried the one additional item that was not present in Study MRIPSx2, thus their scores were reduced by one point in order to compare with the Study MRIPSx2 scores.

PAGE 275

275 Because the groups we re large enough to assume a normal distribution (and from a histogram, a normal distribution appeared to be an acceptable fit), a univariate analysis of variance was conducted. Participants who viewed the procedural checklist in real -time performed significantly more complete breast histories than the historical control group which did not receive the real -time procedural checklist feedback. Study MRIPS -Learning participants outperformed the Study MRIPSx2 participants by an average of nearly 7 items: Study MRIPS Learning = 17.3 + 4.3; Study MRIPSx2 = 10.6 + 3.0. This was significant by ANOVA: F (d.f. = 1, n = 41) = 33.2, p < 0.001. From this result we accept Hypothesis Breast History Completeness Feedback Impact The real -time feedback of the procedural checklist improved performance in breast history completeness over that observed in the historical control group. This result shows that real -time feedback is more effective than no feedback. By investigating the data from the 8 participants who performed three MRIPS interactions, we find that real time feedback in addition to post experiential feedback has benefits over post experiential feedback alone. In all of the SP and MRIPS interactions, participants asked a higher percentage of the items in the real -time feedback than the items reviewed only in the post experiential feedback. This difference in percentages increased with the repetition as shown in Figure 93. For each interaction, the difference is significant by Wilcoxon test, at p = 0.05 for SP Pre -test and at p < 0.01 for the remaining interactions. Both categories of items were asked more often in the SP Post test than in the SP Pre -test, but the improvement in items appearing in the real -time feedback is significant and the

PAGE 276

276 improvement in pos t experiential items was not significant by Wilcoxon test: improvement in real -time items: Z = 2.5, p < 0.01; improvement in post experiential items Z = 0.9, p = 0.5, ns Participant 21 may have benefitted from the post experiential feedback more than the other participants (see Figure 9-2) as this participants improvement from MRIPS #3 to SP Post -test was due to asking items in the post experiential feedback. Interestingly, this participant did not spend more time with the post experiential feedback than other participants. Participant 21 spent ~80 sec. with the post experiential feedback which was similar to the average time of ~75 sec (ranging from ~30 sec. to ~102 sec.). From these observations, it appears as though there is benefit from practicing w ith a visible list of topics to query, along with the ability to cross off the items. One might wonder why a graphical display is needed for this task; would a piece of paper be adequate. Students are not given a list of items during SP or real patient interviews. In the practice scenario of MRIPS, visualizing this information on the screen above the virtual humans head has the advantage of not incurring an increased cognitive load from looking back and forth between the screen and a piece of paper. 9.5 .5 Visual Inspection Learning and Training Transfer To evaluate improvement in visual inspection of the patients breasts, we analyzed how many participants performed any visual inspection in each interaction as well as how many of these visual inspections were complete. An inspection was complete if it involved the three poses of arms relaxed, hands on hips, and arms raised above head. From SP Pre -test to SP Post -test, there was a trend towards significant improvement in the number of participants perform ing visual inspections. In the SP Pre test, two of the eight participants performed a visual inspection. This number increased

PAGE 277

277 to six of eight participants in the SP Post -test. This trended towards significance at p = 0.06 from McNemars test. From SP P re -test to SP Post -test, there was a significant increase in the number of participants who performed complete visual inspections. No participant performed a complete visual inspection in the SP Pre-test, while five of eight performed complete inspections in the SP Post test. This improvement was significant by McNemars at p = 0.03. The pattern of improvement in visual inspection mirrored that of improvement in breast history completeness, as shown in Table 95. The improvement from SP Pretest to MRIPS #1 in the number of participants performing any visual inspection trended towards improvement ( p = 0.063); the improvement in the number of participants performing a complete visual inspection was significant ( p = 0.03). Among the three MRIPS interactions there were no significant changes in the number of participants performing a visual inspection, nor were there significant changes in the number of participants performing a complete visual inspection. It was expected that in all MRIPS interactions, all participants would perform a complete visual inspection, as the three poses were included in the real -time procedural checklist feedback. One participant did not perform a visual inspection in any of the MRIPS or SP interactions. For this participant, no t performing any visual inspection appeared to be due to participant preference, as there were no technical difficulties ex perienced in this task in MRIPS. However, the experimenter did not ask the participant if this was the case.

PAGE 278

278 As the completeness of participants visual inspections increased significantly from the SP Pre test to MRIPS #1 and from the SP Pre test to the SP Post -test, we accept Hypothesis Visual Inspection Learning and Transfer. MRIPS provided repetitive practice which reinforced the n eed to perform a visual inspection and the three poses for a complete visual inspection. This knowledge transferred to the SP Post test, indicating learning and training transfer from practice with MRIPS. 9. 5.6 Discussion Repetitive practice with MRIPS in which the procedural checklist feedback was provided led to improved performance in completeness of both breast history taking and visual inspection. For breast history completeness, real -time feedback led to improved performance in comparison to perform ance when no feedback is provided. Real -time feedback also appears more effective at helping learners retain information than post experiential feedback. Providing real -time feedback for these cognitive tasks requires only a speech interface (i.e. does n ot require a touch interface) thus MRIPS is not unique among interpersonal simulations in being able to provide this feedback. However, MRIPS is the first interpersonal simulation to provide real -time feedback of this type and evaluate its impact. The c ombination of the on-demand (repetitive) learning opportunity and real -time feedback provided by MRIPS led to improvement in cognitive tasks in a real world interpersonal scenario. 9. 6 Psychomotor and Cognitive Psychomotor Performance In our analysis, we grouped the psychomotor and compound cognitivepsychomotor tasks together. The psychomotor and cognitivepsychomotor tasks on which participants were evaluated are: Palpating all breast tissue, regardless of pressure.

PAGE 279

279 o Palpating the entire breast, i.e. complete coverage o Palpating the axillary, supraclavicular, and infraclavicular areas. Using correct (deep) pressure to palpate the entire breast. Palpating along the correct (expert) patternof -search. Determining whether palpated breast tissue is normal or a mass, i.e. finding actual masses and not reporting false positive masses. 9. 6 .1 Measures 9.6.1.1 Coverage and correct pressure MRIPS -CBE is able to automatically calculate the percent area of the cone of the breast that is palpated at each of the levels of pressure (light, medium, high, toohigh). As discussed in Section 7.4, this requires a calibration step in which an expert performs an exam using MRIPS. For Study MRIPS -Learning, the calibration exam was performed by a clinician who was considered to be the breast examination expert at the Medical College of Georgia. Although MRIPS calculates percent area at light, medium, high, and toohigh levels of pressure, these were simplified to three measures for recording learner performance. These three levels were percent area palpated with superficial pressure (light, medium), high pressure, and too-high pressure. An ideal performance would palpate 100% of the breast at high pressure. However, it was not clear how to integrate superficial and toohigh meas ures into a rating of use of correct pressure. For example, is it better to palpate at 0% 70% 20% or 5% 70% 15%? The first palpates 90% at deep pressure, while the second decreases the use of toohigh pressure at the expense of palpating only 85% at deep pressure. We define deep pressure as high or greater pressure (high or too-high pressure). To determine how to integrate superficial and toohigh measures into an overall score of performance, we consulted with two medical experts in CBE. Both experts expressed that it was better to palpate toohard than to not palpate deep enough, e.g. the patient

PAGE 280

280 would rather hear you say sorry for pressing too hard than sorry you missed a lesion because you did not [press hard enough] (Carla M. Pugh, personal com munication). Based on this feedback from experts, we used two measures to evaluate coverage and use of correct pressure. 1 Coverage: The percent area palpated at light or greater pressure is used as the measure of coverage. 2 Correct pressure: The percent are a palpated at high or toohigh pressure is used as the measure of correct deep pressure. An additional question lies in determining what constitutes a significant change in percentage of breast tissue palpated. This significance differs from significance in a statistical sense; here a significant change refers to the smallest percentage change that matters to the outcomes of the exam. As one important outcome is finding masses, we use the size of the masses present in the breast to calculate a significan t percentage. The breast used in MRIPS is approximately 7 (17.8 cm) long by 5 (12.7 cm) wide. The masses used were approximately 2 cm x 2 cm. Thus a mass represents 1.8% or roughly 2% of the area of the breast. Not palpating 2% of the breast area cou ld result in not finding a mass, thus we chose 2% as a threshold of significance when discussing changes in participant performance in coverage and in palpating the breast at deep pressure. If a participant can improve by 2%, this hypothetically increases the participants ability to find masses present in the breast. In determining what c onstituted a passing score in area palpated, our medical collaborators expressed that they would accept competency in percent of breast tissue palpated [ 9 ]. In the SP interacti ons, coverage and use of correct pressure can not be measured with the same precision as in MRIPS -CBE. Although the SP wore a silicone breast vest

PAGE 281

281 (to incorporate breast masses) our method of quantitatively measuring coverage and pressure (Section 7.4) c ould not be applied due to an inability to have the SP lie in the same position for each exam. This would be required to maintain a calibration between the camera which tracked palpation position and the sensors placed in the vest. Additionally, the breast vest could not accommodate sensors, as it did not provide a rigid surface beneath the silicone on which to seat the sensors. The sensors must be placed on a rigid surface in order to maintain a consistent relationship between force applied and value reported. Adding a rigid surface would make the vest uncomfortable for the female SPs. Because of these limitations, to evaluate psychomotor performance in the SP interaction, five experienced medical professionals and two medical students rated performance from reviewing video of participants exams. Three MDs, one RN, one resident, and two 3rdyear medical students performed the video rating. The instrument used to rate psychomotor performance was a validated instrument used to evaluate students in the w omens health clerkship at the Medical College of Georgia. All video raters were trained in using the video rating instrument to reduce variability between the five experienced raters and the two inexperienced (medical student) raters. Items from the vid eo rating instrument pertaining to coverage and pressure are listed in Table 9-6. Palpation of the ancillary areas of tissue included in the CBE was evaluated separately. Palpation of the axillary, supraclavicular, and infraclavicular areas was evaluated as three dichotomous (yes or no) variables. MRIPS -CBE reported palpation of these three areas automatically. For the SP interactions, the video raters determined

PAGE 282

282 if these areas were palpated. Participants performance in this task is rated as the numb er of areas palpated, from 0 to 3. 9.6.1.2 Correct p attern of s earch In evaluating the participants patternof -search, MRIPS -CBE calculated a total deviation of the learners pattern from the experts pattern. This was calculated by Equation 91, a summation over all segments in the learners pattern, in which si is the ith segment of the learners pattern, vi is the matching segment of the experts pattern, and li is the length of the learners ith segment. The total deviation is the most sensitive meas ure calculated by MRIPS -CBE of the difference between learner pattern and expert pattern, but total deviation does not account for the overall length of the pattern. If two learners make one large mistake (one segment with high deviation) both are penaliz ed equally, even if this mistake represents a small percentage of one learners pattern and a large percentage of the other learners pattern. To account for overall pattern length, the total deviation is normalized by the total pattern length, as in Equation 92. This measure is the normalized deviation, and is used to in Study MRIPS -Learning as the measure of correctness of participants patterns of -search in MRIPS -CBE. i i i iv s lo90 / arccos (9 -1) i i i i i il v s lo o90 90 / arccos (9 -2) For the SP i nteractions, pattern of search can not be evaluated quantitatively or with the same precision as in MRIPS. To evaluate participants pattern of search, experts reviewed video to determine whether participants adequately used the vertical

PAGE 283

283 strip pattern, a different systematic pattern (e.g. spiral, spokes), or did not use a systematic pattern. 9.6.1.3 Finding masses In both MRIPS and SP interactions, participants reported the number, location, and quality (fixed or mobile, hard or rubbery) of masses found. This was reported in a note written by participants after the interaction. Participants drew a diagram of the breast which indicated the location of each mass found. The SP breast contained one hard, fixed mass at the base of the breast (Figure 94, B) which was placed at the chest wall. The MRIPS breast contained a similar hard, fixed mass at the chest wall, placed a few centimeters medially from the nipple (Figure 9 -4, A). This mass was judged by clinicians to be of similar difficulty to find as the mass in the SP breast. The MRIPS breast also contained a second mass at the tip of the breast cone. This mass was soft and semi mobile, i.e. it could be moved 1-2 cm but remained in the same area for all participants (Figure 9-4, A). All masses were rou ghly the same size. Masses were not spherical but were generally round, convex shapes and the long axis of each mass was ~2 cm in length. The experimenter and experts reviewed the notes to determine which reported masses were actual masses and which we re false positives. In the SP interactions, participants were graded on whether they found the actual mass and the number of false positives reported. In the MRIPS interactions, participants were graded on the number of actual masses found and false posit ives reported.

PAGE 284

284 9. 6 .2 Hypotheses Hypothesis Coverage L earning : Par ticipants coverage will increase with repeated MRIPS interactions. o Null hypothesis: The percent of breast area palpated at light or higher pressure will not significantly increase with repeated MRIPS interactions. Hypothesis Pressure Learning: Participants use of correct pressure will increase with repeated MRIPS interactions. o Null hypothesis: The percent of breast area palpated at deep or higher pressure will not significantly increase wit h repeated MRIPS interactions. Hypothesis Coverage Transfer : The coverage of participants exams will improve from SP Pre -test to SP Post -test. o Null hypothesis : Expert ratings of participants coverage will not significantly increase from SP Pre test to SP Post -test. Hypothesis Pressure Transfer: The pressure of participants exams will improve from SP Pre -test to SP Post -test. o Null hypothesis : Expert ratings of participants use of correct pressure will not significantly increase from SP Pre-test to SP Po st test. Hypothesis Coverage Feedback Impact : T he presence of real -time feedback of coverage will result in improved coverage in participants exams in MRIPS. o Null hypothesis: Participants coverage in MRIPS #1 will not be significantly more complete than historical coverage data from Study MRIPSx2. Hypothesis Pressure Feedback Impact : The presence of real -time feedback of use of correct pressure will result in improved use of deep pressure in participants exams in MRIPS. o Null hypothesis: The area palpat ed at deep pressure by participants in MRIPS #1 will not be significantly greater than area palpated at deep pressure in historical data from Study MRIPSx2. Hypothesis Patternof -Search Learning: Deviation of participants patternof search will decrease with repeated MRIPS interactions. o Null hypothesis: Participants normalized deviation from the expert pattern of -search will not decrease significantly with repeated MRIPS interactions. Hypothesis Patternof -Search Transfer: Expert ratings of the correctne ss of participants pattern of -search will improve from SP Pre-test to SP Post -test. o Null hypothesis: Expert ratings of the correctness of participants patternof -search will not significantly improve from SP Pre test to SP Post -test. Hypothesis Patterno f -Search Feedback Impact: The presence of real -time feedback of the correctness of participants patternof -search will result in a significantly larger percent of participants using the vertical strip pattern than in historical data from Study MRIPSx2. o Nu ll hypothesis: The percentage of participants using the vertical strip pattern in MRIPS #1 will be significantly larger than the percentage of participants using the vertical strip pattern in Study MRIPSx2. Hypothesis Finding Masses Learning: The number of masses participants find will increase with repeated MRIPS interactions.

PAGE 285

285 o Null hypothesis: The number of masses found will not significantly increase throughout the MRIPS interactions. Hypothesis Finding Masses Transfer: The number of participants finding the mass in the SP breast will increase from SP Pre-test to SP Post test. o Null hypothesis: The number of participants finding masses will not increase significantly from SP Pre test to SP Post -test. Hypothesis False Positive Masses Learning: The number of false positive masses reported by participants will decrease with repeated MRIPS interactions. o Null hypothesis: The number of false positive masses reported by participants will not significantly decrease with repeated MRIPS interactions. Hypothesis False Positive Masses Transfer: The number of false positive masses reported by participants will decrease from SP Pre-test to SP Post -test. o Null hypothesis: The number of false positive masses reported by participants will not significantly decrease from SP Pre test to SP Post test. A summary of results of acceptance and rejection of these hypotheses is shown in Table 9 7. 9. 6 .3 Results: Coverage and Pressure Learning and Transfer 9.6.3.1 Coverage learning Participants performance in palpating the three ancilla ry areas (supraclavicular, infraclavicular, and axilla) is listed in Appendix D.2. In SP Pre-test, only three of eight participants palpated one or more of these areas. The average number of areas palpated was 0.6 0.9. Six of eight participants improv ed by palpating more of these areas in MRIPS #1 than in the SP Pre-test; one participant palpated one less area and one participant palpated the same number of areas. This improvement is significant by Wilcoxon test: Z = 1.9, p = 0.047. This suggests that the real -time feedback provided by MRIPS assisted participants in palpating these areas. From MRIPS #1 to MRIPS #3, five of eight participants improved, two did not change, and one decreased in performance. This improvement was not significant (Wilcoxon test: Z = 0.96, p = 0.17). In MRIPS #1, participants palpated an average of 1.5 1.1 of these areas; this increased in MRIPS #3 to an average of 2.1 1.1 of these areas. From SP Pre -test to

PAGE 286

286 MRIPS #3, six of eight participants improved and two had no change. This improvement was significant by Wilcoxon test, Z = 2.3, p = 0.16. Most importantly, the number of participants palpating all three areas increased from zero to four, and the number of participants palpating at least one area more than doubled from three to seven participants. For the task of palpating the tissue surrounding the breast, practice with MRIPS appears to result in learning. Palpation of the breast was measured with different mechanisms and precisions in MRIPS and SP, so we can no t compare SP Pre -test and MRIPS #1 to investigate learning; instead we are only able to look for improvement throughout the three MRIPS interactions. We observed a ceiling effect occurring in MRIPS #1 for the majority of participants. Five of eight part icipants palpated 90% or more of the breast tissue in MRIPS #1. This is a significantly higher proportion than the 5 of 57 participants palpating at 90% or more in the historical control group of MRIPSx2 (by Fishers exact test, p < 0.005). This suggests that the real -time feedback of coverage resulted in a ceiling effect in MRIPS; further discussion of the impact of feedback on performance is given in Section 9.6.4. Because of their high level of performance in MRIPS #1, most participants decreased in coverage from MRIPS #1 to MRIPS #2. Using our 2% threshold for significant change, 5 of 8 participants decreased from MRIPS #1 to MRIPS #2. All 5 had palpated at >90% in MRIPS #1. Three of the participants whose performance decreased remained at or abov e 90%. The three participants improving from MRIPS #1 to MRIPS #2 improved from <80% to >89%.

PAGE 287

287 MRIPS #3 data was not available for one participant due to an equipment error. From MRIPS #2 to MRIPS #3, four of seven participants increased significantly, one increased by <2%, and two decreased. One of the participants who decreased palpated above 90% in both MRIPS #2 and MRIPS #3. From participants first to last MRIPS interaction, three participants improved significantly, three participants changed by less than 2%, and two participants performance decreased (including the one participant who only had data for MRIPS #1 and MRIPS #2). Overall, participants performed well in this task, as in MRIPS #3, five of seven participants palpated >90% of the breast and another participant palpated >89%. However, due to the ceiling effect, there was not a statistically significant improvement from MRIPS #1 to MRIPS #3, so we must reject Hypothesis Coverage L earning. We do so noting that performance in palpating t he three areas of tissue surrounding the breast improved significantly, and performance in palpation of the breast was significantly higher than performance observed in past MRIPS interactions. A larger population and increased number of repetitions with MRIPS is expected to provide clearer evidence of learning within MRIPS. 9.6.3.2 Coverage transfer From SP Pre -test to SP Post -test, participants significantly increased their coverage, palpating more of the breast tissue. In SP Pre test, 2 of 8 participan ts were rated as completely palpating the cone of the breast only, one participant was rated as completely palpating the entire breast, and 5 of 8 participants were rated as incompletely palpating the cone of the breast. In the SP Post test, 4 of 8 partic ipants were rated as completely palpating the cone of the breast and 4 of 8 participants were rated as completely palpating the entire breast. This represented an improvement for 6

PAGE 288

288 participants and no change for 2 participants. These changes represented significant improvement, by a Wilcoxon test: Z = 2.2, p = 0.02. In palpating the ancillary areas of tissue included in the breast exam (supraclavicular, infraclavicular, and axilla), participants trended towards significant improvement from SP Pre test to SP Post -test. In the SP Pre -test, 2 of 8 participants palpated two of these areas, and one participant palpated one of these areas. The other five participants did not palpate any of these three areas. In the SP Post test, four participants palpated two of the three areas, three participants palpated one of the three areas, and one participant did not palpate any of the three areas. This represented an improvement for five participants, no change for one participant, and a decrease in performance for on e participant. These changes trended towards significant improvement, by a Wilcoxon test: Z = 1.7, p = 0.08. From this trend in improvement in the coverage of ancillary areas of tissue and the significant improvement in ratings of coverage of the breast tissue, we accept Hypothesis Coverage Transfer 9.6.3.3 Pressure learning As with coverage, palpation pressure was measured using different mechanisms and precisions in MRIPS and SP, so we can not compare SP Pre-test and MRIPS #1 to investigate learning; i nstead we are only able to look for improvement throughout the three MRIPS interactions. Performance in the three MRIPS interactions is shown in Figure 95. Participants performed well in MRIPS #1, with 7 of 8 participants (88%) palpating more than 60% of the breast with deep pressure. For comparison, in Study MRIPSx2, only 58% (33 of 57 participants) palpated >60% of the breast with deep pressure. From MRIPS #1 to MRIPS #2, four participants improved, one participant had no significant

PAGE 289

289 change (<2%), and for three participants the percent area palpated decreased. One of the participants who performed worse in MRIPS #2 palpated 95% of the breast with deep pressure in MRIPS #1. It appears as though a ceiling effect occurred for about half of the participa nts. As with coverage, data from MRIPS #3 was obtained from 7 of 8 participants. From MRIPS #2 to MRIPS #3, four of seven participants increased the area palpated at deep pressure, and three participants decreased the area palpated at deep pressure. Two of the participants whose performance decreased experienced large drop offs, from 70% to 34% and from 54% to 31%. These participants may have experienced study fatigue. It is the opinion of our expert reviewers that these participants simply put less ef fort into palpation in MRIPS #3 than in the first two MRIPS exams. Overall, from their first to last MRIPS interaction, 5 of 8 participants increased the area palpated at deep pressure and three participants decreased the area palpated at deep pressure. T hese three included the two participants who may have experienced study fatigue, and the participant who palpated the most area of any participants (95%, in MRIPS #1). Other than the two participants who put less effort into MRIPS #3 than MRIPS #2 and MR IPS #1, the repetitive practice with MRIPS appears to have a positive effect on learners performance in palpating with deep pressure. However, the ceiling effect and lack of effort on the part of two participants resulted in a lack of significant evidenc e of learning, and we must reject Hypothesis Pressure Learning. 9.6.3.4 Pressure transfer From SP Pre -test to SP Post -test, participants significantly increased their use of the three levels of palpation pressure. In SP Pre-test, only 2 of 8 participants palpated with correct deep pressure, 3 of 8 palpated with correct medium pressure (but not deep

PAGE 290

290 pressure), 2 of 8 palpated with correct light pressure (but not deep or medium), and one participant was rated as not palpating with even correct light pressure In the SP Post test, 6 of 8 participants palpated with correct deep pressure and 2 of 8 participants palpated with correct medium pressure. This represented an improvement for 5 participants and no change for 3 participants. These changes represented significant improvement, by a Wilcoxon test: Z = 2.0, p = 0.03. From this result, we accept Hypothesis Pressure Transfer. Appendix D.2 lists performance for each participant in coverage and pressure tasks. 9. 6 .4 Results: Impact of Real -Time Feedback on C overage and Pressure To evaluate whether the coverage and correct pressure feedback given by the touch map improved participants coverage and use of correct pressure in MRIPS, we compared participants coverage and correct pressure use in Study MRIPS -Lear ning with the previous Study MRIPSx2 (Section 4.3). This allows us to compare the performance of novices receiving real -time feedback to both novices and experienced residents and clinicians who did not receive real -time feedback. The same mannequin, numb er of sensors, and density of sensors was used in both Study MRIPSx2 and Study MRIPS -Learning. The only significant difference with respect to palpation coverage and pressure was that a more precise method of evaluating coverage and pressure had been developed in the interim between the two studies. We re processed the historical data to represent this more precise knowledge of what constitutes correct coverage and deep pressure. The historical data from Study MRIPSx2 included unprocessed sensor value dat a from participants exams. In the previous evaluation in Section 4.3, coverage was evaluated as the area palpated at a pressure that was above the noise level for the

PAGE 291

291 sensors. With the creation of the touch map feedback and evaluation (Section 7.4), we have a more precise definition for coverage: the percent area palpated at light or greater pressure. In order to generate coverage data that fit this more precise definition for coverage, we processed the historical data using a threshold which represented a lower bound for light coverage in Study MRIPS Learning. This threshold was calculated as the minimum of the means for the light pressure distributions across the 64 sensors. Applying this threshold changed the percent area covered for only two of the Study MRIPSx2 participants, and by less than a significant amount (<2%). A similar method was used to calculate the percent area palpated at deep or higher pressure in Study MRIPSx2. The threshold used for this calculation was the minimum of the means of the high pressure distributions across the 64 sensors. Because the method of calculating area in Study MRIPSx2 intentionally overestimates the area palpated, and the thresholds used in the calculations were lower bounds, the calculated areas of coverage and deep pressure use for the historical control should be considered upper bounds of actual performance by the historical control participants. Because this between-subjects comparison had larger sample sizes, we used an independent -samples t test to comp are performance between Study MRIPSx2 and Study MRIPS Learning. Results are shown in Table 9 -8. Participants in MRIPS #1 were performing their 2nd CBE ever, their first CBE being the SP Pre test. Participants in the MRIPSx2 Inexperienced group were medic al students with between 0 and 5 prior CBEs performed, meaning the MRIPS exam for which their coverage and pressure was evaluated was their 1st to 6th CBE. Participants in the MRIPSx2 Experienced group were experienced medical students, interns,

PAGE 292

292 residents and clinicians performing from their 7th to >1000th CBE (the experienced students had performed 6 to >10 CBEs; most residents indicated they had performed between 50 and hundreds of CBEs; clinicians indicated they had performed between hundreds and thous ands of CBEs). Study MRIPS Learning participants in MRIPS #1 performed significantly better at the coverage task, palpating significantly more breast tissue at light or higher pressure than either the MRIPSx2 inexperienced or MRIPSx2 experienced groups. T he 12 participants in MRIPS #1 palpated 89.9% 11.6% of the breast at light or higher pressure, compared to 75.3% 12.4% for the MRIPSx2 Inexperienced group and 80.9% 9.0% for the MRIPSx2 Experienced group. The improvement over the MRIPSx2 inexperienc ed group is significant at p < 0.005 and the improvement over the MRIPSx2 experienced group is significant at p < 0.05. Both improvements are also significant at our 2% threshold. Participants in Study MRIPS -Learning performing only their 2nd CBE we re able to perform CBEs with significantly more complete coverage than participants from Study MRIPSx2 with more prior experience in CBE. The independent variable that changed between Study MRIPS Learning and Study MRIPSx2 is the presence of the touch -map real -time feedback of palpation completeness (coverage). From this result we accept Hypothesis Coverage Feedback Impact. The presence of the real -time touchmap feedback guides novice learners to expert -level (or better) performance in the cognitive psy chomotor task of palpating the entire breast. The real time touchmap feedback also resulted in improved performance in the psychomotor task of palpating with correct (deep) pressure. Participants in Study

PAGE 293

293 MRIPS -Learning significantly outperformed the Stu dy MRIPSx2 inexperienced participants. Study MRIPS Learning participants palpated 73.7% 16.9% of the breast with deep pressure, compared to the 61.1% 17.5% percent palpated by the MRIPSx2 .05. Study MRIPS Learning participants also outperformed MRIPSx2 experienced participants, who averaged 62.9% 19.4% percent of tissue palpated at deep pressure. This improvement trended towards significance at p = 0.11. Though touch-map feedback did n ot lead Study MRIPS Learnings novice participants to significantly outperform the more experienced historical control group, they did on average, and significantly outperformed the novice (approximately equally experienced) historical group. This result leads us to accept Hypothesis Pressure Feedback Impact. Receiving real -time feedback of the correctness of the learners palpation pressure causes learners to palpate more of the breast at correct deep pressure than when this feedback is not provided. 9.6 .5 Results: Pattern -of -Search Learning and Transfer In evaluating how closely participants followed the expert pattern in MRIPS, we analyzed only the most sensitive measure of deviation from the expert pattern, the normalizeddeviation measure of Equation 9 2. Examples of patterns giving low and high normalizeddeviation scores are shown in Figure 95. We expected to see the normalizeddeviation decrease throughout the three MRIPS interactions. Performance is shown in Table 99. Seven of eight participants decreased their normalized deviations from MRIPS #1 to MRIPS #2. This decrease is significant by a Wilcoxon test: Z = 2.2, p = 0.01. Of those improving, one participant switched from a spiral

PAGE 294

294 pattern to the vertical strip pattern; the other participa nts improved the correctness of their vertical strip patterns. From MRIPS #2 to MRIPS #3, total deviation increased for 5 participants and decreased for 2 participants. One participant did not have touch map or patternof search map data for MRIPS #3. On average, there was an increase that trended towards significance, by Wilcoxon test: Z = 1.7, p = 0.06. However, the increase from MRIPS #2 to MRIPS #3 was much smaller than the decrease from MRIPS #1 to MRIPS #2. The median change from MRIPS #1 to MRIP S #2 was a decrease of 5.7, while the median change from MRIPS #2 to MRIPS #3 was an increase of 1.9. The smaller increase from MRIPS #2 to MRIPS #3 following the larger decrease from MRIPS #1 to MRIPS #2 may indicate that some participants experienced a ceiling effect in MRIPS #2. Overall, from participants first MRIPS interaction to their last MRIPS interaction, the total deviation decreased for 7 participants and increased for 1 participant. This p = 0.03. Part icipants followed the experts patternof -search with significantly more precision after repeated practice with MRIPS incorporating patternof -search feedback, thus we accept Hypothesis Patternof -Search Learning. Though the patternof -search feedback posi tively impacted learners performances, we did observe that some participants had problems with the pattern map related to errors in the tracking of the infrared-reflective marker on the participants middle fingernail. One problem was occlusion of the tr acking marker. This most often

PAGE 295

295 occurred when participants began palpating with their fingertips instead of the correct method of palpating with the finger pads. We observed two participants that experienced this problem. Participants also occluded the t racking marker by learning over the mannequin too far and blocking the cameras view of the tracking marker. Participants were explicitly warned against this in the instructions before the interaction. Two other participants had this problem. These part icipants eventually figured out what was wrong, or asked the person supervising the study what was causing this problem. This issue did not impact the normalized or total deviation measures, as no segments were added to the pattern map while the marker wa s occluded. Participants were simply asked to start palpation over once the issue was fixed. The final problem was wearing of wristwatches or jewelry on the palpation hand. This occurred for two participants. This problem was more troubling as it caused errors in the pattern map. The watch face or jewelry reflected infrared light from the tracking camera which added noise to the calculated position of the tracking marker. Participants were instructed before the MRIPS interaction to remove any watches, jewelry, or reflective material from their hands, but these participants did not follow instructions. In our analysis of the normalized and total deviation measures, we removed segments which were obviously caused by this problem. These problems occurred for three participants in MRIPS #1, three participants in MRIPS #2, and for two participants in MRIPS #3. Solutions to these problems include having participants better follow instructions, increasing tracking infrastructure (e.g. a larger fiducial plac ed on the fingers that would not require the fingernail to face the camera), and adding more sophisticated tracking capabilities. Increasing tracking infrastructure is undesirable as it may negatively impact user

PAGE 296

296 acceptability of MRIPS. However, tracking could be enhanced to track multiple distinct infrared blobs which could be used to separate the marker position from noise caused by other infrared-reflective objects in the scene. To evaluate training transfer, we reviewed video of the SP interactions to evaluate what type of pattern of -search participants used: vertical -strip pattern, other systematic pattern (e.g. spiral), or no systematic pattern. In the SP Pre test, two of eight participants used a vertical strip pattern, two used a spiral pattern, and four used a non -systematic (no discernable pattern) method of examining the breast. In the SP Post test, seven of eight participants used a vertical strip pattern. One participant used a spiral pattern in SP Post -test. This participant also used the spiral pattern in SP P re -test and MRIPS #1, then chang ed to a vertical strip pattern for MRIPS #2 and MRIPS #3, but reverted to the spiral pattern in the SP Post test. The repetitive practice with the patternof -search feedback may not have been enough t o ingrain in this participant that she should use a vertical strip pattern. However, the other seven participants did learn to use a vertical strip pattern. The change from two of eight (25%) using a vertical strip in SP Pre-test to seven of eight (~88%) using a vertical strip in SP Post -test is significant by McNemars test, p = 0.03. Five of eight participants changed from a non -systematic or spiral pattern to a vertical strip pattern after repeated practice with MRIPS. All of these participants change d to a vertical strip pattern in MRIPS #1. The only information received by participants that instructed them to use a vertical strip pattern was the real time pattern of -search map feedback given in the MRIPS interactions. Given the significant improvem ent in the number of learners using vertical strip patterns and the evidence

PAGE 297

297 that this change is due to the feedback provided in MRIPS, we accept Hypothesis Pattern of -Search Transfer Repeated practice guided by the real -time feedback present in MRIPS caused participants to use the correct vertical strip pattern instead of the incorrect patterns with which they began the study. Participants retained this knowledge learned in MRIPS and applied it to the CBE of the SP in the SP Post -test. 9.6.6 Results: Im pact of Real -Time Feedback on Pattern -of -Search To evaluate whether the presence of the patternof -search feedback guided participants to perform the vertical strip patternof -search, we compared the number of participants using the vertical strip pattern in MRIPS #1 to the number of participants using the vertical strip pattern in the historical control group of Study MRIPSx2 (Chapter 4). The 12 participants who completed MRIPS #1 were included in this analysis along with 18 inexperienced and experienced students in Study MRIPSx2. Eleven students from Study MRIPSx2 were not included in this analysis because video showing their patterns of -search had not been recorded. From Study MRIPSx2, only students were included (not residents or clinicians) because t he students had learned CBE technique at the Medical College of Georgia, where the vertical strip pattern is taught. Clinicians and residents learned CBE elsewhere and were taught a wider variety of techniques. Ten of the twelve participants (83%) in MRIP S #1 used the vertical strip pattern, with two others using a systematic non vertical strip pattern (horizontal strip and spiral patterns). From the historical control, 6 of 18 participants used a vertical strip pattern (33%), with 6 of 18 using a systematic nonvertical strip pattern (horizontal strip, spiral, and spoke patterns) and 6 of 18 using a non-systematic pattern (i.e. no discernable pattern). A significantly larger proportion of Study MRIPS -Learning participants used a vertical strip pattern th an did the historical control group, by Fishers exact test at p <

PAGE 298

298 0.01. The addition of the patternof -search map which was designed to guide learners to use a vertical strip pattern led to a significantly higher percentage of learners to use a vertical strip pattern. From this result, we accept Hypothesis Patternof -Search Feedback Impact. We observed that two participants did not use a vertical strip pattern in MRIPS #1, despite receiving feedback that attempted to guide them to use a vertical strip pattern. Both of these participants used the same pattern they had previously used in the SP Pre -test. Only one of these participants completed MRIPS #2 and MRIPS #3; in these interactions, she used the vertical strip pattern. In MRIPS #1, neither partici pant attempted to follow the vertical strip pattern, so it appears unlikely that a high cognitive load made it too difficult for participants to follow the vertical strip pattern. As both participants used the same pattern as in the SP Pre-test, it seems plausible that they used these incorrect patterns out of familiarity not from using these patterns in previous CBEs, but from misinformation from friends, attending physicians, or outdated educational materials. It is this type of non-standardized learn ing that MRIPS has the potential to combat, through repeated practice with standardized, objective feedback. 9.6.7 Results: Finding Masses and False Positives Learning and Transfer Within the three MRIPS interactions, repeated practice with MRIPS appears t o have positively impacted both finding of real masses and finding of false positive masses, but neither improvement was significant (Table 910). In MRIPS #1, two participants found one mass each, while in MRIPS #2 and MRIPS #3 the number of participants finding masses increased to three. In MRIPS #3, one of these three participants found both masses. This improvement was not significant. The number of participants and number of false positives found decreased with each repetition, but not

PAGE 299

299 significantl y. This was evaluated by a Friedman test, X2(df = 2, n = 8) = 2.0, ns From these tests, we must reject Hypothesis Finding Masses Learning and Hypothesis False Positive Masses Learning. However, it is possible that with a larger population that these hypotheses would be accepted. This is because there is a relationship between finding masses and the amount of the breast palpated with deep pressure. Participants who found one or more real mass also palpated significantly more area with deep pressure than those participants who did not find a real mass. This difference trended towards significance -11) This indicates that if MRIPS was able to improve learners palpation at deep pressure, then MRIPS should improve learners ability to find masses. We expect that with a larger population, and perhaps with more repetitions, a majority of participants would consistently palpate a high percentage (e.g. >90%) of the breast with deep pressure. This would lead more participants to find the real masses. That we did not observe this significant improvement in finding masses is likely due to two participants performing very poorly in palpating with deep pressure (<40%) in MRIPS #3. With these two participants not even attempting to palpate with enough pressure to find masses, only six participants remained who had the potential to find the masses. Fifty percent of participants attempting to find masses did find masses; if the same percen tages held in a larger population, the increase from 25% (2 of 8) to 38% (3 of 8) to 50% (3 of 6) would have been significant.

PAGE 300

300 Although there is insufficient evidence to accept that participants became more adept at finding masses due to repeated exposure to MRIPS, participants performance in finding masses and finding false positives improved from SP Pre-test to SP Post test. In the SP Pre test, 4 of 8 participants found the real mass. All four also found the mass in the SP Post -test. Three additional p articipants did not find the mass in the SP Pre -test but did find the mass in the SP Post -test. The improvement from 50% of participants finding the mass to 87.5% of participants finding the mass trends towards significance by McNemars ( p = 0.13). With a population ~1.5 times larger, the same proportions would be significant. It is worth noting then that if all 12 participants completing the SP Pre-test are included, the figure of 50% finding the mass in SP Pretest still holds. However with only a trend towards significance, we will reject Hypothesis Finding Masses Transfer, but expect that a larger population would provide sufficient evidence to accept this hypothesis. Though participants did not improve significantly in finding the real mass, partici pants improved significantly in distinguishing between normal tissue and masses, as evidenced by a significant reduction in false positives found in the SP breast. In the SP Pre -test, five false positives were found by five participants (mean 1.0 1.1). This number decreased to three false positives found by two participants in SP Post test p = 0.03. From this evidence, we accept Hypothesis False Positive Masses T ransfer. Although we accepted only one of the four related hypotheses, practice with MRIPS does appear beneficial to the cognitive-psychomotor task of determining whether breast tissue is normal or a mass. The small population completing the study

PAGE 301

301 did not provide the power to find a significant improvement in most of the tests, but incremental improvement was observed with repeated practice in MRIPS. Overall, fewer participants found the masses in the MRIPS breast than in the SP breast. This indicates t hat the masses in the MRIPS breast were more difficult to palpate. This was expected of the soft mass, but expert clinicians rated the hard mass as being equally difficult to find as the SP mass. It appears that this was not the case for the novices who made up the study population. Although the majority of participants did not find a mass in MRIPS, we believe that the repeated practice with MRIPS resulted in the improvement from SP Pre-test to SP Post -test. Finding masses is a task in which performance does not improve without repetition and feedback [ 39 ]. These were provided by MRIPS with a frequency and precision otherwise unavailable in traditional curricula. 9.6.8 Discussion Overall, participants performance in the psychomotor and cognitivepsych omotor components of the CBE improved with repeated practice with MRIPS. From SP Pre test to SP Post -test, participants improved significantly in coverage, use of deep pressure, following the expert patternof -search, and not finding false positive masses Participants also trended towards a significant improvement in finding real masses. Although the result of practice with MRIPS appears to be significant improvement in a CBE of an SP, ceiling effects and a limited population size made it difficult to evaluate the veracity of hypotheses of learning within MRIPS. The real time feedback appears to have contributed significantly in improving participants skills in these tasks, as participants in Study MRIPS Learning significantly outperformed participants in Study MRIPSx2 (who did not receive real -time feedback).

PAGE 302

302 Although the real -time feedback led to improved performance, it is likely the cause of the ceiling effects that prevented us from making more solid conclusions of learning. Repeated practice with MRIPS appears to benefit learners performance in psychomotor and cognitivepsychomotor components of the CBE. However, learners appear to need more than three repetitions to learn some of the psychomotor tasks. After three opportunities to practice pal pation at deep pressure, a large amount of variability in participants scores existed. Three participants palpated >90% and another at 88%, but the other four participants palpated <80% of the breast with deep pressure. This contrasts with the cognitive tasks in which the variance among participants scores decreased with each MRIPS interaction. 9. 7 Affective Performance Participants affective performance is primarily concerned with attending to the patients emotional state: displaying empathy and keeping the patient comfortable throughout the exam. In MRIPS, participants received feedback of their affective performances through the real -time thought bubble feedback and post experiential feedback which listed the number of successfully addressed, unsuccessfully addressed, and missed opportunities for comforting and expressing empathy. In the SP interactions, participants feedback concerning their affective performance came solely from verbal and nonverbal communication from the SP. 9. 7 .1 Measures In MRIPS, speech input was parsed to perform an objective evaluation of empathic content of participant responses to eight critical moments. We had intended for medical educators to provide video review of participant responses to these critical moments, but educators were unable to do so. Instead, participant handling of the

PAGE 303

303 critical moments was rated objectively, assigning 0, 1, or 2 points to participant responses, based on a gross approximation of empathic content. In general, 2 points were assigned to responses which acknowledged the fear, concern, or emotional content of the patients statement; 1 point was assigned to responses which provided matter of -fact related information; for all critical moments, 0 points were given to ignoring the participant or providing an unrelated response. The critical moments and specific scoring schemes are listed here: CM1 : Before the physical exam, the patient asks in a fearful voice: Do you think my pain could be because I have cancer? o Scoring 2 points: acknowledgi ng the patients concern, e.g. from your mother passing away, Im sure that must be a concern for you and I understand it must have been hard for you when you found the pain. o Scoring 1 point: stating that a physical exam and tests are needed to determine if the pain is due to cancer, e.g. its hard to say; are you willing to run some more tests to find out? CM2 : As the participant begins palpation, the patient expresses fear, that the exam will find cancer, in the form of a thought bubble: This is so s cary, what if they find cancer? o Further interaction: If the participant responds by talking down to the patient, e.g. theres no need to be scared, the participant responds with the thought: Why shouldnt I be scared? Of course Im scared, what if it s cancer? o Further interaction: If the participant ignores the expression of fear, the patient responds with the thought Maybe they only care about doing the exam, not about how I feel. o Further interaction: If the participant expresses understanding, e.g. I know this is hard for you because you lost your mother to breast cancer, the participant responds with the thought: I guess they do care about me, that makes me feel better. o Scoring 2 points: acknowledging the patients fear or lack of comfort, e.g I understand this is uncomfortable. o Scoring 1 point: explaining the exam, e.g. Im going to start by palpating along your collarbone. No response received this score. CM3 : After the physical exam is complete, the patient asks Well, do you think it c ould be cancer? o Scoring 2 points: acknowledging the patients concern, e.g. I understand your concern about cancer with it running in your family. Soon we will be able to gather more information and tell you more certainly.

PAGE 304

304 o Scoring 1 point: stating that more testing was required to determine if it could be cancer, e.g. Its hard to say thats why we need to run a mammogram, to make sure its not cancer. CM4 : When asked about her family history of cancer, the patient responds I lost my mother to breas t cancer two years ago. I miss her everyday. o Further interaction: If the participant ignores this opportunity for empathy, the patient thinks I guess they just care about doing the exam. o Further interaction: If the participants response is ontopic but does not express understanding or sympathy, e.g. How old was your mother when she died? the patient thinks This is my mother were talking about; this person doesnt care at all. o Further interaction: If the participant expresses empathy (understandin g, e.g. I understand that must make you anxious about your breast pain. Have you been able to talk to anyone about how your mothers passing is affecting you?), or expresses sympathy (e.g. Im sorry to hear about your mother), the patient responds wit h the thought I guess they do care about me, that makes me feel better and the speech thank you, doctor. o Scoring 2 points: expressing empathy (understanding) or sympathy, e.g. I know its hard to come in for something that your mother had as well and, not that you have breast cancer, but its a good decision to come see us or I am so sorry to hear that, it must be tough. o Scoring 1 point: asking more about her mothers cancer, e.g. how old was your mother when she passed away? CM5 : When instructed that she should have a mammogram done, the patient expresses fear of mammograms: I dont know, I mean dont mammograms hurt? o Scoring 2 points: acknowledging that the mammogram would be uncomfortable and/or reassuring the patient that she will be well tak en care of, e.g. Ive heard they are uncomfortable but I actually dont know if they are. The nurses there will do the best job they can to make it comfortable for you. o Scoring 1 point: stated that mammograms are important to diagnosis, e.g. Well, we may need to do it just to make sure we rule out anything more serious. CM6 : When further instructed that a mammogram is needed to ensure she does not have a malignancy, the patient expresses further fear of mammograms: Do I really have to get a mammogram ? I mean, my mom was fine, then she had a mammogram and all of the sudden she was really sick. o Scoring 2 points: acknowledging the patients fear, e.g. I know its scary for you, but we need to do it so that we can figure out whats going on with your br east pain. o Scoring 1 point: stating that the mammogram is important or referencing the loss of the patients mother as a motivating factor to get a mammogram, e.g. mammograms are important diagnostic tools, or she probably had cancer before the mammogr am; the mammogram simply found a cancer. Its important to have a mammogram to detect it early so you can treat it.

PAGE 305

305 CM7 : When asked to disrobe for visual inspection, the patient expresses anxiety over disrobing, stating I guess thats ok Im kind of sh y about taking off my clothes. o Scoring 2 points: acknowledged the patients discomfort, e.g. I understand. Please let me know at any time if you feel uncomfortable, OK? o Scoring 1 point: stating why visual inspection should be performed, e.g. right now Im looking for any asymmetry or any redness or swelling. CM8 : When the patient was asked to raise her arms above her head for visual inspection, the patient thought: This is so awkward; I dont remember doing this for my last doctor. o Scoring 2 points: acknowledging the patients discomfort. No response received this score. o Scoring 1 point: explaining why the patient needed to raise her arms over her head, e.g. the reason Im doing this is to make sure there is no mass in your armpit. Thought bubble feedback was also incorporated into virtual human responses to sexual history questions, but no participants queried sexual history, so we were not able to analyze that use of thought bubble feedback. In evaluating learning, performance in the eight critic al moments was condensed down to two measures. The first measure is the percent of moments scored a 2 (i.e. displaying empathy or acknowledging the patients emotions). The second measure is a normalized score of total performance, which is calculated by Equation 9 -3, where nx is the number of critical moments receiving a score of x 0 1 2 1 2* 2 n n n n n Normscore (9 -3) For the SP interactions, affective performance was rated by medical experts and novices who reviewed video of participants exams. Affec tive performance was assessed by three measures using the instrument of Appendix D.4. These measures were an overall rating of empathy consisting of eight items scored on a 4point forcedranking scale ( strongly disagree, disagree, agree, strongly agree). The other two measures concern the participants responses to four critical moments in which the

PAGE 306

306 virtual human patient expressed concern or prompted for empathy. Participants were rated on the empathic content and appropriateness of their responses. Th e critical moments rated were: SP CM1 : I lost my mother to breast cancer two years ago. I miss her everyday. SP CM2 : Do you think my pain could be because I have cancer. (in a fearful voice, before physical exam) SP CM3 : Well, do you think it could be cancer? (in a fearful voice, after physical exam) SP CM4 : Do I really have to get a mammogram? My mom was fine then she had a mammogram and then all of the sudden she was really sick? 9. 7 .2 Hypotheses Hypothesis Empathy Learning: Participants demonstration of understanding and attentiveness to the patients emotions will improve with repeated MRIPS interactions. o Null hypothesis : Affective performance scores will not significantly increase from MRIPS #1 to MRIPS #3. Hypothesis Empathy Transfer : From SP Pre test to SP Post -test, participants will increase the appropriateness and empathic content of their responses to critical moments. o Null hypothesis : Video reviewers ratings of the appropriateness and empathic content of participants critical moment responses will not significantly increase from SP Pre test to SP Post -test. 9. 7.3 Results: Empathy Learning Performance is given for the three MRIPS -CBE interactions in Table 9-12, individual participant data is shown in Appendix D.6. From MRIPS #1 to MR IPS #2, four participants increased in the percent of moments rated a 2, three participants decreased, and one participant did not change. This was not a significant change by Wilcoxon test: Z = 0.74, p = 0.41. From MRIPS #2 to MRIPS #3, four participants improved, one regressed, and three did not change their performance. This trended towards significant improvement by Wilcoxon test: Z = 1.8, p = 0.063. Overall, from MRIPS #1 to MRIPS #3, four participants improved, two regressed, and two did not

PAGE 307

307 cha nge; this was not a significant change in performance, by Wilcoxon test: Z = 0.3, p = 0.4. In the normalized score, six participants improved from MRIPS #1 to MRIPS #2, one regressed and one did not change. This improvement trended towards significance, b y Wilcoxon test: Z = 1.4, p = 0.10. From MRIPS #2 to MRIPS #3, six participants improved and two participants regressed. This improvement was not significant by Wilcoxon test: Z = 0.84, p = 0.22. Overall, from MRIPS #1 to MRIPS #3, seven participants im proved and one did not change. This improvement was significant p = 0.008. The normalized score is a more sensitive measure, though it also scores affective performance more liberally, i.e. some responses receiving a score of 1 were not empathetic but were more appropriate than ignoring the patient in the critical moment. However, due to the difficulty in evaluating affective performance (earlier difficulties in getting medical education experts to agree in ratin gs of affective performance are described in Sections 4.2 and 4.3), and the small population size, we accept Hypothesis Empathy Learning on the basis of the significant improvement in the normalized score from MRIPS #1 to MRIPS #3. 9. 7 4 Results: Impact of Feedback Affective performance in MRIPS interactions prior to the incorporation of the thought bubble feedback was evaluated using different measures than affective performance in MRIPS in Study MRIPS-Learning. Thus we can not determine the impact of re al -time feedback of affective performance by directly comparing performance with feedback and performance without feedback. Instead, we look at how participants reacted to the thought bubble feedback in Study MRIPS Learning.

PAGE 308

308 There were two critical momen ts (CM) in which virtual human communication consisted solely of thought bubbles: CM8 (awkwardness of visual inspection) and CM2 (fear that exam will find cancer). CM8 used a thought bubble to prompt a participant response. In CM8, only 3 of 7 participan ts (one did not do visual inspection and did not encounter CM8) reacted to the feedback (one participant in MRIPS #2 and two in MRIPS #3). The other four participants ignored the patients discomfort with visual inspection. The three who responded did so to explain the procedure, but did not acknowledge that the patient was uncomfortable with the procedure. In this instance, the thought bubble feedback appears ineffective at communicating the patients discomfort to the learner. CM2 used a thought bubbl e to prompt participant response and a subsequent thought bubble to communicate to the participant whether his response (or lack of response) was appropriate. Only two participants responded to the prompt (one in MRIPS #1 and one in MRIPS #3). The one participant responding to the initial prompt in MRIPS #3 was the only participant who may have been motivated by negative thought bubble feedback (maybe they only care about the exam) to respond to the prompt in a subsequent interaction. With no participants responding to the prompt in two successive interactions, we can not determine whether positive feedback (that makes me feel a bit better) reinforced in the learner to continue to provide empathic responses. Though we observed improvement in affecti ve performance in MRIPS, we do not have any evidence that suggests the thought bubble feedback is directly responsible for improvement in affective performance. However, affective performance improved

PAGE 309

309 significantly from MRIPS #1 to MRIPS #3 and from SP Pr e test to SP Post -test. This improvement may be due to post experiential feedback (listing of successful and missed opportunities for empathy and video of expert reacting to three of the critical moments). 9.7.5 Results: Empathy Transfer Participants improved their use of empathy significantly from SP Pretest to SP Post -test. Results are shown in Table 912 with full data in Appendix D.5. Participants were rated on the empathic content and appropriateness of their responses to the four critical moments listed in Section 9.7.1. We conducted a reliability analysis to determine if empathy and acceptability on each of the four critical moments could be averaged into unified empathy and acceptability scores that represented the overall empathy and acceptabi lity of the participants responses to the critical moments. Cronbachs alpha was calculated for empathy and acceptability measures in each SP 0.7): SP Pre-test emp participant, the expert scores of the four critical moments were averaged. These scores are given in Table 9-12. Participants improved significantly in their use of empathy when responding to critical moments. Only one of eight participants was rated positively on his use of empathy in response to critical moments in the SP Pre -test. All eight participants were rated positively on their use of empathy in response to critical moments in the SP Post test. The improvement from a rating of 2.76 0.65 to 3.95 0.005. The appropriateness of responses to critical moments also improved significantly, from 1 of 8 participants to 8 of 8 participants, with average rating improving

PAGE 310

310 from 2.89 0.56 to 4.02 All participants improved in the empathic content and appropriateness of their handling of the patients concerns expressed in the critical moments. Participants were also given an overall rating of empathy which took into account how the participant trea ted the patients emotions, the amount of concern the participant expressed for the patient, and the appropriateness of the participants nonverbal behavior. Participants improved significantly on this measure, with only 1 of 8 participants receiving a p ositive rating in the SP Pre test and 6 of 8 participants receiving positive ratings in the SP Post test. One of the two participants that did not receive an overall positive rating in the SP Post -test had a score of 2.9, just short of the minimum positiv e score of 3.0; this participant received positive scores from all reviewers except for one reviewer. The other participant not receiving a positive score in the SP Post test had the same overall score in SP Pre-test and SP Post -test. This participant re ceived overall positive scores in the critical moment measure, but was the only participant to receive any negative scores from individual reviewers in the critical moment ratings of the SP Post -test. It appears certain that 7 of the 8 participants improv ed their use of empathy and their handling of patient concerns as a result of repetitive practice with MRIPS. The overall empathy score improved significantly from SP Pre -test ( 2.58 0.39) to SP Post -test (3.11 we accept Hypothesis Empathy Transfer. 9. 7.6 Discussion Affective performance improved from repeated practice with MRIPS CBE and this improvement transferred to the CBE of an SP. However, it is not clear what role the real -time and post experiential feedb ack played in improving affective performance.

PAGE 311

311 Due to the inability of medical educators to provide video review of performance in the MRIPS -CBE interactions, we were not able to directly compare affective performance to control groups. Further investiga tion should focus on evaluating the efficacy of the real time feedback of affective performance. 9. 8 Validity of Results To evaluate the validity of the results of Study MRIPS Learning, we investigated the impact of a single MRIPS -CBE interaction on perfor mance in a CBE of an SP and the impact of an SP interaction on subsequent CBEs of SPs (a description of the evaluation was given in Section 9.2.3). 9.8.1 Impact of Multiple MRIPS Practice Opportunities To determine the impact of a single MRIPS CBE interac tion, we compared performance of participants receiving a MRIPS -CBE practice opportunity before an evaluation in a CBE of an SP, and participants who did not receive an MRIPS CBE practice opportunity before a CBE of an SP. Cognitive performance was evalu ated as the completeness of the breast history and completeness of visual inspection. Psychomotor performance was evaluated as completeness of palpation (coverage), use of correct palpation pressure, palpation of ancillary areas of tissue (supraclavicular infraclavicular, and axilla), and use of a vertical strip pattern of search. Affective performance was evaluated using a four item instrument assessing empathy and appropriateness of verbal and nonverbal communication. Instruments and data are given in Appendix D. 7 Results are shown in Table 9 13. Participants that practiced with MRIPS -CBE performed significantly better on only the three psychomotor tasks. Real time feedback was provided for two of these

PAGE 312

312 tasks. The Study MRIPS Learning participants performed better in the SP Post test than the pilot study participants did in the SP interaction on all metrics other than coverage. However, the MRIPS Learning participants only performed significantly better on the affective and cognitive tasks (Figure 9 -14). That three MRIPS -CBE practice opportunities did not result in significantly better psychomotor performance than one MRIPS -CBE practice opportunity is likely due to the real time feedback available for the coverage and palpation pressure tasks, as well as the small population size. Considering cognitive, psychomotor, and affective skill sets, there is clearly additional benefit to multiple practice opportunities incorporating real -time feedback of all three skill sets. However, this experiment doe s not tell us what number of repetitions will result in diminishing returns (when all participants performance plateaus). 9.8.2 Impact of an SP Pre test Interaction on Subsequent SP Performance To ensure that improvement from SP Pre-test to SP Post -test is not due solely to the practice afforded by the SP Pre-test, we compared performance in CBE of five medical students who performed two SP exams one month apart. For this group, the cognitive task of visual inspection, and the psychomotor tasks of cover age, palpating ancillary areas, using correct pressure, and using the vertical strip patternof -search were analyzed. For visual inspection, four participants had the patient assume two or three visual inspection poses in their first SP interaction and de creased to doing no visual inspection in their second SP interaction; one participant used one pose in each SP interaction. The proportion of participants doing a visual inspection trended towards significant decrease from first to second SP interaction, by McNemars test: p = 0.063.

PAGE 313

313 For coverage, two participants coverage decreased and three participants coverage stayed the same. The average coverage score decreased from 4.8 1.1 to 3.6 2.2, which was not significant by Wilcoxon test, Z = 1.3, p = 0.25. In the psychomotor task of palpating with correct pressure, two participants score decreased, one increased, and two remained the same. The average score decreased from 3.4 1.3 to 3.0 1.4, which was not significant, by Wilcoxon test, Z = 0.3, p = 0.5. Five of five participants palpated at least one ancillary area in their first SP interaction; this decreased to 2 of 5 participants in the second SP interaction. This decrease was not significant by McNemars test, p = 0.13. For the patternof -search, two participants did not use vertical strip in either SP interaction; two used vertical strip in the first SP interaction and did not use vertical strip in the second SP interaction; and one used vertical strip in both interaction. This decrease in performance was not significant by McNemars test. All aspects of psychomotor performance decreased over the one month interval between first and second SP interactions. This is an approximately equal amount of time as between SP Pre-test and SP Post -t est in Study MRIPS Learning. If improvement in MRIPS -Learning from SP Pre test to SP Post -test was due solely to the SP Pre -test experience, we would expect a similar improvement in this pilot study. Instead, we observed decreases in performance, indicat ing that improvement observed in Study MRIPS Learning must be due to the three practice opportunities with MRIPS CBE. 9.9 Study Limitations The limitations of the study lie primarily in the small population. The population size was limited by the availabi lity of medical student volunteers and SPs, as well as

PAGE 314

314 Medical College of Georgia staff to administer the study. Many results trended towards significance and would likely be statistically significant in a larger population. Running a longer -duration study (~1 month from consent to completion) such as Study MRIPS Learning with a large population of medical students encounters a catch -22: to obtain a large population, integration into curriculum is necessary; however, course integration requires that l earning has previously been demonstrated, with such a study. Fortuitously, on the basis of Study MRIPS -Learning results, the Medical College of Georgia plans to incorporate multiple MRIPS -CBE interactions into their womens health clerkship starting in AprilMay 2010. This will provide data for >100 students which will provide future insight into the efficacy of MRIPS for learning real world interpersonal scenarios. Additionally, it would be beneficial to compare learning with the current study design to learning in a sequence of five SP interactions. However, our medical collaborators express that practice with five SP interactions is unlikely in a medical curriculum and has prohibitive logistic and monetary costs. Although this would be an ideal contro l group for Study MRIPS Learning, because we have previously validated MRIPS -CBE as a substitute for CBE of an SP, we should expect statistically similar impacts on performance from the Study MRIPS Learning procedure and the ideal 5-SP procedure. 9.10 Revisiting Meta Hypotheses We restate the goals of Study MRIPS Learning and corresponding meta hypothesis and evaluate their veracity: Determine what learning (cognitive, psychomotor, affective) occurs in users of MRIPS -CBE.

PAGE 315

315 o Meta hypothesis: Participants will improve in cognitive, psychomotor, and affective performance throughout repetitive practice with MRIPS -CBE. Performance in these tasks will significantly improve from the first MRIPS interaction to the third MRIPS interaction. o Accepted. Performance in psychomotor, cognitive, and affective tasks increased significantly with repeated practice in MRIPS CBE. Specifically, performance improved significantly in breast history taking, visual inspection completeness, correctness of patternof -search, and affec tive tasks. Determine whether improvement in skills within MRIPS CBE transfers to the real world, in the form of improvement in performance in CBE of human patients. o Meta hypothesis: After practice with MRIPS -CBE, participants cognitive, psychomotor, and affective performance in CBE of an SP will have significantly improved in relation to baseline levels taken before practice with MRIPS -CBE. o Accepted. Performance in psychomotor, cognitive, and affective tasks increased significantly from SP Pre-test to SP Post test. Specifically, performance improved significantly on breast history completeness, visual inspection completeness, coverage, correctness of palpation pressure, correctness of patternof -search, not finding false positive breast masses, and appropriate use of empathy. Determine whether the presence of real -time feedback causes learners to significantly outperform past users of MRIPS -CBE (without real -time feedback) in cognitive, psychomotor, and affective tasks. o Meta hypothesis: Participants in St udy MRIPS -Learning will perform significantly better in cognitive, psychomotor, and affective tasks (for which real -time feedback was provided) than prior users of MRIPS -CBE (participants in Study MRIPSx2, Section 4.3) who did not receive real -time feedbac k. o Accepted for cognitive and psychomotor tasks. Real time feedback resulted in improved performance in psychomotor, cognitive, and affective performance. Specifically, performance improved in the presence of real time feedback for the tasks of breast hi story completeness, visual inspection completeness, coverage, correctness of palpation pressure, and correctness of pattern of -search. The lack of a direct comparison to a control group left us unable to completely evaluate the impact of the thought bubbl e feedback. However, from within-subjects comparisons, the thought bubble feedback does not appear to be effective in improving affective performance. Yet, affective performance did improve; this may be due to the post experiential affective feedback.

PAGE 316

316 If learning occurs, determine whether skill sets are learned concurrently; or, whether one skill set must be maximized before performance in other skill sets can improve. o Meta hypothesis: We will not observe the following: performance in a single skill set r equires maximization before the other skill sets can improve. o Accepted. With the small population size and small number of repetitions, we provide an observational analysis. Trends in individual participants performance in the three skill sets are visualized in Table 9 16. Four participants clearly demonstrated concurrent learning of all three skill sets. Three participants demonstrated concurrent learning of two skill sets at a time. Two of these participants experienced no change in performance of t he third skill set at all during the MRIPS interactions. This skill set was the cognitive skill set and the lack of change was due to ceiling effect in the breast history taking completeness. So, these participants also could be considered to learn all t hree skill sets concurrently; one skill set was simply maximized in the first practice opportunity. The final participant demonstrated concurrent learning of two skill sets, however this participant could be considered to have not learned concurrently, be cause he maximized affective performance before improving either cognitive or psychomotor performance. Overall, these observations indicate that MRIPS -CBE does afford concurrent learning of all three skill sets. This is made possible by the mix of haptic s, patient interaction, and feedback that puts MRIPS at an advantage over other simulation approaches. From Study MRIPS Learning, we conclude that concurrent learning of cognitive, psychomotor, and affective skills takes place with repeated practice in MRI PSCBE, and this improvement transfers to the real world task of CBE of an SP. Real -time feedback plays a significant role in learning cognitive and psychomotor tasks of the breast exam, however, the role of feedback in affective learning is unclear. Though the study population was small, this study represents an initial step in demonstrating the efficacy of interpersonal simulation for educating real world interpersonal scenarios.

PAGE 317

317

PAGE 318

318 Table 9 1. Instrument used to evaluate the completeness of breast his tory taking in MRIPS and SP interactions. Specific to the MRIPS interactions, the participants asking of shaded items was reviewed only in post experiential feedback. Non-shaded items were visible during the CBE as part of the real time procedural check list feedback, and also reviewed in the post experiential feedback. Breast history completeness instrument Chief complaint History of present illness: Location of pain History of present illness: Description of pain History of present illness: Nipple d ischarge History of present illness: Other changes in breast (e.g. redness) History of present illness: Trauma to breast History of present illness: Pain stationary or radiating History of present illness: Can the pain be made better (e.g. by medicatio n) Medical history: Age Medical history: Onset of menarche (age of first menstrual period) Medical history: Still having periods / are periods regular Medical history: Use of hormones or birth control Medical history: Prior breast problems Medical hi story: Yearly clinical breast examination? Medical history: Monthly self breast examination? Medical history: Prior mammograms? Medical history: Pregnancies Medical history: Hospitalizations Medical history: Surgeries Medical history: Current medica tions Medical history: Other health problems Family History: History of cancer Family History: Other medical problems Social History: Smoker / tobacco use Social History: Alcohol use Social History: Sexually active? Social History: Employment / on t he job health risks Table 9 2. Summary of results for hypotheses relating to cognitive performance. Measure Hypothesis Result Breast history completeness Learning and training transfer Accepted Breast history completeness Improvement from feedback Acce pted Visual inspection completeness Learning and training transfer Accepted

PAGE 319

319 Table 9 3 Performance in breast history taking in Study MRIPS Learning. Interaction Mean Std. dev. Range of scores SP Pre test 12.0 3.0 [ 8, 17] MRIPS #1 20.3 3.1 [15 24] MRIPS #2 20.1 1.7 [18, 23] MRIPS #3 21.3 0.9 [20, 22] SP Post test 18.3 3.9 [12, 25] Table 9 4 Changes in the number of participants asking about specific risk factors. (*) Denotes that the increase is significant at < 0.05 by McNemars test, while ( ) indicates that the increase trends to significance with p = 0.06. Item SP Pre test SP Post test Importance [ 151 ] Onset of menarche 0 of 8 5 of 8 Risk factor Hormone / birth control use 1 of 8 7 of 8 Hormone use i s a risk factor. Birth control can produce benign cysts which could cause the patients pain Alcohol use 2 of 8 5 of 8 Risk factor with daily use Job risks 1 of 8 5 of 8 Trauma may have caused the pain; exposure to environmental or chemical hazards inc reases risk Prior breast problems 1 of 8 5 of 8 Risk factor Patient age 3 of 8 7 of 8 Risk factor. Guides other questions e.g. concerning menopause and screening mammograms Table 9 5 Number of participants performing any visual inspection and complete visual inspections in each interaction. Interaction Performed Visual Inspection Complete Visual Inspection SP Pre test 2 of 8 0 of 8 MRIPS #1 6 of 8 5 of 8 MRIPS #2 7 of 8 4 of 8 MRIPS #3 7 of 8 5 of 8 SP Post test 6 of 8 5 of 8

PAGE 320

320 Table 9 6. Ins trument used to evaluate coverage and use of correct pressure in the SP interactions. Coverage items were mutually exclusive (scores ranged from 0 to 6) and pressure items were cumulative (scores ranged from 0 to 4). Coverage items Thoroughly examined en tire chest (6pts) Thoroughly examined cone only (4 pts) Did not examine cone thoroughly (0 pts) Pressure items Used correct light pressure (1pt) Used correct medium pressure (1pt) Used correct deep pressure (2pts) Table 9 7. Summary of acceptance and rejection of hypotheses of psychomotor and cognitive-psychomotor task performance. Task Hypothesis Result Coverage Learning For the entire breast, rejected (can not determine due to ceiling effect). Significant improvement in palpating supraclavicular, axilla, and infraclavicular areas. Training transfer Accepted Improvement from feedback Accepted Pressure Learning Rejected (can not determine due to ceiling effect) Training transfer Accepted Improvement from feedback Accepted Pattern of sear ch Learning Accepted Training transfer Accepted Improvement from feedback Accepted Finding masses Learning Rejected (improved but not significantly) Training transfer Rejected (trends towards significant improvement) Reducing false positive masses Learning Rejected (improved but not significantly) Training transfer Accepted

PAGE 321

321 Table 9 8 Coverage and use of deep pressure of the Study MRIPS Learning participants in the MRIPS #1 interaction, compared to two historical control groups (from Study MRIP Sx2 in Section 4.3) not receiving real -time feedback: inexperienced medical students (0 to 5 prior CBEs) and experienced medical students, residents, and clinicians (6 to >1000 prior CBEs). (*) Denotes results significant at < 0.005 and (**) denotes results significant at < 0.005. Coverage (percent area at light or higher pressure) Pressure (percent area at deep or higher pressure) Mean Std ev. (%) 95% CI Mean Stdev. (%) 95% CI MRIPS #1 ( n = 12) 89.9 11.6 [82.6, 97.3] 73.7 16.9 [62.9, 84.5] MRIPSx2 Inexperienced ( n = 33) 75.3 12.4 [70.9, 79.6] 61.1 17.5 [54.9, 67.3] MRIPSx2 Experienced ( n = 24) 80.9 9.0 [77.1, 84.7] 62.9 19.4 [54.7, 71.1] MRIPS #1 vs. Inexperienced t(43) = 3.6 p = 0.001** t(43) = 2.2 p = 0.04* MRIPS #1 vs. Experienced t(34) = 2.6 p = 0.014* t(34) = 1.6 p = 0.11 Table 9 9 Total deviation from expert patternof -search in the three MRIPS interactions of Study MRIPS Learning. Interaction Mean Std. dev. 95% CI MRIPS #1 22.6 7.8 [17.2, 28.0] MRIPS #2 16.8 5.2 [13.2, 20.4] MRIPS #3 19.3 3.2 [17.1, 21.5] Table 9 10 Number of participants finding real masses and false positive masses in MRIPS. MRIPS #1 MRIPS #2 MRIPS #3 Masses found 1 2 1 2 1 2 Participants 2 0 3 0 2 1 Participants Masses Participants Masses Participants Masses False positives 5 6 4 4 3 3

PAGE 322

322 Table 9 11 Participants finding masses palpated a larger percentage of the MRIPS breast with deep pressure. The difference is significant in MRIPS #2 and MRIP S #3. MRIPS #1 MRIPS #2 MRIPS #3 n % area n % area n % area Found 1+ masses 4 80.7 17.0 3 85.9 5.0 3 85.4 7.7 Found no masses 8 70.2 17.0 5 61.3 13.4 5 55.7 24.8 Mann Whitney test Z = 1.2, p = 0.14 Z = 2.0, p = 0.029 Z = 1.94, p = 0.036 Table 9 12. Affective performance in MRIPS -CBE interactions. MRIPS #1 MRIPS #2 MRIPS #3 Outcome Percent of moments scored 2 23.4% 16.2% 20.8% 7.5% 26.0% 8.6% Increased, not significantly: Wilcoxon, Z = 0.3, p = 0.4 Normalized score 0.59 0.34 0.77 0.26 0.84 0.21 Increased, significantly: Wilcoxon, Z = 2.4, p = 0.008 Table 9 13. Expert ratings of participants affective performance in the SP interactions. (*) indicates significance at < 0.05; () indicates significance at Meas ure SP Pre test SP Post test Wilcoxon test Overall empathy 2.58 0.39 3.11 0.38 Z = 2.24, p = 0.012 Critical moment empathy 2.76 0.65 3.95 0.51 Z = 2.52, p = 0.004 Critical moment appropriateness 2.89 0.56 4.02 0.45 Z = 2.52, p = 0.004

PAGE 323

323 Table 9 14. Performance in pilot study used to assess impact of a single MRIPS -CBE interaction on a subsequent CBE of an SP. Significant difference at < 0.05 is indicated by a (*) and trend towards significance is indicated by (**). Measure MRIPS + SP ( n = 11 ) SP Only ( n = 8) Significance Breast history 4.82 1.2 (max score 10 items) 4.63 2.0 Mann Whitney: Z = 0.55, p = 0.30, ns Performed any vis ual inspection 6 of 11 participants 5 of 8 participants Fishers exact test, ns Coverage* 4.73 1.0 (0 to 6 scale) 2.50 2.1 Mann-Whitney, Z = 2.5, p = 0.01 Palpation pressure** 3.18 1.4 (1 to 4 scale) 2.0 1.8 Mann Whitney, Z = 1.6, p = 0.065 Pa lpated one or more ancillary area* 8 of 11 participants 1 of 8 participants Fishers exact test, p = 0.02 Used vertical strip 6 of 11 participants 3 of 8 participants Fishers exact test, ns Affective performance 3.06 0.37 (max score 5.0; greater tha n 3.0 positive) 3.14 0.50 Mann Whitney, Z = 0.5, p = 0.33, ns Table 9 15. Performance in a CBE of an SP after three MRIPS -CBE practice opportunities and after one MRIPS -CBE practice opportunity. Significant difference at < 0.005 is indicated by a (*). Measure MRIPS Learning participants after 3 MRIPS interactions (SP Post test) Pilot study participants after 1 MRIPS interaction Significance Breast history* 7.86 1.2 (max score 10 items) 4.82 1.2 Mann-Whitney, Z = 3.5, p < 0.001 Visual inspec tion completeness (0 3) 2.0 1.4 1.18 1.33 Wilcoxon, Z = 1.3, p = 0.14, ns Coverage 4.0 2.6 (0 to 6 scale) 4.73 1.0 Wilcoxon, Z = 0.09, p= 0.49, ns Palpation pressure 3.75 0.71 (1 to 4 scale) 3.18 1.4 Wilcoxon, Z = 0.93, p = 0.17, ns Anc illary completeness 1.5 1.1 1.0 0.78 Wilcoxon, Z = 1.2, p= 0.13, ns Used vertical strip 7 of 8 participants 6 of 11 participants Fishers, p = 0.15, ns Affective performance* 3.69 0.44 (max score 5.0; positive greater than 3.0) 3.06 0.37 Wilcox on, Z = 2.7, p = 0.002

PAGE 324

324 Table 9 16. Concurrent improvement in the three skill sets. For each MRIPS interaction, participants were rated as improving (Up) in a skill set if they improved in >50% of the tasks of that skill set; improving in only 50% r esults in a rating of NC (no change). Participants 20, 33, 34, and 33 demonstrated concurrent learning of all three skill sets. Participants 21, 23, 27, and 37 demonstrated concurrent learning of two skill sets. An argument for lack of concurrent lear ning could be made for participant 37, as this participant maximized affective performance before improving cognitive and psychomotor performance. MRIPS #1 to MRIPS #2 MRIPS #2 to MRIPS #3 Concurrency ID Cog. Psych. Affective Cog. Pysch. Affective 20 Up Up Up Up NC NC All three 21 Up Dn Up NC Up Up Two at a time 22 Up Up Up Up Up Up All three 23 NC Dn Dn NC Up Up Two at a time 24 Up Up Up Up Dn Up All three 27 NC Dn NC NC Up Up Two at a time 33 Up Up Up NC Dn Up All three 37 NC Dn Up Up Up NC N o

PAGE 325

325 Figure 91. Procedure for Study MRIPS -Learning.

PAGE 326

326 Participant ID37 33 27 24 23 22 21 20 Items queried25 20 15 10 5 0 SP Post-test MRIPS #3 MRIPS #2 MRIPS #1 SP Pre-test Figure 92. Participants performance in breast history completeness in the two SP and three MRIPS interactions.

PAGE 327

327 Percent of Items Asked, by Feedback Type 47.6 96.4 73.2 16.7 43.8 89.3 91.1 16.7 25 33.3 0 20 40 60 80 100 SP Pre-test MRIPS #1 MRIPS #2 MRIPS #3 SP Post-test Real-time Items Post-Experiential Items Figure 93. Real -time feedback appears to be more effective than the post exper iential feedback, as the gap in percentageof -items queried grew with repetition.

PAGE 328

328 A B Figure 94. A) Participant drawing of correct locations of the two masses in the MRIPS breast. B) Participant drawing of correct location of the mass in the SP breast. ID37 33 27 24 23 22 21 20 31 77 74 34 88 91 88 54 92 51 70 84 82 81 51 76 66 63 62 95 66 82 42 Percent area palpated100% 80% 60% 40% 20% 0% MRIPS #3 MRIPS #2 MRIPS #1 Percent of breast tissue palpated with deep pressure Figure 95. Participants use of deep pressure in MRIPS.

PAGE 329

329 A B Figure 96 A) A pattern of search that closely followed the experts, receiving deviation scores of 72.1 (total) and 9.4 (normalized). B) A horizontal strip pattern did not closely follow the experts, receiving a total deviation score of 1270.6 and a normalized deviation score of 44.6. Note that the long nonoutlined arrow is not part of the recorded pattern map nor is it included in the deviation score. This arrow points from the last palpation position to where the participants hand left the tracked area (at the end of the exam).

PAGE 330

330 CHAPTER 10 SUMMARY AND FUTURE DIRECTIONS We have developed a new paradigm for interpersonal simulation with virtual humans, in which the users touch of the virtual human and the users manipulation of physical objects are actively sensed by the virtual human and become significant components of the interaction between human and virtual human. Applying this paradigm to scenarios in medical education, we developed two mixed reality interpersonal simulations to train medical students cognitive, psychomotor, and affective skills in clinical breast examination and neurological examination with abnormal findings We also demonstrated the importance of i ncorporating real -time feedback of user performance into interpersonal simulation. 10.1 Review of Results In this dissertation we claimed that: Interpersonal simulation incorporating instrumented haptic interfaces and providing real -time evaluation and feedback of performance improves users scenariospecific psychomotor, cognitive, and affective skills. Skills improvement transfers to the real world interpersonal scenarios being simulated, demonstrated as improved performance in the real world interpersonal scenario. To evaluate this thesis statement, we first designed interfaces which allowed touch between human and virtual human and between virtual human and human, and allowed the human to interact with the virtual human through the manipulation of hand -held tools and handgestures. We then evaluated the validity of the MRIPS approach for practicing and evaluating performance in two interpersonal scenarios in medical education. Content and construct validity were demonstrated for MRIPS -CBE, as cogniti ve, psychomotor, and affective performance in MRIPS CBE were statistically equivalent or noninferior to

PAGE 331

331 performance with an SP; performance in MRIPS -CBE predicted performance with the SP in cognitive and psychomotor tasks; and performance in MRIPS CBE was able to distinguish between inexperienced and experienced users in cognitive, psychomotor, and affective tasks. Content validity was demonstrated for MRIPS -NEURO, as learners were able to use MRIPS NEURO to gather the information required to make a corre ct diagnosis of the virtual humans cranial nerve disorder. Real -time feedback was then developed to provide users of MRIPS with guidance, reinforcement, and motivation for correction in cognitive, psychomotor, and affective tasks. Real -time feedback in MRIPS -NEURO was shown to significantly improve learner performance in the affective task of perspective taking and the efficiency of the eye movements test, and the Patient -Vision feedback provided information sufficient for learners to correctly diagnose the virtual human patients cranial nerve disorder. Real time feedback in MRIPS -CBE was shown to significantly improve coverage, use of correct pressure, use of a vertical strip patternof -search, and completeness of breast history taking. Learning and t raining transfer was evaluated in MRIPS -CBE. Overall, repeated practice with MRIPS -CBE improved cognitive, psychomotor, and affective performance in the CBE of standardized human patients. The small population and ceiling effects limited the strength of these results, but it was clear that learni ng took place in MRIPS CBE and improvement in skills from practice with MRIPS -CBE transferred to the real world interpersonal scenario of CBE of a human patient. 10.2 Future Directions The immediate next step for this work is integration of MRIPS -CBE and MRIPS NEURO into medical school curricula so that more formal, larger studies of learning in

PAGE 332

332 MRIPS can be conducted. Curriculum integration of MRIPS -CBE is planned to occur in the near future at the Medical College of Georgia and the University of Central Florida. MRIPS -NEURO is targeted for integration into the 1styear medical school neuroscience course at the University of Florida. We also foresee the concept of haptic interaction with virtual humans being applied in two additional areas. The first is a continuation of what we have begun with MRIPS. This is the integration of virtual humans with more advanced physical medical simulation, e.g. the Human Patient Simulator (HPS) [55]. By incorporating the medic ally oriented inputs of the HPS (e.g. sensing of administered medication, chest compression, intubation, resuscitation) with the medially oriented (palpation) and communicationoriented (speech, handgestures, tool use, comforting touches) inputs of MRIPS, and the expressiveness and flexibility of the virtual human, a myriad of medical procedures and scenarios could be simulated at a high degree of fidelity. The main challenge in the melding of virtual humans and the HPS is the display of the virtual human. While seethrough head -mounted displays are an option, in our experience, HMDs present a barrier to widespread acceptability within the medical community. Pico projectors or internal projection may provide a solution to the problem of displaying the vi rtual human. The second is the application of haptic interaction with virtual humans beyond the medical domain, for example, t raining business greetings or a tangible virtual museum guide. Expansion into these domains will benefit from two future technological enhancements. One of these is a haptic interface that can move, e.g. walk, bow, shake hands, while remaining registered to the virtual human. The immediate solution is to incorporate more advanced robotics into a haptic interface similar to that of MRIPS -

PAGE 333

333 CBE. The challenge lies in balancing the amount of robotics needed to perform the desired tasks with the amount of robotics visually acceptable to users. In other words, the addition of robotics and increased movement of the haptic interface shoul d not cause the mixed reality human to fall into the uncanny valley. The other technological enhancement required to move into more general domains is the ability of the mixed reality human to interact with more than one human at a time. This will involv e challenges in all aspects of design: enhancing speech recognition and understanding to recognize multiple speakers, tracking the position and attention (head gaze) of multiple users in order for the virtual human to address individual users, and the abil ity to distinguish among touches from multiple users. Within the development required to overcome these challenges, lies an opportunity for unique feedback mechanisms that expand on Patient -Vision feedback. Tracking multiple users will require the additi on of more active sensors to the haptic interface, likely including stereo vision capabilities. Human -like stereo vision could be achieved by adding cameras at the two eyes of the virtual human and engineering the ability to have the cameras move along with the virtual humans eyes. With this setup, Patient -Vision feedback of the real world could be provided by manipulating the video streams of the two cameras and presenting the resulting images to one of the humans int eracting with the virtual human. In this manner, the incorporation of additional sensing of user inputs (e.g. full body pose, proximity) and actuation of additional virtual human outputs (e.g. smell) will serve to further enhance the applicability of interpersonal simulation with virtual humans to a wide variety of real world interpersonal scenarios.

PAGE 334

334 APPENDIX A STUDY MRIPS -SP DATA A.1 Breast History Checklist Data Participant MRIPS (# items) SP (# items) 102 11 11 104 10 11 107 6 5 202 11 11 203 10 9 204 12 12 205 12 11 206 10 9 207 11 8 A.2 Empathy Video Review Instrument 1 Rate the appropriateness of the student's response -how appropriate would this response be if the patient was real (1 ) Exceptionally INappropriate (one of the most inappropriate responses I can imagine) (2 ) Inappropriate (3 ) Borderline (could go either way) (4 ) Appropriate (5 ) Exceptionally appropriate (one of the most appropriate responses I can imagine) Questions 2-4 are from the Empathy subscale of Krupat et al. [41] 2 Empathy: clinician allows patient to express emotions (1 ) Stud ent shows no interest in patient's emotional state and/or discourages or cuts off the expression of emotion by the patient (verbal or nonverbal signals that it is not okay to express emotions). (2 ) (3 ) Student shows relatively little interest or encouragement fo r the patient's expression of emotion; or allows emotions to be shown by actively or subtly encourages patient to move on. (4 ) (5 ) Student openly encourages or is receptive to the expression of emotion (e.g. through use of continuers or appropriate pauses (signal s verbally or nonverbally that it is okay to express feelings). 3 Empathy: clinician validates patient feelings (1 ) Student makes no attempt to respond to/validate the patient's feelings, or possibly belittles or challenges them (e.g. It's ridiculous to be so co ncerned about...). (2 ) (3 ) Student briefly acknowledges patient's feelings but makes no effort to indicate acceptance/validation.

PAGE 335

335 (4 ) (5 ) Student makes comments clearly indicating acceptance/validation of patient's feelings (e.g. I'd feel the same way... I can see how t hat would worry you...). 4 Empathy: clinician explores patient feelings (1 ) Student makes no attempt to identify patient's feelings. (2 ) (3 ) Student makes brief reference to patients' feelings, but does little to explore them by identification or labeling. (4 ) (5 ) Student makes clear attempt to explore patient's feelings by identifying or labeling them (e.g. So how does that make you feel? It seems to me that you are feeling quite anxious about...) 5 Empathy: clinician nonverbal behavior (1 ) Student's nonverbal behavior displays lack of interest and/or concern and/or connection (e.g. little or no eye contact, body orientation or use of space inappropriate, bored voice). (2 ) (3 ) Student's nonverbal behavior shows neither great interest or disinterest (or behaviors over course of visit are inconsistent). (4 ) (5 ) Student displays nonverbal behaviors that express great interest, concern and connection (e.g. eye contact, tone of voice, and body orientation) throughout the visit. A.3 Empathy Video Review Data Critical moment: At the start of exam the patient exclaims Wait! Im scared (7 experts rated this moment). Approp riate Empathy 1 Empathy 2 Empathy 3 Empathy 4 Avg. Emp. ID VP SP VP SP VP SP VP SP VP SP VP SP 208102 4.5 3.5 1.67 1.17 1.67 1.17 1.67 1.17 1.5 1.17 1.63 1.17 208103 2.17 3.84 1. 33 1.5 1.17 1.33 1.17 1.17 1.17 1.5 1.21 1.38 208104 4.0 4.0 1.33 1.5 1.17 1.33 1.33 1.17 1.5 1.5 1.33 1.38 208107 3.5 3.67 1.33 1.33 1.17 1.17 1.17 1.17 1.17 1.33 1.21 1.25 208202 No data 208203 4.0 3.67 1.5 1.5 1.17 1.17 1.17 1.17 1.5 1.33 1.33 1.29 208204 4.17 3.33 1.5 1.5 1.17 1.17 1.17 1.17 1.5 1.5 1.33 1.33 208205 4.17 3.0 1.33 1.33 1.17 1.17 1.17 1.17 1.33 1.33 1.25 1.25 208206 3.67 3.33 1.5 1.5 1.17 1.17 1.17 1.17 1.5 1.5 1.33 1.33 208207 3.33 3.33 1.0 1.33 1.0 1.17 1.0 1.17 1.0 1.17 1.0 1.2 1 Avg: 3.72 3.52 1.39 1.41 1.20 1.20 1.22 1.17 1.35 1.37 1.29 1.29 kappa : 0.22 0.36 0.63 0.58 0.63 0.58 0.63 0.58 0.63 0.58

PAGE 336

336 Critical moment: Patient expresses a fear of cancer Do you think it could be cancer? (6 experts rated this moment). Approp riate Empathy 1 Empathy 2 Empathy 3 Empathy 4 Avg. Emp. ID VP SP VP SP VP SP VP SP VP SP VP SP 208102 4.29 3.14 4.0 2.86 4.14 2.43 3.43 2.29 3.86 2.86 3.86 2.61 208103 3.43 4.0 3.14 2.43 3.14 2.57 2.57 2.43 2.71 3.0 2.89 2.61 208104 3.57 4.57 3.29 4.29 3.29 4.29 3.14 4.42 3.29 4.29 3.25 4.32 208107 4.29 4.0 3.71 3.14 3.57 2.71 3.0 2.57 3.86 3.0 3.54 2.86 208202 2.71 3.0 1.86 2.57 1.71 2.14 1.29 1.86 2.86 3.0 1.93 2.39 208203 4.14 3.71 4.43 3.0 4.86 2.71 4.0 2.0 3.86 3.14 4.29 2.71 208204 4.29 4.29 4 .14 4.29 4.0 4.71 3.86 4.14 4.14 4.29 4.04 4.36 208205 3.57 3.71 3.0 2.71 3.0 2.29 2.86 2.0 2.57 2.71 2.86 2.43 208206 3.29 3.14 2.71 2.71 2.29 2.29 2.14 1.86 2.71 3.29 2.46 2.54 208207 2.57 3.43 1.71 2.29 1.71 2.43 1.14 2.0 2.14 2.57 1.68 2.32 Avg: 3. 61 3.7 3.2 3.03 3.17 2.86 2.74 2.56 3.2 3.21 3.08 2.9 IRR: 0.30 0.43 0.15 0.24 0.31 0.16 0.29 0.14 0.11 0.18 Critical moment: Explaining what will happen to patient What happens next? Appropriate Emp 1 Emp 2 Emp 3 Emp 4 Avg. Emp. ID VP SP VP S P VP SP VP SP VP SP VP SP 208102 4.33 4.17 Not applicable. This item was deemed by reviewers as not requiring empathy. 208103 2.83 3.67 208104 4.33 4.17 208107 4.17 3.83 208202 3.33 3.0 208203 4.0 2.33 208204 4.5 4.0 208205 3.5 3.33 2 08206 3.67 2.17 208207 3.17 2.17 Avg. 3.78 3.28 IRR: 0.18 0.21

PAGE 337

337 APPENDIX B STUDY MRIPS X2 DATA B.1 Study MRIPSx2 Video Review Instrument 1 Rate the appropriateness of the student's response -how appropriate would this response be if the patient w as real? (1 ) Exceptionally INappropriate (one of the most inappropriate responses I can imagine) (2 ) Inappropriate (3 ) Borderline (could go either way) (4 ) Appropriate (5 ) Exceptionally appropriate (one of the most appropriate responses I can imagine) 2 Rate the students use o f empathy (1 ) Student was not at all empathetic (2 ) Student's attempt at empathy was not sincere (e.g. words are associated w/ empathy, but tone of voice demonstrates lack of sincerity) (3 ) Student's attempt at empathy was not successful (e.g. statement may not have been appropriate) (4 ) Student's empathy was appropriate and sincere (5 ) Student's empathy was exceptionally appropriate and sincere B.2 Study MRIPSx2 Video Review Data CM1: Wait, Im scared. What if you find cancer? CM2: I lost my mother to breast cancer two years ago. CM3: Do I really have to get a mammogram? I mean, my mom was fine, then she had a mammogram, and then all of the sudden she was really sick. Blank cells indicate that the participant did not experience this critical moment or that video revi ewers did not evaluate this participants critical moment. Participant 1 2 3 4 5 6 7 8 9 10 12 13 14 15 16 CM1 Acceptable 3.4 4.1 3.4 4.1 4.5 4.7 3.5 3.7 4.2 3.3 2.7 4.0 4.0 3.6 3.8 CM1 Empathetic 2.8 3.9 1.9 3.4 3.5 4.5 1.3 2.4 2.8 1.8 1.8 2.8 2.5 1.4 1 .4 CM2 Acceptable 3.6 3.4 3.4 3.0 4.0 4.3 3.6 3.7 3.8 3.7 3.6 3.5 2.8 3.5 CM2 Empathetic 2.0 3.4 1.3 1.0 1.7 3.2 1.4 2.2 2.3 1.3 1.4 2.3 1.6 1.5 CM3 Acceptable 3.3 2.7 3.6 4.0 4.4 4.3 3.6 3.5 4.0 3.3 4.0 4.0 4.0 CM3 Empathetic 1.9 2.3 1.4 1.0 4.3 3 .2 1.4 1.3 1.3 2.0 1.5 2.2 3.4

PAGE 338

338 (continued) Participant 18 19 20 21 22 23 24 25 26 27 28 29 30 CM1 Acceptable 4.0 4.4 2.6 3.8 4.0 4.0 4.4 3.3 4.0 3.7 4.3 4.0 CM1 Empathetic 2.9 2.6 1.2 3.5 1.8 2.0 4.4 1.7 2.0 2.0 2.7 2.7 CM2 Acceptable 4.0 4.4 4. 0 4.0 1.8 3.0 3.0 3.3 4.3 4.0 4.3 CM2 Empathetic 2.0 3.4 1.4 1.5 1.8 2.0 1.3 1.7 4.3 3.0 3.7 CM3 Acceptable 4.5 4.2 4.3 4.3 4.0 4.0 4.0 4.0 2.7 4.3 4.3 CM3 Empathetic 4.0 3.0 3.0 4.0 4.0 3.0 2.0 2.0 4.3 4.0 B.3 Study MRIPSx2 Breast History Ch ecklist Data Blank cells indicate a clinician or resident (omitted from this analysis) or that a video or transcript was not available for this participant. Participant ID 701 702 703 704 707 708 709 713 714 716 722 Items asked 9 14 10 5 15 14 12 7 13 7 1 0 Participant ID 723 724 725 726 727 728 509 510 511 512 517 Items asked 8 5 12 11 11 6 12 14 10 9 15 Participant ID 518 519 521 523 524 525 526 Items asked 14 13 7 10 9 12 12 B.4 Study MRIPSx2 Palpation Completeness Data Participant ID 701 7 02 703 704 707 708 709 710 712 713 Percent 94.1 89.5 72 70.9 96.5 85.9 89.4 70.9 75.5 94.1 Participant ID 714 716 718 722 723 724 725 726 727 728 Percent 47.1 87.1 90.1 76.2 83.6 74.5 82.5 92.9 73.3 83.7

PAGE 339

339 APPENDIX C STUDY MRIPS -NEURO QUESTIONNAIRES C. 1 Study MRIPS NEURO Post Patient Vision Survey Post -Patient -Vision Survey ID ______________________ 1) Which eye was affected (or none)? __________________ 2) Which cranial nerve was affected (or none)? _______________ 3) Rate the patients ability to carry out everyday tasks (1 = needs assistance in all facets; 5 = normal ability): 1 2 3 4 5 4) Briefly describe how you think the patients double vision affects his everyday life: ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ C.2 Study MRIPS NEURO Post -Exam Survey Post -Exam Survey ID _____________________ 1) For the Patient #1 (history and exam), Which cranial nerve was affected (or none)? _____________ Which eye was affected (or none)? _________________

PAGE 340

340 2) For the Patient #2 (just exam), Which cranial nerve was aff ected (or none)? _____________ Which eye was affected (or none)? _________________ 3) List any concerns you have for Patient #1 (history & exam), or anything you would like to express to the patient and/or his family: ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________

PAGE 341

341 APPENDIX D STUDY MRIPS -LEARNING INSTRUMENTS AND DATA D.1 Study MRIPS -Learning Breast History Checklist Data SP Pre test MRIPS #1 MRIPS #2 MRIPS #3 SP Post test ID RT PE Tot. RT PE Tot. RT PE Tot. RT PE Tot. RT PE Tot. 20 9 1 10 16 0 16 17 1 18 20 2 22 18 3 21 21 11 1 1 2 20 1 21 21 2 23 21 0 21 20 5 25 22 10 1 11 13 2 15 17 2 19 20 2 22 17 3 20 23 9 2 11 21 1 22 20 2 22 20 1 21 14 3 17 24 9 2 11 19 2 21 19 0 19 20 0 20 15 1 16 27 6 2 8 21 0 21 21 0 21 20 0 20 11 1 12 33 12 4 16 21 3 24 19 1 19 20 2 22 14 2 16 37 14 3 17 19 3 22 19 0 19 21 1 22 16 3 19 25 8 2 10 13 2 15 30 6 2 8 8 1 9 34 7 2 9 16 1 17 35 8 2 10 16 0 16 Number of items queried by each participant. RT refers to items in the real time procedural checklist. PE refers to items reviewed only in the post experiential feedback. Tot. refers to the total number of items queried (Tot. = RT + PE).

PAGE 342

342 D .2 Study MRIPS -Learning Coverage and Pressure Data SP Pre test MRIPS #1 MRIPS #2 MRIPS #3 SP Post test ID Cvg Press % Cvg %Deep % Cvg %Deep % Cvg %Deep Cvg Press. 20 0 2 90.1 42.4 81.9 51.2 6 4 21 0 2 99.1 82.1 95.7 80.7 99.2 87.8 6 4 22 4 1 72.1 65.5 89.2 82.0 99.2 91.1 4 4 23 0 1 98.0 94.7 94.2 84.2 99.6 88.5 4 2 24 4 2 66.0 62.1 91.2 70.0 86.6 34.2 6 2 27 0 4 95.0 62.7 86.4 51.3 89.2 74.1 4 4 33 6 4 76.8 66.4 98.3 91.6 90.3 76.7 6 4 37 0 0 92.8 76.0 89.9 53.5 91.1 31.2 4 4 25 0 2 99.5 96.8 30 0 2 96.1 58.1 34 0 2 98.8 94.0 35 4 2 95.0 83.8 Coverage and use of deep pressure in palpating the MRIPS and SP breast. ID SP Pre test MRIPS #1 MRIPS #2 MRIPS #3 SP Post test 20 2 1 2 2 1 21 2 3 2 3 2 22 0 1 2 3 2 23 0 2 2 3 2 24 0 1 1 1 1 27 0 1 0 3 1 33 1 1 3 2 2 37 0 3 0 0 0 Palpation of supraclavicular, infraclavicular, and axilla. Number of the three areas palpated is indicated.

PAGE 343

343 D. 3 Study MRIPS -Learning Pattern -of -Search Data ID SP Pre test MRIPS #1 MRIPS #2 MRIPS #3 SP Post test 20 Non systematic 11.7 8.8 Missing data Vertical strip 21 Vertical strip 16.0 20.1 18.9 Vertical strip 22 Spiral 16.4 9.4 14.1 Vertical strip 23 Non systematic 21.7 15.9 17.8 Vertical strip 24 Non systematic 35.3 18.9 21.1 Vertical strip 27 Non systematic 27.6 22.2 23.2 Vertical strip 33 Vertical strip 23.6 17.9 22.4 Vertical strip 37 Spiral 28.4 21.4 21.3 Spiral Normalized deviation of each parti cipa nts pattern of -search in MRIPS interactions and pattern used in SP interactions. D. 4 Study MRIPS -Learning Empathy Video Review Instrument 1 The participant did not pay attention to the patients emotions when interviewing and examining her (R). (1 ) Strongl y disagree (2 ) Disagree (3 ) Agree (4 ) Strongly agree 2 The participant encouraged the patient to express her emotions. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree (4 ) Strongly agree 3 The participant accepted and/or validated the patients feelings. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree (4 ) Stro ngly agree 4 The participant displayed little interest or concern to the patient. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree (4 ) Strongly agree 5 The participant made little or no attempt to explore the patients feelings. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree (4 ) Strongly agree 6 The participant demonstrated appropriate nonverbal behavior. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree

PAGE 344

344 (4 ) Strongly agree 7 The participant dealt sensitively with embarrassing and disturbing topics and physical pain. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree (4 ) Strongly agree 8 The part icipant expressed support and partnership. (1 ) Strongly disagree (2 ) Disagree (3 ) Agree (4 ) Strongly agree The following two items were used to evaluate participant performance in four critical moments: CM1: I lost my mother to breast cancer two years ago. CM2: Do you think my pain could be because I have cancer (before physical exam) CM3: Well, do you think it could be cancer? (after physical exam) CM4: Do I really have to get a mammogram? My mom was fine then she had a mammogram and then all of the sudden she was really sick? 1 The participants response was appropriate for a real patient. (1 ) Strongly inappropriate; offensive. (2 ) Less appropriate than average. (3 ) An average student might respond this way. (4 ) More appropriate than average. (5 ) Excellent; positive example to other students. 2 The participants response was appropriate for a real patient. (1 ) Strongly inappropriate; offensive. (2 ) Less appropriate than average. (3 ) An average student might respond this way. (4 ) More appropriate than average. (5 ) Excellent; positive example to other students. D. 5 Study MRIPS -Learning Empathy Video Review Data ID Emp.1 Emp.2 Emp.3 Emp.4 Emp.5 Emp.6 Emp.7 Emp.8 Avg. 20 3.29 2.71 3.67 3.29 3.29 3.14 3.29 3.29 3.24 21 1.86 1.57 2.14 2.14 1.86 2.14 2.00 2.00 1.96 22 2.60 2.80 3.00 3.00 2.80 2.80 3.00 2.60 2 .83 23 2.50 2.83 2.83 3.00 2.50 2.83 2.67 3.00 2.77 24 2.25 2.25 2.25 2.38 2.38 2.63 2.57 2.43 2.39 27 2.71 2.50 2.86 2.86 2.83 2.43 2.71 2.86 2.72 33 2.14 2.43 2.29 2.57 2.14 2.57 2.43 2.43 2.38 37 2.57 2.33 2.50 2.33 2.50 2.17 2.33 2.20 2.37

PAGE 345

345 SP Pre -test data from the empathy instrument of Appendix D.4. Ratings greater than 3.0 are considered positive. ID Emp.1 Emp.2 Emp.3 Emp.4 Emp.5 Emp.6 Emp.7 Emp.8 Avg. 20 3.57 3.0 0 3.33 3.50 3.17 3.33 3.0 0 3.17 3.26 21 3.50 3.00 3.38 3.75 3.25 3.38 3.38 3.5 3. 39 22 4.00 3.29 3.57 3.71 3.29 3.29 3.57 3.57 3.54 23 3.25 3.50 3.25 3.38 3.38 3.00 3.00 3.00 3.22 24 2.75 3.00 3.25 3.00 3.13 3.00 3.00 3.13 3.03 27 3.63 3.13 3.50 3.25 3.25 3.13 3.13 3.13 3.27 33 3.00 2.67 3.17 3.14 2.86 2.71 2.83 2.86 2.91 37 2.50 2.29 2.43 2.29 2.14 2.14 2.43 2.29 2.31 SP Post -test data from the empathy instrument of Appendix D.4. Ratings greater than 3.0 are considered positive. CM1 CM2 CM3 CM4 Avg. ID App. Emp. App. Emp. App. Emp. App. Emp. App. Emp. 20 4.13 4.50 4.43 4.43 4.00 3.80 4.00 4.00 4.17 4.23 21 2.13 1.88 2.00 1.63 2.14 2.00 2.71 2.71 2.23 2.03 22 3.17 3.00 2.67 2.67 2.67 2.67 3.40 3.40 2.95 2.91 23 2.86 2.57 2.71 2.57 2.71 2.57 2.86 2.86 2.78 2.64 24 2.38 2.13 2.88 2.75 3.00 3.00 2.75 2.75 2.74 2.65 27 3.00 2.88 2.71 2.43 3.00 2.71 2.80 2.80 2.89 2.70 33 2.25 2.25 2.71 2.43 2.71 2.43 2.50 2.50 2.53 2.40 37 3.25 2.75 2.86 2.43 2.71 2.57 2.20 2.20 2.81 2.52 SP Pre -test critical moment appropriateness and empathy ratings. CM1 CM2 CM3 CM4 Avg. ID App. Emp. App. Emp. App. Emp. App. Emp. App. Emp. 20 4.57 4.00 4.40 4.40 4.50 4.50 4.40 4.40 4.47 4.29 21 4.25 4.25 4.29 4.29 4.38 4.25 4.71 4.57 4.40 4.33 22 4.50 4.50 4.67 4.67 4.71 4.71 4.60 4.80 4.63 4.67 23 3.43 3.57 4.00 3.80 4.00 3.80 4.14 4.00 3.88 3.79 24 3.75 3.88 3.86 3.86 3.88 3.88 4.00 3.88 3.87 3.87 27 4.00 4.00 4.14 4.29 4.00 4.25 3.75 3.75 3.97 4.06 33 3.25 3.50 3.71 3.57 3.50 3.38 4.14 3.71 3.63 3.53 37 3.14 3.00 3.20 3.00 3.25 3.00 3.75 3.25 3.30 3.05 SP Post -test critical moment appropriat eness and empathy ratings.

PAGE 346

346 D.6 Affective Ratings of Participants in MRIPS -CBE Percent of critical moments rated a 2 Normalized score ID MRIPS #1 MRIPS #2 MRIPS #3 MRIPS #1 MRIPS #2 MRIPS #3 20 14.3 25.0 25.0 0.57 1.00 0.63 21 33.3 28.6 37.5 0.67 0.8 6 0.88 22 25.0 28.6 25.0 0.75 0.86 0.88 23 37.5 12.5 25.0 1.00 0.38 1.00 24 0.00 14.3 33.3 0.00 0.43 0.83 27 50.0 28.6 33.3 1.00 1.00 1.17 33 12.5 12.5 12.5 0.50 0.63 0.88 37 14.3 16.7 16.7 0.29 1.00 0.50 D.7 Pilot Study Video Rating Instrument and Data 1 Cognitive: Breast history completeness (queried item, yes/no). a Pain b Discharge c Mass d Breast self examination e Family history f Alcohol use g Smoking history h Birth control pills / Medications i Age of first pregnancy j Onset of period 2 Affective a Attentiveness (in terest in patients problems) (1 ) Did not seem to really be paying attention or listening; interrupted without apology of explanation. (2 ) Attention drifted at times; asked a question that had already been answered without apology. (3 ) Appeared to be paying attention. (4 ) Appeared to be paying attention and responded to verbal or nonverbal cues. b Eye contact (appropriateness of nonverbal behavior) (1 ) Little or no eye contact (2 ) Some eye contact (3 ) Appropriate eye contact at most times (4 ) Appropriate eye contact at all times c Attitude ( appropriateness of verbal behavior) (1 ) Made judgmental comments or criticized patient; or talked down to patient. (2 ) Made 12 comments with inappropriate affect.

PAGE 347

347 (3 ) No judgmental comments; talked to patient as an equal. (4 ) No judgmental comments; talked topatient as an equal; offered praise/encouragements when opportunity arose. d Empathy and Support (1 ) Offered no empathetic comments; no encouragement or support (did not state intention to help). (2 ) Offered only brief supportive or empathetic comment and only in response to a distinct emotional statement by patient. Comments may seem prospective or forced. (3 ) Offered empathetic or supportive comments or stated intention to help. (4 ) Offered empathetic or supportive comments or stated intention to help; despite limited time seemed to be on the way to establishing a caring relationship. 3 Psychomotor: physical exam. a Inspected both breasts arms relaxed (0 or 1). b Inspected both breasts arms flexed (0 or 1). c Inspected both breasts arms raised (0 or 1). d Examined supraclavicular area (0 or 1). e Examined infraclavicular area (0 or 1). f Examined deep central axillary area (0 or 1). g Use of pressure Light only: 1 pt. Light and medium: 2 pts. Deep (and medium and light): 4 pts. h Used vertical strip pattern (0 or 1). i Coverage Examined entire chest area: 6 pts. Examined cone only: 4 pts. Incomplete examination of cone: 0 pts.

PAGE 348

348 ID Visual Inspection (0 to 3) Breast History (0 to 10) Ancillary Coverage (0 to 3) Coverage (0 to 6) Pressure (0 to 4) Vertical Strip Affective (1 to 4) 56 2 2 1 4 1 No 2.38 55 1 4 1 4 1 Yes 3.25 51 3 8 1 4 4 Yes 2.83 53 3 3 0 4 4 No 3.63 49 0 7 1 6 4 Yes 3.25 48 0 4 0 4 4 No 2.53 45 0 4 2 6 4 Yes 3.68 47 1 5 2 6 4 Yes 3.56 46 0 6 2 6 4 Yes 3.10 91 0 4 0 4 1 No 2.90 73 3 3 1 4 4 No 3.35 64 0 6 1 0 4 Yes 2.40 63 0 3 0 4 2 No 3.38 69 3 6 0 4 4 No 3.45 68 1 5 0 4 4 No 2.75 74 0 4 0 0 0 No 3.00 54 3 5 0 4 1 Yes 3.25 70 2 6 0 4 1 Yes 3.50 83 3 5 0 0 0 No 2.57 562nd SP 0 n/a 0 0 4 No n/a 51 2nd SP 0 n/a 1 4 4 Yes n/a 47 2nd SP 1 n/a 3 6 4 No n/a 46 2nd SP 0 n/a 0 4 1 No n/a 73 2nd SP 0 n/a 0 4 2 No n/a

PAGE 349

349 LIST OF REFERENCES [ 1 ] J. Chalabian and G. Dunnington, Do our current assessments assure competency in clinical breast evaluation skills? American Journal of Surgery vol. 175, pp. 497502, 1998. [ 2 ] P.E. Kann, and D.S. Lane, Breast cancer screening knowledge and skills of students upon entering and exiting a medical school Academic Medicine vol. 73, no. 8, pp. 904-906, Aug. 1998. [ 3 ] D.A. Sloan, M.B. Donnelly, R.W. Schwartz, L.C. Munch, M.D. Wells, S.B. Johnson, and W.E. Strodel Assessing medical students and surgery residents clinical competence in problem solving in surgical oncology, Annals of Surgical Oncology vol. 1, no. 3, pp. 204212, May 1994. [ 4 ] S. McDonald, D. Saslow, and M. H. Alciati, Performance and Reporting of Clinical Breast Examination: A Review of the Literature, CA Cancer Journal for Clinicians vol. 54, no. 6, pp. 345 -361, 2004. [ 5 ] C.M. Pugh and P. Youngblood, Development and Validation of Assessment Measures for a Newly Devel oped Physical Examination Simulator, Journal of the American Medical Informatics Association, vol. 9, no. 5, pp. 448460, Sep-Oct. 2002. [ 6 ] P. Dull, and D. J Haines, Methods for teaching physical examination skills to medical students, Family Medicine vo l. 35, no. 5, pp 343348, May 2003. [ 7 ] R. Kneebone, J. Kidd, D. Nestel, S. Asvall, P. Paraskeva, A. Darzi, An innovative model for teaching and learning clinical procedures, Medical Education vol. 36, no. 7, pp. 628-634, July 2002. [ 8 ] D. Nestel and T. Tiern ey, Role play for medical students learning about communication: Guidelines for maximizing benefits BMC Medical Education vol. 7, no. 3, pp. 1-9, Mar. 2007. [ 9 ] D. Saslow, J. Hannan, J. Osuch, M.H. Alciati, C. Baines, M. Barton, J.K. Bobo, C. Coleman, M. Dolan, G. Gaumer, D. Kopans, S. Kitner, D.S. Lane, H. Lawson, H. Meissner, C. Moorman, H. Pennypacker, P. Pierce, E. Sciandra, R. Smith, and R. Coates, Clinical Breast Examination: Practical Recommendations for Optimizing Performance and Reporting, Cancer Journal for Clinicians, vol. 54, pp. 327-344, 2004. [ 10 ] K. Johnsen, R. Dickerson, A. Raij, B. Lok, J. Jackson, M. Shin, J. Hernandez, A. Stevens, and D.S. Lind Experiences in Using Immersive Virtual Characters to Educate Medical Communication Skills, Pro ceedings of IEEE Virtual Reality 2005 Bonn, Germany, 12 16 March 2005, pp. 179186.

PAGE 350

350 [ 11 ] J. Cassell, T. Bickmore, and M. Billinghurst, Embodiment in conversational interfaces: Rea, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 1999) Pittsburgh, PA, May 15-20, 1999, pp. 520-527. [ 12 ] J Westberg and H Jason, Fostering reflection and providing feedback: Helping others learn from experience, New York Springer, 2001. [ 13 ] R.H. Hill Jr, J. Gratch, S. Marsella, J. Ri ckel, W. Swartout, and D. Traum, Virtual Humans in the mi ssion rehearsal exercise system, Artificial Intelligence (AI Journal) vol. 17, no. 4, pp. 5 -10, 2003. [ 14 ] K. Johnsen, The Design and Validation of a Virtual Human System for Interpersonal Skills Education, Unpublish ed doctoral dissertation, University of Florida, 2008. [ 15 ] K. Johnsen, R. Dickerson, A. Raij, C. Harrison, B. Lok, A. Stevens, and D.S. Lind, Evolving an Immersive Medical Communication Skills Trainer, Presence: Teleoperators and Virtual Environments vol. 15, no. 1, pp. 33-46, Feb. 2006. [ 16 ] K. McLaughlin, L. Gregor, A. Jones, and S. Coderre, Can standardized patients replace physicians as OSCE examiners? BMC Medical Education vol. 6, no. 12, pp. 15, Feb. 2006. [ 17 ] D.V. Power, and B.A. Center, Examining the Medical Student Body: Peer Physical Exams and G enital, Rectal, or Breast Exams, Teaching and Learning in Medicine vol. 17, no. 4, pp. 337 -343, Oct. 2005. [ 18 ] E.H. Chang, and D.V. Power, Are Medical Students Comfortable with Practicing Physical Examinations on Each Other? Academic Medicine, vol. 75, no. 4, pp. 384-389, Apr. 2000. [ 19 ] P.A. ONeill, C. Larcombe, K. Duffy and T.L. Dorman, Medical students willingness and reactions to learning basic skills through examining fellow students Medical Teacher v o l 20, no. 5, pp. 433 437, Sept. 1998. [ 20 ] B. Rossen, K. Johnsen, A. Deladisma, D.S. Lind, and B. Lok, Virtual Humans Elicit Skin -Tone Bias Consistent with Real World Skin -Tone Biases, Intelligent Virtual Agents 2008 Tokyo, Japan, Sept. 1-3, 2008, pp. 237 -244. [ 21 ] E. Deaton, C. Barba, T. Santarelli, L. Rosenzweig, V. Souders, C. Mc -Collum, J. Seip, W. Knerr, and J. Singer. Virtual environment cultural training for operational readiness ( VECTOR), Virtual Reality vol. 8, no. 3, pp. 156-167, June 2005.

PAGE 351

351 [ 22 ] G. Frank, C. Guinn, R. Hubal, P. Pope, M. Stanford, and D. Lamm -Weisel. Just talk: An application of responsive virtual human technology Proceedings of the 24th Interservice/Industry Training, Simulation and Education Conference Orlando, FL, Dec. 2002. [ 23 ] M atriculants to U.S. Medical Schools by Race Combinations within Hispanic Ethnicity [Internet]. Association of American Medical Colleges; [updated 2008 Oct. 10; cited 2009 Sept. 14]. Available from: http://www.aamc.org/data/facts/2008/2006to2008detmat.htm [ 24 ] K. Stevenson, and P. Sander, Medical students are from Mars business and psychology students are from Venus University teachers are from Pluto? Medical Teacher vol. 24, no. 1, pp. 2731, Jan 2002. [ 25 ] Richmond County Quick Facts from the U.S. Census B ureau [Internet]. U.S. Census Bureau; [updated 2009 Sept. 4; cited 2009 Sept. 14]. Available from: http://quickfacts.census.gov/qfd/states/13/13245.html [ 26 ] G H McGinn, N E. Weaver, D M McDonald, M vanDriel, and P. A Hancock, RTO Human Factors and Medicine Panel (HFM) Symposium Copenhagen, Denmark, Apr. 21 -23, 2008, pp. 111. [ 27 ] L.D. Howley, Performance Assessment in Medical Education: Where Weve Been and Where Were Going, Evaluation and the Health Professions vol. 27, no. 3, pp. 285303, 2004. [ 28 ] A. Kot ranza, K. Johnsen, J. Cendan, B. Miller, D.S. Lind, and B. Lok, Virtual Multi tools for Hand and Tool -Based Interaction with Life-size Virtual Human Agents, Proceedings of IEEE Symposium on 3D User Interfaces 2009, Lafayette, LA, Mar. 1415, 2009, pp. 2330. [ 29 ] R.C. Hubal and R.S. Day, "Informed Consent Procedures: An Experimental Test Using a Virtual Character in a Dialog Systems Training Application," Journal of Biomedical Informatics vol. 39, no. 5, pp. 532540, Oct. 2006. [ 30 ] A. Manganas, M. Tsiknakis, E. Leisch, M. Ponder, T. Molet, B. Herbelin, N. Magnenat -Thalmann, D. Thalmann, M. Fato, and A. Schenone, The Just Vr Tool: An innovative approach to training personnel for emergency situations using virtual reality techniques, Information Technology in He althcare, vol. 2, no. 6, pp. 399412, 2004. [ 31 ] P. Kenny, T.D. Parsons, J. Gratch, and A.A. Rizzo, Evaluation of Justina: A Virtual Patien t with PTSD, Proceedings of the 8th international Conference on Intelligent Virtual Agents (IVA 2008) Tokyo, Japan, Sept. 1 -3, 2008, pp. 394408.

PAGE 352

352 [ 32 ] P. Kenny, T.D. Parsons, J. Gratch, A. Leuski, and A.A. Rizzo, Virtual Patients for Clinical Therapist Skills Training, Intelligent Virtual Agents Berlin, Springer, 2007, pp. 197210. [ 33 ] W. Swartout, J. Gratch, R.W. Hill, E. Ho vy, S. Mar sella, J. Rickel, and D. Traum, Toward Virtual Humans, AI Magazine, vol. 27, no. 2, pp. 96-108, Jul. 2006. [ 34 ] J. D. Fisher, M. Rytting, and R. Heslin, Hands touching hands: affective and evaluative ef fects of an interpersonal touch, Sociometry vol. 39, no. 4, pp. 416421, Dec. 1976 [ 35 ] D. J. Dolin and M. BoothButterfield, Reach Out and Touch Someone: Analysis of Nonverbal Comforting Responses, Communication Quarterly vol. 41, no. 4, pp. 383-393, Fall 1993. [ 36 ] J.G. Bruhn, The doctor's touch: T a ctile communication in the doctor patient relationship, Southern Medical Journal vol. 71, no. 2, pp. 1469 -1473, Dec. 1978. [ 37 ] H.H. Clark and S.E. Brennan, Grounding in Communication, Perspectives on socially shared cognition, Eds: L.B. Resnick, J.M. Levi ne, and S.D. Teasley. Washington, American Psychological Association, 1991, pp. 127149. [ 38 ] C.M. Pugh and L.H. Salud, Fear of missing a lesion: use of simulated breast models to decrease student anxiety when learning clinical breast examinations, American Journal of Surgery vol. 193, no. 6, pp. 766770, 2007. [ 39 ] C.M. Pugh, Z.B Domont, L.H. Salud, and K.M Blossfield, A Simulation Based Assessment of Clinical Breast Examination Techniques: Do patient and clinician factors affect clinician approach? American Journal of Surgery vol. 195, no. 6, pp. 874880, 2008. [ 40 ] D.W. Johnson, Cooperativeness and social perspective taking, Journal of Personality and Social Psychology vol. 31, pp. 241244, 1975. [ 41 ] E. Krupat, R. Frankel, T. Stein, and J. Irish, The Four Habi ts Coding Scheme: Validation of an instrument to assess cli nicians communication behavior, Patient Education and Counseling, vol. 62, no. 1, pp. 38-45, Jun. 2006. [ 42 ] R Balkissoon, K Blossfield, L Salud, D Ford, and C Pugh, Lost in translation: U nfold ing medical students misconceptions of how to perform a clinical digital rectal examination The American Journal of Surgery vol. 197, no. 4, pp. 525 532, Apr. 2009 [ 43 ] H. L. Lujan and S. E. DiCarlo, First year medical students prefer multiple learning styles Advances in Physiology Education vol. 30, pp. 1316, 2006

PAGE 353

353 [ 44 ] A.K. Madan, S. Aliabadi -Wahle, A.M. Babbo, M. Posner, and D.J. Beech, Education of medical students in clinical breast examination during surgical clerkship American Journal of Surgery vol. 184, no. 6, pp. 637640, Dec. 2002. [ 45 ] M.J. Horner, L.A.G. Ries, M. Krapcho, N. Neyman, R. Aminou, N. Howlader, S.F. Altekruse, E.J. Feuer, L. Huang, A. Mariotto, B.A. Miller, D.R. Lewis, M.P. Eisner, D.G. Stinchcomb, and B.K. Edwards, SEER Cancer Statistics Review, 1975 -2006 National Cancer Institute. Bethesda, MD, 2008. [ 46 ] B.J. Shingleton, M.W. ODonoghue, Blurred vision, New England Journal of Medicine vol. 343, no. 8, pp. 556 562, Aug. 2000. [ 47 ] P. J Kutschke, Taking a history of the patient with diplopia Insight vol. 21, no. 3, pp. 92-95, Sept. 1996. [ 48 ] D.J. Gelb, C.H. Gunderson, K.A. Henry, H.S. Kirshner, and R.F. J ozefowicz The neurology clerkship core curriculum Neurology vol. 58, no. 6, pp. 849852, Mar. 2002. [ 49 ] L E. Davis and M.K. King Assesment of medical student clinical compet encies in the neurology clinic, Neurology vol. 68, pp. 597599, 2007. [ 50 ] E. S. van de Graaf, J Felius, H van Kempen-du Sar, C. W.N. Looman, J Passchier, H Kelderman, H J Simonsz, Construct validation of th e Amblyopia and Strabismus Question naire (A&SQ) by factor analysis, Graufes Archive for Clinical and Experimental Opthalmology vol 247, no. 9, pp. 12631268, Sept 2009. [ 51 ] R.M. C omer, E. D awson, G. P lant, J.F. Acheson, and J.P. Lee, Causes and outcomes for patients presenting with diplopi a to an eye casualty department, Eye vol. 21, pp. 413-418, 2007. [ 52 ] Diplopia: Follow up [Internet]. eMedicine Ophthalmology; [updated 2008 Sept. 8; cited 2009 Sept. 15]. Available from: http://emedicine.medscape.com/arti cle/1214490 -followup [ 53 ] F. Schon, P. Hart and C. Fernandez, Is clinical neurology really so difficult? Journal of Neurology, Neurosurgery, and Psychiatry vol. 72, pp. 557 559, 2002. [ 54 ] F.G.A. Moore and C. Chalk The essential neurologic examination: What should medical students be taught? Neurology vol. 72, no. 23, pp. 20202023, June 2009.

PAGE 354

354 [ 55 ] W.L. van Meurs, M.L. Good, and S. Lampotang, Functional Anatomy of Full -scale Patient Simulators, Journal of Clinical Monitoring and Computing, vol. 13, no. 5, pp. 317324, Sept. 1997. [ 56 ] Limbs and Things Eye Examination Simulator [Internet]. Limbs and Things Co.; [cited 2009 Oct. 4]. Available from: http://www.limbsandthings.com/uk/products.php?id=96 [ 57 ] R Lasslo, J. Keltner, and G Henderson, Virtual patient/eye si mulator E volution of a futuristic teaching tool Proceedings of the AMIA 2002 Annual Symposium p. 1074. [ 58 ] B. Scherokman, K Cannard, and J Q Miller, What should a graduating medical student know about neurology? Neurology vol. 44, pp. 1170 -1176, Jun e 1994. [ 59 ] B. Casabella and M. Aguilar Neurological disorders: A primary care approach, Aten Primaria, vol. 7, no.8, pp. 538544, Sept. 1990. [ 60 ] Bickerstaffs Neurological Examination in Clinical Practice, 6th Edition, Ed: J.A Spillane, Wiley -Blackwell, 19 96. [ 61 ] S.M. Ku rtz, J.D. Silverman, J. Draper, Teaching and Learning Communication Skills in Medicine Radcliffe Medical Press, Oxford, 1998. [ 62 ] C. Pilgrim, C. Lannon, R.P. Harris, W. Cogburn, S.W Fletcher, Improving clinical breast examination training in a m edical school : a randomized controlled trial, Journal of General Internal Medicine, vol. 8, no. 12, pp. 685688, Dec. 1993. [ 63 ] J Biggs, Individual differences in study processes and the Quality of Learning Outcomes, Higher Education, vol 8, no 4, pp. 381 -394, July 1979 [ 64 ] S. Babu, T. Barnes, and L. Hodges, Can immersive virtual humans teach social conversational protocols? Proceesings of IEEE Internactional Conference on Virtual Reality 2007 Charlotte, NC, Mar 10 14 2007, pp. 215218. [ 65 ] D.C. Hall, C.K Adams, G.H. Stein, H.S. Stephenson, M.K. Goldstein, H.S. Pennypacker, Improved detection of human breast lesions following experimental training Cancer vol. 46, no. 2, pp. 408 -414, July 1980 [ 66 ] M.B. Barton, R. Harris, and S.W. Fletcher, Does this pat ient have breast cancer? T he screening clinical breast examination: S hould it be done? How? Journal of the American Medical Association, vol. 282, no. 13, pp. 1270-1280, Oct. 1999.

PAGE 355

355 [ 67 ] D. -P. Pertaub, M. Slater, and C. Barker, An Experiment on Public Speaking Anxiety in Response to Three Different Types of Virtual Audience, Presence: Teleoperators and Virtual Environments vol. 11, no. 1, pp. 68-78, 2002. [ 68 ] P.S. Bordnick, K.M. Graap, H.L. Copp, J. Brooks, and M. Ferrer, "Virtual Reality Cue Reactivity Assessment in Cigarette Smokers ," Cyberpsychology & Behavior vol. 8, no. 5, pp. 487492, 2005. [ 69 ] M. Garau, D. Friedman, H.R. Widenfeld, A. Antley, A. Brogni, and M. Slater, Temporal and Spatial Variations in Presence: Qualitative Analysis of Interviews from an E xperiment on Breaks in Presence, Presence: Teleoperators and Virtual Environments vol. 17, no. 3, pp. 293-309, June 2008. [ 70 ] C.A. Zanbaka, A. Ulinski, P. Goolkasian, and L.F. Hodges, Social responses to virtual humans: implications for future interface de sign Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2007) San Jose, CA, Apr. 28May 3, 2007, pp. 15611570. [ 71 ] J.N. Bailenson, J. Blascovich, A.C. Beall, and J.M. Loomis Equilibrium theory revisited: Mutual gaze and pers onal space in virtual environments PRESENCE: Teleoperators and Virtual Environments vol. 10, no. 6, pp. 583598, Dec. 2001. [ 72 ] C. Zanbaka, P. Goolkasian, and L.F. Hodges, Can a Virtual Cat Persuade You? The Role of Gender and Realism in Speaker Persuasiv eness, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2006) Montreal, Quebec, Canada, Apr. 2227, 2006, pp. 11531162. [ 73 ] E.A. Plant, A.L. Baylor, C.E. Doerr, and R.B. RosenbergKima, Changing middle school students' attit udes and performance regarding engineering wi th computer based social models, Computers and Education, vol. 53, no. 2, pp. 209 215, Sept. 2009. [ 74 ] P. Anderson, E. Zimand, S.K. Schmertz, and M. Ferrer, Usability and utility of a computerized cognitive-beh avioral self -help program for public speaking anxiety, Cognitive & Behavioral Practice vol 14, no. 2, pp. 198-207, 2007. [ 75 ] S. Babu, T. Grechkin, B. Chihak, C. Zeimer, J. Kearney, J. Cremer, and J. Plumert, A Virtual Peer for Investigating Social Influenc es on Childrens Bicycling, Proceedings of IEEE Virtual Reality 2009 Conference Lafayette, LA, Mar. 14 18, 2009, pp. 9198. [ 76 ] A. Leuski, J. Pair, D. Traum, P.J. McNerney, P. Georgiou, and R. Patel, How to Talk to a Hologram, Proceedings of the 2006 ACM International Conference on Intelligent User Interfaces (IUI 2006) Sydney, Australia, Jan. 29 Feb. 1, 2006, pp. 360-362.

PAGE 356

356 [ 77 ] B. Rossen, D.S. Lind, B. Lok, "Human-centered Distributed Conversational Modeling: Efficient Modeling of Robust Virtual Human Convers ations," Proceedings of the 9th International Conference on Intelligent Virtual Agents 2009, Amsterdam, Netherlands, Sept. 1416, 2009, pp. 474-481. [ 78 ] R. Artstein, S. Gandhe, J. Gerten, A. Leuski, and D. Traum, Semi -formal Evaluation of Conversational Char acters, Languages: From Formal to Natural Berlin, Springer, pp. 22 35. [ 79 ] K.R. Thr isson, A Mind Model for Multimodal Communicative Creatures and Humanoids, International Journal of Applied Artificial Intelligence, vol. 13 no. 4, pp. 449486, 1999. [ 80 ] A. Raij, K. Johnsen, R. Dickerson, B. Lok, M. Cohen, M. Duerson, R. P auly, A. Stevens, P. Wagner, and D.S. Lind, "Comparing Interpersonal Interactions with a Virtual Human to Those with a Real Human," IEEE Transactions on Visualization and Computer Graphics v ol. 13, no. 3, pp. 443 -457, May -Jun. 2007. [ 81 ] K. Johnsen, A. Raij, A. Stevens, D.S. Lind, and B. Lok "The Validity of a Virtual Human Experience for Interpersonal Skills Education" Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI 2007) San Jose, CA, Apr. 28 May 3, 2007, pp. 10491058. [ 82 ] A. Kotranza and B. Lok. Virtual Human + Tangible Interface = Mixed Reality Human: An Initial Exploration with a Virtual Breast Exam Patient Proceedings of IEEE Virtual Reality 2008, Reno, N V, March 8 12, 2008, pp. 99106. [ 83 ] R.W. Hill, J. Belanich, H.C. Lane, M. Core, M. Dixon, E. Forbell, J. Kim, and J. Hart, Pedagogically Structured Game-Based Training: Development of the ElECT BiLAT Simulation, Proceedings of the 25th Army Science Conf erence, Orlando, FL, Nov. 27 30, 2006, pp. 1-8. [ 84 ] A. Raij and B. Lok, IPSVIZ: An After -Action Review Tool for HumanVirtual Human Experiences Proceedings of IEEE Virtual Reality 2008 March 8 12, Reno, NV, pp. 91 -98. [ 85 ] A. Raij, A. Kotranza, D.S Lind, C.M. Pugh, and B. Lok, Virtual Experiences for Social Perspective Taking, Proceedings of IEEE Virtual Reality 2009, Lafayette, LA, Mar. 1418, 2009, pp. 99102. [ 86 ] C. Basdogan, C. Ho, M.A. Srinivasan, and M. Slater, An Experimental Study on the Role of Touch in Shared Virtual Environments, ACM Transactions on Computer -Human Interaction, vol. 7, no. 4, pp. 443460, Dec. 2000

PAGE 357

357 [ 87 ] S.E. Jones, and A.E. Yarbrough, Naturalistic Study of the Meanings of Touch, Communication Monographs vol. 52, no. 1, pp. 19 -56, Mar 1985. [ 88 ] J. P. Bardeen, Interpersonal perception through the tactile, verbal, and visual modes. Meeting of the International Communication Association Phoenix, Arizona, Apr. 1971. [ 89 ] P. Ellsworth, H.S. Friedman, D. Perlick, and M. Hoyt Effects of direc t gaze on subjects motivated to s eek or avoid social comparison, Journal of Experimental Social Psychology vol. 14, pp. 6987, 1978. [ 90 ] J. E. Pattison, Effects of touch on self exploration and the therapeutic relationship Consulting and Clinical Psychol ogy vol. 40, no. 2, pp. 170175, Apr. 1973. [ 91 ] D.C. Aguilera, Relationship between physical contact and verbal interaction between nurses and patients, Journal of Psychiatric Nursing and Mental Health, vol. 5, no. 1, pp. 5 21, Jan. -Feb. 1967. [ 92 ] P. Routasal o, The Right to Touch and Be Touched. Nursing Ethics vol. 3, no. 2, pp. 165 176, 1996. [ 93 ] A. Kotranza, B. Lok, A. Deladisma, C.M. Pugh, and D. S. Lind, Mixed Reality Humans: Evaluating Behavior, Usability, and Acceptability, IEEE Transactions on Visua lization and Computer Graphics vol. 15, no. 3, pp. 369382, May -June 2009. [ 94 ] S. Wesugi and Y. Miwa, Dual Embodied Interaction for Creating a Virtual Co existing Space, Proceedings of the 6th Annual International Workshop on Presence, Aalborg, Denmark, Oc t. 6 8, 2003, pp. 40. [ 95 ] I. Oakley, S. Brewster, and P.D. Gray Can You Feel the Force? An Investigation of Haptic Collaboration in Shared Editors, Proceedings of EuroHaptics 2001, Birmingham, UK, pp. 54 -59. [ 96 ] S. Balcisoy, M. Kallman, R. Torre, P. Fua, and D. Thalman, Interaction techniques with virtual humans in mixed environments, Proceedings of the 5th IEEE International Summer School on Biomedical Imaging, June 1513, 2002., pp. 1523. [ 97 ] J N. Bailenson and N. Yee, Virtual Interpersonal Touch: Haptic Interaction and Copresence in Collaborative Virtual Environments International Journal of Multimedia Tools and Applications vol. 37, no. 1, pp. 5-14, Mar. 2008.

PAGE 358

358 [ 98 ] F. Biocca, J. Kim, and Y. Choi, Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross Modal Sensory Illusions Presence: Teleoperators and. Virtual Environments vol. 10, no. 3, pp. 247-265, June 2001 [ 99 ] A. Pusch, O. Martin, and S. Coquillart, HEMP Hand -Displacement Based Pseudo -Haptics: A Study of a Force Field Application Proceedings of IEEE Symposium on 3D User Interfaces 2008, Mar. 8 9, Reno, NV, pp. 5966. [ 100] D. Kondo, R. Kijima, and Y. Takahashi, "Dynamic Anatomical Model for Medical Education using Free Form Projection Display", Proc eedings of 13th International Conference on Virtual Systems and Multi media (VSMM 2007) Brisbane, Australia, Sept. 2326, 2007, pp. 142149. [ 101] A. Kotranza, B. Lok, A. Deladisma, C.M. Pugh, D.S. Lind, Mixed Reality Humans: Evaluating Behavior, Usability, and Acceptability, IEEE Transactions on Visualization and Computer Graphics vol. 15, no. 3, pp. 369382, May -June 2009 [ 102] S.W. Fletcher, M.S. O'Malley, J.L Earp, T.M. Morgan, S. Lin, and D. Degnan, How best to teach women breast self examination, Annals of Internal Medicine, vol. 112, no. 10, pp. 772 -779, May 1990. [ 103] S.A. Fox and J.A. Stein, The Effect of Physician-Patient Communication on Mammography Utilization by Different Ethnic Groups, Medical Care vol. 29, no. 11, pp. 10651082, Nov. 1991. [ 104] J. K. Burgoon, J. B. Walther, and E. J. Baesler, Interpretations, Evaluations, and Consequences of Interpersonal Touch. Human Communication Research vol. 19, no. 2, pp. 237 263, Dec. 1992 [ 105] B.S. Bloom, Taxonomy of Educational Objectives, Handbook I: The Cog nitive Domain. New York: David McKay Co Inc, 1956. [ 106] E.J. Simpson, The Classification of Educational Objectives in the Psychomotor Domain. Washington, DC Gryphon House, 1972. [ 107] J.K. Heard, M. Cantrell, L. Presher, V.S. Klimberg, G.S. San Pedro, and D.O. Erw in, Using standardized patients to teach breast evaluation to sophomore medical students Journal of Cancer Education vol. 10, no. 4, pp. 191-194, 1995. [ 108] J. G affan, J. D acre, and A. Jones, Educating undergraduate medical students about oncology: A literature review, Journal of Clinical Oncology vol. 24, no. 12, pp. 1932-1939, Apr. 2006.

PAGE 359

359 [ 109] A. Tang, C. Owen, F. Biocca, and W. Mou, Comparative effectiveness of augmented reality in object assembly, Proceedings of the SIGCHI Conference on Human Factors i n Computing Systems (CHI 2003) Ft. Lauderdale, FL A pr. 510, 2003, pp. 7380. [ 110] B. Lok, S. Naik, M. Whitton, and F. Brooks, Effects of Interaction Modality and Avatar Fidelity on Task Performance and Sense of Presence in Virtual Environments, Presence: Teleoperators and Virtual Environments vol. 12, no. 6, pp. 615628, 2003. [ 111] M. Piccardi, Background s ubtraction techniques: A review, Proceedings of the 2004 IEEE International Conference on Systems, Man and Cybernetics vol. 4, pp. 30993104, Oct. 2004. [ 112] N. Pinckney, Pulse width modulation fo r microcontroller servo control, IEEE Potentials vol. 25, no. 1, pp. 2729, JanFeb. 2006. [ 113] R.M. Taylor, T.C. Hudson, A. Seeger, H. Weber, J. Juliano, and A.T. Helser, VRPN: A device independent, network -transpar ent VR peripheral sys tem, Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST 2001) Banff, Alberta, Canada, Nov. 15-17, 2001, pp. 55 -61. [ 114] S. Wellek, Testing Statistical Hypotheses of Equivalence, Chapman and Hall, Boca Raton FL, 2003. [ 115] M. Duerson, J. Cendan, J. Woodard, M. Hammoud, Integrating Anatomical Simulators with Standardized Patients in a Clinical Skills Examination, Poster presentation at the 6th Annual Conference of the Association of Standardized Patient Educators Toronto, Canada, June 17-20, 2007. [ 116] S.J. Pocock, The pros and cons of noninferiority trials, Fundamental and Clinical Pharmacology vol. 17, no. 4, pp. 483490, Aug. 2003. [ 117] J.J. Randolph, Free marginal multirater kappa: An alternative to Fleiss' fi xed marginal multirater kappa, Joensuu University Learning and Instruction Symposium 2005 Joensuu, Finland, October 1415th, 2005. [ 118] B. Hodges, N. McNaughton, G. Regehr, R. Tiberius, and M. Hanson, The challenge of creating new OSCE measures to capture t he characteristics of expertis e, Medical Education vol. 36, no. 8, pp. 742748, Aug. 2002. [ 119] The Joint Commission on Accreditation of Healthcare Organizations. The Physicians Promise: Protecting Patients from Harm, 2nd Edition JCR Publishing, 2006.

PAGE 360

360 [ 120] M Ramirez Lassepas, C.E. Espinosa J.J. Cicero K.L. Johnston, R.J. Cipolle and D.L. Barber P redictors of Intracranial Pathologic Findings in Patients Who Seek Eme rgency Care Because of Headache, Archives of Neurology vol. 54, no. 12, pp. 1506-1509, D ec. 1997. [ 121] D.A. Bowman, and C.A. Wingrave, Design and Evaluation of Menu Systems for Immersive Virtual Environments, Proceedings of the Virtual Reality 2001 Conference Washington, DC, Mar. 1317, 2001, p. 149. [ 122] B. Leibe, D. Minnen, J. Weeks, and T. Starner, Integration of Wireless Gesture Tracking, Object Tracking, and 3D Reconstruction in the Perceptive Workbench, Proceedings of the 2nd International Workshop on Computer Vision Systems (ICVS 2001) Vancouver, BC, Canada, July 2001, pp. 73-92. [ 123] R.G. O Hagan, A. Zelinsky, and S. Rougeaux, "Visual gesture interfaces for virtual environments," Interacting with Computers vol. 14, pp. 231250, 2002. [ 124] X. Wang, A. Kotranza, J. Quarles, B. Lok, and D. Allen A Pipeline for Rapidly Incorporating Real Objects i nto a Mixed Environment, Proceedings of the 4th IEEE and ACM International Symposium on Mixed and Augmented Reality Vienna, Austria, 2005, pp. 170173. [ 125] Managed Library for Nintendo's Wiimote [Internet]. Brian Peek; [updated 2007 Mar. 14; cited 2008 Oct. 31]. Available from: http://blogs.msdn.com/coding4fun/archive/2007/03/14/1879033.aspx [ 126] J.C. Lee, "Hacking the Nintendo Wii Remote," IEEE Pervasive Computing vol. 7, no. 3, pp. 3945, Jul -Sept, 2008. [ 127] A. Kotranza, D S. Lind, C M Pugh, and B Lok, Real -T ime In-Situ Visual Feedback of Task Performance in Mixed Environments for Learning Joint Psychomotor -Cognitive Tasks, (to appear) Proceedings of the 8th IEEE/ACM International Symposium on Mixed and Augmented Rea lity (ISMAR 2009) Orlando, FL, Oct. 19 23, 2009. [ 128] E. Burns, S. Razzaque A.T. Panter, M.C. Whitton, M.R. McCallus, F.P. Brooks Jr, The hand is slower than the eye: a quantitative exploration of visual dominance over proprioception Proceedings of IEEE Virtual Reality 2005 (VR 2005) Bonn, German y, Mar. 12 -16, 2005, pp. 310. [ 129] A. Lumma -Sellenthin, Talking with patients and peers: Medical students' difficulties wit h learning communication skills, Medical Teacher vol 31, no. 6, pp 528-534 June 2009

PAGE 361

361 [ 130] R. Barra, J.M. Montero, J. Macas -Guarasa, L.F. DHaro, R. San -Segundo, and R. Crdoba, P rosodic and segmental r ubrics in emotion identification, Proceedings of the 2006 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2006) Toulouse, France, May 14-19, 2006, pp. 10851087. [ 131] B. Horn, Closed Form Solution of Absolute Orientation using Unit Quaternions, Journal of the Optical Society \ A vol. 4, n o. 4, pp. 629-642, April 1987. [ 132] L.A. Breslow, T.J. Gregory, R.M. Ratwani, A perceptual process approach to selecting c olor scales for complex visualizations, Journal of Experimental Psychology: Applied vol. 15, no. 1, pp. 25-34 Mar. 2009 [ 133] D.H Merwin and C.D. Wickens. Comparison of eight color and gray scales for displaying continuous data, Proceedings of the Human F actors Society vol. 2, pp. 13301334, 1993. [ 134] K. Patel, S. Jung, R. Diankov, R. Bajcsy, and J.N. Bailenson, The effects of fully immersive virtual reality on the learning of physical tasks, Proceedings of the 9th Annual International Workshop on Presence, Aug. 24 -26, Cleveland, OH, 2006, pp. 87-94. [ 135] E. To dorov, R. Shadmer, and E. Bizzi, Augmented feedback presented in a virtual environment accelerates learning of a difficult motor task, Journal of Motor Behavior vol. 29, pp. 147 158, 1997 [ 136] M.K. Holden Virtual environments for motor rehabilitation: review Cyberpsychology and Behavior vol.8, no.3, pp.187211, Jun. 2005 [ 137] G. Lacey, D. Ryan, D. Cassidy, D. Young, "Mixed -Reality Simulation of Minimally Invasive Surgeries," IEEE MultiMedia vol. 14, no. 4, pp. 7687, Oct.Dec. 2007. [ 138] R. Tan, H. Huo, J. Qian, and T. Fang, Traffic Video Segmentation Using Adaptive-K Gaussian Mixture, Proceedings of the International Workshop on Intelligent Computing in Pattern Analysis/Synthesis (IWICPAS 2006) Xian, China, Aug. 26 27, 2006, pp. 125 -134. [ 139] H. Gehlbach, A New Perspective on Perspective Taking: A Multidimensional Approach to Conceptualizing an Aptitude, Educational Psychology Review vol. 16, no. 3, pp. 207 -234, Sept. 2004. [ 140] S. Mostafawy, O. Kermani, H. Lubatschowski, Virtual Eye: Retinal Image Visualization of the Human Eye, IEEE Computer Graphics and Applications vol. 17, n o. 1, pp. 812, Jan. 1997.

PAGE 362

362 [ 141] B. A. Barsky, Vision -realistic rendering: simulation of the scanned foveal image from wavefront data of human subjects, Proceedings of the 1st symposium on applied perception in graphics and visualization Los Angeles, CA, Aug. 78, 2004, pp. 73-81. [ 142] B. Jin, Z. Ai, and M. Rasmussen, Simulation of eye disease in virtual reality, Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference Shanghai, China, September 1-4, 2005 pp. 51285131. [ 143] T. Sielhorst, T. Blum, and N. Navab, "Synchronizing 3D movements for quantitative comparison and simultaneous visualization of actions," Proceedings of the 4th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2005), Vienna, Austria, Oct. 58, 2005, pp. 38-47. [ 144] B. Insko, Passive haptics significantly enhances virtual environments Unpublished PhD dissertation, Department of Computer Science, UNC Chapel Hill, Chapel Hill, North Carolina 2001. [ 145] S.Z Jeong, N.Hashimoto, and M.Sato, Haptic Interaction System for Co evolution with Reactive Virtual Human, Technologies for E Learning and Digital Entertainment Berlin, Sp ringer Berlin / Heidelberg, pp. 12521261, 2006. [ 146] F. Faul, E. Erdfelder, A.G. Lang, and A. Buchner G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences, Behavior Research Methods vol. 39, pp. 175191, 2007. [ 147] F. Wilcoxon, Individual comparisons by ranking methods Biometrics vol. 1, pp. 8083, 1945. [ 148] M. Friedman, "The use of ranks to avoid the assumption of normality implicit in the analysis of variance, Journal of the American Statistical Assoc iation vol. 32, no. 200, pp. 675 701, 1937. [ 149] V. L Durkalski, Y Y Palesch, S R Lipsitz, and P. F Rust, Analysis of clustered matched-pair data, Statistics in Medicine vol 22, no. 15, pp 24172428, Aug. 2003. [ 150] R.A. Fisher, from contingency tables, and the calculation of P, Journal of the Royal Statistical Society vol. 85, no. 1, pp. 87-94, Jan. 1922. [ 151] H O Adami, D J Hunter, and D Trichopoulos, Textbook of cancer epidemiology Oxford University Press, New York, NY, 2002

PAGE 363

363 BIOGRAPHICAL SKETCH Aaron Andrew Kotranza was born in 1983 in Tampa, Florida. Aaron graduated cum laude from Berkeley Preparatory School in 2001. He received a full scholarship to attend the University of Florida as a National Merit Scholar. Aaron began working with Dr. Benjamin Lok as an undergraduate in 2004. Upon graduating with honors from the undergraduate computer engineering program, Aaron was awarded a four year UF Alumni Fellowship to pursue a Ph.D. His work has focused on expanding the applicability of virtual humans to simulate social scenarios for interpersonal skills training. His work has received significant recognition in both the fields of computer science and medicine with 11 articles published in leading journals and conferences, including receiving the best paper award at IEEE Virtual Reality 2008 and a featured article in the May/June 2009 issue of IEEE Transactions on Visualization and Computer Graphics Aaron and his collaborators have applied for patents of the technology des cribed in this dissertation. In the near future, Aaron hopes to continue his work in a research position in academia or industry.